MEMORANDUM

TO: Docket No. EPA

‐ HQ

‐ OAR

‐ 2012

‐ 0918

Air Quality Designations for the 2012 PM 2.5 Standards

FROM: Liz P. Naess

Air Quality Assessment Division, OAQPS

SUBJECT: Data Quality Issues in Louisville, Kentucky Affecting Air Quality Designations for

the 2012 PM 2.5 National Ambient Air Quality Standards

The purpose of this memorandum is to discuss PM 2.5 data quality issues identified in Louisville,

Kentucky as a result of a technical systems audit (TSA) of the Louisville Metro Air Pollution

Control District's (LMAPCD) ambient air monitoring program. TSAs are systematic and objective

examinations of a monitoring program or project to determine whether environmental data

collection activities and related results comply with the project's Quality Assurance Project Plan

and other planning documents, are implemented effectively, and are suitable to achieve its

data quality goals.

Kentucky Division for Air Quality conducted a TSA in 2013 of LMAPCD’s ambient air monitoring

program and laboratory, which indicated that the PM 2.5 samples collected and analyzed by

LMAPCD during multiple timeframes ranging from 2009

‐ 2012 did not meet all of the

requirements of the federal reference method for PM 2.5 found in 40 CFR Part 50, Appendix L, or

the requirements of LMAPCD's Quality Assurance Project Plan and Standard Operating

Procedures (see Attachment 1). The TSA also found that in many cases the records maintained

by LMAPCD were not sufficient to verify the validity of the PM 2.5 samples being collected. This is

further discussed in EPA Region 4’s August 8, 2014, letter to the Kentucky Department of

Environmental Protection (KDEP) (see Attachment 2).

EPA Region 4 has made a preliminary determination regarding the data collected and analyzed

by LMAPCD for the period 2009 through 2012. Until a final determination has been made, EPA

has moved the data in the Air Quality System database for the PM 2.5 sites in question from

parameter code 88101 (PM 2.5 at local conditions; used for regulatory data) to parameter code

88501 (PM 2.5 raw data). This change will clarify to data users that data from these sites should

not be compared to the PM 2.5 National Ambient Air Quality Standards until EPA has made a

final validity determination. No other changes to this dataset have been made at this time.

Attachments

1. June 3, 2013, letter from John S. Lyons, Director, Kentucky Division for Air Quality, to

Lauren Anderson, Director, Louisville Metro Air Pollution Control District (LMAPCD),

transmitting a copy of the 2013 LAPCD Weigh Lab Technical Systems Audit. Includes

audit attachment.2. August 8, 2014, letter from Beverly H. Banister, Director, Air Pesticides and Toxics

Management Division, US EPA Region 4, to Sean Alteri, Director, Division for Air Quality,

Kentucky Department for Environmental Protection, regarding data issues in Louisville,

Kentucky.KDAQ TSA of LMAPCD May 2013 Page 1 of 20

INTRODUCTION

In late February 2013, Kentucky Division for Air Qu ality (KDAQ or Division) staff inadvertently observed Louisville Metro Air Polluti on Control District (LMAPCD) October 2012 field blank data in AQS, and found the data greatly exceeded acceptance limits. Upon further review, it appeared the field blank issue spanned t he entire 2012 year. KDAQ staff also observed that, despite this issue, there were no qu alifier codes on any of the 2012 PM 2.5 data in AQS. In March 2013, KDAQ staff contacted LMAPCD to inquire about these observations and learned that the weigh lab had experienced issues w ith static electricity in 2012, which coincided with the receipt of new MTL filters from EPA. The field blank data results were also discussed with LMAPCD staff during the first quarter 2013 per formance audit exit interview. LMAPCD staff assured KDAQ that the static electricity issu e was isolated to only the field blank samples and subsequently had been resolved. KDAQ staff, ho wever, felt the severity of the problem and the impact on all particulate samples may not have been realized by LMAPCD staff and therefore warranted further investigation.

Unfortunately, a review of records showed the last EPA-conducted in-depth technical systems audit (TSA) in the LMAPCD weigh lab was in 2001. G iven the length of time since the last comprehensive weigh lab audit, as well as the stati c electricity issue that impacted an entire year’s worth of data, KDAQ staff felt compelled to conduct a TSA of the LMAPCD weigh lab, as well as an in-depth data review. Thus, on April 23, 2013, Stephanie McCarthy and Stacie Masters, KDAQ Technical Services Branch personnel, began a TSA of the PM 2.5 weigh lab operations of the LMAPCD. The audit also included a review of data handling procedures, as well as an in-depth review of the agency’s quality systems documents. On April 26, 2013, the

audit was concluded.

During the audit, interviews were conducted with LM APCD staff:

• Susan Bowman – Weigh Lab Analyst & agency Quality A ssurance Staff

• Terry Carby – Back-up Weigh Lab Analyst & agency Qu ality Assurance Staff

• Josh Tennen – Data Coordinator

• Marty Layman – LMAPCD Quality Assurance Officer (QA O)

• Billy DeWitt – Air Quality Unit Supervisor

• Cynthia Lee – Air Program Manager

The following quality systems documents were review ed during the TSA, and will be referenced in this report, as well as in the attached Weigh La b Technical Systems Review questionnaire.

• The LMAPCD Quality Assurance Project Plan (QAPP) en titled QAPP for the LMAPCD NCore Ambient Air Quality Monitoring Program, Version 1.0, June 2010 (henceforth referred to as the LMAPCD QAPP). This document is the agency’s most current QAPP and was approved by EPA on May 17, 2011.

• The LMAPCD SOP entitled Gravimetric Analysis for the Measurement of Fine

Particulate Matter as PM 2.5 , SOP Using Mettler MT5 Microbalance in Support of the Federal Reference Method for Measuring PM 2.5 , AQ-PMF2, Version 1.4, updatedKDAQ TSA of LMAPCD May 2013 Page 2 of 20

November 8, 2007 (henceforth referred to as the Wei gh Lab SOP). This document is the most current version of this SOP, and was approved by EPA on June 27, 2008.

• The LMAPCD SOP entitled Field Operations for Measurement of Fine Particulat e

Matter as PM

2.5 , SOP Using Rupprecht & Patashnick or TEI Partisol- Plus 2025 , AQPMF1, Version 1.2, updated October 9, 2008 (hencefo rth referred to as the PM 2.5 Field SOP). This document is the most current version of this SOP, and was approved by EPA on January 8, 2009.

• The LMAPCD SOP entitled Data Handling and Custody SOP for Collecting, Proce ssing, and Reporting Ambient Air Quality Data , Version 1.0, February 2009 (henceforth referred to as the Data Handling SOP). This SOP wa s submitted to EPA in March 2009, but an approval letter has never been received.

FINDINGS AND RECOMMENDATIONS

While addressing the static electricity issue in th e LMAPCD weigh lab, additional areas of concern were found. The findings from this systems audit will be broken into three main sections: weigh lab operations, data handling, and quality assurance. Further details and information regarding the audit findings are docume nted in the accompanying Weigh Lab Technical Systems Review questionnaire.

It is to be noted here that the deviations detailed in this report will focus on PM 2.5 ; however, these deviations will also impact the agen cy’s PM 10 and PM 10-2.5 data, since all of these samples are handled in the LMAPCD weigh lab.

The deficiencies listed below do not necessarily re flect the capabilities or dedication of LMAPCD monitoring staff. However, they do indicate serious concerns with regard to the effectiveness of the LMAPCD quality system.

Findings Regarding the PM 2.5 Weigh Lab Operations:

1. While the minimum and maximum temperatures, as well as relative humidity readings, recorded in the weigh lab are being reviewed daily to discern whether the lab is within

the 20-23 0 C temperature and 30-40% humidity limits, the varia bility in temperature and humidity control in the weigh lab is not being calc ulated or reviewed. LMAPCD staff indicated the variability (standard deviation) has never been calculated for weigh lab operations throughout the entirety of the agency’s PM 2.5 program.

40 CFR 50, Appendix L, Section 8.2, requires the va riation in temperature and humidity control in the weigh lab be determined for each 24- hour period. Specifically, the regulation details in Section 8.2.2 that temperatur e control is ±2oC over 24 hours; Section 8.2.4 specifies the humidity control is ±5% relativ e humidity over 24 hours. LMAPCD’s Weigh Lab SOP further details these same requiremen ts in Section 12.3.4. Moreover, theKDAQ TSA of LMAPCD May 2013 Page 3 of 20

SOP indicates that if these requirements are not me t, filter weighing must be postponed until the weigh room is again within tolerance.

Recommendation : LMAPCD must adhere to regulatory and SOP require ments and calculate the variability in temperature and relati ve humidity control in the weigh lab. These calculations should begin immediately. The s tandard deviation data must be reviewed daily and filter weighing postponed if env ironmental conditions are out of specification.

Additionally, historical temperature and humidity d ata must be reviewed in order to

determine if there have ever been occasions when th e weigh room conditions were out of specification, but filters were weighed. The Divis ion requests that LMAPCD review its available historic minute data for temperature and relative humidity and calculate the variability. In instances when weigh room conditio ns were out of specification, all filters weighed during those time periods should be, at a m inimum, flagged in the AQS database with a critical criteria flag of “1”, or else inval idated.

At a minimum, data from 2009-2013 should be reviewe d and corrected in the manner

indicated above.

However, the Division strongly recommends that all data from 2002 forward be reviewed. If minute data is no longer available to calculate variability, the Division encourages LMAPCD to flag the historical data with an AQS flag of “1”, since the agency would not be able to prove that conditions w ithin the weigh lab met regulatory requirements.

2. The LMAPCD utilizes an in-house developed Access da tabase, referred to as the APCD Ambient Monitoring Data System, for all PM 2.5 weigh lab operations. The Access database is programmed to notify the lab analyst wh enever environmental conditions are out of tolerance in the weigh room, or when require d balance checks exceed specifications. Unfortunately, during this audit, i t was discovered that all limits programmed into the LMAPCD Access database were inc orrect.

40 CFR 50, Appendix L, Section 8.2 details the envi ronmental control specifications for the weigh room. The temperature range in the labor atory is to be maintained between 20- 23°C; the relative humidity range is to be maintain ed between 30-40%. The temperature range programmed into the LMAPCD Access database wa s 20-30°C. The humidity range programmed into the Access database was 20-50 %.

In accordance with EPA Method 2.12, balance checks are required to be within ±3μg. The LMAPCD Access database was programmed to allow a ±25μg deviation.

Recommendation : In the short term, the Division recommends LMAPC D immediately reconfigure the Ambient Monitoring Data System Acce ss database to include the correct specifications for environmental conditions and bal ance checks. Additionally, since these values are strongly relied upon for the determinati on of proper weigh lab operations, it isKDAQ TSA of LMAPCD May 2013 Page 4 of 20

recommended that they be reviewed during the quarte rly audits of the weigh room as well. Looking towards the future, the Division str ongly encourages LMAPCD to purchase a Laboratory Information Management System (LIMS) for the PM 2.5 weigh lab.

3. LMAPCD field blank data during the January 2012 thr ough February 2013 time frame ranged from -175μg to +250μg. Field blank data wa s uploaded to AQS as required; however, no AQS Qualifier Codes were uploaded with that data.

40 CFR 50, Appendix L, Section 8.3.7, as well as EP A Method 2.12, Section 7.7, detail the requirements for field filters blanks. The est ablished acceptance criterion for field blank data is ±30μg. This requirement is reflected in the agency’s Weigh Lab SOP.

In the LMAPCD QAPP, in Section 22.1, the following requirement is stated: “Data identified as suspect, or does not meet the accepta nce criteria, shall be flagged as indicated in Table 21-1”. Additionally, the agency ’s Data Handling SOP supports the QAPP and provides in Section 8.3 a table of AQS nul l codes and qualifier flags to apply to data that does not meet specifications.

Recommendation : As directed by the agency QAPP and Data Handling SOP, all PM 2.5 filter data in AQS from January 2012-February 2013 should be flagged with an AQS qualifier code of “2”, indicating an operational cr iterion of the method was not met. The data should also be flagged with the AQS qualifier code “FB”, indicating the field blank value was above the acceptable limit.

4. A review of laboratory blank data for the January 2 012 through February 2013 time period showed concentration values ranging from -47 7μg to +98μg.

40 CFR 50, Appendix L, Section 8.3.7.2, as well as EPA Method 2.12, Section 7.7, detail the requirements for laboratory filters blanks. Th e established acceptance criterion for laboratory blank data is ±15μg. This requirement i s also reflected in the agency’s Weigh

Lab SOP.

Recommendation : As described in Item #3 above, the LMAPCD QAPP a nd Data Handling SOP directs staff to flag data that does n ot meet acceptance criteria.

Due to the extreme variability in laboratory blank data during the January 2012-February

2013 time frame, all PM 2.5 data in AQS should be flagged with an AQS qualifier code of “LB”, indicating the lab blank value was above the acceptable limit.

5. Multiple deviations were noted regarding lot stabil ity testing.

a. Lot blank stability testing is required prior to us e of a new lot of filters. EPA Method 2.12 requires lot blank stability testing to includ e 9 filters per lot, with each filter demonstrating less than ±15μg change between weighi ngs. The Weigh Lab SOP details this procedure in Part I, Section 12.1.2. The lab analyst was found to be weighing only 3 filters during this process.KDAQ TSA of LMAPCD May 2013 Page 5 of 20 Recommendation : To properly conduct a lot blank stability test, the weigh lab analyst must weigh 9 filters per lot, in adherence with the method and SOP requirement.

b. The Weigh Lab SOP states in Part I, Section 12.1.2, that the lab analyst is to document the LMAPCD Filter Lot Blank Stability Test Form when completing lot stability testing. Per the SOP, “The completed for m is to be turned into the Air Quality Unit Supervisor for verification and then f iled.” This same requirement is detailed in the agency’s Data Handling SOP in Secti on 5.1. However, the lab analyst does not turn the form into the Supervisor for veri fication. Moreover, no one quality assures the lab analyst’s work regarding lot blanks to verify the math (which is computed by hand) and/or ensure the procedure is co mpleted correctly.

Recommendation : In accordance with the agency’s Weigh Lab and Da ta Handling SOPs, the Filter Lot Blank Stability Test Form shou ld be submitted to the Supervisor for verification and then filed. Moreover, any mat hematical computations, especially those completed by hand, should be peer reviewed by the supervisor or other independent staff to ensure accuracy.

c. The Weigh Lab SOP states, “To determine the appropr iate conditioning time for the other filters, make note of how long it takes the w eight of the filters to stabilize. A stabilized filter weight has been achieved when the difference in the 24-hour weighs of each lot blank filter is less than 15μg.” Hence , it may take longer than 24 hours for filters to equilibrate; the conditioning time shoul d be determined per the results of the lot stability testing. However, in 2012, the weigh lab analyst was unable to achieve successful lot conditioning of the blanks as descri bed by the agency’s SOP. Weights would not stabilize within 15μg. Eventually, the l ab analyst decided to disregard the SOP procedure and condition the filters for 24 hour s only, which is the minimum required per regulation.

During the audit, all 2012 Corrective Action Report (CAR) forms generated by

LMAPCD staff were reviewed. There were no CAR forms documenting the stability issues within the laboratory or any corrective acti on that was taken as a result of the stability issues. Moreover, the Air Quality Unit S upervisor was not made aware of this issue in the laboratory until a significant ti me period had past.

Recommendation : Whenever discrepancies are noted in filter weigh ings, or if a lot of filters fails the stability testing requirements , the Air Quality Unit Supervisor should be immediately notified. All discrepancies and troubleshooting should be thoroughly documented in the weigh lab logbook. It is additionally recommended that agency CAR forms be documented and submitted. Ideally, if an issue forces a deviation from the established procedures, the QAO should be prompted to conduct an internal review to ensure a systematic problem d oes not exist.

6. Multiple deviations were noted regarding balance ch ecks.KDAQ TSA of LMAPCD May 2013 Page 6 of 20

a. EPA Method 2.12, Section 7.9, requires balance chec ks to be performed at the beginning of each weigh session, every 10 th sample weighing, and at the end of the weigh session. The method requires ≤ 3μg for this check. Specifically, the method states, “If this measurement disagrees from the ver ified value by more than 3 μ g (i.e., three times the microbalance’s reproducibility), re weigh the standard.” The agency’s Weigh Lab SOP details the balance check procedure i n Sections 12.4 and 13.2. More specifically, the Weigh Lab SOP states in Sections 12.4.5 and 13.2.6 that the ± 3μg (0.003 mg) limit for a balance check is the differe nce between the actual and measured values of the standard weights. The Weigh Lab SOP specifically details

that if the balance check fails, the balance is to be recalibrated prior to weighing

filters.

The lab analyst stated during the audit that balanc e check limits were ±15μg. Moreover, as stated previously in Item # 2, a revie w of the Access database utilized in the weigh lab also revealed that the specification for balance checks was programmed incorrectly to allow a ±25μg deviation.

The auditors requested a printout from the Access d atabase of a batch weighing session; data from January 14 and February 14, 2013 , weigh sessions were reviewed. During this data review, there were exceedances of the balance check specification observed. For example, when reviewing the batch se ssion from February 14, 2013, the 500mg weight standard demonstrated a range from 500.003 to 500.011 mg.

Recommendation : At a minimum, balance checks from all weigh sess ions from the past 4 years should be reviewed to ensure they were within ±3μg. If instances where

the balance check limits were exceeded are discover ed, all filters weighed in that batch should be flagged in AQS with a qualifier cod e of “2”, indicating an operational

criterion of the method was not met.

Additionally, the Division encourages LMAPCD to inv estigate purchasing a new microbalance in the future, given that repair parts are no longer available for the aged Mettler balance being utilized.

b. The Weigh Lab SOP, Section 12.4.10, requires that a t the end of a weigh session, a balance check is to be performed using both weight standards. A review of data from the January 14, 2013, weigh session indicated that only the 200 mg weight was final weighed. However, it is noted that both check stan dards were final weighed during the February 14, 2013, session, for which data was also reviewed.

Recommendation : The weigh lab analyst should adhere to the Weigh Lab SOP and weigh both weight standards at the end of each weig h session.

c. According to EPA Method 2.12, Section 7.9, bullet # 7, “After approximately every 10th filter weighing, the analyst should reweigh at least one of the working standards.” During the LMAPCD data review, it was observed that the analyst wasKDAQ TSA of LMAPCD May 2013 Page 7 of 20 reweighing the 200mg working standard after the 20 th filter weighing, which deviates from the method. Upon discussion with the lab anal yst, it appeared there was a misinterpretation of what the 10 th filter weighing was referencing. The weigh lab analyst weighs each filter twice and after weighing 10 filters (i.e., 20 actual weighings), the check weight is re-weighed.

Recommendation : The weigh lab analyst should check the 200mg wei ght standard every 10 th filter weighing.

As stated previously, the Division strongly encoura ges LMAPCD to purchase a LIMS

for the PM 2.5 weigh lab. A LIMS system would prevent the aforem entioned deviations (items a, b, & c above) from occurring.

7. In 2012, staff determined that static electricity w as causing erroneous data results, which

were believed to be isolated to the field blank dat a only. Because of this determination, additional anti-static accessories (such as an anti -static desk pad, grounding clips for the conditioning trays, and extra polonium strips) were placed in the lab to neutralize static charges. In early 2013, an electrician discovered that the weigh lab was not properly grounded to an earth ground. The findings listed a bove in Items #3, 4, 5, & 6 can be partially attributed to the weigh lab not being pro perly grounded.

However, throughout the course of the year troubles hooting these issues in the weigh lab, the KDAQ auditors found no documentation whatsoever in the lab logbook or within the agency CAR forms to recreate this chain of events, and therefore could not produce a timeline of troubleshooting and corrective action i mplementation within the weigh lab. The Weigh Lab SOP in Sections 15 & 16 instructs sta ff to document the logbook with information regarding preventive maintenance proced ures and troubleshooting. Section 9 of the LMAPCD QAPP details additional documentation requirements.

Recommendation : Foremost, given the weigh lab was found by an e lectrician to not be properly grounded, an AQS qualifier flag of “4” sho uld be added to all January 2012- February 2013 particulate data to indicate a lab is sue.

Additionally, staff should adhere to requirements i n the agency SOPs and QAPP and document accordingly. Moreover, comments should be detailed and descriptive, so that the events can be easily recreated years later. Wi thout detailed documentation, actions taken cannot be proven and the defensibility of dat a is jeopardized.

8. The Weigh Lab SOP in Part I, Section 10.1.2 states, “Once each quarter, verify the working weight standards against the stated mass of the primary weight standards. Weigh the primary standards, then the working stand ards in triplicate, recording each measured mass in the microbalance room logbook.” T his information was not recorded in the microbalance room logbook. The lab analyst stated this procedure was performed on a monthly basis, but it was a “visual check” onl y.KDAQ TSA of LMAPCD May 2013 Page 8 of 20

Recommendation : The weigh lab analyst should adhere to the SOP r equirements and document this information in the logbook. Without written documentation, there is no tangible evidence to prove the procedures were comp leted.

9. The weigh lab analyst maintained separate forceps f or initial (clean) and exposed (dirty) filter weigh sessions; however, the analyst only ha d 1 plastic bag in which to store the 2 forceps. The analyst stated she never placed both forceps in the bag at the same time. Instead, she would rotate which forceps were in the bag for a given length of time, leaving the other set of forceps out.

Recommendation : Forceps for initial and exposed filter weighings should be maintained in separate, appropriately labeled plast ic bags to minimize potential

contamination.

10. Section 12.3.3 of the Weigh Lab SOP instructs the l ab analyst to inspect each filter “in front of a strong light source.” Damaged filters a re to be discarded. When describing the procedure for inspecting each filter, the analyst i ndicated the filter was held up to the ceiling light to determine if there was any damage.

Recommendation : The Division recommends LMAPCD investigate purch asing a small lighted table for this process.

11. Upon returning exposed filters to the weigh lab, th e field technician checks the transport cooler’s temperature with a small thermometer in th e weigh lab that lacks any certification paperwork or statement of accuracy.

Recommendation : The temperature device used for this process sho uld be NISTtraceable and have documentation to verify its trac eability and accuracy. Moreover, annual verifications against a NIST-traceable stand ard are needed for this particular

thermometer.

12. The Weigh Lab SOP, Part I, Section 12.3.1 states, “ Field blanks should amount to between 10-15% of filters used for sampling through out the year... A monitor operating on a 1-in-6 day cycle requires 6-9 field blanks per year; 1-in-3 requires 13-18”. The lab analyst prepares only 1 field blank each month, whi ch would total only 12 filters per

year.

Recommendation : The lab analyst should prepare the amount of fie ld blanks required by the agency SOP.

13. The LMAPCD QAPP in Section 13.5 states that trip bl ank filters are to be used to monitor potential contamination throughout the samp ling process. The Weigh Lab SOP

in Part I, Section 16.3.2 further details the use o f trip blank filters to help isolate problems with field blanks. However, it was found that trip blanks are not utilized by the agency.KDAQ TSA of LMAPCD May 2013 Page 9 of 20

Recommendation : LMAPCD staff should adhere to the requirements i n their QAPP and SOP and prepare trip blank filters. If there are bu dgetary constraints preventing LMAPCD from allotting extra filters for this purpos e, then the requirement should be removed from the agency’s quality documents. Howev er, if filters are available, the Division strongly encourages LMAPCD to utilize trip blanks because they are excellent quality control and troubleshooting tools.

14. The Weigh Lab SOP, Part II, Section 2.3.2 states, “ An analyst other than the one who performed the original weighings should reweigh at least 10% of each lot... Record both measurements in the logbook.” This same requireme nt is further detailed in the agency’s Data Handling SOP in Section 5.1. However, these m easurements were not recorded in the logbook as required. Only the filter identifica tion numbers were documented.

Recommendation : The analyst who reweighs 10% of each lot should completely document the weighing session in the logbook, which includes the filter mass obtained for each weighing.

15. The Weigh Lab SOP details in Part II, Section 2.1.1 that performance audits of the microbalance are to be completed using an independe nt set of weight standards. However, a review of quarterly performance audit fo rms for the weigh lab showed that the auditor typically used the lab analyst’s set of primary standards, and occasionally used the analyst’s working standards, to complete t he quarterly performance audits.

It is to be noted here that 40 CFR 58, Appendix A, requires performance audits to be conducted utilizing independent equipment. Using t he lab analyst’s set of working or primary weight standards does not meet the definiti on of independence, and therefore these quarterly balance checks cannot be classified as audits.

Recommendation : A separate set of weight standards should be pur chased by LMAPCD for the purpose of conducting acceptable qua rterly performance audits of the

microbalance.

Finding and Recommendations Regarding PM 2.5 Data Handling:

1. There were multiple parameters detailed in 40 CFR 5 0, Appendix L that were not being reviewed as part of the routine data verification/v alidation process. Specifically, the time frame between the initial (tare) weight and the sam pling date (< 30 days), the sample pick-up date (7 days 9 hours from the end of sampli ng), the post-sample weigh time, and the sampler’s coefficient of variance for flow (≤2% ) were not reviewed. LMAPCD Quality Assurance staff indicated that it was assum ed that all of these requirements were being met. No documentation was provided to indica te these assumptions were correct.

Recommendation : Although the agency’s PM 2.5 network is small and the lab analyst’s

routine is uniform, data should still be verified i n order to ensure regulatory requirements

are met.KDAQ TSA of LMAPCD May 2013 Page 10 of 20

The Division recommends that the data audit form ut ilized by agency Quality Assurance staff – the “All FRM Particulate Matter Data by Dat e” Report – be modified to include sample weigh dates, sample pick-up dates, and the s ampler’s coefficient of variance. These parameters should be reviewed in relation to the regulatory requirements detailed in 40 CFR 50, Appendix L.

Moreover, given that these critical criteria compon ents have never been reviewed as part of the routine data verification/validation process for PM 2.5 , the Division recommends

that LMAPCD staff review its historical data to ens ure all filters have met these regulatory requirements. At a minimum, 2009-2013 d ata should be verified. If any samples are found to exceed any of the regulatory r equirements, the data should be, at a minimum, flagged with an AQS qualifier code of “1,” or invalidated.

2. The Data Handling SOP requires in Section 5.3.3 tha t PM 2.5 mass concentrations be manually calculated to verify accuracy. LMAPCD sta ff indicated that these calculations are made; however, there was no documentation provi ded to demonstrate that these calculations were indeed being computed and verifie d by hand, as per SOP requirements. Additionally, the results of these calculations are not reviewed by other LMAPCD Quality Assurance staff or by the agency’s QAO.

Recommendation : The Division recommends that LMAPCD Quality Assu rance staff document a logbook that shows the manual, mathemati cal computations completed for the samples specified in Section 5.3.3 of the agenc y SOP. Ideally, the QAO should review the results of these manual computations to ensure accuracy, as well as ensure data has been handled correctly.

3. Interval data files downloaded from the PM 2.5 samplers are not routinely reviewed during the data verification/validation process.

Recommendation : Interval data files from R&P 2025 samplers can p rovide significant insight into health and operation of the 2025 sampl ers, which is a valuable tool for Quality Assurance staff. Interval data can also be a useful troubleshooting diagnostic for

field technicians, as it will illustrate when flow controllers may be faltering. This data should be routinely reviewed during the QA process.

4. The Chain-of-Custody (COC) forms being utilized by agency staff are not the same COC forms shown in the LMAPCD QAPP in Section 12.3 in F igure 12-1. Unfortunately, the COC form currently utilized by the agency is not as robust as the one shown in the

QAPP.

The current COC form is shown in the PM 2.5 Field SOP in Section IV. The form does not contain spaces for field technician or weigh la b analyst comments, the date the samples were picked up from the weigh lab, the date samples were returned to the weigh lab, or the temperature of the samples upon return to the weigh lab. Additionally, theKDAQ TSA of LMAPCD May 2013 Page 11 of 20

COC form only has one space for the lab analyst’s s ignature (“Prepared by”) and the field technician’s signature (“Received by”).

When reviewing documentation on the COC forms, it w as noted by the KDAQ auditors

that, in some instances, COC forms were found to ha ve multiple sets of handwriting that were not attributed to anyone. Additionally, there were multiple temperature readings and multiple dates listed on the individual forms t hat were undefined and, therefore, confusing.

In short, the current COC forms do not provide the necessary level of defensibility to the

data.

Additionally, it is to be noted here that in Sectio n 12.3 of the LMAPCD QAPP, it states that, during the data validation process, notes on the COC forms are used to determine the validity of samples and to assign validation fl ags. However, LMAPCD staff indicated that they do not make comments on the COC forms regarding any sources or events that could impact data integrity. Such info rmation is recorded only in the site logbooks, and is not reviewed during the data valid ation process.

Recommendation : Foremost, the COC form shown in the agency QAPP should be utilized by LMAPCD, but it should be enhanced to in clude additional information. The Division recommends that spaces designated for samp le pick-up date and time (from the field) be added to the COC form. Also, it would pr ovide better insight into the total handling of the filters if the lab analyst signed ( or initialed) and dated the COC form when the magazine was prepared for deployment, as w ell as signed and dated the form upon receiving the samples back into the laboratory . The same holds true for the field technician. Ultimately, the COC forms should be sig ned by any and all staff members who maintained physical custody of the samples for any period of time.

A specific space for “Comments” should also be adde d to the COC forms, where field or lab issues that could impact data validity can be n oted. Moreover, LMAPCD staff should adhere to the agency QAPP and review those COC note s during the data validation process and AQS qualifier codes be added to data, w hen warranted.

5. KDAQ auditors reviewed LMAPCD PM 2.5 , PM 10 , PM 10-2.5 , and lead data, as found in the AQS database, during this audit. Numerous issues w ere found.

a. Inconsistencies in data coding were observed. While CAR forms were

documented in most cases of invalid data, the docum entation on the forms did not always compliment the data found in AQS. In severa l instances, it was noted by KDAQ auditors that similar situations were coded di fferently. When asked to explain why different codes were used to null simil ar events, staff relied heavy on

their collective memories to recreate the events. Unfortunately, the Access database, upon which they primarily depend, does no t contain enough information to help them retrace AQS coding decisions.KDAQ TSA of LMAPCD May 2013 Page 12 of 20 Recommendation : Foremost, all staff members should be trained on data handling procedures, which should include how to co nsistently apply AQS null codes and qualifier flags. A more detailed Data Ha ndling SOP should be developed.

Second, documentation should exist that is thorough enough that any data coding decision can be easily recreated years later. The Division recommends that LMAPCD Quality Assurance staff develop internal dat a audit reports that recount data handling decisions. The Division recommends t hese reports be submitted to the Air Quality Unit Supervisor for approval, as we ll as the QAO.

b. For some data points chosen by the KDAQ auditors to review, LMAPCD staff provided no documentation to justify why the data h ad been invalidated. There

were instances noted where data could have been ret ained, but flagged in AQS with a qualifier code.

Recommendation : The reasons for invalidating data should be clea rly documented. Additionally, the agency should adhere to the requirements of its QAPP and SOPs and apply qualifier codes to data. A pplication of qualifier codes

will allow, in certain instances, for data to be re tained that would otherwise be

invalidated.

c. A review of 2012 lead data indicated that sample re sults were uploaded on days which did not correspond to PM 10 sample days. The low-volume lead analysis is performed on the PM 10 sampled filters, which means the lead results shou ld be uploaded on the same day as the PM 10 results.

Recommendation : The LMAPCD data review process should be enhance d so that Quality Assurance staff, and/or the QAO, revie ws data to ensure that it is entered into AQS on the appropriate days. Entering data on the wrong days will negatively impact data completeness results.

d. For all particulate parameters observed, there appe ared to be missing data in AQS (i.e., scheduled sample days were blank).

Recommendation : All scheduled run dates must have either a conce ntration value or null value code assigned. The LMAPCD data review process should be enhanced so that Quality Assurance staff, and/or th e QAO, reviews data to ensure that all scheduled run days contain the necessary i nformation. Entering data on the wrong days, or not entering data at all, will n egatively impact data completeness results.

6. Precision and accuracy data files are generated uti lizing the P &A Transaction Generator tool (i.e., an Excel program) provided by EPA. Use of this Excel program requires manual data entry. However, there is no independen t review of the information entered into the Transaction Generator to ensure the approp riate data is entered and typographicalKDAQ TSA of LMAPCD May 2013 Page 13 of 20

errors are not made. Staff indicated that a peer re view process for P&A data was unnecessary. In fact, the QAO stated during the au dit that the individual who is responsible for P&A data rarely makes mistakes, and therefore, the QAO does not closely

review his work.

Recommendation : The Division strongly recommends LMAPCD develop a more robust peer review process. All maintenance, precision, a nd audit forms used to generate the precision and accuracy AQS upload file should be re viewed by another independent staff member (ideally, the QAO) to screen for outliers. Additionally, the generated P&A file should also be reviewed by independent staff for tr anscription errors (such as entering precision data on the wrong date, or entering the w rong audit equipment type). Ideally, the QAO should review the completed AQS AMP 250 rep ort against the original precision and accuracy forms to ensure the accuracy of the data entry.

7. The Data Handling SOP does not detail any of the pr ocedures employed by the QAO during the data certification process. Additionall y, neither the Data Handling SOP nor the LMAPCD QAPP clearly details the follow-up proce dures employed by staff to ensure data corrections are correctly implemented after re commendation by the QAO. When asked to detail the data correction process to the KDAQ auditors, the QAO stated that data is reviewed after it is entered into AQS, and if any corrections are deemed necessary, a request is forwarded to the staff member responsi ble for originally coding the data. The QAO assumes the staff member revises the data as re quested and the correction is successfully uploaded into AQS. However, there was no documentation of this process provided to the KDAQ auditors for review. At the end of the year, while performing the annual data certification, the QAO stated that data corrections requested during the course of the year are then reviewed in AQS. Howev er, no reports are printed from the AQS database; it is just a visual check. It was un clear to the KDAQ auditors if all of the requested data corrections were verified or if the QAO picked only 10% of the data corrections and assumed, based on this random sampl ing, that all corrections were in fact submitted correctly as requested.

Recommendation : The Data Handling SOP should be revised to inclu de more specific information regarding the QAO’s data review procedu res and the data certification process itself. Additionally, the Division strongl y recommends that all data corrections

be reviewed at the time of the correction and not d uring the tedious process of data certification, which could occur a year or more aft er the corrections were requested.

Findings and Recommendations Regarding Quality Assu rance :

1. On Friday, April 26, 2013, the weigh lab analyst in formed the KDAQ auditors that the 2002 version of the Weigh Lab SOP was the document being utilized in the lab over the course of the last 4 years. The analyst further ind icated that she had no knowledge of the current, EPA-approved version of the document until the audit conducted by KDAQ.KDAQ TSA of LMAPCD May 2013 Page 14 of 20

Recommendation : Foremost, if the lab analyst has been following an outdated SOP for the entirety of her tenure in the weigh lab, the Di vision recommends that all of the data that she has handled be flagged in AQS with a quali fier code of “6”, indicating a QAPP/SOP issue. Given all of the QAPP/SOP deviation s listed in this audit report, this AQS flag is warranted for the agency’s particulate data, regardless of whether or not the 2002 version of the Weigh Lab SOP had been approved by EPA.

(It is to be noted here that the auditors did not c onduct an in-depth review of the 2002 version of the Weigh Lab SOP during this audit.)

Secondly, all employees should be supplied the most current version of SOPs. There should be a system in place to ensure that employee s not only possess the most current version of SOPs, but that they have read and unders tood them, and are actually following the stated procedures.

Although the agency has a shared network drive (i.e ., the M: drive) where documents are shared and stored, that does not guarantee that emp loyees are actually using the

documents housed on that drive. If older versions o f SOPs are retained on the M:drive, they should be archived or labeled to indicate they are not the current versions.

In Section 19.8.1, the LMAPCD QAPP directs the foll owing: “The Quality Assurance Officer (QAO) maintains overall responsibility for management and administrative aspects of the QA program.” Therefore, the LMAPCD QAO should develop a plan to

resolve these issues.

2. The LMAPCD quality system provides for employee tra ining. The culmination of that training involves a certification of acceptable per formance by the LMAPCD QAO (see the LMAPCD QAPP, Section 4.1.2). The weigh lab anal yst received “on the job training” from the previous weigh lab analyst. However, the weigh lab analyst was never observed by the LMAPCD QAO to ensure that procedures were be ing performed correctly. Moreover, the QAO admitted that an “over the should er review” of the employees in the weigh lab had never been completed, which means tha t the employee who trained the current weigh lab analyst was most likely never “ce rtified” either. Therefore, erroneous or incomplete procedures could have been passed dow n without detection.

Recommendation : The LMAPCD QAPP requires the QAO to certify that staff can perform all duties correctly. Therefore, the QAO m ust certify all employees in all aspects of the air monitoring program – including the weigh lab, which is the foundation of LMAPCD’s particulate network.

3. In the Weigh Lab SOP, Part I, Section 16.3.3, it st ates, “Systems audits are used to determine if procedures are being followed.” This requirement is found in the “Troubleshooting” section of the document, and dire cts that systems audits are especially required when there is a significant issue in the w eigh lab that warrants further investigation. Additionally, the LMAPCD QAPP, in S ection 4.1.2, states the following:KDAQ TSA of LMAPCD May 2013 Page 15 of 20

“The QAO will perform internal audits to identify p otential problems and review quality

control data.”

During the audit, there was no documentation of the se reports in relation to the weigh lab provided.

Recommendation : A technical systems audit is a great tool for as sessing staff understanding of the job duties assigned as well as identifying deficiencies. Based upon the severity and duration of the weigh lab issues i n 2012, an audit should have been conducted and would most likely have revealed that procedures were being followed

from the incorrect SOP.

The QAO should perform internal technical systems a udits as specified in the agency’s SOPs and QAPP for all areas of the air monitoring n etwork. Additionally, reports of the findings from these audits should be disseminated t o staff to ensure that any corrective actions required are properly addressed.

4. During the audit, there appeared to be a disconnect amongst staff members regarding data handling and AQS principles. Staff did not appear to have a clear understanding of the data quality objectives (DQOs) or measurement quali ty objectives (MQOs) for the particulate network. Staff also did not appear to h ave clear understanding of the requirements in the agency QAPP.

Moreover, in regards to the technical issues within the weigh lab in 2012, as mentioned previously, staff attributed the static electricity issues to impacting field blank filters only. Unfortunately, this issue impacted all filters in the lab, including PM 10 , and required that all particulate data from January 2012-February 201 3 be flagged in AQS.

These findings illustrate another issue regarding t he LMAPCD training program. Reading SOPs alone does not guarantee understanding of content or that the fundamental scientific principles upon which these programs are built are comprehended. Moreover, simply watching one employee do his or her job and learning to “mimic” those procedures does not guarantee an understanding of E PA methods or the “big picture” of the monitoring program. Without an understanding o f the mathematics and scientific principles upon which air monitoring programs are i mplemented, the potential for staff producing questionable data increases.

Recommendation : LMAPCD staff would greatly benefit from additiona l training, particularly in the areas of EPA methods, data hand ling, and quality assurance. Specifically, training on data verification versus validation, chain-of-custody, and AQS coding are needed. The Division recommends that th e LMAPCD QAO develop a more robust training program for agency staff. In the i nterim, KDAQ staff will provide additional training to LMAPCD staff.

5. In the LMAPCD QAPP, Section 20.5, it states the fol lowing: “LMAPCD performs technical systems audits of the monitoring system. These audits are performed by theKDAQ TSA of LMAPCD May 2013 Page 16 of 20

QAO and reports are issued to the Air Quality Unit Supervisor.” Unfortunately, however, technical systems audit reports are not is sued to the Air Quality Unit Supervisor, as required by the agency QAPP.

The KDAQ auditors requested copies of the QAO’s tec hnical systems audits reports during the TSA. The auditors were supplied “checkl ists” to represent the systems findings. The QAO visits each site in the field on ce per year. However, a review of these “checklists” showed incomplete documentation. There were sections of the checklists that were left completely blank. On som e of the checklists, the QAO had written comments regarding findings that could impl y field problems or questionable data, but there did not appear to be any follow-up investigation by the QAO regarding these findings. Moreover, without these checklists being translated into reports which are submitted to the Air Quality Unit Supervisor, any q uestionable findings in the field would not be relayed to management so that corrective act ions could be immediately implemented.

Recommendation : First, the QAO should completely document all au dit checklists. Gaps in documentation imply that procedures were no t reviewed. Second, the QAO should augment the systems audit process and develo p a detailed questionnaire to use, as opposed to the abbreviated checklist. The current c hecklist does not provide an independent reviewer with a clear understanding of the systems finding. Third, it is further recommended that the QAO write a detailed r eport regarding the findings from the systems audit, and submit that written report t o the Air Quality Unit Supervisor, as required by the QAPP. The QAO should submit his ch ecklist to the Air Quality Unit Supervisor as well, to supplement the written repor t.

The QAO and the Air Quality Unit Supervisor should mutually determine a deadline for systems audit report submissions. The deadline shou ld be long enough to allow the QAO to detail his findings, but short enough that the A ir Quality Unit Supervisor can direct his staff to implement corrective actions before there is an adverse impact on data.

Finally, the Division encourages the QAO to conduct more than one systems audit per site per year. Given that the LMAPCD network is sm all, multiple systems audits could easily be conducted across the course of the year.

6. An in-depth review of the quality systems documents detailed in the Introduction of this report revealed several areas of concern.

a. When reviewing documents side-by-side, there were i nconsistencies in terminology and procedures noted.

Recommendation : All particulate SOPs should be updated. The QAP P and SOPs should be complimentary documents. The termin ology utilized in the most recent document (i.e., the LMAPCD QAPP) should be t he terminology used in

the other documents.KDAQ TSA of LMAPCD May 2013 Page 17 of 20 b. For the particulate SOPs, it has been 5 or more yea rs since the documents have been updated. Staff admitted that several procedur es in these documents did not

reflect their current activities.

Recommendation : SOPs should be reviewed on an annual basis and r evised when/if procedures change. The QAO should implement and oversee the schedule for annual SOP review, and ensure that all stated p rocedures are current and

accurate.

c. Several of the quality documents utilized by LMAPCD staff had not received EPA-approval, and/or there was confusion as to whet her or not the document had

ever been submitted to EPA.

i. The Data Handling SOP was submitted to EPA in March 2009, but LMAPCD has not received approval for this document.

ii. The LMAPCD Quality Assurance Project Plan (QAPP) en titled QAPP for the LMAPCD Ambient Air Quality Monitoring Program, Version 2.1, December 2009, appears to be the most recent versio n of the agency’s criteria pollutant (non-NCore) QAPP. This criteria pollutant QAPP is identical to the NCore QAPP throughout its sections , except that it does not contain MQO tables for the trace-level analyzer s found at the Cannons

Lane site, and certain references to hi-volume lead sampling have been removed. This criteria pollutant QAPP was submitte d to KDAQ by the previous Air Quality Unit Supervisor as an official document. However, this QAPP has not been approved by EPA, and there i s uncertainty as to

whether or not the document was ever sent to EPA fo r approval.

Recommendation : The LMAPCD QAO should develop a system for docum ent tracking. When SOPs or QAPPs are submitted to EPA, the QAO should followup with EPA if documents have not been approved aft er an extended time frame (e.g., 6 months after submittal).

7. There appeared to be a significant breakdown in the LMAPCD corrective actions process in 2012. Section 19.9 of the LMAPCD QAPP details th e structure and protocols employed by the agency for this purpose.

a. In 2012, weigh lab quality control data (i.e. lot b lank, lab blank, field blank, and balance check data) significantly exceeded agency a cceptance limits, but there were no CAR forms generated that detailed these iss ues or any actions taken as a

result of the issues.

Recommendation : Corrective Action Report (CAR) forms should be g enerated for all issues which could impact data validity. T hese forms should be thoroughly documented and descriptive. It is also recommended that logbooks beKDAQ TSA of LMAPCD May 2013 Page 18 of 20 documented with the corrective action information, in addition to agency CAR forms. Without thorough documentation, defensibilit y of data is jeopardized.

b. Quality control data detailed above exceeded limits throughout the entire 2012 year, which indicates that initial corrective actio ns were unsuccessful and further investigation was warranted. In Part I, Section 16 .3.1, of the Weigh Lab SOP, staff are directed to clean samplers whenever field blank results are outside of the acceptable limits of ±30μg. Likewise, in Section 1 6.3.2, the SOP details that trip

blanks collected when field blanks exceed limits ca n help isolate where contamination may be occurring. Finally, the SOP d irects in 16.3.3 that a systems audit is to be conducted to review filter handling procedures.

A review of all 2012 CAR forms indicated that none of these SOP-directed corrective action procedures were completed during the field blank investigation.

Recommendation : Staff should adhere to agency SOPs. The Air Qua lity Unit Supervisor should ensure that all required correcti ve actions are implemented. The QAO should also evaluate the efficacy of the co rrective action(s) and determine if data quality has been impacted. And, the QAO should complete the required internal systems audits.

c. The LMAPCD QAPP directs that any staff who perceive s the need for corrective action shall present the situation to the Air Quali ty Unit Supervisor within 30 days of perceiving the need. Air monitoring staff are t hen responsible for implementing the corrective actions within 30 days, notwithstanding extenuating circumstances. In 2012, the Air Quality Unit Super visor was not notified within 30 days of the need for corrective action in the we igh lab. And, unfortunately, the situation took over a year to resolve.

Recommendation : Staff should adhere to the agency’s QAPP require ments and notify the Air Quality Unit Supervisor of the need for corrective actions. However, it is to be noted here that the KDAQ audit ors were given the impression that staff did not have a sense of urgency in regar ds to the quality control data issues within the weigh lab, which may be why the A ir Quality Unit Supervisor was not notified within 30 days. Therefore, again, additional training would greatly benefit staff.

Additionally, the Division strongly recommends shor tening the length of time given for notifying management of the need for corr ective actions, as well as the length of time given to implement corrective action s. With data quality at stake, 30 days to report the need for corrective action an d then 30 days to correct it seems extensive. A five-day turnaround time is sug gested.

8. All LMAPCD technicians are responsible for certifyi ng their own standards. There is no single individual designated within the agency to t rack equipment certifications, and therefore ensure that all standards are actually wi thin specification. Without a singleKDAQ TSA of LMAPCD May 2013 Page 19 of 20

individual overseeing this activity, there is incre ased potential for equipment expiring in

the field.

Recommendation : The Division recommends that the LMAPCD QAO be r esponsible for tracking the certification of all standards use d in the agency network.

9. The LMAPCD QAPP states in Section 14.2, “Control ch arts will be used extensively by LMAPCD”. The QAPP specifically requires control ch arting of balance check samples in the weigh lab. The QAPP details control charts for PM 2.5 sampler flow rates as well. Later, in Section 20.7, the review frequency for co ntrol charts is detailed. However, staff admitted that no control charts are developed or re viewed.

Recommendation : Control charts are excellent tools for tracking t rends in data. However, if control charts will not be utilized by staff, the QAPP should be revised to

remove all references to control charts, which are found throughout the document. In the absence of control charts, quality assurance staff should develop another method for analyzing data for trends.

10. LMAPCD staff have an internal deadline established of 45 days for data to be uploaded into AQS, but the goal is 30 days. They routinely meet this expedited deadline, for which staff are to be commended. However, the regulatory deadline for AQS data entry (pursuant to 40 CFR 58.16) is 90 days after the end of the calendar quarter.

Recommendation : The Division strongly encourages LMAPCD to recons ider its stringent data-entry deadline, in order to allow ti me for a more in-depth data review process to occur. Allowing staff more time to anal yze and review data will minimize data entry mistakes.

CONCLUSION

During the course of this audit, numerous significa nt issues were discovered, including regulatory deficiencies. These issues impact all p articulate samples handled in the weigh lab, including PM 2.5 , PM 10 , and PM 10-2.5 . Luckily, several of the operational deviations t hat were identified can be quickly rectified. Other issues, particularly in regards to the agency’s quality system, will require extensive modification and sub sequent employee training. Due to the magnitude of the issues uncovered during this audit , an in-depth historical data review is necessary. At a minimum, data from 2009-forward mu st be reviewed for all criteria identified in this report, which will most likely result in data corrections and subsequent re-certification.

The following summarizes the most significant findi ngs of this audit:

• The grounding issue in the weigh lab seriously impa cted operations and subsequent data results. All January 2012-February 2013 particulate matter data should be flagged in AQS with qualifier codes of 2, 4, 6, FB, and LB.KDAQ TSA of LMAPCD May 2013 Page 20 of 20

• Regulatory and EPA method deviations were found, al l of which have been occurring for at least 4 years. Historical data should be review ed to determine the extent of the

deviations, and data corrections should be entered into AQS.

• The agency’s QAPP and SOPs are not being followed.

• There is a lack of documentation in-house, which we akens the defensibility and quality of

data.

• The structure of the agency’s data handling process is problematic, leading to

inconsistencies.

• Internal systems audits are not being conducted.

• There is a communication and procedural breakdown i n the agency’s corrective action

system.

• Staff, including the QAO, would benefit greatly fro m additional training.

• The agency would greatly benefit from the purchase of a LIMS for the weigh lab.

While the totality of these audit findings represen ts significant areas of concern, KDAQ is confident that corrections within the weigh lab can be swiftly implemented to rectify the abovementioned deficiencies. Adhering to the quality sys tems documents could have prevented many of the audit findings and should be stressed to sta ff moving forward. The development of more robust and unified quality assurance procedures wil l provide staff the stability and consistency that appears to be lacking in the current system.

The Division requests follow-up correspondence from LMAPCD detailing the corrective actions implemented as a result of this audit. The Division specifically requests notification once LMAPCD has completed the review of its historical d ata. As a conclusion to this audit, KDAQ

staff will also review the historical data for conc urrence of the findings and ensure data corrections are complete and appropriate. In the i nterim, the Division is committed to providing training to LMAPCD staff. Questions regarding this audit may be directed to Ms. Stephanie McCarthy, Technical Services Branch Manager.