1 

### Quality Assurance Project Plan

TDEC PM2.5 Filter Investigation Study

Performed by RTI for the Division of Air Pollution Control Ambient Air Quality Monitoring Program

June 17, 20142

Table of Contents

Table of Contents Transferred to Report Table of Contents

................................ ............... 2

Quality Assurance Project Plan Identification and Approval

................................ ............... 3

Background

................................ ................................ ................................ ................................ 4

Project/Task Description

................................ ................................ ................................ .......... 5

Description of Work to be Performed

................................ ................................ ..................... 6

Laboratory Activities

................................ ................................ ................................ ................ 6

Project Records

................................ ................................ ................................ .......................... 6

Data and Measurement Quality Objectives and Criteria

................................ ...................... 6

General Data Quality Objectives

................................ ................................ ........................... 16

Training Requirements

................................ ................................ ................................ ........... 16

Documentation and Records

................................ ................................ ................................ .. 16

Sample Custody Procedure

................................ ................................ ................................ .... 16

Analytical Methods Requirements

................................ ................................ ......................... 16

Quality Control Requirements

................................ ................................ ............................... 17

Equipment Acceptance Testing and Maintenance

................................ ............................... 17

Instrument Calibration

................................ ................................ ................................ ........... 17

Requirements for Supplies and Consumable

................................ ................................ ........ 18

Acquisition of Data

................................ ................................ ................................ .................. 18

Data Storage and Management Activities

................................ ................................ ............. 18

Data Validation

................................ ................................ ................................ ........................ 18

Final Data End User Requirements

................................ ................................ ....................... 183

Quality Assurance Project Plan Identification and Approval

Title: QUALITY ASSURANCE PROJECT PLAN TDEC PM2.5 Filter Investigation Study

The Q u ality Assurance Project Plan is hereby recommended for approval and commits the

State of Tennessee APCD to follow the elements described within.4

Background

The TDH laboratories in Nashville and Knoxville have provided laboratory analysis and
sample m edia preparation services for TDEC Air Pollution Control for a number of years.
In early 2000, the lab began processing PM2.5 filters for Air Pollution Control and other
agencies operating PM2.5 networks in Tennessee. The laboratory was provided funding

t hrough the 103 PM2.5 Grant program to acquire the necessary equipment to outfit and
operate a PM 2.5 weighing lab in Nashville. This was later expanded to include another
operational lab in the Knoxville TDH environmental lab.

EPA provides strict criteria that must be followed by a weighing laboratory if it is
performing weighing analysis for PM2.5 filters. Specialized weighing equipment (micro
balance), temperature and humidity controlled weighing environment and other
requirements must be met and mainta ined on an ongoing basis in order to perform
weighing analysis. Filters that are initially weighed at one laboratory must be returned to
the same laboratory for final weighing and must also be processed within a fixed
timeframe. Filters once exposed must be returned for analysis either immediately or kept
under refrigeration and return shipped to the lab for analysis.

During the period from 2010 through August 2012, the TDH laboratories in both
Knoxville and Nashville began experiencing significant proble ms with the environmental
controls used to maintain the temperature and relative humidity in the PM2.5 weighing
rooms operated by each facility. These deviations from the acceptable ranges of
temperature and humidity resulted in an increasing number of fi lters being identified by
the two laboratories as not meeting the criteria set by EPA for valid PM2.5 data and
potentially subject to being flagged or invalidated because of these excursions in the
environmental conditions within the weighing laboratories.

EPA has defined the acceptable operational conditions for a PM2.5 weighing laboratory
and has periodically evaluated the laboratories in Knoxville and Nashville during
Technical System Audits (TSA) that are performed by EPA normally once every three

years . EPA performed a TSA during the week of September 9 to 13, 2013 during which
the PM2.5 data generated by the state lab(s) were also evaluated for the period from 2010
through 2012. During this audit it was determined that the laboratories failure to pro perly
maintain environmental conditions during PM2.5 filter processing significantly impacted
the quality of the resulting PM2.5 data being reported by the laboratory and provided to
the Division of Air Pollution Control for use in defining air quality and determining
compliance with the NAAQS for fine particulate matter. EPA further recommended that
instead of flagging the affected data impacted by the excursions of laboratory temperature
and humidity conditions during weighing, that invalidation of the a ffected data should be
performed. (November 21, 2013 EPA Final TSA Report Section 2. Summary follows)5 “The TSA for the State of Tennessee revealed two major concerns and several
other concerns with respect to the ambient air monitoring program. The first

m ajor concern was the significant portion of PM2.5 data flagged with a “1” in the
AQS during 2011

- 2012. The State operated two PM 2.5 weigh laboratories at the
time of this audit. Both of these labs began experiencing environmental control
problems (temperature and relative humidity). Efforts were made by the State to
repair the failed environmental systems. These repair efforts were unsuccessful,
with a cycle of repair and failure until the decision was made to contract a private
laboratory to take over all of the weigh laboratory duties until replacement of the
environmental systems could be accomplished. The State decided to close one of its laboratories (Knoxville) and focus all efforts into the Nashville lab. The “1”
flag is an indicator of a “cr itical criteria” for this pollutant measurement method
not being met. EPA guidance recommends data not meeting a critical criteria be
invalidated. EPA Region 4 has consistently followed this guidance and will

recommends the State invalidate this data unles s it can present evidence as why it
should be retained. The data impacted was collected from mid 2011 through mid
2012 for the State, Memphis, Nashville and Knoxville agencies and totals
approximately 30% of the data set for each of these agencies. EPA Reg ion 4 is
recommending the TDEC

- DAPC review data back through 2010 for this issue.”

The impact of the proposed invalidation of the affected data for the state and local
program PM2.5 monitoring networks could be very significant as the network monitors

are required to collect and report at least 75% of the data and to have that data available
for a contiguous 3 year period to use in calculation of the design values for the sites and
areas the monitors represent. An initial evaluation of the entire dataset during the period
from January 1, 2010 to December 31, 2012 indicates that both the state and local
programs that were customers of the TDH environmental laboratories in Knoxville and
Nashville potentially will have insufficient PM2.5 data available to cal culate valid design
values for a number of the monitoring sites operating during this period of time.

Project/Task Description

An investigation was started immediately upon receiving the preliminary EPA remarks
and comments during the exit interview into the extent of data affected and possible
options to resolve the potential loss of a significant portion of the existing PM2.5 datasets already processed and loaded into the EPA’s AQS subsystem.

Some options are being investigated to further identify and possibly recover data that may be currently flagged but statistically meet the definition of the criteria for exclusion but in reality are still valid. Additionally a study is being considered to evaluate the6

effect that excursions of temperature and humidity may have on filter weighing in an effort to assign a specific amount of impact to the ranges of temperature and humidity experienced during filter processing by the TDH labs.

Description of Work to be Performed

Contract with a recognized commercial PM2.5 weighing laboratory to perform reweighing of previously exposed “archived” PM2.5 filters at temperature and humidity ranges outside of the normally acceptable temperature and humidity levels and determine the difference (if any) in the same set of filters all initially weighed at normal laboratory conditions and then at lower and higher conditions relative to the acceptable extreme lower and upper ranges for temperature and humidity.

Laborat ory Activities

The following requirements must be met and maintained by a weighing laboratory that is processing PM2.5 filters. The specific requirement listed in the CFR, QA Guidance Document 2.12 and the excerpt from the Critical Criteria Tables provid ed by EPA in the data validation templates provide the actual requirements PM2.5 weighing labs must meet and maintain in order to properly process PM2.5 filters.

Project Records

Because the previously exposed filters are already identified and catalogued by the TDH labs in their laboratory PM2.5 filter record and data and processing system, there will be additional documentation and records generated by the contract lab during their receipt, processing and weighing/reporting of the mass values generated d uring the filter study. A report of all necessary data will be requested to be provided by the contract lab including all needed documentation of the laboratory conditions to ensure all stability and control requirements are met during both the normal and extreme temperature and humidity weighing operations.

Data and Measurement Quality Objectives and Criteria

The DQO’s (Data Quality Objectives) were determined in part based upon the following questions to consider.

1. What will be the criteria for selection of filters to be included in the reweighing study?

There will be only a limited number of PM2.5 filters available for re

- weighing by the contract lab given that only filters that have already been sampled with and placed into archive will be emp loyed and only those of recent sampling dates will be selected. The entire population of affected and unaffected filters will not be subject to reweighing because of cost, time and age of the filters in question. In this study filters that were collected in late 2013 and early 2014 would qualify to7 be included. An examination of the available filter population revealed that filter meeting these criteria were available for selection.

2. What is the statistically significant number of samples to select to re present the entire population of affected filters?

The contract lab was contacted to determine a statistically valid number of filters to employ in the study. The number suggested by the contract lab was 30

- 50 filters based on the statistical power analy sis for the number of samples needed to be 95% certain a difference between the two populations of weighings actually existed (i.e., two different environmental conditions). This sample size was also based on historical gravimetric precision and that in th e FRM Acceptance Testing that the average any two weights can differ by is up to 10 ug and still be deemed ‘passing’ the moisture pickup acceptance test. The following data suggest that this number of filters may be sufficient for the purposes of this stu dy.

In the table above the total number of filters produced for use in the 2014 sampling year by MTL and acquired for distribution by EPA (TRC) to the state sampling agencies nationwide were slightly over 211,000 filters. TRC supplied 50 filters to RTI to per form the required moisture pickup acceptance testing representing the entire lot of filters. This is approximately 0.02% of the total. Tennessee collected on average 4200 creditable filters each year for the 2010 to 2012 time period. The number of filte rs to select was suggested to be 30 filters (plus one blank MTL filter). This represents approximately 1% of the entire population of ambient filters (excluding field blanks, lab blanks and collocated sites). It is suggested that the percent of samples m ust be in excess of the percent employed for acceptance testing to satisfy the requirement for a statistically valid sample. In reality, the actual number of affected samples is believed to be approximately 30 to 35% of the total filter population.

U sing the reduced number of filters thought to be affected (based on 35%), the number included in the study (30) represents 2% of the total population of filters.

In this case the number of filters selected is 2 orders of magnitude above the

selection leve l considered acceptable for acceptance testing.8

3. What method will be employed to select filters to include in the study?

Only filters that were determined to meet the requirements for both pre and post laboratory conditioning and weighing were considered for acceptance and selection. Filters processed by the TDH weighing lab in Nashville during the 3 most recent months of operation (November, December 2013 and January 2014), were evaluated to determine if they met the criteria and were not flagged by the lab or determined to be invalid or flagged for another reason. The filters that met the criteria were then assigned a number between 1 and 399 and a random number generation program was used to select random numbers within this specified range and matched the random number to a filter number in the data set. In this case field blanks were also included in the population of filters selected to draw the 30 filters from. The criteria employed to determine if the filters were representative of the filter pop ulation is met because 30 unaffected filters were identified as available. The candidate population of study filters was then shared with representatives of the 3 local programs (Knoxville, Nashville and Memphis) to determine if they were agreeable to usi ng these filters to represent the entire population of filters, including the filters form their own sampling networks. All three agreed to use the selected set. If any of the local programs decided to nominate their filters to include, they would have t o directly contact the TDH lab to arrange for their filters to be released as TDEC APCD does not have access to local programs filters as they are under separate contract with TDH for services.

4. What method will be employed to ensure that the contract lab is weighing the study filters in an unbiased manner?

In order to ensure that the filter weighing and processing is performed in an unbiased manner by the contract weighing lab, only filter numbers were provided

to the TDH lab to use to retrieve the selected filters from storage. The TDH lab was instructed to ship the selected filters directly to the contract lab without any additional information about the previous weighing performed by the TDH lab. The filters were shipped according to the TDH lab s expedited shipping protocol and following instructions provided by the contract lab. It is understood that the contract lab received the filters and immediately placed them into storage in preparation for the initial intake weighing to establish a basel ine mass at normal temperature and humidity conditions established for PM2.5 mass weighing.

5. What will be the basis for determining if the level of difference noted in the contract lab reweighing of the filters is significant and can a simple test be perfo rmed to determine if this level of error is or is not exceeded?

In evaluation of possible levels of significance in differences in the mass of the filters being processed, it is critical to note the following levels of significance already recognized by E PA and included in the MQO table:9 Criteria Acceptable Range

Lot Blanks Less than 15 ug change between weighings

Field Blanks ± 30 ug change between weighings

Lab Blanks ± 15 ug change between weighings Duplicate

Filter Weighing ± 15 ug change between weighings

Lower

Detection

Limit < 2 ug/m3

In evaluation of the values stated above, it becomes clear that if a field blank can vary by up to ±30 ug and still be acceptable, the implied variability in the field sampling results is also no more than ±30 ug. In taking this approach further, the lab blank variability may also be considered a reasonable guide to determine if results are in excess of acceptable weighing results. It is proposed that significance in the reweighing operations performed by the contract lab be defined as having an absolute net mass difference either in excess of the lab blank or field blank differences described. Values falling between the two stated numbers would be considered acceptable to the extent that these deviations in

w eights are normally assigned operational status event flagging and are acceptable to be reported to AQS with a “2” flag.

6. What are the other approaches being considered for filter recovery and

revalidation?

The statistical review of the TDH lab measuremen t data for temperature and humidity indicate that a number of reported measurement values may be slightly over or under the stated control ranges for temperature and humidity (within 0.1 % RH or 0.1 Deg C of the control temperatures and humidity’s specifie d by the method. It is understood that this special study will also reinforce any decisions made with respect to validation of the affected datasets at least up to the specified ranges included in the study.

Although not entirely applicable to this speci al study, the current DQO’s for PM2.5 as stated in the current TDEC QAPP approved by EPA for PM2.5 are stated as follows: “The ambient air quality monitoring program will establish an acceptable precision of ±10%, as measured by coefficient of variation, and an acceptable bias of ±10%. By controlling precision and bias at these levels, the decision error probability limit will be

5%.”

The relevant MQO’s (Measurement Quality Objectives) are defined in the following tables provided by EPA.

The following cr itical criteria tables define EPA’s expectations of the data quality and further clarify the requirements for the laboratory performance during filter processing10

when weighing PM2.5 filters. The criteria of most concern with the laboratory weighing

operat ions are identified by a black outline box to help separate them from the other criteria that may be related to field sampling and field equipment checks and quality assurance requirements.111213141516

General Data Quality Objectives

For the purposes of this study, the same DQO’s apply for overall laboratory data quality with the understanding that the contract laboratory is not processing filters that have been collected under

a 30 day expiration from sampling date window as would be the case with regular samples being processed coming in directly from the field. The same qualification applies for the requirements that the same weighing lab must perform both the pre and post weighin g of the filters using the same balance. The filters being examined in the study are previously exposed filters and therefore are not subject to a pre

- weighing requirement to obtain an unexposed mass weight.

Training Requirements

The contract laboratory is a recognized national laboratory and contracts with EPA and other air monitoring agencies in the provision of PM2.5 related gravimetric analysis services. See section A.8 of the QAPP for Chemical Speciation of PM2.5 Filter Samples, Revision No. 13, Da te: January 27, 2014.

Documentation and Records

All of the data and summary reports generated by the contract lab will be provided in support of the study. If additional information is needed relative to the filter processing operations performed, including copies of any logs or other records generated, the contract lab will need to be contacted to supply them. See section A.9 of the QAPP for Chemical Speciation of PM2.5 Filter Samples, Revision No. 13, Date: January 27, 2014

.

Sample Custody Proc edure

For this study, a filter list was developed based on random selection of filters processed by the TDH PM2.5 weighing lab and identified as meeting the laboratory QC requirements for temperature and humidity control during pre and post filter weighin g operations. These filters

are considered to be valid and “un

- flagged” filters that will be reported to AQS “as is” representing actual air quality. The list of filter numbers was provided to the TDH labs to pull from the filter storage archive of recen tly collected filters and ship directly to the contract lab. The contract lab was not provided with any TDH weighing information or data associated with the filters and the only identifying information provided was the actual filter number embedded

on eac h filter as supplied by the MTL filter manufacturer. See section B.3 of the QAPP for Chemical Speciation of PM2.5 Filter Samples, Revision No. 13, Date: January 27, 2014. Filters will be returned to the TDH laboratory for archive and storage.

Analytica l Methods Requirements

Standard Operating Procedure for Particulate Matter (PM) Gravimetric Analysis, PM Gravimetric Analysis, Revision 9, Date: July 8, 2008. Contractor SOPs have been updated annually, but they are not public documents until EPA posts them at the beginning of a new contract. SOPs following the analytical method requirements were adhered to for this study with robotic weighing using the MTL AH

- 225 Precision Weighing System.17

Quality Control Requirements

The laboratory SOP’s employed by the contract lab is included and identifies the QC requirements and methods employed by the contract lab to ensure these requirements are being met. The laboratory utilized a robotic weighing system with excellent precision capabilities to follow the qual ity control requirements. See section B.5 of the QAPP for Chemical Speciation of PM2.5 Filter Samples, Revision No. 13, Date: January 27, 2014 and Standard Operating Procedure for Particulate Matter (PM) Gravimetric Analysis, PM Gravimetric Analysis, Revi sion 9, Date: July 8, 2008.

http://www.epa.gov/ttn/amtic/files/ambient/pm25/spec/qapp.pdf

http://www.epa.gov/ttnamti1/specsop.html

Equipment Acceptance Testing and Maintenance

The contract lab must be capable of performing normal PM2.5 filter weighing activities and also meet the study criteria extremes for both temperature and humidity. The contract laboratory was initially contacted to determine if their weighing facility could meet the following humidity and temperature control levels for testing purposes

Humidity Extreme

Range % Humidity Control

Low 25  5% SD * over 24 hr. High 45  5% SD * over 24 hr.

Temperature Extreme

Range Temperature °C Control

Low 18  2 o C SD * over 24 hr. High 25  2 o C SD * over 24 hr

After undergoing tests for both temperature and humidity, the contract laboratory was able to

meet and maintain the specified levels of control relative to the low and high extreme ranges specified. The contract lab also indicated that these levels were able to be maintained during the weekend weighing periods set aside for the study filter processing including both co nditioning and actual weighing. See section B.6 of the QAPP for Chemical Speciation of PM2.5 Filter Samples, Revision No. 13, Date: January 27, 2014.

Instrument Calibration

All laboratory analytical equipment must be periodically calibrated with weighin g lab equipment referenced to NIST standards. See section B.7 of the QAPP for Chemical Speciation of PM2.5 Filter Samples, Revision No. 13, Date: January 27, 2014.18

Requirements for Supplies and Consumable

Although the contract lab is not responsible for providing the blank filter media for actual field sampling, section B.8 of the QAPP for Chemical Speciation of PM2.5 Filter Samples, Revision No. 13, Date: January 27, 2014 specifies how these items are obtained.

Acquisition of Data

The study undertaken by the contract lab does not require any data acquisition from historical sources. The filters provided are all pre

- numbered with the numbers being supplied by the filter manufacturer. No filter mass weighing data from the previous labs weighing activit ies was provided for the filters employed in the study.

Data Storage and Management Activities

The contract laboratory will be responsible for all data management activities and will process and store the data generated directly by the weighing instrumen ts as electronic data files. See section B.10 of the QAPP for Chemical Speciation of PM2.5 Filter Samples, Revision No. 13, Date: January 27, 2014.

Data Validation

Processing of the exposed filters for this study involves the use of filters that have already been used for PM2.5 air sampling and have subsequently determined by the TDH lab and the TDEC APCD to be valid acceptable unflagged filters. See section D.1 of the QAPP for Chemical Speciation of PM2.5 Filter Samples, Revision No. 13, Date: Januar y 27, 2014 for the procedures employed by the contract lab in typical data review and validation. The filters processed in this special study are not able to be subjected to review which includes the typical field and other

related data elements as these elements were not made available to the contract lab. The data resulting from the special study weighing evaluations will not be employed to directly calculate or generate a data value to be loaded to AQS and therefore will not be required to be calculate d to render an ug/m3 value. However, it is anticipated that the results will be used to specify whether existing data already loaded to AQS may be voided, flagged or annotated to convey unusual or out of the ordinary conditions associated with the affecte d filters.

Final Data End User Requirements

The special weighing study is not intended to be an on

- going laboratory function and as described previously in this document, is designed to examine weight differences found under extremes of temperature and h umidity outside of the normal PM2.5 weighing labs accepted ranges of temperature and humidity. The contract lab will provide a draft study report detailing the preliminary results from the weighing investigations undertaken and in coordination with

TDEC A PCD address comments pertinent to preparation of the final study report.