Report on PM 2.5 Weighing Lab Temperature and Humidity Control
And
The Effect on PM 2.5 Filter Weighing

Aug 2014

Report Prepared By Robert Brawner2



## Table of Contents Background

................................ ................................ ................................ ................................ ................... 3 Proposed Actions

................................ ................................ ................................ ................................ .......... 4 Results and Recommendations

................................ ................................ ................................ ..................... 4 Table 1. Pearson Correlation Data Comparisons

................................ ................................ ..................... 6 Graphic 1 Location Map for TDEC and TVA PM 2.5 Sampling Sites

................................ ............................. 7 Response to Comments Provided by EPA on Review o f the Draft Report

................................ ................... 8 Filter Data Statistical Analysis Performed by TDH, Caleb Wiedeman, MPH, Epidemiologist, Tennessee Department of Health, Communicable and Environmental Diseases and Emergency Preparedness

....... 13 Knox County Filter Data Evaluation 2009 to 2013

................................ ................................ ...................... 21 All Sites Filter Data Evaluation 2009 to 2012

................................ ................................ .............................. 22 Additional Documents Referenced in this Report (Provided as separate attachments)

............................ 23 Environmental Condition Testing of 31 Sampled Measurement Technology Laboratories 46.2

- mm PTFE (Teflon®) Filters by Paige Presler

- Jur, RTI International, July 18, 2014

................................ .......... 23 Quality Assurance Project Plan, TDEC PM2.5 Filter Investigation Study, June 17, 2014, by Robert Brawner, Technical Services Program, Air Pollution Control Division, Tennessee Department of Environment and Con servation

................................ ................................ ................................ .............. 233



## Background

The Environmental Protection Agency (EPA) provides strict criteria that must be followed by a weighing laboratory if it is performing weighing analysis for PM 2.5 filters. Specialized weighing equipment (micro balance), temperature and humidity controlled weighing environment and other requirements must be met and maintained on an ongoing basis in order to perform weighing ana lysis. Filters that are initially weighed at one laboratory must be returned to the same laboratory for final weighing and must also be processed within a fixed timeframe. Filters once exposed must be returned for analysis either immediately or kept unde r refrigeration and return shipped to the lab for analysis.

During the period from 2010 through August 2012, the Tennessee Department of Health ( TDH ) laboratories in both Knoxville and Nashville began experiencing significant problems with the
environmenta l controls used to maintain the temperature and relative humidity in the PM2.5 weighing rooms operated by each facility. These deviations from the acceptable ranges of temperature and humidity resulted in an increasing number of filters being identified b y the two laboratories as not meeting the criteria set by EPA for valid PM 2.5 data and potentially subject to being flagged or invalidated because of these excursions in the environmental conditions within the weighing laboratories.

The impact of the proposed invalidation of the affected data for the state and local program PM 2.5 monitoring networks could be very significant as the network monitors are required to collect and report at least 75% of the data and to have that data available for a contigu ous 3 year period to use in calculation of the design values for the sites and areas the monitors represent. An initial evaluation of the entire dataset during the period from January 1, 2010 to December 31, 2012 indicates that both the state and local pr ograms that were customers of the TDH environmental laboratories in Knoxville and Nashville potentially will have insufficient PM 2.5 data available to calculate valid design values for a number of the monitoring sites operating during this period of time.4



## Proposed Actions

1). All PM 2.5 data should be reviewed for possible invalidation or additional flagging including looking as far back as the 2008 through 2012 datasets to identify all data previously flagged as temperature or humidity affected data

.
2). I nvestigate methods to identify data that may be salvaged based upon Measurement Quality Objectives ( MQO ) criteria being met at the extreme ranges of the MQO acceptance by employing the following techniques

.
Employ rounding to the humidity or temperature measurement data found to be slightly above or below the acceptance criteria.

Apply the stated calibration measurement accuracy range to the temperature and humidity
acceptance ranges.

3). Conduct an investigation into the effects of elevated or lowered t emperature and humidity (outside of the acceptable MQO stated ranges) on the weighing of PM 2.5 filters post exposure.

4). Review collocated datasets where more than one laboratory may have been employed to analyze

PM 2.5 filter data to determine if the data are similar to each other.



## Results and Recommendations

1) All of thePM 2.5 filter data collected from 2008 through 2012 w ere r eviewed and a number of filters were identified including in some cases up to 30% of the available filters with humidity or temperature flags already applied.

2) (a) The results of the rounding study identified some filters that may be considered valid and should have the existing data flags removed.

(b) The results of the calibration measurement accuracy range study also identified additional filters that may be considered valid and should have the existing data flags removed.

3) The results of the filter reweighing study indicate that the filters selected for the study were not substantively affected by changes in the temperature or humidity over the ranges of temperature or humidity selected to investigate.

The attached RTI report detailing these findings shows that there is essentially no difference in filters weighed at normal temperature and humidity and those weighed at higher or lower temperature or relative humidity (based upon the ranges of control specified within the EPA required procedure s).5

The attached statistical evaluation prepared by the TDH staff indicates there is no difference in the RTI data when comparing the effects of weighing’s of exposed filters over the controlled environmental conditions employed in the testing

.

The conclus ions fr o m both of these evaluations are that the differences seen in the filters selected for the RTI study fall well within the acceptable ranges for field blanks, lab blanks and duplicate filter weighing’s

. These level of error s are acceptable differenc es in filter weights and are identified in the MQO’s as operational criteria that when identified are to be flagged but not rising to the level of invalidation

.
4) Filter datasets were evaluated to identify any sampling sites where samples were collected in the same relative location but were analyzed by different weighing laboratories. PM 2.5 filters collected in the Kingston and Harriman areas of Roane County by TDEC and Tennessee Valley Authority ( TVA ) were found to have been analyzed by different suppo rt laboratories

.
There were several site identified that collected filters during the times the TDH weighing laboratory experienced environmental control issues. The filters from these sites were processed by an independent lab and not by the TDH lab. Th e filters were also found to have met the environmental control requirements for weighing. These filters were then evaluated against the filters collected on the same dates that were processed by the TDH laboratory during periods of time when environmenta l conditions in the weighing room were out of control. These data show high correlations between the filters processed by the TDH lab during periods of environmental temperature and humidity instability and filters processed by the TVA
lab under controlle d environmental weighing conditions. The follow ing table 1 provide s the correlation analysis data. An additional graphic “Location Map for the TDEC and TVA Sampling
Sites” depicts the relative locations of the 7 different PM 2.5 samplers that were used in this data correlation study. It is interesting to note that the TDEC site is approximately 2.3 miles from the nearest TVA site and yet has a high level of correlation with all of the TVA sites. The sites
evaluated are listed below in the table “TDEC and TVA PM 2.5 Sampling Sites”. The data evaluated include samples collected from Feb. 2009 through July 2011.6



### Table 1. Pearson Correlation Data Comparisons7



### Graphic 1 Location Map for TDEC and TVA PM 2.5 

### Sampling Sites8



## Response to Comments Provided by EPA on Review of the Draft Report

The EPA auditing team that conducted the TSA during September 2013 in which the laboratory environmental weighing room issues were noted, commented during the exit interview that TDEC APCD
should consider con ducting an investigation into the effect that variable temperature and humidity may
have on PM 2.5 filter processing. This was initially presented to the EPA audit team by the TDEC staff as a “glove box” study and was described as a possible method to help determine the overall effect the variable environmental conditions may have on filters being processed. Following this discussion planning was begun to design a PM 2.5 filter re

- weighing study and to determin e if equipment or a suitable lab might be available to perform the final study.

The EPA Region 4 staff that participated in the TSA, were appraised of the progress in developing the final study and were given an opportunity to review a draft of the study r eport this final document
represents.

EPA staff were requested to provide comments on the draft report and provided comments verbally during a conference call with EPA on 7/7/2014. The EPA provided a summary of the verbal comments in an email provided to TDEC APCD on 8/1/2014.

The following are the written comments received (in bold) and a response to each of the comments provided:

Overall, the study shows very little effect on the weights (of filters selected for this study) due to
varied lab conditions (humidity and temperature). The study’s intent was to replicate the lab conditions that affected TN PM2.5 samples. Specifically, the study should be representative of the TN PM2.5 samples that were weighed outside of temperature and humidity criteria in the Nashville and Knoxville labs during 2008

- 2013 (target population). However, the filters analyzed in the study are less representative of the affected/target filter population for several reasons:

1. TN sent RTI filters that had passed the required 30 day timeframe from sample collection. In this respect, the study filters were not representative of the target population. The target filter population were weighed within 30 days of sample collection, but did not meet temperature and humidity criteria. Th is introduces another variable into the study that would not have affected the target population.

Response: The study filters were selected from filters recently processed by the TDH lab using the same weighing chamber and analytical equipment that were u sed to process the other filters that were outside of the required critical criteria for humidity and temperature. TDEC did not select filters that

were generated during routine sampling to specifically avoid the possibility of having to void the filters9

and cause further data loss or invalidation. The 30 day time frame from sample collection to filter analysis is very short and may not allow for the variable conditioning and weighing to b e performed.

2. It seems that none of the sample filters in the study were selected from samples collected by the local agencies (Memphis/Shelby County, Nashville, or Knoxville). Samples collected by local agencies are more likely to have been exposed to urban PM2.5 and have likely would have had different PM2.5 composition and possibly higher PM2.5 mass than TDEC collected filters. The omission of local agency collected filters creates a potential selection bias. The selection bias could limit the representativeness of the study to the desired target population.

Response: Prior to selection of the final filters used for the study, a draft list of filter that only included net mass gains were circulated to the three local program with affected filters. After evaluation of the filters to be employed, the three local progra ms all agreed that the filters would be suitable for use. Therefore the randomly selected filters were allowed to be used in the study. The filters selected included filters from the following sites

.

Site Name AQS ID# Athens 471071002
Clarksville 471251009
Columbia 471192007 Dyersburg 470450004

Harriman 471450004
Hendersonville 471650007
Jackson 471130006 Kingsport 471631007

Lawrence 470990002
Loudon 471050108 Maryville 470090011

These sites represent areas across the entire state including sites that are in or near the Knoxville MSA and Nashville MSA. The filters also were selected to include recently sampled filters and were collected over the timeframe of 11/03/2013 through 1/2 4/2014. The filters were also selected to include only filters that were processed during period of time when the lab weighing room environmental conditions
were in control and not outside of the humidity or temperature limits established for the method. These filters were selected because they were the most recent available filters with the shortest storage life after the expiration of the typical 30 day time limit. Given the need to process filters relatively quickly to minimize the amount of time pos t 30 day expiration, it was decided to only look at these
filters initially.

3. The filters sent to RTI were weighed under high and low temperatures outside of critical criteria and then the same filters were weighed in humidity conditions outside of critical

. This introduces another factor into the study that would not have affected the target filter10 population. The response to humidity on the study filters could have been confounded by being previously conditioned at high and low temperatures prior to the variation in humidity.

Response: The filters used in the study were initially processed by the TDH lab within the proper humidity and temperature critical criteria. The filters were processed by the contract lab for this study, first under the proper hum idity and temperature controls and within the critical criteria limits. The
filters were then re

- equilibrated under temperature and humidity conditions outside of the critical criteria limits and weighed to determine the affect the temperature and humidit y had on the filters. Although changes in temperature and humidity during the weighing process are mentioned as confounding, the entire sampling and transport process is subject to uncontrolled humidity and temperature variations which can swing w idely in the field. This variable is not accounted for in the sampling methodology and is not addressed as a factor that should be considered in filter processing.

4. The QAPP and study plan should have been provided to RTI and EPA for comment before the study was c onducted.

Response: In fact the draft study plan was presented to RTI for consideration before the study was undertaken because RTI was initially not able to determine if their weighing lab could accommodate the ranges of humidity and temperature called f or in the study. After undergoing stability testing and confirming that their chamber could control at the extreme ranges then be brought back into normal operating parameters for routine work, it was determined the RTI lab would be able to conduct this

r equired testing. Several other labs were also contacted and provided with the details of the study and upon internal investigation, declined to participate based on the inability to control at the ranges specified. It was determined that the scope of the study should not be to investigate the extreme ranges of humidity or temperature actually found during the TDH filter processing , but to investigate the ranges where a majority of the filter weighing occurred and to determine if the actual data recovery f or all sites could be brought up to at least 75% minimum. RTI employs their own QAPP for filter weighing and processing, which should by itself address any questions about the analytical or weighing related operations. TDEC developed and presented a draf t QAPP for review to both RTI and EPA. The draft TDEC study QAPP was presented after EPA requested that this be provided as a part of the overall study.

5. It appears that there was no seasonal variability incorporated into the filters selected for the study and there is an impact on PM2.5 measurements depending on the season.

Response: As was previously discussed, t he filters selected ( collected over the timeframe of 11/03/2013 through 1/24/2014 ), would be broadly classified as seasonally representative of the fall/winter season. If seasonal selection were to be included, the study would have to be extended from January through December and would drastically prolong reporting of any usable data. The contract lab was informed that additional analytical work might be required after the initial result s were presented in the study to help address these types of questions. Provisions are already in place to utilize the lab for further study work and to selectively address additional areas of interest. EPA has discouraged any addi tional work or study proposals.11

The filter subset selected for the TDEC TVA sampling site comparison did include seasonal variability and was representative over a period that ran from Feb. 2009 through July 2011. An analysis of all of the filter data collected from 2008 to 2012 by all of the state and local PM 2.5 sampling sites in Tennessee indicated the following seasonal variation :

The TDEC TVA dataset upon seasonal analysis produced the following graph which provides the number
of filters evaluated by quarter and also shows the relative frequency of filters representing each quarter (season) and therefore the distribution of filters across all seasons.

The entire states dataset indicates that the second and third quarters are more likely to have higher

PM 2.5 filter loading than the first and fourth quarters. The filters selected in the TDEC TVA study appear to adequately address the concern about seasonal variation in the study.

6. The sample size may be too small to be able to m ake any definitive statements about the results (i.e., may not be statistically significant).12

Response: In the process of preparing to begin the study, the contract lab was requested to investigate the number of filters to select from the population of af fected filters (critical criteria exceeded for temperature or humidity during weighing) and determine what the number of filters should be to be able to statistically represent the entire population of filters. The reasoning behind this is included in
the QAPP and is believed to justify the number of filters selected. The requirement to not include any filters that failed to meet the critical criteria for filter weighing reduces the filter population to select from. The number of filters affected appears to be approximately 30 to 35% of the total filters collected, which would reduce the number of filters in the population from which to determine what the actual test sample size should be. The TDH has prepared an additional analysis of the study results and also prepared graphics depicting the number of affected filters in both the Knox County area and for the other areas outside of Knox County. This report immediately follows the comment and response section of the report on the next page and following

.13



## Filter Data Statistical Analysis Performed by TDH , Caleb Wiedeman,
MPH , Epidemiologist, Tennessee Department of Health, Communicable and Environmental Diseases and Emergency Preparedness

Methods

Thirty

- two filters were weighed and then reweighed at under the recommended conditions. The same filters were weighed and reweighed at different relative humidity levels (25%, 45%, and a second weighing under the recommended condition at 35%) and different temperature levels (18 o C, 25 o C, and
a second wei ghing under the recommended condition at 22 o C). Filter # 3654706 was excluded from the analysis due to debris being dislodged from the filter between weight measurements and filter # 14334066 was excluded because it was a different type of filter from the remaining thirty filters.

The Two One

- Sided Test (TOST) procedure was chosen to determine if the mean differences between the filters weighed at control and experimental conditions were greater than the + 15 μg of imprecision allowed between duplicate me asures of weight. TOST is often used in randomized controlled trials to determine if two treatments should be considered equal based on a predetermined equivalence margin. TOST combines the results of two one sided paired t

- tests, one testing the
likelihoo d that the mean difference is larger than the smallest allowable difference, and the other testing whether the mean difference is smaller than the largest allowable mean difference.

Differences in weight were calculated by subtracting the weight of each i ndividual filter weighed under the recommended conditions with the measured weight of the same filter under the experimental conditions. The mean differences in weight were then tested using the TOST procedure, to
determine if the estimated mean difference between the initial measurement and the experimental measurement of each filter fell between the equivalence margin of + 15 μg (the range considered to be the allowable “noise” during a typical repeat measurement). A power calculation was performed (assum ing a mean difference as large as 8 and a standard deviation of 6) and power was assessed to be greater than 0.8 for the TOST procedure using the thirty, paired filter measurements and an alpha of 0.025. All statistics were calculated using SAS version 9.3

.

Results

The mean differences in weight between the filters at control conditions (weighed at the ideal conditions) and the experimental conditions (weighed at non

- ideal conditions) are shown in Figures 1 and 2. The green box in each chart represents the + 15 μg allowable range. The blue diamond markers are the differences in weight when the filters were re

- measured later under the ideal conditions.14 -20 -15 -10 -5 0 5 10 15 20

0 5 10 15 20 25 30 Difference Between Weight Measurements in Micrograms (Weight at 36% RH Used as Control)

Filters Tested 

# Figure 1: Raw Weight Difference in Filters Measured at Different Relative Humidities

Difference in μg at 25% RH
Difference in μg at 45% RH
Difference in μg at 35% RH

Difference in filter weight calculated by using the initial weight at 36% RH and subtracting the weight of the filter in the test condition.

-20 -15 -10 -5 0 5 10 15 20

0 5 10 15 20 25 30 Difference Between Weight Measurements in Micrograms (Weight at 22o C used as Control)

Filters Tested 

# Figure 2: Raw Weight Differences in Filters Measured at Different Temperatures

Difference in μg at 18°C
Difference in μg at 25°C
Difference in μg at 22°C (repeat)

Difference in filter weight calculated by using the initial weight at 22 o C and subtracting the weight of the filter in the test condition.15

Most of the differences in the filter measurements fell within + 15 μg regardless of the conditions. Th ere were some outliers in the measurements taken at different temperatures, but most of these occurred during the repeat weighing two days later under the ideal conditions. Table 1 shows the mean differences, 95% confidence intervals, standard deviations, standard errors, and ranges for all filters
under the different conditions.

Table 1: Mean Differences (From First Measured Weight at 36% RH or 22 ° C), Standard Deviations,

Standard Errors, and Ranges for Filters Under Different Conditions

Mean Differen ce 95% Confidence

Interval Std.

Deviation Std. Error Range

Reweighed at 36% RH 2.4 1.5

- 3.3 2.5 0.5

- 4 to 6 Weighed at 35% RH 0.6

- 0.7

- 2.0 3.7 0.7

- 8 to 10 Weighed at 25% RI 3.3 1.9

- 4.7 3.7 0.7

- 4 to 12 Weighed at 45% RH

- 1.6

- 3

- - 0.1 3.9 0.7

- 9 to 5 Reweighed at 22 degrees C 3.1 1.8

- 4.3 3.4 0.6

- 5 to 9 Weighed at 22 degrees C two days later 5 2.2

- 7.7 5 1.3

- 5 to 22

Weighed at 18 degrees C 5.5 3.9

- 7.2 4.4 0.8

- 4 to 15 Weighed at 25 degrees C 2.1

- 0.1

- 4.3 5.9 1.1

- 7 to 16

Figure 3 shows the mean differences and 95% confidence intervals plotted out for side by side comparison, and demonstrate that the level of variation is well within expected error of measurement.

2.4
0.6 3.3

- 1.6 3.1 5 5.5

2.1

-15 -10 -5 0 5 10 15

Reweighed at 36% RH Weighed at

35% RH Weighed at

25% RI Weighed at

45% RH Reweighed

at 22 degrees C Weighed at

22 degrees C two days later Weighed at 18 degrees C Weighed at

25 degrees C 

### Figure 3: Mean Difference and 95% Confidence Intervals for Weight Difference between Control Conditions (First Measured Weight at 36 %

RH or 22 o 

### C) and Other Conditions

95% CI Lower
95% CI Upper

Mean Difference16

Figures 4, 5, 6, 7, 8, 9, 10, and 11 show box plots and T OST results for the mean differences each filter category. All TOST results concluded that the mean difference between the control and experimental condition was highly likely to be between + 15 μg. Plots were done using SAS, version 9.3.171819

The TOST procedure in all of the experimental cases gave results indicating that the estimated mean differences between the measurements were highly likely to be between + 15 ug.20 Discussion

The majority of the weight measurements taken under the experime ntal conditions fell within + 15 μg of the original measurements (see Figure 1 and 2). Confidence intervals for the mean differences were calculated using the TOST procedure. The mean differences between the filter weights measured
under the recommended co nditions and the experimental conditions were highly likely to fall within + 15 μg, based on the data collected.

All the calculated mean differences seen between the filters weighed under the recommended conditions and the experimental conditions were ver y small. If the changes in the environment had an effect on the weight of the filters, there was not enough precision between the measurements to tease out the difference. For instance, the filters that were weighed at 22 o C under ideal conditions and then immediately reweighed (see Figure 8) showed a “significant” (the 95% confidence interval did not include 0) mean difference in weight between the measures. This is highly unlikely to be a true difference in weight, but is likely a reflection of the random “noise” that occurs between instruments. Per the testing recommendations, the two measurements at the ideal conditions would be considered the same, despite the small amount of noise seen between the measurements.

Given these results, measures of filter weight under the experimental conditions (relative humidity levels of 45% and 25%, and temperatures of 18 o and 25 o C) have little or no measurable effect on the filter weight. Measures within these extended ranges should be considered statistically similar to measures of filter weight made under the recommended conditions with a high degree of confidence.21



## Knox County Filter Data Evaluation 2009 to 201322



## All Sites Fil ter Data Evaluation 2009 to 201223



## Additional Documents Reference d in this Report (Provided as separate attachments)

Environmental Condition Testing of 31 Sampled Measurement Technology Laboratories 46.2

- mm PTFE (Teflon®) Filters by Paige Presler

- Jur, RTI International, July 18, 2014

Qual ity Assurance Project Plan , TDEC PM2.5 Filter Investigation Study , June 17, 2014, by Robert Brawner, Technical Services Program, Air Pollution Control Division, Tennessee Department of Environment and Conservation