1 UNITED STATES ENVIRONMENTAL PROTECTION AGENCY Office of Air Quality Planning and Standards Research Triangle Park, NC 27711

April 5, 2011

MEMORANDUM

SUBJECT: Investigation of 1-hour PM 2.5 Mass Concentration Data from EPA-Approved Continuous Federal Equivalent Method Analyzers

FROM: Mark Evangelista, OAR/OAQPS/AQAD /s/

TO: PM 2.5 NAAQS Review Docket (EPA-HQ-OAR-2007-0492)

Summary

This memorandum documents systematic analyses of hourly PM 2.5 mass concentration data obtained from continuous Federal Equivalent Method (FEM) analyzers operated by state and local air monitoring agencies and reported to EPA’ s Air Quality System (AQS). The quality of hourly data from FEM analyzer is of interest in connection with the current review of the secondary PM 2.5 NAAQS, due to the possible use of such data to calculate PM 2.5 light extinction on a sub-daily basis (e .g., 1 hour or 4 hours).

Calendar year 2009 data from 109 FEM instrument s were obtained from AQS. As discussed later, three sites were eliminated from furthe r consideration based on an obvious data validation failure. Of the remaining 106 instruments (105 site s) only one site had co llocated instruments of the same model, such that instrument readings could be compared to get a sense of data reproducibility. At the other 104 single-instrument sites, the analyses of the quality of hourly data were based on “common sense” expectations for the range and tempor al patterns of actual
ambient concentrations.

The following observations were reached:

 Although FEM instruments are capable of bein g paired with a digital data logging system, not all monitoring programs do so. Some reported concentrations of approximately 1000 μg/m 3 were traced to the pairing of one particular instrument model with older analog data logging systems. Arme d with this awareness, it should be possible for monitoring agencies to avoid reporting such erroneous data in the future, even if they continue to rely on older analog systems.

 Data from the one available pair of collo cated instruments of the same model show periodic inconsistenc ies in hour-to-hour PM 2.5 data. On 26 days in 2009, there was at least one hour during which the inst ruments differed by at least 10 μg/m 3 , with one2 monitor reading below 10 μg/m 3 and the other reading above 20 μg/m 3 . However, the
annual 90 th and 95 th percentile daily maximum 1-hour concentrations for the two instruments, respectively, were within 10 percent of each other, indicating that for this site and this year the errors did not happen in a pattern that would have greatly affected compliance determinations against a st andard with this statistical form. 1 Nevertheless, this observation from a single year of such da ta from a single instrument does not provide
much confidence that measurement errors fo r hourly concentrations would not affect compliance determinations at least some times if NAAQS implementation were based on a network of monitoring sites using this instrument model.

 When data from this same pair of collocate d instruments are used to calculate moving 4- hour average concentrations, the results from the instruments become more similar to each other and the occurrence and magnitude of extreme values are reduced, confirming the expected tendency of multi-hour averages to mute the effect of hourly measurement

errors.

 Among the 106 individual instruments, in stances of very high reported 1-hour concentrations and/or patterns involving isolated high peaks in concentration were never so frequent as to signif icantly affect the annual 95 th percentile or annual 90 th percentile daily maximum 1-hour concentration, if such 1-hour concentrations were assumed to be incorrect readings and were re placed by much lower values.

It has not been possible to asse ss the representativeness of the vast majority of the moderately high concentrations reported by the 104 single-inst rument sites because of the lack of any known value for “true” hourly concentrations. Nevertheless, the findings for the one site with a pair of collocated instruments, the occurre nce of very high and/or highly peaked values at some singleinstrument sites, and the hour-to- hour variability seen in visual time series displays of the concentration data suggest that considerable cau tion is warranted when contemplating the use of hourly data from these FEM instruments in any calculations of PM 2.5 light extinction.

Background

Consideration of potential revi sed secondary standards for PM 2.5 to address PM-related welfare effects involves addressing visi bility impairment, which is caused by light scattering and absorption by suspended particles and gases. In itial drafts of the Policy Assessment for the current review of the secondary PM 2.5 NAAQS contemplated the use of calculated PM 2.5 light extinction, estimated based on 24-hour PM 2.5 speciation data and hourly PM 2.5 mass data, as an indicator for a revised secondary standard. Th e contemplated method includes simple algorithms to estimate light extinction attributed to each of the major PM 2.5 components. By applying the speciation profile associated with a Chemical Speciation Ne twork monitor to the hourly PM 2.5 concentration values reported by a collocated continuous mass monitor implementing a Federal

1 A revised PM 2.5 NAAQS could be based on a three-year 90 th or 95 th percentile form. To calculate these percentiles, I determined a percentile value depending upon the number of da ys in the data set, rounded up to the nearest 10 (for
the 90 th percentile case) or 20 (for the 95 th percentile case) days. For example, fo r a full year of data (365 days) the
95 th percentile daily maximum value is the 19 th highest value. This method is based on the method used to identify
98 th percentile concentrations for the 2006 24-hour PM 2.5 NAAQS, as described in 40 CFR 50 Appendix N.3 Equivalent Method (FEM), hourly values of the major PM 2.5 components could be estimated and input into a version of the IMPROVE algorithm for calculating light extinction.

The analyses in this memo were designed to high light outlier concentrati on values that were so high that they seemed highly unlikely to represen t actual ambient conditions, especially if they were frequent and had no obvious explanation ba sed on the monitoring site. However, some of the same (largely unknown) causes of such very high outlier concentrations may, to a lesser degree, also affect reported concen tration values in or near the ra nge 20-50 μg/m3. This range of
PM 2.5 concentration is of interest because, depending on species mix and relative humidity,
PM 2.5 concentrations in this range can translate into PM 2.5 light extinction levels in the range of 25-30 deciviews, which is the range of mo st interest in the Policy Assessment.

Federal Equivalent Method Monitors and the Data Set Used in this Investigation

Monitor models, manufacturers, and monitoring me thods used to obtain the data set for this investigation are listed in Table 1.

Table 1. Information about monitoring instrumentation. The legend for the table is found in the
first row.

Number
of
Monitors

in 2009
Data Set Monitor Model Name
Website of the monitor model manufacturer Federal Equivalent Method [and reference location]

91* Met One BAM-1020 Mass Monitor w/VSCC

www.metone.com/particulate.php

Method EQPM-0308-170 [Federal Register: April 25, 2008 (Vol. 73, Num 81) Pg. 22362-22363]

1 Thermo Scientific 1405-DF Dichotomous FDMS

www.globalspec.com/FeaturedProducts/Detail/T hermoFisherScientificAirQualityInstruments

Method EQPM-0609-182 [Federal Register: June 17, 2009 (Vol. 74, Num 115) Pg. 28696-28698]

15 Thermo Scientific 8500C FDMS w/VSCC

www.thermoscientific.com/wps/portal/ts/products/detail

Method EQPM-0609-181 [Federal Register: June 17, 2009 (Vol. 74, Num 115) Pg. 28696-28698]

2 Thermo Scientific Mode l 5030 SHARP w/VSCC

www.globalspec.com/FeaturedProducts/Detail/Model_5030_RealTime_Particulate_Monitor/17839/0

Method EQPM-0609-184 [Federal Register: June 17, 2009 (Vol. 74, Num 115) Pg. 28696-28698] * Data from three of the sites with this instrument model were set aside early in the conduct of these analyses, as described in the body of this memo.

I examined hourly PM 2.5 concentration values reported by FEM monitors for 2009. The investigation began in early 2010, and 2009 provided the latest seasonally-balanced data set for an entire year at that time.4 For this investigation, I obtained data from a ll 109 FEM monitors that had some data for 2009. Of the 109 monitors, 53 reported data for the entire year 2009, 21 monitors reported about 9 months of data, 15 monitors reported about six mont hs of data, and 20 monitors reported data for
3 months or less. I included data from all m onitors in the analyses without regard for data completeness. Of course, determination of de sign values for regulatory purposes would depend upon data completeness.

Instructions for access to the data used in th is investigation are given in Appendix A.

First Indication of a Data Quality Issue

One of the first observations in examining the data set was that thr ee sites had reported PM 2.5 concentration values of 985 μg/m 3 . These values were considered to represent self-evident monitoring system errors, and their cause was investigated by EPA staff in the Ambient Air Monitoring Group. It was determined that th e Met One BAM-1020 Mass Monitor used at these three sites, when paired with an analog data logg ing system, is typically run with a voltage range from 0 to 1 volt corresponding to a concentration range from -0.015 mg/m 3 to 0.985 mg/m 3 (i.e., -15 μg/m 3 to 985 μg/m 3 ). This offset allows for slight ne gative values, which may be within the noise of the instrument output when the atmos phere approaches an ambi ent concentration value near zero, to be includ ed in the reported hourly data. As a means of sending an error code when
a malfunction or out-of-limits condition is det ected, the monitor is programmed to send its analog output to the full-scale voltage of 1 volt, rather than report a questionable concentration
value. 2 If the data logger (and/or downstream data processing steps) is de signed to convert every recorded voltage into a concen tration using a simple linear algorithm, the hour for which the voltage output was 1 volt will be treated as having a concentration value of 985 μg/m 3

.

I eliminated these three sites from the investigation data set, leaving data from 106 instruments at 105 monitoring sites. Additionally, OAQPS and Regional Office staff are discussing this issue with Met One and with State monitoring operators to encourage corrective action, and to ensure monitoring agencies remove such clear ly incorrect data before submission.

Analytical Approaches

Several analytical approaches were followed



### .

1. A qualitative, exploratory data inspection in which time series graphics were generated for every FEM monitor with at least one “day light” 1-hour concentration in excess of 150 μg/m 3

. 2. Graphical analysis of the time-matched data fr om the one site with a pair of collocated
instruments.
3. Analysis of the effect of 4-hour averagi ng on the annual daily maximum concentration
values.
4. Application of two different approaches for qualitatively defining (“flagging”) what data could potentially be erroneous. Data identi fied by these approaches were graphically

2 When the instrument is paired with a digital data logger, a separate data channel is used to report such conditions explicitly.5 inspected in temporal and spatial context to determine whether it might be plausible that the data represent legitimate measurements.
5. An analysis in which reported 1-hour concentr ations that arguably could be potentially erroneous, as identified by Approach 4, were replaced with plausible values. The point of the substitution process was to produce an a lternate data set that might better represent true ambient concentrations, to create a ba seline for quantifying th e effect of possible measurement errors in the actual reported data . I examined how the substitutions affected the annual maximum, the annual 95 th percentile, and the annual 90 th percentile values of the daily maximum 1-hour PM 2.5 concentration values. The e ffect of errors/substitutions
on the annual 95 th percentile and annual 90 th percentile values of the daily maximum 1-
hour PM 2.5 concentration values was of interest because a revised PM 2.5 NAAQS could be based on a three-year 90 th or 95 th percentile form.

1. Qualitative, Exploratory Data Inspections

I immediately noticed that some FEM monitors occasionally reported high hourly PM 2.5 concentrations. These concentration spikes sometimes occur for only one hour, with the hours immediately before and after s howing a significantly lower concen tration. Other high hourly
PM 2.5 concentration values appear to take several hours to build a nd subside. Also, there were cases in which a very high reading was unique during the year and other cases where high readings occur more frequently. For example, a time series of hourly data reported by a monitor in Zion National Park, UT (AQS ID 49-053- 0130) for 2009 (Figure 1) shows one hourly concentration of nearly 250 μg/m 3 , but no other hourly concentr ation values gr eater than 80 μg/m 3 were reported in all of 2009. In contrast , Figure 2 shows the hourly time series for 2009 concentrations at a monitor in Allentown, PA (AQS ID 42-095-0025), wher e high concentrations occurred several times during the year. In bot h cases, these high concentration values may represent the actual pattern of ambient particulate matter. Or, they may be erroneous values that are due to instrument issues such as siting, main tenance, calibration, inst rument design, or oneoff instrument defects. Errors in data handling and transfer ar e also a possibility.

I next conducted a visual inspection of the hourly data from each monitor that included at least one “daylight” concentration value in excess of 150 μg/m 3 . This was the lowest round value that would cause both sites mentioned in the previous paragraph to be included. I used a smaller dataset of only “daylight” hours for expediency. 3 In the 2009 data, I identified 12 monitors that had at least one “daylight” concen tration value great er than 150 μg/m 3 . Figure 3 shows the highest daily maximum concentrat ion value for each of these m onitors, compared to the second highest daily maximum concentration value. Figure 4 shows the annua l time series for the site at Turlock, CA (AQS ID 06-099-0006, denoted as mon itor 3 in Figure 3). This time series shows

3 The methods contemplated in the Policy Assessment to estimate visibility from, among other information, PM 2.5 values reported by FEM monitors, involve several steps and inputs. Two such steps are sc reening the data to include only daylight hours, and screening the data to include on ly those hours when relative humidity is reported below a
certain value. To allow full use of values from sites with only a few months of reported data, and because there was

no a priori reason to suspect that the time of day or high humidity play a role in the generation of questionable concentration values, I tested most flagging algorithms on the full data set without filtering for relative humidity or daylight hours. However, computational limitations sometime s required a smaller data set, and in those cases the test was performed on data for only a subset of hours. Those tests performed on this subset—referred to as “daylight” hours, defined as the hours 5:00 AM to 7:00 PM local tim e (no Daylight Savings correc tion)—are clearly identified.6 one value above 150 μg/m 3 , but several other high values as we ll. In this context, perhaps the high value is valid as there are ot her days with relatively high c oncentrations in the same period.

Of the 12 monitors, only one ha d a second highest daily maximu m also greater than 150 μg/m 3

. In fact, four monitors had a sec ond highest daily maximum below 75 μg/m 3 , less than one-half the highest value. These four monitors each appeared to have one high concentration value unique to the rest of the year’s data. Figures 5-7 each depict a time series of the 2009 hourly
concentration values for one of these monitors. In Figure 5, the high value at the Honolulu site may have resulted from an upstream volcanic erup tion. Figure 6 shows another high value that, while it is much lower than the highest value, may lend plausibility to that highest value. Additionally, the high valu e occurs within 90 days of the st art of monitoring operation, perhaps suggesting an instrument issue. This possibility is more clearly illustrate d in the time series for another monitor in Figure 7.

2. Examination of Data from One Site w ith a Pair of Collocated Instruments

In 2009, data was reported from only one site with collocated FEM monitors—Missoula, Montana (AQS ID 30-063-0024, POC3 and POC4) —of the same model and method (Met One BAM-1020 Mass Monitors w/VSCC, Met hod ID EQPM-0308-170). Hourly PM 2.5 mass measurements were reported from both Misso ula monitors for the full year 2009. Figure 8 shows a scatter plot of the paired hourly PM 2.5 measurements reported by both monitors. I identified 26 days on which there was an hour when the concentration measured by one monitor was reported to be greater than 20 μg/m 3 , while the measurement by the collocated monitor for the same hour was reported to be less than 10 μg/m 3 . I examined these indi vidual days in greater detail. Time series for some of these days are found in Figures 9-11. In the figures, differences up to 10 μg/m 3 for the same hour can sometimes be seen between POCs 3 and 4, and these differences generally lasted 4 or more hours when they did occur.

High PM 2.5 concentration measurements are not specific to one single monitor at this site. Both monitors did report high PM 2.5 concentration values, but neve r on the same day. The monitor identified as POC3 reported more high values th an POC4. Figure 9 illustrates all days on which POC3 reported a high PM 2.5 concentration while POC4 did not. Figure 10 illustrates all days on which POC4 reported a high concentration while POC3 did not. Additionally POC4 reported measurements in a more erratic pattern than POC3 on several days, as illustrated by Figure 11, while POC3 exhibited no such erratic pattern.

3. Analysis of 4-Hour Averages

Four-hour averages were determined from the hourly data. If one hourly measurement was missing, an average was determined of the remain ing three hourly measurements. If there were two or more missing hourly measurements, the 4-hour average was not computed.

The 4-hour averaging was applied to the data fr om the one site with a pair of collocated instruments. Scatter plots of the 4-hour averages are show n in Figure 12. Not surprisingly, correlation of the 4-hour averages (0.910) was gr eater than correlation of the hourly data shown in Figure 8 (0.825). Figure 12 does not ha ve any 4-hour values above about 100 μg/m 3 , while
the 1-hour data set depicted in Figure 8 contained thr ee such values. Also, there are fewer cases7 in Figure 12 of data pairs in which the two PO Cs reported much different 4-hour values than there are of cases in Figure 8 in which th ey reported much different 1-hour values.

Figure 13 shows a scatter plot of the annual maximum daily maximum PM 2.5 concentrations for 1-hour vs. 4-hour values. Figures 14 and 15 show the same comparisons for the 95 th and 90 th percentiles.

Averaging would obviously reduce the impact of an outlier value, and it is reasonable to find several points in Figure 13 that represent 1-hour va lues greater than 50 μg/m 3 , and therefore highly likely to correspond to a light extinction value that ex ceeds 30 deciviews (dv), with a corresponding 4-hour average va lue within the 20-50 μg/m 3 range, where light extinction values greater than 30 dv may be less lik ely given other atmospheric circ umstances. Figure 14 shows the same comparison for the 95 th percentile and has a similar numb er of points of the same type as Figure 13. Figure 15, represen ting the comparison for the 90 th percentile has fewer points of this type, as one would expect from the lower percentile. Additionally, some 95 th and 90 th percentile 4-hour average va lues are less than 20 μg/m 3 with a corresponding 1-hour value well within the 20-50 μg/m 3 range.

4. Application of Two Data Flagging Algorithms

Algorithms using two different methods were deve loped for this portion of the investigation: flagging data greater than a certa in threshold value (“‘high value” ), and flagging data exceeding the average of adjacent values by a threshold (“sp ikes”). These algorithms were applied to the full 2009 dataset, and the flagged data were exam ined for plausibility. Similar results were obtained from applying the two algorithms. Fo r illustration, I will desc ribe only the algorithm and results for flagging data spikes exceeding th e average of adjacent values by a threshold.

This algorithm flagged any value more than 75 μg/m 3 greater than the average of concentration values immediately before and after the examined value. In other words, the concentration value was compared to the average of the concentration values for the previous hour and the following hour. This average is the linearly interpolated value between those adja cent hours, which would be a reasonable initial estimate of a missing va lue between these adjacent values. If the
measured concentration value exceeded th e interpolated value by more than 75 μg/m 3 , the concentration value was flagged. This threshol d value was chosen because one hourly value 75 μg/m 3 higher than the adjacent average would be su fficient to create 4-hour averages above 20 μg/m 3 even if the other hourly values in the average were very low.

Of the 106 instruments in the 2009 data, this algo rithm flagged data for 31 instruments. Ten monitors had 2-4 flagged concentration values, and 20 monitors had only one hour flagged. The only monitor with more than 4 flagged con centration values was located in the West Yellowstone City Center, Montana (AQS ID 30-031-0016), which had 9 such hourly values. Most of the flagged hours were in the evening or night and all were in th e cool season, indicating possible wood burning under night time inversions as the cause of legitimate measurements
rather than instrument errors. The 2009 time se ries for this monitor is shown in Figure 16.8 5. Analysis of Data Substitution Algorithms

Hourly concentration values that had been fla gged by two algorithms in Approach 4—high-value and spikes—were eliminated from the data and re placed by predetermined concentration values. Daily maximum PM 2.5 concentrations for the data set in cluding the substituted values were compared to daily maximum PM 2.5 concentrations for the original data without any flagging or substitution. Comparisons involving the two flagging/substitution algor ithms yielded similar results. This analysis involving the spike algorithm is explained in detail.

I assumed all the flagged concentrations values were invalid, and I s ubstituted the linearly interpolated values for the flagged values at these 31 monitors. The annual maximum daily
maximum concentration values for the 2009 data with replacement of flagged values was compared to that for the 2009 data without any replacement in Figure 17. Replacement of data flagged using this algorithm a ppeared to affect the annual 95 th percentile daily maximum
concentration values at about 10% of th e monitors (Figure 18). The annual 90 th percentile daily maximum concentration values at only three mon itors appeared to be affected by the flagging and substitution algorithm (Figure 19).

The flagging and substitution approaches were al so applied to the data from both POCs at the collocated site, and the results compared. Ta ble 2 shows the annual maximum daily maximum, the annual 95th percentile daily maximum, a nd the annual 90th percentile daily maximum for POC3 and POC4, determined from th e original data, as well as from the results of substitution by two different algorithms.

The high-value algorithm did not change the annual 95 th percentile daily maximum or the 90 th percentile daily maximum at POC3, presumably b ecause the day with the flagged concentration value did not fall below the percentile day when re-ranked after substitu tion or, if the day fell
below the 95 th or 90 th percentile day, the next day was also 37.1 μg/m 3 . However, the spike algorithm and substitution did change the annual 95 th and annual 90 th percentile daily maximum values, presumably because the 3 flagged valu es did not all occur on the dame day, and the substituted days changed the ranking of the pe rcentile days. These changes were less than 10 percent.

At POC4, neither substitution algorithm altered the annual 95 th or 90 th percentile daily maximum concentration values, presumably because the da y with the flagged concentration value did not fall below the percentile day when re-ranked after substitution or , if the flagged day fell below
the 95 th or 90 th percentile day, the next day was of the same value as the original 95 th or 90 th percentile day.

The flagging and techniques do not offer much if any insight into the unexpl ained serious lack of reproducibility in the bulk of the collocated data.9 Table 2. Comparison of Annual Daily Maximum values for the collocat ed monitors at the Missoula MT site (AQS ID 30-063-0024). Values are substituted by the algorithm to flag values above a threshold (Hi-Val) and by th e algorithm to flag values considered sudden spikes (Spike). Column designated Original is data without any flagging or substitution.

Annual Maximum

Daily Maximum (μg/m 3 ) Annual 95 th Percentile Daily Maximum (μg/m 3 ) Annual 90 th Percentile Daily Maximum (μg/m 3 ) Hourly Values Flagged

Algorithm Original Hi-Val Spike Original Hi-Val Spike Original Hi-Val Spike Hi-Val Spike

POC 3 187.8 128.6 105.7 37.1 37.1 34.8 30.9 30.9 30.7 1 3

POC 4 201.2 99.0 99.0 40.1 40.1 40.1 32.5 32.5 32.5 1 11011121314151617181920212223242526272829 

## Appendix A

Instructions for Access to Data Used in This Investigation

2009 Hourly PM 2.5 concentration values from EPA-A pproved Continuous Federal Equivalent
Method Monitors.xlsx. This file can be downloaded at: http://www.epa.gov/ttn /analysis/pm.htm

.

Monitoring site information for PM 2.5 monitors, including those used in this inves tigation, can be downloaded at http://www.epa .gov/airexplorer/monitor_kml.htm . Select the file PM 2.5 CSV (11.4MB) under the heading “Criteria Pollutants.”