1

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY 

Office of Air Quality Planning and Standards (OAQPS) 

Research Triangle Park, North Carolina 27711 

April 7, 2011 

MEMORANDUM

SUBJECT: Assessment of PM2.5 FEMs Compared to Collocated FRMs 

FROM: Tim Hanley and Adam Reff, OAQPS /s/ 

TO: PM NAAQS Docket, EPA - HQ - OAR - 2007 - 0492

Objective 

To assess the quality of recently approved PM2.5 continuous Federal Equivalent Method (FEM) 

data as operated in routine monitoring networks. 

Background 

On October 17, 2006, EPA published Revisions to the Ambient Air Monitoring Regulations that 

included new performance criteria for acceptance of continuous PM2.5 monitors as Class III 

Federal Equivalent Methods (FEM). PM2.5 Class III Federal Equivalent Methods (FEMs) are 

analyzers capable of providing ambient air measurements representative of one-hour or shorterperiod integrated PM2.5 concentrations as well as 24-hour measurements determined as, or 

equivalent to, the mean of 24 consecutive one-hour measurements. The testing and performance 

criteria for Class III FEMs can be found in 40 CFR Part 53. 

Since finalizing these new testing and performance criteria several instruments have received 

designation as an FEM. EPA maintains a list of approved FEMs on its web site at: 

http://www.epa.gov/ttn/amtic/criteria.html

Available Data

In the fall of 2010, we retrieved collocated PM2.5 FRM and continuous FEM data that had been 

submitted to the Air Quality System (AQS) database. As of that time, the majority of available 

data were associated with the following two methods: 

1. Met One BAM-1020 Monitor as an automated equivalent method EQPM-0308-170 (61 

sites collocated with an FRM); and 

2. Thermo Scientific Ambient Particulate Monitor with Series 8500C FDMS as an 

automated equivalent method EQPM-0609-181 (17 sites collocated with an FRM) 
2

We also had sufficient data to evaluate a third method, but only at two locations; therefore we are 

including the two sites assessed for this method in the attachments, but are not analyzing the data 

across sites for this method. 

3. Thermo Scientific Model 5030 SHARP Monitor as an automated equivalent method 

EQPM-0609-184 (2 sites collocated with an FRM) 

Methods used to Assess the Collocated Data 

Since the FRM measurement is a 24-hour integrated average, we can only evaluate the FEM for 

the coincident 24 consecutive hours where both the FEM and FRM are operating (i.e., midnight 

to midnight local standard time for days when the FRM is operated). In these assessments, we 

are using data from sites with collocated FRM and FEM as there are often high numbers of 

sample pairs available to assess the FEM performance. Therefore, the data used in this 

assessment are from the collocated FRMs operated by the same monitoring agency1

. 

The statistics used in this assessment are based on the performance criteria for approval of Class 

III FEMs from 40 CFR Part 53, which are derived from the related PM2.5 data quality objectives2. 

There are related data quality goals defined in the Quality Assurance Requirements for Air 

Monitoring (Appendix A to 40 CFR Part 58). The Appendix A goals are for comparison to the 

measurement uncertainty for automated and manual PM2.5 methods. The goal for this acceptable 

measurement uncertainty is defined as 10 percent coefficient of variation (CV) for total precision 

and plus or minus 10 percent total bias. We chose to use the Part 53 performance criteria as 

these statistics provide more detailed information that has the potential to relate to instrument 

set-up, operation, and maintenance procedures. 

We used two approaches to assess the data. First, one-page assessments were produced for each 

FEM collocated with an FRM; these are presented in the attachments. Second, we took some of 

the key statistics (slope, intercept, and mean concentration) and compared the results across sites 

for the two FEMs most commonly available (i.e., Met-One BAM 1020 and Thermo 8500C 

FDMS). These are presented further below. 

The following steps were taken to prepare and assess the data for use in the one-page 

assessments presented in the attachments: 

1. Data were retrieved from EPA’s TTN web site 

(http://www.epa.gov/ttn/airs/airsaqs/detaildata/downloadaqsdata.htm) on October 6, 

                                                           

1 There is an audit program known as the Performance Evaluation Program, where independent auditors conduct 

FRM measurements using samplers that are temporarily collocated at stations with PM2.5 FRMs or FEMs; however, 

these audits are conducted only a few times a year (5-8 samples per year) per monitoring organization. 

2

 A report on this subject was reviewed by CASAC’s technical subcommittee on Ambient Air Monitoring and 

Methods at a meeting in Durham, NC on September 21-22, 2005. The report is available on the web at: 

http://www.epa.gov/ttn/amtic/files/ambient/pm25/casac/att4casac.pdf The report and comments from CASAC 

subcommittee members, in addition to public comments from the proposed rule, were all incorporated in the final 

performance criteria published in Part 53.
3

2010. The TTN provides publically accessible data files from EPA’s Air Quality System 

(AQS) database. 

2. The dataset included all valid data, including data flagged with an exceptional event 

code. 

3. PM2.5 continuous FEM data were used if the following criteria were met: 

a. There were at least 18 valid hours in a day from the FEM 

b. There was a valid PM2.5 FRM and FEM measurement for a given day. 

4. Assessments were conducted using the performance criteria from Table C-4 and Figures 

C-2 and C-4 of 40 CFR Part 53 

5. Assessment periods for each site included: 

a. All data 

b. Each calendar year 

c. Seasons as defined by calendar (e.g., Winter is December 21 to March 20). Note: 

this is not prescribed in regulation, but is a useful way to look at the instruments 

performance according to season. 

6. Assessments by station included: 

a. A scatter plot with a linear regression. A one-to-one line is also drawn on each 

figure. 

b. A time series chart illustrating the difference between the FRM and continuous 

FEM data. 

c. An illustration of the additive and multiplicative bias as compared to the 

acceptable limits for these statistics (Part 53 table C-4). 

d. An illustration of the correlation as compared to the acceptable correlation. Note: 

the statistic used here is r and not r2. 

e. A summary of the number of data points, means of the data, and ratio of 

continuous to FRM is presented in the middle of the page. 

7. While every site with collocated FRM and continuous FEM was analyzed3, we are only 

presenting data from those sites with at last 23 sample pairs, which is the minimum 

number of sample pairs required for each test campaign, (described in Table C-4 to Part 

53). 

Results

Attachments 1, 2, and 3 provide each of the one-page assessments. 

A summary of the slope, intercept, and mean of the FEM and FRM statistics from the one-page 

assessments is presented below. In each case, the X-axis is organized by site from low to high 

with the Y-axis being the slope, intercept, or mean concentration value. The orange boxes 

represent the acceptance criteria for the applicable metric (i.e., slope or intercept). The red 

arrows represent the median site for the slope and intercept figures. Note: the slope and 

                                                           

3 All available sites in AQS were used except for one site in South Dakota (AQS ID: 46-033-0132). This site was 

impacted by a substantial PM event around September 4, 2009. The PM2.5 measured during this event was in the 

hundreds of micrograms per cubic meter and would therefore not be representative of the performance of the 

instrument in the range of interest.
4

intercepts are presented independently here, while the one-page assessments in the attachments 

appropriately consider them together. 

0.7

0.8

0.9

1

1.1

1.2

1.3

1.4

1.5

1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

Slope

Met One BAM 1020 ‐ FEM

Average slope for all sites = 1.081

Average slope for sites with an intercept within +/‐ 2 μg/m3 = 1.066

Figure 1 – Met One BAM 1020 with acceptable slopes illustrated in the box. 

‐6

‐4

‐2

0

2

4

6

8

10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Intercept (μg/m3)

Average intercept for all sites = 1.12 μg/m3

Average intercept for sites with a slope within +/‐ 10% of 1 = 1.34 μg/m3

Met One BAM 1020 ‐ FEM

Figure 2 – Met One BAM 1020 with acceptable intercepts illustrated in the box 
5

Met One BAM 1020 ‐ FEM

0

5

10

15

20

25

30

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59

micrograms per cubic meter

Mean Concentrations ‐ All Seasons

FRM

FEM

Figure 3 – Met One BAM 1020 with mean concentration by site for both the FRM and 

FEM 

Met One BAM 1020 ‐ FEM

0

5

10

15

20

25

30

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39

micrograms per cubic meter

Mean Concentrations for Winter

FRM

FEM

0

5

10

15

20

25

30

1 3 5 7 9 111315171921232527293133353739414345

micrograms per cubic meter

Mean Concentrations for Spring

FRM

FEM

0

5

10

15

20

25

30

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

micrograms per cubic meter

Mean Concentrations for Fall

FRM

FEM

0

5

10

15

20

25

30

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37

micrograms per cubic meter

Mean Concentrations for Summer

FRM

FEM

Figure 4 – Met One BAM 1020 with mean concentration by site by season for both the 

FRM and FEM 
6

8500C FDMS ‐ FEM

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

1.4

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Slope

Average slope for all sites = 0.937

Average slope for sites with an intercept within +/‐ 2 μg/m3 = 0.926

Figure 5 – Thermo 8500C FDMS – FEM with acceptable slopes illustrated in the box 

8500C FDMS ‐ FEM

‐3

‐2

‐1

0

1

2

3

4

5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Intercept (μg/m3)

Average intercept for all sites = 1.40

Average intercept for sites with a slope within +/‐ 10% of 1 = 0.68

Figure 6 – Thermo 8500C FDMS – FEM with acceptable intercepts illustrated in the box 
7

8500C FDMS – FEM

Mean Concentration – All Seasons

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

FRM

FEM

Figure 7 – Thermo 8500C FDMS – FEM with mean concentration by site for both the FRM 

and FEM 

8500C FDMS ‐ FEM

0

5

10

15

20

25

1234

micrograms per cubic meter

Mean Concentrations for Fall

FRM

FEM

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14

micrograms per cubic meter

Mean Concentrations for Winter

FRM

FEM

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

micrograms per cubic meter

Mean Concentrations for Spring

FRM

FEM

0

5

10

15

20

25

1234

micrograms per cubic meter

Mean Concentrations for Summer

FRM

FEM

Figure 8 – Thermo 8500C FDMS – FEM with mean concentration by site by season for 

both the FRM and FEM 
8

Discussion 

There are a wide variety of results for each method. In some cases the collocated FEM is 

producing data that is meeting the performance criteria described in 40 CFR Part 53; however, in 

other cases it is reading differently and often higher than the collocated FRM. 

A summary of each of the major statistics for the two methods most widely deployed is 

described here: 

Slope for the Met One BAM 1020 

About 2/3 of the sites had acceptable slopes; however, on average these acceptable slopes were 

reading about 6-7% high. This may be due to a number of issues under investigation. Some of 

the possible explanations might include under performance of the “Smart Heater” sample 

conditioning, inaccurate calibration of the relative humidity sensor, and inconsistent control of 

the air conditioning during warm months of the year. 

Intercept for the Met One BAM 1020 

About 2/3 of the sites had an acceptable intercept; however on average these acceptable 

intercepts were a little over a microgram per cubic meter higher than zero. Given that the Met 

One BAM 1020 has a zero testing procedure as part of its installation, we might expect the 

average to fall closer to zero, with some scatter above and below zero. 

Mean Concentration (all data) for the Met One BAM 1020 

Most sites either predicted about the same mean as the collocated FRM or were 2-5 micrograms 

per cubic meter higher than the FRM data. Only a few sites had FEM data with a mean 

concentration lower than the FRM and in those few cases these values were usually within two 

micrograms per cubic meter of the FRM mean concentration. 

Mean Concentration (by season) for the Met One BAM 1020 

For most sites each of the seasons still shows a positive bias (the FEM reading higher than the 

FRM); however, the biases appear more pronounced during the spring and summer. This could 

be due to the differences in ambient temperature between winter and spring/summer. 

Slope for the 8500C FDMS 

About half of the FEMs had acceptable slopes; however, figure 5 clearly illustrates that the 

middle of the dataset has a consistent slope of about 0.9. The average slope for sites with an 

acceptable intercept was 0.926, thus the slope is lower than may be expected. 

Intercept for the 8500C FDMS 
9

About 2/3 of the sites had an acceptable intercept and on average these acceptable intercepts 

were about 0.68 micrograms per cubic meter higher than zero. There is also a noticeable tail of 

sites with large and unacceptable intercepts (>2 μg/m3

) possibly indicating a set-up or 

operational problem for these sites. 

Mean Concentration (all data) for the 8500C FDMS 

Most sites either predicted the same mean as the collocated FRM or were 2-4 micrograms per 

cubic meter higher than the FRM data. Only a few sites had FEM data with a mean 

concentration lower than the FRM and in those few cases these values were usually within one 

microgram per cubic meter compared to the FRM. 

Mean Concentration (by season) for the 8500C FDMS 

The seasonal data was very similar for fall and winter. The spring data showed substantially 

higher (4-6 μg/m3) concentrations for 4 sites and very similar data for the others. The summer 

had mixed results, but there were only 4 sites with enough data points to analyze. The higher 

data in the spring at 4 sites may be a result of volatile PM2.5 being readily measured by the 

8500C FDMS FEM, which incorporates measurement of volatile PM as part of its design. 

Conclusions 

The assessments presented illustrate that a sub-set of monitoring sites are generating data that are 

meeting the performance criteria used to approve PM2.5 FEMs. However, there are clearly a 

number of sites with continuous FEMs that are not meeting these performance criteria. In these 

cases, agencies should reexamine their set-up and operational procedures, including data 

reporting, to attempt to maximize their data quality. To assist in the effort to maximize data 

quality, EPA ambient air monitoring staff are working with the monitoring committee of the 

National Association of Clean Air Agencies, EPA Regional Offices, EPA’s Office of Research 

and Development, and instrument manufactures to use these results and other assessments in 

combination with set-up and maintenance information to document best practices for the set-up, 

operation, and reporting of data from approved FEMs. Ideally, a best practices document will be 

used in combination with this and future assessments to target troubleshooting of 

instrumentation. 

The lack of acceptable performance data at some sites with FEMs when compared to collocated 

FRMs, on a 24-hour basis, calls into question the use of these continuous FEMs. For the PM2.5 

primary standard, monitoring agencies have the option of continuing to use FRMs4, or where 

applicable, using a well performing continuous FEM. The annual monitoring network plan 

(described in §58.10), due to the applicable EPA Regional Office by July 1 of each year, is the 

appropriate place for monitoring agencies to identify the methods and sampling frequencies it 

will operate in its network. In cases where a PM2.5 continuous FEM is not meeting the part 53 

                                                           

4 The data quality of each monitoring agencies FRM operation is available in assessments on the EPA web site at:

http://www.epa.gov/ttn/amtic/anlqa.html.  National statistics on the FRM data quality are described in the PM

ISA.   
10

performance criteria, we recommend keeping the PM2.5 FRM as the Primary monitor5 while 

working towards improvements in FEM data quality. For those agencies with well performing 

PM2.5 continuous FEMs, we support the use of these instruments in the agencies network. 

The lack of more uniform and acceptable performance data for PM2.5 continuous FEMs 

compared to FRMs on a 24-hour basis clearly calls into question the use of FEM data to support 

a possible secondary standard with a sub-daily averaging time (e.g., one-hour or four-hour). 

While some agencies are achieving good data quality with PM2.5 FEMs on a 24-hour basis, 

others are not, at this time. Since a possible secondary standard with a sub-daily averaging time 

would necessitate having monitoring agencies use continuous PM2.5 FEMs, we are concerned 

that such a requirement could not be implemented for all agencies. Until such time as there are 

continuous PM2.5 FEMs operated across the nation in State and local networks that can be 

demonstrated to meet the acceptable performance criteria defined in 40 CFR Part 53 (which is 

based on 24-hour measurements), we should move cautiously about considering wider use of 

these instruments for sub-daily measurement needs. 

 Attachments: 

1. Met One BAM 1020 Assessments – 61 Sites. 

2. Thermo Scientific Ambient Particulate Monitor with Series 8500C FDMS Assessments - 

17 sites. 

3. Thermo Scientific Model 5030 SHARP Assessments - 2 sites. 

                                                           

5 A Primary monitor is the monitor designated by the State, local or Tribal agency in their annual monitoring

network plan to be compared to the NAAQS.  Primary monitors must be FRMs, FEMs, or ARMs.