February 22, 2012

1

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY

RESEARCH TRIANGLE PARK, NC 27711

OFFICE OF AIR QUALITY PLANNING AND STANDARDS

Technical Note - PM2.5 Continuous Monitor Comparability Assessment

The PM2.5 continuous monitor comparability assessment tool is available at:

http://www.epa.gov/airquality/airdata/ad_rep_frmvfem.html

Summary: 

This tool provides a one-page technical report that assesses the comparability of a PM2.5 continuous monitor when 

collocated with an FRM sampler. These reports are intended to assist monitoring agencies in understanding if the 

PM2.5 continuous monitors operated in their network are appropriate for their intended monitoring objective (i.e., 

comparison to the NAAQS and/or reporting the AQI). Data are summarized by season across years, by year, and 

for all data. The most appropriate way to interpret the comparability of the PM2.5 continuous monitors is to look at 

either the entire data set, designated as “AllData” or “A”, or view the last complete year of data. The 

comparability assessments are presented in the context of several benchmark tests to assist with that evaluation. 

The assessment methods are described in detail in the following memo - Assessment of PM2.5 FEMs Compared to 

Collocated FRMs available at: http://www.epa.gov/ttnnaaqs/standards/pm/data/HanleyandReff040711.pdf. The 

memo also provides a set of graphical outputs for comparing the most widely used FEMs to the one-page 

assessments described in this technical note, allowing users to understand if their FEMs are consistent with FEMs 

operated in other agencies.

Description of Data and Assessments:

The following information describes the data and assessments in the one-page reports:

Table 1 – Dataset Descriptors

Dataset

Short 

Descriptor

Color of 

descriptor or 

data points Description

AllData A black Represents all the data in the assessment

Winter W blue Winter is represented by the dates December 21 – March 20.

Spring R green Spring is represented by the dates March 21 – June 20.

Summer S red Summer is represented by the dates June 21 – September 20.

Fall F brown Fall is represented by the dates September 21– December 20.

Year 0-9 black Full calendar year

Note: Seasons are fixed dates regardless of year.

Illustration of Linear Regression Relationship:

On the top left of the page a regression relationship is illustrated and the regression equation is presented along 

with the correlation of the equation. A 1:1 line is drawn on the regression relationship to quickly assess if data 

points are above, below, or straddling the 1:1 line. The FRM is presented on the X-axis, while the continuous 

method is presented on the Y-axis.

Illustration of Difference Trend:

On the top right of the page a trend is illustrated based on the difference between the continuous and FRM 

method. Data are presented in a color coded manner to determine any seasonal patterns, should they exist. 

Part 53 Specifications:
February 22, 2012

2

On the middle left side of the one-page assessment an illustration of the seasonal, yearly, and full data set for 

slope (multiplicative bias) and intercept (additive bias) is provided. This test is based on the specifications for 

PM2.5 candidate FEMs described in Table C-4 and illustrated in Figure C-2 from Subpart C to 40 CFR Part 53. 

From a linear regression output (y = mx + b), the slope (m) is plotted along the horizontal axis, while the intercept 

(b) is plotted along the vertical axis. 

On the middle right side of the one-page assessment, an illustration of the seasonal, yearly, and full data set for 

correlation coefficient (i.e., r and not r2) as compared to the concentration coefficient of variation (CCV) is 

provided. This figure is based on the Figure C-4 to Subpart C of Part 53. CCV is calculated using equation 22 in 

Part 53. The CCV is a statistic that describes the spread of the sample population. For example, a location with a 

concentration range of 0 – 25 μg/m3is expected to have a lower CCV than a location with a concentration range 

of 0 – 50 μg/m3. Datasets at or above the solid line meet the part 53 correlation criteria used in approving 

continuous PM2.5 FEMs. Data at or above the dashed line (r =0.9) meet the correlation criteria identified in 

guidance for reporting the AQI1. 

Mean of PM2.5 Dataset:

On the bottom left of the page, the mean for the FRM and continuous method are presented for all data, seasons, 

and years used in the assessment. A simple ratio of the continuous method over the FRM is calculated in the right 

hand column. 

Appendix A to Part 58 Statistics:

On the bottom right of the one-page assessment, equation 1 from Appendix A to Part 58 is used to calculate bias. 

In the bias calculation we first calculate individual paired biases as % difference = [[continuous –

FRM]/FRM]*100. Then we take the average of all the paired biases. These are described in detail in section 

4.3.2 of Appendix A. Statistical output for the Appendix A bias calculation is presented in the left column for all 

observations and on the right for those cases where both the FRM and continuous PM2.5 monitor are greater than 3 

μg/m3. Appendix A calls for only using data when the both observations are greater than 3 μg/m3; however, we 

calculate both options for users to see how low concentration data affects this statistic.

Interpreting the Comparability Assessment:

The one-page PM2.5 continuous monitor comparability assessment is intended to provide a concise description and 

illustration of the comparability of each operating PM2.5 continuous monitor that is collocated with an FRM. The 

assessment assumes that the operating FRM at the site represents a true value when compared to the PM2.5 

continuous monitor, even though the FRM will have its own uncertainty. Changes in the set-up or operating 

procedures of the PM2.5 continuous monitor (e.g., upgrading the firmware) during the period of the assessment 

may result in changes to the outputs. If changes have occurred, the time series difference assessment at the top 

right hand side of the one-page output may provide a useful tool to differentiate before and after the change. 

EPA has previously published or finalized a number of supporting documents and/or regulatory statements that 

describe acceptable performance criteria for PM2.5 continuous monitors. However, these documents or regulatory 

statements have common origins in the PM2.5 data quality objectives, which resulted in goals for measurement 

uncertainty that are published in the monitoring regulations. Since PM2.5 continuous methods inherently provide a 

daily sample, there is a statistical advantage over filter-based methods operating at less frequent sampling (e.g., 1-

in-3 day or 1-in-6 day sampling). This statistical advantage is captured in the acceptable multiplicative and 

additive biases presented in the middle left of the page where data may be within the slope and intercept 

boundaries, but not have a mean ratio of 1.00 +/- 0.10 as tabulated on the bottom left or be within 10% bias for 

the Appendix A statistics tabulated on the bottom right of the page. Therefore, while meeting the goal for 

measurement uncertainty bias provides confidence that the data are suitable for their intended purpose, not 

meeting this goal does not necessarily result in the rejection of the data or method; rather, it provides an 

 

1 Data Quality Objectives (DQOs) for Relating Federal Reference Method (FRM) and Continuous PM2.5 Measurements to Report an Air 

Quality Index (AQI), EPA 454/B-02-002, November 2002.
February 22, 2012

3

opportunity to investigate the performance of the method and consider the measurement uncertainty in the context 

of a decision error (performed as part of a DQO exercise) when data from a method are near the level of the 

NAAQS. The goals for measurement uncertainty are stated in Section 2.3.1.1 of Appendix A to Part 58 as:

Measurement Uncertainty for Automated and Manual PM2.5 Methods. The goal for acceptable measurement 

uncertainty is defined as 10 percent coefficient of variation (CV) for total precision and plus or minus 10 percent 

for total bias.

Total precision is not readily available from the PM2.5 continuous monitor comparability assessment since this 

statistic requires collocated data from the same make and model. However, bias is illustrated and calculated a few 

different ways (i.e., the illustration of additive and multiplicative bias, the ratio of the datasets, the Appendix A 

bias calculation for all data and for only those cases with data above 3 μg/m3). As described in the summary, the 

most appropriate way to interpret the comparability of the PM2.5 continuous monitors is to look at either the entire 

data set, designated as “AllData” or “A”, or view the last complete year of data. Monitoring agencies can also use 

this tool to identify outliers or to investigate seasonal patterns. More details on interpreting the comparability 

assessment are expected as users across monitoring agencies share their experiences and we collectively learn 

how to optimize the operation of each PM2.5 continuous method.

Parameter Codes:

At the top of the PM2.5 continuous monitor comparability assessment there are notes on the FRM and continuous 

PM2.5 methods and parameter codes used in the assessment. The tool will provide an output for any case when a 

PM2.5 continuous method is reported for the first four parameter codes below (i.e., 88101, 88500, 88501, and 

88502) and the site has a collocated PM2.5 FRM. EPA has details on these codes available on its web site at: 

http://www.epa.gov/ttn/amtic/datamang.html

For convenience these parameter codes are explained here:

Table 2 – Parameter codes available for use with PM2.5 continuous monitors.

Parameter Name

Parameter

Code Purpose Notes

PM2.5 LOCAL CONDITIONS 88101 Appropriate code for all FRM/FEM/ARMs

All PM2.5 continuous FEMs are to use this 

parameter code. Data are eligible for use in 

NAAQS calculations, unless labeled as 

“SPM” and “non-regulatory” for first 24 

months of operation. 

PM2.5 TOTAL ATMOSPHERIC 88500

Valid data from methods measuring total PM2.5

aerosols in the atmosphere, including those that can 

be volatilized from the FRM. 

This is the full characterization from any 

FDMS (i.e., base – volatile channel). Data 

are not used in NAAQS calculations.

PM2.5 RAW DATA 88501 Valid uncorrected data that does not reasonably 

match the FRM.

This code is expected when a PM2.5 

continuous method has any post processing 

of its data and there is a desire to have a 

historical record of the original data. Data 

are not used in NAAQS calculations.

ACCEPTABLE PM2.5 AQI & 

SPECIATION MASS 88502

Valid data that does reasonably match the FRM with 

or without correction, but not to be used in NAAQS 

decisions 

This parameter code is expected in most 

cases for pre-FEM versions of PM2.5 

continuous monitors. Data in this

parameter code are used in AQI reports. 

Data are not used in NAAQS calculations.

PM2.5 VOLATILE CHANNEL 88503 Stores important related data such as the FDMS 

reference channel 

This parameter code is not used in the onepage comparability report described above. 

Data are not used in NAAQS calculations.

Additional Notes:

The data source is the EPA AQS Data Mart, which is updated each week night from AQS.

A 24 hour average for the PM2.5 continuous method is produced for each day with at least 18 valid hours.

Data are only presented in cases with at least 23 valid sample pairs.

The processing of an assessment may take less than a minute to several minutes.

The assessment will provide an output of the last three years of data.
February 22, 2012

4

Data used in the mean for both the FRM and continuous monitor have been rounded to one decimal place, 

while the ratios are carried to two decimal places.

The mean PM2.5 concentrations will likely not represent exactly the overall mean of the site. On one hand, 

we only use days where both a valid FRM and continuous monitor data point are available; on the other 

hand, we use all the available data, even where identified as an exceptional event. However, it’s still 

likely that these mean values will be very close to a mean annual average for a given site.