February 2 2 , 2012

1 UNITED STATES ENVIRONMENTAL PROTECTION AGENCY RESEARCH TRIANGLE PARK, NC 27711 OFFICE OF AIR QUALITY PLANNING AND STANDARDS



## Technical Note

- PM 2.5 

## Continuous M onitor Comparability Assessment

The PM 2.5 c o ntinuous monitor comparability a ssessment t ool is available at: http://www.epa.gov/airquality/airdata/ad_rep_frmvfem.html

Summary: This tool provides a one

- page technical report that assesses the comparability of a PM 2.5 continuous monitor when collocated with an FRM sampler. These reports are intended to assist monitoring agencies in understanding if the

PM 2.5 continuous monitors operated in their network are appropriate for their intended monitoring objective (i.e.,

com parison to the NAAQS and/or reporting the AQI). Data are summarized by season across years, by year, and for all data. The most appropriate way to interpret the comparability of the PM 2.5 continuous monitors is to look at either the entire data set, design ated as “AllData” or “A”, or view the last complete year of data. The comparability assessments are presented in the context of several benchmark tests to assist with that evaluation. The assessment methods are described in detail in the following memo

- Assessment of PM 2.5 FEMs Compared to

Collocated FRMs available at: http://www.epa.gov/ttnnaaqs/standards/pm/data/HanleyandReff040711.pdf

. The memo also provides a set of graphical outputs for comparing the most widely used FEMs to the one

- page assessments described in this technical note, allowing users to understand if their FEMs are consistent with FEMs operated in other agencies.

Description of Data and Assessments : The following information describes the data and assessments in the one

- page reports :

Table 1

– Dataset Descriptors

Dataset Short

Descriptor Color of

descriptor or data points Description

AllData A b lack Represents all the data in the assessment

Winter W blue Winter is represented by the dates December 21

– March 20. Spring R green Spring is represented by the dates March 21

– June 20.

Summer S red Summer is represented by the dates June 21

– September 20.

Fall F brown Fall is represented by the dates September 21

– December 20.

Year 0

- 9 black Full calendar year Note: Seasons are fixed dates regardless of year.

Illustration of L inear Regression R elationship: On the top left of the page a regression relationship is illustrated and the regression equation is present ed along with the correlation of the equation . A 1:1 line i s drawn on the regression relationship to quickly ass ess if data points are above, b elow, or straddling the 1:1 line . The FRM is presented on the X

- axis, while the continuous method is presented on the Y

- axis.

Illustration of Difference Trend: On the top right of the page a trend is illustrated based on the difference between the continuous and FRM method. Data are presented in a color coded man ner to determine any seasonal patterns, should they exist.

Part 53 Specifications:February 2 2 , 2012

2 O n the middle left side of the one

- page assessment a n illustration of the seasonal, yearly, and full data set for slope (multiplicative bias) and intercept (additive bias) is provided. This test is based on the specifications for

PM 2.5 candidate FEMs described in Table C

- 4 and illustrated in Figure C

- 2 from Subpart C to 40 CFR P art 53. From a linear regression output (y = mx + b), the slope ( m ) is plotted along the horizontal axis, while the intercept ( b ) is plotted along the vertical axis.

On the middle right side of the on e

- page assessment, a n illustration of the seasonal, yearly, and full data set for
correlation coefficient ( i.e.

, r and not r 2 ) as compared to the concentration coefficient of variation (CCV) is provided. This figure is based on the F igure C

- 4 to Subpart C of Part 53. CCV is calculated usi ng equation 22 in
Part 53. The CCV is a statistic that describes the spread of the sample population. For example, a location with a concentration range of 0

– 25 μ g/m 3 is expected to have a lower CCV than a location with a concentration range
of 0

– 50 μg/m 3 . Datasets at or above the solid line meet the part 53 correlation criteria used in approving
continuous PM 2.5 FEMs. Data at or above the dashed line (r =0.9) meet th e correlation criteria identified in guidance for reporting the AQI 1

.

Mean of PM

2.5 Dataset: On the bottom left of the page

, the mean for the FRM and continuous method are presented for all data, seasons, and years used in the assessment . A simple ratio of the continuous method over the FRM is calculated in the right
hand column.

Appendix A to Part 58 Statistics: On the bottom right of the one

- page assessment

, equation 1 from Appendix A to Part 58 is used to calculate bias

. In the bias calculation we first calculate individual paired biases as % difference = [[continuous

– FRM]/FRM]*100 . Then we take the average of all the paired biases. These are described in detail in section 4.3.2 of Appendix A. Stat istical output for t he Appendix A bias calculation is presented in the left column for all
observations and on the right for those cases where both the FRM and continuous PM 2.5 monitor are greater than 3 μg/m 3 . Appendix A calls for only using data when the both observations are greater than 3 μg/m 3 ; however, we calculate both options for users to see how low concentration data affects this statistic.

Interpreting the Comparability Assessment:
The one

- page PM 2.5 continuous monitor comparability a ssessment is intended to provide a concise descript ion and illustration of the comparability of each operating PM 2.5 continuous monitor that is collocated with an FRM . The assessment assumes that the operating FRM at the site represents a true value when compared to the PM 2.5 cont inuous monitor, even though the FRM will have its own uncertainty. Changes in the set

- up or operating procedures of the PM 2.5 continuous monitor (e.g., upgrading the firmware) during the period of the assessment

may r esult in ch anges to the outputs. If changes have occurred, the time series difference assessment at the top right hand side of the one

- page output may provide a useful tool to differentiate before and after the change.

EPA has previously published or finalized a number of supporting documents and/or regulatory statements that describe acceptable performance criteria for PM 2.5 continuous monitors. However, these documents or regulatory

statements have common origin s in the PM 2.5 data quality objectives , which resulted in goals for measurement uncertainty that are published in the monitoring regulations. Since PM 2.5 continuous methods inherently pro vide a daily sample, there is a statistical advantage over filter

- based methods operating at less frequent sampling (e.g., 1

- in

- 3 day or 1

- in

- 6 day sampling)

. This statistical advantage is captured in the acceptable multiplicative and additive biases presented in the middle left of the page where data may be within the slope and intercept
boundaries , but not have a mean ratio of 1.00 +/

- 0.10 as tabulated on the bottom left or be within 10% bias for
the Appendix A statistics tabulated on the bottom right of the page. Therefore

, while meeting the goal for

measureme n t uncertainty bias provides confide nce that the data are suitable for their intended purpose, not meeting this goal does not necessarily result in the rejection of the data or method; rather, it provides an

1 Data Quality Objectives (DQOs) for Relating Federal Reference Method (FRM) and Continuous PM 2.5 Measurements to Report an Air Quality Index (AQI), EPA 454/B

- 02

- 002 , November 2002.February 2 2 , 2012

3 opportunity to investigate the performance of the method and consider the measurem e n t uncertainty in the contex t of a decision error (performed as part of a DQO exercise) when data from a method are near the level of the NAAQS

. The goals for measurement uncertainty are stated in Section 2.3.1.1 of Appendix A to Part 58 as:

Measurement Uncertainty for Automated and Manual PM 2.5 Methods. The goal for acceptable measurement uncertainty is defined as 10 percent coefficient of variation (CV) for total precision and plus or minus 10 percent for total bias.

Total precision is not readily ava i la ble from the PM 2.5 continuous monitor comparability a ssessment since this
statistic r e qu i res collocated data from the same make and model. However, bias is illustrated and calculated a few different ways (i.e., the illustration of additive and multiplicative bias, the ratio of the datasets, the A ppendix A bias calculation for all data and for only those cases with data above 3 μg/m 3 ). As de scribed in the summary, the most appropriate way to interpret the comparability of the PM 2.5 continuous monitors is to look at either the entire data set, designated as “AllData” or “A”, or view the last complete year of data. Monitoring agencies can also use this tool to ide ntify outliers or to investigate seasonal patterns. More details on interpreting the comparability assessment are expected as users across monitoring agencies share their experiences and we collectively learn how to optimize the operation of each PM 2.5 continuous method

.

Parameter Codes: At the top of the PM 2.5 continuous monitor comparability assessment there are notes on the FRM and continuous

PM 2.5 methods and parameter codes used in the assessment. The tool will provide an o utput for any case when a

PM 2.5 continuous method is reported for the first four parameter codes below (i.e., 88101, 88500, 88501, and 88502) and the site has a collocated PM 2.5 FRM. EPA has details on these codes available on its web site at: http://www.epa.gov/ttn/amtic/datamang.html For convenience these parameter codes are explained here:

Table 2

– Parameter codes available for use with PM 2.5 continuous monitors.

Parameter Name Parameter
Code Purpose Notes

PM2.5 LOCAL C ONDITIONS 88101 Appropriate code for all FRM/FEM/ARMs All PM 2.5 continuous FEMs are to use this parameter code. Data are eligible for use in NAAQS calculations, unless labeled as
“SPM” and “non

- regulatory” for first 24 months of operation.

PM2.5 TOTAL ATMOSPHERIC 88500 Valid data from methods measuring total PM 2.5 aerosols in the atmosphere, including those that can
be volatilized from the FRM

. This is the full characterization from any FDMS (i.e., base

– volatile channel) . Data are not used in NAAQS calculations.

PM2.5 RAW DATA 88501 Valid uncorrected data that does not reasonably
match the FRM. This code is expected when a PM 2.5 continuous method has any post processing
of its data and there is a desire to have a historical record of the original data. Data are not used in NAAQS calculations.

ACCEPTABLE P M2.5 AQI &
SPECIATION MASS 88502 Valid data that does reasonably match the FRM with or without correction, but not to be used in NAAQS
decisions This parameter code is expected in most cases for pre

- FEM versions of PM 2.5 continuous monitors. Data in this parameter code are used in AQI reports. Data are not used in NAAQS calculations.

PM2.5 VOLATILE CHANNEL 88503 Stores important related data such as the FDMS
reference channel This parameter code is not used in the one

page comparability report described above. Data are not used in NAAQS calculations.

Additional Notes: The data source is the EPA AQS Data Mart, which is updated each week night from AQS. A 24 hour average for the PM 2.5 continuous method is produced for each day w ith at least 18 valid hours. Data are only pre sented in cases with at least 23 valid sample pairs

. The processing of an assessment may take less than a minute to several minutes. The assessment will provide an output of the last three years of data.February 2 2 , 2012

4 Data used in the mean for both the FRM and continuous monitor have been rounded to one decimal place, while the ratios are carried to two decimal places.

The mean PM 2.5 concentrations will likely not represent exactly the overall mean of the site. On one hand, we only use days where both a valid FRM and continuous monitor data point are available; on the other hand, we use all the available data, even where identified as an exceptional event. However, it’s still likely that these mean values will be very close to a mean annual average for a given site.