1 UNITED STATES ENVIRONMENTAL PROTECTION AGENCY

RESEARCH TRIANGLE PARK, NC 27711



### MEMORANDUM

SUBJECT : Assessments of One- Point QC Data in Response to Comments on Revisions to the Ambient Air Quality Assurance Regulation contained in 40 CFR part 58 Appendix A

FROM: Lewis Weinstock and Mike Papp (OAQPS/AQAD/AAMG)

TO: Ambient Monitoring Rule Docket (EPA-HQ-OAR-2013-0619)

This memorandum discusses potential changes to quality assurance requirements in 40 CFR Part 58. As part of a proposed rule - Revisions to Ambient Monitoring Quality Assurance and Other Requirements – 79 FR 54356 - OAQPS received a number of comments related to the proposal to lower the concentrations of the one-point QC check and to requi re that the selection of the check concentration be

based on the mean or median concentration of th e measurements within the ambient air monitoring

network. The comments that were received were used to revise the proposed language for the final rule to provide more flexibility for the monitoring organizati ons. However, EPA also performed some additional assessments of monitoring data that was used to pr ovide some rationale for our initial decision to propose the changes in the regulations and we believe provi de a framework for future changes to operational procedures in the SLAMS network. Th ese additional details are provided below.

BACKGROUND

The EPA proposed to lower the audit concentrations (current section 3.2.1) of the one-point quality control (QC) checks to between 0.005 and 0.08 parts per million (ppm) for SO 2 , NO 2 , and O 3 (currently 0.01 to 0.1 ppm), and to between 0.5 and 5 ppm for CO monitors (currently 1 and 10 ppm). With the development of more sensitive monitoring instru ments with lower detection limits, technical improvements in calibrators, and lower ambient air con centrations in general, the EPA felt this revision would better reflect the precision and bias of the ambien t air data being measured at a majority of the sites

across the network.

The majority of the comments (19 of 26 respondi ng to the quality assurance proposal) received on appendix A related to this proposed change. Most of the commenters expressed similar technical concerns which can be categorized below:

 The SLAMS network is in place mainly for decisi ons related to the NAAQS therefore QC checks should be around NAAQS values.

 Some of the federal reference methods (FRM) or federal equivalent methods (FEM) that are still in use may operate acceptably at concentrati ons around the NAAQS but these older methods are not as sensitive at lower concentrations (i.e., mean or median concentrations) so QC checks at these lower levels are beyond the limits of the instrumentation.2  The instrumentation necessary to challenge the monito rs at the lower concentrations (calibrators with additional mass flow controllers or gas cylin ders of lower concentrations) would be required to comply and therefore represen t an added expense and burden.

 The lower concentrations affect the percent different statistic so there is more chance that the QC check will fail the acceptance requirements and th erefore invalidate data that the monitoring organization feels is of acceptable quality.

DATA ANALYSIS

In order to provide some context to the proposal, EP A extracted routine data and all one-point QC data for the 4 gaseous criteria pollutants for SLAMS site s for calendar year 2013. The following evaluation provides summary information about the routine data a nd the one-point QC checks reported by states. It must be noted that the one-point QC regula tion, prior to the proposal, stated that “ the QC check gas

concentration selected should be related to the routin e concentrations normally measured at sites within the monitoring network in order to appropriately reflect the precision and bias at these routine concentration ranges” 1

. Based on the 2013 data presented below, it is evident that many monitoring organizations did not follow this recommendation and was the reason EPA formally proposed the revision to this requirement.

Nitrogen Dioxide - In Figure 1 (and similar figures for the other gaseous pollutants), the graph on the left represents the mean, 99th percentile and max value for 1-hour routine ambient air data for 43 states and territories (x-axis) reporting NO 2 data in 2013. The y-axis is conc entration in ppb. The green line (80 ppb) represents the proposed upper range of the one- point QC check while the red line represents the current upper range (100 ppb) for the one-point QC check. Of the 43 states reporting hourly NO 2 data, only three states had maximum hourly values (highest value for the year) above the NAAQS (100 ppb) and 99 percent of all the states had hourly values that were below 60 ppb. The graph on the right is a frequency distribution of the one-point QC checks for 2013. 85 percent of the one-point QC checks reported for 2013 were above 99 percent of the routine data.

1 40 CFR Part 58 Appendix A section 3.2.13 Figure 2 below (and similar figures for the other gaseous pollutants), provides an assessment of the one-point QC checks using box and whisker plots (example on left). In the figures presented below, the mean is not identified. In Figure 2, the graph on the left provides the difference between the measured value and the audit standard value; the graph on the right is the same data set but reports the percent difference which is the statistic presently in use for the gaseous pollutants 2 . The one-point QC data is segregated into 10 ppb segments (0-10 ppb, 10-20 ppb etc.) in order to evaluate whether the lower QC concen trations have an effect on the precision and bias estimates. The green vertical line on the graph on th e right represents the proposed 80 ppb high QC range and the blue shaded area represents where the QC chec ks would be selected if monitoring organizations selected a QC concentration related to the mean or median routine air concentration. The red lines represent the current percent difference acceptance criteria (+15%) 3 . In this case, the NO 2 variability,

based on the 25 th – 75 th percentile spread, does not appear to be significantly different at lower QC concentrations than the high er QC concentration data.

2 40 CFR Part 58 Appendix A Section 4.1.1

3 Validation Templates in QA Handbook Volume II and on AMTIC http://www.epa.gov/ttnamti1/qalist.html4 Ozone (Fig. 3) - Of the 51 states and territories reporting hourly ozone data, 48 states had maximum values over the NAAQS (75 ppb) but had 99 percent of their hourly values below the NAAQS. In addition, 90 percent of the one-point QC check s were above 99 percent of the routine data.

Similar to NO 2 , Figure 4 presents the differences (left graph) and percent differences (right graph) of the one-point QC checks segregated into the ten concentra tion ranges. The ozone acceptance criteria is tighter

than NO 2 (+7 percent difference) but the variability of th e 25th – 75th percentile spread at these lower ranges does not appear to be significantly different from the one-point QC checks variability at higher concentration ranges.5 Sulfur Dioxide (Fig. 5) - In order to provide a readable graph, the left hand graph of Figure 5 does not

include maximum values since some states did measure SO 2 values that were quite high. Of the 48 states and territories providing hourly sulfur dioxide data , 23 states had maximum values over the NAAQS (75 ppb) and 47 states had 99 percent of their values below 36 ppb. 75 percent of the one-point QC checks were above 99 percent of the routine data.

Figure 6 presents the differences (left graph) and per cent differences (right graph) of the one-point QC checks segregated into the ten concentration ranges. The SO 2 acceptance criteria is +10% (percent difference). The variability, based on the 25th – 75t h percentile spread, at these lower ranges does not appear to be significantly different from the one- point QC checks at higher concentration ranges.6 Carbon Monoxide (Fig. 7) - of the 51 states and territories reporting hourly carbon monoxide data, no state was reporting maximum valu es greater than the one-hour NAAQS (35 ppm) and only three states were reporting hourly maximum values above the eight hour NAAQS (9 ppm). In addition, 99 percent of all states hourly data was below 3.0 ppm. States we re providing one-point QC values at lower ranges (graph on right) but about 60 percent of the one-point QC checks were being performed at concentrations greater than 99 percent of the routine ambient air data.

Similar to the other gaseous pollutants, the variabilit y of percent differences, based on the 25th – 75th percentile spread of the box and whisker plots, does not appear to be significantly different between the higher and the lower one-point QC concentration values.7 The 2013 data at the lower concentration levels used in this assessment may represent monitoring organizations that have invested in the technology (i.e., analyzers, calibration devices and standards at NCore sites) necessary to adequately calibrate a nd estimate precision and bias at the concentrations

measured at ambient levels more reflective of mean concentrations. However, the data also demonstrates that the technology is available to measure and re port precision and bias at mean/median ambient air

concentration levels.

As indicated in these results descr ibed above, challenging ambient ai r analyzers with a one-point QC check at the level of the NAAQS provi des an incomplete characterization of the precision and bias of the data actually reported to the AQS since, in most cases, the precision and bias estimates from the 2013 QC checks represent the uncertainty at a concentration level above 99 percen t of the SLAMS data reported to AQS. These estimates may not be reflective of the precision and bias of these instruments at concentrations routinely measured.

IMPLICATIONS

Over the years, EPA has developed guidance to reflect a different paradigm for the use of a number of QC checks in the ambient air quality system. Figure 9 re presents a view of the way various QC checks are currently implemented. This example chart represen ts data from an ozone monitor with an operating range of 500 ppb (typical operating ranges are 500 or 1000 ppb). The routine data is from a monitoring organization and represents the distri bution of typical values seen in a r outine monitoring organization. For the 2013 data reported above, the mean and median concentrations for all states reporting ozone was 30 ppb (lower than the 50 ppb mean reported in Fig.9).8 Calibrations - Multipoint calibrations were evenly sp aced over the operating range (0-500 ppb).

Span Check - a span check is normally set at 80% of the operating range (in this case 400 ppb).

One-point QC check- The current one-point QC check in appe ndix A (prior to the proposal) had a range of 10 – 100 ppb and alth ough the regulation suggested that the checks should represent routine concentrations, 90 percent of the checks in the 2013 data for ozone were between 60-100 ppb. Based on this scenario (Fig. 9) and the concen trations of routine data in the ambient air network, one can see that few quality control points reflect where the majority of the data are measured.

EPA has made strides, with the assistance of mon itoring organizations, to de velop more useful quality control information, as illustrated in Figure 10. In this case, instead of calibrating all the way to the operating range, monitoring organizations can select a calibration scale that is based on the maximum values reported at the site (or monitoring agency) or at some value just above a historical max. This allows more calibration points to be placed at con centration ranges found in routine data. The span point can still be placed at 80 percent of the calibrati on range but can also be selected to represent a concentration close to the NAAQS. This would elim inate the concern by commenters that the one-point QC should represent the NAAQS since the span point is implemented at the same frequency (minimally every two weeks) as the one-point precision check. Th e one-point QC check can then be measured in a concentration range that is more refl ective of the routine concentrations.9 EPA also realizes that the distribution of routine data does not always fall as “nicely” as the ozone data in Figure 9 indicate. Figure 11 provides a national frequency distribution of the SO 2 data. This figure represents average state values for the various per centiles across all states (with the exception of Hawaii which has some very extreme SO2 values due to natura l events). For example, the high value (104 ppb) is the average of the 47 state max values (not includ ing Hawaii). What this is meant to demonstrate is

that EPA will need to evaluate ea ch pollutant in order to determin e the best process to utilize the QC information so that it can reflects the precision and bi as of the routine ambient air being measured as well as the precision and bias around the NAAQS.10 

### SUMMARY

As has been shown, monitoring agencies can test and achieve acceptable precision and bias results at lower concentration levels. Providing data users with estimates of precision and bias where the majority

of our ambient air data are measured should be a programmatic goal and monito ring organizations should be working with EPA Regional Offices to devel op the budgets necessary for purchasing the updated equipment and revising related procedures. The EPA w ill continue to endorse this approach to make the QC checks more meaningful and will consider future re visions to Appendix A to either require QC checks at two concentration levels (i.e., one around the mean concentrations and one related to the NAAQS) or require the span check to be reported to AQS. In addition, to alleviate concerns about failing the acceptance criteria at lower QC concentrations, EPA will evaluate suggestions by monitoring organizations to raise acceptance criteria or look at alternative acceptance criteri a (e.g. difference instead of percent difference). Since acceptance criteria is in cluded in guidance, EPA will have the opportunity to perform the evaluations without effecting the regulat ion. In 2011, EPA developed similar guidance for

lower concentration levels of the annual performance evaluation audits.