Milwaukee PM2.5 Clean Data Determination

Technical Support Document

Michael Rizzo

USEPA Region 5

Air and Radiation Division

December 2011Technical Support Document

Table of Contents

Background

................................ ................................ ................................ ................................ ................... 3
Review of Historical Monitoring Data

................................ ................................ ................................ ........... 5
2010 Data Imputation for 55

- 079

- 0043 and 55

- 079

- 0099

................................ ................................ ........... 7

Conclusions

................................ ................................ ................................ ................................ ................. 13

References

................................ ................................ ................................ ................................ .................. 13

### Background

As part of the 2010 annual network review, the Wisconsin Department of Natural Resources requested to discontinue two sites (AQS ID: 550790043 and AQS ID: 550790099) within the Milwaukee, WI metropolitan area to save resources. The State cited two other violating
monitors (AQS ID: 550790026 and AQS ID: 550790010) within the area which could be used for attainment designation purposes (Figure 1) . At the time of the request four monitors within Milwaukee were violating the 24

- hour PM2.5 National Ambient Air Quality Standards (NAAQS) including the two sites the State requested to shut down. During the review, Region 5 mistakenly agreed that the S tate could shut down the two sites even though 40 CFR 58.14 does not allow any violating site to be discontinued. In 2010, the remaining two violating sites in Milwaukee had data showing that they attained the standard for the 2008

- 2010 time period.
Wisc onsin DNR requests a clean data finding based on those two sites so that the area can be redesignated from nonattainment for the 24

- hour PM2.5 NAAQS to attainment. However, because the two sites which were allowed to be terminated at the end of 2009 were also violating, the clean data determination cannot be granted without first determining whether or not those two sites could have met the NAAQS if they had been allowed to continue operating.Figure 1 . Violating PM2.5 Sites wit hin the Milwaukee WI CSA. Sites with blue markers are

the sites which were discontinued.

Past issues with incomplete PM2.5 data have been settled using statistical techniques to impute missing data, calculate the three year design value, and determine if the monitor in question meets the NAAQS within some margin of error. Usually, these tests are only allowed after all due diligence has been exhausted to assure that all measures have been taken to confirm that the data were collected and monitoring netwo rks were maintained according to the requirements prescribed in 40 CFR. That did not happen in this case. The Region believes, however, that the use of historical data along with the statistical techniques to impute the missing data to determine whether or not the two sites would attain the 24

- hour PM2.5 NAAQS can provide enough evidence to make a definitive decision on the redesignation of the Milwaukee area. Region 5 also will require Wi sconsin DNR to re

- establish the Milwaukee Fire Department Headqua rters site (550790099) beginning January 1, 2012 so that further measurements can be collected for an amount of time to be negotiated between the State and
EPA.

### Review of Historical Monitoring Data

The Milwaukee area had a total of four monitors which wer e violating the 24

- hour PM2.5 NAAQS at the time the two fire station sites were discontinued. Figure 2 shows the 98 th percentiles for all four sites from their inception in 1999 to the present

. For many years, the two discontinued sites had 98 th percentile values (blue lines in Figure 2 ) which were greater than those seen at either 550790010 or 550790026. One important observation that needs to be noted is that starting around 2007 and continuing through 2009, the 98 th percentiles from the two d iscontinued sites tracked the other two violating sites well in that all four sites had
98 th percentiles that rose and fell with each other. If this trend continued into 2010, it is a good possibility that the two discontinued sites would have had 98 th pe rcentile values that would have been less than those seen in 2009. The three year design values at the four sites (Figure 3 ) show that the two discontinued sites had values that were greater than the level of the NAAQS almost every year they were in opera tion with none of the four sites consistently having a higher set of design va lues over the remaining three. B ecause three consecutive years worth of data area required to determine attainment , 2010 98 th percentile critical values show what each sites con centrations could have been to still attain the 24

- hour NAAQS. Those concentrations were 37.1 ug/m3 and 36.9 ug/m3, respectively, for site 550790043 and site
550790099 (gray lines in Figure 2 ) . Table I shows the 98 th percentiles for the four sites from
2 007 through 2009. For all three years, the 98 th percentile concentrations were within 2

- 3 ug/m3 of each other. If 2010 were consistent with 2007 through 2009, the 98 th percentiles for the two missing sites would most likely be below the maximum critical values necessary for the
2008

- 2010 design values to be in attainment for the two sites.

Table I: Annual 98 th Percentile Concentrations (ug/m3) for Violating Sites within Milwaukee WI

Year Virginia

Street Fire Station (550790043) Milwaukee Fire

Department HQ (550790099) Wisconsin

DNR SE HQ (550790026) 16th Street

Health Clinic (550790010)

2007 38 40.7 39.8 40.6
2008 27.4 29 27.5 27.3

2009 41.7 40.3 39 39.1
2010 31.9 30.9Without some more definitive estimate of the 2010 data, however, a determination of each

monitor’s attainment status still cannot be made. Thus, a set of statistical regression techniques have been used to provide a larger weight of evidence of the two si tes’ attainment

status.

Figure 2 . Annual 98th Percentile Concentrations for Violating Sites within Milwaukee WIFigure 3 . 24

- hour PM2.5 Design Values for the Violating Sites within Milwaukee WI



### 2010 Data Imputation for 55

- 079

- 0043 and 55

- 079

- 0099 An analysis has been completed to determine the potential of two discontinued monitors within the Milwaukee, WI CSA of violating the 24 hour PM2.5 National Ambient Air Quality Standards (NAAQS) for the t hree year period covering 2008

- 2010. The monitors were shut down at the end of 2009 and have two complete years of Federal Reference Method (FRM) data from 2008

- 2009. Without 2010 data, the two violating sites c an not be compared to the NAAQS to determ ine if they are, in fact, meeting the 24

- hour standard. Comparing the two discontinued sites to nearby sites allows for the 2008

- 2010 design values estimation which indicates whether or not the two sites could have met the standard if they had remained

op erational for 2010. Three slightly different techniques were utilized to predict the 2010 98 th percentiles for the two sites and then compute the 24

- hour NAAQS. All three techniques uselinear regression to relate the concentrations from the discontinued sites to a base site

containing 2010 data. The residuals from the regression provide an indication as to the overall error in the calculated design values.

T he Office of Air Quality Pl anning and Standards (OAQPS) developed the first technique to originally aid in predict ing annual PM2.5 design value s in areas which did not meet specific data completeness requirements. A linear relationship between the monitor missing data and one that can be used as a comparison site is created using ordinary least squares (OLS). This type of relationship can be expressed in the form:

(1)

with

( 2 )

The comparison monitor is chosen from among sites within the general vicinity of the site with missing data and is usually the highest correl at ed site based on historical data

. Equation 1 predicts the 2010 concentrations at the two sites by taking 2010 data from the comparison site and multiplying those concentrations by the slope and adding the intercept from the linear regression. The linea r regression yields a set of residuals (the error term in Equation 2 ) which is the difference between what the concentration the model predicts and what the site actually measured. Through a process called bootstrapping, t he residuals are randomly sampled 1000
times with replacement and then added to the prediction. The residual addition accounts for the random error from the regression. In the case of Milwaukee, the comparison site had a total of 177 observations for 2010. A total of 1000 sets of 177 o bservations were therefore produced for each missing site. A 2010 98 th percentile and
, subsequently
, a 2008

- 2010 design value for each set were calculated yielding 1000 individual design values

. OAQPS accepts a monitor as meeting the standard when at least approximately 90% of the boot strap ped design values meet the standard which would mean that around 900 of the 1000 bootstraps would have to meet the standard. Depending on how the residuals are randomly chosen, the number of bootstrapped design values greater than the standard could vary by analysis. Nonparametric statistical tests are employed to demonstrate that the majority of the boo tstrap samples meet the NAAQS (Conover, 1999)

.

The boots trapped residuals can also be used to determine the variability in the slope and intercept of the relationship between the discontinued and comparison sites. The resampled residuals are added to the historical predictions and then the regression is recalc ulated resulting in a slightly different slope and intercept. The residuals from this regression are then sampled as described above to give a set of 98 th percentiles and design values. The wholeprocess is repeated 1000 times and the final results are p rocessed in the same manner as

described above. Region 5 used this technique for the two discontinued Milwaukee sites and showed that it is comparable to the OAQPS technique.

The two previously described analyses assume that there is only error in the m easurements made at the two discontinued sites and that the concentrations at the comparison monitor are essentially error free. Because both the discontinued and comparison monitors are from the same monitoring network and the measured PM2.5 concentratio ns are subject to the same operating and quality assurance procedures
, an error free assumption for the comparison monitor is unrealistic. To account for the error in both the concentrations from the discontinued and comparison sites, Region 5 also impute d the missing data from the two discontinued sites using Deming regression which is also known as orthogonal regression. This type of regression is similar to the OLS regression displayed in Equation s 1 and 2 with the exception that the PM2.5 comparison si te term also has an error term associated with it . The Deming regression model can be expressed in the form:

(3)

with

( 4)

(5)

Equations 3 through 5 are solved iteratively to get estimates for the slope and intercept terms.
T he residuals (the error terms in Equations 4 and 5) are resampled or bootstrapped similarly to what was done to the error term for the Region 5 OLS analysis. The bootstrapping is repeated 1000 times like the processes described above and the results are similarly analyzed.

When building the relationships between each discontinued site and the potential comparison
si te s, censoring criteria determine which potential sites could be utilized as comparison monitors. To be eligible to be a comparison site, there ha s to be at least 100 paired samples available over the course of 20 calendar quarter s (5 years of data) and n o set of calendar quarters over the 5 year period c an have less than 20 paired samples. The correlation coefficient (R2) was calculated for each candidate missing/comparison site pair and the comparison site with the highest correlation to the missing sit e was chosen to be used for the final analysis. In both cases for Milwaukee , the comparison site which had the highest correlation to each of the discontinued sites was the Wisconsin DNR Southeast Headquarters
site (AQS ID: 550790026). The results of the correlation analysis along with the regression results for each site are shown in Figures 4 and 5

.Figure 4

. PM2.5 Relationship between the Virginia Street Fire Station Site (550790043) and

the Wisconsin DNR Southeast Headquarters Site (550790026)Figure 5 . PM2.5 Relationship between the Milwaukee Fire Department Headquarters Site

(55 0790099) and the Wisconsin DNR Southeast Headquarters Site (550790026)Table I I: Results from the data imputation techniques for the two discontinued PM2.5 sites in

Milwaukee, WI

Monitor Comparison

Monitor R 2 Method Number of bootstraps below NAAQS Number of

bootstraps above NAAQS % of

bootstraps greater than NAAQS 3 Year

Design Value What is the chance that

we could incorrectly conclude that the monitor violates the 24

- hour PM2.5 NAAQS ? What is the chance that we

could incorrectly conclude th at the monitor does not violate the 24

- hour PM2.5 NAAQS?

550790043 550790026 0.91 Region 5 995 5 1% 34 100% 0% (Virginia Street Fire Station) (Wisconsin

DNR SE HQ) OAQPS 999 1 0% 34 100% 0%

Deming 1000 0 0% 34 100% 0%

550790099 550790026 0.86 Region 5 898 102 10% 34 44% 60% (Milwaukee Fire Department HQ) (Wisconsin

DNR SE HQ) OAQPS 903 97 10% 34 64% 39%

Deming 920 80 8% 34 98% 2%

Table II shows the monitor ID for the discontinued site and the comparison monitor with the highest correlation to the discontinued site. The correlation coefficient (R2) is provided in the third column with the fourth column giving the method used for the analys is. The three year design value is the estimate directly predicted from the regression equations (Equations 1 and 3) . The last two columns display the results of the nonparametric quantile tests and are displayed in a way that reflects the probability of making an erroneous conclusion which would cause the null hypothesis to be rejected and the acceptance of the alternative. A monitor that attains the standard from the bootstrap results would have a very high probability for Test #1 (>=95%) and a very lo w probability for Test #2 (<=5%) . In cases where the probabilities from the two tests do not meet these requirements, the monitor is either leaning more toward attainment or nonattainment. Th is “gray” zone signifies that other evidence may be necessary

t o aid in deciding whether or not the monitor is attaining the standard.

The OAQPS results listed in Table II represent an independent running of the OAQPS regression technique by Region 5. OAQPS’ results showed both discontinued sites having three year design values of 34 ug/m3. The Virginia Street Fire Station site (550790043) had 2 bootstrap simulations that produced a failing design value (Test #1 probability: 100% and Test #2 probability: 0%) while the Milwaukee Fire Department Headquarters site (55 0790099) had 101 simulations that produced a failing design value (Test #1 probability: 48% and Test #2 probability: 56%).For both sites, the regression results predict attainment with all techniques returning three

year design values of 34 ug/m3. The re lationship between a discontinued site’s concentrations and its comparison site is high for both sites. The number of bootstraps that are below the NAAQS for the Virginia Street fire Station indicates attainment of the 24

- hour standard

. The Milwaukee Fir e Department Headquarters site , however, falls more into the “gray” zone for both the Region 5 and OAQPS methods making it difficult to determine whether the site is attaining the NAAQS. The Deming regression results, however, do indicate attainment and m ay be considered more reliable because the error in the comparison site’s measurements is also accounted for

.



### Conclusions

 The trends in the annual 98 th percentiles show that the two discontinued sites were tracking the other two violating sites for 2007

- 2009 and that the 98 th percentile concentration differences were around 2

- 3 ug/m3 between the four sites. If this trend continued for 2010, the two disc ontinued sites would probably have had 98 th percentile concentrations less than the calculated 2010 critical values of approximately 37 ug/m3
and the t wo sites would have attain ed the standard.

 If we use OAQPS’ criteri on to determine whether or not a monit or passes using the regression methods, the Virginia Street Fire Station (550790043) attains the standard. The Milwaukee Fire Department HQ site’s status lies more in a “gray” zone, but the Deming regression which accounts for the error in both the discon tinued and comparison sites’ data shows that the site attains the 24

- hour NAAQS.

 With th e weight of evidence from both the trends in the 98 th percentile concentrations and the results from the regression analyses , AMAS would support a clean data determinat ion for the Milwaukee area.

 However, the Milwaukee Fire Department Headquarters site must be re

- established as a means to confirm attainment and monitor ambient PM2.5 concentrations within Milwaukee for a period consistent with the requirements of 40 CFR P art 58. The site should be operational by January 1, 2012 and operate on a one in three day schedule.



### References Conover, W. J.; Practical Nonparametric Statistics ; John Wiley and Sons; New York; 1999; third edition; pp. 135

- 143.