UNITED STATES ENVIRONMENTAL PROTECTION AGENCY

REGION 4

Technical Support Document 

For 

Ambient Monitoring

EPA’s Notice of Proposed Rulemaking For:

Air Plan Approval and Air Quality Designation; GA; Redesignation of the Atlanta, GA, 

1997 Annual Fine Particulate Matter Nonattainment Area to Attainment

June 2015
2

A. BACKGROUND

On July 18, 1997 (62 FR 36852), EPA established an annual PM2.5 NAAQS at 15.0 micrograms per 

cubic meter (μg/m3) based on a 3-year average of annual mean PM2.5 concentrations. At that time, EPA 

also established a 24-hour standard of 65 μg/m3(today’s action does not address the 24-hour standard). 

See 40 CFR 50.7. On January 5, 2005 (70 FR 944), EPA published its air quality designations and 

classifications for the 1997 PM2.5 NAAQS based upon air quality monitoring data for calendar years 

2001–2003. These designations became effective on April 5, 2005. The Atlanta Area (comprised of

Barrow, Bartow, Carroll, Cherokee, Clayton, Cobb, Coweta, DeKalb, Douglas, Fayette, Forsyth, Fulton, 

Gwinnett, Hall, Heard (partial), Henry, Newton, Paulding, Putnam (partial), Rockdale, Spalding and 

Walton Counties) was designated nonattainment for the 1997 PM2.5 NAAQS. See 40 CFR 81.301.

On October 17, 2006 (71 FR 61144), EPA retained the annual PM2.5 NAAQS at 15.0 μg/m3based on a 

3-year average of annual mean PM2.5 concentrations. On November 13, 2009, EPA clarified the 

designations promulgated in 2005 for the 1997 PM2.5 NAAQS, stating that the Atlanta Area was 

designated as nonattainment for the annual standard and unclassifiable/attainment for the 24-hour

standard. 

B. EPA REQUIREMENTS

An area is deemed to have attaining data for the 1997 Annual PM2.5 NAAQS when the design valuea

meets the requirements stipulated in 40 CFR Part 50, section 50.13 as follows:

The annual primary and secondary PM2.5 standards are met when the annual arithmetic mean 

concentration, as determined in accordance with appendix N of this part, is less than or equal to 

15.0 μg/m3.

Upon determination by EPA that an area designated nonattainment for the PM2.5 NAAQS has attained

the standard, 40 CFR section 51.1004(c) provides that the requirement for such area to submit 

attainment demonstrations and associated reasonably available control measures, reasonable further 

progress plans, contingency measures, and other planning SIPs related to attainment of the PM2.5

NAAQS shall be suspended until the area is redesignated to attainment, at which time the requirement 

no longer applies. However, if EPA subsequently determines that the area has violated the PM2.5

NAAQS, the requirement to submit such plans is reinstated.

Determinations of attainment are based on three years of complete, quality-assured data. Appendix N 

does not provide for examining partial years of data because various seasons of the year reflect various 

influences on PM2.5 concentrations, and a partial year’s data may not be representative of values that 

would be determined from a full year’s data set. Nevertheless, any assessment should consider 

additional quality-assured data to the extent that it exists. 

 

a Design values are the metrics that are compared to the NAAQS levels to determine attainment. The annual design value is 

calculated as the average of three consecutive annual means. (See 40 CFR part 50, Appendix N.)
3

C. EPA’S ANALYSIS OF MONITORED DATA

EPA’s determination that the Atlanta Area has current attaining data for the 1997 Annual PM2.5 NAAQS

is based upon 2012-2014 data. This data has been quality assured, quality controlled and certified as 

required by 40 CFR Part 58. The monitors and data used to determine the 2012-2014 Annual PM2.5

design value for the Atlanta Area are included in Table 1 below. All design values presented in this 

analysis are annual PM2.5 design values and are calculated using the procedures provided in 40 CFR Part 

50, Appendix N. Unless otherwise noted, all data presented in this analysis was retrieved from EPA’s 

Air Quality System Database and have been certified as accurate by the State of Georgia, as required by 

40 CFR §58.15.

Table 1: 2012-2014 Annual Average PM2.5 Concentrations for Monitors in the Atlanta, Georgia,

Nonattainment Area

Location Site Number

2012

Annual 

Average

(μg/m3)

2013

Annual 

Average

(μg/m3)

2014

Annual 

Average

(μg/m3)

Annual 

Design 

Value 

(μg/m3)

Maximum 

Bootstrapped 

Annual Design 

Value (μg/m3)

Georgia DOT 13-063-0091 10.8 9.7 10.6 10.4 -

GA Nat’l Guard 13-067-0003 10.1 9.6 10.3 10.0 -

South DeKalb 13-089-0002 10.2 9.3 10.1 9.9 -

Fire Station #8 13-121-0039 11.3 10.4 11.2 11.0* 11.1

Gwinnett Tech 13-135-0002 10.3 8.9 9.3 9.5 -

Gainesville 13-139-0003 9.3 8.5 8.9 8.9 -

Yorkville 13-223-0003 8.8 8.5 8.7 8.7 -

* Fire Station #8 did not produce a valid design value for 2012-2014 due to incomplete data.

All of the sites in the approved monitoring network recorded valid, attaining design values during 2012-

2014 except for the Fire Station #8 site. This site had incomplete data during the 3rd quarter of 2012 and 

is not eligible for the high value substitution test in 40 CFR Part 50 Appendix N because the data 

completeness in this quarter is below 50 percent. The site has incomplete data for this period because the 

PM2.5 sampler failed a monthly one-point flow rate verification (required by 40 CFR Part 58,

Appendix A, Section 3.2.3) during this period, and several samples were invalidated as a result. An 

analysis of the data completeness for all sites in the area from 2010-2014 is shown in Figure 1. EPA 

approved three sites (Macland Aquatic Center, 13-067-0004; Doraville, 13-089-2001; and E. Rivers 

School, 13-121-0032) to shut down at the end of 2012, and they are not included in this analysis. These 

sites are not considered incomplete because they are no longer part of the approved PM2.5 monitoring 

network for the area. The GA National Guard site collected incomplete data in 3rd quarter of 2014, but 

this site has a valid, attaining design value after passing the high value substitution test in 40 CFR Part 

50, Appendix N.
4

Figure 1: Atlanta Data Completeness by Site, 2010-2014

Fire Station #8 Site: Analysis of Incomplete Data

Since the Fire Station #8 site has historically recorded the highest annual design value in the Atlanta 

area, EPA conducted additional statistical analysis which indicates, as a weight of evidence, that the site 

would likely have attained the 1997 annual PM2.5 NAAQS during 2012-2014 had it collected complete 

data. This analysis estimates the missing samples at the incomplete site using a linear regression model 

and uses bootstrapping to evaluate the model and approximate the probable 2012-2014 design value at 

Fire Station #8. A similar approach has been applied to areas with incomplete data in the past.

1,2

 This 

approach was determined to be appropriate for this area for the reasons discussed below, but it may or 

may not be suitable for other areas with incomplete data. EPA will evaluate the appropriateness of this 

analytical approach on a case-by-case basis for determinations regarding each area with incomplete data.

Step 1 – Organize data and perform linear regression between incomplete site and candidate sites

The first step in this analysis was to develop a linear regression model for the incomplete site (Fire 

Station #8) using historical data from the last five years collected at other sites in the area. Linear 

regressions were developed using any sites with at least 20 sample pairs (historical samples collected at 

the predictor site and the incomplete site on the same day) in each calendar quarter (1st, 2nd, 3rd, and 4th

quarter) over the five-year period, and at least 100 total sample pairs. In this analysis, a Theil-Sen linear 

regression model3,4 was used. The Theil-Sen regression is a nonparametric linear model and is typically 

less likely to be influenced by outliers than a least squares linear regression.
5

Step 2 – Select the best candidate site as the predictor site

For each model, the Spearman correlation coefficient was evaluated to determine the correlation of the 

incomplete site and the predictor site. To evaluate the prediction error of the model, a k-folds crossvalidation5 was performed. This method randomly removes subsets of the original dataset and predicts 

these known values using the same model method (in this case the Theil-Sen regression) to determine 

the model error on the dataset. In this analysis, 10 folds were used and a standard error estimate, or root 

mean squared error (RMSE), was calculated from the k-folds results. 

The models using different predictor sites were ranked using the Spearman correlation and the RMSE to 

determine the best model to use for predicting the incomplete sample values. Summary statistics about 

the models using each of the potential predictor sites are shown in Table 2, ranked by the Spearman

correlation. The selected predictor site must also have collected enough samples during the calendar 

quarter(s) that are incomplete to generate at least 75 percent data completeness in each quarter in the 

most recent three-year period after modeling the missing samples. Based on the above criteria, the 

correlation and cross-validation results indicated that the South DeKalb site (Spearman correlation ρ = 

0.91, RMSE = 2.09 μg/m3) is the best predictor for concentrations at Fire Station #8. Overall, these 

results indicate a strong linear relationship between the South DeKalb and Fire Station #8 sites. A plot 

of the Theil-Sen regression model and the paired data from both sites is shown in Figure 2.

Table 2: Linear Regression Model Comparison for Candidate Predictor Sites

Incomplete 

Site

Predictor 

Site

n 

Sample 

Pairs

n pairs

Q1

n pairs 

Q2

n pairs 

Q3

n pairs 

Q4

Minimum 

Pairs Met

13-121-0039 13-089-0002 519 124 135 121 139 Yes

13-121-0039 13-063-0091 497 121 122 118 136 Yes

13-121-0039 13-135-0002 474 110 128 112 124 Yes

13-121-0039 13-139-0003 462 113 114 113 122 Yes

13-121-0039 13-223-0003 477 111 125 112 129 Yes

Incomplete 

Site

Predictor 

Site

Theil 

Slope

Theil 

Intercept

Spearman 

ρ

Theil 

RMSE 

(μg/m3)

13-121-0039 13-089-0002 1.030 0.676 0.910 2.095

13-121-0039 13-063-0091 1.000 0.449 0.894 2.519

13-121-0039 13-135-0002 1.007 1.243 0.885 2.712

13-121-0039 13-139-0003 0.965 2.600 0.759 3.744

13-121-0039 13-223-0003 0.984 2.218 0.737 3.555
6

Figure 2: Correlation plot and Theil-Sen regression between the South DeKalb (13-089-0002) and 

Fire Station #8 (13-121-0039) sites, using PM2.5 data (μg/m3) from 2010-2014

The missing samples at Fire Station #8 in any quarter with data completeness below 75 percent were 

estimated using the Theil-Sen regression model. Missing samples were not estimated in quarters with 

data completeness of 75 percent or greater. 

Step 3 – Calculate a test design value

Next, a test 2012-2014 design value was calculated using the combination of the collected samples and 

the estimated samples. This procedure produced a test design value of 10.9 μg/m3, below the 1997 

PM2.5 NAAQS of 15 μg/m3.

Step 4 - Bootstrapping Analysis

In order to estimate the variance of the predicted design value, a bootstrapping analysis was conducted. 

Bootstrapping is a statistical procedure that uses resampling of a dataset to estimate the variance of a 

statistic.

6

In this analysis, the following bootstrapping procedure was used:

a. Residuals were calculated from the Theil-Sen regression model

b. For each missing sample that was estimated in Step 2, a regression residual was randomly 

selected with replacement from the pool of residuals and added (or subtracted, in the case of a 

negative residual) to the estimated value. 

c. Repeat (b) 10,000 times.

d. For each of the 10,000 runs, the predicted 3-year design value was calculated as in Step 3.

The resulting 10,000 predicted design values are interpreted as an approximate probability distribution 

of the design value at the incomplete site, had it collected complete data. The distribution of the 

resulting bootstrapped design values is shown in Figure 3. The results of the 10,000 bootstrapped design 
7

values are summarized in Table 3. Since none of the 10,000 bootstrapping runs resulted in a design 

value above the NAAQS, the Fire Station #8 site is considered to have met the NAAQS.

Figure 3: Distribution of design value bootstrap results for Fire Station #8

Table 3: Summary of Fire Station #8 design value bootstrap results

Incomplete 

Site Predictor Site Bootstrap Runs

Predicted

DV

(μg/m3)

95% Confidence 

Interval (μg/m3)

Max Bootstrapped 

DV (μg/m3)

13-121-0039 13-089-0002 10,000 10.9 [10.9 – 11.0] 11.0

Additional Analysis of Seasonal Model Performance

The bootstrapping method described above assumes that the underlying dataset is independent and 

identically distributed (i.i.d.). This assumption can be inappropriate for time series data, and several 

alternative bootstrap methods (e.g., a block bootstrap) have been developed for these types of data.

7

PM2.5 concentrations often exhibit a strong seasonal pattern. In this analysis, the general bootstrap 

method could overpredict or underpredict concentrations in certain seasons if the relationship between 

the incomplete site and the predictor site changes seasonally. To investigate this possibility, the 

concentration difference between Fire Station #8 and South DeKalb was plotted over time (Figure 4). 
8

The Theil-Sen model RMSE and median model residual value (to identify a positive or negative 

prediction bias) were also calculated for each calendar quarter. The quarterly RMSE is shown in Figure 

5, and the quarterly median residual is shown in Figure 6. These analyses do not show a seasonal 

pattern in the relationship between the two sites or in the predictive accuracy of the linear model. 

Therefore, the general bootstrap method was considered appropriate for this dataset, and a time series 

bootstrap was not conducted.

Figure 4: PM2.5 concentration difference time series for each sample pair collected at Fire Station 

#8 and South DeKalb, 2010-2014

Figure 5: Theil-Sen regression model standard error by year and calendar quarter
9

Figure 6: Theil-Sen regression model median residual value by year and calendar quarter

Air Quality Trend for the Nonattainment Area

In the years since the Atlanta area was designated nonattainment in 2005 for the 1997 Annual PM2.5

NAAQS, all of the monitors in the area have shown improvements in annual average ambient PM2.5

concentrations. The annual PM2.5 design values for all sites in the area from 2001-2003 (the period used 

for the original nonattainment designation) to 2012-2014 (the period used in this action) are shown in 

Figure 7.

Figure 7: Atlanta Annual PM2.5 Design Values, 2003 – 2014

Invalid Design 

Value due to 

Incomplete Data
10

D. CONCLUSIONS AND RECOMMENDED AGENCY ACTION

The monitors in the Atlanta Area that collected complete data for the period 2012 - 2014 have average 

PM2.5 concentrations that are all in attainment with the 1997 Annual PM2.5 NAAQS. The Fire Station 

#8 site had incomplete data during the 3rd quarter of 2012 and is not eligible for the high value 

substitution test in 40 CFR Part 50, Appendix N. However, based upon the analysis described in this 

Technical Support Document, EPA has determined that the upper end of the range of probable 2012 -

2014 design values at the Fire Station #8 site did not exceed the NAAQS. In addition, the GA National 

Guard site collected incomplete data in 3rd quarter of 2014 but has a valid, attaining design value after 

passing the high value substitution test. On the basis of this review, EPA has concluded that the Atlanta 

Area continues to meet the 1997 Annual PM2.5 NAAQS of 15.0 μg/m3for the period 2012 - 2014, the 

most recent 3-year period of certified data availability. 
11

E. REFERENCES

1. Approval and Promulgation of Implementation Plans and Designations of Areas for Air Quality 

Planning Purposes; Georgia: Atlanta; Determination of Attaining Data for the 1997 Annual Fine 

Particulate Matter Standards, 76 Fed. Reg. 56701, September 14, 2011.

2. New York Citation http://www.regulations.gov/#!documentDetail;D=EPA-R02-OAR-2009-0659-

0003

3. R Core Team (2014). R: A language and environment for statistical computing. R Foundation for 

Statistical Computing, Vienna, Austria. URL http://www.R-project.org/. 

4. Lukasz Komsta (2013). mblm: Median-Based Linear Models. R package version 0.12. 

http://CRAN.R-project.org/package=mblm

5. John H. Maindonald and W. John Braun (2014). DAAG: Data Analysis And Graphics data and 

functions. R package version 1.20. http://CRAN.R-project.org/package=DAAG

6. Efron, B. Bootstrap Methods: Another Look at the Jackknife. Ann. Statist. 7 (1979), no. 1, 1--

26. doi:10.1214/aos/1176344552. http://projecteuclid.org/euclid.aos/1176344552. 

7. Jens-Peter Kreiss, Soumendra Nath Lahiri, 1 - Bootstrap Methods for Time Series, In: Tata Subba 

Rao, Suhasini Subba Rao and C.R. Rao, Editor(s), Handbook of Statistics, Elsevier, 2012, Volume 

30, Pages 3-26, ISSN 0169-7161, ISBN 9780444538581, http://dx.doi.org/10.1016/B978-0-444-

53858-1.00001-6. 

8. Angelo Canty and Brian Ripley (2015). boot: Bootstrap R (S-Plus) Functions. R package version 

1.3-14.

9. Davison, A. C. & Hinkley, D. V. (1997) Bootstrap Methods and Their Applications. Cambridge 

University Press, Cambridge. ISBN 0-521-57391-2

10. H. Wickham. ggplot2: elegant graphics for data analysis. Springer New York, 2009.