## Testing of the Simplified Light Extinction Estimates at the SEARCH Site in Birmingham Alabama

Ivar Tombach
2 November 2010



### Introduction

C omments I provided to the EPA on 30 August 2010, concerning EPA’s second draft Policy Assessment Document (PA D) for the PM 2.5 NAAQS, 1 included results of a comparison of fine particle light extinction (b ext,fp ) estimates using both the “simplified
method ” in the PAD and values derived from hourly measurements of particulate matter chemical composition that were m ade by the SEARCH program.

The Southeastern Aerosol Research and Characterization Study (SEARCH) network has been analyzing 24

- hr filter samples and operating semi

- continuous analyzers at multiple
locations in the Southeast. The semi

- continuous analyzers provide measurements of the
concentrations of PM 2.5 mass and of sulfates, nitrates, ammonium, total carbon, and
elemental carbon with one

- hour resolution. Such data allow application of the IMPROVE algorithm on an hourly basis 2 without recourse to the var ious scaling steps required to
estimate light extinction applying EPA’s simplified Speciated PM 2.5 Mass

- calculated Light Extinction (SPMLE) method to daily speciated PM concentration measurements

, following the method described in the draft PAD.

As EPA st rives to simplify its method for estimating hourly light extinction in urban areas, one needs to be concerned whether those simplifications produce erroneous estimates of light extinction or smooth out relevant actual diurnal or day

- to

- day variations

1 Comments Concerning the Vis ibility

- Related Portion of the Second External Review Draft of the Policy Assessment for the Review of the PM NAAQS , by Ivar Tombach, 30 August 2010.

2 Except that concentrations of the elements that make up fine soil are not measured hourly. Rather, con centrations from 24

- hr filter samples serve as estimates for every hour of the day.2 in th at extinction. A nalyses carried out with SEARCH data , as described in this memo, explore some of these questions.

To evaluate the simplified method , the hourly PM 2.5 light extinction results obtained by using EPA’s SPMLE calculation method were compared with hourly light extinction calculated from hourly speciated concentration measurements, using EPA’s variant of the original IMPROVE algorithm in both cases. For this analysis, I originally used both 24

- hr
and 1

- hr measurements made at the Jefferson Stree t (JST), Atlanta, SEARCH site, during

2005

- 2007 (the same years as those analyzed by EPA in the PAD).

The analysis contained in my August comments compared the two methods for a metric based on hourly values of b ext,fp over all daylight hours with RH ≤ 9 0%. That analysis, as extracted from my August comments, is Attachment 1 to this memo. That attachment also provides more information about the data sources.

Subsequently, i n response to a request from EPA staff, I carried out a second, similar analysis that addressed an alternative metric of daily maximum hourly values of b ext,fp , for
the same SEARCH JST data set under the same constraints. The results of that second analysis were submitted to the EPA on 27 September 2010. 3 That document is
Attachment 2 to this memo.

These analyses showed reasonable agreement between estimates of hourly light extinction by EPA’s simplified method and those derived by applying the same modified IMPROVE algorithm using hourly PM 2.5 component measurements in Atlanta by the SEARCH program. There wasn’t sufficient time during the formal comment period to carry out analysis for other locations, both urban and rural, but EPA staff expressed an interest in seeing such results if they became available . In response, this memo prov ides results of analyses using data from the SEARCH site at Birmingham, Alabama (BHM)

. The analyses herein are identical to the two sets of analyses carried out for JST and
described in the two attachments.



### Summary of Approach

In the PAD, the EPA elected to use a modified version of the traditional IMPROVE algorithm to estimate light extinction from the concentrations of various chemical components in particulate matter. In my original comments, I expressed reservations
about the choice of the traditional algorithm and about the modifications EPA made to it. However, for the sake of consistency

, all of my analyses of SEARCH data use EPA’s modified IMPROVE algorithm

.

To review, the 10 steps of EPA’s new approach , as summarized on pages 4

- 34 and 4

- 35
of the PAD

, are reproduced here

. M y comments in italics describe adjustments I made to
be able to apply the SEARCH data.

3 Comparison of D aylight Maximum Hour Calculation s , by Ivar Tombach, 27 September 2010

.3 • Step i. Estimate 24

- hour organic carbonaceous mass from the CSN organic carbon measurement, via a multiplier, rathe r than by the SANDWICH me thod. For CSN,
substitute SEARCH. Although I remain skeptical about EPA’s choice of a multiplier of 1.7 , I have used it here for my calculations with the SEARCH data.

• Step ii. Calculate 24

- hour fine soil/crustal from the CSN measurements of the
related PM 2.5 crustal elements. In lieu of CSN 24

- hr speciated concentration measurements, I used the similar 24

- hr sulfate, nitrate, OC, EC, and fine soil constituent measurements made by the SEARCH Particle Composition Monitor (PCM)

. As I’ve noted previously , the fine soil fo rmula in the PAD differs slightly from the IMPROVE soil formula. My calculations use the IMPROVE formu la for soil, but this choice should have negligible impact on the results of my
evaluations

.

• Step iii. Calculate 24

- hour neutralized, dry sul fate and nitrate in the same manner as in the IMPROVE network and Regional Haze program, by multiplication by factors of 1.375 and 1.29 respectively. This step implies that the CSN sulfates and nitrates are in the forms of ammonium sulfate and ammonium nit rate , which is
consistent with the assumed forms in the IMPROVE algorithm but will overstate the sulfate concentration at times when the sulfate is not fully neutralized, as is often the case in the Southeast in summer. For my analyses I used sulfate and

n itrate concentrations determined from sampling by the SEARCH PCM and

, consistent with the EPA, also assumed that the sulf ate is fully neutralized.

• Step iv. Sum the five components, for each day. The result is a version of the PM 2.5

mass co ncentration, san s fine sea salt, i.e., [PM 2.5 ] = [AmSulfate] + [AmNitrate] + [O C M] + [EC] + [Soil]

.

• Step v. Calculate the five component fractions (i.e., percentages) of the sum

- of

- five, for each day.

• Step vi. Looking across all CSN sampling days in a given calendar mo nth, average the daily component fractions for sulfate, for nitrate, etc. These monthly

- average component fractions will be applied on every day in that month

. For CSN,
substitute SEARCH PCM.

• Step vii. For each day, determine a scaling factor to reconcile [ the 24

- hr average of

] hourly P M 2.5 mass measurements from a continuous instrument with the 24

- hour P M 2.5 mass determined by FRM sampling. The continuous PM 2.5 instrument
in SEARCH is a Torsion

- Element Optical Microbalance (TEOM), operated at a temperatur e of 30° C. The FRM PM 2.5 concentrations were determined using a Rupprecht & Patashnick Model 2025 FRM sampler

.

• Step viii. For each day, apply this scaling factor to the 1

- hour measurements of

P M 2.5 mass from the continuous instrument.4 • Step ix. For each day and daylight hour in the month, multiply the scaled value of hourly P M 2.5 mass by the component fractions from Step v i

.

• Step x. These hourly speciated PM 2.5 concentrations are then combined with collocated hourly averaged relative humidity as input t o the IMPROVE algorithm to calculate the speciated PM 2.5 mass

- calculated light extinction. As with many of EPA’s calculations, t his analysis excluded hours with values of RH > 90%

.

Note that my calculations are not a validation of the ability of the simpl ified method, or of any of the other methods considered by the EPA, to accurately portray hour

- by

- hour actual light extinction. Rather, they should be considered as initial exploration of whether the simplifications in EPA’s method bias or alter the struct ure of the extinction estimates.



### Results for BHM

The component mass fractions for BHM that result from Step (v) of the SPMLE approach
are summarized in Table 1. A lack of valid 24

- hr nitrate measurements from mid May to the end of September 2005 means th at averages could not be calculated for June through September, and the average for May 2005 is not based on a complete month of data

.

Table 1 . Monthly average s of filter sample 24

- hr fractional contributions of chemical componen ts in the PM 2.5 mass concentration at BHM

.

AmSulf
Fraction Am Nitr
Fraction OCM
Fraction EC
Fraction Soil
Fraction

Jan 0.281 0.123 0.420 0.097 0.081

Feb 0.256 0.144 0.415 0.112 0.073

Mar 0.326 0.101 0.384 0.112 0.077

Apr 0.312 0.055 0.427 0.115 0.091

May 0.379 0.055 0.404 0.099 0.064

June 0.400 0.029 * 0.393 0.100 0.078

July 0.435 0.034 * 0.360 0.059 0.113

Aug 0.467 0.026 * 0.326 0.076 0.105
Sept 0.429 0.025 * 0.358 0.095 0.093

Oct 0.338 0.058 0.393 0.107 0.104

Nov 0.276 0.075 0.450 0.115 0.085

Dec 0.252 0.110 0.444 0.125 0.069

3

- yr Avg. 0.336 0.073 0.402 0.103 0.085

2005 Avg 0.302 0.092 0.422 0.112 0.072

2006 Avg 0.347 0.067 0.395 0.102 0.090
2007 Avg 0.349 0.067 0.395 0.099 0.090

* 2

- year (2006 & 2007) averages5 Although the day

- to

- day values of the component mass fractions are highly variable, note in Table 1 that, on average, ammonium sulfate and OCM combined account for about 80% of the fine mass. With the 1.7 multiplier, OCM accounts on average for slightly
more of the mass than does ammonium sulfate, which is also the case for JST. Annually, ammonium nitrate accounts for 7 to 9%, EC for 10 to 11% (vs. 8 to 9% at JST), and the fine soil for 7 to 9% of the mass (vs. 4 to 5% at JST). These fractions vary with month,
sometimes with large changes, although the year

- to

- year fractions are relatively stable. The monthly averages in Table 1 hide the day

- to

- day and hour

- to

- hour variation, of

course.

The SEARCH hourly speciated measurements provide concentration results that are analogous to the products of Step (ix) in EPA’s method , except that the values are not constrained by monthly

- average factors required by the EPA procedure, but rather reflect
the actual hour

- by

- hour and day

- by

- day variation of the composition of the fine PM. The

o nly exception is fine soil, for which hourly measurements were not made and therefore the daily average concentration of soil was used for every hour of the day. At JST soil is typically a minor fraction of the PM 2.5 , as note d in the attachments; its contr ibution at BHM is a bit larger, but using the 24

- hr average will generally not affect the quality of the hourly estimates. Brief episodic dust events would be an exception.

In part icular , some areas of Birmingham have occasional episodes of high concentrations
of

industrial “dust” that contains substantial amounts of metals, such as zinc. Such metals

are not accounted for in the soil term of IMPROVE formula , and hence are not reflected in Table 1, but yet have been observed for short periods of time with concentra tions of
such metals that are many tens of micrograms per cubic meter

.

The SEARCH hourly speciated concentrations were then inserted into the modified traditional IMPROVE algorithm, as described in Step (x). Two metrics were evaluated
for BHM

– (1) light extinction during all daylight hours and (2) the maximum hourly light extinction on each day. The results for these analyses are described below in a format patterned after the presentation of the results of the JST analyses. The analogous
results for JST are described in Attachments 1 and 2, respectively.6 

### Results

– All Daylight Hours

Figure 1 shows every valid hourly value of b ext,fp (PM 2.5 light extinction) at BHM during

2005

- 07

, as produced by the two methods

– the SPMLE of the PAD in the top panel and as calculated directly from hourly component measurements in the bottom panel. Only hours designated as daylight hours are shown and conditions with RH>90% have been excluded from the analysis.

Figure 1 . Time series plots of all hourly daylight val ues of b ext,fp at BHM generated by the PAD method (top panel) and the direct calculation method (lower panel). RH
≤ 90%.7 Figure 1 illustrates several points:

• Both methods can be plagued by data gaps. In the top panel of Figure 1, t he

4

- month absence of 24

- hr nitrate measurements on filters derails the calculation of monthly fine PM 2.5 mass concentration in step ( iv) of SPMLE method that was summarized above, and hence prevents calculation of component fractions in Step (v) , spe ciated concentrations in Step ( ix) and light extinction in Step (x)

. Hence, i f the filter samplers for any component are out of commission for a month, then the
PAD method cannot be used. There are also small breaks in the data because filter samples were only taken on every 3 rd day in 2005 at BHM; daily sampling resumed in 2006. Generally, however, the monthly averages of the component
frac tions are quite robust, since only a valid hourly PM 2.5 concentration and the RH are needed for each hourly point in the PAD method (top panel). In contrast, the direct calculation method (the bottom panel) has more short gaps in the data, a consequence of needing five valid hourly concentration measurements and a value
of RH for each hour.

• Both plots show similar patterns in the main body of the data, but “spikes” in one plot are not always present in the other. A notable discrepancy of this type is

asso ciated with the very large spikes, up to above 800 Mm - 1 , in the latter part of May 2007 in the PAD plot; there is no corresponding pattern in the direct calculation plot because missing concentration data precluded the calculation of light extinction then.

The same data sets are compar ed in a scatter plot in Figure 2 . The only points plotted are
those for hours for which both methods produced a value. Consequently, the high PAD
b ext,fp values produced by the spike in May 2007 do not appear because there we re no corresponding hourly values from the direct calculation. The correlation between the two
data sets is respectable (R 2 = 0 .80) and just slightly weaker tha n the corresponding comparison at JST. The black line is the ordinary least squares (OLS) regres sion line and the red line is the OLS line forced through the origin. The red line has a slope of 1.00, which suggests that one data set is not biased relative to the other ; JST also showed a slope of 1.00. The dashed green lines denote a factor of 2 aroun d the 1:1 line (which just happens to be the OLS line through the origin). The dense data “cloud” spills outside the two green lines when b ext,fp is less than about 150 Mm - 1 , which indicates that the hourly
b ext,fp values produced by the two methods can r eadily differ by more than a factor of two. This scatter is greate r than what was observed at JST. Relative agreement improves at higher v alues of extinction.8 Figure 2 . Comparison of hourly daylight values of PAD b ext,fp with di rectly

calculated va lues for BHM during 2005

- 2007, for RH ≤ 90%. A few points lie outside the plot area but were used in the calculations

. Ordinary least squares regression lines and formulas are shown in black and red

. The dashed green lines
delineate a factor of two around the 1:1 line.

We can gain insight into the significance of any differences between the results of the two methods by looking at their distributions. Table 2 summarizes some statistics about
the two data sets for each of the three year s

. In particular, the 98 th percentile of all the valid hourly values of b ext,fp is a metric that appears in the draft PAD for consideration.9 Table 2 . Statistical comparison of the two b ext,fp data sets for all daylight hours for

BHM

.

2005 2006 2007

PAD Direct PAD Direct PAD Direct

# of daylight hours 4107 4107 4107 4107 4107 4107

# of daylight hours (RH ≤ 90%) w/ valid
data 1101 794 2942 2103 3101 2181

Maximum b ext,fp (Mm - 1 ) 673 533 780 738 823 626

98 th %

- ile b ext,fp (Mm - 1 ) 362 387 264 261 331 239

Note the following in Table 2:

• Although the number of possible daylight hours (as defined by the UFVA) per year is 4107, that number was reduced appreciably by the RH ≤ 90% cutoff and by the need for valid values for all needed calculations for t hat hour. The once

every

- third

- day sampling also took its toll in 2005. T he PAD method produced more hourly estimates of b ext,fp than the direct method.

• For 2006, b oth methods calculated very similar highest extinction values

. A lthough they did not occur on the same day , t hey did occur two days apart, during the same episode in December. In the other two years, t he maximum
estimated values of extinction differed to a greater degree and occurred several months apart, which demonstrates that the SPMLE method can miss the mark by a wide margin at times

.

• The 98 th percentile values of b ext,fp are expected to be less sensitive to the extremes reflected by the maximum values, and indeed this is the case. The b ext,fp values calculated by the two methods agreed cl osely in 2006 , although those values occurred on different days.

Table 2 shows that b oth methods give design values (i.e., 3

- year average 98 th

- percentile
values of b ext,fp with RH ≤ 90%) that exceed the upper end of the SPMLE range of 64 to
191 Mm - 1 recomm ended by EPA staff on page 4

- 50 of the draft PAD. The annual 98 th
percentile values that make up these desig n values are in the range of 239 to 387 Mm - 1 (depending on method)

.10 

### Results

– Daily Maximum Daylight Hours

The above analysi s compared the two methods for a metric that was based on hourly
values of b ext,fp over al l daylight hours with RH ≤ 90%. A new analysis that addresses, similarly, the alternative metric of daily maximum hourly values of b ext,fp , for the same
data set under the same constraints

, follows below

.

The scatter plot in Figure 3 dis plays the relationship for 2005

– 2007 between values of daily maximum hourly b ext,fp derived using the method in the PAD a nd the values calculated directly from the chemical measurements using the modified traditi onal IMPROVE algorithm. Figure 3 is the counterpart of Figure 2 in the previous section.

In Figure 3, as in Figure 2 , the black line is the ordinary least squares ( OLS) linear regression line and the red line is the OLS line forced through the origin. Since there is only one point per day, th ere are fewer points on Figure 3 than in its all

- hours
counterpart. The coefficient of determination (R 2 ) between the data sets is rather poor

– R 2 = 0.39 for the line through the origin. (It was a more respectable 0.72 for JST.) Unlike the wedge

- shaped data “cloud” in the earlier figure, here the bounds on the data scatter appear to be ill defined. T he scatter does not appear to be in the form of a systematic pattern of offsets from the central line

. As a reference for the eye, two green lines are
drawn at a distance of ± 50 Mm - 1 horizontally or vertically from a 1:1 line (essen tially the red line). Disagreements between the two methods are still relatively common outside that range, though, and a range of more than ± 100 Mm - 1 is needed to contain 90 to 95%
of the data.

Table 3 summarizes some statistics about the two data sets that were plotted in Figure 3, for each of the three years 2005 through 2007. The table displays both the 98 th and 90 th
percentiles of all the daily maximum hourly values of b ext,fp during each year, metrics that
were considered in the second draft PAD. Al though there is some variability in these percentiles from year to year, the 3

- year averages of various percentiles of light extinction are consistently greater for the PAD SPMLE method than for the direct method, a systematic bias that was not apparent in the JST analysis. Note though, the 90 th percentiles in 2006 are identical, although t hey occurred three months apart!

Table 3 shows that both methods considered here give design values (i.e., 3

- year average 98 th

- percentile values of daily maximum hourly b ext,fp whenever RH ≤ 90%) that greatly exceed the upper end of the SPMLE range of 64 to 191 Mm - 1 recommended by EPA
staff. The annual 98 th percentile values that make up these design values are in the range
of 300 to 540 Mm - 1 (depending on method and year), and ar e considerably greater than the roughly 250 to 350 Mm - 1 values for all dayligh t hours that appeared in Table 2

.11 Figure 3 . Comparison of daily maximum hourly daylight values of PAD b ext,fp with directly calculated values for BHM during 2005

- 2007, for RH ≤ 90%. A few points lie outside the plot area, but are included in the regression calculations. Ordinary least squares regression lines and formulas are shown in black and red. The dashed green lines delineate deviations of ± 50 Mm - 1 from the 1:1 line.

Table 3 . Statistical comparison of the two daily maximum hour b ext,fp data sets for

BHM.

2005 2006 2007

PAD Direct PAD Direct PAD Direct

# of days considered, (RH<90% and valid data) 322 342 335

# of valid max hour
extinction values 238 183 290 25 1 308 242

Maximum b ext,fp (Mm - 1 ) 673 487 676 738 823 626

98 th %

- ile of daily max
b ext,fp values (Mm - 1 ) 475 314 484 540 5 50 399

90 th %

- ile of daily max
b ext,fp values (Mm - 1 ) 229 188 272 272 305 24912 A comparison of Tables 2 and 3 shows that the obser vation made in EPA’s documents,
that the extinction at the 98 th percentile of all hourly value s is approximately the same as that at the 90th percentile of maximum daily values, is reasonably correc t for 2006, a
little less so for 2007, but totally misses the mark for 2005. L ower data completeness in

2005 , relative to 2006 and 2007, could be a contributing factor, though.



### Conclusions

In summary, these new analyses at BHM show greater differences between the two
methods than were observed at JST. Birmingha m is a more industrial city than Atlanta, so it has greater haze and that haze is more likely to be less uniform spatially and temporally, and hence to vary more with meteorological conditions

. There is often reasonable agreement between extinction estimat es made by the two methods considered
here , but substantial and consistent differences were also observed. Setting aside the 2005 comparison because of less complete data than in 2006 and 2007 , note that PAD 98 th
percentile extinction values in 2007 for bo th metrics were greater by 30

- 35% (roughly
100 to 140 Mm - 1 ) than those of the direct analyses

. As one goes farther from the highest values, down to the 90 th percentile for the maximum daily value metric, agreement between the methods improves, both in abso lute and relative senses, but the maximum daily hour metric is generally more inconsistent, even at 90 th percentile, than the all hours
metric

One aspect that is more troublesome for BHM than JST is limited temporal agreement
between the estimates of hazi ness by the two methods , especially when using the daily
max extinction metric

. As an example, according to the SPMLE method, the 8th highest (roughly 98 th percentile) daily max extinction at BHM in 2007 was 531 Mm - 1 on March

13 th , while the direct calcul ation method arrived at a very different 8th highest extinction
of 372 Mm - 1 on August 15 th , 5 months later. On March 13 th both methods estimated relatively similar extinction, but on August 15 th

, the SPMLE calculated an extinction of
190 Mm - 1 , which doesn’t compare well with the 372 Mm - 1 estimate of the direct method.

Therefore, the conclusion of the JST analyses has to be tempered somewhat for BHM to reflect the consistent bias of one method relative to the other

, the non

- robust nature of the daily max hour metric (esp ecially at 98 th percentile), and the greater lack of temporal agreement as to when during the year the greatest h a ziness occurs. Although individual hourly estimates of b ext,fp may differ considerably, the agreement between the two methods improves when mo re stable percentile levels are chosen, i.e, ones farther down
from the 99 th percentile, and the all hours metric is used instead of the daily maximum hour. It is likely that a multi

- year index , such as the average of 90 th percentiles over 3 years, will al so be more stable and consistent , but that hypothesis was not tested here.

One approach that does not improve matters is that of using a range of percentiles

– e.g., the average of all values between the 92 nd and the 90 th percentiles. The biases between the methods tend to be consistent over a range of extinction values, so nothing is gained by adding this complexity.13 If some of these concepts are followed, the EPA’s simplified method of estimating hourly light extinction may still provide usable estima tes of b ext,fp , as compared to estimates
made with actual on

- site hourly PM 2.5 component concentration data.

An important caveat : These conclusions are based on analyses of data from two Southern cities. The results may not be applicable to other cities, especially in other areas of the
country, to other city sizes, a nd to other pollutant mixes. Also, all of the comparisons
here have been between t w o calculation methods for estimating light extinction , neither of which has been tested against actual visibi lity measurement . It is important that research be carried out to link these extinction estimates to actual optical measurements of haze, so that the haze management program will aim at goals that are meaningful for society

.

## ATTACHMENT 1

Comparison of All Daylight Hour Calculations at JST

The following text and figure are extracted , with some minor editing, from pages 21 to 20
of Comments Concerning the Visibility

- Related Portion of the Second External Review Draft of the Policy Assessment for the Review of the PM NAAQS, by Ivar Tombach, 30 August 2010 :

I have compared the hourly PM 2.5 light extinction results obtained by using EPA’s simplified SPMLE [ Speciated PM 2.5 Mass

- calculated Light Extinction ] calculation
method wi th the results of the hourly light extinction calculated from hourly speciated concentration measurements, using the same variant of the IMPROVE algorithm in both cases. For this analysis, I used the every

- day measurements made at the Jefferson Street (JST ), Atlanta, SEARCH site, during 2005

- 2007 (the same years as those analyzed by EPA in the PAD [Policy Assessment Document]).

Note that my calculations are not a validation of the ability of the simplified method, or of any of the other methods considered by the EPA, to accurately portray hour

- by

- hour actual light extinction. Rather, they should be considered as initial exploration of whether the simplifications in EPA’s latest method alter the fine structure of the extinction estimates to such a degree th at they cannot be considered usable.

The same data sets are compared in a scatter plot in Figure 5. The only points plotted are those for hours for which both methods produced a value. Consequently, the high PAD
b ext,fp values produced by the spike in May 2007 do not appear because there were no corresponding hourly values from the direct calculation [because of a carbon analyzer malfunction] . The correlation between the two data sets is respectable (R 2 = 0.85). The
blac k line is the ordinary least squares (OLS) regression line and the red line is the OLS line forced through the origin. The red line has a slope of 1.00, which suggests that one data set is not biased relative to the other. The dashed green lines denote a f actor of 2 around the 1:1 line (which just happens to be the OLS line through the origin). The dense data “cloud” just barely fits between the two green lines when b ext,fp is less than about

150 Mm - 1 , which indicates that the hourly b ext,fp values produce d by the two methods can readily differ by up to a factor of two. Relative agreement improves at higher values of
extinction. Even above 300 Mm - 1 there are occasional values that differ by a factor of two from their counterparts, though.2 Figure 5. Comparison of hourly daylight values of PAD b ext,fp with directly calculated values for JST during 2005

- 2007, for RH ≤ 90%. Ordinary least squares regression lines and formulas are shown in black and red. The dashed green lines
delineate a factor of two ar ound the 1:1 line.

We can gain insight into the significance of any differences between the results of the two methods by looking at their distributions. Table 2 summarizes some statistics about the two data sets for each of the three years. In particular , the 98 th percentile of all the valid hourly values of b ext,fp is a metric that appears in the draft PAD for consideration. y = 0.93x + 9.33

R 2 = 0.85 y = 1.00x

R 2 = 0.84

0 100 200 300 400 500 600

0 100 200 300 400 500 600

bextfp (Mm-1)PAD bextfp (Mm-1)3 Table 1 . Statistical comparison of the two b ext,fp data sets for JST.

2005 2006 2007

PAD Direct PAD Direct PAD Direct

# of daylight hours 4107 4107 4107 4107 4107 4107

# of daylight hours (RH ≤ 90%) w/ valid
data 2614 2106 3023 2589 2952 2564

Maximum b ext,fp (Mm - 1 ) 494 533 444 453 910 452

98 th %

- ile b ext,fp (Mm - 1 ) 255 255 229 235 236 207

Note the following in Table 2:

• Although the number of possible daylight hours (as defined by the UFVA) per year is 4107, that number was reduced appreciably by the RH ≤ 90% cutoff and by the need for valid values for all needed calculations for that hour. As we discussed earlier, the PA D method produced more hourly estimates of b ext,fp than
the direct method.

• For 2006, both methods calculated very similar highest extinction values and determined them to occur at the very same hour of the year. In the other two years, the maximum estimated values of extinction differed to a greater degree and did not occur on the same day. The very large discrepancy in 2007 is due to missing total carbon data during a high PM episode in May 2007, which caused the direct method to miss the episode, as was discussed above in connection with Figure 4.

• The 98 th percentile values of b ext,fp are expected to be less sensitive to the extremes reflected by the maximum values, and indeed this is the case. The b ext,fp values calculated by the two methods agre ed exactly in 2005, although those identical values occurred on different days. Agreement was also very good in 2006 but less so in 2007, again a consequence of the missing total carbon data during the high PM episode in May 2007.

Table 2 shows that both methods give design values (i.e., 3

- year average 98 th

- percentile
values of b ext,fp with RH ≤ 90%) that exceed the upper end of the SPMLE range of 64 to

191 Mm - 1 recommended by EPA staff on page 4

- 50 of the draft PAD. The annual 98 th percentile values that ma ke up these design values are in the range of 207 to 255 Mm - 1 (depending on method)

.4 Although this analysis suggests that, at least at the SEARCH JST site, the EPA’s simplified method of estimating hourly light extinction does not 1 appear to materially
d egrade the quality of the estimates relative to those made with actual on

- site hourly

PM 2.5 component concentration data, this exercise does not demonstrate any of the following:

• Equivalency between the simplified method (Method F) and the method the EPA has used previously (Method A) ;

• The ability of EPA’s variant of the original IMPROVE algorithm to produce reliable estimates of hourly light extinction from PM 2.5 component concentrations in urban areas;

• Equivalency between the simplified method (or, for that matter, of any of the methods considered by the EPA) and actual hourly light extinction. T hat very

essential evaluation requires comparison with on

- site hourly measurements of light extinction.

1 [This “not” was inadvertently left out of my August comments, although this conclusion was stated correctly elsewhere in that document.]

## ATTACHMENT 2

Comparison of Daylight Maximum Hour Calculations

Ivar Tombach 27 September 2010

The comments I su b mitted to the EPA on 30 August 2010, concerning EPA’s second draft Policy Assessment Document (PAD) for the PM 2.5 NAAQS, 2 included results of a comparison of fine partic le light extinction (b ext,fp ) estimates using both the “simplified method” in the PAD and values derived from hourly measurements of the principal chemical components in the original IMPROVE algorithm, using daily aerosol
measurements made at the Jefferson Street (JST), Atlanta, SEARCH monitoring site during 2005

- 2007.

The analysis in that report compared the two methods for a metric that was based on hourly values of b ext,fp over all daylight hours with RH ≤ 90%. (That analysis, as extracted from my Augus t comments, is attached to this memo.) This memo provides a new analysis that addresses, similarly, the alternative metric of daily maximum hourly values
of b ext,fp , for the same data set under the same constraints. Information about the data
sources is co ntained in the Attachment.

The scatter plot in Figure 1 displays the relationship for 2005

– 2007 between values of daily maximum hourly of b ext,fp derived using the method in the PAD and the values calculated directly from the chemical measurements usin g the traditional IMPROVE algorithm. Figure 1 is the counterpart of Figure 5 in my earlier report, which is included
in the Attachment.

In Figure 1, the black line is the ordinary least squares (OLS) linear regression line and
the red line is the OLS lin e forced through the origin. Since there is only one point per day, there are fewer points on Figure 1 than in its all

- hours counterpart. Unlike the wedge

- shaped data “cloud” in the earlier figure, here the bounds on the data scatter appear to be parallel to the central line, i.e., the scatter appears to be in the form of offsets from the central line that don’t depend on the magnitude of the light extinction, rather than the multiplicative form in the figure in my earlier report. As a reference for the eye

, the two green lines are at a distance of ± 50 Mm - 1 horizontally or vertically from a 1:1 line (essentially the red line). Disagreements between the two methods are still relatively common outside that range, though, and a range of ± 100 Mm - 1 is needed to contain 90 to
95% of the data.

2 Comments Concerning the Vis ibility

- Related Portion of the Second External Review Draft of the Policy Assessment for the Review of the PM NAAQS , by Ivar Tombach, 30 August 2010.2 Figure 1 . Comparison of daily maximum hourly daylight values of PAD b ext,fp with directly calculated values for BHM during 2005

- 2007, for RH ≤ 90%. Ordinary least squares regression lines and formulas are shown in black and red. The dashed green
lines delineate deviations of ± 50 Mm - 1 from the 1:1 line.

Table 1 summarizes some s tatistics about the two data sets that were displayed in Figure 1, for each of the three years 2005 through 2007. (This new table is analogous to Table 2 in my earlier report, attached.) Table 1 displays both the 98 th and 90 th percentiles of all the daily maximum hourly values of b ext,fp during each year, which are metrics that were considered in the second draft PAD. Although there is some variability in these percentiles from year to year, the 3

- year averages for the two methods are within about
5% of eac h other, with neither method showing obvious bias relative to the other. y = 0.70x + 54.99

R 2 = 0.49 y = 0.95x

R 2 = 0.39

0 100 200 300 400 500 600

0 100 200 300 400 500 600

Daily Max Hour bextfp (Mm-1)PAD Daily Max Hr bextfp (Mm-1)3 Table 2 . Statistical comparison of the two daily maximum hour b ext,fp data sets for

BHM.

2005 2006 2007

PAD Direct PAD Direct PAD Direct

# of days considered, (RH<90% and valid data) 322 342 335

# of valid max hour
extinction values 238 183 290 251 308 242

Maximum b ext,fp (Mm - 1 ) 673 487 676 738 823 626

98 th %

- ile of daily max
b ext,fp values (Mm - 1 ) 475 314 484 540 541 399

90 th %

- ile of daily max
b ext,fp values (Mm - 1 ) 229 188 272 272 305 267

Table 1 shows that both methods considered here give design values (i.e., 3

- year average 98 th

- percentile values of daily maximum hourly b ext,fp whenever RH ≤ 90%) that greatly exceed the upper end of the SPMLE range of 64 to 191 Mm - 1 recommended by EPA
staff. The annual 98 th percentile values that make up these design values are in the range
of 300 to 400 Mm - 1 (depending on method and year), which are also considerably greater than the roughly 200 to 250 Mm - 1 values for all daylight hours that appeared in Table 2 in my earlier report.

The PAD included the observation that the 98 th percentile of all hours is about the same
as the 90 th percentile of the daily maximum hours. When one compares Table 1 above with Table 2 in the Attachment, that relationship also seems to apply to the SEARCH JST data analyzed here.

In summary, these new analyses with daily maximum hour extinction lead to conclusions similar to those of the earlier analyses with extinction during all daylight hours, with RH

≤ 90% in both cases. Specifically, at least at the SEARCH JST site, although i ndividual hourly estimates of b ext,fp may differ considerably, when data are aggregated over time the EPA’s simplified method of estimating hourly light extinction appears not to materially degrade the quality of estimates of various percentile levels b ext ,fp , as compared to
estimates made with actual on

- site hourly PM 2.5 component concentration data.

It is worth repeating that this exercise does not demonstrate any of the following,
however:

• Equivalency between the simplified method used here (Method F) and the method the EPA has used previously (Method A);4 • The ability of EPA’s variant of the original IMPROVE algorithm to produce reliable estimates of hourly light extinction from PM 2.5 component concentrations in urban areas;

Equivalency between the sim plified method (or, for that matter, of any of the methods considered by the EPA) and actual hourly light extinction. That very

- essential evaluation requires comparison with on

- site hourly measurements of light extinction.