1

Comments to CASAC

on Reconsideration of the Ozone National Ambient Air Quality Standard

in advance of February 18, 2011 CASAC Conference Call

Comments by

Anne E. Smith, Ph. D.

Charles River Associates

Comments sponsored by American Petroleum Institute

February 7, 2011

The U.S. Environmental Protection Agency (EPA) will convene a conference call of the CASAC Ozone

Reconsideration Panel on February 18, 2011 to obtain CASAC’s advice on a number of questions

regarding the strength and uncertainties in evidence supporting  the ozone National Ambient Air Quality

Standard (NAAQS).  The evidence to be considered is limited to that which was available and in the

record at the time of the March 2008 final rule which lowered the primary ozone standard to 75 ppb

from its effective prior level of 84 ppb (4th highest daily maximum 8‐hour average).  This written

statement summarizes points in my written comments from the 2008 ozone review process regarding

uncertainties in EPA’s quantitative health risk assessment and exposure assessment, which are relevant

to CASAC’s present Charge Questions 6 and 8.   

My key points are:

 Very little of EPA’s quantitative risk estimates come from exposures on days when 8‐hour ozone

levels exceed 70 ppb, or even 60 ppb.

 The majority of EPA’s risk estimates come from ozone exposures that are so low that they are

subject to rapidly expanding epidemiological uncertainty regarding magnitude and causality.

 The majority of the risk reduction estimated from a tightened ozone standard is also highly

uncertain because it is based on assumptions of relatively large reductions in ozone

concentrations that are already very low and likely near the limit of further reduction.

 92 to 100 percent of EPA’s risk estimate disappears if EPA’s assumed “policy relevant

background” ozone level is increased to 40 ppb, within its range of uncertainty.

My original written comments in the record (Riker and Smith, 2006; Smith and Gibbs, 2007; Smith, 2008)

provide more data and analysis supporting the points that I make here.  URLs to access copies of those

documents are provided in the References section.

EPA’s Risk Analysis Attributes Very Little Risk to Days with 8‐Hour Ozone Levels Above 60 ppb

The level at which EPA is reconsidering setting the ozone standard is between 60 and 70 ppb (8‐hour

average).  An in‐depth review of EPA’s quantitative risk analysis finds that very little of the total

estimated risk  is due to 8‐hour ozone exposures exceeding 70 ppb, or even 60 ppb.  Rather, the

majority of EPA’s quantitative risk estimates are due to days when the maximum 8‐hour average
2

exposure lies below 60 ppb.  That majority of the quantitatively estimated risk is subject to much greater

uncertainty regarding causality, because (1) the uncertainty of any epidemiologically‐derived risk

estimate expands at the lower end of the exposure range and (2) there is no supporting clinical evidence

of relevant effects in that range of 8‐hour exposures.  Furthermore, as discussed in the next section,

there is uncertainty that those lower ozone concentrations will actually decline if the standard is

tightened.

In the Proposed Rule in 2007, the Administrator stated that it was his judgment that for mortality and

hospitalizations – health effects that some epidemiological studies have associated with ambient ozone

– the causality of those associations becomes highly dubious for ozone exposures below 70 ppb (8‐hour

average).   Given this view, one would want to know what part of the total mortality and morbidity

estimates is attributable to 8‐hour average exposures above and below 70 ppb.  EPA’s Risk Analysis and

Staff Paper did not provide that information, but I was able to compute it from the same data and

assumptions that EPA used for its Risk Analysis.  The results for mortality risk are provided in Tables 1

and 2 for risk at exact attainment of the then‐current NAAQS of 84 ppb, and for exact attainment of the

alternative NAAQS of 74 ppb, respectively.  74 ppb results serve as a close approximation of the now‐

current NAAQS of 75 ppb.  These tables show the number of deaths and contribution to annual

mortality rates in EPA’s risk analysis that come from days on which the maximum 8‐hour ozone

exposure is predicted to exceed 70 ppb.  All of the rest of the estimated mortality risk in EPA’s Risk

Analysis is due to days on which the maximum 8‐hour ozone concentration is less than 70 ppb.    

One can conclude from Table 1 that even the older 84 ppb standard did not entail any meaningful

contribution to mortality rates when ozone peak concentrations exceeded 70 ppb.  In 2008, EPA revised

the ozone NAAQS to 75 ppb, which is obviously even more protective.  Table 2 shows that the newer 75

ppb standard entails even less risk from peak ozone concentrations above 70 ppb.  For the 84 ppb

NAAQS, 86% of the total risk estimated for each health endpoint is attributable to days when the

maximum 8‐hour average ozone is less than 70 ppb.  For the 74 ppb alternative, 93% of the risk is

attributable to such days.  

For this Reconsideration of the Ozone NAAQS decision, it would be useful if I could also report the

percentage of total risk that is attributable to days when the maximum 8‐hour average ozone is less

than 60 ppb.  However, given the focus of the Administrator on the 70 ppb cut‐point, I did not make that

calculation.  Based on other data that I entered into the record, I am able to determine that at exact

attainment of the 84 ppb standard, 72% of the total risk estimate is attributable to days when peak

ozone was less than 60 ppb.  I cannot however, provide the comparable estimate under exact

attainment of the current NAAQS of 75 ppb from data in the record.   

   
3

Table 1.  Total Annual Non‐Accidental Deaths by Risk Analysis City and Number of Those Deaths

Attributed to Days When the Maximum 8‐Hour Average Ozone Level Exceeds 70 ppb When Just

Attaining the 84 ppb Standard (for Average of 2002 and 2004 Data; 95% Confidence Intervals in

Parentheses)  

 

Total Non-Accidental 

Deaths per Year 

Numbers of Deaths 

Attributable to Ozone > 

70 ppb (8-hr avg) 

Percent of Mortality 

Incidence 

Atlanta 9,233 2 

(-8 , 11) 

0.02% 

(-0.08, 0.12%) 

Cleveland 14,749 10 

(-6 , 25) 

0.07% 

(-0.04, 0.17%) 

Detroit 18,817 7 

(-2 , 16) 

0.04% 

(-0.01, 0.08%) 

Houston 18,122 4 

(0 , 7) 

0.02% 

(0.00, 0.04%) 

Los Angeles 54,157 0 

(0 , 0) 

0.00% 

(0.00, 0.0%) 

Sacramento 8,390 1 

(-3 , 5) 

0.01% 

(-0.04, 0.06%) 

St. Louis 3,992 1 

(-2 , 4) 

0.03% 

(-0.04, 0.10%) 

Source:  Smith and Gibbs (2007), Table 1.

Table 2.  Total Annual Non‐Accidental Deaths by Risk Analysis City and Number of Those Deaths

Attributed to Days When the Maximum 8‐Hour Average Ozone Level Exceeds 70 ppb When Just

Attaining the a 74 ppb Standard (for Average of 2002 and 2004 Data; 95% Confidence Intervals in

Parentheses)  

 

Total Non-Accidental 

Deaths per Year 

Numbers of Deaths 

Attributable to Ozone > 

70 ppb (8-hr avg) 

Percent of Mortality 

Incidence 

Atlanta 9,233 0.6 

(-2.6 , 3.7) 

0.01% 

(-0.03, 0.04%) 

Cleveland 14,749 4.0 

(-2.5 , 10.4) 

0.03% 

(-0.02, 0.07%) 

Detroit 18,817 1.9 

(-0.6 , 4.5) 

0.01% 

(-0.00, 0.02%) 

Houston 18,122 1.5 

(0.1 , 3.0) 

0.02% 

(0.00, 0.04%) 

Los Angeles 54,157 0 

(0 , 0) 

0% 

(0, 0%) 

Sacramento 8,390 0.1 

(-0.3 , 0.5) 

0.00% 

(-0.00, 0.01%) 

St. Louis 3,992 0.4 

(-0.8 , 1.6) 

0.01% 

(-0.02, 0.04%) 

Source:  Smith and Gibbs (2007), Appendix A.

Although Tables 1 and 2 report mortality risk estimates, the same percentage attributions are true for

the various morbidity risk estimates as well.1

  This is because the quantitative risk estimates for all of the

epidemiologically‐based health endpoints are based on the same set of data of hourly ozone levels.

                                                           

1 Direct evidence supporting this statement can be found in Appendix B of Riker and Smith (2007).
4

These ambient concentration data imply that only a fraction of days would have ozone exceeding 70

ppb, and therefore only a fraction of the total calculated risk would be attributable to exposures above

that level, whether that risk is for mortality or any morbidity endpoint.   

I note also that the mortality risk estimates are statistically insignificant for most of the cities

considered.  Thus, even when focusing on the risks attributable to days with moderate to high ozone

exposure, the quantitative estimates of risk are represented by a wide range of uncertainty that centers

on zero.  Further, the statistical insignificance of the risk estimates is only one element of a much larger

degree of uncertainty regarding epidemiologically‐based risk estimates.  Other uncertainties such as

model selection, non‐linearity of the concentration‐response function at lower and lower

concentrations, and the role of other pollutants need also to be considered.   If these additional

uncertainties had been integrated into the primary risk estimates, even smaller population risk

estimates would gain in likelihood.

Large Uncertainties in Quantitative Risk Analysis Due to EPA’s “Roll Back” Formula

The above section focused on facts about EPA’s own quantitative risk estimates, without any comment

or deviation from the assumptions that EPA used to perform those calculations.  The fact that a majority

of the estimated risk at attainment of the current standard of 75 ppb is attributable to days when ozone

is below 60 ppb may give some people pause.   Others, however, may not feel that reduced weight

should be given to risks estimated at very low exposure levels just due to epidemiological uncertainties.  

Even if one believes that the same relative risks apply to ozone exposures below 8‐hour averages of 60

ppb, there is a separate aspect of EPA’s Risk Analysis methodology that implies very large uncertainty

that a tightening of the ozone NAAQS will produce any reduction in health effects that are estimated

due to those lower ozone exposures.   In other words, even if the health effects exist, the benefits that

EPA estimates from changing the standard may not exist.

EPA’s risk estimates for each alternative ozone standard are derived by first assuming how a tighter

ozone standard would affect ozone on every hour of the ozone season.  This is called the “rollback”.  For

ozone, EPA applies a “quadratic rollback” method that reduces higher ozone concentrations more than

smaller concentrations (Rizzo, 2005 and 2006).  This general concept has some commonsense merit, but

a relative paucity of empirical support.  Empirical support is especially lacking for how well EPA’s rollback

method can simulate how an ozone distribution that is in attainment with the either the 84 ppb or 75

ppb standard would be further reduced to attain a yet‐tighter standard in the range of 60 to 70 ppb.  

Nevertheless, all of the estimates of changes in risk due to tighter standards are predicated on this

rollback formula, and thus uncertainty in the estimates of health benefits from tightening the standard

is created by uncertainty in the rollback.

A little‐appreciated aspect of EPA’s rollback methodology is that it does not account for any level of

background ozone.  Although EPA does not compute risks for days when ozone falls below its assumed

“policy relevant background” (PRB) level, EPA’s rollback method assumes that ozone levels that are

already below background will actually be reduced even lower if the ozone standard is tightened.   
5

This does not create any bias in the risk reduction estimates as long as that day is below background

even before the rollback.   The bias arises from the majority of the days on which ozone is relatively low,

but still above background.  For those days, EPA has calculated that the rollback relative to 0 ppb, rather

than relative to the background level of ozone.  This may seem like a minor point, but it is not.  Even a

single ppb of overstatement by the rollback formula on the days with relatively low ozone (e.g., days

with ozone peaks in the range of 30 to 50 ppb) can cause EPA’s estimated change in risk to be

overstated by a factor of two or more.  This is because the vast majority of EPA’s estimated risk is

attributable to small changes in ozone concentrations (e.g., less than 2 ppb reduction) that occur on

low‐ozone days.   It is true that relatively large changes in concentrations (e.g., 5 to 15 ppb) will occur on

the few days that exceed or nearly exceed the standard in order to bring a region into attainment with a

tighter standard, and that those large changes are quite certain to occur if attainment is to occur.  

However, there are so few such days in the overall ozone distributions that EPA is using in its Risk

Analysis that they account for only a very small share of the total estimated reduction in risk (or

“benefit”) due to a tightened standard.    

The bulk of the health benefits that EPA estimates come from assumed reductions on days that are not

only considered “clean” but also which fall into the realm that is very close to, or even within, levels

considered to be uncontrollable under SIPs.  It is for these days that the bias from EPA’s rollback formula

is largest, and these types of days also comprise a majority of the ozone distribution.  This can be seen in

Figure 1 which shows the distribution of ozone EPA derived to simulate Detroit under exact attainment

of the 84 ppb NAAQS (presented as the curve marked by green diamonds in Figure 1), starting from the

observed “as‐is” ozone distribution shown by the black line.  The days that are most liable to have an

overstated ozone reduction due to the use of a rollback method that does not account for background

are those in the range where background levels may exist, but still well above 0 ppb.  This points to the

days with peak ozone levels between about 25 ppb and 40 ppb.2

  From Figure 1, one can see that fully

60% of the ozone‐season days are in this range.3

  The horizontal distance between the green and black

lines reflects the amount by which ozone at each as‐is level is reduced by EPA’s quadratic rollback

formula.  As can be seen, for the majority of days whose 8‐hour peak is already below 40 ppb, the

rollback method assumes about a 3 to 5 ppb of further reduction.  Whether such reductions can

realistically be expected to occur is a great uncertainty.  Whether such large percentage reductions can

be expected to continue to occur as that distribution is reduced to yet‐tighter standards such as 75 ppb,

or 65 ppb, is an even greater uncertainty.  One of the most important elements of that uncertainty is

whether those days’ ozone is as controllable as the quadratic rollback formula assumes.   The red line in

Figure 2 presents an alternative rolled back distribution that also just attains the 84 ppb standard, but

one in which the rollback formula is quadratic relative to a background level of 40 ppb.  If this is a more

realistic way of estimating the distribution’s shift to meet the 84 ppb standard, the overstatement in

EPA’s approach would be roughly proportional to the horizontal gap between the green and red lines.  

                                                           

2 The 24‐hour averages for these days, which are what is actually used in the risk calculation, are even lower. 3 That is, 25 ppb is at the 15% percentile and 40 ppb at about the 75% percentile, implying that 60% of days fall in

this range.    
6

As can be seen in Figure 1, that overstatement largely comes from the large percentage of “clean” days

in the range from 25 to 40 ppb 8‐hour average peaks.

Source:  Riker and Smith (2006), Figure 1.

Even modest uncertainty about that degree of rollback on those low‐ozone days translates into a very

significant potential overstatement in the estimated benefits from tightening the standard.  This is

because a majority the benefits are due to days in a range that has great rollback uncertainty.  Thus,

even if one were to believe that exposure to ozone in these lower concentrations causes the kinds and

amount of health effects that EPA has calculated in its Risk Analysis, the ability to actually reduce those

exposures by tightening the ozone standard is subject to very large and untested uncertainties.  This

rollback uncertainty should not be discounted when evaluating the usefulness of the quantitative risk

analysis for determining whether to tighten the ozone standard further.

Large Uncertainty in Risk Estimates Due to Uncertainty about Policy Relevant Background

The uncertainty in the roll back method raises a related point, which is uncertainty in the EPA

quantitative risk estimates due to its assumption about the level of PRB.  The problem I have identified

in the rollback method is driven by uncertainty in the level of ozone that cannot effectively be reduced

further due to being from natural or non‐US manmade precursor emissions, or even difficult‐to‐control

U.S. emissions sources.  This is a somewhat different concept than PRB, given how EPA has chosen to
7

define PRB.  The point of resistance to further rollback is an empirical matter that cannot be affected by

any change in the definition of PRB, or estimate of PRB.  It will remain an important element of the

uncertainty in benefits estimates even if EPA were to decide dispense with the notion of PRB in its risk

analysis.  Nevertheless, CASAC may be interested to also understand how utterly sensitive EPA’s

quantitative risk estimates are to its assumption about PRB.  Please note that statements regarding PRB

that follow are in terms of 24‐hour average ozone concentrations.  Thus, the numerical values that

follow are substantially lower (and not directly comparable to) the maximum daily 8‐hour average

concentrations that have been used in all the previous parts of my comments.   

EPA’s 2008 Risk Analysis bases its estimate of PRB on output of a specific global emissions scenario from

a global atmospheric model called GEOS‐CHEM.  The use of GEOS‐CHEM was new during the 2008 ozone

review.  Prior to that review, EPA had relied on evidence from remote monitoring locations to estimate

PRB, and used a value of 40 ppb.  The GEOS‐CHEM PRB levels are substantially lower, ranging from 14 to

34 ppb, depending on the location and the month.  Tables 3 and 4 compare the estimates of mortality

when using the GEOS‐CHEM outputs for the PRB assumption, and when using the monitor‐based

estimate of 40 ppb.  Table 3 is for air quality rolled back from as‐is to just attain the 84 ppb NAAQS and

Table 4 shows the same sensitivity after further rollback to exactly attain a 74 ppb NAAQS, which is a

close approximation of the current ozone standard of 75 ppb.  (The estimates using GEOS‐CHEM PRB

match EPA’s estimates of total mortality under each respective NAAQS standard, and come from the

same calculations as the data in Tables 1 and 2 above.)  The last column of Tables 3 and 4 show the

percent reduction in the total mortality risk estimate that occurs by simply replacing the current PRB

value with the earlier value of 40 ppb:  the risk estimate falls by 92% to 100% in the case of the 74 ppb

NAAQS, and nearly as much for the 84 ppb NAAQS.   These same percentage change sensitivities apply

to all the quantitative risk estimates in the EPA Risk Analysis for these two respective NAAQS standards,

including all the morbidity risk estimates and all alternative mortality risk estimates.  

The results in Tables 3 and 4 are so dramatic that they merit some discussion.  They reveal that almost

all of EPA’s quantitative estimates of risk are due to days when the 24‐hour average of ozone is below

40 ppb.  The “correct level” of PRB is subject to considerable debate and uncertainty, and my sensitivity

analysis cannot help resolve that debate.  However, it brings to light the exceptional degree to which

EPA’s risk estimates depend on this single assumption regarding a highly nebulous concept.4

  It also

brings to light the fact that if EPA were to perform its risk analysis without any change in its assumption

on PRB since the prior review cycle, its mortality and morbidity risk estimates at a standard

                                                           

4 I also caution CASAC that this problem regarding the rather arbitrary notion of PRB cannot be resolved by simply

dropping PRB from the risk analysis.  This is because, as I have noted above, the roll back methodology does not

account for the point where ozone starts to become resistant to further emissions control effort.  The use of PRB in

the risk calculation that proceeds using those rolled back distributions screens out at least some of the unrealistic

reductions in ozone.  PRB cannot be dropped altogether from the risk estimation unless and until the roll back

methodology is revised to account for estimates of ozone level s that cannot be further reduced by domestic

emissions control measures.
8

approximating the current 75 ppb standard would be 92 to 100 percent smaller.  Therefore, the increase

in EPA’s quantitative estimates of risk since the previous review is almost entirely because EPA changed

its assumption regarding the PRB level.  Any suggestion that new information since the last ozone

review has increased the estimates of ozone’s health risks is not true.  Given that the actual values for

PRB, and especially its day‐to‐day variability, are unknown, the high degree of sensitivity to this

uncertain assumption should be an important concern in relying on the quantitative risk analysis for

standard setting purposes.   

Table 3.  Sensitivity of Quantitative Risk Estimates PRB Assumption at Exact Attainment of 84 ppb

NAAQS  (Average of 2002 and 2004 Air Quality Data)  

 

Numbers of Deaths 

Estimated Using 

GEOS-CHEM for PRB 

Assumptions 

Numbers of Deaths 

Estimated Using 

PRB = 40 ppb 

Assumption 

Percent Reduction in 

Risk Estimate from 

Change in PRB 

Assumption 

Atlanta 

 6.5 0.3 95% 

Cleveland 

 37.4 4.3 88% 

Detroit 

 37.9 1.8 95% 

Houston 

 22.7 1.5 94% 

Los Angeles 

 38.5 0.1 99% 

Sacramento 

 11.0 0.2 98% 

St. Louis 

 4.4 0.3 93% 

Source:  Smith and Gibbs, 2007, Appendix C.

Table 4.  Sensitivity of Quantitative Risk Estimates PRB Assumption at Exact Attainment of a 74 ppb

NAAQS  (Average of 2002 and 2004 Air Quality Data)  

 

Numbers of Deaths 

Estimated Using 

GEOS-CHEM for PRB 

Assumptions 

Numbers of Deaths 

Estimated Using 

PRB = 40 ppb 

Assumption 

Percent Reduction in 

Risk Estimate from 

Change in PRB 

Assumption 

Atlanta 

 5.3 0.1 98% 

Cleveland 

 31.7 2.6 92% 

Detroit 

 30.2 0.7 98% 

Houston 

 17.8 0.7 96% 

Los Angeles 

 28.6 0.0 100% 

Sacramento 

 9.5 0.1 99% 

St. Louis 

 3.4 0.2 96% 

Source:  Smith and Gibbs, 2007, Appendix C.   
9

Conclusion

In conclusion, the risks and risk reductions that EPA has implicitly estimated for exposures to ozone

above 70 ppb and even 60 ppb (8‐hour average) are quantitatively small.  EPA has not reported its Risk

Analysis results in a way that one enables a reader to see this fact.  At the same time, the majority of the

risk reductions that EPA has estimated would result from tightening the ozone NAAQS come from

projected health effects occurring at ozone exposures below 60 ppb (8‐hour average).  That risk

reduction benefit is subject to much more substantial uncertainty for two independent reasons.  One is

the widening scientific uncertainty that effects such as premature death can be reasonably ascribed to

maximum 8‐hour ozone exposures below 60 ppb, even if epidemiological statistics find an association.  

The second uncertainty is whether maximum 8‐hour average ozone concentrations in the range of 25 to

45 ppb are likely to be able to roll back to the extent that the quadratic formula used by EPA assumes.   

Finally, I have shown that EPA’s decision to alter the PRB assumption is almost solely responsible for the

much larger quantitative risk estimates in the 2008 ozone review as compared to the prior ozone

review.  It is not new risk information that explains this increase in estimated risk, but an altered

assumption about a nebulous and uncertain concept called “policy relevant background.”

References

Riker, D. and Smith, A.  2006. Comments on EPA’s Draft Ozone Health Risk Assessment, submitted as part

of UARG comments, Docket # EPA‐HQ‐OAR‐2005‐0172‐0049.1, September 18, 2006.  Available at

http://www.crai.com/ConsultingExpertise/listingdetails.aspx?id=13566&tID=9154&subtID=0&tertID=0&

fID=34&SectionTitle=Climate+%26+Sustainability.

Rizzo, M.  2006. “A Distributional Comparison between Different Rollback Methodologies Applied to

Ozone Concentrations,” memorandum to Ozone NAAQS Review Docket (OAR‐2005‐0172), November 7,

2005.  Available at

http://www.epa.gov/ttn/naaqs/standards/ozone/data/20060823_rizzo_rollback_evaluation.pdf.

Rizzo, M.  2005.  “A Comparison between Different Rollback Methodologies Applied to Ambient Ozone

Concentrations,” memorandum to Ozone NAAQS Review Docket (OAR‐2005‐0172), November 7, 2005.  

Available at http://www.epa.gov/ttn/naaqs/standards/ozone/data/rizzo‐memo‐11‐05.pdf.

Smith, A. and Gibbs, B.  2007. Comments on EPA’s Proposed Rule to Revise the National Ambient Air

Quality Standards for Ozone (72 FR 37818), submitted as part of UARG comments, Docket # EPA‐HQ‐

OAR‐2005‐0172‐4183.1, October 9, 2007. Available at

http://www.crai.com/ConsultingExpertise/listingdetails.aspx?id=13565&tID=9154&subtID=0&tertID=0&

fID=34&SectionTitle=Climate+%26+Sustainability.

Smith, A.  2008.  Letter to Ms. Lucinda Minton Langworthy (reviewing data in memo dated November

27, 2007 from Abt Associates that was placed in Ozone NAAQS Docket after end of comment period),

letter submitted to docket by UARG, Docket # EPA‐HQ‐OAR‐2005‐7004, January 9, 2008. Available at

http://www.regulations.gov/#!documentDetail;D=EPA‐HQ‐OAR‐2005‐0172‐7004.1.  Supplemental data
10

added to Docket # EPA‐HQ‐OAR‐2005‐7051 on February 4, 2008 is available at

http://www.regulations.gov/#!documentDetail;D=EPA‐HQ‐OAR‐2005‐0172‐7051.1.