Draft – Do Not Cite or Quote xlix September 2011

PREAMBLE

Process of ISA Development

1 This preamble outlines the general process for developing an Integrated Science 

2 Assessment (ISA) including the framework for evaluating weight of evidence and 

3 drawing scientific conclusions and causal judgments. The ISA provides a concise review, 

4 synthesis, and evaluation of the most policy-relevant science to serve as a scientific 

5 foundation for the review of the National Ambient Air Quality Standards (NAAQS). The 

6 general process for NAAQS reviews is described at 

7 http://www.epa.gov/ttn/naaqs/review.html; information for individual NAAQS reviews is 

8 available at www.epa.gov/ttn/naaqs. This preamble is a general discussion of the basic 

9 steps and criteria used in developing an ISA; for each ISA, specific details and 

10 considerations are included in the introductory section for that assessment.

11 The fundamental process for developing an ISA includes:

12  literature searches;

13  study selection;

14  evaluation and integration of the evidence; and 

15  development of scientific conclusions and causal judgments.

16 An initial step in this process is publication of a call for information in the Federal 

17 Register that invites the public to provide information relevant to the assessment, such as 

new publications on health or welfare1 18 effects of the pollutant, or from atmospheric and 

19 exposure sciences fields. EPA maintains an ongoing literature search process for 

20 identification of relevant scientific studies published since the last review of the NAAQS. 

21 Search strategies are designed for pollutants and scientific disciplines and iteratively 

22 modified to optimize identification of pertinent publications. Papers are identified for 

23 inclusion in several additional ways: specialized searches on specific topics; independent 

24 review of tables of contents for journals in which relevant papers may be published; 

25 independent identification of relevant literature by expert scientists; review of citations in 

26 previous assessments and identification by the public and CASAC during the external 

27 review process. Publications considered for inclusion in the ISA are added to the Health 

28 and Environmental Research Online (HERO) database developed by EPA 

29 (http://hero.epa.gov/); the references in the ISA include a hyperlink to the database. 

 1 Welfare effects as defined in Clean Air Act section 302(h) [42 U.S.C. 7602(h)] include, but are not limited to, “effects on soils, 

water, crops, vegetation, man-made materials, animals, wildlife, weather, visibility and climate, damage to and deterioration of 

property, and hazards to transportation, as well as effects on economic values and on personal comfort and well-being.”
Draft – Do Not Cite or Quote l September 2011

1 Studies that have undergone scientific peer review and have been published or accepted 

2 for publication and reports that have undergone review are considered for inclusion in the 

3 ISA. Analyses conducted by EPA using publicly available data are also considered for 

4 inclusion in the ISA. All relevant epidemiologic, controlled human exposure, 

5 toxicological, and ecological and welfare effects studies published since the last review 

6 are considered, including those related to exposure-response relationships, mode(s) of 

7 action (MOA), and potentially at-risk populations and lifestages. Studies on atmospheric 

8 chemistry, environmental fate and transport, dosimetry, toxicokinetics and exposure are 

9 also considered for inclusion in the document, as well as analyses of air quality and 

10 emissions data. References that were considered for inclusion in a specific ISA can be 

11 found using the HERO website (http://hero.epa.gov).

12 Each ISA builds upon the conclusions of previous assessments for the pollutant under 

13 review. EPA focuses on peer reviewed literature published following the completion of 

14 the previous review and on any new interpretations of previous literature, integrating the

15 results of recent scientific studies with previous findings. Important older studies may be 

16 discussed in detail to reinforce key concepts and conclusions or for reinterpretation in

17 light of newer data. Older studies also are the primary focus in some areas of the 

18 document where research efforts have subsided, or if these older studies remain the 

19 definitive works available in the literature. 

20 Selection of studies for inclusion in the ISA is based on the general scientific quality of 

21 the study, and consideration of the extent to which the study is informative and policy22 relevant. Policy relevant and informative studies include those that provide a basis for or 

23 describe the relationship between the criteria pollutant and effects, including studies that 

24 offer innovation in method or design and studies that reduce uncertainty on critical issues, 

25 such as analyses of confounding or effect modification by copollutants or other variables, 

26 analyses of concentration-response or dose-response relationships, or analyses related to 

27 time between exposure and response. Emphasis is placed on studies that examine effects 

28 associated with pollutant concentrations relevant to current population and ecosystem 

29 exposures, and particularly those pertaining to concentrations currently found in ambient 

30 air. Other studies are included if they contain unique data, such as a previously 

31 unreported effect or MOA for an observed effect, or examine multiple concentrations to 

32 elucidate exposure-response relationships. In general, in assessing the scientific quality 

33 and relevance of health and welfare effects studies, the following considerations have 

34 been taken into account when selecting studies for inclusion in the ISA.

35  Are the study populations, subjects, or animal models adequately selected, and 

36 are they sufficiently well defined to allow for meaningful comparisons 

37 between study or exposure groups? 
Draft – Do Not Cite or Quote li September 2011

1  Are the statistical analyses appropriate, properly performed, and properly 

2 interpreted? Are likely covariates adequately controlled or taken into account 

3 in the study design and statistical analysis? 

4  Are the air quality data, exposure, or dose metrics of adequate quality and 

5 sufficiently representative of information regarding ambient conditions?

6  Are the health, ecological or welfare effect measurements meaningful, valid 

7 and reliable?

8  Do the analytical methods provide adequate sensitivity and precision to 

9 support conclusions?

10 Considerations specific to particular disciplines include the following. In selecting 

11 epidemiologic studies, EPA considers whether a given study: (1) presents information on 

12 associations with short- or long-term pollutant exposures at or near ambient conditions; 

13 (2) addresses potential confounding by other pollutants; (3) assesses potential effect 

14 modifiers; (4) evaluates health endpoints and populations not previously extensively 

15 researched; and (5) evaluates important methodological issues related to interpretation of 

16 the health evidence (e.g., lag or time period between exposure and effects, model 

17 specifications, thresholds, mortality displacement). 

18 Considerations for the selection of research evaluating controlled human exposure or 

19 animal toxicological studies includes a focus on studies conducted using relevant 

20 pollutant exposures. For both types of studies, relevant pollutant exposures are 

21 considered to be those generally within one or two orders of magnitude of ambient 

22 concentrations. Studies in which higher doses were used may also be considered if they 

23 provide information relevant to understanding MOA or mechanisms, as noted below.

24 Evaluation of controlled human exposure studies focuses on those that approximated 

25 expected human exposure conditions in terms of concentration and duration. Studies 

26 should include control exposures to filtered air, as appropriate. In the selection of 

27 controlled human exposure studies, emphasis is placed on studies that: (1) investigate 

28 potentially at-risk populations and lifestages such as people with asthma or 

29 cardiovascular diseases, children or older adults; (2) address issues such as concentration30 response or time-course of responses; and (3) have sufficient statistical power to assess 

31 findings. 

32 Review of the animal toxicological evidence focuses on studies that approximate 

33 expected human dose conditions, which vary depending on the dosimetry, toxicokinetics 

34 and biological sensitivity of the particular laboratory animal species or strains studied. 

35 Emphasis is placed on studies that: (1) investigate animal models of disease that can 

36 provide information on populations potentially at increased risk of effects; (2) address 
Draft – Do Not Cite or Quote lii September 2011

1 issues such as concentration-response or time-course of responses; and (3) have sufficient 

2 statistical power to assess findings. Due to resource constraints on exposure duration and 

3 numbers of animals tested, animal studies typically utilize high-concentration exposures 

4 to acquire data relating to mechanisms and assure a measurable response. Emphasis is 

5 placed on studies using doses or concentrations generally within 1-2 orders of magnitude 

6 of current levels. Studies with higher concentration exposures or doses are considered to 

7 the extent that they provide useful information to inform our understanding of 

8 interspecies differences and potential differences between healthy and susceptible human 

9 populations. Results from in vitro studies may also be included if they provide 

10 mechanistic insight or further support for results demonstrated in vivo.

11 These criteria provide benchmarks for evaluating various studies and for focusing on the 

12 policy-relevant studies in assessing the body of health, ecological and welfare effects 

13 evidence. As stated initially, the intent of the ISA is to provide a concise review, 

14 synthesis, and evaluation of the most policy-relevant science to serve as a scientific 

15 foundation for the review of the NAAQS, not extensive summaries of all health, 

16 ecological and welfare effects studies for a pollutant. Of most relevance for inclusion of 

17 studies is whether they provide useful qualitative or quantitative information on 

18 exposure-effect or exposure-response relationships for effects associated with pollutant 

19 exposures at doses or concentrations relevant to ambient conditions that can inform 

20 decisions on whether to retain or revise the standards. 

21 In developing an ISA, EPA reviews and summarizes the evidence from: studies of 

22 atmospheric sciences and exposure; the health effects evidence from toxicological, 

23 controlled human exposure and epidemiologic studies; and ecological and welfare effects 

24 evidence. In the process of developing the first draft ISA, EPA may convene a public 

25 workshop in which EPA and non-EPA experts review the scientific content of 

26 preliminary draft materials to ensure that the ISA is up to date and focused on the most 

27 policy-relevant findings, and to assist EPA with integration of evidence within and across 

28 disciplines. 

29 EPA integrates the evidence from across scientific disciplines or study types and 

30 characterizes the weight of evidence for relationships between the pollutant and various 

31 outcomes. The integration of evidence on health, and ecological or welfare effects, 

32 involves collaboration between scientists from various disciplines. As an example, an 

33 evaluation of health effects evidence would include the integration of the results from 

34 epidemiologic, controlled human exposure, and toxicological studies, and application of 

35 the causal framework (described below) to draw conclusions. Using the causal 

36 framework described in the following section, EPA scientists consider aspects such as 

37 strength, consistency, coherence, and biological plausibility of the evidence, and develop 
Draft – Do Not Cite or Quote liii September 2011

1 draft causality determinations on the nature of the relationships. Causality determinations 

2 often entail an iterative process of review and evaluation of the evidence. Two drafts of 

3 the ISA are typically released for review by the CASAC and the public, and comments 

4 received on the characterization of the science as well as the implementation of the causal 

5 framework are carefully considered in revising and completing the final ISA.

EPA Framework for Causal Determination

6 EPA has developed a consistent and transparent basis to evaluate the causal nature of air 

7 pollution-related health or welfare effects for use in developing ISAs. The framework 

8 described below establishes uniform language concerning causality and brings more 

9 specificity to the findings. This standardized language was drawn from sources across the 

10 federal government and wider scientific community, especially the National Academy of 

11 Sciences (NAS) Institute of Medicine (IOM) document, Improving the Presumptive 

12 Disability Decision-Making Process for Veterans (2008), a comprehensive report on 

13 evaluating causality. This framework:

14  describes the kinds of scientific evidence used in establishing a general causal 

15 relationship between exposure and health effects; 

16  characterizes the evidence necessary to reach a conclusion about the existence 

17 of a causal relationship; 

18  identifies issues and approaches related to uncertainty; and

19  provides a framework for classifying and characterizing the weight of 

20 evidence in support of a general causal relationship.

21 Approaches to assessing the separate and combined lines of evidence 

22 (e.g., epidemiologic, controlled human exposure, and animal toxicological studies) have 

23 been formulated by a number of regulatory and science agencies, including the IOM of 

24 the NAS (2008), International Agency for Research on Cancer (2006), EPA Guidelines 

25 for Carcinogen Risk Assessment (2005), and Centers for Disease Control and Prevention 

26 (2004). Causal inference criteria have also been described for ecological effects evidence 

27 (U.S. EPA, 1998; Fox, 1991). These formalized approaches offer guidance for assessing 

28 causality. The frameworks are similar in nature, although adapted to different purposes, 

29 and have proven effective in providing a uniform structure and language for causal 

30 determinations.
Draft – Do Not Cite or Quote liv September 2011

Evaluating Evidence for Inferring Causation 

1 The 1964 Surgeon General’s report defined “cause” as a “significant, effectual 

2 relationship between an agent and an associated disorder or disease in the host” (HEW); 

3 more generally, a cause is defined as an agent that brings about an effect or a result. An 

4 association is the statistical relationship among variables; alone, however, it is 

5 insufficient proof of a causal relationship between an exposure and a health outcome. 

6 Unlike an association, a causal claim supports the creation of counterfactual claims, that 

7 is, a claim about what the world would have been like under different or changed 

8 circumstances (Samet and Bodurow, 2008).

9 Many of the health and environmental outcomes reported in these studies have complex 

10 etiologies. Diseases such as asthma, coronary heart disease (CHD) or cancer are typically 

11 initiated by multiple agents. Outcomes depend on a variety of factors, such as age, 

12 genetic susceptibility, nutritional status, immune competence, and social factors (Samet 

13 and Bodurow, 2008; Gee and Payne-Sturges, 2004). Effects on ecosystems are often also 

14 multifactorial with a complex web of causation. Further, exposure to a combination of 

15 agents could cause synergistic or antagonistic effects. Thus, the observed risk may 

16 represent the net effect of many actions and counteractions. 

17 In estimating the causal influence of an exposure on health or environmental effects, it is 

18 recognized that scientific findings incorporate uncertainty. “Uncertainty” can be defined 

19 as having limited knowledge to exactly describe an existing state or future outcome, 

20 e.g., the lack of knowledge about the correct value for a specific measure or estimate. 

21 Uncertainty analysis may be qualitative or quantitative in nature. In many cases, the 

22 analysis is qualitative, and can include professional judgment or inferences based on 

23 analogy with similar situations. Quantitative uncertainty analysis may include use of 

24 simple measures (e.g., ranges) and analytical techniques. Quantitative uncertainty 

25 analysis might progress to more complex measures and techniques, if needed for decision 

26 support. Various approaches to evaluating uncertainty include classical statistical 

27 methods, sensitivity analysis, or probabilistic uncertainty analysis, in order of increasing 

28 complexity and data requirements. However, data may not be available for all aspects of 

29 an assessment and those data that are available may be of questionable or unknown 

30 quality. Ultimately, the assessment is based on a number of assumptions with varying 

31 degrees of uncertainty. The ISA generally evaluates uncertainties qualitatively in 

32 assessing the evidence from across studies; in some situations quantitative analysis 

33 approaches, such as meta-regression, may be used.

34 Publication bias is a source of uncertainty regarding the magnitude of health risk 

35 estimates. It is well understood that studies reporting non-null findings are more likely to 

36 be published than reports of null findings, and publication bias can also result in 
Draft – Do Not Cite or Quote lv September 2011

1 overestimation of effect estimate sizes (Ioannidis, 2008). For example, effect estimates 

2 from single-city epidemiologic studies have been found to be generally larger than those 

3 from multicity studies (Bell et al., 2005). 

Consideration of evidence from scientific disciplines

4 Moving from association to causation involves the elimination of alternative explanations 

5 for the association. The ISA focuses on evaluation of the findings from the body of 

6 evidence, drawing upon the results of all studies determined to meet the criteria described 

7 previously. Causality determinations are based on the evaluation and synthesis of 

8 evidence from across scientific disciplines. The relative importance of different types of 

9 evidence varies by pollutant or assessment, as does the availability of different types of 

10 evidence for causality determination. Three general types of studies inform consideration 

11 of human health effects: controlled human exposure, epidemiologic and toxicological 

12 studies. Evidence on ecological or welfare effects may be drawn from a variety of 

13 experimental approaches (e.g., greenhouse, laboratory, field) and numerous disciplines 

14 (e.g., community ecology, biogeochemistry and paleological/historical reconstructions).

15 The most direct evidence of a causal relationship between pollutant exposures and human 

16 health effects comes from controlled human exposure studies. Controlled human 

17 exposure studies experimentally evaluate the health effects of administered exposures in 

18 human volunteers under highly controlled laboratory conditions. Also referred to as 

19 human clinical studies, these experiments allow investigators to expose subjects to known 

20 concentrations of air pollutants under carefully regulated environmental conditions and 

21 activity levels. In some instances, controlled human exposure studies can also be used to 

22 characterize concentration-response relationships at pollutant concentrations relevant to 

23 ambient conditions. Controlled human exposures are typically conducted using a 

24 randomized crossover design, with subjects exposed both to the pollutant and a clean air 

25 control. In this way, subjects serve as their own controls, effectively controlling for many

26 potential confounders. However, controlled human exposure studies are limited by a 

27 number of factors, including small sample size and short exposure time. For example, 

28 exposure patterns relevant to understanding real-world exposures, especially long-term 

29 exposures, are generally not practical to replicate in a laboratory setting. In addition, 

30 although subjects do serve as their own controls, personal exposure to pollutants in the 

31 hours and days preceding the controlled exposures may vary significantly between and 

32 within individuals. Finally, controlled human exposure studies require investigators to 

33 adhere to stringent health criteria for subjects included in the study, and therefore the 

34 results cannot necessarily be generalized to an entire population. Although some 

35 controlled human exposure studies have included health-compromised individuals such 

36 as those with respiratory or cardiovascular disease, these individuals must also be 
Draft – Do Not Cite or Quote lvi September 2011

1 relatively healthy and may not represent the most sensitive individuals in the population. 

2 In addition, the study design is limited to exposures and endpoints that are not expected 

3 to result in severe health outcomes. Thus, not observing an effect in controlled human 

4 exposure studies does not necessarily mean that a causal relationship does not exist. 

5 While controlled human exposure studies provide important information on the biological 

6 plausibility of associations observed in epidemiologic studies, observed effects in these 

7 studies may underestimate the response in certain populations.

8 Epidemiologic studies provide important information on the associations between health 

9 effects and exposure of human populations to ambient air pollution. In epidemiologic or 

10 observational studies of humans, the investigator generally does not control exposures or 

11 intervene with the study population. Broadly, observational studies can describe 

12 associations between exposures and effects. These studies fall into several categories: 

13 e.g., cross-sectional, prospective cohort, panel and time-series studies. “Natural 

14 experiments” offer the opportunity to investigate changes in health related to a change in 

15 exposure, such as closure of a pollution source. 

16 In evaluating epidemiologic studies, consideration of many study design factors and 

17 issues must be taken into account to properly inform their interpretation. One key 

18 consideration is evaluation of the potential contribution of the pollutant to a health 

19 outcome when it is a component of a complex air pollutant mixture. Reported effect 

20 estimates in epidemiologic studies may reflect: independent effects on health outcomes; 

21 effects of the pollutant acting as an indicator of a copollutant or a complex ambient air 

22 pollution mixture; effects resulting from interactions between that pollutant and 

23 copollutants.

24 In the evaluation of epidemiologic evidence, one important consideration is potential 

25 confounding. Confounding is “... a confusion of effects. Specifically, the apparent effect 

26 of the exposure of interest is distorted because the effect of an extraneous factor is

27 mistaken for or mixed with the actual exposure effect (which may be null)” (Rothman 

28 and Greenland, 1998). One approach to remove spurious associations due to possible 

29 confounders is to control for characteristics that may differ between exposed and 

30 unexposed persons; this is frequently termed “adjustment.” Scientific judgment is needed 

31 to evaluate likely sources and extent of confounding, together with consideration of how 

32 well the existing constellation of study designs, results, and analyses address this 

33 potential threat to inferential validity. A confounder is associated with both the exposure 

34 and the effect; for example, confounding can occur between correlated pollutants that are 

35 associated with the same effect. 

36 Several statistical methods are available to detect and control for potential confounders, 

37 with none of them being completely satisfactory. Multivariable regression models 
Draft – Do Not Cite or Quote lvii September 2011

1 constitute one tool for estimating the association between exposure and outcome after 

2 adjusting for characteristics of participants that might confound the results. The use of 

3 multipollutant regression models has been the prevailing approach for controlling 

4 potential confounding by copollutants in air pollution health effects studies. Finding the 

5 likely causal pollutant from multipollutant regression models is made difficult by the 

6 possibility that one or more air pollutants may be acting as a surrogate for an unmeasured 

7 or poorly measured pollutant or for a particular mixture of pollutants. In addition, more 

8 than one pollutant may exert similar health effects, resulting in independently observed 

9 associations for multiple pollutants. The number and degree of diversity of covariates, as 

10 well as their relevance to the potential confounders, remain matters of scientific 

11 judgment. Despite these limitations, the use of multipollutant models is still the 

12 prevailing approach employed in most air pollution epidemiologic studies and provides 

13 some insight into the potential for confounding or interaction among pollutants.

14 Confidence that unmeasured confounders are not producing the findings is increased 

15 when multiple studies are conducted in various settings using different subjects or 

16 exposures, each of which might eliminate another source of confounding from 

17 consideration. For example, multicity studies which use a consistent method to analyze 

18 data from across locations with different levels of covariates can provide insight on 

19 potential confounding in associations. Intervention studies, because of their quasi20 experimental nature, can be particularly useful in characterizing causation. 

21 Another important consideration in the evaluation of epidemiologic evidence is effect 

22 modification, which occurs when the effect differs between subgroups or strata; for 

23 example, effect estimates that vary by age group or potential risk factor. “Effect-measure 

24 modification differs from confounding in several ways. The main difference is that, 

25 whereas confounding is a bias that the investigator hopes to prevent or remove from the 

26 effect estimate, effect-measure modification is a property of the effect under study . . . In 

27 epidemiologic analysis one tries to eliminate confounding but one tries to detect and 

28 estimate effect-measure modification” (Rothman and Greenland, 1998). When a risk 

29 factor is a confounder, it is the true cause of the association observed between the 

30 exposure and the outcome; when a risk factor is an effect modifier, it changes the 

31 magnitude of the association between the exposure and the outcome in stratified analyses. 

32 For example, the presence of a preexisting disease or indicator of low socioeconomic 

33 status may be an effect modifier in causing increased risk of effects related to air 

34 pollution exposure. It is often possible to stratify the relationship between health outcome 

35 and exposure by one or more of these potential effect modifiers. For variables that 

36 modify the association, effect estimates in each stratum will be different from one another 

37 and different from the overall estimate, indicating a different exposure-response 

38 relationship may exist in populations represented by these variables.
Draft – Do Not Cite or Quote lviii September 2011

1 Another key consideration for epidemiologic evidence is exposure measurement error. 

2 There are several components that contribute to exposure measurement error in 

3 epidemiologic studies, including the difference between true and measured ambient 

4 concentrations, the difference between average personal exposure to ambient pollutants 

5 and ambient concentrations at central monitoring sites, and the use of average population 

6 exposure rather than individual exposure estimates. 

7 The third main type of health effects evidence, animal toxicological studies, provides 

8 information on the pollutant’s biological action under controlled and monitored exposure 

9 circumstances. Taking into account physiological differences of the experimental species 

10 from humans, these studies inform characterization of health effects of concern, 

11 exposure-response relationships and MOAs. Further, animal models can inform 

12 determinations of at-risk or susceptible populations. These studies evaluate the effects of 

13 exposures to a variety of pollutants in a highly controlled laboratory setting and allow 

14 exploration of toxicological pathways or mechanisms by which a pollutant may cause

15 effects. Understanding the biological mechanisms underlying various health outcomes 

16 can prove crucial in establishing or negating causality. In the absence of human studies 

17 data, extensive, well-conducted animal toxicological studies can support determinations 

18 of causality, if the evidence base indicates that similar responses are expected in humans 

19 under ambient exposure conditions. 

20 Interpretations of animal toxicological studies are affected by limitations associated with 

21 extrapolation between animal and human responses. The differences between humans 

22 and other species have to be taken into consideration, including metabolism, hormonal 

23 regulation, breathing pattern, and differences in lung structure and anatomy. Also, in spite 

24 of a high degree of homology and the existence of a high percentage of orthologous 

25 genes across humans and rodents (particularly mice), extrapolation of molecular 

26 alterations at the gene level is complicated by species-specific differences in 

27 transcriptional regulation. Given these differences, there are uncertainties associated with 

28 quantitative extrapolations of observed pollutant-induced pathophysiological alterations 

29 between laboratory animals and humans, as those alterations are under the control of 

30 widely varying biochemical, endocrine, and neuronal factors.

31 For ecological effects assessment, both laboratory and field studies (including field 

32 experiments and observational studies) can provide useful data for causality 

33 determination. Because conditions can be controlled in laboratory studies, responses may 

34 be less variable and smaller differences easier to detect. However, the control conditions 

35 may limit the range of responses (e.g., animals may not be able to seek alternative food 

36 sources), so they may not reflect responses that would occur in the natural environment. 

37 In addition, larger-scale processes are difficult to reproduce in the laboratory.
Draft – Do Not Cite or Quote lix September 2011

1 Field observational studies measure biological changes in uncontrolled situations, and 

2 describe an association between a disturbance and an ecological effect. Field data can 

3 provide important information for assessments of multiple stressors or where site-specific 

4 factors significantly influence exposure. They are also often useful for analyses of larger 

5 geographic scales and higher levels of biological organization. However, because 

6 conditions are not controlled, variability is expected to be higher and differences harder 

7 to detect. Field surveys are most useful for linking stressors with effects when stressor 

8 and effect levels are measured concurrently. The presence of confounding factors can 

9 make it difficult to attribute observed effects to specific stressors. 

10 Intermediate between laboratory and field are studies that use environmental media 

11 collected from the field to examine response in the laboratory, and experiments that are 

12 performed in the natural environment while controlling for some environmental 

13 conditions (i.e. mesocosm studies). This type of study in manipulated natural 

14 environments can be considered a hybrid between a field experiment and laboratory study 

15 since some aspects are performed under controlled conditions but others are not. They 

16 make it possible to observe community and/or ecosystem dynamics, and provide strong 

17 evidence for causality when combined with findings of studies that have been made 

18 under more controlled conditions.

Application of Framework for Causal Determination

19 In its evaluation of the scientific evidence on health or welfare effects of criteria 

20 pollutants, EPA determines the weight of evidence in support of causation and 

21 characterizes the strength of any resulting causal classification. EPA also evaluates the 

22 quantitative evidence and draws scientific conclusions, to the extent possible, regarding 

23 the concentration-response relationships and the loads to ecosystems, exposure doses or 

24 concentrations, duration and pattern of exposures at which effects are observed. 

To aid judgment, various “aspects”2 25 of causality have been discussed by many 

26 philosophers and scientists. The 1964 Surgeon General’s report on tobacco smoking 

27 discussed criteria for the evaluation of epidemiologic studies, focusing on consistency, 

28 strength, specificity, temporal relationship, and coherence (HEW, 1964). Sir Austin 

29 Bradford Hill (1965) articulated aspects of causality in epidemiology and public health 

30 that have been widely used (Samet and Bodurow, 2008; IARC, 2006; U.S. EPA, 2005; 

31 HHS, 2004). These aspects (Hill, 1965) have been modified (Table I) for use in causal 

 2 The “aspects” described by Hill (1965) have become, in the subsequent literature, more commonly described as “criteria.” The 

original term “aspects” is used here to avoid confusion with “criteria” as it is used, with different meaning, in the Clean Air Act.
Draft – Do Not Cite or Quote lx September 2011

Table I Aspects to aid in judging causality

Consistency of the 

observed association

An inference of causality is strengthened when a pattern of elevated risks is observed 

across several independent studies. The reproducibility of findings constitutes one of the 

strongest arguments for causality. If there are discordant results among investigations, 

possible reasons such as differences in exposure, confounding factors, and the power of 

the study are considered.

Coherence An inference of causality from one line of evidence (e.g., epidemiologic, clinical or animal 

studies) may be strengthened by other lines of evidence that support a cause-and-effect 

interpretation of the association. Evidence on ecological or welfare effects may be drawn 

from a variety of experimental approaches (e.g., greenhouse, laboratory, and field) and 

subdisciplines of ecology (e.g., community ecology, biogeochemistry and 

paleological/historical reconstructions). The coherence of evidence from various fields 

greatly adds to the strength of an inference of causality. In addition, there may be 

coherence in demonstrating effects across multiple study designs or related health 

endpoints within one scientific line of evidence.

Biological plausibility. An inference of causality tends to be strengthened by consistency with data from 

experimental studies or other sources demonstrating plausible biological mechanisms. A 

proposed mechanistic linking between an effect and exposure to the agent is an important 

source of support for causality, especially when data establishing the existence and 

functioning of those mechanistic links are available. 

Biological gradient 

(exposure-response 

relationship)

A well-characterized exposure-response relationship (e.g., increasing effects associated 

with greater exposure) strongly suggests cause and effect, especially when such 

relationships are also observed for duration of exposure (e.g., increasing effects observed 

following longer exposure times). 

Strength of the observed 

association

The finding of large, precise risks increases confidence that the association is not likely 

due to chance, bias, or other factors. However, it is noted that a small magnitude in an 

effect estimate may represent a substantial effect in a population.

Experimental evidence Strong evidence for causality can be provided through “natural experiments” when a 

change in exposure is found to result in a change in occurrence or frequency of health or 

welfare effects. 

Temporal relationship of 

the observed association

Evidence of a temporal sequence between the introduction of an agent, and appearance 

of the effect, constitutes another argument in favor of causality. 

Specificity of the 

observed association

Evidence linking an exposure to a specific outcome can provide a strong argument for 

causation. However, it must be recognized that rarely, if ever, does exposure to a pollutant 

invariably predict the occurrence of an outcome, and that a given outcome may have 

multiple causes. 

Analogy Structure activity relationships and information on the agent’s structural analogs can 

provide insight into whether an association is causal. Similarly, information on mode of 

action for a chemical, as one of many structural analogs, can inform decisions regarding 

likely causality. 

1 determinations specific to health and welfare effects for pollutant exposures (U.S. EPA, 

2009d).

3 2 Although these aspects provide a framework for assessing the evidence, they do 

3 not lend themselves to being considered in terms of simple formulas or fixed rules of 

4 evidence leading to conclusions about causality (Hill, 1965). For example, one cannot 

5 simply count the number of studies reporting statistically significant results or 

 3 The Hill aspects were developed for interpretation of epidemiologic results. They have been modified here for use with a broader 

array of data, i.e., epidemiologic, controlled human exposure, ecological, and animal toxicological studies, as well as in vitro data, 

and to be more consistent with EPA’s Guidelines for Carcinogen Risk Assessment.
Draft – Do Not Cite or Quote lxi September 2011

1 statistically nonsignificant results and reach credible conclusions about the relative 

2 weight of the evidence and the likelihood of causality. Rather, these aspects are taken into 

3 account with the goal of producing an objective appraisal of the evidence, informed by 

4 peer and public comment and advice, which includes weighing alternative views on 

5 controversial issues. In addition, it is important to note that the aspects in Table I cannot 

6 be used as a strict checklist, but rather to determine the weight of the evidence for 

7 inferring causality. In particular, not meeting one or more of the principles does not 

8 automatically preclude a determination of causality [see discussion in (HHS, 2004)].

Determination of Causality

9 In the ISA, EPA assesses the body of relevant literature, building upon evidence available 

10 during previous NAAQS reviews, to draw conclusions on the causal relationships 

11 between relevant pollutant exposures and health or environmental effects. ISAs use a 

five-level hierarchy that classifies the weight of evidence for causation4 12 . In developing

13 this hierarchy, EPA has drawn on the work of previous evaluations, most prominently the 

14 IOM’s Improving the Presumptive Disability Decision-Making Process for Veterans

15 (Samet and Bodurow, 2008), EPA’s Guidelines for Carcinogen Risk Assessment (U.S. 

16 EPA, 2005), and the U.S. Surgeon General’s smoking report (HHS, 2004). This weight of 

17 evidence evaluation is based on various lines of evidence from across the health and 

18 environmental effects disciplines. These separate judgments are integrated into a 

19 qualitative statement about the overall weight of the evidence and causality. The five 

20 descriptors for causal determination are described in Table II.

21 Determination of causality involves the evaluation of evidence for different types of 

22 health, ecological or welfare effects associated with short- and long-term exposure 

23 periods. In making determinations of causality, evidence is evaluated for major outcome 

24 categories and then conclusions are drawn based upon the integration of evidence from 

25 across disciplines and also across the spectrum of related endpoints. In making causal 

26 judgments, the ISA focuses on major outcome categories (e.g., respiratory effects, 

27 vegetation growth), by evaluating the coherence of evidence across a spectrum of related 

28 endpoints (e.g., health effects ranging from inflammatory effects to respiratory mortality) 

29 to draw conclusions regarding causality. In discussing the causal determination, EPA 

30 characterizes the evidence on which the judgment is based, including strength of 

31 evidence for individual endpoints within the major outcome category.

 4 It should be noted that the Center for Disease Control (CDC) and IOM frameworks use a four-category hierarchy for the strength 

of the evidence. A five-level hierarchy is used here to be consistent with the EPA Guidelines for Carcinogen Risk Assessment and to 

provide a more nuanced set of categories.
Draft – Do Not Cite or Quote lxii September 2011

Table II Weight of evidence for causal determination

Health Effects Ecological and Welfare Effects

Causal 

relationship

Evidence is sufficient to conclude that there is a causal 

relationship with relevant pollutant exposures (i.e., doses 

or exposures generally within one to two orders of 

magnitude of current levels). That is, the pollutant has 

been shown to result in health effects in studies in which 

chance, bias, and confounding could be ruled out with 

reasonable confidence. For example: a) controlled 

human exposure studies that demonstrate consistent 

effects; or b) observational studies that cannot be 

explained by plausible alternatives or are supported by 

other lines of evidence (e.g., animal studies or mode of 

action information). Evidence includes replicated and 

consistent high-quality studies by multiple investigators.

Evidence is sufficient to conclude that there is a causal 

relationship with relevant pollutant exposures i.e., doses 

or exposures generally within one to two orders of 

magnitude of current levels). That is, the pollutant has 

been shown to result in effects in studies in which 

chance, bias, and confounding could be ruled out with 

reasonable confidence. Controlled exposure studies 

(laboratory or small- to medium-scale field studies) 

provide the strongest evidence for causality, but the 

scope of inference may be limited. Generally, 

determination is based on multiple studies conducted by 

multiple research groups, and evidence that is considered 

sufficient to infer a causal relationship is usually obtained 

from the joint consideration of many lines of evidence that 

reinforce each other.

Likely to be a 

causal 

relationship

Evidence is sufficient to conclude that a causal 

relationship is likely to exist with relevant pollutant 

exposures, but important uncertainties remain. That is, 

the pollutant has been shown to result in health effects in 

studies in which chance and bias can be ruled out with 

reasonable confidence but potential issues remain. For 

example: a) observational studies show an association, 

but copollutant exposures are difficult to address and/or 

other lines of evidence (controlled human exposure, 

animal, or mode of action information) are limited or 

inconsistent; or b) animal toxicological evidence from 

multiple studies from different laboratories that 

demonstrate effects, but limited or no human data are 

available. Evidence generally includes replicated and 

high-quality studies by multiple investigators.

Evidence is sufficient to conclude that there is a likely 

causal association with relevant pollutant exposures. That 

is, an association has been observed between the 

pollutant and the outcome in studies in which chance, 

bias and confounding are minimized, but uncertainties 

remain. For example, field studies show a relationship, 

but suspected interacting factors cannot be controlled, 

and other lines of evidence are limited or inconsistent. 

Generally, determination is based on multiple studies in 

multiple research groups.

Suggestive of 

a causal 

relationship

Evidence is suggestive of a causal relationship with 

relevant pollutant exposures, but is limited. For example, 

(a) at least one high-quality epidemiologic study shows 

an association with a given health outcome but the 

results of other studies are inconsistent; or (b) a wellconducted toxicological study, such as those conducted 

in the National Toxicology Program (NTP), shows effects 

in animal species.

Evidence is suggestive of a causal relationship with 

relevant pollutant exposures, but chance, bias and 

confounding cannot be ruled out. For example, at least 

one high-quality study shows an effect, but the results of 

other studies are inconsistent.

Inadequate to 

infer a causal 

relationship

Evidence is inadequate to determine that a causal 

relationship exists with relevant pollutant exposures. The 

available studies are of insufficient quantity, quality, 

consistency or statistical power to permit a conclusion 

regarding the presence or absence of an effect.

The available studies are of insufficient quality, 

consistency or statistical power to permit a conclusion 

regarding the presence or absence of an effect.

Not likely to 

be a causal 

relationship

Evidence is suggestive of no causal relationship with 

relevant pollutant exposures. Several adequate studies, 

covering the full range of levels of exposure that human 

beings are known to encounter and considering at-risk 

populations, are mutually consistent in not showing an 

effect at any level of exposure. 

Several adequate studies, examining relationships with 

relevant exposures, are consistent in failing to show an 

effect at any level of exposure.

1 In drawing judgments regarding causality for the criteria air pollutants, the ISA focuses 

2 on evidence of effects in the range of relevant pollutant exposures or doses, and not on 

3 determination of causality at any dose. Emphasis is placed on evidence of effects at doses 

4 (e.g., blood lead concentration) or exposures (e.g., air concentrations) that are relevant to, 

5 or somewhat above, those currently experienced by the population. The extent to which 

6 studies of higher concentrations are considered varies by pollutant and major outcome 
Draft – Do Not Cite or Quote lxiii September 2011

1 category, but generally includes those with doses or exposures in the range of one to two 

2 orders of magnitude above current or ambient conditions. Studies that use higher doses or 

3 exposures may also be considered to the extent that they provide useful information to 

4 inform our understanding of mode of action, interspecies differences or factors that may 

5 increase risk of effects for a population. Thus, a causality determination is based on 

6 weight of evidence evaluation for health, ecological or welfare effects, focusing on the 

7 evidence from exposures or doses generally ranging from current levels to one or two 

8 orders of magnitude above current levels.

9 In addition, EPA evaluates evidence relevant to understand the quantitative relationships 

10 between pollutant exposures and health, ecological or welfare effects. This includes 

11 evaluation of the form of concentration-response or dose-response relationships and, to 

12 the extent possible, drawing conclusions on the levels at which effects are observed. The 

13 ISA also draws scientific conclusions regarding important exposure conditions for effects 

14 and populations that may be at greater risk for effects, as described in the following 

15 section.

Quantitative relationships: Effects on Human Populations

16 Once a determination is made regarding the causal relationship between the pollutant and 

17 outcome category, important questions regarding quantitative relationships include: 

18  What is the concentration-response, exposure-response, or dose-response 

19 relationship in the human population?

20  What is the interrelationship between incidence and severity of effect?

21  What exposure conditions (dose or exposure, duration and pattern) are 

22 important?

23  What populations and lifestages appear to be differentially affected (i.e., more 

24 at risk of experiencing effects)?

25 To address these questions, the entirety of quantitative evidence is evaluated to 

26 characterize pollutant concentrations and exposure durations at which effects were 

27 observed for exposed populations, including populations and lifestages potentially at 

28 increased risk. To accomplish this, evidence is considered from multiple and diverse 

29 types of studies, and a study or set of studies that best approximates the concentration30 response relationships between health outcomes and the pollutant may be identified. 

31 Controlled human exposure studies provide the most direct and quantifiable exposure32 response data on the human health effects of pollutant exposures. To the extent available, 

33 the ISA evaluates results from across epidemiologic studies that use various methods to 
Draft – Do Not Cite or Quote lxiv September 2011

1 characterize the form of relationships between the pollutant and health outcomes and 

2 draws conclusions on the shape of these relationships. Animal data may also inform 

3 evaluation of concentration-response relationships, particularly relative to MOAs and 

4 characteristics of susceptible populations. 

5 An important consideration in characterizing the public health impacts associated with 

6 exposure to a pollutant is whether the concentration-response relationship is linear across 

7 the range of concentrations or if nonlinear relationships exist along any part of this range. 

8 Of particular interest is the shape of the concentration-response curve at and below the 

9 level of the current standards. Various sources of variability and uncertainty, such as low 

10 data density in the lower concentration range, possible influence of exposure 

11 measurement error, and variability between individuals in susceptibility to air pollution 

12 health effects, tend to smooth and “linearize” the concentration-response function, and 

13 thus can obscure the existence of a threshold or nonlinear relationship. Since individual 

14 thresholds vary from person to person due to individual differences such as genetic level 

15 susceptibility or preexisting disease conditions (and even can vary from one time to 

16 another for a given person), it can be difficult to demonstrate that a threshold exists in a 

17 population study. These sources of variability and uncertainty may explain why the 

18 available human data at ambient concentrations for some environmental pollutants 

19 (e.g., particulate matter [PM], O3, lead [Pb], environmental tobacco smoke [ETS], 

20 radiation) do not exhibit thresholds for cancer or noncancer health effects, even though 

21 likely mechanisms include nonlinear processes for some key events. These attributes of

22 human population dose-response relationships have been extensively discussed in the 

23 broader epidemiologic literature (Rothman and Greenland, 1998).

24 Finally, identification of the population groups or lifestages that may be at greater risk of 

25 health effects from air pollutant exposures contributes to an understanding of the public 

26 health impact of pollutant exposures. In the ISA, the term “at-risk population” is used to 

27 encompass populations variously described as susceptible, vulnerable, or sensitive. “At28 risk populations” is defined here as those populations or lifestages that have a greater 

29 likelihood of experiencing health effects related to exposure to an air pollutant due to a 

30 variety of factors. These factors may be intrinsic, such as genetic or developmental 

31 factors, race, gender, lifestage, or the presence of preexisting diseases, or they may be 

32 extrinsic, such as socioeconomic status (SES), activity pattern and exercise level, reduced 

33 access to health care, low educational attainment, or increased pollutant exposures (e.g., 

34 near roadways). Epidemiologic studies can help identify populations potentially at 

35 increased risk of effects by evaluating health responses in the study population. Examples 

36 include testing for interactions or effect modification by factors such as gender, age 

37 group, or health status. Experimental studies using animal models of susceptibility or 
Draft – Do Not Cite or Quote lxv September 2011

1 disease can also inform the extent to which health risks are likely greater in specific 

2 population groups.

Quantitative relationships: Effects on Ecosystems or 

Public Welfare

3 Key questions for understanding the quantitative relationships between exposure (or 

4 concentration or deposition) to a pollutant and risk to ecosystems or the public welfare 

5 include: 

6  What elements of the ecosystem (e.g., types, regions, taxonomic groups, 

7 populations, functions, etc.) appear to be affected, or are more sensitive to 

8 effects? Are there differences between locations or materials in welfare effects 

9 responses, such as impaired visibility or materials damage?

10  Under what exposure conditions (amount deposited or concentration, duration 

11 and pattern) are effects seen? 

12  What is the shape of the concentration-response or exposure-response 

13 relationship? 

14 Evaluations of causality generally consider the probability of quantitative changes in 

15 ecological and welfare effects in response to exposure. A challenge to the quantification 

16 of exposure-response relationships for ecological effects is the great regional and local 

17 variability in ecosystems. Thus, exposure-response relationships are often determined for 

18 a specific ecological system and scale, rather than at the national or even regional scale. 

19 Quantitative relationships therefore are available site by site and may differ greatly 

20 between ecosystems. 

Concepts in Evaluating Adversity of Health Effects

21 In evaluating health evidence, a number of factors can be considered in delineating 

22 between adverse and nonadverse health effects resulting from exposure to air pollution. 

23 Some health outcomes, such as hospitalization for respiratory or cardiovascular diseases, 

24 are clearly considered adverse. It is more difficult to determine the extent of change that 

25 constitutes adversity in more subtle health measures. These include a wide variety of 

26 responses, such as alterations in markers of inflammation or oxidative stress, changes in 

27 pulmonary function or heart rate variability, or alterations in neurocognitive function 

28 measures. The challenge is determining the magnitude of change in these measures when 

29 there is no clear point at which a change become adverse; for example, what percentage 

30 change in a lung function measure represents an adverse effect. What constitutes an 
Draft – Do Not Cite or Quote lxvi September 2011

1 adverse health effect may vary between populations. Some changes that may not be 

2 considered adverse in healthy individuals would be potentially adverse in more 

3 susceptible individuals. 

4 For example, the extent to which changes in lung function are adverse has been discussed 

5 by the American Thoracic Society (ATS) in an official statement titled What Constitutes 

6 an Adverse Health Effect of Air Pollution? (2000b). An air pollution-induced shift in the 

7 population distribution of a given risk factor for a health outcome was viewed as adverse, 

8 even though it may not increase the risk of any one individual to an unacceptable level. 

9 For example, a population of asthmatics could have a distribution of lung function such 

10 that no identifiable individual has a level associated with significant impairment. 

11 Exposure to air pollution could shift the distribution such that no identifiable individual 

12 experiences clinically relevant effects. This shift toward decreased lung function, 

13 however, would be considered adverse because individuals within the population would 

14 have diminished reserve function and therefore would be at increased risk to further 

15 environmental insult. The committee also observed that elevations of biomarkers, such as 

16 cell number and types, cytokines and reactive oxygen species, may signalrisk for ongoing 

17 injury and clinical effects or may simply indicate transient responses that can provide 

18 insights into mechanisms of injury, thus illustrating the lack of clear boundaries that 

19 separate adverse from nonadverse effects.

20 It is important to recognize that the more subtle health outcomes may be connected 

21 mechanistically to health events that are clearly adverse. For example, air pollution may 

22 affect markers of transient myocardial ischemia such as ST-segment abnormalities and 

23 onset of exertional angina. These effects may not be apparent to the individual, yet may 

24 still increase the risk of a number of cardiac events, including myocardial infarction and 

25 sudden death. Thus, small changes in physiological measures may not appear to be 

26 clearly adverse when considered alone, but contribute to a coherent and biologically 

27 plausible group of related health outcomes, including responses that are very clearly 

28 adverse.

Concepts in Evaluating Adversity of Ecological Effects

29 Adversity of ecological effects can be understood in terms ranging in scale from the 

30 cellular level to the individual organism and to the population, community and ecosystem 

31 levels. In the context of ecology, a population is a group of individuals of the same 

32 species, and a community is an assemblage of populations of different species interacting 

33 with one another that inhabit an area. An ecosystem is the interactive system formed from 

34 all living organisms and their abiotic (physical and chemical) environment within a given 

35 area (IPCC, 2007a). The boundaries of what could be called an ecosystem are somewhat 
Draft – Do Not Cite or Quote lxvii September 2011

1 arbitrary, depending on the focus of interest or study. Thus, the extent of an ecosystem 

2 may range from very small spatial scales to, ultimately, the entire Earth (IPCC, 2007a).

3 Effects on an individual organism are generally not considered to be adverse, however if 

4 effects occur to enough individuals within a population, communities and ecosystems 

5 may be disrupted. Changes to populations, communities and ecosystems can in turn result 

6 in an alteration of ecosystem processes. Ecosystem processes are defined as the metabolic 

7 functions of ecosystems including energy flow, elemental cycling, and the production, 

8 consumption and decomposition of organic matter (U.S. EPA, 2002). Growth, 

9 reproduction, and mortality are species-level endpoints that can be clearly linked to 

10 community and ecosystem effects and are considered to be adverse when negatively 

11 affected. Other endpoints such as changes in behavior and physiological stress can 

12 decrease ecological fitness of an organism, but are harder to link unequivocally to effects 

13 at the population, community and ecosystem level. The degree to which pollutant 

14 exposure is considered adverse may also depend on the location and its intended use (i.e. 

15 city park, commercial cropland). Support for consideration of adversity beyond the 

16 species level by making explicit the linkages between stress-related effects at the species 

17 and effects at the ecosystem level is found in A Framework for Assessing and Reporting 

18 on Ecological Condition: an SAB report (U.S. EPA, 2002). Additionally, the National 

19 Acid Precipitation Assessment Program (NAPAP) uses the following working definition 

20 of adverse ecological effects in the preparation of reports to Congress mandated by the 

21 Clean Air Act: “any injury (i.e. loss of chemical or physical quality or viability) to any 

22 ecological or ecosystem component, up to and including at the regional level, over both 

23 long and short terms.”

24 On a broader scale, ecosystem services may provide indicators for ecological impacts. 

25 Ecosystem services are the benefits that people obtain from ecosystems (UNEP, 2003). 

26 According to the Millennium Ecosystem Assessment, ecosystem services include: 

27 “provisioning services such as food and water; regulating services such as regulation of 

28 floods, drought, land degradation, and disease; supporting services such as soil formation 

29 and nutrient cycling; and cultural services such as recreational, spiritual, religious and 

30 other nonmaterial benefits.” For example, a more subtle ecological effect of pollution 

31 exposure may result in a clearly adverse impact on ecosystem services if it results in a 

32 population decline in a species that is recreationally or culturally important.
Draft – Do Not Cite or Quote lxviii September 2011

PREFACE

Legislative Requirements for the NAAQS Review

1 Two sections of the Clean Air Act (CAA) govern the establishment and revision of 

2 the National Ambient Air Quality Standards (NAAQS). Section 108 (42 USC §7408) 

3 directs the Administrator to identify and list certain air pollutants and then to issue air 

4 quality criteria for those pollutants. The Administrator is to list those air pollutants 

5 that in her “judgement; cause or contribute to air pollution which may reasonably be 

6 anticipated to endanger public health or welfare” and whose “presence...in the 

7 ambient air results from numerous or diverse mobile or stationary sources” (CAA, 

8 1990a). Air quality criteria are intended to “accurately reflect the latest scientific 

9 knowledge useful in indicating the kind and extent of identifiable effects on public 

10 health or welfare which may be expected from the presence of [a] pollutant in 

11 ambient air . . . [42 USC §7408(b)].

12 Section 109 (CAA, 1990b) directs the Administrator to propose and promulgate 

13 “primary” and “secondary” NAAQS for pollutants for which air quality criteria have 

14 been issued. Section 109(b)(1) defines a primary standard as one “the attainment and 

15 maintenance of which in the judgment of the Administrator, based on such criteria 

16 and allowing an adequate margin of safety, are requisite to protect the public 

health.”5 17 A secondary standard, as defined in section 109(b)(2), must “specify a level 

18 of air quality the attainment and maintenance of which, in the judgment of the 

19 Administrator, based on such criteria, is required to protect the public welfare from 

20 any known or anticipated adverse effects associated with the presence of [the] 

pollutant in the ambient air.”6 21

22 The requirement that primary standards include an adequate margin of safety was 

23 intended to address uncertainties associated with inconclusive scientific and technical 

24 information available at the time of standard setting. It was also intended to provide a 

25 reasonable degree of protection against hazards that research has not yet identified. 

26 See Lead Industries Association v. EPA, 647 F.2d 1130, 1154 (D.C. Cir 1980), cert. 

27 denied, 449 U.S. 1042 (1980); American Petroleum Institute v. Costle, 665 F.2d 

28 1176, 1186 (D.C. Cir. 1981), cert. denied, 455 U.S. 1034 (1982). Both kinds of 

29 uncertainties are components of the risk associated with pollution at levels below 

 5 The legislative history of section 109 indicates that a primary standard is to be set at “the maximum permissible ambient air level 

. . . which will protect the health of any [sensitive] group of the population,” and that for this purpose “reference should be made to a 

representative sample of persons comprising the sensitive group rather than to a single person in such a group” [S. Rep. No. 91-

1196, 91st Cong., 2d Sess. 10 (1970)].

6 Welfare effects as defined in section 302(h) include, but are not limited to, “effects on soils, water, crops, vegetation, man-made 

materials, animals, wildlife, weather, visibility and climate, damage to and deterioration of property, and hazards to transportation, as 

well as effects on economic values and on personal comfort and well-being” (CAA, 2005).
Draft – Do Not Cite or Quote lxix September 2011

1 those at which human health effects can be said to occur with reasonable scientific 

2 certainty. Thus, in selecting primary standards that include an adequate margin of 

3 safety, the Administrator is seeking not only to prevent pollution levels that have 

4 been demonstrated to be harmful but also to prevent lower pollutant levels that may 

5 pose an unacceptable risk of harm, even if the risk is not precisely identified as to 

6 nature or degree.

7 In selecting a margin of safety, the EPA considers such factors as the nature and 

8 severity of the health effects involved, the size of the sensitive population(s) at risk, 

9 and the kind and degree of the uncertainties that must be addressed. The selection of 

10 any particular approach to providing an adequate margin of safety is a policy choice 

11 left specifically to the Administrator’s judgment. See Lead Industries Association v. 

12 EPA, supra, 647 F.2d at 1161-1162.

13 In setting standards that are “requisite” to protect public health and welfare, as 

14 provided in Section 109(b), EPA’s task is to establish standards that are neither more 

15 nor less stringent than necessary. In so doing, EPA may not consider the costs of 

16 implementing the standards. [See generally Whitman v. American Trucking 

17 Associations, 531 U.S. 457, 465-472, 475-76.]

18 Section 109(d)(1) requires that “not later than December 31, 1980, and at 5-year 

19 intervals thereafter, the Administrator shall complete a thorough review of the criteria 

20 published under section 108 and the national ambient air quality standards ... and 

21 shall make such revisions in such criteria and standards and promulgate such new 

22 standards as may be appropriate...” Section 109(d)(2) requires that an independent 

23 scientific review committee “shall complete a review of the criteria ... and the 

24 national primary and secondary ambient air quality standards ... and shall 

25 recommend to the Administrator any new . . . standards and revisions of existing 

26 criteria and standards as may be appropriate ...” Since the early 1980s, this 

27 independent review function has been performed by CASAC.

History of the NAAQS for Ozone

28 Tropospheric (ground-level) O3 is the indicator for the mix of photochemical 

29 oxidants (e.g., peroxyacetyl nitrate, hydrogen peroxide) formed from biogenic and 

30 anthropogenic precursor emissions. Naturally occurring O3 in the troposphere can 

31 result from biogenic organic precursors reacting with naturally occurring nitrogen 

32 oxides (NOX) and by stratospheric O3 intrusion into the troposphere. Anthropogenic 

33 precursors of O3, especially NOX, and volatile organic compounds (VOCs), originate 

34 from a wide variety of stationary and mobile sources. Ambient O3 concentrations 
Draft – Do Not Cite or Quote lxx September 2011

1 produced by these emissions are directly affected by temperature, solar radiation, 

2 wind speed, and other meteorological factors.

3 NAAQS are comprised of four basic elements: indicator, averaging time, level, and 

4 form. The indicator defines the pollutant to be measured in the ambient air for the 

5 purpose of determining compliance with the standard. The averaging time defines the 

6 time period over which air quality measurements are to be obtained and averaged or 

7 cumulated, considering evidence of effects associated with various time periods of 

8 exposure. The level of a standard defines the air quality concentration used (i.e., an 

9 ambient concentration of the indicator pollutant) in determining whether the standard 

10 is achieved. The form of the standard specifies the air quality measurements that are 

11 to be used for compliance purposes (e.g., the annual fourth-highest daily maximum 

12 8-hour concentration, averaged over 3 years), and whether the statistic is to be 

13 averaged across multiple years. These four elements taken together determine the 

14 degree of public health and welfare protection afforded by the NAAQS.

Table III Summary of primary and secondary NAAQS promulgated for ozone

during the period 1971-2008

Final Rule Indicator Avg 

Time

Level 

(ppm) Form

1971 (36 FR 8186) Total photochemical 

oxidants 1-h 0.08 Not to be exceeded more than 1 hour per year

1979 (44 FR 8202) O3 1-h 0.12

Attainment is defined when the expected number of days per 

calendar year, with maximum hourly average concentration greater 

than 0.12 ppm, is ≤ 1

1993 (58 FR 13008) EPA decided that revisions to the standards were not warranted at the time.

1997 (62 FR 38856) O3 8-h 0.08 Annual fourth-highest daily maximum 8-h concentration averaged 

over 3 years

2008 (73 FR 16483) O3 8-h 0.075 Form of the standards remained unchanged relative to the 1997 

standard

15 Table III summarizes the O3 NAAQS that have been promulgated to date. In each 

16 review, the secondary standard has been set to be identical to the primary standard. 

17 These reviews are briefly described below.

18 EPA first established primary and secondary NAAQS for photochemical oxidants in 

19 1971 . Both primary and secondary standards were set at a level of 0.08 parts per 

20 million (ppm), 1-h avg, total photochemical oxidants, not to be exceeded more than 

21 1 hour per year. The standards were based on scientific information contained in the 

22 1970 AQCD.

23 In 1977, EPA announced the first periodic review of the 1970 AQCD in accordance 

24 with Section 109(d)(1) of the Clean Air Act. In 1978, EPA published an AQCD. 
Draft – Do Not Cite or Quote lxxi September 2011

1 Based on the 1978 AQCD, EPA published proposed revisions to the original NAAQS 

2 in 1978 (U.S. EPA, 1978b) and final revisions in 1979 (U.S. EPA, 1979a). The level 

3 of the primary and secondary standards was revised from 0.08 to 0.12 ppm; the 

4 indicator was revised from photochemical oxidants to O3; and the form of the 

5 standards was revised from a deterministic to a statistical form, which defined 

6 attainment of the standards as occurring when the expected number of days per 

7 calendar year with maximum hourly average concentration greater than 0.12 ppm is 

8 equal to or less than one. 

9 In 1982, EPA announced plans to revise the 1978 AQCD (U.S. EPA, 1978a). In 1983, 

10 EPA announced that the second periodic review of the primary and secondary 

11 standards for O3 had been initiated (U.S. EPA, 1983). EPA subsequently published 

12 the 1986 O3 AQCD (U.S. EPA, 1986) and 1989 Staff Paper (U.S. EPA, 1989). 

13 Following publication of the 1986 O3 AQCD, a number of scientific abstracts and 

14 articles were published that appeared to be of sufficient importance concerning 

15 potential health and welfare effects of O3 to warrant preparation of a Supplement to 

16 the 1986 O3 AQCD (Costa et al., 1992). Under the terms of a court order, on August 

17 10, 1992, EPA published a proposed decision (U.S. EPA, 1992) stating that revisions 

18 to the existing primary and secondary standards were not appropriate at the time 

19 (U.S. EPA, 1992). This notice explained that the proposed decision would complete 

20 EPA’s review of information on health and welfare effects of O3 assembled over a 

21 7-year period and contained in the 1986 O3 AQCD (U.S. EPA, 1986) and its 

22 Supplement to the 1986 O3 AQCD (Costa et al., 1992). The proposal also announced 

23 EPA’s intention to proceed as rapidly as possible with the next review of the air 

24 quality criteria and standards for O3 in light of emerging evidence of health effects 

25 related to 6- to 8-hour O3 exposures. On March 9, 1993, EPA concluded the review 

26 by deciding that revisions to the standards were not warranted at that time (U.S. EPA, 

27 1993).

28 In August 1992, EPA announced plans to initiate the third periodic review of the air 

29 quality criteria and O3 NAAQS (U.S. EPA, 1992). On the basis of the scientific 

30 evidence contained in the 1996 O3 AQCD and the 1996 Staff Paper (U.S. EPA, 

31 1996e), and related technical support documents, linking exposures to ambient O3 to 

32 adverse health and welfare effects at levels allowed by the then existing standards, 

33 EPA proposed to revise the primary and secondary O3 standards on December 13, 

34 1996 (U.S. EPA, 1996d). The EPA proposed to replace the then existing 1-hour 

35 primary and secondary standards with 8-h avg O3 standards set at a level of 0.08 ppm 

36 (equivalent to 0.084 ppm using standard rounding conventions). The EPA also 

37 proposed, in the alternative, to establish a new distinct secondary standard using a 

38 biologically based cumulative seasonal form. The EPA completed the review on July 
Draft – Do Not Cite or Quote lxxii September 2011

1 18, 1997 by setting the primary standard at a level of 0.08 ppm, based on the annual 

2 fourth-highest daily maximum 8-h avg concentration, averaged over 3 years, and 

3 setting the secondary standard identical to the revised primary standard (U.S. EPA, 

4 1997).

5 On May 14, 1999, in response to challenges to EPA’s 1997 decision by industry and 

6 others, the U.S. Court of Appeals for the District of Columbia Circuit (D.C. Cir.) 

7 remanded the O3 NAAQS to EPA, finding that Section 109 of the CAA, as 

8 interpreted by EPA, effected an unconstitutional delegation of legislative authority. In 

9 addition, the D.C. Cir. directed that, in responding to the remand, EPA should 

10 consider the potential beneficial health effects of O3 pollution in shielding the public 

11 from the effects of solar ultraviolet (UV) radiation, as well as adverse health effects. 

12 On January 27, 2000, EPA petitioned the U.S. Supreme Court for certiorari on the 

13 constitutional issue (and two other issues) but did not request review of the D.C. Cir., 

14 ruling regarding the potential beneficial health effects of O3. On February 27, 2001, 

15 the U.S. Supreme Court unanimously reversed the judgment of the D.C. Cir. on the 

16 constitutional issue, holding that Section 109 of the CAA does not delegate 

17 legislative power to the EPA in contravention of the Constitution, and remanded the 

18 case to the D.C. Cir. to consider challenges to the O3 NAAQS that had not been 

19 addressed by that Court’s earlier decisions. On March 26, 2002, the D.C. Cir. issued 

20 its final decision, finding the 1997 O3 NAAQS to be “neither arbitrary nor 

21 capricious,” and denied the remaining petitions for review. On November 14, 2001, 

22 in response to the D.C. Cir. remand to consider the potential beneficial health effects 

23 of O3 pollution in shielding the public from effects of solar (UV) radiation, EPA 

24 proposed to leave the 1997 8-h O3 NAAQS unchanged (U.S. EPA, 2001). After 

25 considering public comment on the proposed decision, EPA published its final 

26 response to this remand on January 6, 2003, reaffirming the 8-h O3 NAAQS set in 

27 1997 (U.S. EPA, 2003). On April 30, 2004, EPA announced the decision to make the 

28 1-h O3 NAAQS no longer applicable to areas 1 year after the effective date of the 

29 designation of those areas for the 8-h NAAQS (2004). For most areas, the date that 

30 the 1-h NAAQS no longer applied was June 15, 2005.

31 EPA initiated the next periodic review if the air quality criteria and O3 standards in 

32 September 2000 with a call for information (U.S. EPA, 2000). The schedule for 

33 completion of that rulemaking later became governed by a consent decree resolving a 

34 lawsuit filed in March 2003 by a group of plaintiffs representing national 

35 environmental and public health organizations. Based on the 2006 O3 AQCD (U.S. 

36 EPA, 2006b) published in March 2006, the Staff Paper (U.S. EPA, 2007b) and related 

37 technical support documents, the proposed decision was published in the Federal 

38 Register on July 11, 2007 (U.S. EPA, 2007a). The EPA proposed to revise the level of 
Draft – Do Not Cite or Quote lxxiii September 2011

1 the primary standard to a level within the range of 0.075 to 0.070 ppm. Two options 

2 were proposed for the secondary standard: (1) replacing the current standard with a 

3 cumulative, seasonal standard, expressed as an index of the annual sum of weighted 

4 hourly concentrations cumulated over 12 daylight hours during the consecutive 

5 3-month period within the O3 season with the maximum index value, set at a level 

6 within the range of 7 to 21 ppm-h; and (2) setting the secondary standard identical to 

7 the revised primary standard. The EPA completed the rulemaking with publication of 

8 a final decision on March 27, 2008 (U.S. EPA, 2008e), revising the level of the 

9 8-hour primary O3 standard from 0.08 ppm to 0.075 ppm and revising the secondary

10 standard to be identical to the primary standard. 

References

ATS. (American Thoracic Society). (2000b). What constitutes an adverse health effect of air pollution? This 

official statement of the American Thoracic Society was adopted by the ATS Board of Directors, July 

1999. Am J Respir Crit Care Med 161: 665-673. 

Bell, ML; Dominici, F; Samet, JM. (2005). A meta-analysis of time-series studies of ozone and mortality with 

comparison to the national morbidity, mortality, and air pollution study. Epidemiology 16: 436-445. 

CAA. (Air quality criteria and control techniques, Section 108 of the Clean Air Act). 42. § 7408, (1990a).

CAA. (National primary and secondary ambient air quality standards, Section 109 of the Clean Air Act). 42. § 

7409, (1990b).

CAA. (Definitions, Section 302 of the Clean Air Act). 42. § 7602, (2005).

Costa, DL; Folinsbee, LJ; Raub, JA; Tilton, B; Tingey, DT. (1992). Summary of selected new information on 

effects of ozone on health and vegetation: Supplement to 1986 air quality criteria for ozone and other 

photochemical oxidants. (EPA/600/8-88/105F). Research Triangle Park, NC: U.S. Environmental 

Protection Agency.

Fox, GA. (1991). Practical causal inference for ecoepidemiologists. J Toxicol Environ Health A 33: 359-373. 

http://dx.doi.org/10.1080/15287399109531535.

Gee, GC; Payne-Sturges, DC. (2004). Environmental health disparities: A framework integrating psychosocial 

and environmental concepts. Environ Health Perspect 112: 1645-1653. 

http://dx.doi.org/10.1289/ehp.7074.

HEW. (U.S. Department of Health, Education and Welfare). (1964). Smoking and health: Report of the advisory 

committee to the surgeon general of the public health service. Washington, DC: U.S. Department of 

Health, Education, and Welfare.

HHS. (U.S. Department of Health and Human Services, Office of the Surgeon General). (2004). The health 

consequences of smoking: A report of the Surgeon General. Washington, DC: U.S. Department of 

Health and Human Services. http://www.surgeongeneral.gov/library/smokingconsequences/.

Hill, AB. (1965). The environment and disease: Association or causation? Proc R Soc Med 58: 295-300. 

IARC. (International Agency for Research on Cancer). (2006). Preamble to the IARC monographs. Lyon, 

France. http://monographs.iarc.fr/ENG/Preamble/.

Ioannidis, JPA. (2008). Why most discovered true associations are inflated. Epidemiology 19: 640-648. 

http://dx.doi.org/10.1097/EDE.0b013e31818131e7.

IPCC. (Intergovernmental Panel on Climate Change). (2007a). Summary for policymakers. In: Impacts, 

Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the 

Intergovernmental Panel on Climate Change. In Climate Change 2007. Cambridge, UK: Cambridge 

University Press.

Rothman, KJ; Greenland, S. (1998). Modern epidemiology. In (2nd ed.). Philadelphia, PA: Lippincott, Williams, 

& Wilkins.
Draft – Do Not Cite or Quote lxxiv September 2011

Samet, JM; Bodurow, CC. (2008). Improving the presumptive disability decision-making process for veterans. In 

JM Samet; CC Bodurow (Eds.). Washington, DC: National Academies Press.

U.S. EPA. (U.S. Environmental Protection Agency). (1971). National primary and secondary ambient air quality 

standards. Fed Reg 36: 8186-8201. 

U.S. EPA. (U.S. Environmental Protection Agency). (1978a). Air quality criteria for ozone and other 

photochemical oxidants. (EPA/600/8-78/004). Washington, DC.

U.S. EPA. (U.S. Environmental Protection Agency). (1978b). Photochemical oxidants: Proposed revisions to the 

national ambient air quality standards. Fed Reg 43: 26962-26971. 

U.S. EPA. (U.S. Environmental Protection Agency). (1979a). National primary and secondary ambient air quality 

standards: Revisions to the national ambient air quality standards for photochemical oxidants. Fed Reg 

44: 8202-8237. 

U.S. EPA. (U.S. Environmental Protection Agency). (1982). Air quality criteria document for ozone and other 

photochemical oxidants. Fed Reg 47: 11561. 

U.S. EPA. (U.S. Environmental Protection Agency). (1983). Review of the national ambient air quality standards 

for ozone. Fed Reg 48: 38009. 

U.S. EPA. (U.S. Environmental Protection Agency). (1986). Air quality criteria for ozone and other 

photochemical oxidants. (EPA-600/8-84-020aF - EPA-600/8-84-020eF). Research Triangle Park, NC.

U.S. EPA. (U.S. Environmental Protection Agency). (1989). Review of the national ambient air quality standards 

for ozone: Assessment of scientific and technical information: OAQPS staff report. (EPA/450/2-92-001). 

Research Triangle Park, NC. http://nepis.epa.gov/Exe/ZyPURL.cgi?Dockey=2000LOW6.txt.

U.S. EPA. (U.S. Environmental Protection Agency). (1992). National ambient air quality standards for ozone; 

Proposed decision. Fed Reg 57: 35542-35557. 

U.S. EPA. (U.S. Environmental Protection Agency). (1993). National ambient air quality standards for ozone -

Final decision. Fed Reg 58: 13008-13019. 

U.S. EPA. (U.S. Environmental Protection Agency). (1996d). National ambient air quality standards for ozone: 

Proposed decision. Fed Reg 61: 65716-65750. 

U.S. EPA. (U.S. Environmental Protection Agency). (1996e). Review of national ambient air quality standards 

for ozone: Assessment of scientific and technical information: OAQPS staff paper. (EPA/452/R-96/007). 

Research Triangle Park, NC.

U.S. EPA. (U.S. Environmental Protection Agency). (1997). National ambient air quality standards for ozone; 

final rule. Fed Reg 62: 38856-38896. 

U.S. EPA. (U.S. Environmental Protection Agency). (1998). Guidelines for ecological risk assessment. 

(EPA/630/R-95/002F). Washington, DC. http://www.epa.gov/raf/publications/guidelines-ecological-riskassessment.htm.

U.S. EPA. (U.S. Environmental Protection Agency). (2000). Air quality criteria for ozone and related 

photochemical oxidants. Fed Reg 65: 57810. 

U.S. EPA. (U.S. Environmental Protection Agency). (2001). National ambient air quality standards for ozone: 

Proposed response to remand. Fed Reg 66: 57268-57292. 

U.S. EPA. (U.S. Environmental Protection Agency). (2002). A framework for assessing and reporting on 

ecological condition: An SAB report. Washington, DC.

U.S. EPA. (U.S. Environmental Protection Agency). (2003). National ambient air quality standards for ozone: 

Final response to remand. Fed Reg 68: 614-645. 

U.S. EPA. (U.S. Environmental Protection Agency). (2004). Final rule to implement the 8-hour ozone national 

ambient air quality standard-phase 1. Fed Reg 69: 23951-24000. 

U.S. EPA. (U.S. Environmental Protection Agency). (2005). Guidelines for carcinogen risk assessment. 

(EPA/630/P-03/001F). Washington, DC. http://www.epa.gov/cancerguidelines/.

U.S. EPA. (U.S. Environmental Protection Agency). (2006b). Air quality criteria for ozone and related 

photochemical oxidants. (EPA/600/R-05/004AF). Research Triangle Park, NC: U.S. Environmental 

Protection Agency, Office of Research and Development. 

http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=149923.

U.S. EPA. (U.S. Environmental Protection Agency). (2007a). National ambient air quality standards for ozone. 

Fed Reg 72: 37818-37919. 
Draft – Do Not Cite or Quote lxxv September 2011

U.S. EPA. (U.S. Environmental Protection Agency). (2007b). Review of the national ambient air quality 

standards for ozone: Policy assessment of scientific and technical information: OAQPS staff paper. 

(EPA/452/R-07/003). Research Triangle Park, NC.

U.S. EPA. (U.S. Environmental Protection Agency). (2008e). National ambient air quality standards for ozone. 

Fed Reg 73: 16436-16514. 

U.S. EPA. (U.S. Environmental Protection Agency). (2009d). Integrated science assessment for particulate 

matter. (EPA/600/R-08/139F). Research Triangle Park, NC. 

http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=216546.

UNEP. (United Nations Environment Programme). (2003). Millennium Ecosystem Assessment: Ecosystems and 

human well-being: A framework for assessment. Washington, DC: Island Press.