Draft – Do Not Cite or Quote liv June 2012

PREAMBLE

Process of ISA Development

1 This preamble outlines the general process for developing an Integrated Science 

2 Assessment (ISA) including the framework for evaluating weight of evidence and 

3 drawing scientific conclusions and causal judgments. The ISA provides a concise review, 

4 synthesis, and evaluation of the most policy-relevant science to serve as a scientific 

5 foundation for the review of the National Ambient Air Quality Standards (NAAQS). The 

6 general process for NAAQS reviews is described at 

7 http://www.epa.gov/ttn/naaqs/review.html. Figure I depicts the general NAAQS review 

8 process and information for individual NAAQS reviews is available at 

9 www.epa.gov/ttn/naaqs. This preamble is a general discussion of the basic steps and 

10 criteria used in developing an ISA; for each ISA, specific details and considerations are 

11 included in the introductory section for that assessment.

12 The fundamental process for developing an ISA includes:

13  literature searches;

14  study selection;

15  evaluation and integration of the evidence; and 

16  development of scientific conclusions and causal judgments.

17 An initial step in this process is publication of a call for information in the Federal 

18 Register that invites the public to provide information relevant to the assessment, such as 

new publications on health or welfare1 19 effects of the pollutant, or from atmospheric and 

20 exposure sciences fields. EPA maintains an ongoing literature search process for 

21 identification of relevant scientific studies published since the last review of the NAAQS. 

22 Search strategies are designed for pollutants and scientific disciplines and iteratively 

23 modified to optimize identification of pertinent publications. Papers are identified for 

24 inclusion in several additional ways: specialized searches on specific topics; independent 

25 review of tables of contents for journals in which relevant papers may be published; 

26 independent identification of relevant literature by expert scientists; review of citations in 

27 previous assessments and identification by the public and CASAC during the external 

28 review process. This literature search and study selection process is depicted in Figure II. 

29 Publications considered for inclusion in the ISA are added to the Health and 

 1 Welfare effects as defined in Clean Air Act section 302(h) [42 U.S.C. 7602(h)] include, but are not limited to, “effects on soils, 

water, crops, vegetation, man-made materials, animals, wildlife, weather, visibility and climate, damage to and deterioration of 

property, and hazards to transportation, as well as effects on economic values and on personal comfort and well-being.”
Draft – Do Not Cite or Quote lv June 2012

1 Environmental Research Online (HERO) database developed by EPA 

2 (http://hero.epa.gov/); the references in the ISA include a hyperlink to the database. 

3 Studies that have undergone scientific peer review and have been published or accepted 

4 for publication and reports that have undergone review are considered for inclusion in the 

5 ISA. Analyses conducted by EPA using publicly available data are also considered for 

6 inclusion in the ISA. All relevant epidemiologic, controlled human exposure, 

7 toxicological, and ecological and welfare effects studies published since the last review 

8 are considered, including those related to exposure-response relationships, mode(s) of 

9 action (MOA), and potentially at-risk populations and lifestages. Studies on atmospheric 

10 chemistry, environmental fate and transport, dosimetry, toxicokinetics and exposure are 

11 also considered for inclusion in the document, as well as analyses of air quality and 

12 emissions data. References that were considered for inclusion in a specific ISA can be 

13 found using the HERO website (http://hero.epa.gov).

Figure I Illustration of the key steps in the process of the review of National 

Ambient Air Quality Standards.

Workshop 

on sciencepolicy issue

EPA final 

decision on 

standards 

Interagency 

review

CASAC review and 

public comment

Peer-reviewed 

scientific studies

Agency decision 

making and draft 

final notice

Public hearings 

and comments 

on proposal

Integrated Review 

Plan: timeline and 

key policy-relevant 

issues and scientific 

questions 

Integrated Science Assessment (ISA): 

concise evaluation and synthesis of most 

policy-relevant studies

Clean Air Scientific Advisory 

Committee (CASAC) review and 

public comment

Agency decision 

making and draft 

proposal notice

Interagency 

review

Policy Assessment:

staff analysis of 

policy options based 

on integration and 

interpretation of 

information in the ISA 

and REA

Risk/Exposure Assessment (REA):

concise quantitative assessment 

focused on key results, observations, 

and uncertainties

EPA 

proposed 

decision on 

standards

National Ambient Air Quality Standard Review Process
Draft – Do Not Cite or Quote lvi June 2012

Figure II Illustration of processes for literature search and study selection 

used for development of ISAs.

1 Each ISA builds upon the conclusions of previous assessments for the pollutant under 

2 review. EPA focuses on peer reviewed literature published following the completion of 

3 the previous review and on any new interpretations of previous literature, integrating the

4 results of recent scientific studies with previous findings. Important earlier studies may 

5 be discussed in detail to reinforce key concepts and conclusions or for reinterpretation in 

6 light of newer data. Earlier studies also are the primary focus in some areas of the 

7 document where research efforts have subsided, or if these earlier studies remain the 

8 definitive works available in the literature. 

9 Selection of studies for inclusion in the ISA is based on the general scientific quality of 

10 the study, and consideration of the extent to which the study is informative and policy11 relevant. Policy relevant and informative studies include those that provide a basis for or 

12 describe the relationship between the criteria pollutant and effects, including studies that 

13 offer innovation in method or design and studies that reduce uncertainty on critical issues, 

14 such as analyses of confounding or effect modification by copollutants or other variables, 

15 analyses of concentration-response or dose-response relationships, or analyses related to 

16 time between exposure and response. Emphasis is placed on studies that examine effects 

17 associated with pollutant concentrations relevant to current population and ecosystem 

Screened

Considered

Included

Title

No

Abstract Full text

No No

Recommendations 

during Peer Review

Literature 

Search 

Strategies

Yes Yes Yes

Citations from 

Past Assessments

Criteria for study evaluation 

include:

Are the study populations, subjects, 

or animal models adequately 

selected, and are they sufficiently 

well defined to allow for meaningful 

comparisons between study or 

exposure groups? 

Are the statistical analyses 

appropriate, properly performed, and 

properly interpreted? Are likely 

covariates adequately controlled or 

taken into account in the study 

design and statistical analysis? 

Are the air quality data, exposure, 

or dose metrics of adequate quality 

and sufficiently representative of 

information regarding ambient 

conditions?

Are the health, ecological or welfare 

effect measurements meaningful, 

valid and reliable?

Do the analytical methods provide 

adequate sensitivity and precision to 

support conclusions?
Draft – Do Not Cite or Quote lvii June 2012

1 exposures, and particularly those pertaining to concentrations currently found in ambient 

2 air. Other studies are included if they contain unique data, such as a previously 

3 unreported effect or MOA for an observed effect, or examine multiple concentrations to 

4 elucidate exposure-response relationships. In general, in assessing the scientific quality 

5 and relevance of health and welfare effects studies, the following considerations have 

6 been taken into account when selecting studies for inclusion in the ISA.

7  Are the study populations, subjects, or animal models adequately selected, and 

8 are they sufficiently well defined to allow for meaningful comparisons 

9 between study or exposure groups? 

10  Are the statistical analyses appropriate, properly performed, and properly 

11 interpreted? Are likely covariates adequately controlled or taken into account 

12 in the study design and statistical analysis? 

13  Are the air quality data, exposure, or dose metrics of adequate quality and 

14 sufficiently representative of information regarding ambient conditions?

15  Are the health, ecological or welfare effect measurements meaningful, valid 

16 and reliable?

17  Do the analytical methods provide adequate sensitivity and precision to 

18 support conclusions?

19 Considerations specific to particular disciplines include the following. In selecting 

20 epidemiologic studies, EPA considers whether a given study: (1) presents information on 

21 associations with short- or long-term pollutant exposures at or near conditions relevant to 

22 ambient exposures; (2) addresses potential confounding by other pollutants; (3) assesses 

23 potential effect modifiers; (4) evaluates health endpoints and populations not previously 

24 extensively researched; and (5) evaluates important methodological issues related to 

25 interpretation of the health evidence (e.g., lag or time period between exposure and 

26 effects, model specifications, thresholds, mortality displacement). 

27 Considerations for the selection of research evaluating controlled human exposure or 

28 animal toxicological studies includes a focus on studies conducted using relevant 

29 pollutant exposures. For both types of studies, relevant pollutant exposures are 

30 considered to be those generally within one or two orders of magnitude of ambient 

31 concentrations. Studies in which higher doses were used may also be considered if they 

32 provide information relevant to understanding MOA or mechanisms, as noted below.

33 Evaluation of controlled human exposure studies focuses on those that approximated 

34 expected human exposure conditions in terms of concentration and duration. Studies

35 should include control exposures to filtered air, as appropriate. In the selection of 

36 controlled human exposure studies, emphasis is placed on studies that: (1) investigate 
Draft – Do Not Cite or Quote lviii June 2012

1 potentially at-risk populations and lifestages such as people with asthma or 

2 cardiovascular diseases, children or older adults; (2) address issues such as concentration3 response or time-course of responses; and (3) have sufficient statistical power to assess 

4 findings. 

5 Review of the animal toxicological evidence focuses on studies that approximate 

6 expected human dose conditions, which vary depending on the dosimetry, toxicokinetics 

7 and biological sensitivity of the particular laboratory animal species or strains studied. 

8 Emphasis is placed on studies that: (1) investigate animal models of disease that can 

9 provide information on populations potentially at increased risk of effects; (2) address 

10 issues such as concentration-response or time-course of responses; and (3) have sufficient 

11 statistical power to assess findings. Due to resource constraints on exposure duration and 

12 numbers of animals tested, animal studies typically utilize high-concentration exposures 

13 to acquire data relating to mechanisms and assure a measurable response. Emphasis is 

14 placed on studies using doses or concentrations generally within 1-2 orders of magnitude 

15 of current levels. Studies with higher concentration exposures or doses are considered to 

16 the extent that they provide useful information to inform understanding of interspecies 

17 differences and potential differences between healthy and potentially at-risk human 

18 populations. Results from in vitro studies may also be included if they provide 

19 mechanistic insight or further support for results demonstrated in vivo.

20 These criteria provide benchmarks for evaluating various studies and for focusing on the 

21 policy-relevant studies in assessing the body of health, ecological and welfare effects 

22 evidence. As stated initially, the intent of the ISA is to provide a concise review, 

23 synthesis, and evaluation of the most policy-relevant science to serve as a scientific 

24 foundation for the review of the NAAQS, not extensive summaries of all health, 

25 ecological and welfare effects studies for a pollutant. Of most relevance for inclusion of 

26 studies is whether they provide useful qualitative or quantitative information on 

27 exposure-effect or exposure-response relationships for effects associated with pollutant 

28 exposures at doses or concentrations relevant to ambient conditions that can inform 

29 decisions on whether to retain or revise the standards. 

30 In developing an ISA, EPA reviews and summarizes the evidence from: studies of 

31 atmospheric sciences and exposure; the health effects evidence from toxicological, 

32 controlled human exposure and epidemiologic studies; and ecological and welfare effects

33 evidence. In the process of developing the first draft ISA, EPA may convene a public

34 workshop in which EPA and non-EPA experts review the scientific content of 

35 preliminary draft materials to ensure that the ISA is up to date and focused on the most 

36 policy-relevant findings, and to assist EPA with integration of evidence within and across 

37 disciplines. The general process for ISA development is illustrated in Figure III. 
Draft – Do Not Cite or Quote lix June 2012

1 EPA integrates the evidence from across scientific disciplines or study types and 

2 characterizes the weight of evidence for relationships between the pollutant and various 

3 outcomes. The integration of evidence on health, and ecological or welfare effects, 

4 involves collaboration between scientists from various disciplines. As an example, an 

5 evaluation of health effects evidence would include the integration of the results from 

6 epidemiologic, controlled human exposure, and toxicological studies, and application of 

7 the causal framework (described below) to draw conclusions. Using the causal 

8 framework described in the following section, EPA scientists consider aspects such as 

9 strength, consistency, coherence, and biological plausibility of the evidence, and develop 

10 causality determinations on the nature of the relationships. Causality determinations often 

11 entail an iterative process of review and evaluation of the evidence. Two drafts of the ISA 

12 are typically released for review by the CASAC and the public, and comments received 

13 on the characterization of the science as well as the implementation of the causal 

14 framework are carefully considered in revising and completing the final ISA.
Draft – Do Not Cite or Quote lx June 2012

Figure III Characterization of the general process of ISA development.

Evergreen Literature Search and

Study Selection

(See Figure II)

Integrated Science Assessment Development Process

Characterization of Evidence 

Develop initial sections or “building blocks” of scientific evidence for 

assessment: review and summarize new study results, by 

outcome/effect category and discipline, for example, toxicological 

studies of lung function. Summarize findings and conclusions from 

previous assessment. As appropriate, develop initial conclusions about 

the available evidence.

Development of Conclusions and Causal Determinations

Evaluate weight of evidence and develop judgments regarding causality for health or welfare effect categories, integrating health or welfare effects evidence with information on mode of action and exposure 

assessment. Develop conclusions regarding concentration- or dose- response relationships, potentially at-risk populations or ecosystems.

Peer Input

Review of initial draft materials for scientific 

quality and to facilitate integration. 

Draft Integrated Science Assessment released for review Clean Air Scientific Advisory Committee

review in public meeting; anticipated review of two drafts of ISA

Final Integrated Science Assessment

Evaluation, Synthesis, and Integration of Evidence 

Integrate evidence from scientific disciplines or study types – for 

example, toxicological, controlled human exposure and epidemiologic 

study findings for particular health outcome. Evaluate evidence for related groups of endpoints or outcomes to draw conclusions regarding 

health or welfare effect categories.
Draft – Do Not Cite or Quote lxi June 2012

EPA Framework for Causal Determination

1 EPA has developed a consistent and transparent basis to evaluate the causal nature of air 

2 pollution-related health or welfare effects for use in developing ISAs. The framework 

3 described below establishes uniform language concerning causality and brings more 

4 specificity to the findings. This standardized language was drawn from sources across the 

5 federal government and wider scientific community, especially the National Academy of 

6 Sciences (NAS) Institute of Medicine (IOM) document, Improving the Presumptive 

7 Disability Decision-Making Process for Veterans (2008), a comprehensive report on 

8 evaluating causality. This framework:

9  describes the kinds of scientific evidence used in establishing a general causal 

10 relationship between exposure and health effects; 

11  characterizes the evidence necessary to reach a conclusion about the existence 

12 of a causal relationship; 

13  identifies issues and approaches related to uncertainty; and

14  provides a framework for classifying and characterizing the weight of 

15 evidence in support of a general causal relationship.

16 Approaches to assessing the separate and combined lines of evidence 

17 (e.g., epidemiologic, controlled human exposure, and animal toxicological studies) have 

18 been formulated by a number of regulatory and science agencies, including the IOM of 

19 the NAS (2008), International Agency for Research on Cancer (2006), U.S. EPA (2005), 

20 and Centers for Disease Control and Prevention (2004). Causal inference criteria have 

21 also been described for ecological effects evidence (U.S. EPA, 1998a; Fox, 1991). These 

22 formalized approaches offer guidance for assessing causality. The frameworks are similar 

23 in nature, although adapted to different purposes, and have proven effective in providing 

24 a uniform structure and language for causal determinations.

Evaluating Evidence for Inferring Causation 

25 The 1964 Surgeon General’s report defined “cause” as a “significant, effectual 

26 relationship between an agent and an associated disorder or disease in the host” (HEW, 

27 1964); more generally, a cause is defined as an agent that brings about an effect or a 

28 result. An association is the statistical relationship among variables; alone, however, it is 

29 insufficient proof of a causal relationship between an exposure and a health outcome. 

30 Unlike an association, a causal claim supports the creation of counterfactual claims, that 
Draft – Do Not Cite or Quote lxii June 2012

1 is, a claim about what the world would have been like under different or changed 

2 circumstances (Samet and Bodurow, 2008). 

3 Many of the health and environmental outcomes reported in these studies have complex 

4 etiologies. Diseases such as asthma, coronary heart disease (CHD) or cancer are typically 

5 initiated by multiple agents. Outcomes depend on a variety of factors, such as age, 

6 genetic susceptibility, nutritional status, immune competence, and social factors (Samet 

7 and Bodurow, 2008; Gee and Payne-Sturges, 2004). Effects on ecosystems are often also 

8 multifactorial with a complex web of causation. Further, exposure to a combination of 

9 agents could cause synergistic or antagonistic effects. Thus, the observed risk may 

10 represent the net effect of many actions and counteractions. 

11 In estimating the causal influence of an exposure on health or environmental effects, it is 

12 recognized that scientific findings incorporate uncertainty. “Uncertainty” can be defined 

13 as having limited knowledge to exactly describe an existing state or future outcome, 

14 e.g., the lack of knowledge about the correct value for a specific measure or estimate. 

15 Uncertainty analysis may be qualitative or quantitative in nature. In many cases, the 

16 analysis is qualitative, and can include professional judgment or inferences based on 

17 analogy with similar situations. Quantitative uncertainty analysis may include use of 

18 simple measures (e.g., ranges) and analytical techniques. Quantitative uncertainty 

19 analysis might progress to more complex measures and techniques, if needed for decision 

20 support. Various approaches to evaluating uncertainty include classical statistical 

21 methods, sensitivity analysis, or probabilistic uncertainty analysis, in order of increasing 

22 complexity and data requirements. However, data may not be available for all aspects of 

23 an assessment and those data that are available may be of questionable or unknown 

24 quality. Ultimately, the assessment is based on a number of assumptions with varying 

25 degrees of uncertainty. The ISA generally evaluates uncertainties qualitatively in 

26 assessing the evidence from across studies; in some situations quantitative analysis 

27 approaches, such as meta-regression, may be used.

28 Publication bias is a source of uncertainty regarding the magnitude of health risk 

29 estimates. It is well understood that studies reporting non-null findings are more likely to 

30 be published than reports of null findings, and publication bias can also result in 

31 overestimation of effect estimate sizes (Ioannidis, 2008). For example, effect estimates 

32 from single-city epidemiologic studies have been found to be generally larger than those 

33 from multicity studies (Bell et al., 2005). 
Draft – Do Not Cite or Quote lxiii June 2012

Consideration of Evidence from Scientific Disciplines

1 Moving from association to causation involves the elimination of alternative explanations 

2 for the association. The ISA focuses on evaluation of the findings from the body of 

3 evidence, drawing upon the results of all studies determined to meet the criteria described 

4 previously. Causality determinations are based on the evaluation and synthesis of 

5 evidence from across scientific disciplines. The relative importance of different types of 

6 evidence varies by pollutant or assessment, as does the availability of different types of 

7 evidence for causality determination. Three general types of studies inform consideration 

8 of human health effects: controlled human exposure, epidemiologic and toxicological 

9 studies. Evidence on ecological or welfare effects may be drawn from a variety of 

10 experimental approaches (e.g., greenhouse, laboratory, field) and numerous disciplines 

11 (e.g., community ecology, biogeochemistry and paleontological/historical 

12 reconstructions).

13 Direct evidence of a relationship between pollutant exposures and human health effects 

14 comes from controlled human exposure studies. Controlled human exposure studies 

15 experimentally evaluate the health effects of administered exposures in human volunteers 

16 under highly controlled laboratory conditions. Also referred to as human clinical studies, 

17 these experiments allow investigators to expose subjects to known concentrations of air 

18 pollutants under carefully regulated environmental conditions and activity levels. In some 

19 instances, controlled human exposure studies can also be used to characterize 

20 concentration-response relationships at pollutant concentrations relevant to ambient 

21 conditions. Controlled human exposures are typically conducted using a randomized 

22 crossover design, with subjects exposed both to the pollutant and a clean air control. In 

23 this way, subjects serve as their own controls, effectively controlling for many potential 

24 confounders. However, controlled human exposure studies are limited by a number of 

25 factors, including small sample size and short exposure time. For example, exposure 

26 patterns relevant to understanding real-world exposures, especially long-term exposures, 

27 are generally not practical to replicate in a laboratory setting. In addition, although 

28 subjects do serve as their own controls, personal exposure to pollutants in the hours and 

29 days preceding the controlled exposures may vary significantly between and within 

30 individuals. Finally, controlled human exposure studies require investigators to adhere to 

31 stringent health criteria for subjects included in the study, and therefore the results often 

32 cannot be generalized to an entire population. Although some controlled human exposure 

33 studies have included health-compromised individuals such as those with respiratory or 

34 cardiovascular disease, these individuals must also be relatively healthy and may not 

35 represent the most sensitive individuals in the population. In addition, the study design is 

36 limited to exposures and endpoints that are not expected to result in severe health 

37 outcomes. Thus, not observing an effect in controlled human exposure studies does not 
Draft – Do Not Cite or Quote lxiv June 2012

1 necessarily mean that a causal relationship does not exist. While controlled human 

2 exposure studies provide important information on the biological plausibility of 

3 associations observed in epidemiologic studies, observed effects in these studies may 

4 underestimate the response in certain populations.

5 Epidemiologic studies provide important information on the associations between health 

6 effects and exposure of human populations to ambient air pollution. In epidemiologic or 

7 observational studies of humans, the investigator generally does not control exposures or 

8 intervene with the study population. Broadly, observational studies can describe 

9 associations between exposures and effects. These studies fall into several categories: 

10 e.g., cross-sectional, prospective cohort, panel and time-series studies. “Natural 

11 experiments” offer the opportunity to investigate changes in health related to a change in 

12 exposure, such as closure of a pollution source. 

13 In evaluating epidemiologic studies, consideration of many study design factors and 

14 issues must be taken into account to properly inform their interpretation. One key 

15 consideration is evaluation of the potential contribution of the pollutant to a health 

16 outcome when it is a component of a complex air pollutant mixture. Reported effect 

17 estimates in epidemiologic studies may reflect: independent effects on health outcomes; 

18 effects of the pollutant acting as an indicator of a copollutant or a complex ambient air 

19 pollution mixture; effects resulting from interactions between that pollutant and 

20 copollutants.

21 In the evaluation of epidemiologic evidence, one important consideration is potential 

22 confounding. Confounding is “... a confusion of effects. Specifically, the apparent effect 

23 of the exposure of interest is distorted because the effect of an extraneous factor is 

24 mistaken for or mixed with the actual exposure effect (which may be null)” (Rothman 

25 and Greenland, 1998). One approach to remove spurious associations due to possible 

26 confounders is to control for characteristics that may differ between exposed and 

27 unexposed persons; this is frequently termed “adjustment.” Scientific judgment is needed 

28 to evaluate likely sources and extent of confounding, together with consideration of how 

29 well the existing constellation of study designs, results, and analyses address this 

30 potential threat to inferential validity. A confounder is associated with both the exposure 

31 and the effect; for example, confounding can occur between correlated pollutants that are 

32 associated with the same effect. 

33 Several statistical methods are available to detect and control for potential confounders, 

34 with none of them being completely satisfactory. Multivariable regression models 

35 constitute one tool for estimating the association between exposure and outcome after 

36 adjusting for characteristics of participants that might confound the results. The use of 

37 multipollutant regression models has been the prevailing approach for controlling 
Draft – Do Not Cite or Quote lxv June 2012

1 potential confounding by copollutants in air pollution health effects studies. Finding the 

2 likely causal pollutant from multipollutant regression models is made difficult by the 

3 possibility that one or more air pollutants may be acting as a surrogate for an unmeasured 

4 or poorly measured pollutant or for a particular mixture of pollutants. In addition, more 

5 than one pollutant may exert similar health effects, resulting in independently observed 

6 associations for multiple pollutants. The number and degree of diversity of covariates, as 

7 well as their relevance to the potential confounders, remain matters of scientific 

8 judgment. Despite these limitations, the use of multipollutant models is still the 

9 prevailing approach employed in most air pollution epidemiologic studies and provides 

10 some insight into the potential for confounding or interaction among pollutants.

11 Confidence that unmeasured confounders are not producing the findings is increased 

12 when multiple studies are conducted in various settings using different subjects or 

13 exposures, each of which might eliminate another source of confounding from 

14 consideration. For example, multicity studies can provide insight on potential 

15 confounding through the use of a consistent method to analyze data from across locations 

16 with different levels of copollutants and other covariates . Intervention studies, because of 

17 their quasi-experimental nature, can be particularly useful in characterizing causation. 

18 Another important consideration in the evaluation of epidemiologic evidence is effect 

19 modification, which occurs when the effect differs between subgroups or strata; for 

20 example, effect estimates that vary by age group or potential risk factor. “Effect-measure 

21 modification differs from confounding in several ways. The main difference is that, 

22 whereas confounding is a bias that the investigator hopes to prevent or remove from the 

23 effect estimate, effect-measure modification is a property of the effect under study ... In 

24 epidemiologic analysis one tries to eliminate confounding but one tries to detect and 

25 estimate effect-measure modification” (Rothman and Greenland, 1998). When a risk 

26 factor is a confounder, it is the true cause of the association observed between the 

27 exposure and the outcome; when a risk factor is an effect modifier, it changes the 

28 magnitude of the association between the exposure and the outcome in stratified analyses. 

29 For example, the presence of a preexisting disease or indicator of low socioeconomic 

30 status may be an effect modifier in causing increased risk of effects related to air 

31 pollution exposure. It is often possible to stratify the relationship between health outcome 

32 and exposure by one or more of these potential effect modifiers. For variables that 

33 modify the association, effect estimates in each stratum will be different from one another 

34 and different from the overall estimate, indicating a different exposure-response 

35 relationship may exist in populations represented by these variables.

36 Exposure measurement error, which refers to the uncertainty associated with using 

37 exposure metrics to represent the actual exposure of an individual or population, can be 
Draft – Do Not Cite or Quote lxvi June 2012

1 an important contributor to variability in air pollution epidemiologic study results. 

2 Exposure error can under- or over-estimate epidemiologic associations between ambient 

3 pollutant concentrations and health outcomes by biasing effect estimates toward or away 

4 from the null, and tends to widen confidence intervals around those estimates. There are 

5 several components that contribute to exposure measurement error in air pollution 

6 epidemiologic studies, including the difference between true and measured ambient 

7 concentrations, the difference between average personal exposure to ambient pollutants 

8 and ambient concentrations at central monitoring sites, and the use of average population 

9 exposure rather than individual exposure estimates. Factors that could influence exposure 

10 estimates include nonambient sources of exposure, topography of the natural and built 

11 environment, meteorology, measurement errors, time-location-activity patterns and extent 

12 to which ambient pollutants penetrate indoor environments. The importance of exposure 

13 misclassification varies with study design and is dependent on the spatial and temporal 

14 aspects of the design.

15 The third main type of health effects evidence, animal toxicological studies, provides 

16 information on the pollutant’s biological action under controlled and monitored exposure 

17 circumstances. Taking into account physiological differences of the experimental species 

18 from humans, these studies inform characterization of health effects of concern, 

19 exposure-response relationships and MOAs. Further, animal models can inform 

20 determinations of at-risk populations. These studies evaluate the effects of exposures to a 

21 variety of pollutants in a highly controlled laboratory setting and allow exploration of 

22 toxicological pathways or mechanisms by which a pollutant may cause effects. 

23 Understanding the biological mechanisms underlying various health outcomes can prove 

24 crucial in establishing or negating causality. In the absence of human studies data, 

25 extensive, well-conducted animal toxicological studies can support determinations of

26 causality, if the evidence base indicates that similar responses are expected in humans 

27 under ambient exposure conditions. 

28 Interpretations of animal toxicological studies are affected by limitations associated with 

29 extrapolation between animal and human responses. The differences between humans 

30 and other species have to be taken into consideration, including metabolism, hormonal 

31 regulation, breathing pattern, and differences in lung structure and anatomy. Also, in spite 

32 of a high degree of homology and the existence of a high percentage of orthologous 

33 genes across humans and rodents (particularly mice), extrapolation of molecular 

34 alterations at the gene level is complicated by species-specific differences in 

35 transcriptional regulation. Given these differences, there are uncertainties associated with 

36 quantitative extrapolations of observed pollutant-induced pathophysiological alterations 

37 between laboratory animals and humans, as those alterations are under the control of 

38 widely varying biochemical, endocrine, and neuronal factors.
Draft – Do Not Cite or Quote lxvii June 2012

1 For ecological effects assessment, both laboratory and field studies (including field 

2 experiments and observational studies) can provide useful data for causality 

3 determination. Because conditions can be controlled in laboratory studies, responses may 

4 be less variable and smaller differences easier to detect. However, the control conditions 

5 may limit the range of responses (e.g., animals may not be able to seek alternative food 

6 sources), so they may not reflect responses that would occur in the natural environment. 

7 In addition, larger-scale processes are difficult to reproduce in the laboratory.

8 Field observational studies measure biological changes in uncontrolled situations, and 

9 describe an association between a disturbance and an ecological effect. Field data can 

10 provide important information for assessments of multiple stressors or where site-specific 

11 factors significantly influence exposure. They are also often useful for analyses of larger 

12 geographic scales and higher levels of biological organization. However, because 

13 conditions are not controlled, variability is expected to be higher and differences harder 

14 to detect. Field surveys are most useful for linking stressors with effects when stressor 

15 and effect levels are measured concurrently. The presence of confounding factors can 

16 make it difficult to attribute observed effects to specific stressors. 

17 Intermediate between laboratory and field are studies that use environmental media 

18 collected from the field to examine response in the laboratory, and experiments that are 

19 performed in the natural environment while controlling for some environmental 

20 conditions (i.e., mesocosm studies). This type of study in manipulated natural 

21 environments can be considered a hybrid between a field experiment and laboratory study 

22 since some aspects are performed under controlled conditions but others are not. They 

23 make it possible to observe community and/or ecosystem dynamics, and provide strong 

24 evidence for causality when combined with findings of studies that have been made 

25 under more controlled conditions.

Application of Framework for Causal Determination

26 In its evaluation of the scientific evidence on health or welfare effects of criteria 

27 pollutants, EPA determines the weight of evidence in support of causation and 

28 characterizes the strength of any resulting causal classification. EPA also evaluates the 

29 quantitative evidence and draws scientific conclusions, to the extent possible, regarding 

30 the concentration-response relationships and the loads to ecosystems, exposure doses or 

31 concentrations, duration and pattern of exposures at which effects are observed. 
Draft – Do Not Cite or Quote lxviii June 2012

Table I Aspects to aid in judging causality

Consistency of the 

observed association

An inference of causality is strengthened when a pattern of elevated risks is observed across several 

independent studies. The reproducibility of findings constitutes one of the strongest arguments for 

causality. If there are discordant results among investigations, possible reasons such as differences in 

exposure, confounding factors, and the power of the study are considered.

Coherence An inference of causality from one line of evidence (e.g., epidemiologic, clinical or animal studies) may 

be strengthened by other lines of evidence that support a cause-and-effect interpretation of the 

association. Evidence on ecological or welfare effects may be drawn from a variety of experimental 

approaches (e.g., greenhouse, laboratory, and field) and subdisciplines of ecology (e.g., community 

ecology, biogeochemistry and paleontological/historical reconstructions). The coherence of evidence 

from various fields greatly adds to the strength of an inference of causality. In addition, there may be 

coherence in demonstrating effects across multiple study designs or related health endpoints within one 

scientific line of evidence.

Biological plausibility. An inference of causality tends to be strengthened by consistency with data from experimental studies 

or other sources demonstrating plausible biological mechanisms. A proposed mechanistic linking 

between an effect and exposure to the agent is an important source of support for causality, especially 

when data establishing the existence and functioning of those mechanistic links are available. 

Biological gradient 

(exposure-response 

relationship)

A well-characterized exposure-response relationship (e.g., increasing effects associated with greater 

exposure) strongly suggests cause and effect, especially when such relationships are also observed for 

duration of exposure (e.g., increasing effects observed following longer exposure times). 

Strength of the observed 

association

The finding of large, precise risks increases confidence that the association is not likely due to chance, 

bias, or other factors. However, it is noted that a small magnitude in an effect estimate may represent a 

substantial effect in a population.

Experimental evidence Strong evidence for causality can be provided through “natural experiments” when a change in 

exposure is found to result in a change in occurrence or frequency of health or welfare effects. 

Temporal relationship of 

the observed association

Evidence of a temporal sequence between the introduction of an agent, and appearance of the effect, 

constitutes another argument in favor of causality. 

Specificity of the 

observed association

Evidence linking a specific outcome to an exposure can provide a strong argument for causation. 

However, it must be recognized that rarely, if ever, does exposure to a pollutant invariably predict the 

occurrence of an outcome, and that a given outcome may have multiple causes. 

Analogy Structure activity relationships and information on the agent’s structural analogs can provide insight into 

whether an association is causal. Similarly, information on mode of action for a chemical, as one of 

many structural analogs, can inform decisions regarding likely causality. 

To aid judgment, various “aspects”1 1 of causality have been discussed by many 

2 philosophers and scientists. The 1964 Surgeon General’s report on tobacco smoking 

3 discussed criteria for the evaluation of epidemiologic studies, focusing on consistency, 

4 strength, specificity, temporal relationship, and coherence (HEW, 1964). Sir Austin 

5 Bradford Hill (Hill, 1965) articulated aspects of causality in epidemiology and public 

6 health that have been widely used (Samet and Bodurow, 2008; IARC, 2006; U.S. EPA, 

7 2005; CDC, 2004). These aspects (Hill, 1965) have been modified (Table I) for use in 

8 causal determinations specific to health and welfare effects for pollutant exposures (U.S. 

EPA, 2009d).

2 9 Although these aspects provide a framework for assessing the evidence, 

10 they do not lend themselves to being considered in terms of simple formulas or fixed 

 1 The “aspects” described by Sir Austin Bradford Hill (Hill, 1965) have become, in the subsequent literature, more commonly 

described as “criteria.” The original term “aspects” is used here to avoid confusion with “criteria” as it is used, with different meaning, 

in the Clean Air Act.

2 The Hill aspects were developed for interpretation of epidemiologic results. They have been modified here for use with a broader 

array of data, i.e., epidemiologic, controlled human exposure, ecological, and animal toxicological studies, as well as in vitro data, 

and to be more consistent with EPA’s Guidelines for Carcinogen Risk Assessment.
Draft – Do Not Cite or Quote lxix June 2012

1 rules of evidence leading to conclusions about causality (Hill, 1965). For example, one 

2 cannot simply count the number of studies reporting statistically significant results or 

3 statistically nonsignificant results and reach credible conclusions about the relative 

4 weight of the evidence and the likelihood of causality. Rather, these aspects are taken into 

5 account with the goal of producing an objective appraisal of the evidence, informed by 

6 peer and public comment and advice, which includes weighing alternative views on 

7 controversial issues. In addition, it is important to note that the aspects in Table I cannot 

8 be used as a strict checklist, but rather to determine the weight of the evidence for 

9 inferring causality. In particular, not meeting one or more of the principles does not 

10 automatically preclude a determination of causality [see discussion in (CDC, 2004)]. 

Determination of Causality

11 In the ISA, EPA assesses the body of relevant literature, building upon evidence available 

12 during previous NAAQS reviews, to draw conclusions on the causal relationships 

13 between relevant pollutant exposures and health or environmental effects. ISAs use a 

five-level hierarchy that classifies the weight of evidence for causation1 14 . In developing 

15 this hierarchy, EPA has drawn on the work of previous evaluations, most prominently the 

16 IOM’s Improving the Presumptive Disability Decision-Making Process for Veterans

17 (Samet and Bodurow, 2008), EPA’s Guidelines for Carcinogen Risk Assessment (U.S. 

18 EPA, 2005), and the U.S. Surgeon General’s smoking report (CDC, 2004). This weight 

19 of evidence evaluation is based on various lines of evidence from across the health and 

20 environmental effects disciplines. These separate judgments are integrated into a 

21 qualitative statement about the overall weight of the evidence and causality. The five 

22 descriptors for causal determination are described in Table II.

23 Determination of causality involves the evaluation of evidence for different types of 

24 health, ecological or welfare effects associated with short- and long-term exposure 

25 periods. In making determinations of causality, evidence is evaluated for major outcome 

26 categories and then conclusions are drawn based upon the integration of evidence from 

27 across disciplines and also across the spectrum of related endpoints. In making causal 

28 judgments, the ISA focuses on major outcome categories (e.g., respiratory effects, 

29 vegetation growth), by evaluating the coherence of evidence across a spectrum of related 

30 endpoints (e.g., health effects ranging from inflammatory effects to respiratory mortality) 

31 to draw conclusions regarding causality. In discussing the causal determination, EPA 

 1 The Center for Disease Control (CDC) and IOM frameworks use a four-category hierarchy for the strength of the evidence. A 

five-level hierarchy is used here to be consistent with the EPA Guidelines for Carcinogen Risk Assessment and to provide a more 

nuanced set of categories.
Draft – Do Not Cite or Quote lxx June 2012

1 characterizes the evidence on which the judgment is based, including strength of 

2 evidence for individual endpoints within the major outcome category.

3 In drawing judgments regarding causality for the criteria air pollutants, the ISA focuses 

4 on evidence of effects in the range of relevant pollutant exposures or doses, and not on 

5 determination of causality at any dose. Emphasis is placed on evidence of effects at doses 

6 (e.g., blood lead concentration) or exposures (e.g., air concentrations) that are relevant to, 

7 or somewhat above, those currently experienced by the population. The extent to which 

8 studies of higher concentrations are considered varies by pollutant and major outcome 

9 category, but generally includes those with doses or exposures in the range of one to two 

10 orders of magnitude above current or ambient conditions. Studies that use higher doses or 

11 exposures may also be considered to the extent that they provide useful information to 

12 inform understanding of mode of action, interspecies differences, or factors that may 

13 increase risk of effects for a population. Thus, a causality determination is based on 

14 weight of evidence evaluation for health, ecological or welfare effects, focusing on the 

15 evidence from exposures or doses generally ranging from current levels to one or two 

16 orders of magnitude above current levels.

17 In addition, EPA evaluates evidence relevant to understand the quantitative relationships 

18 between pollutant exposures and health, ecological or welfare effects. This includes 

19 evaluation of the form of concentration-response or dose-response relationships and, to 

20 the extent possible, drawing conclusions on the levels at which effects are observed. The 

21 ISA also draws scientific conclusions regarding important exposure conditions for effects 

22 and populations that may be at greater risk for effects, as described in the following 

23 section.
Draft – Do Not Cite or Quote lxxi June 2012

Table II Weight of evidence for causal determination

Health Effects Ecological and Welfare Effects

Causal 

relationship

Evidence is sufficient to conclude that there is a causal 

relationship with relevant pollutant exposures 

(i.e., doses or exposures generally within one to two 

orders of magnitude of current levels). That is, the 

pollutant has been shown to result in health effects in 

studies in which chance, bias, and confounding could 

be ruled out with reasonable confidence. For example: 

a) controlled human exposure studies that demonstrate 

consistent effects; or b) observational studies that 

cannot be explained by plausible alternatives or are 

supported by other lines of evidence (e.g., animal 

studies or mode of action information). Evidence 

includes multiple high-quality studies

Evidence is sufficient to conclude that there is a causal 

relationship with relevant pollutant exposures i.e., doses 

or exposures generally within one to two orders of 

magnitude of current levels). That is, the pollutant has 

been shown to result in effects in studies in which 

chance, bias, and confounding could be ruled out with 

reasonable confidence. Controlled exposure studies 

(laboratory or small- to medium-scale field studies) 

provide the strongest evidence for causality, but the 

scope of inference may be limited. Generally, 

determination is based on multiple studies conducted by 

multiple research groups, and evidence that is 

considered sufficient to infer a causal relationship is 

usually obtained from the joint consideration of many 

lines of evidence that reinforce each other.

Likely to be a 

causal 

relationship

Evidence is sufficient to conclude that a causal 

relationship is likely to exist with relevant pollutant 

exposures, but important uncertainties remain. That is, 

the pollutant has been shown to result in health effects 

in studies in which chance and bias can be ruled out 

with reasonable confidence but potential issues remain. 

For example: a) observational studies show an 

association, but copollutant exposures are difficult to 

address and/or other lines of evidence (controlled 

human exposure, animal, or mode of action information) 

are limited or inconsistent; or b) animal toxicological 

evidence from multiple studies from different 

laboratories that demonstrate effects, but limited or no 

human data are available. Evidence generally includes 

multiple high-quality studies.

Evidence is sufficient to conclude that there is a likely 

causal association with relevant pollutant exposures. 

That is, an association has been observed between the 

pollutant and the outcome in studies in which chance, 

bias and confounding are minimized, but uncertainties 

remain. For example, field studies show a relationship, 

but suspected interacting factors cannot be controlled, 

and other lines of evidence are limited or inconsistent. 

Generally, determination is based on multiple studies in 

multiple research groups.

Suggestive of 

a causal 

relationship

Evidence is suggestive of a causal relationship with 

relevant pollutant exposures, but is limited. For 

example, (a) at least one high-quality epidemiologic 

study shows an association with a given health outcome 

but the results of other studies are inconsistent; or (b) a 

well-conducted toxicological study, such as those 

conducted in the National Toxicology Program (NTP), 

shows effects in animal species.

Evidence is suggestive of a causal relationship with 

relevant pollutant exposures, but chance, bias and 

confounding cannot be ruled out. For example, at least 

one high-quality study shows an effect, but the results of 

other studies are inconsistent.

Inadequate to 

infer a causal 

relationship

Evidence is inadequate to determine that a causal 

relationship exists with relevant pollutant exposures. 

The available studies are of insufficient quantity, quality, 

consistency, or statistical power to permit a conclusion 

regarding the presence or absence of an effect.

The available studies are of insufficient quality, 

consistency, or statistical power to permit a conclusion 

regarding the presence or absence of an effect.

Not likely to 

be a causal 

relationship

Evidence is suggestive of no causal relationship with 

relevant pollutant exposures. Several adequate studies, 

covering the full range of levels of exposure that human 

beings are known to encounter and considering at-risk 

populations, are mutually consistent in not showing an 

effect at any level of exposure. 

Several adequate studies, examining relationships with 

relevant exposures, are consistent in failing to show an 

effect at any level of exposure.
Draft – Do Not Cite or Quote lxxii June 2012

Quantitative Relationships: Effects on Human Populations

1 Once a determination is made regarding the causal relationship between the pollutant and 

2 outcome category, important questions regarding quantitative relationships include: 

3  What is the concentration-response, exposure-response, or dose-response 

4 relationship in the human population?

5  What is the interrelationship between incidence and severity of effect?

6  What exposure conditions (dose or exposure, duration and pattern) are 

7 important?

8  What populations and lifestages appear to be differentially affected (i.e., more 

9 at risk of experiencing effects)?

10 To address these questions, the entirety of quantitative evidence is evaluated to 

11 characterize pollutant concentrations and exposure durations at which effects were 

12 observed for exposed populations, including populations and lifestages potentially at 

13 increased risk. To accomplish this, evidence is considered from multiple and diverse 

14 types of studies, and a study or set of studies that best approximates the concentration15 response relationships between health outcomes and the pollutant may be identified. 

16 Controlled human exposure studies provide the most direct and quantifiable exposure17 response data on the human health effects of pollutant exposures. To the extent available, 

18 the ISA evaluates results from across epidemiologic studies that use various methods to 

19 characterize the form of relationships between the pollutant and health outcomes and 

20 draws conclusions on the shape of these relationships. Animal data may also inform 

21 evaluation of concentration-response relationships, particularly relative to MOAs and 

22 characteristics of at-risk populations. 

23 An important consideration in characterizing the public health impacts associated with 

24 exposure to a pollutant is whether the concentration-response relationship is linear across 

25 the range of concentrations or if nonlinear relationships exist along any part of this range. 

26 Of particular interest is the shape of the concentration-response curve at and below the 

27 level of the current standards. Various sources of variability and uncertainty, such as low 

28 data density in the lower concentration range, possible influence of exposure 

29 measurement error, and variability between individuals in susceptibility to air pollution 

30 health effects, tend to smooth and “linearize” the concentration-response function, and 

31 thus can obscure the existence of a threshold or nonlinear relationship. Since individual 

32 thresholds vary from person to person due to individual differences such as genetic level 

33 susceptibility or preexisting disease conditions (and even can vary from one time to 

34 another for a given person), it can be difficult to demonstrate that a threshold exists in a 

35 population study. These sources of variability and uncertainty may explain why the 
Draft – Do Not Cite or Quote lxxiii June 2012

1 available human data at ambient concentrations for some environmental pollutants 

2 (e.g., particulate matter [PM], O3, lead [Pb], environmental tobacco smoke [ETS], 

3 radiation) do not exhibit thresholds for cancer or noncancer health effects, even though 

4 likely mechanisms include nonlinear processes for some key events. These attributes of 

5 human population dose-response relationships have been extensively discussed in the 

6 broader epidemiologic literature (Rothman and Greenland, 1998). 

7 Finally, identification of the population groups or lifestages that may be at greater risk of 

8 health effects from air pollutant exposures contributes to an understanding of the public 

9 health impact of pollutant exposures. In the ISA, the term “at-risk population” is used to 

10 encompass populations or lifestages that have a greater likelihood of experiencing health 

11 effects related to exposure to an air pollutant due to a variety of factors; other terms used 

12 in the literature include susceptible, vulnerable, and sensitive. These factors may be 

13 intrinsic, such as genetic or developmental factors, race, gender, lifestage, or the presence 

14 of preexisting diseases, or they may be extrinsic, such as socioeconomic status (SES), 

15 activity pattern and exercise level, reduced access to health care, low educational 

16 attainment, or increased pollutant exposures (e.g., near roadways). Epidemiologic studies 

17 can help identify populations potentially at increased risk of effects by evaluating health 

18 responses in the study population. Examples include testing for interactions or effect 

19 modification by factors such as gender, age group, or health status. Experimental studies 

20 using animal models of susceptibility or disease can also inform the extent to which 

21 health risks are likely greater in specific population groups.

Quantitative Relationships: Effects on Ecosystems or Public 

Welfare

22 Key questions for understanding the quantitative relationships between exposure (or 

23 concentration or deposition) to a pollutant and risk to ecosystems or the public welfare 

24 include: 

25  What elements of the ecosystem (e.g., types, regions, taxonomic groups, 

26 populations, functions, etc.) appear to be affected, or are more sensitive to 

27 effects? Are there differences between locations or materials in welfare effects 

28 responses, such as impaired visibility or materials damage?

29  Under what exposure conditions (amount deposited or concentration, duration 

30 and pattern) are effects seen? 

31  What is the shape of the concentration-response or exposure-response 

32 relationship? 

33 Evaluations of causality generally consider the probability of quantitative changes in 

34 ecological and welfare effects in response to exposure. A challenge to the quantification 
Draft – Do Not Cite or Quote lxxiv June 2012

1 of exposure-response relationships for ecological effects is the great regional and local 

2 spatial variability, as well as temporal variability, in ecosystems. Thus, exposure3 response relationships are often determined for a specific ecological system and scale, 

4 rather than at the national or even regional scale. Quantitative relationships therefore are 

5 available site by site and may differ greatly between ecosystems. 

Concepts in Evaluating Adversity of Health Effects

6 In evaluating health evidence, a number of factors can be considered in delineating 

7 between adverse and nonadverse health effects resulting from exposure to air pollution. 

8 Some health outcomes, such as hospitalization for respiratory or cardiovascular diseases, 

9 are clearly considered adverse. It is more difficult to determine the extent of change that 

10 constitutes adversity in more subtle health measures. These include a wide variety of 

11 responses, such as alterations in markers of inflammation or oxidative stress, changes in 

12 pulmonary function or heart rate variability, or alterations in neurocognitive function 

13 measures. The challenge is determining the magnitude of change in these measures when 

14 there is no clear point at which a change become adverse; for example, what percentage 

15 change in a lung function measure represents an adverse effect. What constitutes an 

16 adverse health effect may vary between populations. Some changes that may not be 

17 considered adverse in healthy individuals would be potentially adverse in more at-risk

18 individuals. 

19 For example, the extent to which changes in lung function are adverse has been discussed 

20 by the American Thoracic Society (ATS) in an official statement titled What Constitutes 

21 an Adverse Health Effect of Air Pollution? (2000b). An air pollution-induced shift in the 

22 population distribution of a given risk factor for a health outcome was viewed as adverse, 

23 even though it may not increase the risk of any one individual to an unacceptable level. 

24 For example, a population of asthmatics could have a distribution of lung function such 

25 that no identifiable individual has a level associated with significant impairment. 

26 Exposure to air pollution could shift the distribution such that no identifiable individual 

27 experiences clinically relevant effects. This shift toward decreased lung function, 

28 however, would be considered adverse because individuals within the population would 

29 have diminished reserve function and therefore would be at increased risk to further 

30 environmental insult. The committee also observed that elevations of biomarkers, such as 

31 cell number and types, cytokines and reactive oxygen species, may signalrisk for ongoing 

32 injury and clinical effects or may simply indicate transient responses that can provide 

33 insights into mechanisms of injury, thus illustrating the lack of clear boundaries that 

34 separate adverse from nonadverse effects.
Draft – Do Not Cite or Quote lxxv June 2012

1 The more subtle health outcomes may be connected mechanistically to health events that 

2 are clearly adverse. For example, air pollution may affect markers of transient myocardial 

3 ischemia such as ST-segment abnormalities and onset of exertional angina. These effects 

4 may not be apparent to the individual, yet may still increase the risk of a number of 

5 cardiac events, including myocardial infarction and sudden death. Thus, small changes in 

6 physiological measures may not appear to be clearly adverse when considered alone, but 

7 may be a part of a coherent and biologically plausible chain of related health outcomes 

8 that range up to responses that are very clearly adverse, such as hospitalization or 

9 mortality.

Concepts in Evaluating Adversity of Ecological Effects

10 Adversity of ecological effects can be understood in terms ranging in scale from the 

11 cellular level to the individual organism and to the population, community, and 

12 ecosystem levels. In the context of ecology, a population is a group of individuals of the 

13 same species, and a community is an assemblage of populations of different species 

14 interacting with one another that inhabit an area. An ecosystem is the interactive system 

15 formed from all living organisms and their abiotic (physical and chemical) environment 

16 within a given area (IPCC, 2007a). The boundaries of what could be called an ecosystem 

17 are somewhat arbitrary, depending on the focus of interest or study. Thus, the extent of an 

18 ecosystem may range from very small spatial scales to, ultimately, the entire Earth 

19 (IPCC, 2007a).

20 Effects on an individual organism are generally not considered to be adverse to public 

21 welfare. However if effects occur to enough individuals within a population, then 

22 communities and ecosystems may be disrupted. Changes to populations, communities 

23 and ecosystems can in turn result in an alteration of ecosystem processes. Ecosystem 

24 processes are defined as the metabolic functions of ecosystems including energy flow, 

25 elemental cycling, and the production, consumption and decomposition of organic matter 

26 (U.S. EPA, 2002). Growth, reproduction, and mortality are species-level endpoints that 

27 can be clearly linked to community and ecosystem effects and are considered to be 

28 adverse when negatively affected. Other endpoints such as changes in behavior and 

29 physiological stress can decrease ecological fitness of an organism, but are harder to link 

30 unequivocally to effects at the population, community, and ecosystem level. The degree 

31 to which pollutant exposure is considered adverse may also depend on the location and its 

32 intended use (i.e., city park, commercial, cropland). Support for consideration of 

33 adversity beyond the species level by making explicit the linkages between stress-related 

34 effects at the species and effects at the ecosystem level is found in A Framework for 

35 Assessing and Reporting on Ecological Condition: an SAB report (U.S. EPA, 2002). 

36 Additionally, the National Acid Precipitation Assessment Program (NAPAP) uses the 
Draft – Do Not Cite or Quote lxxvi June 2012

1 following working definition of “adverse ecological effects” in the preparation of reports 

2 to Congress mandated by the Clean Air Act: “any injury (i.e., loss of chemical or physical 

3 quality or viability) to any ecological or ecosystem component, up to and including at the 

4 regional level, over both long and short terms.” 

5 On a broader scale, ecosystem services may provide indicators for ecological impacts. 

6 Ecosystem services are the benefits that people obtain from ecosystems (UNEP, 2003). 

7 According to the Millennium Ecosystem Assessment, ecosystem services include: 

8 “provisioning services such as food and water; regulating services such as regulation of 

9 floods, drought, land degradation, and disease; supporting services such as soil formation 

10 and nutrient cycling; and cultural services such as recreational, spiritual, religious and 

11 other nonmaterial benefits.” For example, a more subtle ecological effect of pollution 

12 exposure may result in a clearly adverse impact on ecosystem services if it results in a 

13 population decline in a species that is recreationally or culturally important.
Draft – Do Not Cite or Quote lxxvii June 2012

References

ATS (American Thoracic Society). (2000b). What constitutes an adverse health effect of air pollution? This 

official statement of the American Thoracic Society was adopted by the ATS Board of Directors, July 

1999. Am J Respir Crit Care Med 161: 665-673. 

Bell, ML; Dominici, F; Samet, JM. (2005). A meta-analysis of time-series studies of ozone and mortality 

with comparison to the national morbidity, mortality, and air pollution study. Epidemiology 16: 436-

445. http://dx.doi.org/10.1097/01.ede.0000165817.40152.85

CDC (Centers for Disease Control and Prevention). (2004). The health consequences of smoking: A report of 

the Surgeon General. Washington, DC: U.S. Department of Health and Human Services. 

http://www.surgeongeneral.gov/library/smokingconsequences/

Fox, GA. (1991). Practical causal inference for ecoepidemiologists. J Toxicol Environ Health A 33: 359-373. 

http://dx.doi.org/10.1080/15287399109531535

Gee, GC; Payne-Sturges, DC. (2004). Environmental health disparities: A framework integrating 

psychosocial and environmental concepts. Environ Health Perspect 112: 1645-1653. 

http://dx.doi.org/10.1289/ehp.7074

HEW (U.S. Department of Health, Education and Welfare). (1964). Smoking and health: Report of the 

advisory committee to the surgeon general of the public health service. Washington, DC: U.S. 

Department of Health, Education, and Welfare. 

http://profiles.nlm.nih.gov/ps/retrieve/ResourceMetadata/NNBBMQ

Hill, AB. (1965). The environment and disease: Association or causation? Proc R Soc Med 58: 295-300. 

IARC (International Agency for Research on Cancer). (2006). Preamble to the IARC monographs. Lyon, 

France. http://monographs.iarc.fr/ENG/Preamble/

Ioannidis, JPA. (2008). Why most discovered true associations are inflated. Epidemiology 19: 640-648. 

http://dx.doi.org/10.1097/EDE.0b013e31818131e7

IPCC (Intergovernmental Panel on Climate Change). (2007a). Climate change 2007: Impacts, adaptation and 

vulnerability. Cambridge, UK: Cambridge University Press. 

Rothman, KJ; Greenland, S. (1998). Modern epidemiology (2nd ed.). Philadelphia, PA: Lippincott, Williams, 

& Wilkins. 

Samet, JM; Bodurow, CC. (2008). Improving the presumptive disability decision-making process for 

veterans. In JM Samet; CC Bodurow (Eds.). Washington, DC: National Academies Press. 

http://www.nap.edu/openbook.php?record_id=11908

U.S. EPA (U.S. Environmental Protection Agency). (1998a). Guidelines for ecological risk assessment [EPA 

Report]. (EPA/630/R-95/002F). Washington, DC. http://www.epa.gov/raf/publications/guidelinesecological-risk-assessment.htm

U.S. EPA (U.S. Environmental Protection Agency). (2002). A framework for assessing and reporting on 

ecological condition: An SAB report [EPA Report]. Washington, DC. 

http://www.ntis.gov/search/product.aspx?ABBR=PB2004100741

U.S. EPA (U.S. Environmental Protection Agency). (2005). Guidelines for carcinogen risk assessment [EPA 

Report]. (EPA/630/P-03/001F). Washington, DC. http://www.epa.gov/cancerguidelines/

U.S. EPA (U.S. Environmental Protection Agency). (2009d). Integrated science assessment for particulate 

matter [EPA Report]. (EPA/600/R-08/139F). Research Triangle Park, NC. 

http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=216546

UNEP (United Nations Environment Programme). (2003). Millennium Ecosystem Assessment: Ecosystems 

and human well-being: A framework for assessment. Washington, DC: Island Press.