Office of Air Quality Planning and Standards

U.S. Environmental Protection Agency

Research Triangle Park, NC 27711

EPA

AA

United States

Environmental 

Protection Agency

Ozone National Ambient Air Quality 

Standards: Scope and Methods Plan 

for Health Risk and Exposure 

Assessment

April 2011

 EPA-452/P-11-001 

April 2011 

Ozone National Ambient Air Quality Standards: 

Scope and Methods Plan 

for Health Risk and Exposure Assessment 

U.S. Environmental Protection Agency 

Office of Air and Radiation 

Office of Air Quality Planning and Standards 

Health and Environmental Impacts Division 

Risk and Benefits Group 

Research Triangle Park, North Carolina 27711 
DISCLAIMER

This planning document has been prepared by staff from the Office of Air Quality 

Planning and Standards, U.S. Environmental Protection Agency. Any opinions, findings, 

conclusions, or recommendations are those of the authors and do not necessarily reflect the 

views of the EPA. This document is being circulated to facilitate consultation with the Clean Air 

Scientific Advisory Committee (CASAC) and to obtain public review. For questions concerning 

this document, please contact Mr. John Langstaff (langstaff.john@epa.gov) or Dr. Zach Pekar 

(pekar.zachary@epa.gov), U.S. Environmental Protection Agency, Office of Air Quality 

Planning and Standards, C504-06, Research Triangle Park, North Carolina 27711.
i

Table of Contents

1 INTRODUCTION........................................................................................................... 1-1

1.1 Background on Last Ozone NAAQS Review ...................................................... 1-3

1.1.1 Overview of Exposure Assessment for Ozone from Last Review...... 1-4

1.1.2 Overview of Health Risk Assessment for Ozone from Last Review .. 1-5

1.2 Goals for Framing the Assessments in the Current Review ................................. 1-7

1.3 Overview of Current Assessment Plan ................................................................ 1-7

1.3.1 Air Quality Assessment .................................................................... 1-8

1.3.2 Exposure Assessment ....................................................................... 1-9

1.3.3 Health Risk Assessment.................................................................... 1-9

2 AIR QUALITY CONSIDERATIONS............................................................................. 2-1

2.1 Introduction ........................................................................................................ 2-1

2.2 Air Quality Inputs to Risk and Exposure Assessments ........................................ 2-1

2.2.1 Recent Air Quality............................................................................ 2-1

2.2.2 Air Quality Data Related to Exceptional Events................................ 2-3

2.3 Development of Estimates of Ozone Air Quality Assuming ―Just Meeting‖ Current 

NAAQS and Potential Alternative NAAQS ....................................................... 2-3

2.3.1 Background and Conceptual Overview ............................................. 2-3

2.3.2 Historical Approach.......................................................................... 2-4

2.4 Policy Relevant Background ............................................................................... 2-4

2.5 Broader Air Quality Characterization.................................................................. 2-7

3 APPROACH FOR POPULATION EXPOSURE ANALYSIS......................................... 3-1

3.1 Introduction ........................................................................................................ 3-1

3.2 The APEX Population Exposure Model .............................................................. 3-1

3.3 Populations Modeled .......................................................................................... 3-7

3.4 Outcomes to be Generated .................................................................................. 3-7

3.5 Selection of Urban Areas and Time Periods........................................................ 3-8

3.6 Development of Model Inputs............................................................................. 3-8

3.6.1 Population Demographics................................................................. 3-9

3.6.2 Commuting....................................................................................... 3-9

3.6.3 Ambient OzoneConcentrations......................................................... 3-9

3.6.4 Meteorological Data ....................................................................... 3-10

3.6.5 Specification of Microenvironments ............................................... 3-10

3.6.6 Indoor Sources................................................................................ 3-12

3.6.7 Activity Patterns............................................................................. 3-12

3.7 Exposure Modeling Issues ................................................................................ 3-16

3.8 Uncertainty and Variability............................................................................... 3-18

4 ASSESSMENT OF HEALTH RISK BASED ON CONTROLLED HUMAN EXPOSURE 

STUDIES........................................................................................................................ 4-1

4.1 Introduction ........................................................................................................ 4-1

4.2 Selection of Health Endpoints............................................................................. 4-1

4.3 Selection of Exposure-Response Functions......................................................... 4-2

4.4 Approach to Calculating Risk Estimates ............................................................. 4-2

4.5 Alternative Approach Under Consideration For Calculating Risk Estimates........ 4-4
ii

4.6 Uncertainty and Variability................................................................................. 4-6

5 ASSESSMENT OF HEALTH RISK BASED ON EPIDEMIOLOGIC STUDIES ........... 5-1

5.1 Introduction ........................................................................................................ 5-1

5.2 Framework for the Ozone Health Risk Assessment ............................................. 5-5

5.2.1 Overview of Modeling Approach...................................................... 5-5

5.2.2 Air Quality Considerations ............................................................... 5-7

5.3 Selection of Health Effects Endpoint Categories ............................................... 5-10

5.3.1 Selection of Epidemiological Studies and Specification of 

Concentration-Response Functions................................................. 5-12

5.3.2 Selection of Urban Study Areas...................................................... 5-16

5.3.3 Baseline Health Effects Incidence Data and Demographic Data ...... 5-17

5.3.4 Assessing Risk In Excess of Policy-Relevant Background .............. 5-18

5.4 Characterization of Uncertainty and Variability in the Context of the Ozone Risk 

Assessment ...................................................................................................... 5-20

5.4.1 Overview of Approach for Addressing Uncertainty and Variability 5-20

5.4.2 Addressing Variability.................................................................... 5-23

5.4.3 Uncertainty Characterization – Qualitative Assessment .................. 5-25

5.4.4 Uncertainty Characterization – Quantitative Analysis..................... 5-26

5.4.5 Representativeness Analysis........................................................... 5-32

6 PRESENTATION OF RISK ESTIMATES TO INFORM CONSIDERATION OF 

STANDARDS................................................................................................................. 6-1

7 SCHEDULE AND MILESTONES.................................................................................. 7-1

8 REFERENCES................................................................................................................ 8-3
iii

List of Tables

Table 3-1. Microenvironments to be Modeled ........................................................................ 3-6

Table 3-2. Studies In CHAD To Be Used For Exposure Modeling ....................................... 3-14

Table 5-1. Planned Sensitivity Analyses for the Epidemiologic-Based Risk Assessment ...... 5-28

Table 7-1. Key Milestones for the Exposure Analysis and Health Risk Assessment for the Ozone 

NAAQS Review ..................................................................................................... 7-1

List of Figures

Figure 1-1. Overview of Risk Assessment Based on Epidemiologic Studies......................... 1-12

Figure 3-1. Overview of the APEX Model ............................................................................. 3-3

Figure 3-2. The Mass Balance Model................................................................................... 3-11

Figure 4-1. Major Components of Ozone Health Risk Assessment Based on Controlled Human 

Exposure Studies .................................................................................................... 4-7

Figure 5-1. Overview of Risk Assessment Model Based on Epidemiologic Studies................ 5-8

Figure 5-2. Overview of Approach For Uncertainty Analysis of Risk Assessment Based on 

Epidemiologic Studies .......................................................................................... 5-30
iv

List of Acronyms/Abbreviations

Act Clean Air Act

AHRQ Agency for Healthcare Research and Quality

APEX EPA’s Air Pollutants Exposure model, version 4

AQS EPA’s Air Quality System

BenMAP Benefits Mapping Analysis Program 

CASAC Clean Air Scientific Advisory Committee

CHAD EPA’s Consolidated Human Activity Database

CONUS Continental United States

C-R Concentration-response relationship

CSA Consolidated Statistical Area

CTM Chemical transport models

EPA United States Environmental Protection Agency

FEM Federal Equivalent Method

FIP Federal Implementation Plan

FRM Federal Reference Method

ISA Integrated Science Assessment

NAAQS National Ambient Air Quality Standards

NAPS National Air Pollution Surveillance

NCEA National Center for Environmental Assessment

NEI National Emissions Inventory

NERL National Exposure Research Laboratory

NCDC National Climatic Data Center

NCore National Core Monitoring Network

NOx Nitrogen oxides

O3 Ozone

OAQPS Office of Air Quality Planning and Standards

OAR Office of Air and Radiation

OMB Office of Management and Budget

ORD Office of Research and Development

PRB Policy-Relevant Background

QA Quality assurance

QC Quality control

RR Relative risk

SAB Science Advisory Board

REA Risk and Exposure Assessment

SEDD State Emergency Department Databases

SID State Inpatient Database

SO2 Sulfur Dioxide

SOx Sulfur Oxides

TSP Total suspended particulate 

VOC Volatile organic compounds
1-1

1 1 INTRODUCTION

2 The U.S. Environmental Protection Agency (EPA) is presently conducting a review of 

3 the national ambient air quality standards (NAAQS) for ozone. Sections 108 and 109 of the 

4 Clean Air Act (Act) govern the establishment and periodic review of the NAAQS. These 

5 standards are established for pollutants that may reasonably be anticipated to endanger public 

6 health and welfare, and whose presence in the ambient air results from numerous or diverse 

7 mobile or stationary sources. The NAAQS are to be based on air quality criteria, which are to 

8 accurately reflect the latest scientific knowledge useful in indicating the kind and extent of 

9 identifiable effects on public health or welfare that may be expected from the presence of the 

10 pollutant in ambient air. The EPA Administrator is to promulgate and periodically review, at 

11 five-year intervals, ―primary‖ (health-based) and ―secondary‖ (welfare-based) NAAQS for such 

pollutants.112 Based on periodic reviews of the air quality criteria and standards, the Administrator 

13 is to make revisions in the criteria and standards, and promulgate any new standards, as may be 

14 appropriate. The Act also requires that an independent scientific review committee advise the 

15 Administrator as part of this NAAQS review process, a function now performed by the Clean Air 

16 Scientific Advisory Committee (CASAC).

17 EPA’s overall plan and schedule for this ozone NAAQS review are presented in the

18 Integrated Review Plan for the Ozone National Ambient Air Quality Standards Review (U.S. 

19 EPA, 2011a). That plan outlines the Clean Air Act (CAA) requirements related to the 

20 establishment and reviews of the NAAQS, the process and schedule for conducting the current 

21 ozone NAAQS review, and two key components in the NAAQS review process: an Integrated 

22 Science Assessment (ISA) and a Risk and Exposure Assessment (REA). It also lays out the key 

23 policy-relevant issues to be addressed in this review as a series of policy-relevant questions that 

24 will frame our approach to determining whether the current primary and secondary NAAQS for 

25 ozone should be retained or revised. 

 

1

Section 109(b)(1) [42 U.S.C. 7409] of the Act defines a primary standard as one ―the attainment and 

maintenance of which in the judgment of the Administrator, based on such criteria and allowing an adequate margin 

of safety, are requisite to protect the public health.‖ 
1-2

1 The ISA prepared by EPA’s Office of Research and Development (ORD), National 

2 Center for Environmental Assessment (NCEA), provides a critical assessment of the latest 

3 available policy-relevant scientific information upon which the NAAQS are to be based. The 

4 ISA will critically evaluate and integrate scientific information on the health and welfare effects 

5 associated with exposure to ozone in the ambient air. The REA, prepared by EPA’s Office of 

6 Air and Radiation (OAR), Office of Air Quality Planning and Standards (OAQPS), will draw

7 from the information assessed in the ISA. The REA will include, as appropriate, quantitative 

8 estimates of human and ecological exposures and/or risks associated with recent ambient levels 

9 of ozone, with levels simulated to just meet the current standards, and with levels simulated to

10 just meet possible alternative standards.

11 The REA will be developed in two parts addressing: (1) human health risk and exposure 

12 assessment and (2) other welfare-related effects assessment. This document describes the scope 

13 and methods planned to conduct the human health risk and exposure assessments to support the 

14 review of the primary (health-based) ozone NAAQS. A separate document describes the scope 

15 and methods planned to conduct quantitative assessments to support the review of the secondary 

16 (welfare-based) ozone NAAQS. Preparation of these two planning documents coincides with the 

17 development of the first draft ozone ISA (U.S. EPA, 2011b) to facilitate the integration of 

18 policy-relevant science into all three documents. 

19 This planning document is intended to provide enough specificity to facilitate 

20 consultation with CASAC, as well as for public review, in order to obtain advice on the overall 

21 scope, approaches, and key issues in advance of the conduct of the risk and exposure analyses 

22 and presentation of results in the first draft REA. NCEA has compiled and assessed the latest 

23 available policy-relevant science available to produce a first draft of the ISA (U.S. EPA, 2011b). 

24 The first draft ISA has been reviewed by staff and used in the development of the approaches 

25 described below. This includes information on source emissions, atmospheric chemistry, air 

26 quality, human exposure, and related health effects. CASAC consultation on this planning 

27 document coincides with its review of the first draft ISA. CASAC and public comments on this 

28 document will be taken into consideration in the development of the first draft REA, the 

29 preparation of which will coincide with and draw from the second draft ISA. The second draft 
1-3

1 REA will draw on the final ISA and will reflect consideration of CASAC and public comments 

2 on the first draft REA. The final REA will reflect consideration of CASAC and public 

3 comments on the second draft REA. The final ISA and final REA will inform the policy 

4 assessment and rulemaking steps that will lead to a final decision on the ozone NAAQS.

5 This introductory chapter includes background on the current ozone standards and the 

6 quantitative risk assessment conducted for the last review; the key issues related to designing the 

7 quantitative assessments in this review, building upon the lessons learned in the last review; and 

8 an overview introducing the planned assessments that are described in more detail in later 

9 chapters. The planned assessments are designed to estimate human exposures and/or health risks 

10 that are associated with recent ambient levels, with ambient levels simulated to just meet the 

11 current standards, and with ambient levels simulated to just meet alternative standards that may 

12 be considered. The major components of the assessments (e.g., air quality analyses, quantitative 

13 exposure assessment, and quantitative health risk assessments) briefly outlined in the Integrated 

14 Review Plan (U.S. EPA, 2011a), are conceptually presented in Figure 1-1, and are described in 

15 more detail below in Chapters 2 – 6. The schedule for completing these assessments is presented 

16 in Chapter 7. 

17 1.1 Background on Last Ozone NAAQS Review

18 As a first step in developing this planning document, we considered the work completed 

19 in previous reviews of the primary NAAQS for ozone (U.S. EPA, 2011a, see section 1.3) and in 

20 particular the quantitative assessments supporting those reviews. EPA completed the most 

21 recent review of the ozone NAAQS with publication of a decision on March 27, 2008 (73 FR 

22 16436). Based on the final CD (U.S. EPA, 2006) published in March of 2006, and on the final 

23 Staff Paper (U.S EPA, 2007a) published in July of 2007, the previous EPA Administrator 

24 decided to revise the level of the 8-hour primary ozone standard from 0.08 ppm to 0.075 ppm 

25 and to revise the secondary to be identical to the primary. As discussed in more detail in the 

26 Integrated Review Plan, the current EPA Administrator has decided to reconsider the March 27, 

27 2008 decisions on the revisions to the primary and secondary ozone NAAQS.
1-4

1 1.1.1 Overview of Exposure Assessment for Ozone from Last Review

2 The exposure and health risk assessment conducted in the review completed in 

3 March 2008 developed exposure and health risk estimates for 12 urban areas across the U.S., 

4 which were chosen, based on the location of ozone epidemiological studies and to represent a 

5 range of geographic areas, population demographics, and ozone climatology. That analysis was 

6 in part based upon the exposure and health risk assessments done as part of the review completed 

in 1997.17 The exposure and risk assessment incorporated air quality data (i.e., 2002 through 

8 2004) and provided annual or ozone season-specific exposure and risk estimates for these recent 

9 years of air quality and for air quality scenarios simulating just meeting the existing 8-hour

10 ozone standard and several alternative 8-hour ozone standards. Exposure estimates were used as 

11 an input to the risk assessment for lung function responses (a health endpoint for which 

12 exposure-response functions were available from controlled human exposure studies). Exposure 

13 estimates were developed for the general population and population groups including school age 

14 children with asthma as well as all school age children. The exposure estimates also provided 

15 information on population exposures exceeding potential health effect benchmark levels that 

16 were identified based on the observed occurrence of health endpoints not explicitly modeled in 

17 the health risk assessment (e.g., lung inflammation, increased airway responsiveness, and 

18 decreased resistance to infection) associated with 6-8 hour exposures to ozone in controlled 

19 human exposure studies.

20 The exposure analysis took into account several important factors including the 

21 magnitude and duration of exposures, frequency of repeated high exposures, and breathing rate 

22 of individuals at the time of exposure. Estimates were developed for several indicators of 

23 exposure to various levels of ozone air quality, including counts of people exposed one or more 

24 times to a given ozone concentration while at a specified breathing rate, and counts of person25 occurrences which accumulate occurrences of specific exposure conditions over all people in the 

26 population groups of interest over an ozone season. 

 

1 In the 1994-1997 Ozone NAAQS review, EPA conducted exposure analyses for the general population, children 

who spent more time outdoors, and outdoor workers. Exposure estimates were generated for 9 urban areas for ―as 

is‖ air quality and for just meeting the existing 1-hour standard and several alternative 8-hour standards. Several 

reports (Johnson et al., 1996a,b,c) that describe these analyses can be found at: 

http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_pr.html.
1-5

1 As discussed in the 2007 Staff Paper and in Section IIa of the ozone Final Rule (73 FR 

2 16440 to 16442, March 27, 2008), the most important uncertainties affecting the exposure 

3 estimates were related to modeling human activity patterns over an ozone season, modeling of 

4 variations in ambient concentrations near roadways, and modeling of air exchange rates that 

5 affect the amount of ozone that penetrates indoors. Another important uncertainty, discussed in 

6 more detail in the Staff Paper (section 4.3.4.7), was the uncertainty in energy expenditure values 

7 which directly affected the modeled breathing rates. These were important since they were used 

8 to classify exposures occurring when children were engaged in moderate or greater exertion and 

9 health effects observed in the controlled human exposure studies generally occurred under these 

10 exertion levels for 6 to 8-hour exposures to ozone concentrations at or near 0.08 ppm. Reports 

11 that describe these analyses (U.S. EPA, 2007a,b; Langstaff, 2007) can be found at: 

12 http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_index.html.

13 1.1.2 Overview of Health Risk Assessment for Ozone from Last Review

14 The human health risk assessment presented in the review completed in March 

15 2008 was designed to estimate population risks in a number of urban areas across the U.S., 

16 consistent with the scope of the exposure analysis described above. The risk assessment

17 included risk estimates based on both controlled human exposure studies and epidemiological 

18 and field studies. Ozone-related risk estimates for lung function decrements were generated

19 using probabilistic exposure-response relationships based on data from controlled human 

20 exposure studies, together with probabilistic exposure estimates from the exposure analysis. For 

21 several other health endpoints, ozone-related risk estimates were generated using concentration22 response relationships reported in epidemiological or field studies, together with ambient air 

23 quality concentrations, baseline health incidence rates, and population data for the various 

24 locations included in the assessment. Health endpoints included in the assessment based on

25 epidemiological or field studies included: hospital admissions for respiratory illness in four urban 

26 areas, premature mortality in 12 urban areas, and respiratory symptoms in asthmatic children in 1 

27 urban area.

28 In the health risk assessment conducted in the previous review, EPA recognized that there 

29 were many sources of uncertainty and variability in the inputs to the assessment and that there 
1-6

1 was a high degree of uncertainty in the resulting risk estimates. The statistical uncertainty 

2 surrounding the estimated ozone coefficients in concentration-response functions as well as the 

3 shape of the exposure-response relationship chosen were addressed quantitatively. Additional 

4 uncertainties were addressed through sensitivity analyses and/or qualitatively. The risk 

5 assessment conducted for that ozone NAAQS review incorporated some of the variability in key 

6 inputs to the assessment by using location-specific inputs (e.g., location-specific concentration7 response function, baseline incidence rates and population data, and air quality data for 

8 epidemiological–based endpoints, location specific air quality data and exposure estimates for 

9 the lung function risk assessment). In that review, several urban areas were included in the 

10 health risk assessment to provide some sense of the variability in the risk estimates across the 

11 U.S.

12 Key observations and insights from the ozone risk assessment, in addition to important 

13 caveats and limitations, were addressed in Section II.B of the Final Rule notice (73 FR 16440 to 

14 16443, March 27, 2008). In general, estimated risk reductions associated with going from 

15 current ozone levels to just meeting the current and alternative 8-hour standards showed patterns 

16 of decreasing estimated risk associated with just meeting the lower alternative 8-hour standards 

17 considered. Furthermore, the estimated percentage reductions in risk were strongly influenced 

18 by the baseline air quality year used in the analysis, which was due to significant year-to-year 

19 variability in ozone concentrations. There was also noticeable city-to-city variability in the

20 estimated ozone-related incidence of morbidity and mortality across the 12 urban areas. 

Uncertainties associated with estimated policy-relevant background (PRB) concentrations121 were 

22 also addressed and revealed differential impacts on the risk estimates depending on the health 

23 effect considered as well as the location. EPA also acknowledged that there were considerable 

24 uncertainties surrounding estimates of ozone coefficients and the shape for concentration25 response relationships and whether or not a population threshold or non-linear relationship exists 

26 within the range of concentrations examined in the epidemiological studies.

 

1 For the purposes of the risk and exposure assessments, policy-relevant background (PRB) ozone has been defined 

in previous reviews as the distribution of ozone concentrations that would be observed in the U.S. in the absence of 

anthropogenic (man-made) emissions of ozone precursor emissions (e.g., VOC, CO, NOx) in the U.S., Canada, and 

Mexico. 
1-7

1 1.2 Goals for Framing the Assessments in the Current Review

2 A critical step in designing the quantitative risk and exposure assessments is to clearly 

3 identify the policy-relevant questions to be addressed by these assessments. As identified above, 

4 the Integrated Review Plan presents a series of key policy questions (U.S. EPA, 2011a, section

5 3.1). To answer these questions, EPA will integrate information from the ISA and from air 

6 quality, risk, and exposure assessments as we evaluate both evidence-based and risk-based 

7 considerations. 

8 More specifically, to focus the REA, we have identified the following goals for the

9 exposure and risk assessment: (1) to provide estimates of the number of people in the general 

10 population and in sensitive populations with ozone exposures above benchmark levels; (2) to 

11 provide estimates of the number of people in the general population and in sensitive populations 

12 with impaired lung function resulting from exposures to ozone; (3) to provide estimates of the 

13 potential magnitude of premature mortality and/or selected morbidity health effects in the 

14 population, including sensitive populations, where data are available to assess these subgroups, 

15 associated with recent ambient levels of ozone and with just meeting the current suite of ozone

16 standards and any alternative standards that might be considered in selected urban study areas; 

17 (4) to develop a better understanding of the influence of various inputs and assumptions on the 

18 risk estimates to more clearly differentiate alternative standards that might be considered

19 including potential impacts on various sensitive populations; and (5) to gain insights into the 

20 distribution of risks and patterns of risk reduction and uncertainties in those risk estimates. In 

21 addition, we are considering conducting an assessment to provide nationwide estimates of the 

22 potential magnitude of premature mortality associated with ambient ozone exposures to more 

23 broadly characterize this risk on a national scale, to assess the extent to which we have captured 

24 the upper end of the risk distribution, and to support the interpretation of the more detailed risk 

25 results generated for the selected urban study areas. 

26 1.3 Overview of Current Assessment Plan

27 This plan is designed to outline the scope and approaches and highlight key issues in the 

28 estimation of population exposures and health risks posed by ozone under existing air quality 

29 levels (―as is‖ exposures and health risks), upon attainment of the current ozone primary 
1-8

1 NAAQS, and upon meeting various alternative standards in selected sample urban areas. This 

2 plan is intended to facilitate consultation with the CASAC, as well as public review, and to 

3 obtain advice on the overall scope, approaches, and key issues in advance of the completion of 

4 such analyses and presentation of results in the first draft of the ozone Policy Assessment. 

5 The planned ozone exposure analysis and health risk assessment address short-term and 

6 long-term exposures to ozone and associated health effects. These assessments cover a variety 

7 of health effects for which there is adequate information to develop quantitative risk estimates. 

8 However, there are some health endpoints for which there currently is insufficient information to 

9 develop quantitative risk estimates. Staff plans to discuss these additional health endpoints 

10 qualitatively in the ozone Policy Assessment. The risk assessment is intended as a tool that, 

11 together with other information on these health endpoints and other health effects evaluated in 

12 the ozone ISA and ozone Policy Assessment, can aid the Administrator in judging whether the 

13 current primary standard is requisite to protect public health with an adequate margin of safety, 

14 or whether revisions to the standard are appropriate.

15 Staff plans to perform exposure and health risk analyses using the three most recent years 

16 of air quality data available at this time, 2008-2010. The time period to be analyzed will be the 

17 ozone season, which in the urban areas to be included in this assessment, varies from April to 

18 October to the entire year depending on the region of the country.

19 1.3.1 Air Quality Assessment 

20 Chapter 2 describes assessments planned for the current review of the primary NAAQS 

21 for ozone including air quality analyses to be conducted to support quantitative risk and exposure 

22 assessments in selected urban study areas as well as to support evidence-based considerations 

23 and to place the results of the quantitative assessments into a broader public health perspective. 

24 Air quality inputs will include: (1) recent air quality data for ozone from suitable monitors for 

25 each selected urban study area; (2) estimates of background concentrations for each selected 

26 urban study area, and (3) simulated air quality that reflects changes in the distribution of ozone

27 air quality estimated to occur when an area just meets the current or alternative ozone standards 

28 under consideration.
1-9

1 1.3.2 Exposure Assessment

2 Chapter 3 discusses our plan to conduct a quantitative exposure assessment in this 

3 review. The exposure assessment will build upon the methodology, analyses, and lessons 

4 learned from assessments conducted for other recent NAAQS reviews. EPA plans to model 

5 population exposures to ambient ozone in three or more of the 12 urban areas modeled in the 

6 previous review (Atlanta, Boston, Chicago, Cleveland, Detroit, Houston, Los Angeles, New 

7 York, Philadelphia, Sacramento, St. Louis, and Washington D.C.), as well as a high-elevation 

8 area such as Denver. The number of areas modeled will depend on the available resources. 

9 These areas were selected to be generally representative of a variety of populations, geographic 

10 areas, climates, and different ozone and co-pollutant levels, and are areas where epidemiologic 

11 studies have been conducted that are planned to be used to support the quantitative risk 

12 assessment. In addition to providing population exposures for estimation of lung function 

13 effects, the exposure modeling will provide a characterization of urban air pollution exposure 

14 environments and activities resulting in the highest exposures, differences in which may partially 

15 explain the heterogeneity across urban areas seen in the risks associated with ozone air pollution.

16 1.3.3 Health Risk Assessment

17 The health risk assessment will estimate various health effects associated with ozone 

18 exposures for current ozone levels, based on 2008-2010 air quality data, as well as reductions in 

19 risk associated with attaining the current 8-hour ozone NAAQS and alternative ozone standards, 

20 based on adjusting 2008-2010 air quality data. Risk estimates will be developed for several

21 urban areas located throughout the U.S., including the areas for which exposure modeling will be 

22 performed. Health endpoints to be examined in the risk assessment include: lung function 

23 decrements, respiratory symptoms in asthmatic children, school absences, emergency department 

24 visits for respiratory causes, respiratory- and cardiac-related hospital admissions, and mortality.

25 At this time, two general types of human studies are particularly relevant for deriving 

26 quantitative relationships between ozone levels and human health effects: (1) controlled human 

27 exposure studies and (2) epidemiological and field studies. Controlled human exposure studies 

28 involve volunteer subjects who are exposed while engaged in different exercise regimens to 

29 specified levels of ozone under controlled conditions for specified amounts of time. The 
1-10

1 responses measured in such studies have included measures of lung function, such as forced 

2 expiratory volume in one second (FEV1), respiratory symptoms, airway hyper-responsiveness, 

3 and inflammation. Prior EPA risk assessments for ozone have included risk estimates for lung 

4 function decrements and respiratory symptoms based on analysis of individual data from 

5 controlled human exposure studies. For the current health risk assessment, staff plans to use the 

6 probabilistic exposure-response relationships which were based on analyses of individual data 

7 that describe the relationship between a measure of personal exposure to ozone and the 

8 measure(s) of lung function recorded in the study. The measure of personal exposure to ambient 

9 ozone is typically some function of hourly exposures – e.g., 1-hour maximum or 8-hour

10 maximum. Therefore, a risk assessment based on exposure-response relationships derived from 

11 controlled human exposure study data requires estimates of personal exposure to ozone, typically 

12 on a 1-hour or multi-hour basis. Because data on personal hourly ozone exposures are not 

13 available, estimates of personal exposures to varying ambient concentrations are derived through 

14 exposure modeling, as described in Chapter 3. 

15 The risk assessment based on controlled human exposure studies is described in 

16 Chapter 4. In contrast to the exposure-response relationships derived from controlled human 

17 exposure studies, epidemiological and field studies provide estimated concentration-response

18 relationships based on data collected in real world settings. Ambient ozone concentration is 

19 typically measured as the average of monitor-specific measurements, using population-oriented 

20 monitors. Population health responses for ozone have included population counts of school 

21 absences, emergency room visits, hospital admissions for respiratory and cardiac illness, 

22 respiratory symptoms, and premature mortality. As described more fully below in Chapter 5 and 

23 outlined in Figure 1-1, a risk assessment based on epidemiological studies typically requires 

24 baseline incidence rates and population data for the risk assessment locations.

25 The characteristics that are relevant to the planning and structure of a risk assessment 

26 based on controlled human exposure studies versus one based on epidemiology or field studies 

27 can be summarized as follows:

28 A risk assessment based on controlled human exposure studies uses exposure29 response functions, and thus requires estimates of personal exposures. It therefore 
1-11

1 involves an exposure modeling step that is not needed in a risk assessment based on 

2 epidemiology or field studies, which uses concentration-response functions.

3 Epidemiological and field studies are carried out in specific real world locations (e.g., 

4 specific urban areas). To minimize uncertainty, a risk assessment based on 

5 epidemiological studies can be performed for the locations in which the studies were 

6 carried out. Controlled human exposure studies, carried out in laboratory settings, are 

7 generally not specific to any particular real world location. A controlled human 

8 exposure studies-based risk assessment can therefore appropriately be carried out for 

9 any locations for which there are adequate air quality data on which to base the 

10 modeling of personal exposures. There are, therefore, some locations for which a 

11 controlled human exposure studies-based risk assessment could appropriately be 

12 carried out but an epidemiological studies-based risk assessment could not, according 

13 to our criteria for city selection.

14 The adequate modeling of hourly personal exposures associated with ambient 

15 concentrations requires more complete ambient monitoring data than are necessary to 

16 estimate average ambient concentrations used to calculate risks based on 

17 concentration-response relationships. Therefore, there may be some locations in 

18 which an epidemiological studies-based risk assessment could appropriately be 

19 carried out but a controlled human exposure studies-based risk assessment would 

20 have increased uncertainty.

21 To derive estimates of risk or risk reduction from concentration-response 

22 relationships estimated in epidemiological studies, it is usually necessary to have 

23 estimates of the baseline incidences of the health effects involved. Such baseline 

24 incidence estimates are not needed in a controlled human exposure studies-based risk 

25 assessment.

26 Overviews of the scope and methods for each type of risk assessment are discussed in 

27 Chapters 4 and 5 below.
1-12

Overview of Risk Assessment Model

Air Quality

•Recent air quality

•Air quality simulated to just 

meet current and alternative 

NAAQS

•Policy relevant background

Concentration-Response

•C-R functions derived from 

epidemiological studies for 

various health endpoints

Baseline Incidence and Demographics

•Baseline health effects incidence rates

•Population data

Health

Risk

Model

Risk Estimates

•Recent air quality

•Current or alternative 

NAAQS scenarios

1

2

3 Figure 1-1. Overview of Risk Assessment Based on Epidemiologic Studies

4
2-1

1 2 AIR QUALITY CONSIDERATIONS

2 2.1 Introduction

3 A number of air quality analyses are planned to provide inputs for the risk and exposure 

4 assessments that will be conducted for selected urban study areas as well as to provide a broader 

5 understanding of ozone air quality, in order to inform: (1) evidence-based considerations; (2) 

6 our understanding of the risk and exposure assessment results to better characterize potential 

7 nationwide public health impacts associated with exposures to ozone; and (3) policy 

8 considerations related to evaluating possible alternative NAAQS. Specific goals for the planned 

9 air quality assessments include: 

10 Characterizing air quality in various locations across the U.S. in terms of ozone

11 considering differences in ozone ambient concentrations, and spatial and temporal 

12 patterns to help inform the selection of specific cities that we plan to include in the risk 

13 and exposure assessments. 

14 Characterizing background concentrations of ozone based on chemical transport 

15 modeling (U.S. EPA, 2011b, section 3.4).

16 Providing air quality distributions for ozone for a number of alternative scenarios in the 

17 selected urban study areas including:

18 o Recent air quality;

19 o Simulation of air quality to just meet the current primary standard; and 

20 o Simulation of air quality to just meet potential alternative primary standards 

21 for ozone under consideration.

22 Providing a broader characterization of current ozone concentrations nationally (beyond 

23 the locations evaluated in the risk and exposure assessments).

24 2.2 Air Quality Inputs to Risk and Exposure Assessments

25 Important inputs to the ozone risk and exposure assessments are ambient ozone air 

26 quality data. For these assessments, EPA plans to use 2008-2010 air quality data obtained from 

27 EPA’s Air Quality System (AQS), as these are the most recent data available.

28 2.2.1 Recent Air Quality

29 For ozone, in general, only data collected by Federal reference or equivalent methods 

30 (FRMs or FEMs) will be used in the risk and exposure assessments, consistent with the use of 
2-2

1 such data in most of the health effects studies. However, if an epidemiologic study used non2 FRM/FEM data from ozone monitors in a concentration-response function, consideration will be 

3 given to using the same type of data in the quantitative risk assessment for the same location. In 

4 order to be consistent with the approach generally used in the epidemiological studies that used 

5 estimated ozone concentration-response (C-R) functions for short-term effects, we plan to 

6 average ambient ozone concentrations on each day for which measured data are available for 

7 estimating health effects associated with 24-hour ambient concentrations. If epidemiologic 

8 studies used a composite monitor, then we will consider developing a data set for each 

9 assessment location based on a composite of all monitors according to the method in the 

10 epidemiologic study. As in the last review, some monitoring sites may be omitted, if needed, to 

11 best match the set of monitors that were used in the epidemiological studies.

12 In addition to matching our characterization of air quality at each assessment location to 

13 the approach from the epidemiological study used in modeling risk, we will also consider

14 alternative approaches for characterizing air quality, if they produce estimates of exposure that 

15 are potentially more representative for the populations being assessed (even if they do not match 

16 the approach used in the epidemiology studies). For example, we may consider the use of 

17 monitor data (as described above) fused with photochemical modeling results for ozone. With 

18 this approach, we would use the monitor data to characterize absolute ozone levels across the 

19 urban study area (subject to the limitations of the monitoring framework’s coverage) with the 

20 modeled results used to fill in the spatial pattern or gradient between monitors. We may also 

21 consider alternative approaches for generating composite monitor estimates that do not rely on 

22 modeling. For example, given the potential importance of commuting and workplace exposure in 

23 driving overall exposure profiles, we might consider generating composite monitor estimates 

24 that weight each monitor by ―population exposure‖ (e.g., the person-hours of exposure 

25 associated with the immediate area surrounding a given monitor). With this approach, we would 

26 use the results of micro-environmental exposure modeling to estimate the amount of time that a 

27 simulated population spends in the vicinity of each monitor (see Section 5.2.2 for additional 

28 detail on these alternative approaches being considered for assessing current exposure). 
2-3

1 Important factors to consider in deciding how to characterize current ambient ozone 

2 levels include the degree of spatial and temporal heterogeneity in monitored levels seen within a 

3 given assessment location. As part of planning for the analysis, we will consider trends in spatial 

4 and temporal gradients across ozone monitor data in the urban study areas we are considering. 

5 2.2.2 Air Quality Data Related to Exceptional Events

6 State and local agencies and EPA are in the process of reviewing ozone data for purposes 

7 of making decisions regarding the exclusion of data under the Exceptional Events Rule. We will

8 include these decisions regarding specific data that should be excluded from consideration when 

9 determining the amount of rollback of air quality to meet the current or alternative ozone

10 standards. 

11 2.3 Development of Estimates of Ozone Air Quality Assuming “Just Meeting” Current 

12 NAAQS and Potential Alternative NAAQS

13 2.3.1 Background and Conceptual Overview

14 In order to simulate air quality concentrations that ―just meet‖ the current or potential 

15 alternative ozone standards in a study area, we consider what mathematical approach (commonly 

16 referred to as rollback) should be used to transform recent air quality into profiles of adjusted air 

17 quality that simulate just meeting the current or alternative standards under consideration. 

18 The challenge in developing estimates of ozone air quality for a scenario in which an 

19 assessment location is ―just meeting‖ the current standards or alternative standards under 

20 consideration is to estimate as realistically as possible how concentrations for all hours at all 

21 monitors will be affected, not just how the design value from the controlling monitor (or set of 

22 monitors being averaged) will be affected. The definition of ―just meeting‖ alternative ozone

23 standards uses the same approach as ―just meeting‖ the current standards. 

24 There are many possible ways to create characterizations of air quality to 

25 represent scenarios ―just meeting‖ specified ozone standards. The previous two reviews have 

26 used a method called quadratic rollback, which is described below in section 2.3.2. This choice 

27 was based on analyses of historical ozone data which found, from comparing the reductions over 

28 time in daily ambient ozone levels in two locations with sufficient ambient air quality data, that 
2-4

1 reductions tended to be roughly quadratic (Abt Associates, 2005, Appendix B). We recognize 

2 that the pattern of changes that have occurred in the past may not necessarily reflect the temporal 

3 and spatial patterns of changes that would likely result from future efforts to attain the ozone

4 standards; therefore, we are considering examining an alternative prospective approach for 

5 rollback, as described in section 2.3.3.

6 2.3.2 Historical Approach 

7 Prior ozone risk assessments simulated ozone reductions that would result from just 

8 meeting a set of standards using a quadratic adjustment (―quadratic rollback‖) which decreased

9 non-background ozone levels on all hours for all concentrations exceeding the policy-relevant 

10 background (PRB). The portion of the distribution below the estimated PRB concentration was 

11 not rolled back, since air quality strategies adopted to meet the standards would not be expected 

12 to reduce the PRB contribution to ozone concentrations. The percentage amount of rollback was 

13 just enough so that the standard under consideration was not exceeded.

14 In the risk assessment for this review, we will again evaluate the quadratic rollback 

15 approach by comparing it with historical changes in distributions of ozone concentrations in 

16 selected locations. Specifically, EPA plans to evaluate historical ozone air quality changes to 

17 assess the implications of using a quadratic rollback approach. We also plan to consider the 

18 premises and outcomes of the quadratic rollback approach against our insights regarding known 

19 and likely future emission reductions, e.g., whether it is reasonable to expect that future patterns 

20 of changes in ozone air quality would generally be similar to historical patterns of changes in air 

21 quality. 

22 2.4 Policy Relevant Background

23 A key issue to be addressed in the ozone Policy Assessment is the characterization of 

24 policy-relevant background ozone levels in the U.S. Historically, PRB has been defined as the 

25 distribution of ozone concentrations that would be observed in the U.S. in the absence of 

26 anthropogenic (man-made) emissions of ozone precursors in the U.S., Canada, and Mexico. This 

27 definition allows for analyses that focus on the effects and risks associated with pollutant levels 
2-5

1 that have the potential to be controlled by U.S. regulations, through international agreements 

2 with border countries, or by voluntary emissions reductions in the U.S. and elsewhere.

3 For this assessment, we are planning to estimate concentrations for different background 

4 scenarios using the global-scale chemistry-transport model GEOS-Chem (Bey et al., 2001) to 

5 inform a discussion of how to characterize PRB. The GEOS-Chem model is run on a global 

6 scale and will be used to provide estimates of transported pollutants from emissions of natural 

7 and anthropogenic sources from various geographic areas. The details of this modeling approach

8 are briefly summarized below.

9 The GEOS-Chem modeling system will be run using emissions and meteorological data 

10 for three annual periods (2006, 2007, 2008). EPA staff is considering how to best use these 

11 model results in the exposure and risk analyses, which will be based on 2008 – 2010 air quality. 

12 The GEOS-Chem model will be run using two nested grids. The outer grid will be global in 

13 extent and utilize a grid resolution of 2.0 by 2.5 degrees. The inner grid will be centered over 

14 North America, cover the area from 140-40W / 10-70N, and use a horizontal resolution of 0.50 

15 by 0.67 degrees. Four scenarios will be modeled. First, a current atmosphere (base case) 

16 simulation will be completed using all global anthropogenic and natural emissions sources. A 

17 model performance evaluation will be completed for this scenario using surface air quality 

18 measurements and satellite estimates of atmospheric air pollutant concentrations.

19 In addition to the ―current atmosphere‖ or base case run which includes all anthropogenic 

20 and biogenic emissions, GEOS-Chem will be run for three additional emissions scenarios to 

21 isolate the contributions of internationally transported air pollutants to ozone concentrations in 

22 the U.S.:

23 A simulation in which U.S. anthropogenic emissions of nitrogen oxides (NOX), non24 methane volatile organic compounds (nMVOC), and carbon monoxide (CO) are set to 

25 zero, while anthropogenic emissions outside of the U.S. are maintained at their 

26 current levels.

27 A simulation in which U.S., Canada, and Mexico anthropogenic emissions of NOX, 

28 nMVOC, and CO are set to zero, while anthropogenic emissions outside of these 

29 areas are maintained at their current levels. This was referred to as policy relevant 

30 background (PRB) in the previous review of the ozone NAAQS.
2-6

1 A simulation in which global anthropogenic emissions of NOX, nMVOC, and CO are 

2 set to zero.

3 These simulations will allow us to quantify the contribution of U.S. anthropogenic, 

4 Canada and Mexico anthropogenic, international (excluding Canada and Mexico) anthropogenic, 

5 and natural ozone sources to U.S. ozone health risk individually. Since emissions of methane are 

6 at current levels for all of these simulations, we will consider differentiating the contribution of 

7 global methane emissions from natural sources using recently published modeling studies that 

8 examine the effect of perturbations in the methane mixing ratio on global and U.S. air quality 

9 (Fiore et al., 2008, 2009).

10 A growing body of observational and modeling studies suggests that the international 

11 anthropogenic contribution to U.S. background ozone levels is substantial and is expected to rise 

12 in the future as rapid economic development continues around the world. Of particular concern 

13 is rising Asian emissions of nitrogen oxides (NOx), which can influence U.S. ozone 

14 concentrations in the near-term, and methane, which affects background ozone concentrations 

15 globally over decadal time scales. The model simulations of current anthropogenic emissions 

16 described above will not allow for projections of future ozone background concentrations nor the 

17 contribution from specific global methane sources on ozone in the present. However, a large 

18 multi-model ensemble assessment convened by the Task Force on Hemispheric Transport of Air 

19 Pollution (TF HTAP) has produced estimates that may be informative for estimating future 

20 global background concentrations and concentrations transported from upwind regions. In 

particular, the HTAP 2010 Assessment Report121 estimated that the contribution of NOx, non22 methane VOC, and CO emissions in Europe, South Asia, and East Asia to North American 

23 ozone concentrations at relatively unpolluted sites is 32% of the contribution of emissions from 

24 all four regions (including North America) combined. That contribution is projected to rise to 

25 49% in a conservative emissions growth scenario and to 52% in a scenario of aggressive global 

26 economic development. The report also concluded that approximately 40% of the mean global 

27 ozone increase since the preindustrial era is due to methane, and that rising global methane 

28 emissions will have a large influence on future U.S. ozone concentrations. These results may be 

 

1 Available at http://www.htap.org/
2-7

1 used to inform estimates of growth of international transport in the future and how those changes 

2 might affect our estimation of future ozone health risks.

3 2.5 Broader Air Quality Characterization

4 Information presented in the REA will draw upon air quality data analyzed in the ISA as 

5 well as national and regional trends in air quality as evaluated in EPA’s Air Quality Status and 

6 Trends document (U.S. EPA, 2008a), and EPA’s Report on the Environment (U.S. EPA, 2008b). 

7 We plan to use this information, and additional analyses, as needed, to develop a broad 

8 characterization of current air quality across the nation. For example, tables of areas and 

9 population in the U.S. exceeding current ozone standards and potential alternative standards will

10 be prepared. Additional information will be generated on the expected number of days on which 

11 the ozone standards are exceeded, adjusting for the number of days monitored. Further, ozone

12 levels in locations and time periods relevant to areas assessed in key short-term epidemiological 

13 studies discussed further in Section 5.3.2 will be characterized. Information on the spatial and

14 temporal characterization of ozone across the national monitoring network will be compiled. To 

15 the extent possible, we plan to compare these data to the same parameters in the selected urban 

16 study areas considered in the quantitative risk assessment to help place the results of that 

17 assessment into a broader context.
3-1

1 3 APPROACH FOR POPULATION EXPOSURE ANALYSIS

2 3.1 Introduction

3 Population exposure to ambient ozone levels will be evaluated using the current version 

4 of the Air Pollutants Exposure (APEX) model, a model based on the current state of knowledge 

5 of inhalation exposure modeling. Exposure estimates will be developed for current ozone levels, 

6 based on 2008-2010 air quality data, and for ozone levels associated with just meeting the 

7 current 8-hour ozone NAAQS and alternative ozone standards, based on adjusting 2008-2010 air 

8 quality data. Exposure estimates will be modeled for 3 to 12 urban areas located throughout the 

9 U.S. for 1) the general population, 2) school-age children (ages 5 to 18), 3) asthmatic school-age 

10 children, 4) outdoor workers, and 5) the elderly population (aged 70 and older). This choice of 

11 population groups includes a strong emphasis on children, which reflects the results of the last 

12 review in which children, especially those who are active outdoors, were identified as the most 

13 important at-risk group.

14 The exposure estimates will be used as an input to that part of the health risk assessment 

15 that is based on exposure-response relationships derived from controlled human exposure 

16 studies, discussed in Section 4.3 below. The exposure analysis will also provide information on 

17 population exposure exceeding levels of concern that are identified based on evaluation of health 

18 effects that are not included in the quantitative risk assessment. It will also provide a 

19 characterization of populations with high exposures in terms of exposure environments and 

20 activities.

21 3.2 The APEX Population Exposure Model

22 APEX, also referred to as the Total Risk Integrated Methodology/Exposure (TRIM.Expo) 

23 model (U.S. EPA, 2008c,d), has its origins in the NAAQS Exposure Model (NEM), which was 

24 developed in the early 1980’s (Biller et al., 1981; McCurdy, 1994, 1995). APEX simulates the 

25 movement of individuals through time and space and their exposure to a given pollutant in 

26 indoor, outdoor, and in-vehicle microenvironments. Figure 3-1 provides a schematic overview 

27 of the APEX model. The model stochastically generates simulated individuals using census28 derived probability distributions for demographic characteristics (Figure 3-1, steps 1-3). The 
3-2

1 population demographics are from the 2000 Census data at the tract or block level, and a national 

2 commuting database based on 2000 Census data provides home-to-work commuting flows 

3 between tracts. A large number of simulated individuals are modeled, and collectively, they 

4 represent a random sample of the study area population.

5 Diary-derived time activity data are used to construct a sequence of activity events for 

6 each simulated individual consistent with the individual’s demographic characteristics and 

7 accounting for effects of day type (e.g., weekday, weekend) and outdoor temperature on daily 

8 activities (Figure 3-1, step 4). APEX calculates the concentration in the microenvironment 

9 associated with each event in an individual’s activity pattern and sums the event-specific

10 exposures within each hour to obtain a continuous time series of hourly exposures spanning the 

11 time period of interest (Figure 3-1, steps 5 and 6). From these exposure estimates, APEX 

12 calculates exposures for averaging times greater than one hour.
- National

database

- Area-specific

input data

- Intermediate step

or data

- Simulation

step

- Data processor - Output data

Sector location 

data

(latitude, longitude)

Defined study area (sectors within a city radius and with air quality and meteorological data within their radii of influence)

Sector population data 

(age/gender/race)

Population within 

the study area

Commuting flow data

(origin/destination sectors)

Age/gender-specific physiological distribution data (body weight, height, etc)

Distribution functions forprofile variables (e.g, probability of air conditioning)

Locations of air quality and 

meteorological measurements; 

radii of influence

Stochastic 

profile generator

Distribution functions for seasonal and daily varying profile variables (e.g., window status, car speed)

A sfoll• H• W• A• G• Ra• Em• H• H• H• Ca• Ph

(h

2000 Census tract-level data for the entire U.S. (sectors=tracts for the NAAQS ozone exposure application)

Age/gender/tract-speci

employment probabilit

1. Characterize study area 2. Characterize study population 3. Generate N nu

simulated individua

Figure 3-1. Overview of the APEX Model

3-3
Diary events/activities 

and personal information 

(e.g., from CHAD)

Maximum/mean daily

temperature data

Activity diary pools by day 

type/temperature category

Each day in the simulation period is assigned to an activity pool based on day type and temperature category

Selected diary records for each day in the simulation period, resulting in a sequence of events(microenvironments visited, minutes spent, and activity) in the simulation period, for an individual

Stochastic diary 

selector using 

age, gender, and 

employment

Stochastic

calculation of energy

expended per event (adjusted for

physiological limits and EPOC)

and ventilation

rates

Physiological 

parameters from 

profile

Sequence of 

indi

Profile for an 

individual

4. Construct sequence of activity events 

for each simulated individual

Figure 3-1. Overview of the APEX Model, continued

3-4
Hourly air quality 

data for all sectors

Select calculation method for each microenvironment:• Factors• Mass balance

Concentrations for all events 

for each simulated individual

Hourly concentrations and minutes spent in each

microenvironment visited by 

the simulated individual

Average exposuresfor simulated person, stratified by ventilation rate:• Hourly• Daily 1-hour max• Daily 8-hour max• Daily... 

Microenvironments defined by 

grouping of CHAD location codes

Calculate hourly

concentrations in

microenvironments

visited

Populatioindicator• Total po• Childre• Asthma

Sequence of events for 

each simulated individual

Calculate

concentrations in all

microenvironments

5. Calculate concentrations in 

microenvironments for all events for 

each simulated individual

6. Calculate hourly 

exposures for each 

simulated individual

7. 

popula

s

Figure 3-1. Overview of the APEX Model, concluded

3-5
3-6

1 APEX employs a flexible approach for simulating microenvironmental concentrations, 

2 where the user can define any number of microenvironments to be modeled and their 

3 characteristics. For this modeling application, we propose modeling the microenvironments 

4 listed in Table 3-1.

5 Table 3-1. Microenvironments to be Modeled

Microenvironment Method

Indoors – residences mass balance

Indoors – restaurants mass balance

Indoors – schools mass balance

Indoors – offices mass balance

Indoors – shopping mass balance or factors

Indoors – other mass balance or factors

Outdoors – public garages and parking lots factors

Outdoors – near road (walking, bicycling) factors

Outdoors – other (e.g., playgrounds, parks) factors

In vehicle – cars and light trucks mass balance or factors

In vehicle – heavy trucks mass balance or factors

In vehicle – school buses mass balance or factors

In vehicle – mass transit vehicles – buses and trolleys factors

In vehicle – mass transit vehicles – underground (subways) factors

6 We plan to calculate the concentrations in each microenvironment using either a factors 

or mass-balance approach17 , depending upon data availability, with probability distributions

8 representing variability of the parameters that enter into the calculations (e.g., indoor-outdoor air 

9 exchange rates) supplied as inputs to the model. These distributions represent the variability (not 

10 uncertainty) of parameters, and can vary spatially and can be set up to depend on the values of 

11 other variables in the model. For example, the distribution of air exchange rates in a home, 

 

1 The factors and mass-balance approaches are described in section 3.6.5.
3-7

1 office, or car depends on the ambient temperature and the type of heating and air conditioning 

2 present. The value of a stochastic parameter can be kept constant for an individual for the entire 

3 simulation (e.g., house volume), or a new value can be drawn hourly, daily, or seasonally from 

4 specified distributions. APEX also allows the specification of diurnal, weekly, and seasonal 

5 patterns for microenvironmental parameters.

6 3.3 Populations Modeled

7 A detailed consideration of the population residing in each modeled area will be included. 

8 The exposure assessment will include the general population residing in each area modeled as 

9 well as susceptible and vulnerable populations as identified in the ISA. The population groups 

10 that we plan to include in the exposure assessment are:

11 The general population

12 School-age children (ages 5 to 18)

13 Asthmatic school-age children

14 Outdoor workers

15 The elderly population (aged 70 and older)

16 Due to the increased amount of time spent outdoors engaged in relatively high levels of 

17 physical activity, school-age children as a group are particularly at risk for experiencing ozone18 related health effects as a result of to their increased dose rates. The proportion of the population 

19 of school-age children characterized as being asthmatic will be estimated by statistics on asthma 

20 prevalence rates from the National Health Interview Survey (CDC, 2010) and other sources.

21 3.4 Outcomes to be Generated

22 There are several useful indicators of exposure of people to various levels of air 

23 pollution. Factors that are important in defining such indicators include the magnitude and 

24 duration of exposures, frequency of repeated high exposures, and ventilation rate (i.e., breathing 

25 rate) of the individual at the time of exposure. In this analysis, exposure indicators will include 

26 daily maximum 1- and 8-hour average ozone exposures, stratified by equivalent ventilation rates 

27 (i.e., ventilation normalized by body surface area). 

28 APEX calculates two general types of exposure estimates: counts of people and person29 occurrences. The former counts the number of individuals exposed one or more times per ozone 
3-8

1 season to the exposure indicator (e.g., exposure level and ventilation rate) of interest. In the case 

2 where the exposure indicator is a benchmark concentration level, the model estimates the number 

3 of people who experience that level of air pollution, or higher, at least once during the modeled 

4 period. The person-occurrences measure counts the number of times per ozone season that an 

5 individual is exposed to the exposure indicator of interest and then accumulates counts over all 

6 individuals. Therefore, the person-occurrences measure conflates people and occurrences: using 

7 this measure, 1 occurrence for 10 people is counted the same as 10 occurrences for 1 person.

8 Analyses of the APEX results will provide distributions of the numbers of people with 8-

9 hour average exposure above benchmark levels of 0.06, 0.07, and 0.08 ppm-8 hours, 

10 distributions of the numbers of people with lung function decrements above 10, 15, and 20 

11 percent decreases in FEV1, and characterization of the attributes of highly exposed individuals.

12 3.5 Selection of Urban Areas and Time Periods

13 EPA plans to model population exposures to ambient ozone in three or more of the 12 

14 urban areas modeled in the previous review (Atlanta, Boston, Chicago, Cleveland, Detroit, 

15 Houston, Los Angeles, New York City, Philadelphia, Sacramento, Seattle, St. Louis, 

16 Washington, D.C.) and a high-altitude city, such as Denver. These were selected to be generally 

17 representative of a variety of populations, geographic areas, climates, and different ozone and co18 pollutant levels, and are areas where epidemiologic studies have been conducted that are planned 

19 to be used to support the quantitative risk assessment.

20 The exposure periods to be modeled will be the ozone-monitoring seasons for each urban 

21 area. These encompass the periods when high ambient ozone levels are likely to occur, and are 

22 the periods for which routine hourly ozone monitoring data are available. The ozone seasons for 

23 the selected study areas generally range from April through either September or October for most 

24 of the locations in the eastern U.S. to all year in locations in southern California and Texas. 

25 3.6 Development of Model Inputs

26 In this section, we describe the plan for developing the inputs to the APEX model.
3-9

1 3.6.1 Population Demographics

2 We plan to use tract-level population counts from the 2000 Census of Population and 

Housing Summary File 113 . Summary File 1 (SF 1) contains the 100-percent data, which is the 

4 information compiled from the questions asked of all people and about every housing unit.

In the 2000 U.S. Census, estimates of employment were developed by census tract25 . The 

6 file input to APEX will be broken down by gender and age group, so that each gender/age group 

7 combination is given an employment probability fraction (ranging from zero to 1) within each 

8 census tract. The age groupings in this file are: 16-19, 20-21, 22-24, 25-29, 30-34, 35-44, 45-54, 

9 55-59, 60-61, 62-64, 65-69, 70-74, and greater than 75 years of age. Children under 16 years of 

10 age will be assumed to be not employed.

11 3.6.2 Commuting

12 As part of the population demographics inputs, it is important to integrate working 

13 patterns into the assessment. In addition to using estimates of employment by tract, APEX also 

14 incorporates home-to-work commuting data. We plan to use the national commuting database 

15 provided with APEX in this analysis. Commuting data were derived from the 2000 Census and 

16 were collected as part of the Census Transportation Planning Package (CTPP) (U.S. DOT, 

2000)317 . The data used to generate APEX inputs were taken from the ―Part 3-The Journey To 

18 Work‖ files. These files contain counts of individuals commuting from home to work locations 

19 at a number of geographic scales. These data have been processed to calculate fractions for each 

20 tract-to-tract flow to create the national commuting data distributed with APEX. This database 

21 contains commuting data for each of the 50 states and Washington, D.C. This data set does not 

22 differentiate people that work at home from those that commute within their home tract.

23 3.6.3 Ambient Ozone Concentrations

24 We plan to conduct exposure modeling based on ozone concentrations measured at 

25 ambient air monitors in and near the areas being modeled. Sources for these data include the 

 

1

 http://www.census.gov/prod/cen2000/doc/sf1.pdf

2 Employment data from the 2000 Census can be found on the U.S. Census web site:

http://www.census.gov/population/www/cen2000/phc-t28.html (Employment Status: 2000- Supplemental Tables). 

3 These data are available from the U.S. DOT Bureau of Transportation Statistics (BTS) at the web site:

http://transtats.bts.gov/. 
3-10

1 hourly concentration measurements from the monitoring data maintained in EPA’s Air Quality 

2 System (AQS).

3 3.6.4 Meteorological Data

4 Surface meteorological observations will be obtained from the National Climatic Data 

Center15 to provide hourly temperatures for input to APEX. We plan to use all meteorological 

6 stations within and nearby each selected urban study area. 

7 3.6.5 Specification of Microenvironments

8 Parameters defining each microenvironment will be specified by distributions which 

9 reflect the variability of these parameters. The parameters needed depend on whether a 

10 microenvironment is modeled using the factors model or the mass balance model.

11 We plan to use the factors model to model simple environments, like outdoor areas, that 

12 do not contain pollutant sources, or microenvironments for which data are not available to use 

13 the mass-balance model. Two parameters affect the pollutant concentration calculation in the 

14 factors method, the proximity and infiltration factors. The proximity factor (FPR) is a unitless 

15 parameter that represents the relationship of the ambient concentration outside of the 

16 microenvironment (CO) to the concentration at a monitoring station (CA) by the equation CO = 

17 FPR CA. The infiltration factor (Finf) is a unitless parameter that represents the equilibrium

18 fraction of pollutant entering a microenvironment from outside the microenvironment. The 

19 concentration inside the microenvironment (CI) is estimated by the equation CI = Finf CO. The

20 infiltration factor in the factors model is often expressed as:

a k

Pa Finf

21

22 where P is a penetration coefficient, a is an air exchange rate, and k is a loss rate. APEX draws 

23 values of these parameters from microenvironment-specific distributions specified by the user, to 

24 model the stochastic nature of these factors.

25 The mass balance model is more appropriate for complex environments. The mass 

26 balance method assumes that an enclosed microenvironment (e.g., a room in a residence) is a 

 

1

See http://www.ncdc.noaa.gov/oa/ncdc.html
3-11

1 single well-mixed volume in which the air concentration is approximately spatially uniform. 

2 APEX estimates the concentration of an air pollutant in such a microenvironment by using the 

3 following four processes (as illustrated in Figure 3-2):

4 Inflow of air into the microenvironment;

5 Outflow of air from the microenvironment;

6 Removal of a pollutant from the microenvironment due to deposition, filtration, and chemical 

7 degradation; and 

8 Emissions from sources of a pollutant inside the microenvironment.

9 Figure 3-2. The Mass Balance Model

10 Considering the microenvironment as a distinct, well-mixed volume of air, the mass 

11 balance relationship for a pollutant can be described by:

dt

dC t

dt

dC t

dt

dC t

dt

dC t

dt

dC t

in out loss source ( ) ( ) ( ) ( ) ( )

12

13 where:

C(t) = Concentration in the microenvironment at time t (μg/m314 )

dt

dC t

in ( )

15 = Rate of change in C(t) due to air entering the micro

Air

inflow

Indoor sources

Removal due to:

•Chemical reactions

•Deposition

•Filtration

Air

outflow

Microenvironment

Air

inflow

Indoor sources

Removal due to:

•Chemical reactions

•Deposition

•Filtration

Air

outflow

Microenvironment
3-12

dt

dC t

out ( )

1 = Rate of change in C(t) due to air leaving the micro

dt

dC t

loss ( )

2 = Rate of change in C(t) due to all removal processes

dt

dC t

source ( )

3 = Rate of change in C(t) due to all source terms

4 In addition to proximity factors, this method supports parameter distributions for time 

5 varying emissions sources, decay rate, air exchange rate, volume, and removal rate. We plan to 

6 estimate the distributions of these microenvironment-specific parameters based on available data 

7 and a review of the literature.

8 3.6.6 Indoor Sources

9 We are considering modeling indoor sources of ozone in this analysis, although our focus 

10 is on exposure to ozone of ambient origin. Indoor sources of ozone would not be subject to 

11 ―rollback.‖

12 3.6.7 Activity Patterns

13 Exposure models use human activity pattern data to predict and estimate exposure to 

14 pollutants. Different human activities, such as outdoor exercise, indoor reading, or driving, have 

15 different pollutant exposure characteristics. In addition, different human activities require 

16 different metabolic rates, and higher rates lead to higher doses. To accurately model individuals 

17 and their exposure to pollutants, it is critical to have a firm understanding of their daily activities.

18 The Consolidated Human Activity Database (CHAD) provides data on human activities 

19 through a database system of collected human diaries, or daily activity logs (McCurdy et al., 

20 2000; U.S. EPA, 2002; Graham and McCurdy, 2004). The purpose of CHAD is to provide a 

21 basis for conducting multi-route, multi-media exposure assessments (McCurdy et al., 2000). The 

22 data contained within CHAD come from multiple surveys with varied structures (Table 3-2). In 

23 general, the surveys have a data foundation based on daily diaries of human activity. Individuals 

24 filled out diaries of their daily activities and this information was entered and stored in CHAD. 

25 Relevant data for these individuals, such as age, are included as well. In addition, CHAD 

26 contains activity-specific metabolic distributions developed from literature-derived data, which 

27 are used to provide an estimate of metabolic rates of respondents through their various activities.
3-13

1 The locations used in the CHAD diaries must be assigned appropriately to the APEX 

2 microenvironments listed in Table 3-1. Each of the microenvironments is designed to simulate 

3 an environment in which people spend time during the day. There are many more CHAD 

4 locations than microenvironments being modeled (there are over 100 CHAD locations and 14

5 proposed microenvironments modeled in this assessment) thus, most of the microenvironments 

6 have multiple CHAD locations mapped to them.
3-14

1 Table 3-2. Studies In CHAD To Be Used For Exposure Modeling

Study name Geographic 

coverage

Study time 

period

Subject

ages

Diary−

days

Number of 

subjects

Diary type and 

study design

Reference

Baltimore One building 

in Baltimore

1/1997−2/1997, 

7/1998−8/1998

72 − 93 391 26 Diary Williams et al. (2000)

California Adults 

(CARB)

California 10/1987−9/1988 18 − 94 1,552 1,552 Recall (next day 

telephone survey); 

Random

Robinson et al. (1989), 

Wiley et al. (1991a)

California Children 

(CARB)

California 04/1989−2/1990 <1 − 11 1,200 1,200 Recall (next day 

telephone survey); 

Random

Wiley et al. (1991b)

California 

Adolescents (CARB)

California 10/1987−9/1988 12 − 17 181 181 Recall (next day 

telephone survey); 

Random

Robinson et al. (1989), 

Wiley et al. (1991a)

Cincinnati (EPRI) Cincinnati 

metro. area

3/1985, 8/1985 <1 − 86 2,601 884 Diary; Random Johnson (1989)

Denver (EPA) Denver 

metro. area

11/1982−2/1983 18 − 70 798 438 Diary; Random Johnson (1984), Akland 

et al. (1985)

Los Angeles: 

Elementary School

Los Angeles 10/1989 10 − 12 49 17 Diary Spier et al. (1992)

Los Angeles: High 

School

Los Angeles 10/1990 13 − 17 42 19 Diary Spier et al. (1992)

NHAPS2−Air National 9/1992−9/1994 <1 − 93 4,338 4,338 Recall; Random Klepeis et al. (1996), 

Tsang and Klepeis 

(1996)

NHAPS−Water National 9/1992−9/1994 <1 − 93 4,349 4,349 Recall; Random Klepeis et al. (1996), 

Tsang and Klepeis 

(1996)
3-15

Study name Geographic 

coverage

Study time 

period

Subject

ages

Diary−

days

Number of 

subjects

Diary type and 

study design

Reference

PSID CDS3I National 3/1997−6/1997, 

9/1997−12/1997

<1 − 13 4,989 2,706 Diary; Random Hofferth et al. (1999)

PSID CDS II National 10/2002−6/2003 5 − 19 4,774 2,505 Diary; Random Mainieri et al. (2004)

Seattle Seattle, WA 12/2000−5/2001 6 − 91 1,688 178 Diary Liu et al. (2003)

RTI Ozone Averting 

Behavior

National 7/2002–9/2002 2 – 12 2,882 773 Recall; Random Mansfield and Corey 

(2003), Mansfield et al. 

(2004; 2006)

RTP Panel Chapel Hill, 

Raleigh, NC

6/2000−5/2001 55 − 85 1,000 37 Diary Williams et al. (2003a,b)

RTI NSAS 8 cities46/2009–9/2009 35-92 4,383 1,194 Recall; Random Knowledge Networks 

(2009)

Washington, D.C. Wash., D.C. 

metro. area

11/1982−2/1983 18 − 98 699 699 Diary; Random Hartwell et al. (1984), 

Akland et al. (1985)

Totals 35,916 21,096

1 NOTE: The counts in this table refer to subsets of the studies in CHAD for which data are suitable for use in APEX.

2

2 National Human Activity Pattern Survey. http://www.exposurescience.org/NHAPS

3

3 The Panel Study of Income Dynamics, Child Development Supplement. http://psidonline.isr.umich.edu/ 

4

4 Atlanta, Chicago, Dallas, Houston, Philadelphia, Sacramento/San Joaquin, St. Louis, Washington D.C.
3-16

1 3.7 Exposure Modeling Issues 

2 In this section, we highlight some aspects of the proposed exposure modeling that have 

3 the potential to significantly contribute to uncertainties in the exposure analysis. These aspects 

4 of people’s exposures are either not modeled or are based on limited information.

5 Representativeness of Personal Activity Patterns

6 The human activity data will be drawn from the CHAD developed and maintained by the 

7 Office of Research and Development’s (ORD) National Exposure Research Laboratory 

8 (NERL). The CHAD includes data from several surveys covering specific time periods 

9 at city, state, and national levels, with varying degrees of representativeness. The extent 

10 to which the human activity database provides a balanced representation of the 

11 population being modeled varies across areas. Although the algorithm that constructs 

12 activity sequences attempts to account for the effects of population demographics and 

13 local climate on activity, this adjustment procedure does not fully account for all intercity 

14 differences in people's activities. Activity patterns are affected by many local factors, 

15 including topography, land use, traffic patterns, mass transit systems, and recreational 

16 opportunities. If time and resources permit, to improve the representativeness of the 

17 activity patterns, diaries from the American Time Use Survey (Bureau of Labor Statistics, 

18 2010; Tudor-Locke et al., 2009, 2010) will be included in the activity pattern database 

19 used by APEX. The American Time Use Survey (ATUS) provides nationally 

20 representative estimates of how and where Americans spend their time. The ATUS data 

21 files include information collected from over 98,000 interviews conducted from 2003 to 

22 2009. The current CHAD database has about 40,000 diaries that APEX can use (Table 

23 3-2), so this has the potential to be a significant improvement. The ATUS data are 

24 collected through telephone interviews asking about the previous day's activities, and 

25 measure the amounts of time people that spend doing various activities, such as work, 

26 childcare, housework, watching television, exercising, and socializing.

27 Longitudinal Personal Activity Patterns

28 In the previous review, it was found that APEX significantly underestimates the 

29 frequency of occurrence of individuals experiencing repeated 8-hour average exposures 

30 greater than 0.06, 0.07, and 0.08 ppm (Langstaff, 2007). The assignment of activity 

31 diaries to individuals is the primary determinant of the frequency of repeated exposures 

32 for individuals. This is an important consideration, since multiple exposures pose a 

33 greater health concern than single exposures. The current methodology for the 

34 construction of a year-long activity sequence for each individual does increase the 
3-17

1 similarity of daily activities for a given simulated individual in terms of the time spent 

2 outdoors, compared to a random assignment of diaries from CHAD to modeled 

3 individuals (Glen et al., 2008). However, repeated routine behavior from one weekday to 

4 the next is not simulated. For example, there are no simulated individuals representing 

5 children in summer camps who spend a large portion of their time outdoors, or adults 

6 with repeating weekday schedules. Improvement of the current approach for creating 

7 year-long activity sequences will be undertaken if sufficient resources are available. We 

8 believe an appropriate approach should adequately account for the day-to-day and week9 to-week repetition of activities common to individuals while maintaining realistic 

10 variability between individuals.

11 Averting Behavior

12 Behavior changes in response to ozone pollution or in response to air quality index (AQI) 

13 notification (―averting behavior‖) can affect the population distribution of exposures, and 

14 was not modeled in the previous review. Eiswerth et al. (2005) find that increased ozone 

15 levels appear to influence the amount of time that asthmatic adults spend in different 

16 activities. In a national survey, Mansfield and Corey (2003) find a significant fraction of 

17 the people surveyed modifying their activities in response to ozone alerts. Significant 

18 research on averting behavior has been conducted since the last review (Di Novi, 2010; 

19 Neidell, 2010, 2005a, 2005b; Neidell et al., 2010; Semenza, 2008; Wen et al., 2009). A 

20 methodology for accounting for averting behavior will be developed for this assessment

21 if sufficient resources are available.

22 Modeling Near-Traffic Outdoor Environments and Public Transportation

23 Modeling activities such as walking next to roads, waiting at bus stops, bicycling, and

24 riding motorcycles, buses, subways and trains is difficult due to the limited information 

25 available about these activities. It is also difficult to estimate the ambient concentrations 

26 in these environments. Ozone concentrations in these environments are typically lower 

27 than measurements at centrally located monitors as a result of the titration of ozone by 

28 the NO emissions of the vehicles. A number of near-road monitoring studies have been 

29 conducted since the last review. An analysis of monitoring data to improve estimates of 

30 exposures near-roadways will be undertaken if sufficient resources are available.

31 Metabolic equivalent (MET) distributions for activities

32 The distributions of activity-specific MET values are of fundamental importance to the 

33 physiological model in APEX and therefore to the estimates of lung function decrements. 

34 Johnson (2003, section 9.6) states:
3-18

1 Perhaps the weakest link in the algorithm is the step which requires the 

2 analyst to provide a distribution of possible MET values for each activity 

3 code. These distributions are currently based on distributions provided by the 

4 developers of CHAD (McCurdy et al., 2000). Because available data were 

5 often insufficient to accurately define a distribution for each activity code, the 

6 developers tended to follow a conservative approach and over-estimate the 

7 variability of each distribution. Consequently, the Ve values produced by the 

8 ventilation rate algorithm may exhibit an excessive degree of variability.

9 McCurdy et al. (2000), in a paper describing the development of the MET distributions in 

10 CHAD, state:

11 At this stage of development, the METs distribution assignment effort should 

12 be viewed as being preliminary in nature. More work is needed to better 

13 relate activity codes used in human activity pattern surveys to those long used 

14 by exercise physiologists and clinical nutritionists.

15 Staff will review the recent literature related to MET distributions and update the 

16 distributions used by APEX, if sufficient resources are available.

17 3.8 Uncertainty and Variability

18 The primary difficulty in performing an exposure modeling uncertainty analysis is the 

19 quantitative characterization of the uncertainties of the model inputs and model formulation. 

20 Information about the variability of model inputs or the variability and uncertainty combined is 

21 often available, but it is usually difficult to estimate the uncertainty separately from the 

22 variability. In considering the use of APEX for an ozone exposure assessment, EPA has 

23 considered the availability of information to provide plausible distributions or ranges for the 

24 uncertainties of all of the model inputs. EPA plans to build upon the APEX exposure modeling 

25 uncertainty analysis conducted in support of the previous review of the ozone NAAQS 

26 (Langstaff, 2007). We plan to improve on these distributions of variability and uncertainty, 

27 where data are available to do so, and to extend the analysis of model formulation uncertainty.

28 Once estimates of the uncertainty of the model inputs have been developed, we plan to 

29 propagate these uncertainties through the model to quantify the resultant uncertainty of the 

30 model predictions. The APEX uncertainty analysis methodology incorporates a 2-dimensional 

31 Monte Carlo sampling approach that explicitly characterizes and models the variability and 

32 uncertainty in inputs and outputs. Essentially, this approach entails performing thousands of
3-19

1 model runs with model inputs randomly sampled from specified distributions reflecting 

2 uncertainty of the model inputs, while each single APEX run simulates distributions of 

3 variability. This 2-dimensional Monte Carlo method allows for the separate characterization of 

4 the variability and uncertainty in the model results (Morgan and Henrion, 1990; Cullen and Frey, 

5 1999). This approach allows for great flexibility in specifying uncertainty distributions for any 

6 of the model inputs and parameters that are supplied to APEX by input files. Furthermore, this 

7 allows us to specify conditional distributions and joint distributions between parameters for 

8 which we have data, which can be critically important in modeling uncertainty (Haas, 1997; 

9 Haas, 1999; Wu and Tsang, 2004).

10 Uncertainties are inherent in modeled representations of physical reality due to 

11 simplifying assumptions and other aspects of model formulation. The methods for assessing 

12 input parameter uncertainty and model formulation or structure uncertainty are different. It is 

13 difficult to incorporate the uncertainties due to the model formulation into a quantitative 

14 assessment of uncertainty in a straightforward manner. The preferred way to assess model 

15 formulation uncertainty is by comparing model predictions with measured values, while having 

16 fairly complete knowledge of the uncertainty due to input parameters. EPA plans to ascertain 

17 whether sufficient data are available to perform such an evaluation. For example, we will 

consider using the data collected in the Detroit Study (DEARS118 ) for this purpose. In the absence 

19 of measurements that can be used to estimate model uncertainty, our planned approach to 

20 assessing model formulation uncertainty will be to partition this uncertainty into that of the 

21 components, or sub-models, of APEX. For each of the sub-models within APEX, we plan to 

22 discuss the simplifying assumptions and those uncertainties associated with the sub-models 

23 which are distinct from the input data uncertainties. Where possible, we plan to evaluate these 

24 sub-models by comparing their predictions with measured data. Alternatively, we may formulate 

25 an informed judgment as to a range of plausible uncertainties for the sub-models. We plan to 

26 quantitatively assemble the different types of uncertainties and variability to present an 

27 integrated analysis of uncertainty and variability.

 

1

See http://www.epa.gov/dears
4-1

1 4 ASSESSMENT OF HEALTH RISK BASED ON CONTROLLED HUMAN 

2 EXPOSURE STUDIES

3 4.1 Introduction 

4 The major components of the portion of the health risk assessment based on data from 

5 controlled human exposure studies are illustrated in Figure 2. The air quality and exposure 

6 analysis components that are integral to this portion of the risk assessment are discussed above in 

7 Sections 2 and 3, respectively. As described in the draft ozone ISA (U.S. EPA, 2011b) and 

8 previous ozone Criteria Documents (U.S. EPA, 1996b, 2006), there are numerous controlled 

9 human exposure studies reporting lung function decrements (as measured by changes in FEV1), 

10 as well as changes in other measures of lung function, airway responsiveness, respiratory 

11 symptoms, and various markers of inflammation. Most of these studies have involved voluntary 

12 exposures with healthy adults, although a few studies have been conducted with mild and 

13 moderate asthmatics and one study reported lung function decrements for children 8-11 years old 

14 (McDonnell et al., 1985).

15 Staff plans to develop lung function decrement risk estimates for the general population, 

16 school age children, asthmatic school age children, outdoor workers, and the elderly population 

17 (aged 70 and older) living in 12 urban areas in the U.S. These areas, identified previously in 

18 Section 3.2, represent a range of geographic areas, population demographics, and ozone 

19 climatology. As discussed further in Section 4.4.2, the selection of these areas was also 

20 influenced by whether other health endpoints could be examined in the same urban area based on 

21 concentration-response relationships developed from epidemiological or field studies.

22 4.2 Selection of Health Endpoints 

23 In the last review, the health risk assessment estimated lung function decrements (≥ 10, ≥ 

24 15, and ≥ 20% changes in FEV1) in children 5-18 years old associated with 8-hour exposures at 

25 moderate exertion. At that time EPA staff and the CASAC Ozone Panel judged that it was 

26 reasonable to estimate the exposure-response relationships for children 5-18 years old based on 

27 data from adult subjects (18-35 years old). As discussed in the 1996 Ozone Staff Paper (EPA, 

28 1996a) and 1996 ozone Criteria Document (EPA, 1996b), findings from other chamber studies 

29 (McDonnell et al., 1985) for children 8-11 years old and summer camp field studies in at least 
4-2

1 six different locations in the United States and Canada found lung function changes in healthy 

2 children similar to those observed in healthy adults exposed to ozone under controlled chamber 

3 conditions. Staff intends to use the same approach in this assessment.

4 4.3 Selection of Exposure-Response Functions 

5 The health risk assessment conducted in this new review will build on the approach 

6 developed and applied in the 2008 rulemaking. In that previous assessment, risk estimates for 

7 lung function responses associated with 8-hour exposures while engaged in moderate exertion 

8 were developed. These estimates were based in part on exposure-response relationships 

9 estimated from the combined data sets from multiple ozone controlled human exposure studies. 

10 Data from the studies by Folinsbee et al. (1988), Horstman et al. (1990), and McDonnell et al. 

11 (1991) in addition to more recent data from Adams (2002, 2003, 2006) were used to estimate 

12 exposure-response relationships for ≥ 10, 15, and 20% decrements in FEV1. 

13 The data from these controlled human exposure studies are corrected for effects 

14 observed with exposure to clean air to remove any systematic bias that might be present in the 

15 data attributable to exercise, diurnal variation, or other effects in addition to those of ozone

16 during the course of an exposure. Generally, this correction for exercise in clean air is small 

17 relative to the total effects measures in the ozone-exposed cases. Regression techniques are then 

18 used to fit a function to the data. A Bayesian approach is used then to characterize uncertainty 

19 attributable to sampling error based on sample size considerations. Response rates are calculated 

20 for 21 fractiles (for cumulative probabilities from 0.05 to 0.95 in steps of 0.05, plus probabilities 

21 of 0.01 and 0.99) at a number of ozone concentrations (see U.S. EPA, 2007a for details of this 

22 approach).

23 4.4 Approach to Calculating Risk Estimates

24 Staff plans to generate several risk measures for this portion of the risk assessment. In 

25 addition to the estimates of the number of school age children and other groups experiencing one

26 or more occurrences of a lung function decrement > 10, > 15, and > 20% in an ozone season, 

27 risk estimates also will be developed for the total number of occurrences of these lung function 

28 decrements in school age children and active school age children.
4-3

1 A headcount risk estimate for a given lung function decrement (e.g., ≥ 20% change in 

2 FEV1) is an estimate of the expected number of people who will experience that lung function 

3 decrement. Since EPA is interested in risk estimates associated with ozone concentrations in 

4 excess of policy relevant background concentrations, staff plans to (1) estimate expected risk, 

5 given the personal exposures associated with ambient ozone concentrations, (2) estimate 

6 expected risk, given the personal exposures associated with estimated background ambient ozone 

7 concentrations, and (3) subtract the latter from the former. As shown in equation 4-1 below, the 

8 headcount risk is then calculated by multiplying the resulting expected risk by the number of 

9 people in the relevant population. Because response rates are calculated for 21 fractiles, 

10 estimated headcount risks are similarly fractile-specific. 

The risk (i.e., expected fractional response rate) for the kth 11 fractile, Rk is:

( | ) ( | )

1 1

b

k i

N

i

b

k j i

N

j

k j R P x RR e P x RR e

b

12 (4-1)

13 where: 

14 ej = (the midpoint of) the jth category of personal exposure to ozone, under ―as is‖ 

15 ambient ozone concentrations;

b

i

16

e

= (the midpoint of) the ith category of personal exposure to ozone, under background 

17 ambient ozone concentrations;

18 Pj = the fraction of the population having personal exposures to ozone concentration of ej

19 ppm, under ―as is‖ ambient ozone concentrations;

b

20

Pi

= the fraction of the population having personal exposures to ozone concentration of 

b

i

21

e

ppm, under background ambient ozone concentrations;

k j

22

RR | e

= k-fractile response rate at ozone concentration ej;

b

k i RR | e

= k-fractile response rate at ozone concentration 

b

i

23

e

; and

24 N = number of intervals (categories) of ozone personal exposure concentration, under ―as 

25 is‖ ambient ozone concentrations; and

26

Nb

= number of intervals of ozone personal exposure concentration, under background 

27 ambient ozone concentrations.

28 For example, if the median expected response rate under ―as is‖ ambient concentrations is 

29 0.065 (i.e., the median expected fraction of the population responding is 6.5%) and the median 

30 expected response rate under background ambient concentrations is 0.001 (i.e., the median 
4-4

1 expected fraction of the population responding is 0.1%), then the median expected response rate 

2 associated with ―as is‖ ambient concentrations above policy relevant background concentrations 

3 is 0.065 – 0.001 = 0.064. If there are 300,000 people in the relevant population, then the 

4 headcount risk is 0.064 x 300,000 = 19,200. 

5 4.5 Alternative Approach Under Consideration For Calculating Risk Estimates 

6 In this new review, if adequate resources are available, staff intends to investigate the 

7 possibility of using an improved model that estimates FEV1 responses for individuals associated 

8 with short-term exposures to ozone (McDonnell, Stewart, and Smith, 2010). This model is based 

9 on the controlled human exposure data included in the prior lung function risk assessment as 

10 well as additional data sets for different averaging times and breathing rates. These data were 

11 from 15 controlled human ozone exposure studies that included exposure of 541 volunteers (ages 

12 18–35 years) on a total of 864 occasions (see McDonnell et al., 2007, for a description of these 

13 data).

14 This model calculates the FEV1 decrement due to ozone exposure for each diary event as: 

15 (4-2) 

16 where X is given by the solution of the differential equation (4-3): 

17 (4-3) 

18 In APEX, because the exposure concentration, exertion level, and ventilation rate are 

19 constant over an event, this equation has an analytic solution for each event (events range in 

20 duration from 1 to 60 minutes): 

21 (4-4) 

22 where 

23 C(t) is the exposure concentration at time t (ppm), 

V(t) = VE(t)/BSA is the effective ventilation rate at time t (L min-2 m

-2 24 ),

VE(t) is the expired minute volume at time t (L min-2 25 ), 

BSA is the body surface area (m226 ),

27 X0 is the value of X(t) at time t0, 

28 t is the time (minutes), t0 is the time at the start of the event, 
4-5

1 yijk = α1 Age + α2, (age in years; α1 and α2 depend on age range) and

2 Ui = subject-level random effect (zero mean).

3

4 The y term is a function of the age of the individual in years. In the equation from 

5 McDonnell, Stewart, and Smith (2010), y is given as [Ageijk – 25], however this term was 

6 developed using only data from individuals aged 18-35. Using a larger data set that included 

7 individuals with ages ranging from 8 to 76, we performed a piecewise linear fit of the form

8 y = α1 Age + α2, for different ranges of ages. The three linear fits (ages 8-16, 16-35, 35-100) 

9 match at each boundary to form a continuous function of age. Exposure data used for the youth 

10 fit came from McDonnell et al. 1985, Avol et al. 1987, and Avol et al. 1985. Exposure data for 

11 the 36-76 age range were taken from Drechsler-Parks et al. 1987, Drechsler-Parks et al. 1989, 

12 Gong et al. 1997, and Hazucha et al. 2003.

13 For youth, we found %ΔFEV1 to be highly correlated with age, with a linear regression 

14 giving: y = –3.16 Age + 41.58; but for older adults, we found it only varied weakly with age: 

15 y = 0.02 Age + 9.3. The middle age range (y = Age – 25) of 18-35 was extended to younger 

16 ages, 16-35, based on discussions with McDonnell. No data exist for the range age < 8, and due 

17 to rapid changes in the physiology of children (as opposed to adults), extension of the fit to lower 

18 age ranges is increasingly uncertain and will not be done. Accordingly, %ΔFEV1 will not be 

19 modeled for children under 8 years of age. The parameters α1 and α2 are age-dependent and are 

20 specified in the APEX physiology input file. Staff will conduct further analyses to inform the 

21 choice of these parameters.

22 Here, β1 - β6 and the variance of the {Ui} are unitless fitted model parameters (see 

23 McDonnell, Stewart, and Smith (2010) for details of fit). Values of U are drawn from a 

24 Gaussian distribution with mean zero and variance var(U). They are chosen once for each 

25 individual and remain constant throughout the simulation. The best fit values for these 

26 parameters given by McDonnell, Stewart, and Smith (2010) are as follows (to 3 significant 

27 figures) : β1 = 9.90, β2 = –0.411, β3 = 0.0164, β4 = 46.9, β5 = 0.00375, β6 = 0.912, Var(U) = 

28 0.835.

29 Staff intends to perform an evaluation of this model based on data from clinical studies 

30 that were not used in the development of the model.
4-6

1 4.6 Uncertainty and Variability

2 Staff plans to conduct a 2-dimensional Monte Carlo analysis of the uncertainty and 

3 variability of the risk estimates based on data from controlled human exposure studies. This will 

4 of necessity be integrated with the exposure modeling uncertainty assessment.
4-7

Air Quality Adjustment

Modeled Hour-by-Hour

Exposures Resulting From

(1) “As is” Ambient Ozone

Levels and (2) When

Standards are Met

Exposure-Response

Relationships

Using Exposure

Metrics Based on

Hourly Ozone

Exposures

Ambient PopulationOriented Monitoring

for Selected Urban

Areas

Estimates of Lun

Function Risk

Reduction Associa

with Meeting

Standards

Estimates of Lun

Function Risk

Associated with "A

is" Ozone Levels O

Background

"As is" Ambient

Ozone Levels

Current or Alternative

Standards

Exposure

Modeling

Controlled Human

Exposure Studies

(various lung

function endpoints)

Modeled Hour-by-Hour

Exposures Resulting From

(1) “As is” Ambient Ozone

Levels and (2) Background

Ozone Levels Modeled

Background

Ozone Levels

1 2 Figure 4-1. Major Components of Ozone Health Risk Assessment Based on Controlled Human Exposure 
5-1

1 5 ASSESSMENT OF HEALTH RISK BASED ON EPIDEMIOLOGIC STUDIES

2 5.1 Introduction 

3 As discussed in the draft ozone ISA (EPA, 2011b), a significant number of 

4 epidemiological and field studies examining a variety of health effects associated with ambient 

5 ozone concentrations in various locations throughout the U.S., Canada, Europe, and other 

6 regions of the world have been published since the last NAAQS review. As a result of the 

7 availability of these epidemiological and field studies and air quality information, staff plans to 

8 expand the ozone risk assessment to include an assessment of selected health risks attributable to 

9 ambient ozone concentrations over policy relevant background concentration and health risk 

10 reductions associated with attainment of current and alternative ozone standards in selected 

11 urban locations in the U.S. The major components of the portion of the health risk assessment 

12 based on data from epidemiological and field studies are illustrated in Figure 5-1. The 

13 approaches used by staff to select health endpoint categories, urban areas, and epidemiology and 

14 field studies to consider for inclusion in the risk assessment are discussed below.

15 This chapter presents an overview of the design of the human health risk assessment to be 

16 conducted in the current review of the ozone NAAQS. This design reflects goals laid out in the 

17 Integrated Review Plan (U.S. EPA, 2011a, section 5.5) including: (1) to provide estimates of the 

18 potential magnitude of premature mortality and/or selected morbidity health effects in the 

19 population associated with recent ambient ozone levels and with just meeting the current suite of 

20 ozone standards and any alternative standards that might be considered in selected urban study 

21 areas; (2) to develop a better understanding of the influence of various inputs and assumptions on 

22 the risk estimates; and (3) to gain insights into the distribution of risks and patterns of risk 

23 reduction and uncertainties in those risk estimates.

24 Based upon the information assessed in the first draft ISA, we plan to focus the risk 

25 assessment on health effect endpoints for which the weight of the evidence as assessed in the 

26 ISA supports the judgment that the overall health effect category is at least likely caused by 

27 exposure to ozone either alone and/or in combination with other pollutants. The planned 

28 quantitative risk assessment, is designed to estimate risks associated with short-term (≥ 24-hour 
5-2

1 average) and long-term (e.g., annual- or seasonal- average) ambient ozone concentrations in 

2 selected urban study areas. We are considering expanding the focus of this risk assessment to 

3 include additional health effect categories beyond those classified as casual or likely causal, 

4 when available evidence presented in the ISA is sufficiently suggestive of a causal association to 

5 support conducting quantitative risk assessment and when inclusion of that endpoint category 

6 will allow us to address potentially important policy issues related to reviewing the ozone 

7 NAAQS. For example, we are considering including information on birth outcome effects 

8 associated with ambient ozone which would allow us to evaluate additional potentially sensitive

9 populations (i.e., pregnant women and infants) not previously evaluated in the quantitative risk 

10 assessment conducted in the last review. In addition, we are also considering estimating 

respiratory mortality associated with long-term exposure to ozone.

1

11 EPA recognizes that a 

12 decision to include these additional endpoint categories needs to consider the increased 

13 uncertainty that their inclusion could introduce into the risk assessment; specifically, the 

14 potential for these endpoints not to be causally linked with ozone exposure, despite the statistical 

15 associations observed in epidemiological studies.

16 Building upon the assessment completed in the last review, we plan to focus the ozone

17 assessment on modeling risk for a set of selected urban study areas, chosen in order to provide 

18 population coverage and to portray the observed heterogeneity in ozone-related risk across 

19 selected urban study areas. EPA is considering ways to put the quantitative risk assessment 

20 results conducted for a limited number of locations and selected health endpoints into a broader 

21 context to better characterize the nature, magnitude, extent, variability, and uncertainty of the 

22 public health impacts associated with ozone exposures. 

23 In designing the risk assessment, we expect to identify multiple options for specifying 

24 specific elements of the risk model (e.g., several concentration response functions for a particular 

25 health endpoint; several approaches for characterizing ambient ozone levels within urban areas 

26 using monitors and/or modeling data). In these instances, to the extent possible given available 

27 information, we will identify those options that we believe has the greatest support in the 

 

1 As noted in section 5.1, the decision to model long-term exposure-related respiratory mortality is complicated by 

the fact that, while the draft ISA classifies all-cause mortality related to long-term ozone exposure has having a 

suggestive of a casual association, the draft ISA assigns respiratory morbidity as likely to have a causal association. 
5-3

1 literature. These modeling elements will then be used, to generate a core (base case) set of risk 

2 estimates. The remaining options identified for specifying elements of the risk model will be 

3 used as part of the sensitivity analysis (see below) to generate an additional set of reasonable risk 

4 estimates that can be used to provide a context, with regard to uncertainty, within which to assess 

5 the set of core (base case) risk results. Note, that in general, those health effects endpoints 

6 falling within health effects endpoint categories assigned a causal or likely causal association 

7 with ozone exposure will be included in the core analysis, while endpoints assigned a suggestive 

8 of a causal association (if modeled quantitatively) would likely be included as part of the 

9 sensitivity analysis. As noted earlier, respiratory mortality associated with long-term exposure 

10 represents a special case. Depending on how we ultimately interpret the degree of support for an 

11 association with this endpoint and ozone exposure, we may include this endpoint as part of the 

12 core estimate, or retain it as a component of the sensitivity analysis.

13 As part of the risk assessment, we will address both uncertainty and variability. In the 

14 case of uncertainty, we are planning to use a four-tiered approach developed by the World Health 

15 Organization (WHO) and used in the risk assessment completed for the last PM NAAQS review. 

16 The WHO’s four-tiered approach matches the sophistication of the assessment of uncertainty to 

17 the overall complexity of the risk assessment, while also considering the potential magnitude of 

18 the impact that the risk assessment can have from a regulatory/policy perspective (e.g., risk 

19 assessments that are complex and are associated with significant regulatory initiatives would 

20 likely be subjected to more sophisticated uncertainty analysis). The WHO framework includes 

21 the use of sensitivity analysis both to characterize the potential impact of sources of uncertainty 

22 on core risk estimates and (as noted earlier) to generate an alternative set of reasonable risk 

23 estimates that supplement the core risk estimates.

24 In the case of variability, we will identify key sources of variability associated with ozone 

25 risk (for both short-term and long-term exposure-related endpoints included in the risk 

26 assessment) and discuss the degree to which these sources of variability are reflected in the 

27 design of the risk assessment. Note, that in those cases where a particular source of variability is 

28 not sufficiently reflected in core risk estimates, this can introduce uncertainty and potentially bias 

29 into the risk estimates since representativeness can be reduced (in certain cases, the sensitivity 
5-4

1 analysis may also explore these sources of variability given their potential to introduce 

2 uncertainty into core risk estimates). 

3 As part of the analysis, we will also complete a representativeness analysis designed to 

4 support the interpretation of risk estimates generated for the set of urban study areas included in 

5 the risk assessment. The representativeness analysis will focus on comparing the urban study 

6 areas to national-scale distributions for key ozone-risk related attributes (e.g., demographics 

7 including socioeconomic status, air-conditions use, baseline incidence rates and ambient ozone 

8 levels). The goal with these comparisons will be to assess the degree to which the urban study 

9 areas provide coverage for different regions of the country as well as for areas likely to 

10 experience elevated ozone-related risk due to their specific mix of attributes related to ozone 

11 risk. As part of the representativeness analysis, we are also considering a broader national-scale 

12 assessment of mortality (both short- and long-term exposure-related). These national-scale 

13 mortality estimates would also allow us to assess the degree to which the urban study areas 

14 included in the risk assessment provide coverage for areas of the country expected to experience 

15 elevated mortality rates due to ozone-exposure. We note that a national-scale assessment such as 

16 this was completed for the risk assessment supporting the latest PM NAAQS review (U.S. EPA, 

17 2010) with the results of the analysis being used to support an assessment of the 

18 representativeness of the urban study areas (assessed in the PM NAAQS risk assessment), as 

19 described here for ozone. Additional detail on the representativeness analysis is presented in 

20 section 5.4.5. 

21 The following discussion begins by presenting the framework for the risk assessment 

22 developed to evaluate ozone with more detailed discussions of key components of the risk 

23 assessment model including air quality considerations (section 5.2). Next, we discuss the

24 selection of health effects endpoints to include in the assessment, including the specification of 

25 concentration-response (C-R) functions, baseline incidence data and demographic data (section

26 5.3). We conclude with the discussion of how uncertainty and variability will be addressed in 

27 the analysis (section 5.4). This discussion also includes an overview of the representativeness 

28 analysis planned for the assessment.
5-5

1 5.2 Framework for the Ozone Health Risk Assessment 

2 5.2.1 Overview of Modeling Approach 

3 Consistent with the last review, we plan to quantify the number of ozone-related adverse 

4 health outcomes by using a health impact function, the components of which are illustrated in 

5 equation 5-1. The health impact function combines information about changes in ambient ozone

6 air quality concentrations (∆x) with C-R relationships (reflected by β, the ozone coefficient 

7 derived from epidemiological studies) and baseline health incidence data for specific health 

8 endpoints (y) to derive estimates of the change in incidence (∆y) of specific health effects 

9 attributable to ambient ozone concentrations during the period examined among a particular 

population (Pop).

1

10

11 (5-1)

12 This type of risk model applies risk coefficients drawn from epidemiological studies that 

13 characterize the relationship between ambient ozone levels measured at fixed-site population14 oriented monitors and the risk of specific health endpoints in the population. Therefore, it does 

15 not require more detailed individual-level exposure modeling described above and relies instead 

16 on the use of ambient monitoring data. Specifically, a change in the level of ambient ozone is 

17 translated through the risk coefficient (β) to a change in the baseline rate of a particular health 

18 effect(s) in the study population. This adjustment to the baseline incidence rate can then be 

19 combined with population estimates (Pop) to generate a change in the incidence of a specific 

20 health endpoint(s) attributable to a change in ambient ozone.

21 In this review we plan to use the environmental Benefits Mapping and Analysis Program 

22 (BenMAP) (Abt, 2008) to perform this calculation across multiple health impact functions and 

23 urban areas. This GIS-based computer program draws upon a database of population, baseline 

24 incidence and effect coefficients to automate the calculation of health impacts. EPA has 

25 traditionally relied upon the BenMAP program to estimate the health impacts avoided and 

26 economic benefits associated with adopting new air quality rules. For this analysis, EPA will use 

 

1 The health risk model given in Equation 5-1 is based on a concentration-response function in which the natural 

logarithm of the incidence of the health effect is a linear function of ozone concentration. We plan to consider other 

mathematical forms where epidemiological studies have reported effects using other model forms. 
5-6

1 the model to estimate ozone-related impacts among the health endpoints, and within the urban 

2 areas, discussed below. BenMAP already contains much of the population and baseline 

3 incidence data, and many of the effect coefficients, needed to perform this analysis; where it 

4 does not, we will specify the model with the appropriate data. The following diagram

5 summarizes the data inputs (in black text) and outputs (in blue text) for a typical BenMAP 

6 analysis.

7

8 BenMAP offers several advantages in terms of modeling population exposure and risk. 

9 First, once we have properly specified the BenMAP software, the program can produce risk 

10 estimates for an array of modeling scenarios across a large number of urban areas. Second, the 

11 program can accommodate a variety of sensitivity analyses. For example, we may consider the 

12 sensitivity of our risk estimates to alternative specifications of concentration-response functions 

13 for the same endpoint. Third, BenMAP would be useful to performing a national assessment of 

14 ozone mortality for the purposes of a representativeness analysis (as discussed earlier in 

15 section 5.1). 

16 As described in Figure 5-1, this risk assessment approach requires specifying a number of 

17 modeling components related to (a) characterizing air quality, (b) establishing the C-R functions, 

Census 

Population 

Data

Air Quality 

Monitoring

Health 

Functions

Population 

Projections

Air Quality 

Modeling

Incidence and 

Prevalence 

Rates

Population 

Estimates

Population 

Exposure

Adverse 

Health Effects

BenMAPdata inputs and outputs
5-7

1 and (c) specifying the baseline incidence rates and population demographics. The remainder of 

2 this section discusses each of these modeling components in detail. 

3 5.2.2 Air Quality Considerations 

4 There are several air quality inputs to the risk assessment as illustrated in Figure 5-1. 

5 These have been described in Chapter 2 and include: (a) characterization of recent air quality 

6 (i.e., ambient ozone levels) for each selected urban study area, (b) background concentrations for 

7 each selected urban study area, and (c) projections of ambient air quality for both current and 

8 alternative ozone NAAQS under consideration. Additional detail on these inputs is presented 

9 below:

10 Characterizing recent ambient ozone levels for selected urban study areas using 

11 monitoring data: EPA plans to use 3 years (2008-2010) of ambient ozone measurement 

12 data to characterize recent air quality conditions (see section 2.2). In aggregating 

13 monitoring data (to form composite monitor(s) for each study area) and linking those 

14 monitors to study populations within a particular study area, as noted earlier in section 

15 2.2, we are considering two approaches. As in the previous ozone NAAQS risk 

16 assessment, we plan to match, to the extent possible, the approach for analyzing air 

17 quality data used in the epidemiological studies from which the C-R functions are 

18 obtained. For example, in order to be consistent with the approach generally used in the

19 epidemiological studies from which C-R functions have been estimated for effects 

20 associated with long-term ozone exposures, we plan to develop and use ambient data for 

21 a single composite monitor based on monitored data from all eligible monitors in that 

22 study area. Some epidemiological studies have used more sophisticated (and spatially23 refined) methods for associating ambient ozone data with a study population. In cases 

24 where we include C-R functions from studies using alternative methods to link ambient 

25 ozone concentrations with health effects information in our risk assessment, we may 

26 consider a more refined approach for linking ozone monitoring data with study 

27 populations, to match the approach used in the study. However, in addition to a risk 

28 simulation where we attempt to match our use of monitoring data to the approach used in 

29 the underlying epidemiological studies, we are also considering an alternative approach 

30 where we focus on developing composite monitors that are more representative of 

31 exposure profiles experienced by populations currently. As noted in section 2.2, this 

32 could involve an alternative weighting scheme for deriving composite monitors where 

33 we use the results of micro-environmental exposure modeling (used in the 
5-8

1 Figure 5-1. Overview of Risk Assessment Model Based on Epidemiologic Studies

2

Ambient Monitoring for 

Selected Urban Study 

Areas (possibly 

augmented with CMAQ 

modeling data)

Air Quality Adjustment 

Procedures (Rollback) to 

Simulate Just Meeting 

Current and Alternative 

NAAQS

Changes in 

Distribution of 

Ozone Air 

Quality

Risk Estimates:

Recent Air Quality

- Risk above PRB

Simulating Just Meeting 

Current NAAQS

- Risk above PRB 

- Incremental difference 

in risk compared to 

recent air quality 

Simulating Just Meeting 

Alternative NAAQS 

Under Consideration

- Risk above PRB

- Incremental difference 

in risk compared to just 

meeting current NAAQS

Health 

Risk 

Model

City-specific Demographic Data

Selection of 

Epidemiological Studies to 

Provide ConcentrationResponse Functions

Specification of 

Concentration –

Response

Functions

Concentration-Response

Air Quality

Policy Relevant 

Background (PRB) 

Recent Ambient 

Ozone Levels

Selection of health 

effects endpoints

Estimates of City-specific 

Baseline Health Effects 

Incidence Rates

Baseline Health Effects Incidence Rates and Demographics
5-9

1 exposure analysis – see Section 3) to generate weights for each monitor in an urban 

2 study area reflecting the fraction of population exposure associated with that monitor 

3 (e.g., the fraction of simulated person-ozone hours associated with the area surrounding 

4 a given monitor). The details of this approach are still being developed. 

5 Characterizing recent ambient ozone levels for selected urban study areas using a 

6 combination of monitoring and modeled data: As discussed in section 2.2, we are 

7 also considering the use of monitor data augmented with modeling in characterizing 

8 recent ambient ozone conditions. This type of a fused surface has the benefit of 

9 retaining the ambient characterization of absolute ozone levels (at monitors), while using 

10 modeled concentrations to characterize the spatial gradient between monitors. Note, 

11 however, that we would still need to specify how this more spatially-refined surface 

12 would be related to population in order to generate an exposure surrogate (e.g., would we 

13 conduct risk simulation at a more spatially-refined grid cell-level, or would we use the 

14 more differentiated ozone surface to generate a composite measurement value – a single 

15 value for the entire study area)? The decision as to whether to pursue this type of fused 

16 (model-monitor) surface and if so, how to use it in characterizing exposure, would likely 

17 rely heavily on our assessment of the spatial heterogeneity of ozone levels across a 

18 subset of our urban study areas – see bullet below.

19 Assessment of spatial heterogeneity of ozone across urban study areas: As noted in 

20 section 2.2., a potentially important component of designing the risk assessment involves 

21 an assessment of the spatial heterogeneity of ambient ozone levels across prospective 

22 urban study areas. This assessment would likely be based on consideration for the 

23 pattern of ozone levels (daily time series and seasonal/annual averages) across monitors 

24 within a given urban study area. If spatial heterogeneity across monitors is found to be 

25 low (and more specifically, if temporal profiles for the monitors within a given study 

26 area are similar), then there may be little benefit in using more sophisticated approaches 

27 for linking ambient ozone levels and populations within a given study area. Conversely, 

28 if there is substantial spatial heterogeneity, then the representativeness of an exposure 

29 surrogate (e.g., surrogate monitor) could be enhanced by more closely linking ambient 

30 ozone levels to demographics.

31 Characterizing PRB: As noted in section 2.4, we will rely on characterization of PRB 

32 provided by GEOS-Chem modeling to obtain values specific to each urban study area 

33 included in the risk assessment. However, we will likely consider several different 

34 background definitions (e.g., U.S. background, a North American background, and 

35 natural background).
5-10

1 Method for adjusting ambient air quality levels to simulate air quality just meeting 

2 current and potential alternative ozone NAAQS: As discussed in section 2.3, EPA is 

3 planning to use the quadratic rollback approach to simulate ozone levels to just meet 

4 current and alternative NAAQS standards. Note, that we may explore the degree of 

5 spatial heterogeneity associated with ambient ozone levels within our urban study areas. 

6 If we find that there is substantial spatial heterogeneity, then this could mean that the 

7 pattern of rollback potentially associated with attainment of an alternative (lower) level 

8 could be more complex from a spatial standpoint (i.e., there is potentially, increased 

9 uncertainty in simulating attainment of either the current or alternative standards). 

10 Conversely, if we find that there is limited spatial heterogeneity, then we expect that 

11 uncertainty associated with simulating alternative standard levels would be relatively 

12 lower. 

13 5.3 Selection of Health Effects Endpoint Categories

14 As noted in section 5.1, based on review of the first draft ISA, we plan to focus the risk 

15 assessment on ozone, estimating potential health impacts associated with both short-term and 

16 long-term exposures to ozone. In selecting health effects endpoints to include in the risk 

17 assessment, we have considered the following factors based upon review of the first draft ISA 

18 (U.S. EPA, 2011b; Chapters 2, 6, and 7): (a) the extent to which the health effect endpoints are 

19 considered significant from a public health standpoint, (b) the overall weight of the evidence 

20 from the collective body of epidemiological, clinical, and toxicological studies and the inferences 

21 made in the first draft ISA as to whether there is a causal or likely causal relationship between 

22 ozone and the health effect endpoint category, (c) whether there is sufficient evidence to support 

23 a causal or likely causal relationship for the specific health endpoint within the health effect 

24 category to warrant inclusion in the risk assessment, and (d) whether there are well-conducted 

25 studies reporting estimated C-R functions for specific health endpoints associated with ambient 

26 ozone levels. 

27 Based upon review of the first draft ISA, we plan to consider the following health effect 

28 endpoint categories in this assessment:

29 Health Effect Categories Associated with Short-term Ozone Exposure

30 respiratory morbidity ( causal association)

31 mortality (likely causal association) 
5-11

1 Health Effect Categories Associated with Long-term Ozone Exposure

2 respiratory morbidity (likely casual association)

3 In addition to the health effect categories presented above, we are considering expanding 

4 the focus of the ozone risk assessment to include additional endpoints from health effect 

5 categories that have been initially judged in the first draft ozone ISA to have a suggestive causal 

6 association with ambient ozone measurements. We plan to consider including these additional 

7 endpoints when they allow us to address potentially important policy issues related to reviewing 

8 the current ozone standards. Risk estimates for endpoints within these additional health effects 

9 categories would likely not be presented as part of the core risk assessment, but rather would be 

10 included as part of the sensitivity analysis examining additional potential health effects 

11 endpoints. Potential health effect endpoint categories being considered for inclusion in the 

12 sensitivity analysis include: (a) long-term exposure-related birth outcome effects (allows us to 

13 evaluate potentially sensitive populations, including pregnant women and infants), and (b) long14 term exposure-related respiratory mortality (due to the clear public health significance of this 

15 endpoint). 

16 The respiratory mortality endpoint deserves some additional discussion. While the 

17 general all-cause mortality category was given a suggestive of causal association (for long-term 

18 exposure) in the draft ISA, it is important to note that the respiratory morbidity category (again 

19 for long-term exposure) was assigned a likely causal association classification in the draft ISA. If 

20 we consider focusing an assessment of long-term exposure related mortality on respiratory 

21 mortality (which would be the endpoint most supported by the latest reanalysis of the ACS data), 

22 then the appropriate causality association classification for this specific mortality endpoint may

23 be more complicated to determine. While this endpoint falls within a category of mortality 

24 (which was assigned a suggestive of casual association), it is also a type of respiratory health 

25 effect (which is given a likely casual association). Therefore, EPA will continue to review 

26 information presented in the next draft ISA and look to input provided by both the public and 

27 CASAC to make a determination as to the appropriate degree of support to assign to the long28 term exposure-related respiratory mortality category. Ultimately, this determination will result in 
5-12

1 an estimate of respiratory mortality (if it is indeed generated in the first place) to be included as 

2 part of the core estimate, or retained as part of the sensitivity analysis. 

3 5.3.1 Selection of Epidemiological Studies and Specification of Concentration4 Response Functions 

5 As noted above, the risk assessment conducted in this review will build on the approach 

6 developed and applied in the last review. EPA will rely on a weight-of evidence approach, based 

7 on the ISA’s evaluation of new and previously reviewed epidemiologic studies including 

8 identification of relevant C-R functions that characterize the relationships between short- and 

9 long-term ozone exposures and health outcomes, particularly those conducted at or near current 

10 ambient concentrations. Quantitative relationships provided in the specific studies (or to be 

11 derived by EPA from the data presented in the epidemiologic studies) describe the change in 

12 concentration (generally based on ambient fixed-site monitors) associated with a change in 

13 health response. These C-R relationships will be combined with air quality data, baseline 

14 incidence data, and population data to develop population health risk estimates. 

15 We plan to use specific criteria to select the epidemiological studies that will be used to 

16 provide C-R functions for the quantitative risk assessment including:

17 The study addresses one of the health effects endpoint categories identified for 

18 inclusion in the risk assessment.

19 The study was peer-reviewed, evaluated in the first draft ISA, and judged adequate by 

20 EPA staff for purposes of inclusion in the risk assessment. Criteria considered by 

21 staff include: whether the study provides C-R relationships for locations in the U.S., 

22 whether the study has sufficient sample size to provide effect estimates with a 

23 sufficient degree of precision and power, whether the study is a multi-city study, and 

24 whether adequate information is provided to characterize statistical uncertainty.

25 The study is not superseded by another study (e.g., if a later study is an extension or 

26 replication of a former study, the later study would effectively replace the former 

27 study), unless the earlier study has characteristics that are clearly preferable.

28 In addition to the above criteria, other factors, which may be specific to a particular 

29 health effect endpoint, or even to a set of studies, may be considered. For example, several of 

30 the studies have improved upon the method of estimating the exposure metric used in most 

31 studies which have generally relied upon population-oriented monitoring data. Instead of 
5-13

1 assigning the same ambient ozone concentration to all individuals in a city (based on a central 

2 monitor or the average of several monitors in a city), these studies have assigned ―exposures‖ 

3 according to monitors that better approximate conditions near subjects’ residences. These and 

4 similar studies may provide additional insights into whether reductions in mortality are 

5 attributable to recent, or more historical changes in patterns of long-term ozone exposure. 

6 We also plan to consider the overall study design, including the method used to adjust for 

7 covariates (including confounders and effects modifiers) in identifying candidate studies. For 

8 example, if a given study uses ecological-defined variables (e.g., smoking rates) as the basis for 

9 controlling for confounding, concerns may be raised as to the effectiveness of that control. 

10 These factors related to confounding control and consideration of effects modification also will 

11 be considered in identifying studies for use as the basis of C-R functions.

12 Once the final set of epidemiological studies is chosen, the next step will be the selection 

13 of C-R functions from those studies. A number of factors need to be considered in specifying C14 R functions related to short- and long-term exposure studies. The factors being considered in 

15 selecting C-R functions include:

16 Single- and multi-pollutant models (pertains to both short-term and long-term 

17 exposure studies): Epidemiological studies often consider health effects associated with 

18 ambient ozone independently as well as together with co-pollutants (e.g., PM, nitrogen 

19 dioxide, sulfur dioxide, carbon monoxide). To the extent that any of the co-pollutants 

20 present in the ambient air may have contributed to health effects attributed to ozone in 

21 single pollutant models, risks attributed to ozone may be overestimated if C-R functions 

22 are based on single pollutant models. This would argue for inclusion of models reflecting 

23 consideration of co-pollutants. Conversely, in those instances where co-pollutants are 

24 highly correlated with ozone, inclusion of those pollutants in the health impact model can 

25 produce unstable and statistically insignificant effect estimates for both ozone and the co26 pollutants. This situation would argue for inclusion of a model based exclusively on 

27 ozone. Given that single and multi-pollutant models each have potential advantages and 

28 disadvantages, we plan to include both types of C-R functions in the risk assessment.

29 Single-city versus multi-city studies (typically a factor in short-term exposure studies): 

30 All else being equal, we judge C-R functions estimated in the assessment location as 

31 preferable to a function estimated in some other location, to avoid uncertainties that may 
5-14

1 exist due to differences associated with geographic location. There are several 

2 advantages, however, to using estimates from multi-city studies versus studies carried out 

3 in single cities. Multi-city studies are applicable to a variety of settings, since they 

4 estimate a central tendency across multiple locations. Multi-city studies also tend to have 

5 more statistical power and provide effect estimates with relatively greater precision than 

6 single-city studies due to larger sample sizes, reducing the uncertainty around the 

7 estimated health coefficient. By contrast, single-city studies, while often having lower 

8 statistical power and varying study designs which can make comparison across cities 

9 challenging, do reflect location-specific factors such as differences in underlying health 

10 status, and differences in exposure-related factors such as air conditioner use and urban 

11 density with larger populations exposed near high-traffic roads. Because single- and 

12 multi-city studies have different advantages, we plan to include both types of functions in 

13 this analysis, where they are available. We plan to place greater weight on the use of C-R 

14 relationships reflecting adjusted single-city estimates from multi-city studies. This would 

15 include empirical Bayes adjusted city-specific estimates. These types of effect estimates

16 benefit both from increased statistical power, as well as the potential for specification of 

17 city-specific effect estimates. Conversely, if a multi-city study only provides aggregated 

18 effect estimates, but does differentiate those estimates regionally, we plan to use those 

19 regional-specific estimates rather than a single national-level estimate by matching 

20 selected urban study areas to these regions. 

21 Multiple lag models (pertinent to short-term exposure time-series studies): If 

22 information is available for a distributed lag model, we plan to use that model. Where 

23 there are multiple lags presented, but a distributed lag model is not included, we plan to 

24 consider information presented in the first draft ISA to determine if there is biological 

25 support for selecting a specific lag period for a given health effect endpoint.

26 Interactions between pollutants and temperature: To the extent that studies explore 

27 (a) interactions between pollutant(s) and ozone and (b) interactions between temperature 

28 and ozone, we will consider that information in modeling specific endpoints to the extent 

29 that relevant concentration-response functions (taking into consideration these factors are 

30 available) and/or use this information to help interpreting risk estimates. 

31 Seasonally-differentiated effects estimates (pertinent to short-term studies): In those 

32 instances where studies presented effect estimates associated with short-term ambient 

33 ozone concentrations differentiated by season, we plan to use these seasonal estimates. 

34 We plan to link seasonal effect estimates with seasonal ozone air quality data in 

35 conducting the risk assessment for selected urban study areas. 
5-15

1 Shape of the functional form of the risk model: In the risk assessment conducted in 

2 the last review, EPA included C-R relationships that reflected linear or log-linear C-R 

3 functions that extended down to estimated background levels for effects related to short4 term exposure and down to lowest measured ambient levels for effects related to long5 term exposure, as well as adjusting these models to reflect various alternative ―cutpoint‖ 

6 models. The alternative cutpoint models imposed an assumed threshold on the original 

7 C-R function, below which there is little or no population response. The first draft ISA 

8 concludes that there is little support in the literature for a population threshold for short9 term exposure-related effects, although in the case of mortality, the first draft ISA notes 

10 that the nature of the mortality effect as well as study design may mean that these studies 

11 are not well suited to identify a threshold should it exist (see U.S. EPA, 2011b, section 

12 2.5.3.2). In the case of long-term exposure related endpoints (specifically for birth 

13 outcomes), the first draft ISA notes that study results suggest a clear association with 

14 ozone above approximately 30 ppb, with that relationship no longer being statistically 

significant below that level (i.e., a 95th 15 % confidence interval on the effect estimate 

16 including zero below this ambient ozone level). Given the above observation from the 

17 first draft ISA regarding the potential for thresholds, we are planning to (a) for all short18 term exposure related endpoints, not consider a threshold either in the core analysis, or as 

19 part of sensitivity analyses and (b) for long-term exposure-related endpoints, consider a 

20 range of threshold levels (e.g., 20, 30 and 40 ppm) along with a no-threshold scenario. 

21 Note, that as discussed earlier, all simulations for birth outcomes (if conducted) will be 

22 presented as part of the sensitivity analysis. However, long-term exposure-related 

23 mortality (if run) could be included as part of the core risk estimate or as part of 

24 sensitivity analyses, depending on how we ultimately interpret the degree of support for a 

25 casual association. In either case, for long-term exposure-related mortality, we would also 

26 likely simulate a series of potential thresholds (e.g., 20, 30 and 40 ppb) with risk 

27 estimates for these simulations being included as part of the sensitivity analysis.

28 In addition to the factors listed above, there are additional factors related to the design of 

29 individual epidemiological studies which we plan to consider in selecting the C-R functions to be 

30 included in the assessment. For example, studies often include adjustment for covariates with 

31 varying degrees of freedom, reflecting the tradeoff between bias and over-adjustment (loss of 

32 efficiency). In these cases, we plan to consider any information provided for specific studies 

33 within the first draft ISA and also plan to consider which model form has the strongest statistical 

34 fit, while still considering overall biological plausibility. An additional factor that we will 

35 consider in selecting C-R functions to include in the risk assessment is ongoing research into the 
5-16

1 potential mitigating and averting effect that air quality alerts such as the EPA’s AirNOW can 

2 have on ozone-related exposure and risk. If available research is found to provide effect 

3 estimates (or information that can be used to derive effect estimates) that reflect this averting 

4 and/or mitigating activity by the exposed population, then we would consider including risks 

5 based on these adjusted effect estimates as part of our Sensitivity Analysis. 

6 5.3.2 Selection of Urban Study Areas 

7 We plan to build on the risk assessment conducted for the last review and continue to 

8 focus the risk assessment on a set of selected urban study areas. The decision to continue to 

9 focus on modeling a set of selected urban study areas reflects the goal of providing risk estimates 

10 that have higher overall confidence due to the use of location-specific data when available for 

11 these urban locations. In addition, given the greater availability of location-specific data, a more

12 rigorous evaluation of the impact of uncertainty and variability can be conducted for a set of 

13 selected urban study areas than would be possible for a broader regional or national-scale 

14 analysis. We plan to consider the following factors in the selection of urban study areas:

15 Air quality data: The urban area has sufficient recent (2008-2010) air quality data to 

16 conduct the risk assessment (See section 2.2.1).

17 Location-specific C-R functions: There are C-R functions available from 

18 epidemiological studies that we ultimately select to use as the basis for deriving 

19 concentration-response functions, for one or more of the selected health endpoints. This 

20 primarily applies to short-term epidemiological studies, which more often include city21 specific effect estimates. C-R functions available from long-term epidemiological studies 

22 generally combine data from multiple cities. Specific cities evaluated in the key long23 term studies would be considered for inclusion in the risk assessment. We plan to include 

24 urban study areas that have been assessed in epidemiological studies that have evaluated 

25 health effects associated with both short- and long-term ozone exposures and, to the 

26 extent possible, locations where both morbidity and mortality health endpoints have been 

27 evaluated.

28 Baseline incidence rates and demographic data: The required urban area-specific 

29 baseline incidence rates and population data are available for a recent year for at least one 

30 of the health endpoints.
5-17

1 Geographic heterogeneity: Because ozone distributions and population characteristics 

2 vary geographically across the U.S., we plan to select a set of urban study areas in which 

3 each region of the country is represented. We plan to define these regions in such a way 

4 as to reflect differences in factors related to ozone distributions, sources, co-pollutants, 

5 exposure, and/or effect estimates.

6 Representing areas with relatively larger vulnerable populations: Baseline incidence 

7 rates (e.g., mortality rates) and ozone exposures are higher in some parts of the country 

8 than others. We plan to select a set of urban study areas that will include representation 

9 of sensitive populations (e.g., those with higher baseline incidence rates of the health 

10 effect endpoints being evaluated, lower air conditioning usage which has been related to 

11 higher ambient ozone exposures).

12 Consideration of epidemiology studies with more refined exposure metrics: We plan 

13 to include urban study areas for which there is a C-R function estimated using a more 

14 refined metric of exposure (e.g., smaller geographic units linked to nearest ozone 

15 monitors, rather than constructing a single composite monitor for an entire metropolitan 

16 area), where available.

17 5.3.3 Baseline Health Effects Incidence Data and Demographic Data

18 As noted earlier (section 5.2.1), the most common epidemiological-based health risk 

19 model expresses the reduction in health risk (∆y) associated with a given reduction in ozone

20 concentrations (∆x) as a percentage of the baseline incidence (y). To accurately assess the 

21 impact of ozone air quality on health risk in the selected urban areas, information on the baseline 

22 incidence of health effects (i.e., the incidence under recent air quality conditions) in each 

23 location is needed. Where at all possible, we plan to use county-specific incidences or incidence 

24 rates (in combination with county-specific populations). A summary of available baseline 

25 incidence data for specific categories of effects is presented below:

26 Availability of baseline incidence data on mortality: County-specific (and, if desired, 

27 age- and race-specific) baseline incidence data are available for all-cause and cause-
5-18

specific mortality from CDC Wonder.11 The most recent year for which data are available 

online is 2005.22 

3 Availability of baseline incidence data for hospital admissions and emergency room 

4 (ER) visits: 

5 o Cause-specific hospital admissions baseline incidence data are available for each 

6 of 40 states from the State Inpatient Databases (SID).

7 o Cause-specific ER visit baseline incidence data are available for 26 states from 

8 the State Emergency Department Databases (SEDD).

9 o SID and SEDD are both developed through the Healthcare Cost and Utilization 

10 Project (HCUP), sponsored by the Agency for Healthcare Research and Quality 

11 (AHRQ). 

12 o The data generated from HCUPnet (HCUP’s online interactive tool) are state13 level summary statistics, whereas the data from the HCUP distributor are at the 

14 individual discharge level. 

15 o In addition to being able to estimate State-level rates, SID and SEDD can also be 

16 used to obtain county-level hospital admission and ER visit counts by aggregating 

17 the discharge records by county. 

18 EPA is in the process of obtaining the county-specific hospital admission and ER visit 

19 baseline incidence data for the most recent single year available for most of the States included 

20 in the HCUP data. While we recognize that there is year-to-year variability in baseline incidence 

21 data, a single year of data is being obtained due to resource constraints. We plan to examine the 

22 potential variability in baseline incidence data and the impact this might have on the risk 

23 estimates in sensitivity analyses based on endpoints and locations where we can obtain multi24 year baseline incidence data at little or no cost and by examining the variability in baseline 

25 incidence rates at the State level.

26 5.3.4 Assessing Risk In Excess of Policy-Relevant Background

27 As noted above, staff plans to assess risks associated with ozone concentrations in excess 

28 of policy-relevant background concentrations, and to assess risk reductions associated with just 

 

1

 http://wonder.cdc.gov/mortsql.html

2

 Note: For years 1999 – 2005, CDC Wonder uses ICD-10 codes; for years prior to 1999, it uses ICD-9 codes. 

Since most of the studies use ICD-9 codes, this means that EPA will have to create or find a mapping from ICD-9 

codes to ICD-10 codes if the most recent data available are to be used. 
5-19

1 meeting current and alternative ozone standards. Following the methods used in the prior ozone 

2 risk assessment, risks based on a concentration-response function estimated in an 

3 epidemiological or field study will be assessed down to the estimated policy relevant 

4 background.

5 To assess risks associated with ozone concentrations in excess of policy-relevant 

6 background concentrations, staff will first calculate the difference between ―as is‖ ozone levels 

7 and policy-relevant background. Staff will then calculate the corresponding change in incidence 

8 of the health effect associated with that change in ambient ozone concentration. If x denotes 

9 the change in ozone level from ―as is‖ concentration to the background concentration, then the 

10 corresponding change in incidence of the health effect, y, for a log-linear concentration11 response function (the most common functional form), is

y = y [1-e

-β x

12 ] (5-1)

13 where y denotes the baseline incidence and β is the coefficient of ozone in the 

14 concentration-response function. A similar calculation would be made if the concentration15 response function is of a logistic form. 

16 To assess the risk reduction associated with just meeting the current standard in those 

17 locations that do not currently meet this standard, the procedure will be the same, except that in 

18 this part of the risk assessment x will be the difference between ―as is‖ ozone levels and the 

19 ozone levels that will be estimated to exist if the current standards are just met.

20 To assess the risk reductions associated with just meeting alternative, more stringent 

21 standards, above and beyond the risk reductions that would be achieved by just meeting the 

22 current standards, x will be the difference between ozone levels that will be estimated to exist if 

23 the current standards are just met and ozone levels that will be estimated to exist if the 

24 alternative, more stringent, standards are just met.

25 Because the ozone coefficient, β, is estimated rather than known, there is uncertainty 

26 surrounding that estimate. This uncertainty is characterized as a normal distribution, with mean 

27 equal to the ozone coefficient reported in the study, and standard deviation equal to the standard 
5-20

1 error of the estimate, also reported in the study. From this information, staff plans to construct a 

2 95 percent confidence interval around the reported risk or risk reduction (number of cases of the 

3 health effect avoided), with that confidence interval primarily reflecting sampling error 

associated with the underlying effect estimate.14

5 5.4 Characterization of Uncertainty and Variability in the Context of the Ozone Risk 

6 Assessment

7 5.4.1 Overview of Approach for Addressing Uncertainty and Variability

8 An important component of a population health risk assessment is the characterization of

9 both uncertainty and variability. Variability refers to the heterogeneity of a variable of interest

10 within a population or across different populations. For example, populations in different regions 

11 of the country may have different behavior and activity patterns (e.g., air conditioning use, time 

12 spent indoors) that affect their exposure to ambient ozone and thus the population health

13 response. The composition of populations in different regions of the country may vary in ways

14 that can affect the population response to exposure to ozone – e.g., two populations exposed to 

15 the same levels of ozone might respond differently if one population is older than the other. 

16 Variability is inherent and cannot be reduced through further research. Refinements in the design 

17 of a population risk assessment are often focused on more completely characterizing variability 

18 in key factors affecting population risk – e.g., factors affecting population exposure or response –

19 in order to produce risk estimates whose distribution adequately characterizes the distribution in 

20 the underlying population(s).

21 Uncertainty refers to the lack of knowledge regarding the actual values of inputs to an

22 analysis. Models are typically used in analyses, and there is uncertainty about the true values of

23 the parameters of the model (parameter uncertainty) – e.g., the value of the coefficient for ozone

24 in a C-R function. There is also uncertainty about the extent to which the model is an accurate

25 representation of the underlying physical systems or relationships being modeled (model

26 uncertainty) – e.g., the shapes of C-R functions. In addition, there may be some uncertainty

27 surrounding other inputs to an analysis due to possible measurement error—e.g., the values of

 

1 The confidence interval will not reflect the impact of other sources of uncertainty such as alternative model choice 

associated with deriving the effect estimate (although this source of uncertainty may be addressed as part of the 

sensitivity analysis – see Section 5.4.4).
5-21

1 daily ozone concentrations in a risk assessment location, or the value of the baseline incidence

rate for a health effect in a population.12 In any risk assessment, uncertainty is, ideally, reduced to 

3 the maximum extent possible through improved measurement of key variables and ongoing

4 model refinement. However, significant uncertainty often remains, and emphasis is then placed

5 on characterizing the nature of that uncertainty and its impact on risk estimates. The

6 characterization of uncertainty can be both qualitative and, if a sufficient knowledgebase is

7 available, quantitative.

8 The characterization of uncertainty associated with risk assessment is often addressed in

9 the regulatory context using a tiered approach in which progressively more sophisticated

10 methods are used to evaluate and characterize sources of uncertainty depending on the overall

11 complexity of the risk assessment (WHO, 2008). Guidance documents developed by EPA for

12 assessing air toxics-related risk and Superfund Site risks (U.S. EPA, 2004 and 2001, 

13 respectively) as well as recent guidance from the World Health Organization (WHO, 2008) 

14 specify multitier approaches for addressing uncertainty. 

15 For the ozone risk assessment, as noted above in section 5.1, we are planning to use a 

16 tiered framework developed by WHO to guide the characterization of uncertainty. The WHO 

17 guidance presents a four-tiered approach, where the decision to proceed to the next tier is based 

18 on the outcome of the previous tier’s assessment. The four tiers described in the WHO guidance 

19 include:

20 Tier 0: recommended for routine screening assessments, uses default uncertainty actors 

21 (rather than developing site-specific uncertainty characterizations);

22 Tier 1: the lowest level of site-specific uncertainty characterization, involves qualitative

23 characterization of sources of uncertainty (e.g., a qualitative assessment of the general 

24 magnitude and direction of the effect on risk results);

25 Tier 2: site-specific deterministic quantitative analysis involving sensitivity analysis, 

26 interval-based assessment, and possibly probability bound (high- and low-end) 

27 assessment; and

 

1

It is also important to point out that failure to characterize variability in an input used in modeling can also 

introduce uncertainty into the analysis. This reflects the important link between uncertainty and variability with the 

effort to accurately characterize variability in key model inputs actually reflecting an effort to reduce uncertainty. 
5-22

1 Tier 3: uses probabilistic methods to characterize the effects on risk estimates of sources 

2 of uncertainty, individually and combined.

3 With this four-tiered approach, the WHO framework provides a means for systematically

4 linking the characterization of uncertainty to the sophistication of the underlying risk assessment.

5 Ultimately, the decision as to which tier of uncertainty characterization to include in a risk

6 assessment will depend both on the overall sophistication of the risk assessment and the

7 availability of information for characterizing the various sources of uncertainty.

8 The risk assessment to be completed for the ozone NAAQS review is relatively complex, 

9 thereby warranting consideration of a full probabilistic (WHO Tier 3) uncertainty analysis. 

10 However, we anticipate that limitations in available information will likely prevent this level of

11 analysis from being completed. In particular, the incorporation of uncertainty related to key 

12 elements of C-R functions (e.g., competing lag structures, alternative functional forms, etc.) into 

13 a full probabilistic WHO Tier 3 analysis would require that probabilities be assigned to each 

14 competing specification of a given model element (with each probability reflecting a subjective 

15 assessment of the probability that the given specification is the ―correct‖ description of reality). 

16 However, for many model elements we expect that there will be insufficient information on 

17 which to base these probabilities. One approach that has been taken in such cases is expert 

18 elicitation; however, this approach is resource- and time-intensive and consequently, it is not 

feasible to use this technique in support of the ozone risk assessment.119

20 For most elements of this risk assessment, rather than conducting a full probabilistic

21 uncertainty analysis, we do expect to include a qualitative discussion of the potential impact of

22 uncertainty on risk results (WHO Tier1) and/or completed sensitivity analyses assessing the

23 potential impact of sources of uncertainty on risk results (WHO Tier 2). In conducting sensitivity 

24 analyses, we are planning to use both single- and multi-factor approaches (to look at the 

25 individual and combined impacts of sources of uncertainty on risk estimates). In addition, in 

 

1 Note, that while we anticipate that a full probabilistic uncertainty analysis will not completed for this risk 

assessment, we are expecting to use confidence intervals associated with effects estimates (obtained from 

epidemiological studies) to incorporate statistical uncertainty associated with sample size considerations in the 

presentation of risk estimates. Technically, this type of probabilistic simulation represents a Tier 3 uncertainty 

analysis, although as noted here, it will be limited and only address uncertainty related to the fit of the C-R 

functions.
5-23

1 conducting sensitivity analyses, we expect to use only those alternative specifications for input 

2 parameters or modeling approaches that are deemed to have scientific support in the literature 

3 (and so represent alternative reasonable input parameter values or modeling options). This means 

4 that, as discussed earlier in section 5.1, the alternative risk results generated in the sensitivity

5 analyses are expected to represent reasonable risk estimates that can be used to provide a context, 

6 with regard to uncertainty, within which to assess the set of core (base case) risk results. 

7 Potential sources of uncertainty included in the sensitivity analysis are presented below in

8 section 5.3.4.

9 The remainder of this section discusses how we are planning to address variability and 

10 uncertainty within the ozone NAAQS risk assessment. The treatment of variability is discussed 

11 first (section 5.4.2) by identifying sources of variability associated with the modeling of ozone12 related risk and noting which of those sources are reflected in the risk modeling approach 

13 presented here. Next, the treatment of uncertainty is addressed, which will include both a 

14 qualitative and quantitative component. The qualitative component is described first (section

15 5.4.3), including plans for identifying and describing key sources of uncertainty, and noting 

16 whether those sources of uncertainty are addressed quantitatively in the risk assessment model. 

17 A preliminary list of key sources of uncertainty for the risk assessment is provided as part of this 

18 discussion. The quantitative component of the uncertainty characterization approach, which is 

19 structured around single-factor and multi-factor sensitivity analysis methods, is then described 

20 (section 5.4.4). The representativeness analysis planned to support interpretation of the urban 

21 study area-level risk estimates is discussed in section 5.4.5.

22 5.4.2 Addressing Variability 

23 Key sources of variability associated with the modeling of population-level risk 

24 associated with ozone exposure are presented below, including whether, and to what extent, we 

25 plan to address each source of variability:

26 Spatial gradients in ozone (and related population exposure): This source of 

27 variability is likely to be less-well captured in the risk assessment primarily because the 

28 majority of epidemiological studies providing effect estimates are themselves limited in 

29 reflecting more detailed patterns of ozone exposure among populations. More 
5-24

1 specifically, the epidemiological studies typically use an average ambient concentration 

2 developed across population-oriented monitors as a surrogate for exposure. Note, 

3 however that the exposure assessment described in Chapter 3 may allow this issue to be 

4 investigated to some degree, particularly as it impacts on exposure error misclassification 

5 in the epidemiological studies underpinning the C-R functions used in this risk 

6 assessment. In addition, a few epidemiological studies being considered for inclusion in 

7 this analysis include more refined characterization of population-level exposure (e.g., 

8 based on more spatially differentiated linkages between population-level monitors and 

9 segments of the study population). We plan to consider the use of those studies with more 

10 refined population exposure characterization to examine the issue of spatial gradients in 

11 ozone and demographics and the degree to which this source of variability impacts risk 

12 estimates. 

13 Demographics (i.e., greater concentrations of susceptible populations in certain 

14 locations): We plan to include multiple urban study areas reflecting differences in 

15 demographics in different regions of the country to address this issue. In addition, as 

16 noted in the previous bullet, we plan to consider studies with more refined 

17 characterization of population-level exposure, to provide insights into the degree to which 

18 this source of variability impacts risk estimates. 

19 Behavior related to ozone exposure (e.g., outdoor time, air conditioning use): We 

20 plan to include multiple urban study areas reflecting differences in a variety of factors 

21 related to ozone exposure (e.g., time spent outdoors, air conditioner use, housing stock, 

22 which can affect ozone infiltration, and commuting patterns).

23 Susceptibility to specific populations to ozone exposure (note – this could include a 

24 number of factors e.g., magnitude of the effect estimate, underlying health status): We 

25 plan to consider this source of variability by using effect estimates and lag structures 

26 specific to each urban study location.

27 Differences in baseline incidence of disease: This source of variability would 

28 potentially be captured through the use of localized baseline incidence data (e.g., county29 level).

30 Longer-term temporal variability in ambient ozone levels (reflecting meteorological 

31 trends, as well as future changes in the mix of ozone sources and regulations affecting

32 ozone): This is more difficult to incorporate into the analysis and reflects a combination 

33 of variability as well as uncertainty.
5-25

1 5.4.3 Uncertainty Characterization – Qualitative Assessment

2 As noted in section 5.4.1, we are planning to base the uncertainty analysis carried out for 

3 this risk assessment on the framework outlined in the WHO guidance document (WHO, 2008). 

4 That guidance calls for the completion of a Tier 1 qualitative uncertainty analysis, provided the 

5 initial Tier 0 screening analysis suggests the-re is concern that uncertainty associated with the 

6 analysis is sufficient to significantly affect risk results (i.e., to potentially affect decision making 

7 based on those risk results). Ozone risk assessments completed for previous NAAQS reviews 

8 have clearly identified sources of uncertainty that could have significant impacts on risk 

9 estimates, thereby allowing us to skip a Tier 0 assessment and proceed directly to a Tier 1 

10 analysis (i.e., a qualitative discussion of potential sources of uncertainty including an assessment 

11 of the nature, magnitude and potential direction of impact of each source of uncertainty on the 

12 core risk estimates). A preliminary list of potentially important sources of uncertainty likely to 

13 be included in a Tier 1 assessment has been developed for this plan and is presented below (note, 

14 some of these sources may be addressed in the quantitative uncertainty analysis, when feasible): 

15 Procedure for characterizing recent air quality for urban study areas: There is 

16 uncertainty associated with characterizing recent air quality conditions at individual 

17 urban study areas. This uncertainty is reflected in the number of decisions or options that 

18 must be considered in designing an approach for characterizing recent air quality 

19 including: (a) whether to rely on monitoring data or to combine it with modeling data, (b) 

20 how to match ambient ozone levels to potentially exposed populations from a spatial 

21 standpoint (e.g., use a single composite monitor or a more differentiated polygon-based 

22 exposure surface, and (c) if a composite monitor approach is used, whether to design that 

23 composite monitor to most closely match the way monitoring data were used in the 

24 underlying epidemiological study providing the C-R function or design it to be more 

25 representative of potentially current exposures (e.g., weight it by activity profiles 

26 simulated for the current population). 

27 Procedures for adjusting air quality to simulate alternate standard levels: There is 

28 uncertainty in developing the method for adjusting current ambient ozone levels (at 

29 individual monitors used in the risk assessment) to simulate just attaining alternative 

30 standard (methods available are likely to include both retrospective empirical monitor31 based trend analysis and forward-looking model-based predictions – see section 3.2.1 and 

32 section 2.3 for additional detail).
5-26

1 Estimates of policy-relevant background ozone levels in a particular location. There 

2 is uncertainty associated with characterizing background for individual locations (see 

3 Section 5.2.2 for additional detail).

4 The impact of historical air quality on estimates of health risk from long-term ozone

5 exposures (i.e., the amount of time that a population experiences new lower ambient 

6 ozone levels before there is a noticeable reduction in health effect incidence): Some 

7 studies of long-term mortality provide effect estimates differentiated by consecutive, 

8 multi-year time periods. These studies may provide insights into this issue and the degree 

9 to which it could affect risk estimates (by providing different effect estimates). 

10 Statistical uncertainty associated with the fit of the C-R function.

11 Shape of the C-R function: Of particular concern is uncertainty related to the shape of 

12 the C-R function at lower exposure levels.

13 Potential role of co-pollutants and different lag structures: these are related to the C14 R function (and nature of the associated effects estimate).

15 Transferability of C-R functions from study locations to urban study area locations: 

16 this reflects variation in (a) ozone distributions, (b) the possible role of copollutants in 

17 influencing risk, (c) relationship between ambient ozone and actual exposure, and (d) 

18 differences in population characteristics. However, it is anticipated that the transferability 

19 issue will play less of a role in the upcoming analysis, since studies used to derive C-R 

20 functions will often be matched to our urban study area locations. However, there may 

21 still be transferability issues arising from changes in these factors between the time 

22 period when the C-R functions were estimated and the time period of this risk analysis.

23 5.4.4 Uncertainty Characterization – Quantitative Analysis

24 In addition to the Tier 1 qualitative assessment of uncertainty discussed in the previous 

25 section, we are also anticipating that we will complete a Tier 2 assessment of uncertainty, which 

26 involves application of deterministic methods including sensitivity analysis and bounding 

analyses.

1

27 For this ozone risk assessment, we are planning to focus primarily on single and 

28 multi-factor sensitivity analyses which are intended to (a) help identify which uncertainty factors 

29 (acting either alone, or in concert with other factors) have a significant impact on the core risk 

 

1 As noted earlier, uncertainty related to the statistical fit of the C-R functions will be addressed using probabilistic 

simulation to derive confidence intervals around the core risk estimates (i.e., a Tier 3 probabilistic approach towards 

characterizing the impact of uncertainty on core risk estimates). 
5-27

1 estimates and (b) generate an alternative reasonable set of risk estimates that supplement the core 

risk estimates and involve overall consideration of uncertainty in the risk estimates.12 This 

3 quantitative uncertainty analysis would likely focus on a subset of the sources of uncertainty 

4 identified above for the Tier 1 assessment with this subset reflecting sources of uncertainty for 

5 which we can clearly identify competing datasets or modeling approaches with some degree of 

6 support in the literature. Table 5-1 identifies those modeling elements that are being considered

7 for inclusion in the sensitivity analysis and includes identification of the options for each 

8 modeling element that could be considered in the sensitivity analysis. Note, that in each case, 

9 one of these options will likely be identified for the core analysis, with the remaining option(s) 

10 either (a) being included as sensitivity analyses, or (b) discussed qualitatively as part of the 

11 overall uncertainty analysis. However, we are not prepared at this stage in the planning process 

12 to identify core versus sensitivity analysis options, or to specific which alternative approaches 

13 will be included in the quantitative sensitivity analysis versus covered qualitatively. 

14 The step-wise procedure for conducting the deterministic uncertainty analysis is 

15 illustrated in Figure 5-2. It is important to point out that we plan to generate a core set of risk 

16 estimates prior to conducting the uncertainty analysis. This core set of risk estimates would be 

17 derived by first applying the criteria discussed in preceding sections (sections 5.2 and 5.3) to 

18 identify those options for key modeling elements which have the strongest scientific support 

(with these determinations being based primarily on the evaluation provided in the ISA).219 The 

20 core set of risk estimates will be generated for each combination of urban study area and air 

21 quality scenario.

 

1 As noted earlier, ideally, we would also include a 2-dimensional probabilistic analysis of uncertainty and 

variability (i.e., a Tier 3 assessment in the WHO framework), since this would allow us to provide a more complete 

and integrated characterization of uncertainty and variability associated with risk estimates. However, limitations in 

our ability to assign rigorous and defensible confidence levels to competing modeling approaches and input datasets 

is expected to prevent us from completing this type of analysis. 

2

For example, as noted in section 5.3.1, if a study provides both single-day and distributed lag models, the 

distributed lag model would be used in the core analysis, while the individual day lags, if retained in the risk 

assessment, would be included in the uncertainty analysis. With regard to non-linearity in functions, including the 

potential for thresholds, generally non-threshold models will be used in the core analysis (based on information 

provided in the ISA) and thresholds models, if they are considered at all, would be reserved for the uncertainty 

analysis. It is also important to point out that for some of the modeling elements, multiple options may be included 

as part of the core simulation (e.g., both multi- and single-component models may be used in core simulations for 

specific health endpoints).
5-28

1 Table 5-1. Planned Sensitivity Analyses for the Epidemiologic-Based Risk Assessment

Component of 

the Risk 

Assessment

Options Potentially Considered for the Sensitivity Analysis 

(Note, these include all options identified for a particular component or 

modeling element – one of the identified options will likely be identified 

for the core analysis, with the remainder being included in the sensitivity 

analysis)

Air quality

Characterization 

of recent air 

quality at urban 

study areas

use of composite monitors (or other aggregations of monitors) that are 

linked to the method used for representing ambient ozone levels in the 

underlying epidemiological studies providing the C-R functions,

use of a composite monitor approach that weights monitors by their 

contribution to population exposure (as reflected in 

microenvironmental exposure modeling) rather than matching 

structure to approach used in underlying epidemiological studies

use of model-monitor fused surfaces. Note, that these alternative 

methods for characterizing current air quality may only be assessed at 

a subset of urban study areas as part of the sensitivity analysis. 

Background 

concentrations

use monitor data alone, or in combination with modeling data, 

whether to use composite monitors or a more spatially-differentiated 

exposure surface. 

Key design 

element associated 

with air quality 

sensitivity 

analysis

As noted in section 2.2 and 5.1, a key aspect of designing the approach 

for characterizing air quality (including core and sensitivity analyses) is 

consideration for the degree of spatial heterogeneity in monitored ozone 

levels. If the spatial gradient within a study area in ozone levels is not that 

substantial, reflecting the dominance of secondary formation for ozone, 

than there may be little utility in considering varied approaches for 

deriving exposure surrogates and conducting rollbacks (i.e., in a study 

area with fairly uniform ozone levels at a given point in time, alternative 

approaches for these modeling steps may not produce meaningfully 

different results). Conversely , if a study area does have significant spatial 

heterogeneity in ozone levels, then the approach used to characterize 

current ozone levels and link it to population and the approach used to 

conduct rollback could have a notable impact on risk estimates. 
5-29

Selection of health effect endpoint categories (and endpoints)

Health effect 

endpoints to 

model

As noted in section 5.3, the core analysis will focus on endpoints 

contained within health effect endpoint categories assigned a causal or 

likely causal association with ozone exposure. The Sensitivity Analysis 

may include endpoints (e.g., developmental) that are assigned a 

suggestive of casual association classification. As noted in section 5.3, the 

specific category of long-term exposure-related respiratory mortality (if 

modeled), may be included in the core analysis, or as part of the 

Sensitivity Analysis depending on how the degree of support for an 

association with ozone exposure is ultimately assessed.

Exposure-response functions

Extrapolation 

below lowest 

levels of exposure 

used in studies 

Extend model without modification below lowest exposure level

Extend model to the lowest exposure level reflected in the underlying 

epidemiologic study (with this reflecting a higher-confidence 

calculation)

Consider alternate model forms for points below lowest exposure 

level (if there is some rationale for this supported by study data, 

including toxicological information). 

Consideration for thresholds (only for long-term exposure-related 

endpoints). 

Consideration for 

alternative C-R 

functions 

reflecting different 

model constructs 

(e.g., single vs.

multi-pollutant 

functions, single 

vs. multi city 

studies)

As noted in section 5.3.1, if an epidemiological study provides multiple 

functions reflecting for example, a single versus multi-pollutant model, 

we will include both forms. Looking more broadly, we will also include 

C-R functions (for a given endpoint) from single and multi-city studies 

given the strengths afforded by each (assuming that each study meets 

criteria for a sound analysis). 

Bayesian-adjusted 

county-level 

estimates from 

multi-city shortterm studies

Consideration for impact of using county-level Bayesian-adjusted C-R 

functions (extracted form multi-city short-term studies) versus application 

of the national-level effect estimates originally provided by these multicity studies.

Baseline Incidence

Aggregation scale Consideration for more aggregate baseline incidence data (national, state, 

etc.) versus county-specific information in the county with the best local 

baseline incidence data

1

2
5-30

1

2 Figure 5-2. Overview of Approach For Uncertainty Analysis of Risk Assessment Based on Epidemiologic Studies

Ambient Monitoring Data for Selected 

Urban Areas (specification of composite 

monitor)

Air Quality Adjustment (roll-back) 

Procedures

City-specific Demographic Data

Selection of Human Epidemiological 

Studies to Provide ConcentrationResponse Functions

Specification of Concentration -

Response Relationships – consider:

• single- v.s. multi-chemical

• single- v.s. multi-city

• lag

• seasonally-differentiated

• slope of CR function (threshold)

Concentration-Response:

Air Quality:

Modeled Background Concentrations 

(PRB)

Overview of Uncertainty Analysis Approach Developed for the Ozone NAAQS Risk Assessment

Estimates of City-specific Baseline 

Health Effects Incidence Rates

Baseline Health Effects Incidence Rates 

and Demographics:

Risk Assessment Modeling Elements

Apply criteria for 

selecting subset of 

modeling element 

options for use in 

the core analysis

Generate core set of risk 

estimates for each 

combination of urban study 

area and air quality scenario

Apply single-factor 

sensitivity analysis to 

identify those 

modeling elements 

with a significant 

impact on risk 

results

Identify additional plausible 

modeling options (distinct from 

those used in the core analysis) 

for this set of key modeling 

elements

Uncertainty Analysis 

Producing Additional Set of 

Plausible Risk Estimates

Conduct multi-factor sensitivity 

analysis (for this subset of key 

modeling options) to generate 

alternative sets of reasonable risk 

estimates for a subset of urban 

study areas and air quality 

scenarios

Primary Analysis Generating Core Set of Risk Estimates
5-31

1 Once the core set of risk estimates has been generated, the uncertainty analysis will begin 

2 with a single-factor sensitivity analysis intended to identify those modeling elements (comprising 

3 the ozone risk assessment framework) that have the potential to significantly impact risk 

4 estimates. This set of key modeling elements would form the basis for the uncertainty analysis. 

5 Next, plausible modeling options (distinct from those used in the core analysis) would be

6 specified for each of these key modeling elements. In identifying these plausible modeling 

7 options, we plan to place emphasis on identifying input factors or modeling approaches, which, 

8 while representing alternatives to those used in the core simulation, still have some degree of 

9 scientific support in the literature. Consequently, while we may have less confidence in risk 

10 estimates generated using these alternate modeling options relative to the core risk estimates, 

11 they could still considered reasonable and consequently may be interpreted as providing 

12 additional perspective on overall uncertainty associated with the core set of risk estimates. 

13 Once the set of plausible modeling options is specified for the key modeling elements, we 

14 plan to use a multi-factor sensitivity analysis to generate a set of reasonable alternative risk 

15 estimates. Specifically, various combinations of these alternative modeling options would be

used to generate risk estimates, each representing an uncertainty simulation. 16 1 We plan to 

17 generate this set of alternative risk estimates for a subset of the urban study areas and air quality 

18 scenarios.

19 The combined sets of core results and alternative risk estimates (for a combination of 

20 urban study area and air quality scenario) could be interpreted as representing an initial 

21 characterization of risk for that combination of urban study area and air quality scenario, 

22 reflecting recognized sources of uncertainty in risk modeling. However, this interpretation needs 

23 to be tempered by consideration of several factors: (a) this does not represent a characterization 

24 of a distribution of uncertainty around the core set of risk estimates, it merely represents several 

25 point estimates likely falling within that uncertainty distribution and (b) the set of modeled risk 

26 estimates may not contain actual upper-bound and lower-bound risk estimates given 

27 scientifically defensible modeling options. Despite these caveats, the risk estimates defined by 

 

1 Note, that care would be taken in linking these modeling options together to insure that they are compatible and do 

not represent combinations that are scientifically not defensible.
5-32

1 the sets of core and alternative risk estimates should be useful to characterize confidence 

2 associated with the results of the application of the ozone risk assessment model.

3 5.4.5 Representativeness Analysis

4 As discussed in section 5.1, we are planning to complete a representativeness analysis 

5 designed to support the interpretation of risk estimates generated for the set of urban study areas 

6 included in the risk assessment. The representativeness analysis will focus on comparing urban 

7 study area-level values to national-scale distributions for key ozone-risk related attributes (e.g., 

8 demographics including socioeconomic status, air-conditions use, baseline incidence rates and 

9 ambient ozone levels). The goal, with these comparisons will be to assess the degree to which 

10 the urban study areas provide coverage for different regions of the country as well as for areas of 

11 the country likely to experience elevated ozone-related risk due to their specific mix of attributes 

12 related to ozone risk. 

13 The national-scale distributions of ozone risk-related parameters would be specified at 

14 the country-level and would be based on generally available data, e.g. from the 2000 Census, 

15 CDC, or other sources. The specific values of these parameters for the selected urban study areas 

16 would then be plotted on these national-scale distributions, and an evaluation of how 

17 representative the selected study areas are of the individual parameters, relative to the national 

18 distributions, could be completed. The specific choices of parameters for which we would 

19 examine the representativeness of the selected urban study areas would be informed through an 

20 assessment of the epidemiology literature. We plan to particularly focus on meta-analyses and 

21 multi-city studies which have identified parameters that influence heterogeneity in ozone effect 

22 estimates, and exposure studies which have explored determinants of differences in personal 

23 exposures to ambient ozone. While personal exposure is not generally incorporated directly into 

24 epidemiology studies evaluating ambient ozone-related effects, differences in the ozone effect 

25 estimates between cities clearly is impacted by differing levels of those exposure determinants. 

26 Once we have identified these parameters, we plan to develop city-specific distributions for those 

27 parameters (or reasonable surrogates) based on readily available data sources. Formal 

28 comparisons of parameter distributions for the set of urban study areas and the city-specific

29 parameter distributions using standard statistical tests (e.g. the Kolmogorov-Smirnov test for 
5-33

1 equality of distributions) would not be useful in this context, since we are more interested in 

2 meaningful differences than statistical significance. Therefore, we plan to use graphical

3 comparisons using probability density functions, cumulative distribution functions, and boxplots.

4 As noted in section 5.1, as part of the representativeness analysis, we are also considering

5 generating national-scale ozone mortality estimates (based both on long-term and short-term 

6 exposure). In the context of both short-term and long-term exposure-related mortality, these 

7 national-level estimates could be used to assess the degree to which our urban study areas 

8 capture urban areas across the U.S. that potentially experience the greatest ozone-related 

9 mortality. For long-term mortality, we would consider a national-scale assessment conducted at 

10 the county-level using the same national-level effect estimate obtained from the cohort study 

11 used in modeling each urban study area (with this assessment likely focusing on respiratory 

12 mortality, as discussed above and in section 5.1). For short-term exposure-related mortality, 

13 rather than generating a comprehensive national-estimate that has full coverage (i.e., covers all 

14 counties in the U.S.), we would likely model the set of urban areas included in the time series 

15 study that provided the effect estimates used in the primary estimate of short-term mortality 

16 generated for our urban study areas (i.e., we would model mortality for each of the urban study 

17 areas included in the underlying time series study). While the mortality estimate for short-term 

18 exposure would not be truly national (in that it would not cover all counties in the country), by 

19 including most of the larger urban areas in the U.S. it would provide close to a national estimate. 
6-1

1 6 PRESENTATION OF RISK ESTIMATES TO INFORM CONSIDERATION OF 

2 STANDARDS

3 This section discusses the nature of the risk estimates that we plan to generate as part of 

4 the review of the ozone NAAQS. We plan to conduct the risk assessment in two phases. Phase 1 

5 would include analysis of risk associated with recent air quality and simulating air quality to just 

6 meet the current NAAQS. Phase 2 would focus on evaluating risk associated with simulating air 

7 quality that just meets alternative NAAQS under consideration. 

8 We plan to present risk estimates in two ways: (1) total (absolute) health effects 

9 incidence (above background) for recent air quality and simulations of air quality just meeting 

10 the current and alternative NAAQS under consideration, and (2) risk reduction estimates,

11 reflecting the difference between (a) risks associated with recent air quality compared to risks 

12 associated with just meeting the current NAAQS and (b) reflecting the difference between risks 

13 associated with just meeting the current NAAQS compared to risks associated with just meeting

14 alternative NAAQS under consideration. 

15 In presenting risk estimates, we plan to emphasize the core (base-case) estimates given 

16 that these would include risk estimates with greater overall confidence. We plan to also present 

17 additional risk estimates generated as part of the uncertainty analyses in order to provide 

18 additional context for understanding the potential impact of uncertainty on the risk estimates and 

19 particularly on the core estimates of risk. The results of the representativeness analysis 

20 (discussed in section 5.4.5) would likely be presented using a combination of (a) cumulative 

21 probability plots (for the national-level distribution of ozone risk-related parameters) with the 

22 locations where the individual urban study areas fell within those distributions noted in the plots 

23 and (b) box and whisker plots, again contrasting the national-scale distribution of the ozone risk24 related parameters with the values for individual urban study areas. Similar types of plots would 

25 be used to present national-scale mortality estimates (should they be generated); contrasting them 

26 with estimates generated for individual urban study areas. 
7-1

1 7 SCHEDULE AND MILESTONES

2 The Integrated Review Plan provides an overview of ozone review schedule. Table 7-1

3 below includes the key milestones for the exposure analysis and health risk assessment that will 

4 be conducted as part of the current ozone NAAQS review. A consultation with the CASAC 

5 Ozone Panel is planned for May 19-20, 2011 to obtain input on this draft Scope and Methods 

6 Plan. Staff will then proceed to develop exposure and health risk estimates associated with 

7 recent ozone levels and levels adjusted to just meet the current 8-hour ozone standard. These 

8 estimates and the methodology used to develop them will be discussed in the first draft ozone 

9 exposure analysis and health risk assessment. This draft report will be released for CASAC and 

10 public review in October 2011. EPA will receive comments on these draft documents from the 

11 CASAC Ozone Panel and general public at a meeting in November 2011. The revised exposure 

12 analysis and risk assessment reports will include estimates associated with just meeting any 

13 alternative standards that may be recommended by staff for consideration. The revised analyses 

14 will be released in May 2012 for review by CASAC and the public at a meeting to be held in 

15 July 2012. Staff will consider these review comments and prepare a final exposure analysis and 

16 risk assessment report by October 2012.

17 Table 7-1. Key Milestones for the Exposure Analysis and Health Risk Assessment for the 

18 Ozone NAAQS Review

Milestone Date

First Draft Integrated Science Assessment (ISA) March 2011

Scope and Methods Plan for the Exposure Analysis and Health Risk 

Assessment

April 2011

CASAC/public review and meeting on First Draft ISA May 19-20, 2011

CASAC consultation on Scope and Methods Plan May 19-20, 2011

Second Draft ISA September 2011

First Draft Exposure Analysis and Risk Assessment October 2011

CASAC/public review and meeting on Second Draft ISA and First 

Draft Exposure Analysis and Risk Assessment

November 2011

Final ISA February 2012

Second Draft Exposure Analysis and Risk Assessment May 2012

First Draft Policy Assessment June 2012
7-2

1

CASAC/public review and meeting on Second Draft Exposure 

Analysis and Risk Assessment and First Draft Policy Assessment

July 2012

Final Exposure Analysis and Risk Assessment October 2012

Second Draft Policy Assessment November 2012

CASAC/public review of Second Draft Policy Assessment January 2013

Final Policy Assessment March 2013

Proposed Rulemaking September 2013

Final Rulemaking June 2014
8-1

1 8 REFERENCES 

2 Abt Associates Inc. (2005). Particulate Matter Health Risk Assessment for Selected Urban 

3 Areas. Final Report. Bethesda, MD. Prepared for the Office of Air Quality Planning and 

4 Standards, U.S. Environmental Protection Agency, Contract No. 68-D-03-002. EPA 452/R5 05-007A. Available: http://www.epa.gov/ttn/naaqs/standards/pm/s_pm_cr_td.html.

6 Abt Associates Inc. (2007). Ozone Health Risk Assessment for Selected Urban Areas. Prepared 

7 for Office of Air Quality Planning and Standards, U.S. Environmental Agency, Research 

8 Triangle Park, NC, July 2007. Available at: 

9 http://epa.gov/ttn/naaqs/standards/ozone/data/ozone_ra_final_tsd_7-2007.pdf. 

10 Abt Associates Inc. (2008). Environmental Benefits Mapping and Analysis Program (Version 

11 3.0). Bethesda, MD. Prepared for Environmental Protection Agency, Office of Air Quality 

12 Planning and Standards, Air Benefits and Cost Group. Research Triangle Park, NC. 

13 Adams, W.C. (2002). Comparison of chamber and face-mask 6.6-hour exposures to ozone on 

14 pulmonary function and symptoms responses. Inhalation Toxicology. 14:745-764. 

15 Adams, W.C. (2003). Comparison of chamber and face-mask 6.6-hour exposure to 0.08 ppm 

16 ozone via square-wave and triangular profiles on pulmonary responses. Inhalation 

17 Toxicology. 15: 265-281. 

18 Adams, W.C. (2006). Comparison of chamber 6.6-hour exposures to 0.04-0.08 ppm ozone via 

19 square-wave and triangular profiles on pulmonary responses. Inhalation Toxicology. 18: 

20 127-136. 

21 Akland, G. G., Hartwell, T. D., Johnson, T. R., Whitmore, R. W. (1985). Measuring human 

22 exposure to carbon monoxide in Washington, D. C. and Denver, Colorado during the winter 

23 of 1982-83. Environ Sci Technol. 19: 911-918. 

24 Avol EL, Linn WS, Shamoo DA, Spier CE, Valencia LM, Venet TG, Trim SC, Hackney JD. 

25 1987. Short-term respiratory effects of photochemical oxidant exposure in exercising 

26 children. JAPCA. 37(2):158-62. 

27 Avol EL, Linn WS, Shamoo DA, Valencia LM, Anzar UT, Venet TG, Hackney JD. 1985. 

28 Respiratory effects of photochemical oxidant air pollution in exercising adolescents. Am 

29 Rev Respir Dis. 132(3):619-22. 

30 Bell, M,A. McDermott, S.L. Zeger, J.M. Samet, and F. Dominici (2004). ―Ozone and short-term 

31 mortality in 95 U.S. urban communities, 1987-2000.‖ JAMA 292(19):2372-2378. 

32 Bey I., et al. (2001). Global modeling of tropospheric chemistry with assimilated meteorology: 

33 Model description and evaluation, J. Geophys. Res., 106, 23,073-23,096, 2001. 

34 Biller, W., Feagans, T., Johnson, T., Duggan, G., Paul, R., McCurdy, T. and Thomas, H. (1981). 

A general model for estimating exposure associated with alternative NAAQS. 74th 35 annual 

36 meeting of the APCA, Philadelphia, PA. 

37 Blanken, Peter D., Jennifer Dillon, Genevieve Wismann. 2001. The impact of an air quality 

38 advisory program on voluntary mobile source air pollution reduction, Atmospheric 

39 Environment, Volume 35, Issue 13, May 2001, Pages 2417-2421. 
8-2

1 Bureau of Labor Statistics, U.S. Department of Labor. (2010). The American Time-Use Survey, 

2 http://www.bls.gov/tus/. 

3 Centers for Disease Control and Prevention (CDC). (2010). National Center for Health 

4 Statistics. National Health Interview Survey (NHIS) Public Use Data Release. Available at: 

5 http://www.cdc.gov/nchs/nhis.htm 

6 Cullen, A.C. and Frey, H.C. (1999). Probabilistic Techniques in Exposure Assessment: A 

7 Handbook for Dealing with Variability and Uncertainty in Models and Inputs. Springer. 

8 Di Novi, C. 2010. The influence of traffic-related pollution on individuals' life-style: results from 

9 the BRFSS. Health Econ. 2010 Nov;19(11):1318-44. 

10 Drechsler-Parks DM, Bedi JF, Horvath SM. 1987. Pulmonary function responses of older men 

11 and women to ozone exposure. Exp Gerontol. 22(2):91-101. 

12 Drechsler-Parks DM, Bedi JF, Horvath SM. 1989. Pulmonary function responses of young and 

13 older adults to mixtures of O3, NO2 and PAN. Toxicol Ind Health. 5(3):505-17. 

14 Eiswerth, M., Shaw, D., and Yen, S. (2005). Impacts of ozone on the activities of asthmatics: 

15 Revisiting the data. J. Env. Mgt. 77, 1:56-63. 

16 Fiore, A.M., ET AL. (2009). Multimodel estimates of intercontinental source-receptor 

17 relationships for ozone pollution, J. Geophys. Res., 114, D04301. 

18 Fiore, A.M., J. J. West, L. W. Horowitz, V. Naik, and M. D. Schwarzkopf (2008). Characterizing 

19 the tropospheric ozone response to methane emission controls and the benefits to climate 

20 and air quality, J. Geophys. Res., 113, D08307. 

21 Folinsbee, L.J.; McDonnell, W.F.; Horstman, D.H. (1988). Pulmonary function and symptom 

22 responses after 6.6-hour exposure to 0.12 ppm ozone with moderate exercise. JAPCA. 38: 

23 28-35. 

24 Glen, G., Smith, L., Isaacs, K., McCurdy, T. and Langstaff, J. (2008). A new method of 

25 longitudinal diary assembly for human exposure modeling. Journal of Exposure Science 

26 and Environmental Epidemiology 18, 299–311. 

27 Gong H Jr, Shamoo DA, Anderson KR, Linn WS. 1997. Responses of older men with and 

28 without chronic obstructive pulmonary disease to prolonged ozone exposure. Arch Environ 

29 Health. 52(1):18-25. 

30 Graham, S. E. and McCurdy, T. (2004). Developing meaningful cohorts for human exposure 

31 models. J Expo Anal Environ Epidemiol. 14: 23-43. 

32 Haas, C. (1997). Importance of Distributional Form in Characterizing Inputs to Monte Carlo 

33 Risk Assessments. Risk Analysis, 17:107-113. 

34 Haas, C. (1999). On Modeling Correlated Random Variables in Risk Assessment. Risk 

35 Analysis, 19:1205-1214. 

36 Hartwell, T. D., Clayton, C. A., Ritchie, R. M., Whitmore, R. W., Zelon, H. S., Jones, S. M., 

37 Whitehurst, D. A. (1984). Study of Carbon Monoxide Exposure of Residents of 

38 Washington, DC and Denver, Colorado. Research Triangle Park, NC: U.S. Environmental 
8-3

1 Protection Agency, Office of Research and Development, Environmental Monitoring 

2 Systems Laboratory. EPA-600/4-84-031. 

3 Hazucha MJ, Folinsbee LJ, Bromberg PA. 2003. Distribution and reproducibility of spirometric 

4 response to ozone by gender and age. J Appl Physiol. 95(5):1917-25. 

5 Hofferth, S., Davis-Kean, P.E., Davis, J., & Finkelstein, J. (1999). The Child Development 

6 Supplement to the Panel Study of Income Dynamics: 1997 User Guide. Available at: 

7 http://psidonline.isr.umich.edu/CDS/usergd.html

8 Horstman, D.H.; Folinsbee, L.J.; Ives, P.J.; Abdul-Salaam, S.; McDonnell, W.F. (1990). Ozone 

9 concentration and pulmonary response relationships for 6.6-hr exposures with five hrs of 

10 moderate exercise to 0.08, 0.10, and 0.12 ppm. Am Rev Respir Dis. 142:1158-1163. 

11 Johnson, T. (1984). A Study of Personal Exposure to Carbon Monoxide in Denver, Colorado. 

12 Research Triangle Park, NC: U.S. Environmental Protection Agency, Environmental 

13 Monitoring Systems Laboratory; Report No. EPA-600/4-84-014. 

14 Johnson, T. (1989). Human Activity Patterns in Cincinnati, Ohio. Palo Alto, CA: Electric 

15 Power Research Institute. EPRI EN-6204. 

16 Johnson, T., Capel, J., McCoy, M., and Warnasch, J. (1996a). Estimation of Ozone Exposures 

17 Experienced by Outdoor Children in Nine Urban Areas Using a Probabilistic Version of 

18 NEM. Prepared by International Technology Air Quality Services for Office of Air Quallity 

19 Planning and Standards, EPA, Research Triangle Park, NC. Available at: 

20 http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_pr.html 

21 Johnson, T., Capel, J., and McCoy, M. (1996b). Estimation of Ozone Exposures Experienced by 

22 Urban Residents Using a Probabilistic Version of NEM and 1990 Population Data. 

23 Prepared by International Technology Air Quality Services for Office of Air Quallity 

24 Planning and Standards, EPA, Research Triangle Park, NC. Available at: 

25 http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_pr.html 

26 Johnson, T., Capel, M. McCoy, and J. Warnasch. (1996c). Estimation of Ozone Exposures 

27 Experienced by Outdoor Workers in Nine Urban Areas Using a Probabilistic Version of 

28 NEM. 1996. Prepared by International Technology Air Quality Services for U.S. 

29 Environmental Protection Agency, OAQPS; contract no. 63-D-30094; Research Triangle 

30 Park, NC, April, 1996. Available at: 

31 http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_pr.html 

32 Klepeis, N. E., Tsang, A. M., Behar, J. V. (1996). Analysis of the National Human Activity 

33 Pattern Survey (NHAPS) Respondents from a Standpoint of Exposure Assessment. 

34 Washington, DC: U.S. Environmental Protection Agency, Office of Research and 

35 Development. EPA/600/R-96/074. 

36 Knowledge Networks (2009). Field Report: National Scale Activity Survey (NSAS). Conducted 

37 for Research Triangle Institute by Knowledge Networks, Menlo Park, CA. 

38 Langstaff, J.E. (2007). OAQPS Staff Memorandum to ozone NAAQS Review Docket (EPA39 HQ-OAR-2005-0172). Subject: Analysis of Uncertainty in Ozone Population Exposure 

40 Modeling. [January 31, 2007]. Available at: 

41 http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_cr_td.html
8-4

1 Liu L-JS, Box M, Kalman D, Kaufman J, Koenig J, Larson T, et al. (2003). Exposure assessment 

2 of particulate matter for susceptible populations in Seattle, WA Environ Health Perspect 

3 111:909–918. 

4 Mainieri, T., Blackburn, Z., Stafford, F., Eccles, J., Schoeni, R., Yeung, W., and McGonagle, K. 

5 (2004). The Panel Study of Income Dynamics Child Development Supplement: User Guide 

6 for CDS-II. The Institute for Social Research, Ann Arbor, MI. Available at: 

7 http://psidonline.isr.umich.edu/CDS 

8 Mansfield, C. and Corey, C. (2003). Analysis of Survey Data on Ozone Alert Days, Final 

9 Report, September 2003. Prepared for the U.S. Environmental Protection Agency by RTI 

10 International, Research Triangle Park, NC. EPA Contract Number 68-D-99-024. 

11 Mansfield, C., F. Reed Johnson, George Van Houtven. (2006). The missing piece: Valuing 

12 averting behavior for children’s ozone exposures. Resource and Energy Economics 28:215-

13 228. 

14 Mansfield, C., Van Houtven G, et al. (2004). Parental Averting Behavior With Respect To 

15 Ozone Alerts. Research Triangle Park NC: Research Triangle Institute. 

16 McCurdy, T. (1994). "Human exposure to ambient ozone," pp. 85-127 in: D.J. McKee (ed.) 

17 Tropospheric Ozone. Ann Arbor MI: Lewis Publishers. 

18 McCurdy, T. (1995). Estimating Human Exposure to Selected Motor Vehicle Pollutants Using 

19 the NEM Series of Models: Lessons To Be Learned. J. Expo. Anal. and Env. Epidemiol. 

20 5(4):533-550. 

21 McCurdy, T., Glen, G., Smith, L., Lakkadi, Y. (2000). The National Exposure Research 

22 Laboratory's Consolidated Human Activity Database. J Expo Anal Environ Epidemiol. 10: 

23 566-578. 

24 McDonnell W.F., Stewart P.W., Smith M.V. (2007). The temporal dynamics of ozone-induced 

25 FEV1 changes in humans: an exposure-response model. Inhal Toxicol 19:483–494. 

26 McDonnell W.F., Stewart P.W., Smith M.V. (2010). Prediction of ozone-induced lung function 

27 responses in humans. Inhal Toxicol. 22(2):160-8. 

28 McDonnell, W.F. et al. (1985). ―Reproducibility of individual responses to ozone exposure.‖ 

29 American Review of Respirtory Disease 131:36-40. 

30 McDonnell, W.F. et al. (1991). ―Respiratory response of humans exposed to low levels of ozone 

31 for 6.6 hours.‖ American Review of Respirtory Disease 147:804-810. 

32 McDonnell, W.F., Chapman RS, Leigh MW, Strope GL, Collier AM. 1985. Respiratory 

33 responses of vigorously exercising children to 0.12 ppm ozone exposure. Am Rev Respir 

34 Dis. 132(4):875-9. 

35 Morgan, M.G. and M. Henrion (1990). Uncertainty: A Guide to Dealing with Uncertainty in 

36 Qualitative Risk and Policy Analysis. Cambridge University Press. 

37 Neidell, M. (2005a). Public Information and Avoidance Behavior - Do People Respond to Smog 

38 Alerts? CISES Technical Report No. 24, The University of Chicago Center for Integrating 

39 Statistical and Environmental Science, Chicago, Illinois. 
8-5

1 Neidell, M. (2005b). Incorporating Behavior into Biological Models: Understanding the Health 

2 Effects of Ozone. CISES Technical Report No. 25, The University of Chicago Center for 

3 Integrating Statistical and Environmental Science, Chicago, Illinois. 

4 Neidell, M. (2010). Air quality warnings and outdoor activities: evidence from Southern 

5 California using a regression discontinuity design. J Epidemiol Community Health. 

6 64(10):921-6. Epub 2009 Oct 12. 

7 Neidell, Matthew, Patrick L. Kinney. 2010. Estimates of the association between ozone and 

8 asthma hospitalizations that account for behavioral responses to air quality information, 

9 Environmental Science & Policy, Volume 13, Issue 2, April 2010, Pages 97-103. 

10 Robinson, J. P., Wiley, J. A., Piazza, T., Garrett, K., and Cirksena, K. (1989). Activity Patterns 

11 of California Residents and their Implications for Potential Exposure to Pollution. 

12 California Air Resources Board, Sacramento, CA. CARB-A6-177-33. 

13 Semenza, Jan C., Daniel J. Wilson, Jeremy Parra, Brian D. Bontempo, Melissa Hart, David J. 

14 Sailor, Linda A. George. 2008. Public perception and behavior change in relationship to hot 

15 weather and air pollution, Environmental Research, Volume 107, Issue 3, July 2008, Pages 

16 401-411. 

17 Spier, C. E., Little, D. E., Trim, S. C., Johnson, T. R., Linn, W. S., Hackney, J. D. (1992). 

18 Activity patterns in elementary and high school students exposed to oxidant pollution. J 

19 Expo Anal Environ Epidemiol. 2: 277-293. 

20 Tsang A. M., and Klepeis, N. E. (1996). Descriptive Statistics Tables from a Detailed Analysis 

21 of the National Human Activity Pattern Survey (NHAPS) Data. U.S. Environmental 

22 Protection Agency. EPA/600/R-96/148. 

23 Tudor-Locke C, Washington TL, Ainsworth BE, Troiano RP. (2009). Linking the American 

24 Time Use Survey (ATUS) and the Compendium of Physical Activities: Methods and 

25 Rationale. J Phys Act Health 2009 May;6(3):347-53. 

26 Tudor-Locke, Catrine, William D. Johnson, Peter T. Katzmarzyk (2010). Frequently Reported 

27 Activities by Intensity for U.S. Adults: The American Time Use Survey, American Journal 

28 of Preventive Medicine, Volume 39, Issue 4, October 2010, Pages e13-e20. 

29 U.S. Department of Transportation and U.S. Census Bureau (2000). Census of Population and 

30 Housing, 2000 long-form (sample) data, Census Transportation Planning Package (CTPP) 

31 2000. Available at: http://transtats.bts.gov/

32 U.S. Environmental Protection Agency (1996a). Review of National Ambient Air Quality 

33 Standards for Ozone: Assessment of Scientific and Technical Information - OAQPS Staff 

34 Paper. EPA/452/R-96-007. Office of Air Quality Planning and Standards, Research 

35 Triangle Park, NC. Available from: NTIS, Springfield, VA; PB96–203435. Available at: 

36 http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_pr.html

37 U.S. Environmental Protection Agency (1996b). Air Quality Criteria for Ozone and Related 

38 Photochemical Oxidants. EPA/600/P-93/004aF-cF. Office of Research and Development, 

39 National Center for Environmental Assessment, Research Triangle Park, NC. Available at: 

40 http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=2831. 
8-6

1 U.S. Environmental Protection Agency (2002). Consolidated Human Activities Database 

2 (CHAD) Users Guide. Database and documentation available at: 

3 http://www.epa.gov/chadnet1/

4 U.S. Environmental Protection Agency (2006). Air Quality Criteria for Ozone and other 

5 Photochemical Oxidants. Research Triangle Park, NC: Office of Research and 

6 Development; report no. EPA/600/R-05/004aB-cB. Available at: 

7 http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=149923 . 

8 U.S. Environmental Protection Agency (2007a). Review of the National Ambient Air Quality 

9 Standards for Ozone: Policy Assessment of Scientific and Technical Information. OAQPS 

10 Staff Paper. Research Triangle Park, NC: Office of Air Quality Planning and Standards; 

11 report no. EPA-452/R-07-007a. Available at: 

12 http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_cr_sp.html . 

13 U.S. Environmental Protection Agency (2007b). Ozone Population Exposure Analysis for 

14 Selected Urban Areas. Office of Air Quality Planning and Standards, U.S. Environmental 

15 Protection Agency, Research Triangle Park, NC. Available at: 

16 http://epa.gov/ttn/naaqs/standards/ozone/data/2007_07_o3_exposure_tsd.pdf . 

17 U.S. Environmental Protection Agency (2008a). National Air Quality: Status and Trends 

18 Through 2007. Office of Air Quality Planning and Standards, U.S. Environmental 

19 Protection Agency, Research Triangle Park, NC. EPA-454/R-08-006. November 2008. 

20 Available at: http://www.epa.gov/airtrends/2008/index.html. 

21 U.S. Environmental Protection Agency (2008b). EPA’s Report on the Environment. U.S. 

22 Environmental Protection Agency Washington, DC EPA/600/R-07/045F. May 2008. 

23 Available at: http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=190806 

24 U.S. Environmental Protection Agency (2008c). Total Risk Integrated Methodology (TRIM) - 

25 Air Pollutants Exposure Model Documentation (TRIM.Expo / APEX, Version 4) Volume I: 

26 User’s Guide. Office of Air Quality Planning and Standards, U.S. Environmental Protection 

27 Agency, Research Triangle Park, NC. EPA-452/B-08-001a. Available at: 

28 http://www.epa.gov/ttn/fera/human_apex.html. 

29 U.S. Environmental Protection Agency (2008d). Total Risk Integrated Methodology (TRIM) -

30 Air Pollutants Exposure Model Documentation (TRIM.Expo / APEX, Version 4) Volume II: 

31 Technical Support Document. Office of Air Quality Planning and Standards, U.S. 

32 Environmental Protection Agency, Research Triangle Park, NC. EPA-452/B-08-001b. 

33 Available at: http://www.epa.gov/ttn/fera/human_apex.html. 

34 U.S. Environmental Protection Agency (2010). Quantitative Health Risk Assessment for 

35 Particulate Matter. Office of Air Quality Planning and Standards, Research Triangle Park, 

36 NC. EPA-452/R-10-005. Available at: 

37 http://www.epa.gov/ttn/naaqs/standards/pm/data/PM_RA_FINAL_June_2010.pdf 

38 U.S. Environmental Protection Agency (2011a). Integrated Review Plan for the Ozone National 

39 Ambient Air Quality Standards Review. Office of Air Quality Planning and Standards, 

40 Research Triangle Park, NC. Available at 

41 http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_2008_pd.html 
8-7

1 U.S. Environmental Protection Agency (2011b). Integrated Science Assessment for Ozone and 

2 Other Related Photochemical Oxidants. First External Review Draft. National Center for 

3 Environmental Assessment, Research Triangle Park, NC. Available at: 

4 http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=217463. 

5 U.S. Environmental Protection Agency. (2001). Risk assessment guidance for Superfund. Vol. 

6 III, Part A. Process for conducting probabilistic risk assessment (RAGS 3A). Washington, 

7 DC, United States Environmental Protection Agency, EPA 540-R-02-002; OSWER 9285.7-

8 45; PB2002 963302; http://www.epa.gov/oswer/riskassessment/rags3adt/index.htm. 

9 U.S. Environmental Protection Agency. (2004) EPA’s risk assessment process for air toxics: 

10 History and overview. In: Air toxics risk assessment reference library. Vol. 1. Technical 

11 resource manual. Washington, DC, United States Environmental Protection Agency, pp. 3-1 

12 – 3-30, EPA-453-K-04-001A; http://www.epa.gov/ttn/fera/data/risk/vol_1/chapter_03.pdf. 

13 Wen X., Balluz L., and Mokdad A. (2009). ―Association between media alerts of Air Quality 

14 Index and change of outdoor activity among adult asthma in six states, BRFSS, 2005.‖ J. 

15 Comm. Health 34: 40-46. 

16 Wiley, J. A., Robinson, J. P., Cheng, Y.-T., Piazza, T., Stork, L., Pladsen, K. (1991b). Study of 

17 Children's Activity Patterns: Final Report. California Air Resources Board, Sacramento, CA. 

18 ARB-R-93/489. 

19 Wiley, J. A., Robinson, J. P., Piazza, T., Garrett, K., Cirksena, K., Cheng, Y.-T., Martin, G. 

20 (1991a). Activity Patterns of California Residents: Final Report. California Air Resources 

21 Board, Sacramento, CA. ARB/R93/487. Available from: NTIS, Springfield, VA., PB94-

22 108719. 

23 Williams, R., Suggs, J., Rea, A., et al. (2003a). The Research Triangle Park particulate matter 

24 panel study: ozone mass concentration relationships. Atmos Environ. 37: 5349-5363. 

25 Williams, R., Suggs, J., Rea, A., et al. (2003b). The Research Triangle Park particulate panel 

26 study: modeling ambient source contribution to personal and residential ozone mass 

27 concentrations. Atmos Environ 37: 5365-5378. 

28 Williams, R., Suggs, J., Creason, J., Rodes, C., Lawless, P., Kwok, R., Zweidinger, R., Sheldon, 

29 L. (2000). The 1998 Baltimore particulate matter epidemiology-exposure study: Part 2. 

30 Personal exposure associated with an elderly population. J Expo Anal Environ Epidemiol. 

31 10(6): 533-543. 

32 World Health Organization (2008). Harmonization Project Document No. 6. Part 1: Guidance 

33 Document on Characterizing and Communicating Uncertainty in Exposure Assessment. 

34 Available at: http://www.who.int/ipcs/methods/harmonization/areas/exposure/en/.

35 Wu, F. and Tsang, Y. (2004). Second-order Monte Carlo uncertainty/variability analysis using 

36 correlated model parameters. Ecological Modeling 177, 393-414. 

37
United States

Environmental Protection

Agency

Office of Air Quality Planning and Standards

Health and Environmental Impacts Division

Research Triangle Park, NC

Publication No. EPA-452/P-11-001 

April 2011