1

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY 

 WASHINGTON D.C. 20460 

 

OFFICE OF THE ADMINISTRATOR 

SCIENCE ADVISORY BOARD 

October 28, 2008 

EPA-CASAC-09-001 

The Honorable Stephen L. Johnson 

Administrator 

U.S. Environmental Protection Agency 

1200 Pennsylvania Avenue, N.W. 

Washington, D.C. 20460 

Subject: Clean Air Scientific Advisory Committee’s (CASAC) Peer Review of 

Draft Chapter 8 of EPA’s Risk and Exposure Assessment to Support the 

Review of the NO2 Primary National Ambient Air Quality Standard

Dear Administrator Johnson: 

 The Clean Air Scientific Advisory Committee (CASAC), augmented by subjectmatter-experts to form the CASAC Oxides of Nitrogen Primary National Ambient Air 

Quality Standards (NAAQS) Review Panel (hereafter referred to as the panel, roster 

provided in Enclosure A) held a public teleconference on October 22, 2008 to review 

draft chapter 8 of EPA’s Risk and Exposure Assessment (REA) to Support the Review of 

the NO2 Primary National Ambient Air Quality Standard. Chapter 8 is entitled "Exposure 

Assessment and Health Risk Characterization." Additionally, CASAC offers comments 

on the schedule for NAAQS revision, prompted by the Staff’s presentation of the 

proposed outline for Chapter 10 of the REA. 

 Chapter 8 is critical to the REA because it provides a characterization of health 

risk from NO2 exposure based on the estimated number of exceedances obtained from a 

comprehensive personal exposure model for asthmatics in Atlanta. The CASAC panel 

was generally impressed with the careful work done over a short time with available tools 

and resources, but it has concerns about the implications of certain model inputs and 

assumptions and about the general presentation of the approach and findings. The panel 

asks for changes in two main areas: in the interpretive presentation and in 

characterization of potential biases. The CASAC plans to review the completed REA at 

its December 5, 2008 public teleconference and will provide recommendations for EPA's 

consideration in developing the Advance Notice of Proposed Rulemaking (ANPR) for 

NO2 during and immediately after that teleconference. 
2

Response to EPA Charge Questions

 EPA requested CASAC’s responses to four charge questions related to the 

exposure assessment and health risk characterization of the draft chapter 8 provided for 

review. The charge questions and responses follow immediately below. 

1. To what extent is the assessment, interpretation, and presentation of the results of 

the exposure analysis technically sound, clearly communicated, and appropriately 

characterized?

 Staff has produced a solid application of the APEX model. While recognizing 

that there have been significant improvements since the first draft of the REA and the 

update at the September CASAC meeting, the panel still has concerns about the 

application of AERMOD. Staff should give serious consideration to improving 

AERMOD model performance in comparison to the concentrations measured by the 

monitors in Atlanta. For example, AERMOD output could be adjusted to better reflect 

observed peak values and diurnal profiles. Many factors influence the AERMOD 

predictions and the panel is concerned that the overall uncertainty, reflective of the 

uncertainties from each of these factors, is not appropriately characterized. In particular, 

the tails of the exposure distribution could be strongly biased, notably with respect to onroad and near-road exposures. These exposures are particularly important, since they are 

the major contributors to the benchmark exceedances. On-road exposures may be biased 

high because receptors are in the middle of the road; the exhaust is not diluted much; and 

observed data suggest that the on-road vs. background ratios are much more narrowly 

distributed than the AERMOD predictions. Near-road exposures may similarly be 

affected by biases in the on-road estimates. In addition, near-road exposures may not be 

appropriately weighted to the Atlanta population because a greater proportion of residents 

may live closer to major roads than is represented by the assigned locations of model 

receptors at the census block centroids used in the AERMOD model. 

In evaluating population exposure under alternative standards, Staff elected to roll 

back health benchmark values as a time-saving substitute for rolling up ambient 

concentrations. In view of the potentially strong influence of this decision on the 

numerical determination of benchmark exceedances, the chapter needs to provide a 

clearer justification for this approach and consideration needs to be given to validating 

the procedure. 

2. The draft risk and exposure assessment document evaluates exposures in Atlanta. 

What are the views of the Panel on the approach taken and on the interpretation 

of the results of this analysis?

 The chapter needs to better evaluate the generalizability of findings for Atlanta. 

In particular, three aspects of the selection of Atlanta should be addressed more carefully. 

First, the selection of Atlanta should be reviewed with consideration of the extent to 

which the city represents exposures nationally, since the model results are being used as 

the basis for setting a national ambient air quality standard. What are typical features of 

the population behaviors, building use (e.g., air conditioning), housing stock, and 

residential patterns that suggest similarities and differences with other major cities, 
3

particularly with respect to anticipated high-end exposures to NO2? As a minimum, the 

panel suggests that a table be added to describe variation of key parameters across cities. 

Ideally, a sensitivity analysis that assessed the importance of such parameters would also 

be included. Second, is the model applied to Atlanta adequately reflecting the local 

population? In particular, in assigning model receptors to census tract centroids, is the 

model systematically missing the fraction of the population who lives closest to roads and 

is consequently regularly exposed to high near-road exposures? The fraction of the 

Atlanta population that lives within 50 meters of a major roadway could be much higher 

than the 1% figure represented by the census tract centroids. We encourage EPA to seek 

an independent estimate of this important quantity. Third, the application of AERMOD 

to Atlanta suggests a significant potential to predict concentrations that are biased 

upwards at the high end of the distribution. This bias should be more thoroughly 

discussed, along with likely implications. 

3. What are the views of the Panel regarding the adequacy of the assessment of 

uncertainty and variability with respect to characterization of exposures and 

health risks associated with those exposures? 

 A complete analysis of uncertainty is an essential component of a thorough 

exposure and risk assessment. The panel is concerned that there is the potential for more 

bias and uncertainty in the results than has been characterized in the current uncertainty 

and variability assessment and that no rationale has been provided for assigning weight or 

priority to identified uncertainties. A better characterization of uncertainty is needed, 

with attention to prioritization of the multiple sources of uncertainty, and where possible, 

quantification of their impact. Qualitative approaches that might be followed for this 

purpose can be found in other reports1

 

In addition, previous suggestions made by individual panel members on this topic 

have not been explicitly considered; the concerns raised should be addressed in the final 

report. More thorough characterization of the assumptions and model features that most 

strongly influence the estimated number of exceedances is needed. Most notably, the onroad and near-road exposures are important determinants of the number of exceedances 

of specified concentration levels experienced by the population. In addition to on- and 

near-road characterization, and locations of receptors, assumptions about penetration into 

various microenvironments, particularly transportation environments, should be 

discussed. The implications of the significant biases discussed in the application of 

AERMOD need to be covered in this section. Value-laden statements in Section 8.4.8 

about the acceptability of AERMOD over-predictions should be relocated to Section 8.10 

and discussed in the context of variability and uncertainty. 

4. To what extent is the assessment, interpretation, and presentation of health risk 

characterization included in Chapter 8 technically sound, clearly communicated, 

and appropriately characterized? 

 

1

 See, for example, U.S. EPA. Limitations in the 1996 National-Scale Air Toxics Assessment 1996 [cited 

10/23/08. Available from http://www.epa.gov/ttn/atw/nata/natsalim2.html] and World Health Organization 

International Programme on Chemical Safety. Part 1: guidance document on characterizing and 

Communicating uncertainty in exposure assessment. World Health Organization In Press. [cited 10/23/08. 

Available from http://www.who.int/ipcs/methods/harmonization/areas/uncertainty%20.pdf]. 
4

 The overall presentation of the chapter needs to be improved in order to help 

readers to understand the broad framework of the modeling and to give a context for the 

health risk characterization. In addition to expanding the overview section, several 

figures and tables should be added to show the relationships among the various models 

and to summarize the data inputs. Furthermore, this chapter needs to note that the health 

risk characterization focuses exclusively on exposure benchmark comparisons through 

additional discussion and/or cross-referencing to other sections of the REA. EPA should 

add a concluding section that summarizes the primary results and implications from the 

large number of models and analyses presented in the chapter. 

Comments on the Schedule for NAAQS Revision

Staff provided an overview of plans for Chapter 10 of the REA, which will be 

critical for considering options for the NAAQS for NO2. The timeframe of the 

teleconference permitted only a brief exchange around the general approach that will be 

taken. The panel was told that time might not permit the EPA to respond to the CASAC 

recommendations concerning Chapter 10 and the implications of that summary chapter 

for developing the ANPR, following the CASAC public teleconference planned for 

December 5, 2008. The REA and the ANPR are critical parts of the new review process 

and the panel would find it unacceptable to be unable to provide advice on these 

documents in an effective manner. Time for these essential CASAC review steps should 

be included in any revisions to the timetable for completion of the NOx review. 

This is only one example of the implications of the rapid schedule for completion 

of the NAAQS revision process. EPA is producing and revising analyses and documents 

at a pace that is compromising CASAC’s review function and consequently the quality of 

the scientific foundation for revising the NAAQS. CASAC recognizes the drivers of the 

schedule for NAAQS revision but cautions that there are now warnings that CASAC's 

review role, mandated by the Clean Air Act, is likely to be diminished by the compressed 

schedule. 

In closing, the panel noted the substantial progress in the development of this 

chapter of the Risk and Exposure Assessment. We look forward to reviewing the final 

version of the entire document later this year. 

 

Sincerely, 

 /Signed/ 

Dr. Jonathan M. Samet, Chair 

Clean Air Scientific Advisory Committee 

cc: Marcus Peacock, Deputy Administrator 
5

NOTICE 

This report has been written as part of the activities of the EPA's Clean Air 

Scientific Advisory Committee (CASAC), a Federal advisory committee 

independently chartered to provide extramural scientific information and advice to the 

Administrator and other officials of the EPA. The CASAC provides balanced, expert 

assessment of scientific matters related to issues and problems facing the Agency. 

This report has not been reviewed for approval by the Agency and, hence, the 

contents of this report do not necessarily represent the views and policies of the EPA, 

nor of other agencies within the Executive Branch of the Federal government. In 

addition, any mention of trade names or commercial products does not constitute a 

recommendation for use. CASAC reports are posted on the EPA Web site at: 

http://www.epa.gov/casac. 
6

Enclosure A 

U.S. Environmental Protection Agency 

Clean Air Scientific Advisory Committee 

Oxides of Nitrogen Primary NAAQS Review Panel 

CHAIR 

Dr. Jonathan M. Samet, Professor and Chair of the Department of Epidemiology, Bloomberg 

School of Public Health, Johns Hopkins University, Baltimore, MD 

CASAC MEMBERS 

Dr. Joseph Brain, Philip Drinker Professor of Environmental Physiology, Department of 

Environmental Health, Harvard School of Public Health, Harvard University, Boston, MA 

Dr. Ellis B. Cowling, University Distinguished Professor At-Large, Emeritus, Colleges of 

Natural Resources and Agriculture and Life Sciences, North Carolina State University, Raleigh, 

NC 

Dr. James Crapo, Professor of Medicine, Department of Medicine, National Jewish Medical 

and Research Center, Denver, CO 

Dr. H. Christopher Frey, Professor, Department of Civil, Construction and Environmental 

Engineering, College of Engineering, North Carolina State University, Raleigh, NC, USA 

Dr. Donna Kenski, Data Analyst, Lake Michigan Air Directors Consortium, Des Plaines, IL 

Dr. Armistead (Ted) Russell, Professor, Department of Civil and Environmental Engineering, 

Georgia Institute of Technology, Atlanta, GA 

CONSULTANTS 

Professor Ed Avol, Professor, Preventive Medicine, Keck School of Medicine, University of 

Southern California, Los Angeles, CA 

Dr. John R. Balmes, Professor, Department of Medicine, Division of Occupational and 

Environmental Medicine, University of California, San Francisco, CA 

Dr. Douglas Crawford-Brown, Emeritus Professor and Director, Department of Environmental 

Sciences and Engineering, University of North Carolina at Chapel Hill, Chapel Hill, NC 

Dr. Terry Gordon, Professor, Environmental Medicine, NYU School of Medicine, Tuxedo, NY 

Dr. Dale Hattis, Research Professor, Center for Technology, Environment, and Development, 

George Perkins Marsh Institute, Clark University, Worcester, MA 
7

Dr. Rogene Henderson, Scientist Emeritus, Lovelace Respiratory Research Institute, 

Albuquerque, NM 

Dr. Patrick Kinney, Associate Professor, Department of Environmental Health Sciences, 

Mailman School of Public Health , Columbia University, New York, NY 

Dr. Steven Kleeberger, Professor, Lab Chief, Laboratory of Respiratory Biology, National 

Institute of Environmental Health Sciences, National Institutes of Health, Research Triangle 

Park, NC 

Dr. Timothy V. Larson, Professor, Department of Civil and Environmental Engineering, 

University of Washington, Seattle, WA, USA 

Dr. Kent Pinkerton, Professor, Regents of the University of California, Center for Health and 

the Environment, University of California, Davis, CA 

Dr. Edward Postlethwait, Professor and Chair, Department of Environmental Health Sciences, 

School of Public Health, University of Alabama at Birmingham, Birmingham, AL 

Dr. Richard Schlesinger, Associate Dean, Department of Biology, Dyson College, Pace 

University, New York, NY 

Dr. Christian Seigneur, Director, Atmospheric Environment Center, Université Paris-Est, 

Champs-sur-Marne, France 

Dr. Elizabeth A. (Lianne) Sheppard, Research Professor, Biostatistics and Environmental & 

Occupational Health Sciences, Public Health and Community Medicine, University of 

Washington, Seattle, WA 

Dr. Frank Speizer, Edward Kass Professor of Medicine, Channing Laboratory, Harvard 

Medical School, Boston, MA 

Dr. George Thurston, Professor, Environmental Medicine, NYU School of Medicine, New 

York University, Tuxedo, NY 

Dr. James Ultman, Professor, Chemical Engineering, Bioengineering Program, Pennsylvania 

State University, University Park, PA 

Dr. Ronald Wyzga, Technical Executive, Air Quality Health and Risk, Electric Power Research 

Institute, Palo Alto, CA 

SCIENCE ADVISORY BOARD STAFF 

Dr. Angela Nugent, Designated Federal Officer 
8

Enclosure B: Compilation of Individual Panel Member Comments on Chapter 8 of 

EPA’s Risk and Exposure Assessment (REA) to Support the Review of the NO2

Primary National Ambient Air Quality Standard 

This enclosure contains final written comments of individual members of the Clean Air 

Scientific Advisory Committee (CASAC) Oxides of Nitrogen Primary National Ambient 

Air Quality Standards (NAAQS) Review Panel. The comments are included here to 

provide both a full perspective and a range of individual views expressed by panel 

members during the review process. These comments do not represent the views of the 

CASAC or the CASAC Panel. 

Comments Received: 

Comments from Prof. Ed Avol ........................................................................................... 9 

Comments from Dr. John Balmes..................................................................................... 11 

Comments from Dr. Joseph Brain .................................................................................... 13 

Comments from Dr. James Crapo..................................................................................... 14 

Comments from Dr. Douglas Crawford-Brown ............................................................... 15 

Comments from Dr.H. Christopher Frey .......................................................................... 19 

Comments from Dr. Terry Gordon ................................................................................... 20 

Comments from Dr. Dale Hattis ....................................................................................... 22 

Comments from Dr. Rogene Henderson........................................................................... 26 

Comments from Dr. Timothy Larson ............................................................................... 27 

Comments from Dr. Kent Pinkerton................................................................................. 29 

Comments from Dr. Armistead Russell............................................................................ 31 

Comments from Dr. Jonathan Samet................................................................................ 33 

Comments from Dr. Richard Schlesinger......................................................................... 34 

Comments from Dr. Christian Seigneur ........................................................................... 35 

Comments from Dr. Elizabeth “Lianne” Sheppard .......................................................... 36 

Comments from Dr. Frank Speizer................................................................................... 38 

Comments from Dr. George Thurston.............................................................................. 40 

Comments from Dr. James Ultman .................................................................................. 42 
9

Comments from Prof. Ed Avol

Comments on Second Draft NOx REA, Chapter 8 

Ed Avol 

General Comments:

My expertise is not in modeling, so I am not familiar with AERMOD and will have 

to defer to others regarding the appropriate application of it here. I did appreciate 

that there was a considerable effort undertaken to lay out the modeling approach, 

and Staff should be commended for those efforts. Although a great deal of detail 

was provided in the chapter, I did not feel it was properly synthesized or 

summarized for the reader. A final few paragraphs asserting what it all means 

would have been useful. There was considerable discussion about uncertainty, 

but it, too, seemed almost philosophical in nature, rather than specific – can 

anything focused to the effort at hand be said regarding the magnitude or degree 

of uncertainty related to this modeling exercise? 

Specific Comments:

Table of Contents, Section 8.8 title, (typographical error), check font type size 

P4, Section 8.3.1, line 21 – the statement that “...Atlanta was selected as the 

second-case study location...” is made, with no reference, comment, or 

explanation about the first study case (Philadelphia), the reasons for going to a 

second, the generalizability of either of these selections, or the selection 

process...something needs to be said as to how these choices were made, even 

if it is to a discussion deferred to the appendix (although the appropriateness of 

this selection would seem to be an issue to be addressed in the main body of the 

document). 

P5, Section 8.3.3, lines 5-7 – the assertion is made that 3 years of met data 

(2001-2003) are used to achieve a degree of stability in dispersion and exposure 

model estimates. Given recent fluctuations in meteorological patterns, how 

“stable” or “usual” are the 2001-2003 weather patterns for the area? Are they, for 

example, unusual drought years, wet years, hot years, or cold years? Is it 

reasonable (and credible) to use them to generalize for the air quality modeling? 

P6, Section 8.3.4, lines 7 and 8 – specify units (years) 

P6, Section 8.4.1, line 23 – change “are” to “were” 

P6, Section 8.4.1, line 27 – typo at end of word “included” 

P7, Section 8.4.2.1, line21 – insert “airport” in Atlanta Hartsfield name. 

P8, Section 8.4.2.4, line 16 – insert “was” after “10%” 
10

P11, Table 8.2 – Comment – the HDV fractions seem high (10%-25%) for a 

metropolitan area with commuters (most of whom will use LDV for 

transportation)...? 

P19, Section 8.4.7, line 14-15 – how does assignment of receptors to a height of 

0 meters make the model more efficient (let alone realistic)? 

P21, Section 8.4.8.1, lines 15-17 – does that observation that there were only 

two predicted exceedances of 150ppb NO2 at one monitor and no estimated 

NO2 concentrations above 200ppb for 1hour say something about the modeling 

results, or the location of the samplers? 

P29, Section 8.5, line 4 – 64,000 asthmatics/500,000 children is about 13%; is 

this a high value or in conformance with other estimates for the region? 

P33, Section 8.7.2.1, lines 5-18 – this discussion exclusively focuses on gas 

cooking as the only indoor source of NO2 interest. What about un-vented or 

poorly-vented room or wall heaters or fireplaces? 

P37, line 8 – change ”an” to “and” 

P38, Section 8.9.1, lines 22-23 – This statement is not necessarily true, so it 

would be better to just say the exposure results focused on asthmatics. 

P39, Section 8.9.2, line 13 – change “individual” to “individuals” 

P39, Section 8.9.2, lines 18-21 – this is poorly phrased and confusing, since 

these are two different thoughts. Suggest breaking in to two sentences: “...with 

most containing less than 30ppb.” “About 5% of these receptors...” 

P53, Figure 8-17 – Why are there three Figure 8-17a illustrations? How can the 

first Figure 8-17a (>100ppb) be so different from the rest? 
11

Comments from Dr. John Balmes

Characterization of Exposure and Health Risks Associated with Exposure (Chapter 8 and 

Appendix B): John Balmes’ comments 

GENERAL COMMENTS 

The agency staff members responsible for the work presented in Chapter and Appendix B 

are to be commended for their hard work over a short timeframe. 

1. To what extent is the assessment, interpretation, and presentation of the results of 

the exposure analysis technically sound, clearly communicated, and appropriately 

characterized? 

I do not have modeling expertise, but I found the logic and results of the modeling efforts 

conducted to be presented clearly. The discussion of assumptions and uncertainties was 

also clear. A general concern that I have is that the overall modeling exercise is complex 

and dependent on many sources of data with varying quality and uncertainties. Multiple 

assumptions are required. Is it necessary to support the air quality standard setting with 

such a complex model. Put another way, could a simpler model with less assumptions 

and less data inputs be built that provides reasonable bounds to the likely population 

exposures to NO2 above the benchmark values, but that is also easier for non-modelers to 

understand? A specific concern that I have is that APEX apparently uses only one 

temperature value per day (p. 63, sub-section 8.10.3). Given the overall complexity of 

the exposure model, one would think that temperature could be better parameterized. 

2. The draft risk and exposure assessment document evaluates exposures in Atlanta. 

What are the views of the Panel on the approach taken and on the interpretation of 

the results of this analysis? 

Atlanta is a large metropolitan area that has both relatively good ambient air quality and 

health effects data. Given limitations of time and staff resources, I think the decision to 

focus Atlanta is reasonable. My major concern about the decision is that because of its 

climate, the penetration of air conditioning is high (apparently almost 100%). This 

makes modeled NO2 exposures in Atlanta less representative of such exposures in areas 

with more temperate climates. 

3. What are the views of the Panel regarding the adequacy of the assessment of 

uncertainty and variability with respect to characterization of exposures and 

health risks associated with those exposures? 

The variability and uncertainty section of the chapter (section 8.10) reads well, but there 

is no overall assessment of the impact of the sources of uncertainty on the exposure 

estimates presented in section 8.9. 

4. To what extent is the assessment, interpretation, and presentation of health risk 

characterization included in Chapter 8 technically sound, clearly communicated, 
12

and appropriately characterized? 

There is no presentation of health risk characterization. In my view, presentation of the 

number of potential exposures to NO2 of the total population or persons with asthma 

does not constitute sufficient health risk characterization. The impact of the estimated 

exposures on the health of the total and asthmatic populations should be discussed in the 

chapter. 

SPECIFIC COMMENTS 

p. 17, line 5 For what the abbreviation, SCC, stands should be spelled out the first time 

it is used.. 

p. 33, line 11 American Housing Survey should be spelled out with its first use. 

p. 33, lines 17-18 Should be “...CHAD diaries, stratified by hour of the day and 

normalized to the expected value of daily food preparation events, 1.4 (Table 8-11). 

p. 39, line 29 Persons do not “contain” exposures to ambient exposures. 

p. 41, lines 6-8 Ibid. 

p. 42, line 12 Should be “...cooking fuel as an influential variable...” 

p. 42, line 13 Should be “...of whom 13 had personal...” 

p. 42, line 17 Should be “...an exposure distribution was constructed for each 

individual,...” 

p. 43, line 3 Should be “1999-2000”. 

p. 51, line 7 Should be “...the times spent...” 

p. 51, line 12 Should be “...regarding the effect of indoor sources...” 

p. 51, line 13 “...microenvironments, however, changes...” 

p. 51, line 16 Delete “a” between “to” and “comprising”. 

p. 56, lines 27-28 “Results for asthmatics when exposures were estimated...” 

p. 66, line 15, line 26 For what the abbreviations, CMSA and CSA stand should be 

spelled out the first time they are used. 
13

Comments from Dr. Joseph Brain

 Chapter 8 and other parts of the REA document rely heavily on analyses of 

Atlanta data. An essential question which deserves much more attention is the extent to 

which these findings can be generalized. In terms of NOx levels and other aspects of 

exposure assessment, we need to know how Atlanta compares to other population centers 

throughout the United States, such as Los Angeles, Houston, or Chicago. What is the 

range of NOx exposure in these other cities? Other critical parameters should also be 

compared? What is the prevalence of air conditioning? What do we know about human 

activity in different cities, e.g. time spent indoors vs. outdoors, and what about relevant 

confounders such as temperature and humidity? 

 

 I am sensitive to the limited time available to address their issues. There may not 

be time to carry out what should be done – a sensitivity analysis. To what extent do the 

predictions and models change when we substitute data from other cities? One could, 

however, even with the existing Atlanta model, change various parameters, and see what 

happens to the outcomes. Another approach would be to create a table which lists these 

parameters for selected US cities. That would reinforce the point that safety factors need 

to be built in because of uncertainty about the extent to which Atlanta is typical. 
14

Comments from Dr. James Crapo

Comments on Chapter 8, REA NO2 Document, dated October 2008 

In general the EPA staff have done an excellent job in compiling a very 

complex model to predict the number of individuals in Atlanta who would 

be exposed to NO2 levels above various potential Health Effect Benchmark 

Levels. The models are appropriate and the descriptions of how the data 

was used in the models, including the assumptions involved, are clearly 

described. This will provide a reasonable basis on which to estimate 

the population impact of conformance to various NOx NAAQS. My concerns 

are: 

1. The model is extremely complex with a large number of assumptions at 

almost every aspect of the model. In most cases it is not known how 

these assumptions may aggregate to create a significant positive or 

negative bias in the results. Would a much simpler model provide 

estimates of populations at risk that would be of similar - or better - 

final accuracy? This complex model may create a false sense of 

accuracy. My point is that this model needs to be validated with 

experimental data. The model should be used to predict testable 

outcomes such as ER admissions for asthma, etc., which can then be 

compared to actual observations. This could be done in a prospective 

fashion or by applying the model to prior time periods not used to 

create the model, but where outcome data is already known. 

2. The overview for the chapter states that the intent is to document 

the methodology and data used in the inhalation exposure assessment and 

in the associated health risk characterization for NO2. The chapter 

only models the temporally and spatially variable NO2 concentrations and 

simulates the predicted human contact with those modeled NO2

concentrations. The chapter needs to go on to use these simulations to 

predict health risk in terms of testable outcomes - i.e.: mortality, 

hospital admissions, ER admissions for asthma, exacerbation of 

respiratory symptoms, etc. This is the ultimate purpose in creating 

these models and will provide a means to eventually validate the utility 

of the models. 
15

Comments from Dr. Douglas Crawford-Brown

Comments on Risk and Exposure Assessment to Support the Review of the NO2

Primary National Ambient Air Quality Standard: Chapter 8 and Appendix 

Doug Crawford-Brown 

April 22, 2008-04-27 

These comments focus on Chapter 8 and the appendix of the Risk and Exposure Draft. 

My review is organized around the four charge questions. 

1. To what extent is the assessment, interpretation, and presentation of the results of 

the exposure analysis technically sound, clearly communicated, and appropriately 

characterized? 

I was in general quite pleased with by the analysis and the way it was described both in 

the text and in the appendix (which was, by the way, well written). There are a number of 

important issues from the previous review that have now been addressed adequately, and 

there has been an impressive amount of work done in short order to provide these 

improvements. I found it easy to understand the analytic steps, and the justification for 

formulating them as the staff did. 

The characterization of AERMOD could use a bit more work. My sense is that it biases 

results upwards, which would then bias upwards the predicted number of effects. This 

aspect of the model is not adequately described, nor is it reflected in the uncertainty 

section. I also was never completely clear as to how the percentiles of ambient 

concentrations were being used in the Atlanta and Philadelphia examples, or whether 

there even was consistency between the two case studies in regards to percentiles used. 

There needs to be a concise paragraph or two explaining more clearly how the upper tails 

of distributions were used and why these particular percentiles were selected. Otherwise, 

it strikes me – if I were a first time reader – that there is a lot of conservatism built into 
16

the analysis, especially using parts of a distribution (the upper tails) where I am less 

confident in the results.. 

The discussion of receptor locations in Section 8.4.7 could use improvement. I was never 

clear in reading that section how the receptor locations were related to the APEX 

modelling, which can follow people through an actual exposure field rather than through 

representative locations of exposure. As I read the section, it sounded to me as if a few 

receptor locations were selected as representative of ambient concentrations associated 

with a given activity, and I was not sure how that fit into the APEX approach based on 

census tracts. I can imagine how it is done, but the section didn’t give me enough 

description to understand the issue fully. 

I repeat here a point I raised in previous drafts. I generally approve the proportional rollup or roll-down methods based on current maximum concentration at a specific site. And 

I agree that the adjustment of the benchmarks produces the same result mathematically. 

But it continues to make no sense scientifically, and. the savings in processing time don’t 

appear to me sufficient to justify a method that people will fail to understand as 

mathematically equivalent. 

 

2. The draft risk and exposure assessment document evaluates exposures in Atlanta. 

What are the views of the Panel on the approach taken and on the interpretation of 

the results of this analysis? 

I agree with the approach taken, subject to the comments in Charge Question 1. It seemed 

to me, however, that the analyses for Atlanta and Philadelphia were not quite the same, 

making comparisons difficult. I particularly liked the generation of results on person-days 

of effects in Atlanta, and wish the same had been done for Philadelphia. 

I think the interpretation of results for Atlanta is appropriate, given the assumptions 

made. However, as mentioned earlier, AERMOD appears to bias results high, or at least 

above the monitoring results. This should factor into the interpretation, or at least into the 

uncertainty analysis, and I didn’t find that to be the case. 
17

I also didn’t know how the staff were interpreting the results that divide effects between 

categories of with or without indoor sources included. In some cases, the large majority 

of effects appear to be from indoor (and hence uncontrolled) sources, while the results as 

the ambient levels rise show the dominance of the ambient exposures. There should be a 

better description of how the staff believe these results should be interpreted. However, I 

do agree with the results themselves, again given the assumptions and models employed. 

The development of the longitudinal activity sequences is a sophisticated piece of work, 

and is described adequately. The one comment I have here is that at one point, the staff 

had to decide whether to use correlated daily activities or uncorrelated. They used a 

cluster approach, which I support. However, it was a bit difficult, reading the text alone, 

to understand the cluster approach selected or why the authors believe it produces results 

intermediate between the other two results. It might have been useful to present a 

representative set of results under each of the three approaches and see how much 

variation it produces (a kind of sensitivity analysis). 

 

3. What are the views of the Panel regarding the adequacy of the assessment of 

uncertainty and variability with respect to characterization of exposures and 

health risks associated with those exposures? 

The uncertainty analysis remains inadequate to produce any sort of confidence bounds on 

the analytic results, although it improves with each iteration we have seen. And it now 

gives the reader a sense of the contribution to variance from each component. I am not 

pushing for a formal nested variability-uncertainty analysis, but it does seem to me that 

something akin to what went into NATA might be appropriate, where at least the overall 

uncertainty was categorized. And there remains the issue of the influence of the 

conservatism of AERMOD on whether final results should be seen as simply uncertain, 

or biased upwards. 
18

On a minor point, there is an odd part of Page 67 where the authors talk of a negative 

value for the GSD and for the GM. I must be reading it incorrectly (since these negative 

values are not possible), but I cannot understand these sentences. 

4. To what extent is the assessment, interpretation, and presentation of health risk 

characterization included in Chapter 8 technically sound, clearly communicated, 

and appropriately characterized? 

My comments are the same as in Charge Question 1, although I note here that the 

presentation of health effects results was quite effective in this document. The analysis of 

repeat effects is interesting scientifically, but the policy implications are not clear. I 

would suspect that the person-days above the benchmark is more likely to inform any 

final decision. Some clarification on the significance of this analysis (of multiple effects 

for an individual) would be useful 

I am not convinced by the argument that the daily activity patterns for asthmatics and 

non-asthmatics are the same, especially on high ozone days where an alert has been 

issued. There is a passing comment about a study suggesting otherwise, but I don’t 

believe that study examined the effect of air quality indices being published. I know RTI 

were doing such a study at one time (Carol Mansfield was heading it), so perhaps the 

staff might find the results of that study and see if the conclusions are changed. 

I end with a comment I have made in almost all similar cases reviewed by CASAC. The 

modelling performed here is impressive and represents state-of-the-science. But I worry 

that it may be too elaborate for the purposes of establishing a NAAQS. There are many, 

many assumptions built into the assessment, from air modelling to activity patterns. And 

all of these rest on spatial resolution of exposure fields and receptor locations. The 

science is great, but I feel the current analysis is pushing the resolution of the results just 

a bit. Some comments in the document mentioning this issue, and why the results still are 

useful for a NAAQS determination, would be good. 
19

Comments from Dr.H. Christopher Frey

The following guidance document should be referenced and cited in Chapter 8: 

"GUIDANCE DOCUMENT ON CHARACTERIZING AND COMMUNICATING 

UNCERTAINTY IN 

EXPOSURE ASSESSMENT" 

http://www.who.int/ipcs/methods/harmonization/areas/exposure/en/index.html 

Also, , the assumption of 10% NO2 versus total NO for source strength may be 

conservative for existing mobile and stationary sources. However, for diesel vehicles 

equipped with particle traps and other postcombustion controls, there is evidence that the 

NO2/NOx ratio is higher than this. As such vehicles penetrate the onroad fleet, this may 

have some effect on the overall ration of NO2/NOx. 

I recommend that EPA consider that one of the useful outputs of this study will be a 

prioritized list of key uncertainties, as discussed today. Such a list could be developed 

from qualitative weight of evidence approaches, and communicated in broad categories 

(e.g., low, medium, high uncertainty) or could be developed quantitatively (preferred 

where possible). One benefit of this information is that it could help prioritize data 

collection or research to improve the state of knowledge for future reviews and revisions 

of the NO2 NAAQS. 

However, my concern is that we don't want to imply that what can be done in a short 

amount of time under a court-order is what SHOULD be done for the best science. The 

qualitative approach is a first step, but not (in my view) sufficient. Along these lines, 

EPA should acknowledge any unresolved issues, possible uncertainties that are not fully 

characterized, and better methodologies for dealing with these issues, in order to guide 

future work. 
20

Comments from Dr. Terry Gordon

1. To what extent is the assessment, interpretation, and presentation of the results of 

the exposure analysis technically sound, clearly communicated, and appropriately 

characterized? 

Although my expertise does not allow me to evaluate whether the exposure analysis is 

technically sound, the analysis was clearly communicated and characterized. 

Explanations as to why certain decisions were made for the chosen methods were 

particularly clear. One minor communication suggestion would be to expand the figure 

legends and labeling for the figures. 

2. The draft risk and exposure assessment document evaluates exposures in Atlanta. 

What are the views of the Panel on the approach taken and on the interpretation of 

the results of this analysis? 

The approach and interpretation were very good, but there still exists the potential 

problem or bias associated with using an analysis of one city in the Southeast (and 

Philidelphia) to encompass a risk analysis for the entire country. In particular, it appears 

that 12 urban centers were used to estimate population exposures for ozone (stated in 

Section 8.2, page 1, line 26). Perhaps this will be better addressed in the final REA 

chapter. 

3. What are the views of the Panel regarding the adequacy of the assessment of 

uncertainty and variability with respect to characterization of exposures and 

health risks associated with those exposures? 

The assessment of uncertainty and variability factors is very clear although a table 

delineating the direction of bias that each factor might introduce would be a good 

addition either in Chapter 8 or 9 (where I believe there may already be one). 

4. To what extent is the assessment, interpretation, and presentation of health risk 

characterization included in Chapter 8 technically sound, clearly communicated, 

and appropriately characterized? 

EPA did a great job under tight time constraints to finish this health risk characterization. 

It is technically sound and well communicated and characterized, although a conclusion 

section summarizing the relevance of the characterized exceedances/exposed asthmatics 

would solidify this chapter (rather than waiting until the final chapter). 

Minor comments: 

1. Table 8-1 – It would help the reader to define Fringe, Rural, Arterial, Freeway, 

etc. below the table. 

2. Table 8-5 – Again, please define FIPS, Major and Minor Links, and % Minor. 

3. page 15, line 9 – Define NEI here and in Table 8-6. 

4. page 26, line 8 – typo = lcocated. 
21

5. page 27, Table 8-7 – Define p0 to p100. Including ‘N’ as the first row is 

confusing as it’s not really On-Road Hourly NO2 (ppb) or Number of hours 

greater than x00 ppb. 

6. page 29, line 2 – Given the use of 4 central counties, is it correct to look at the 

total population of Atlanta? 

7. page 34, line 10 – typo = ‘rate and the cooking...’. 

8. page 68 – Should there be a discussion of the uncertainty in applying 1 city (or 2) 

to the rest of the U.S.? 
22

Comments from Dr. Dale Hattis

Comments on the Treatment of Uncertainty and Variability in the Revised Chapter 

8 Exposure Analysis 

Dale Hattis, Clark University 

Over the course of reviewing the APEX and air quality 

modeling for past meetings, I and others have offered 

quite a few constructive suggestions for improving the 

modeling, the distributional assumptions (on such 

parameters as indoor air elimination rates and on 

road/off-road adjustment factors) and other issues. 

These include the following comments on the first draft 

REA reviewed in the May 2008 meeting: 

“Air exchange distributions contingent on temperature and presence or absence of 

air conditioning. Overall the panel does not have any objection to the idea of 

using lognormal distributions with very broad limits (.1 and 10 air changes/hr). 

However the detailed results seem to show different patterns with temperature 

arbitrarily blocked into a few ranges. There does not appear to be any great 

consistency or overall theory for this analysis. A better description of the data as 

a whole might be produced by a more extensive regression study using 

temperature or some transform of temperature as a continuous variable and either 

fixed-effect or mixed effects modeling of differences among cities and for the air 

conditioner presence variable. 

NO2 removal rate distribution--p. 101. At least one panelist expressed an 

objection to the narrow fixed limits used for the removal rate distribution based 

on six values from Spicer et al (1993). The abstract to the Spicer paper makes it 

clear that all six observations were made in a single house, and that there are 

additional complications from the presence of HONO, an apparently longer-lived 

NOx species: 

p. 101—The same panelist also objected to the fixed limits used for the removal 

rate distribution based on six values from Spicer et al (1993). The abstract reads. 

Transformations, lifetimes, and sources of NO2, HONO, and HNO3 in 

indoor environments. 

Spicer CW, Kenny DV, Ward GF, Billick IH. 

Air Waste. 1993 Nov;43(11):1479-85. 

Battelle, Columbus, OH 43201-2693. 

Recent research has demonstrated that nitrogen oxides are transformed to 

nitrogen acids in indoor environments, and that significant concentrations 
23

of nitrous acid are present in indoor air. The purpose of the study reported 

in this paper has been to investigate the sources, chemical transformations 

and lifetimes of nitrogen oxides and nitrogen acids under the conditions 

existing in buildings. An unoccupied single family residence was 

instrumented for monitoring of NO, NO2, NOy, HONO, HNO3, CO, 

temperature, relative humidity, and air exchange rate. For some 

experiments, NO2 and HONO were injected into the house to determine 

their removal rates and lifetimes. Other experiments investigated the 

emissions and transformations of nitrogen species from unvented natural 

gas appliances. We determined that HONO is formed by both direct 

emissions from combustion processes and reaction of NO2 with surfaces 

present indoors. Equilibrium considerations influence the relative 

contributions of these two sources to the indoor burden of HONO. We 

determined that the lifetimes of trace nitrogen species varied in the order 

NO approximately HONO > NO2 > HNO3. The lifetimes with respect to 

reactive processes are on the order of hours for NO and HONO, about an 

hour for NO2, and 30 minutes or less for HNO3. The rapid removal of 

NO2 and long lifetime of HONO suggest that HONO may represent a 

significant fraction of the oxidized nitrogen burden in indoor air. 

The uniform distribution with its fixed boundaries (0% probability assumed for 

values outside of the defined limits) is particularly inappropriate when the data are 

limited, as in this case. Use of the uniform distribution artificially reduces the 

likelihood of more extreme values of the modeled parameter than happen to be 

present in the limited available data. This in turn limits the model-predicted 

variability of NO2 concentrations, which critically determines the number of 

exceedances of the high hourly NO2 levels that are the focus of the risk 

assessment modeling. It would likely be far better to use a lognormal here as an 

initial hypothesis, but in the light of the fact that different houses with different 

internal materials might well destroy NO2 at different rates, expert judgment 

might well be needed to expand the likely distribution beyond what can be 

derived from a simple data fit. 

The same panelist also strongly objected to the use of uniform distribution of 

concentrations of NO2 from use of gas stoves (p. 101). The very breadth of the 

bounds derived (4 – 188) ppb argues against a uniform distribution and in favor of 

something more skewed, such as a lognormal. The lognormal guarantees a 

positive contribution, and doesn’t have the unfortunate property of implying zero 

chance that the indoor contribution will be above the derived maximum. 

Moreover, if a mass balance approach is being used to model indoor NO2, then 

the input per cooking event should be in terms of mass units of NO2, not 

concentration. Concentration will depend on house- and temperature specific 

factors such as air exchange rates, NO2 removal rates and residual contributions 

from HONO, among other things. Because these observations were from a single 

house in California, there must be extra allowance for variability and uncertainty 

in these estimates that must clearly extend beyond the mass equivalent of the 

concentration range quoted. 
24

Finally the assumption that all cooking events contributing to indoor NO2 last 

exactly one hour also artificially limits the variability in NO2 inputs and therefore 

exposures represented in the model.” 

In addition to these suggestions from the last meeting, 

earlier suggestions on the planning document for the 

REA included explicit modeling to deal with the fact 

that the air quality monitors are generally greater 

than people’s breathing zones, leading to a systematic 

underestimation of air concentrations to which people 

are exposed in the air quality characterizations. 

My reading of the current chapter 8 is that the EPA 

authors have completely blown off all of these 

suggestions for revision of the uncertainty/variability 

analysis using the APEX modeling tool and the air 

quality modeling system. Not only that, the most 

recent version of the REA appears to omit sections of 

the previous exposure analysis draft that gave the 

reader a clue that improved quantitative 

characterization of the uncertainty and variability was 

possible. By not acknowledging these sources of 

uncertainty, and discussing the previous suggestions 

for improvement it seems to me they have not just been 

neglectful. It is fine if EPA wants to disagree with 

my/our previous review of their analysis on technical 

grounds. It is also fine if they want to say that they 

cannot make the improvements in their analyses that we 

suggested because of time and resource constraints. 

But to completely omit discussion of the previous 

critiques and suggestions for improvement of the 

analysis is unacceptable and could be interpreted as 

actively misleading. At the very minimum, the REA 

authors need to provide quantitative documentation and 

discussion of: 

• The height distributions for the air quality 

monitors, both nationally (as in earlier drafts of 

the REA and/or ISA appendices) and specifically 

for the case study cities (Philadelphia and 

Atlanta), 
25

• The 30-odd values of the on-road/off-road ratios 

that the authors used for their empirical 

distribution of this key parameter, 

• The six values for the indoor NO2 removal rate 

distribution that underlies the calculations of 

indoor air concentrations. 

Documentation of the individual data are needed for 

these parameters in order for the committee to 

adequately review the conclusions of the final REA and 

the eventual comparative evaluations of alternative 

standards. The “margin of safety” language of the 

Clean Air Act, it seems to me, requires the 

Administrator of EPA to consider in some detail the 

degree of confidence that can be achieved in 

conclusions that alternative NAAQS adequately protect 

public health. Therefore, absent a fair analysis and 

discussion of the likely quantitative effects of 

uncertainty and variability of these parameters on the 

estimates of population exposure under different 

regulatory scenarios, I would have the committee advise 

the Administrator that the current document is so 

deficient that it cannot be a reasonable basis for 

informed decision-making. 
26

Comments from Dr. Rogene Henderson

Preliminary Comments on Chapter 8 of the NOx REA 

Submitted by Rogene Henderson, PhD 

October 16, 2008 

My comments are given from the viewpoint a non-modeler. 

1.The details of the models for exposure were fully described and discussed. I think it is 

important to compare the modeling values with actual measurements to see how well the 

models reflect reality. This was described well in Chapter 8. I am concerned that the 

diurnal comparisons showed the models overestimated NO2 exposures at the beginning 

and ending of the day. Models can be useful when they are wrong because they tell us 

that we are not taking something(s) into account that influences, in this case, the diurnal 

variation in NO2. Has any thought been given to what that might be? 

2. Using only one city as the basis for the exposure analysis has the potential for not 

being representative of the total population of the US. However, from a practical 

viewpoint, limited resources prevent one from analyzing more cities in depth. I think the 

document describes the limitations of the approach well and I agree with it. 

3.The limitations of the models for exposure are discussed in great detail. 

4. I think of "health risk characterization" as comparing what we know about the toxicity 

or hazard potential of the pollutant with what we know about exposures to come up with 

the potential health risk. But in this chapter I only see the exposure portion of the 

characterization discussed. Can you add a discussion of the health effects you might 

expect from the described exposures? 
27

Comments from Dr. Timothy Larson

Comments by Tim Larson on Chapter 8 of the NO2 REA 

General Comments 

EPA is to be commended for the framework outlined in Chapter 8. The inclusion of 

deterministic approaches to spatial extrapolation of measurements from sparse network 

for purposes of exposure estimation is a welcome addition. It is an improvement on 

simple interpolation schemes that ignore the spatial distribution of important sources. 

My main concern with this analysis centers around the assumption that the census block 

centroids describe the location of the exposed population. My reading of the chapter says 

that only 1% of the exposed population is assumed to be living within 100 meters of 

relatively small roadways (>15,000 vpd). If this is the underlying assumption in the final 

assignment of ambient NO2 by AERMOD, then it does not reflect the reality in most 

urban areas. In fact, if this assumption is what was made, the exposed population in the 

Atlanta assessment could underestimated that in most urban areas by at least an order of 

magnitude. While I agree that the AERMOD model probably overpredicts the on-road 

exposures by up to a factor of 2, this is outweighed by the exposure location assignment 

bias. 

Something should also be mentioned in the uncertainty section regarding the roll-up 

methods. There is an apparent correlation between the average NO2 concentrations at 

EPA sites and the peak to mean ratio. In the roll-up method used by EPA, the 

assumption is that these two quantities are not correlated and therefore the peak to mean 

ratio remains constant during roll-up. The data suggest otherwise. Therefore the peak 

values may be overpredicted for a scenario just meeting the current annual standard. This 

is important when drawing conclusions about how well the current standard protects 

against the exceedance of peak hourly benchmark values. The analysis presented by 

Paine in the September 26 public comments suggests that the EPA approach is very 

conservative. There needs to be acknowledgement of this issue and/or a rebuttal 

argument. 

Specific Comments 

P6 line 19 Strictly speaking, AERMOD is a bi-Gaussian model 

P 7 line 8 clarify wording 

P23 what about smaller NOx point soujrces with short stacks? Is emission rate the only 

criterion? 

P25 only 1% near roadways? 

Fig 8-7 The overprediction of AERMOD could be the effect of ignoring vehicle induced 

turbulence as well as including too much fresh ozone. Comparison of NOx predictions 

would be useful in assessing the pure dispersion estimates. 
28

P31 locally high values? Do you mean within 100 meters of the road? 

Table 8-8 How do you get asthma prevalence rate for a 0-1 year old? 

P53 line 16 should read “spend more time” 

Fig 8-16 the in-transit estimates are probably generous given the assumption of pen=1. 

Also, given that people spend most time in their homes, even if the pen=0.5 for homes 

near roads, this could be important. In Atlanta, only 1% of the population lives within 50 

m of a major road?? 

P55 line 2 should read figure 8-19 

P61 line 13 in travel exposures may be underestimated due to ignoring the communting 

route on-road levels 

P63 line 24 a bit optimistic- e.g. compare with line 20 on p64 

P64 line 27 the vehicle wake effect is most pronounced for wind directions parallel to the 

road, where the wakes of multiple vehicles can reduced on-road concentrations more than 

a simple one-time adjustment factor (e.g. 1.7) applied to all wind directions (including 

cross-winds). 

P66-67 discussion of between city variation in APEX results for ozone not necessarily 

applicable to NO2, given the proximity to road effect for NO2. 

Other comments (including material in Appendix B) 

Discuss whether there any visual check on the conflation of TDM links with the actual 

street map. Using an algorithm that is based on closest road to the TDM node will not 

always put the link on the correct road (case of nearly parallel roads and also underpass 

links). 

Add a few sentences about the use of segmented lengths- discuss how much mismatch 

there is with actual road locations. Could be large for curved roads and may affect both 

on-road and off-road estimates from AERMOD. 

Initial NO2 to NOx ratio from tailpipe emissions- include a few sentences about the 

effect of catalytic converters on newer heavy duty vehicles. 

Add to the discussion of vehicle induced turbulence- initial sigma z values are not 

adjusted for wind path length along the roadway, an effect most pronounced for near 

parallel wind directions. 

Discuss the uncertainties of assuming a superimposed, steady state area source plume 

model for aggregated small roadway area sources- if travel times are greater than one 

hour, then the steady state assumption breaks down. 
29

Comments from Dr. Kent Pinkerton

COMMENTS: Kent E. Pinkerton, University of California, Davis 

CHARGE QUESTION #1: To what extent is the assessment, interpretation, and 

presentation of the results of the exposure analysis technically sound, clearly 

communicated, and appropriately characterized? 

REPLY: The authors of chapter 8 have done an incredible job to organize this chapter. 

The presentation of information within the chapter is extremely thorough and carefully 

described. The methodology and data used in inhalation exposure assessment and health 

risk characterization for NO2 as a criteria pollutant is up to date and highly innovative in 

its application as presented in Chapter 8. The estimation of temporal and spatial variation 

of NO2 concentrations appears to be a highly complicated and complex process, but the 

authors have done an reasonable job to provide a sound description and rationale. The 

authors describe approaches taken using the Air Pollutants Exposure Model (APEX) to 

estimate human population exposure, while AERMOD is used to characterize ambient air 

quality. The APEX model simulates exposure using 1) characterization of the study area, 

2) generation of simulated individuals, 3) construction of a sequence of activities, 4) 

calculation of hourly concentrations in mcroenvironments and 5) estimation of exposure. 

Each step in Chapter 8 is systematically described with potential concerns for potential 

error described. It is my evaluation that the assessment, interpretation, and presentation 

of the results of the exposure analysis in Chapter 8 have been done in a technically sound, 

clearly communicated, and appropriately characterized fashion. 

CHARGE QUESTION #2: The draft risk and exposure assessment document evaluates 

exposures in Atlanta. What are the views of the Panel on the approach taken and on the 

interpretation of the results of this analysis? 

REPLY: Atlanta appears to be an ideal location for crafting this risk and exposure 

assessment, based on the historical richness for NO2 concentrations in this region and the 

availability of an excellent database of information to use in creating this model. The 

inclusion of four counties covering much of urban and suburban Atlanta seems to provide 

a reasonable base to model and to make predictions for risk and exposure assessment. 

However, it is critical that the authors fully justify the selection of a single city or region 

for making predictions for the entire nation in the assessment of NO2 for the REA for 

NOx. 

CHARGE QUESTION #3: What are the views of the Panel regarding the adequacy of 

the assessment of uncertainty and variability with respect to characterization of exposures 

and health risks associated with those exposures? 

REPLY: The authors have carefully laid out the parameters used for the modeling to 

estimate NO2 exposure risk to individuals. The authors acknowledge the limitations of 

their model that must use assumptions, but have applied APEX which from the authors’ 

point of view provides the most power and flexibility to estimate exposure assessment 

and risk characterization. The authors acknowledge CHAD (Consolidated Human 

Activity Database) used in their model is likely to produce the greatest uncertainty. This 
30

concern is based on 1) the variability of human activity patterns and 2) much of the 

information used is over 20 years old. The authors have attempted to correct for some of 

these uncertainties, but there appears to still remain potential errors and/or factors that 

may not be able to be accounted for and/or are unknown. Population and commuting 

data, meterological data, and air quality data are only a few of the many parameters that 

must be factored into this model. Based on these parameters as well as indoor and 

outdoor factors, air exchange rates and others, it is impressive that the authors can make 

any predictions at all with any degree of certainty! 

CHARGE QUESTION #4: To what extent is the assessment, interpretation, and 

presentation of health risk characterization included in Chapter 8 technically sound, 

clearly communicated, and appropriately characterized? 

REPLY: The implementation of the exposure modeling and health risk characterization 

is very thorough, well documented and reasonably explained. However, concern remains 

when multiple assumptions must be made in a model where minor errors or large 

variations in values are possible, can easily be compounded, thus leading to a highly 

misleading exposure model or health risk characterization. Again, the authors simply 

need to make these limitations of the model clearly evident for the reader. 
31

Comments from Dr. Armistead Russell

Review of Chapter 8: 

EPA Staff and their contractor are to be complimented on the amount of work conducted, 

and how quickly, to address concerns discussed at the last NOx-primary meeting. While 

they have reduced some of the largest differences between the modeled and observed 

NO2, some remain. Those, for the most part, can be explained by some of the 

assumptions made: 

1. 10% NO2 in exhaust. This could be lower, and would, in particular, lower 

the early morning NO2 peak. It was chosen to be conservative. This is 

fine, and should be recognized as leading to a possible bias, and the issue 

observed. 

2. The temporal NOx emissions pattern would show less hourly variation 

than is likely, and, in particular, the smoothing used will put more 

emissions before 6 am than is likely the case. I drive here. There are few 

on the road before 6 am, and it is ghostish before 5 am. They would have 

done better to use the hourly temporal distributions available in SMOKE. 

This, alone, might get rid of the aberrant peak at 5 am. 

3. They could choose greater initial mixing values, though this would be 

small except for on-road estimates. 

On the other hand, I would actually think the way they did the NOx emissions estimation 

would be more accurate than the NEI, and the 28% is too big of an expected change (and 

would go in the other direction). 

There is a major bias in the AERMOD results that can not be readily dismissed, that 

being that the on-road/non-road ratio gets much higher for AERMOD. There is a 

consistent tendency, in all of the approaches, for AERMOD to have higher on-road than 

might be indicated by the observations. This is discussed, but should be highlighted 

more, particularly later on when interpreting the APEX results. As you will note, most of 

the exposures to high levels come from on- and near-road activities. 

The application of APEX appears solid, though the evaluation and interpretation is 

flawed. While I am glad to see the evaluation performed, it is misleading. In particular, 

Fig. 8-11 should show maximums, not P97.5 for comparison with Fig. 8-10. Indeed, I 

suspect we will see that the maximums predicted by APEX (which drive the later 

analysis) are much higher than observed. I would actually like to see a figure where they 

bin the APEX results in deciles (e.g., cumulative %ile deciles), and plot the min, median, 

97.5%ile and max. They also might estimate the 97.5%ile of the observations from Suh 

et al., for comparison. Call Helen. 

The reason all of the above is important (at least to me) is that we are compounding some 

apparently large biases at the upper end of the distribution to the point that I think we are 

well overestimating the population that is being exposed to greater than the benchmarks 

chosen. Remember, there has been one observation of over 200 ppb of NO2 over the 

three year period. What was the second high? Does that support the distribution found? 
32

In the end, I am uncomfortable with the compounding of biases for factors leading to the 

upper tail of the NO2 exposure distribution. While they may have to work with this, they 

should be extremely cautious about how the results are communicated. 
33

Comments from Dr. Jonathan Samet

Comments: Draft Chapter 8 of EPA’s Risk and Exposure Assessment to Support the 

Review of the NO2 Primary National Ambient Air Quality Standard 

General Comments: 

The draft Chapter 8 on Exposure Assessment and Health Risk Characterization is 

improved and now complete. The Staff made substantial progress in developing the 

chapter and it will prove useful as the basis for health risk characterization. As far as 

major concerns, I offer the following: 

• The document is detailed and lengthy, and does not offer a sufficiently 

informative overview of the approach that would be taken. It does have an 

introductory section, but the description is inadequate. I recommend the 

incorporation of appropriate diagrams that would lay out the various 

models that would be used and the data inputs for the modeling. This 

diagram would be useful for describing what has been done and also for 

describing the uncertainties that arise. 

• At a number of points, judgments are made with regard to what evidence 

shows and what determinations can be made. However, I find that the 

criteria for these determinations are lacking. For example, on page 27, 

lines 7-9, the conclusion is reached that adjustment of model 

concentrations to the ambient monitors was not necessary. I cannot find a 

clear rationale. Similarly, on page 24, lines 1-3, “general agreement” is 

said to be found between observed and modeled values. Again, what is 

“general agreement?” 
34

Comments from Dr. Richard Schlesinger

I do not have expertise in the area of exposure assessment. However, that said, the 

revised chapter did present material in a coherent and understandable manner. As I have 

noted in prior types of analyses such as these, I am concerned about having one site 

become the model site for all of the analyses. Thus, there should be a more 

comprehensive justification for use of this one site. 
35

Comments from Dr. Christian Seigneur

Comments on Chapter 8 of the 2nd draft of the REA 

Christian Seigneur 

CEREA, Université Paris-Est 

 

OAQPS has accomplished a large amount of work in a short time. Their overall 

methodology to calculate exposure and risk for Atlanta seems sound. However, I have 

some questions regarding the results of the air quality modeling. 

Section 8.4.2.4: It seems that the ozone limitation techniques were applied to AERMOD 

by source category ((mobile sources, airport, etc.). Thus, if two source categories were to 

impact significantly a same location, would it be possible to double-count the ozone 

available for oxidizing NO to NO2? 

Section 8.4.8: The report presents a thorough comparison of modeled and measured NO2 

concentrations. Both the cumulative distributions and the diurnal profiles show that 

AERMOD tends to overestimate the measured concentrations. Such results suggest that 

some adjustment of the modeled values is warranted to better reflect the actual 

concentrations. However, the document states that “it was determined that adjustment of 

the modeled air quality based on the three monitors was not necessary”. Such a statement 

could apply if the model showed error with no or negligible bias. However, the model 

shows a systematic bias (overestimation) for concentrations above the 60th percentile of 

the observed values (i.e., those concentrations of most interest for health effects). 

Therefore, it seems that some adjustment is warranted to scale the modeled values down 

toward the values observed at the monitors. 

Section 8.10.4: The discussion of the uncertainties associated with the air quality values 

does not highlight the significant bias that was shown in the AERMOD evaluation 

section. 
36

Comments from Dr. Elizabeth “Lianne” Sheppard

General comments: 

I wish to complement EPA staff for completing so much work in a short time. The 

APEX model for NO2 is complex and requires more complex inputs than some previous 

APEX applications, particularly given the 1-hour averaging time and the need to capture 

strong spatial gradients in the ambient exposure. Adequate spatio-temporal modeling of 

ambient pollutants is a new focus area for the ERA, and a strong effort has been made to 

apply available tools appropriately. While further improvements to the modeling and its 

assessment are desirable (and possibly necessary), significant revisions since the first 

draft REA have resulted in 1) better predictions and 2) a model assessment that is much 

better aligned with the modeling goals. 

Important issues that need additional attention before the REA is finalized: 

• Philadelphia vs. Atlanta. Make sure to bring out in several places in the report (intro 

to Appendix B (B-1), Appendix B case study descriptions (B-3 and B-4), intro to 

Chapter 8, and as added comments in conjunction with text references to the 

Philadelphia analysis (e.g. p 4, line 21)) the fact that these two modeling efforts were 

sequential and the improvements in Atlanta were not applied to Philadelphia. While 

it is reasonable to include the Philadelphia analysis in the appendix, the presence of 

the two will mean some parties will make an effort to compare the results. Since the 

modeling efforts were significantly different, this is not appropriate. Readers need to 

be told this. 

• AERMOD application: 

o On-road model assumptions: The strong discrepancy between the data and 

the model predictions plotted in Figure 8-7 suggest much more attention is 

needed to this model feature, particularly since it is an important driver of 

exceedances. Do the existing published data provide an adequate foundation 

for this direct comparison? Is the comparison of ratios of AERMOD receptors 

fair (since there may be a large amount of variability in locations of 

AERMOD receptors relative to the empirical studies)? 

o Census block centroids and the consequent impact on near-road exposures: 

The assumption that the Atlanta population resides at census block centroids 

must be evaluated more carefully since this could have a huge impact on 

estimation of exceedances. Exposure to high pollution falls off rapidly in the 

first 100 m from a road. The data show 1% of the simulated population 

(located at block centriods) is within 50m of a road. Since census block 

boundaries may not cross busy roads, the fraction of population living near 

busy roads as defined by census block centroids is most likely underestimated. 

For comparison, here are data from the 6 cities that are a part of the MESA air 

study. The percentages of 6,014 MESA Air study subjects living within 50 

meters of a busy road (census class A1-A3) are approximately 16% (WinstonSalem), 18% (Baltimore, Los Angeles, Minneapolis-St Paul), 26% (Chicago), 

and 48% (New York City). 

• Discussion and interpretation: Additional text is needed to discuss the results, 

particularly in terms of the large fraction of the simuated population experiencing 

exceedances and the dominant locations/activities where exceedances occur (which 
37

appear to me to be predominantly driven by near-road and on-road exposures). The 

main locations for exceedances are not only important from a policy perspective, but 

should also guide the discussion of uncertainty and variability. 

• Uncertainty and variability: The key questions to address with respect to uncertainty 

and variability are 1) what features are most influential in modeling the tails of the 

distributions, and 2) are there major sources of bias and uncertainty in these features. 

Both aspects need more attention. Make sure concerns about on-road and near-road 

exposures are addressed. One approach for structuring a revision may be to list every 

single assumption and model input (e.g. receptor height of 0 m) and discuss its 

possible contribution to bias and uncertainty. If more extensive analytical work were 

to be done in support of this REA, better quantification of bias and variability is most 

likely where the bulk of the attention should be focused. 

Specific comments: 

• Appendix B-4.14: It is good to summarize the receptor location distribution. This 

should be compared to information about the Atlanta population residence 

distribution. 

• p 1 l 7: insert “ambient” before NO2. 

• p 21 l19-21: Fix wording 

• p 25 l 5: Replace “the same seasonal pattern” with “similar seasonal relationships”. 

• p 25 l 16-19: This statement needs more analysis to support it. 

• Section 8.4.8.3: The value-laden statements in this section should be moved to the 

uncertainty and variability section. 

• p 37 l 11 and others following: Improve notation. Why define b if b=1-a? Include 

indexes, at least for individuals and time, to reflect that the predictions are across 

individuals and hours. 

• p 47, 52: The fraction of the population exposed to exceedances is huge and deserves 

further comment (most likely in the discussion section to be added). 

• p 49: The evaluation of microenvironments to reflect locations of exceedances is 

valuable. 
38

Comments from Dr. Frank Speizer

Comments on Chapter 8, REA NO2 Document, dated October 2008 

Submitted by : Frank E. Speizer, MD 

Pre Conference Call submission. 10/15/08 

In general I thought this chapter reads rather well. Most of my comments are minor. 

Charge Question 1: The first part of chapter 8 does an excellent job of describing what 

was done to make the exposure estimates. The only question would be whether it would 

be worth reviewing and expanding on both the selection of Atlanta and the 

reasonableness of Atlanta as a place to derive estimates for the whole country. As I recall 

some of this is discussed in previous chapter and there may be no need to repeat here 

except to reinforce the selection. 

Page 10-11, Tables 8-1 and 8-2: It is not clear how Central Business District (CBD) 

differs from Urban. May need an expanded footnote. 

Charge Question 2: The logic used in developing the assessment is well laid out. In table 

8.12 it might be worth constructing an addition set of figures for the 24 hour measure (or 

8 hour), since in the end we will need to advise on one number and it would be good to 

be able to justify how that one number differs from or covers the same at risk groupings 

as other time periods. At 100ppb there appears to be little difference in the 98th vs 99th

percentiles. Only be dropping the level to 50ppb does the form appear to have an effect. 

Without additional calculations, it would appear that this argues strongly for a margin of 

safety for safety sake rather than as an estimate of uncertainty (see below). 

Page 54, Figure 8-17: the labels on the figures need to be corrected. (a-a-a, should be ab-c). 

Charge Question 3: Although there is an excellent description of the uncertainties that 

are associated with the various analyses presented, there appears to be no comment in 

either direction of the magnitudes of what these uncertainties do to the estimates 

provided. It seems clear that in some cases they would actually reduce the variance of 

the estimates and in other cases may increase it. As indicated on the top of page 66, by 

analogy with O3 may not be adequate, and therefore some sensitivity analyses with NO2 

need to be performed. By example, the sensitivity test performed that generally showed 

low variation helped to reinforce the findings as presented, and thus reduce the potential 

importance of some of the issues mentioned. 

Page 68, section 8.10.8 is useful but raises an issue that is generic to the consideration of 

Atlanta. This section states that almost 100 % households are air conditioned. Thus 

understanding both penetration from outdoors as well as indoor source contribution, 

although understood for this region, may not be adequate for generalizing beyond the 

Atlanta region, particularly to regions that are not so saturated by air conditioning. 

Charge Question 4:Although the chapter does a reasonable job in characterizing the 

exposure of the at risk population, it seems to fall short in estimation of outcomes from 
39

those exposures. For example it never goes on to provide estimates of asthma 

exacerbations, or hospitalizations based on the estimates of exposure. This is particularly 

troublesome in that the lowest estimated exposure result in substantial numbers of at risk 

subjects. It would therefore be useful to indicate what those at risk subject will suffer 

from, and the degree that true health impacts will occur from those exposures. I would 

think this next step is necessary and look forward to hearing it presented in the 

conference call (or seeing a few charts or tables before the conference call). 

Section B-4.2.6. Supplemental Exposure Results 

 It appears that there is little impact on the exposure results until the Air Quality 

Adjustment gets to 50 ppb for the form (98 vs 99 percentile) to make any difference. 

This would suggest that estimates based on a lower bound of 100ppb are too high, and 

that they should be recalculated and presented as low as 50 ppb, if not lower. 

 In reviewing the rest of Appendix B it is clear that there is a repeat of the material 

from the first draft on Philadelphia, but because of the way that analysis was done and the 

way the current analysis of Atlanta was changed, they really are not comparable. I 

therefore am concerned that these two very different urban centers (eg. One warm and 

almost 100% air conditioned, and the other more typical of urban Northeast) can’t be 

judged side by side and therefore cannot figure out how comfortable to be with just using 

Atlanta. Right now it looks as if the estimates for Phili are done to a level of 200 ppb 

whereas for Atlanta they go down to 50 ppb. I would like to see similar analysis of what 

would be predicted exposures in Phili at the same levels seen for example in table B-50 

on page B-114. (There may be a reason why this can’t be done---as long as it is not that 

there is just not enough time to do it, and if so, so be it.) 
40

Comments from Dr. George Thurston

Prof. George Thurston’s Initial Comments for CASAC on Chpt. 8 of the Second Draft of: 

Risk and Exposure Assessment to Support the Review of the NO2 Primary National 

Ambient Air Quality Standard 

1. To what extent is the assessment, interpretation, and presentation of the results of the 

exposure analysis technically sound, clearly communicated, and appropriately 

characterized? 

 The EPA staff should be congratulated for their arduous effort at a formidable 

task to reach the ultimate goal of the chapter: to conduct a case study of an exposure and 

health risk characterization for various alternative NO2 standards. While the approach is 

basically technically sound, it involved numerous challenging modeling steps, each with 

their own uncertainties and possible biases that are potentially compounded from step to 

step. The populations considered and the 3 receptor classifications considered in these 

analyses seem very appropriate, and the modeling applied is state-of-the art. One major 

concern I have with regard to the modeling, though, is the continuing overestimation of 

peak values, as displayed (for example) in Figures 8-5 and 8-6. I wonder if it is the fact 

that the roadway emissions are reportedly modeled here using the Ozone Limiting 

Method (as noted on page 8, lines 10-12), an approach which I believe tends to give 

higher (more conservative) NO2 level estimates, since it is my understanding that this 

model assumes all available ozone is reacted with NO to form NO2, which will not likely 

be met at peak times of the day, seemingly consistent with the results in Figure 8-6. Is 

this the cause of these overestimates? If so, this assumption is fine for screening analysis, 

but may not be satisfactory for this application. 

2. The draft risk and exposure assessment document evaluates exposures in Atlanta. 

What are the views of the Panel on the approach taken and on the interpretation of the 

results of this analysis? 

The choice of Atlanta seems appropriate, given the existing pollution conditions and 

availability of modeling input information, but more discussion as to how these results 

might vary in other cities and regions would seem appropriate. Section 8.10.7.1 needs 

much further elaboration (e.g., sensitivity analysis of potentially varying conditions such 

as %AC, and discussion of the potential health effects/policy implications of this 

variability). 

3. What are the views of the Panel regarding the adequacy of the assessment of 

uncertainty and variability with respect to characterization of exposures and health risks 

associated with those exposures? 

 To me, this is the greatest weakness of the chapter as it is now written: no 

quantitative or qualitative propagation of errors by the sequential modeling steps is 

provided in Section 8.10. It is my belief that such a comprehensive uncertainty analysis 

would indicate large uncertainties in this approach to achieve the final estimates. A 
41

consideration of these uncertainties would, I think, indicate that, while a useful exercise, 

such a human exposure-health effects analysis is not sufficient to alone assess the health 

implications of alternative standards, and that a consideration of the results of an 

application of the NO2-health effects epidemiology are also required in each such REA 

(as is the case for this particular document) to obtain a fuller appreciation of the health 

implications of various alternative standards. 

4. To what extent is the assessment, interpretation, and presentation of health risk 

characterization included in Chapter 8 technically sound, clearly communicated, and 

appropriately characterized? 

 Very little is provided in the way of an interpretation of the results (e.g., putting 

the numbers into a context useful for regulatory assessment). I suggest an additional 

Section 8.11 to provide just such information. 
42

Comments from Dr. James Ultman

Comments on Second Draft NOx REA, Chapter 8 

James Ultman 

General Comments

I commend staff on completing an extensive body of work in such a short time. The 

document is clearly written. It conveys the methodology (including the description of 

computer models, sources of input data and assumptions) in as detailed a manner as is 

possible within the confines of a single chapter. The results are also summarized in a 

logical fashion. 

I do, however, feel somewhat overwhelmed by the complexity that emerges from the 

layering of different models, input data sets and accompanying assumptions that I feel are 

not always justified or validated. Moreover, lacking an integrated summary at the end of 

the chapter, it is not clear what key conclusions should be drawn. 

Specific Comments

I am skeptical that rolling back the health benchmark values in the exposure analysis is 

equivalent to rolling up of ambient concentrations (and visa versa). The document does 

not present clear evidence—either numerical or mathematical—that this is the case. 

Moreover, the equation that is used to justify a parallel roll-back in indoor source 

concentration (pg 38, line 5) fails to recognize that Cindoor is determined, in part, by 

Cambient. In particular, if Cambient is rolled up by a factor f (pg. 38, line 3), Cindoor must also 

increase. Thus, its original value cannot be used to compute Cthresh (see equation on pg. 

38, line 3). Moreover, the relationship between Cindoor and Cambient is algebraically nonlinear when the mass balance model is used to determine Cindoor. 

Given the importance of on-road exposures, I am also concerned with the discrepancy 

between the AERMOD-based and data-based determination of the on-road/non-road 

concentration ratio. On the other hand, this is not surprising given that the data was 

measured in a number of locales where meteorology and emission patterns are different 

than in Atlanta. 

Finally, I believe that the document provides an extensive discussion of areas of 

uncertainty, but it does not attempt to quantify the uncertainty. In retrospect, I feel that it 

would have been wiser to spend less time in the modeling/simulation effort and more 

time in analyzing those uncertainties that are judged to be the most important.