![](media/image1.wmf){width="1.0833333333333333in"
height="1.0291666666666666in"}

> **UNITED STATES ENVIRONMENTAL PROTECTION AGENCY**

**WASHINGTON D.C. 20460**

> **OFFICE OF THE ADMINISTRATOR**
>
> **SCIENCE ADVISORY BOARD**

October 28, 2008

EPA-CASAC-09-001

The Honorable Stephen L. Johnson

Administrator

U.S. Environmental Protection Agency

1200 Pennsylvania Avenue, N.W.

Washington, D.C. 20460

Subject: Clean Air Scientific Advisory Committee's (CASAC) Peer Review
of Draft Chapter 8 of EPA's *Risk and Exposure Assessment to Support the
Review of the NO~2~ Primary National Ambient Air Quality Standard*

Dear Administrator Johnson:

The Clean Air Scientific Advisory Committee (CASAC), augmented by
subject-matter-experts to form the CASAC Oxides of Nitrogen Primary
National Ambient Air Quality Standards (NAAQS) Review Panel (hereafter
referred to as the panel, roster provided in Enclosure A) held a public
teleconference on October 22, 2008 to review draft chapter 8 of EPA's
*Risk and Exposure Assessment (REA) to Support the Review of the NO~2~
Primary National Ambient Air Quality Standard.* Chapter 8 is entitled
\"Exposure Assessment and Health Risk Characterization.\" Additionally,
CASAC offers comments on the schedule for NAAQS revision, prompted by
the Staff's presentation of the proposed outline for Chapter 10 of the
REA.

Chapter 8 is critical to the REA because it provides a characterization
of health risk from NO*~2~* exposure based on the estimated number of
exceedances obtained from a comprehensive personal exposure model for
asthmatics in Atlanta. The CASAC panel was generally impressed with the
careful work done over a short time with available tools and resources,
but it has concerns about the implications of certain model inputs and
assumptions and about the general presentation of the approach and
findings. The panel asks for changes in two main areas: in the
interpretive presentation and in characterization of potential biases.
The CASAC plans to review the completed REA at its December 5, 2008
public teleconference and will provide recommendations for EPA\'s
consideration in developing the Advance Notice of Proposed Rulemaking
(ANPR) for NO~2~ during and immediately after that teleconference.

[Response to EPA Charge Questions]{.underline}

EPA requested CASAC's responses to four charge questions related to the
exposure assessment and health risk characterization of the draft
chapter 8 provided for review. The charge questions and responses follow
immediately below.

1\. *To what extent is the assessment, interpretation, and presentation
of the results of the exposure analysis technically sound, clearly
communicated, and appropriately characterized?*

Staff has produced a solid application of the APEX model. While
recognizing that there have been significant improvements since the
first draft of the REA and the update at the September CASAC meeting,
the panel still has concerns about the application of AERMOD. Staff
should give serious consideration to improving AERMOD model performance
in comparison to the concentrations measured by the monitors in Atlanta.
For example, AERMOD output could be adjusted to better reflect observed
peak values and diurnal profiles. Many factors influence the AERMOD
predictions and the panel is concerned that the overall uncertainty,
reflective of the uncertainties from each of these factors, is not
appropriately characterized. In particular, the tails of the exposure
distribution could be strongly biased, notably with respect to on-road
and near-road exposures. These exposures are particularly important,
since they are the major contributors to the benchmark exceedances.
On-road exposures may be biased high because receptors are in the middle
of the road; the exhaust is not diluted much; and observed data suggest
that the on-road vs. background ratios are much more narrowly
distributed than the AERMOD predictions. Near-road exposures may
similarly be affected by biases in the on-road estimates. In addition,
near-road exposures may not be appropriately weighted to the Atlanta
population because a greater proportion of residents may live closer to
major roads than is represented by the assigned locations of model
receptors at the census block centroids used in the AERMOD model.

In evaluating population exposure under alternative standards, Staff
elected to roll back health benchmark values as a time-saving substitute
for rolling up ambient concentrations. In view of the potentially strong
influence of this decision on the numerical determination of benchmark
exceedances, the chapter needs to provide a clearer justification for
this approach and consideration needs to be given to validating the
procedure.

2\. *The draft risk and exposure assessment document evaluates exposures
in Atlanta. What are the views of the Panel on the approach taken and on
the interpretation of the results of this analysis?*

The chapter needs to better evaluate the generalizability of findings
for Atlanta. In particular, three aspects of the selection of Atlanta
should be addressed more carefully. First, the selection of Atlanta
should be reviewed with consideration of the extent to which the city
represents exposures nationally, since the model results are being used
as the basis for setting a national ambient air quality standard. What
are typical features of the population behaviors, building use (e.g.,
air conditioning), housing stock, and residential patterns that suggest
similarities and differences with other major cities, particularly with
respect to anticipated high-end exposures to NO*~2~*? As a minimum, the
panel suggests that a table be added to describe variation of key
parameters across cities. Ideally, a sensitivity analysis that assessed
the importance of such parameters would also be included. Second, is the
model applied to Atlanta adequately reflecting the local population? In
particular, in assigning model receptors to census tract centroids, is
the model systematically missing the fraction of the population who
lives closest to roads and is consequently regularly exposed to high
near-road exposures? The fraction of the Atlanta population that lives
within 50 meters of a major roadway could be much higher than the 1%
figure represented by the census tract centroids. We encourage EPA to
seek an independent estimate of this important quantity. Third, the
application of AERMOD to Atlanta suggests a significant potential to
predict concentrations that are biased upwards at the high end of the
distribution. This bias should be more thoroughly discussed, along with
likely implications.

*3. What are the views of the Panel regarding the adequacy of the
assessment of uncertainty and variability with respect to
characterization of exposures and health risks associated with those
exposures?*

A complete analysis of uncertainty is an essential component of a
thorough exposure and risk assessment. The panel is concerned that there
is the potential for more bias and uncertainty in the results than has
been characterized in the current uncertainty and variability assessment
and that no rationale has been provided for assigning weight or priority
to identified uncertainties. A better characterization of uncertainty is
needed, with attention to prioritization of the multiple sources of
uncertainty, and where possible, quantification of their impact.
Qualitative approaches that might be followed for this purpose can be
found in other reports[^1]

In addition, previous suggestions made by individual panel members on
this topic have not been explicitly considered; the concerns raised
should be addressed in the final report. More thorough characterization
of the assumptions and model features that most strongly influence the
estimated number of exceedances is needed. Most notably, the on-road and
near-road exposures are important determinants of the number of
exceedances of specified concentration levels experienced by the
population. In addition to on- and near-road characterization, and
locations of receptors, assumptions about penetration into various
microenvironments, particularly transportation environments, should be
discussed. The implications of the significant biases discussed in the
application of AERMOD need to be covered in this section. Value-laden
statements in Section 8.4.8 about the acceptability of AERMOD
over-predictions should be relocated to Section 8.10 and discussed in
the context of variability and uncertainty.

4\. *To what extent is the assessment, interpretation, and presentation
of health risk characterization included in Chapter 8 technically sound,
clearly communicated, and appropriately characterized?*

The overall presentation of the chapter needs to be improved in order to
help readers to understand the broad framework of the modeling and to
give a context for the health risk characterization. In addition to
expanding the overview section, several figures and tables should be
added to show the relationships among the various models and to
summarize the data inputs. Furthermore, this chapter needs to note that
the health risk characterization focuses exclusively on exposure
benchmark comparisons through additional discussion and/or
cross-referencing to other sections of the REA. EPA should add a
concluding section that summarizes the primary results and implications
from the large number of models and analyses presented in the chapter.

[Comments on the Schedule for NAAQS Revision]{.underline}

Staff provided an overview of plans for Chapter 10 of the REA, which
will be critical for considering options for the NAAQS for NO~2~. The
timeframe of the teleconference permitted only a brief exchange around
the general approach that will be taken. The panel was told that time
might not permit the EPA to respond to the CASAC recommendations
concerning Chapter 10 and the implications of that summary chapter for
developing the ANPR, following the CASAC public teleconference planned
for December 5, 2008. The REA and the ANPR are critical parts of the new
review process and the panel would find it unacceptable to be unable to
provide advice on these documents in an effective manner. Time for these
essential CASAC review steps should be included in any revisions to the
timetable for completion of the NO~x~ review.

This is only one example of the implications of the rapid schedule for
completion of the NAAQS revision process. EPA is producing and revising
analyses and documents at a pace that is compromising CASAC's review
function and consequently the quality of the scientific foundation for
revising the NAAQS. CASAC recognizes the drivers of the schedule for
NAAQS revision but cautions that there are now warnings that CASAC\'s
review role, mandated by the Clean Air Act, is likely to be diminished
by the compressed schedule.

In closing, the panel noted the substantial progress in the development
of this chapter of the *Risk and Exposure Assessment.* We look forward
to reviewing the final version of the entire document later this
year**.**

> Sincerely,
>
> ***/Signed/***
>
> Dr. Jonathan M. Samet, Chair
>
> Clean Air Scientific Advisory Committee

cc: Marcus Peacock, Deputy Administrator

**NOTICE**

> This report has been written as part of the activities of the EPA\'s
> Clean Air Scientific Advisory Committee (CASAC), a Federal advisory
> committee independently chartered to provide extramural scientific
> information and advice to the Administrator and other officials of the
> EPA. The CASAC provides balanced, expert assessment of scientific
> matters related to issues and problems facing the Agency. This report
> has not been reviewed for approval by the Agency and, hence, the
> contents of this report do not necessarily represent the views and
> policies of the EPA, nor of other agencies within the Executive Branch
> of the Federal government. In addition, any mention of trade names or
> commercial products does not constitute a recommendation for use.
> CASAC reports are posted on the EPA Web site at:
> [[http://www.epa.gov/casac]{.underline}](http://www.epa.gov/casac_).

**Enclosure A**

**U.S. Environmental Protection Agency**

**Clean Air Scientific Advisory Committee**

**Oxides of Nitrogen Primary NAAQS Review Panel**

**CHAIR**

**Dr. Jonathan M. Samet**, Professor and Chair of the Department of
Epidemiology, Bloomberg School of Public Health, Johns Hopkins
University, Baltimore, MD

**CASAC MEMBERS**

**Dr. Joseph Brain**, Philip Drinker Professor of Environmental
Physiology, Department of Environmental Health, Harvard School of Public
Health, Harvard University, Boston, MA

**Dr. Ellis B. Cowling**, University Distinguished Professor At-Large,
Emeritus, Colleges of Natural Resources and Agriculture and Life
Sciences, North Carolina State University, Raleigh, NC

**Dr. James Crapo**, Professor of Medicine, Department of Medicine,
National Jewish Medical and Research Center, Denver, CO

**Dr. H. Christopher Frey**, Professor, Department of Civil,
Construction and Environmental Engineering, College of Engineering,
North Carolina State University, Raleigh, NC, USA

**Dr. Donna Kenski**, Data Analyst, Lake Michigan Air Directors
Consortium, Des Plaines, IL

**Dr. Armistead (Ted) Russell**, Professor, Department of Civil and
Environmental Engineering, Georgia Institute of Technology, Atlanta, GA

**CONSULTANTS**

**Professor Ed Avol**, Professor, Preventive Medicine, Keck School of
Medicine, University of Southern California, Los Angeles, CA

**Dr. John R. Balmes**, Professor, Department of Medicine, Division of
Occupational and Environmental Medicine, University of California, San
Francisco, CA

**Dr. Douglas Crawford-Brown**, Emeritus Professor and Director,
Department of Environmental Sciences and Engineering, University of
North Carolina at Chapel Hill, Chapel Hill, NC

**Dr. Terry Gordon**, Professor, Environmental Medicine, NYU School of
Medicine, Tuxedo, NY

**Dr. Dale Hattis**, Research Professor, Center for Technology,
Environment, and Development, George Perkins Marsh Institute, Clark
University, Worcester, MA

**Dr. Rogene Henderson**, Scientist Emeritus, Lovelace Respiratory
Research Institute, Albuquerque, NM

**Dr. Patrick Kinney**, Associate Professor, Department of Environmental
Health Sciences, Mailman School of Public Health , Columbia University,
New York, NY

**Dr. Steven Kleeberger**, Professor, Lab Chief, Laboratory of
Respiratory Biology, National Institute of Environmental Health
Sciences, National Institutes of Health, Research Triangle Park, NC

**Dr. Timothy V. Larson**, Professor, Department of Civil and
Environmental Engineering, University of Washington, Seattle, WA, USA

**Dr. Kent Pinkerton**, Professor, Regents of the University of
California, Center for Health and the Environment, University of
California, Davis, CA

**Dr. Edward Postlethwait**, Professor and Chair, Department of
Environmental Health Sciences, School of Public Health, University of
Alabama at Birmingham, Birmingham, AL

**Dr. Richard Schlesinger**, Associate Dean, Department of Biology,
Dyson College, Pace University, New York, NY

**Dr. Christian Seigneur**, Director, Atmospheric Environment Center,
Université Paris-Est, Champs-sur-Marne, France

**Dr. Elizabeth A. (Lianne) Sheppard**, Research Professor,
Biostatistics and Environmental & Occupational Health Sciences, Public
Health and Community Medicine, University of Washington, Seattle, WA

**Dr. Frank Speizer**, Edward Kass Professor of Medicine, Channing
Laboratory, Harvard Medical School, Boston, MA

**Dr. George Thurston**, Professor, Environmental Medicine, NYU School
of Medicine, New York University, Tuxedo, NY

**Dr. James Ultman**, Professor, Chemical Engineering, Bioengineering
Program, Pennsylvania State University, University Park, PA

**Dr. Ronald Wyzga**, Technical Executive, Air Quality Health and Risk,
Electric Power Research Institute, Palo Alto, CA

**SCIENCE ADVISORY BOARD STAFF**

**Dr. Angela Nugent**, Designated Federal Officer

**Enclosure B: Compilation of Individual Panel Member Comments on
Chapter 8 of EPA's *Risk and Exposure Assessment (REA) to Support the
Review of the NO~2~ Primary National Ambient Air Quality Standard***

This enclosure contains final written comments of individual members of
the Clean Air Scientific Advisory Committee (CASAC) Oxides of Nitrogen
Primary National Ambient Air Quality Standards (NAAQS) Review Panel. The
comments are included here to provide both a full perspective and a
range of individual views expressed by panel members during the review
process. These comments do not represent the views of the CASAC or the
CASAC Panel.

**[Comments Received]{.underline}**:

[Comments from Prof. Ed Avol 9](#__RefHeading___Toc212907964)

[Comments from Dr. John Balmes 11](#__RefHeading___Toc212907965)

[Comments from Dr. Joseph Brain 13](#__RefHeading___Toc212907966)

[Comments from Dr. James Crapo 14](#__RefHeading___Toc212907967)

[Comments from Dr. Douglas Crawford-Brown
15](#__RefHeading___Toc212907968)

[Comments from Dr.H. Christopher Frey 19](#__RefHeading___Toc212907969)

[Comments from Dr. Terry Gordon 20](#__RefHeading___Toc212907970)

[Comments from Dr. Dale Hattis 22](#__RefHeading___Toc212907971)

[Comments from Dr. Rogene Henderson 26](#__RefHeading___Toc212907972)

[Comments from Dr. Timothy Larson 27](#__RefHeading___Toc212907973)

[Comments from Dr. Kent Pinkerton 29](#__RefHeading___Toc212907974)

[Comments from Dr. Armistead Russell 31](#__RefHeading___Toc212907975)

[Comments from Dr. Jonathan Samet 33](#__RefHeading___Toc212907976)

[Comments from Dr. Richard Schlesinger 34](#__RefHeading___Toc212907977)

[Comments from Dr. Christian Seigneur 35](#__RefHeading___Toc212907978)

[Comments from Dr. Elizabeth "Lianne" Sheppard
36](#__RefHeading___Toc212907979)

[Comments from Dr. Frank Speizer 38](#__RefHeading___Toc212907980)

[Comments from Dr. George Thurston 40](#__RefHeading___Toc212907981)

[Comments from Dr. James Ultman 42](#__RefHeading___Toc212907982)

[]{#__RefHeading___Toc212907964 .anchor}[Comments from Prof. Ed
Avol]{.underline}

Comments on Second Draft NOx REA, Chapter 8

Ed Avol

***[General Comments:]{.underline}***

My expertise is not in modeling, so I am not familiar with AERMOD and
will have to defer to others regarding the appropriate application of it
here. I did appreciate that there was a considerable effort undertaken
to lay out the modeling approach, and Staff should be commended for
those efforts. Although a great deal of detail was provided in the
chapter, I did not feel it was properly synthesized or summarized for
the reader. A final few paragraphs asserting what it all means would
have been useful. There was considerable discussion about uncertainty,
but it, too, seemed almost philosophical in nature, rather than specific
-- can anything focused to the effort at hand be said regarding the
magnitude or degree of uncertainty related to this modeling exercise?

***[Specific Comments:]{.underline}***

Table of Contents, Section 8.8 title, (typographical error), check font
type size

P4, Section 8.3.1, line 21 -- the statement that "...Atlanta was
selected as the second-case study location..." is made, with no
reference, comment, or explanation about the first study case
(Philadelphia), the reasons for going to a second, the generalizability
of either of these selections, or the selection process...something
needs to be said as to how these choices were made, even if it is to a
discussion deferred to the appendix (although the appropriateness of
this selection would seem to be an issue to be addressed in the main
body of the document).

P5, Section 8.3.3, lines 5-7 -- the assertion is made that 3 years of
met data (2001-2003) are used to achieve a degree of stability in
dispersion and exposure model estimates. Given recent fluctuations in
meteorological patterns, how "stable" or "usual" are the 2001-2003
weather patterns for the area? Are they, for example, unusual drought
years, wet years, hot years, or cold years? Is it reasonable (and
credible) to use them to generalize for the air quality modeling?

P6, Section 8.3.4, lines 7 and 8 -- specify units (years)

P6, Section 8.4.1, line 23 -- change "are" to "were"

P6, Section 8.4.1, line 27 -- typo at end of word "included"

P7, Section 8.4.2.1, line21 -- insert "airport" in Atlanta Hartsfield
name.

P8, Section 8.4.2.4, line 16 -- insert "was" after "10%"

P11, Table 8.2 -- Comment -- the HDV fractions seem high (10%-25%) for a
metropolitan area with commuters (most of whom will use LDV for
transportation)...?

P19, Section 8.4.7, line 14-15 -- how does assignment of receptors to a
height of 0 meters make the model more efficient (let alone realistic)?

P21, Section 8.4.8.1, lines 15-17 -- does that observation that there
were only two predicted exceedances of 150ppb NO2 at one monitor and no
estimated NO2 concentrations above 200ppb for 1hour say something about
the modeling results, or the location of the samplers?

P29, Section 8.5, line 4 -- 64,000 asthmatics/500,000 children is about
13%; is this a high value or in conformance with other estimates for the
region?

P33, Section 8.7.2.1, lines 5-18 -- this discussion exclusively focuses
on gas cooking as the only indoor source of NO2 interest. What about
un-vented or poorly-vented room or wall heaters or fireplaces?

P37, line 8 -- change "an" to "and"

P38, Section 8.9.1, lines 22-23 -- This statement is not necessarily
true, so it would be better to just say the exposure results focused on
asthmatics.

P39, Section 8.9.2, line 13 -- change "individual" to "individuals"

P39, Section 8.9.2, lines 18-21 -- this is poorly phrased and confusing,
since these are two different thoughts. Suggest breaking in to two
sentences: "...with most containing less than 30ppb." "About 5% of these
receptors..."

P53, Figure 8-17 -- Why are there three Figure 8-17a illustrations? How
can the first Figure 8-17a ([\>]{.underline}100ppb) be so different from
the rest?

[]{#__RefHeading___Toc212907965 .anchor}[Comments from Dr. John
Balmes]{.underline}

Characterization of Exposure and Health Risks Associated with Exposure
(Chapter 8 and

Appendix B): John Balmes' comments

GENERAL COMMENTS

The agency staff members responsible for the work presented in Chapter
and Appendix B are to be commended for their hard work over a short
timeframe.

1\. To what extent is the assessment, interpretation, and presentation
of the results of

the exposure analysis technically sound, clearly communicated, and
appropriately

characterized?

I do not have modeling expertise, but I found the logic and results of
the modeling efforts conducted to be presented clearly. The discussion
of assumptions and uncertainties was also clear. A general concern that
I have is that the overall modeling exercise is complex and dependent on
many sources of data with varying quality and uncertainties. Multiple
assumptions are required. Is it necessary to support the air quality
standard setting with such a complex model. Put another way, could a
simpler model with less assumptions and less data inputs be built that
provides reasonable bounds to the likely population exposures to NO2
above the benchmark values, but that is also easier for non-modelers to
understand? A specific concern that I have is that APEX apparently uses
only one temperature value per day (p. 63, sub-section 8.10.3). Given
the overall complexity of the exposure model, one would think that
temperature could be better parameterized.

2\. The draft risk and exposure assessment document evaluates exposures
in Atlanta.

What are the views of the Panel on the approach taken and on the
interpretation of

the results of this analysis?

Atlanta is a large metropolitan area that has both relatively good
ambient air quality and health effects data. Given limitations of time
and staff resources, I think the decision to focus Atlanta is
reasonable. My major concern about the decision is that because of its
climate, the penetration of air conditioning is high (apparently almost
100%). This makes modeled NO2 exposures in Atlanta less representative
of such exposures in areas with more temperate climates.

3\. What are the views of the Panel regarding the adequacy of the
assessment of

uncertainty and variability with respect to characterization of
exposures and

health risks associated with those exposures?

The variability and uncertainty section of the chapter (section 8.10)
reads well, but there is no overall assessment of the impact of the
sources of uncertainty on the exposure estimates presented in section
8.9.

4\. To what extent is the assessment, interpretation, and presentation
of health risk

characterization included in Chapter 8 technically sound, clearly
communicated,

and appropriately characterized?

There is no presentation of health risk characterization. In my view,
presentation of the number of potential exposures to NO2 of the total
population or persons with asthma does not constitute sufficient health
risk characterization. The impact of the estimated exposures on the
health of the total and asthmatic populations should be discussed in the
chapter.

SPECIFIC COMMENTS

p\. 17, line 5 For what the abbreviation, SCC, stands should be spelled
out the first time it is used..

p\. 33, line 11 American Housing Survey should be spelled out with its
first use.

p\. 33, lines 17-18 Should be "...CHAD diaries[,]{.underline} stratified
by hour of the day and normalized to the expected value of daily food
preparation events[,]{.underline} 1.4 (Table 8-11).

p\. 39, line 29 Persons do not "contain" exposures to ambient exposures.

p\. 41, lines 6-8 Ibid.

p\. 42, line 12 Should be "...cooking fuel as [an]{.underline}
influential variable..."

p\. 42, line 13 Should be "...of whom 13 had personal..."

p\. 42, line 17 Should be "...an exposure distribution was constructed
[for]{.underline} each individual,..."

p\. 43, line 3 Should be "1999-[20]{.underline}00".

p\. 51, line 7 Should be "...the time[s]{.underline} spent..."

p\. 51, line 12 Should be "...regarding the [e]{.underline}ffect of
indoor sources..."

p\. 51, line 13 "...microenvironments[,]{.underline}
however[,]{.underline} changes..."

p\. 51, line 16 Delete "a" between "to" and "comprising".

p\. 56, lines 27-28 "Results for [asthmatics when]{.underline}
exposure[s]{.underline} were estimated..."

p\. 66, line 15, line 26 For what the abbreviations, CMSA and CSA stand
should be spelled out the first time they are used.

[]{#__RefHeading___Toc212907966 .anchor}[Comments from Dr. Joseph
Brain]{.underline}

Chapter 8 and other parts of the REA document rely heavily on analyses
of Atlanta data. An essential question which deserves much more
attention is the extent to which these findings can be generalized. In
terms of NOx levels and other aspects of exposure assessment, we need to
know how Atlanta compares to other population centers throughout the
United States, such as Los Angeles, Houston, or Chicago. What is the
range of NOx exposure in these other cities? Other critical parameters
should also be compared? What is the prevalence of air conditioning?
What do we know about human activity in different cities, e.g. time
spent indoors vs. outdoors, and what about relevant confounders such as
temperature and humidity?

I am sensitive to the limited time available to address their issues.
There may not be time to carry out what should be done -- a sensitivity
analysis. To what extent do the predictions and models change when we
substitute data from other cities? One could, however, even with the
existing Atlanta model, change various parameters, and see what happens
to the outcomes. Another approach would be to create a table which lists
these parameters for selected US cities. That would reinforce the point
that safety factors need to be built in because of uncertainty about the
extent to which Atlanta is typical.

[]{#__RefHeading___Toc212907967 .anchor}[Comments from Dr. James
Crapo]{.underline}

Comments on Chapter 8, REA NO2 Document, dated October 2008

In general the EPA staff have done an excellent job in compiling a very

complex model to predict the number of individuals in Atlanta who would
be exposed to NO~2~ levels above various potential Health Effect
Benchmark Levels. The models are appropriate and the descriptions of how
the data was used in the models, including the assumptions involved, are
clearly described. This will provide a reasonable basis on which to
estimate the population impact of conformance to various NO~x~ NAAQS. My
concerns are:

1\. The model is extremely complex with a large number of assumptions at
almost every aspect of the model. In most cases it is not known how
these assumptions may aggregate to create a significant positive or
negative bias in the results. Would a much simpler model provide
estimates of populations at risk that would be of similar - or better -
final accuracy? This complex model may create a false sense of accuracy.
My point is that this model needs to be validated with experimental
data. The model should be used to predict testable outcomes such as ER
admissions for asthma, etc., which can then be compared to actual
observations. This could be done in a prospective fashion or by applying
the model to prior time periods not used to create the model, but where
outcome data is already known.

2\. The overview for the chapter states that the intent is to document
the methodology and data used in the inhalation exposure assessment and
in the associated health risk characterization for NO~2~. The chapter
only models the temporally and spatially variable NO~2~ concentrations
and simulates the predicted human contact with those modeled NO~2~
concentrations. The chapter needs to go on to use these simulations to
predict health risk in terms of testable outcomes - i.e.: mortality,
hospital admissions, ER admissions for asthma, exacerbation of
respiratory symptoms, etc. This is the ultimate purpose in creating
these models and will provide a means to eventually validate the utility
of the models.

[]{#__RefHeading___Toc212907968 .anchor}[Comments from Dr. Douglas
Crawford-Brown]{.underline}

**Comments on Risk and Exposure Assessment to Support the Review of the
NO2 Primary National Ambient Air Quality Standard: Chapter 8 and
Appendix**

**Doug Crawford-Brown**

**April 22, 2008-04-27**

These comments focus on Chapter 8 and the appendix of the Risk and
Exposure Draft. My review is organized around the four charge questions.

1\. To what extent is the assessment, interpretation, and presentation
of the results of

the exposure analysis technically sound, clearly communicated, and
appropriately

characterized?

I was in general quite pleased with by the analysis and the way it was
described both in the text and in the appendix (which was, by the way,
well written). There are a number of important issues from the previous
review that have now been addressed adequately, and there has been an
impressive amount of work done in short order to provide these
improvements. I found it easy to understand the analytic steps, and the
justification for formulating them as the staff did.

The characterization of AERMOD could use a bit more work. My sense is
that it biases results upwards, which would then bias upwards the
predicted number of effects. This aspect of the model is not adequately
described, nor is it reflected in the uncertainty section. I also was
never completely clear as to how the percentiles of ambient
concentrations were being used in the Atlanta and Philadelphia examples,
or whether there even was consistency between the two case studies in
regards to percentiles used. There needs to be a concise paragraph or
two explaining more clearly how the upper tails of distributions were
used and why these particular percentiles were selected. Otherwise, it
strikes me -- if I were a first time reader -- that there is a lot of
conservatism built into the analysis, especially using parts of a
distribution (the upper tails) where I am less confident in the
results..

The discussion of receptor locations in Section 8.4.7 could use
improvement. I was never clear in reading that section how the receptor
locations were related to the APEX modelling, which can follow people
through an actual exposure field rather than through representative
locations of exposure. As I read the section, it sounded to me as if a
few receptor locations were selected as representative of ambient
concentrations associated with a given activity, and I was not sure how
that fit into the APEX approach based on census tracts. I can imagine
how it is done, but the section didn't give me enough description to
understand the issue fully.

I repeat here a point I raised in previous drafts. I generally approve
the proportional roll-up or roll-down methods based on current maximum
concentration at a specific site. And I agree that the adjustment of the
benchmarks produces the same result mathematically. But it continues to
make no sense scientifically, and. the savings in processing time don't
appear to me sufficient to justify a method that people will fail to
understand as mathematically equivalent.

2\. The draft risk and exposure assessment document evaluates exposures
in Atlanta.

What are the views of the Panel on the approach taken and on the
interpretation of

the results of this analysis?

I agree with the approach taken, subject to the comments in Charge
Question 1. It seemed to me, however, that the analyses for Atlanta and
Philadelphia were not quite the same, making comparisons difficult. I
particularly liked the generation of results on person-days of effects
in Atlanta, and wish the same had been done for Philadelphia.

I think the interpretation of results for Atlanta is appropriate, given
the assumptions made. However, as mentioned earlier, AERMOD appears to
bias results high, or at least above the monitoring results. This should
factor into the interpretation, or at least into the uncertainty
analysis, and I didn't find that to be the case.

I also didn't know how the staff were interpreting the results that
divide effects between categories of with or without indoor sources
included. In some cases, the large majority of effects appear to be from
indoor (and hence uncontrolled) sources, while the results as the
ambient levels rise show the dominance of the ambient exposures. There
should be a better description of how the staff believe these results
should be interpreted. However, I do agree with the results themselves,
again given the assumptions and models employed.

The development of the longitudinal activity sequences is a
sophisticated piece of work, and is described adequately. The one
comment I have here is that at one point, the staff had to decide
whether to use correlated daily activities or uncorrelated. They used a
cluster approach, which I support. However, it was a bit difficult,
reading the text alone, to understand the cluster approach selected or
why the authors believe it produces results intermediate between the
other two results. It might have been useful to present a representative
set of results under each of the three approaches and see how much
variation it produces (a kind of sensitivity analysis).

3\. What are the views of the Panel regarding the adequacy of the
assessment of

uncertainty and variability with respect to characterization of
exposures and

health risks associated with those exposures?

The uncertainty analysis remains inadequate to produce any sort of
confidence bounds on the analytic results, although it improves with
each iteration we have seen. And it now gives the reader a sense of the
contribution to variance from each component. I am not pushing for a
formal nested variability-uncertainty analysis, but it does seem to me
that something akin to what went into NATA might be appropriate, where
at least the overall uncertainty was categorized. And there remains the
issue of the influence of the conservatism of AERMOD on whether final
results should be seen as simply uncertain, or biased upwards.

On a minor point, there is an odd part of Page 67 where the authors talk
of a negative value for the GSD and for the GM. I must be reading it
incorrectly (since these negative values are not possible), but I cannot
understand these sentences.

4\. To what extent is the assessment, interpretation, and presentation
of health risk

characterization included in Chapter 8 technically sound, clearly
communicated,

and appropriately characterized?

My comments are the same as in Charge Question 1, although I note here
that the presentation of health effects results was quite effective in
this document. The analysis of repeat effects is interesting
scientifically, but the policy implications are not clear. I would
suspect that the person-days above the benchmark is more likely to
inform any final decision. Some clarification on the significance of
this analysis (of multiple effects for an individual) would be useful

I am not convinced by the argument that the daily activity patterns for
asthmatics and non-asthmatics are the same, especially on high ozone
days where an alert has been issued. There is a passing comment about a
study suggesting otherwise, but I don't believe that study examined the
effect of air quality indices being published. I know RTI were doing
such a study at one time (Carol Mansfield was heading it), so perhaps
the staff might find the results of that study and see if the
conclusions are changed.

I end with a comment I have made in almost all similar cases reviewed by
CASAC. The modelling performed here is impressive and represents
state-of-the-science. But I worry that it may be too elaborate for the
purposes of establishing a NAAQS. There are many, many assumptions built
into the assessment, from air modelling to activity patterns. And all of
these rest on spatial resolution of exposure fields and receptor
locations. The science is great, but I feel the current analysis is
pushing the resolution of the results just a bit. Some comments in the
document mentioning this issue, and why the results still are useful for
a NAAQS determination, would be good.

[]{#__RefHeading___Toc212907969 .anchor}[Comments from Dr.H. Christopher
Frey]{.underline}

The following guidance document should be referenced and cited in
Chapter 8: \"GUIDANCE DOCUMENT ON CHARACTERIZING AND COMMUNICATING
UNCERTAINTY IN

EXPOSURE ASSESSMENT\"

http://www.who.int/ipcs/methods/harmonization/areas/exposure/en/index.html

Also, , the assumption of 10% NO2 versus total NO for source strength
may be conservative for existing mobile and stationary sources. However,
for diesel vehicles equipped with particle traps and other
postcombustion controls, there is evidence that the NO~2~/NO~x~ ratio is
higher than this. As such vehicles penetrate the onroad fleet, this may
have some effect on the overall ration of NO~2~/NO~x~.

I recommend that EPA consider that one of the useful outputs of this
study will be a prioritized list of key uncertainties, as discussed
today. Such a list could be developed from qualitative weight of
evidence approaches, and communicated in broad categories (e.g., low,
medium, high uncertainty) or could be developed quantitatively
(preferred where possible). One benefit of this information is that it
could help prioritize data collection or research to improve the state
of knowledge for future reviews and revisions of the NO~2~ NAAQS.

However, my concern is that we don\'t want to imply that what can be
done in a short amount of time under a court-order is what SHOULD be
done for the best science. The qualitative approach is a first step, but
not (in my view) sufficient. Along these lines, EPA should acknowledge
any unresolved issues, possible uncertainties that are not fully

characterized, and better methodologies for dealing with these issues,
in order to guide future work.

[]{#__RefHeading___Toc212907970 .anchor}[Comments from Dr. Terry
Gordon]{.underline}

1\. To what extent is the assessment, interpretation, and presentation
of the results of

the exposure analysis technically sound, clearly communicated, and
appropriately

characterized?

Although my expertise does not allow me to evaluate whether the exposure
analysis is technically sound, the analysis was clearly communicated and
characterized. Explanations as to why certain decisions were made for
the chosen methods were particularly clear. One minor communication
suggestion would be to expand the figure legends and labeling for the
figures.

2\. The draft risk and exposure assessment document evaluates exposures
in Atlanta.

What are the views of the Panel on the approach taken and on the
interpretation of

the results of this analysis?

The approach and interpretation were very good, but there still exists
the potential problem or bias associated with using an analysis of one
city in the Southeast (and Philidelphia) to encompass a risk analysis
for the entire country. In particular, it appears that 12 urban centers
were used to estimate population exposures for ozone (stated in Section
8.2, page 1, line 26). Perhaps this will be better addressed in the
final REA chapter.

3\. What are the views of the Panel regarding the adequacy of the
assessment of

uncertainty and variability with respect to characterization of
exposures and

health risks associated with those exposures?

The assessment of uncertainty and variability factors is very clear
although a table delineating the direction of bias that each factor
might introduce would be a good addition either in Chapter 8 or 9 (where
I believe there may already be one).

4\. To what extent is the assessment, interpretation, and presentation
of health risk

characterization included in Chapter 8 technically sound, clearly
communicated,

and appropriately characterized?

EPA did a great job under tight time constraints to finish this health
risk characterization. It is technically sound and well communicated and
characterized, although a conclusion section summarizing the relevance
of the characterized exceedances/exposed asthmatics would solidify this
chapter (rather than waiting until the final chapter).

Minor comments:

1.  Table 8-1 -- It would help the reader to define Fringe, Rural,
    Arterial, Freeway, etc. below the table.

2.  Table 8-5 -- Again, please define FIPS, Major and Minor Links, and %
    Minor.

3.  page 15, line 9 -- Define NEI here and in Table 8-6.

4.  page 26, line 8 -- typo = lcocated.

5.  page 27, Table 8-7 -- Define p0 to p100. Including 'N' as the first
    row is confusing as it's not really On-Road Hourly NO2 (ppb) or
    Number of hours greater than x00 ppb.

6.  page 29, line 2 -- Given the use of 4 central counties, is it
    correct to look at the total population of Atlanta?

7.  page 34, line 10 -- typo = 'rate and the cooking...'.

8.  page 68 -- Should there be a discussion of the uncertainty in
    applying 1 city (or 2) to the rest of the U.S.?

    []{#__RefHeading___Toc212907971 .anchor}[Comments from Dr. Dale
    Hattis]{.underline}

**Comments on the Treatment of Uncertainty and Variability in the
Revised Chapter 8 Exposure Analysis**

**Dale Hattis, Clark University**

Over the course of reviewing the APEX and air quality modeling for past
meetings, I and others have offered quite a few constructive suggestions
for improving the modeling, the distributional assumptions (on such
parameters as indoor air elimination rates and on road/off-road
adjustment factors) and other issues. These include the following
comments on the first draft REA reviewed in the May 2008 meeting:

> ["Air exchange distributions contingent on temperature and presence or
> absence of air conditioning.]{.underline} Overall the panel does not
> have any objection to the idea of using lognormal distributions with
> very broad limits (.1 and 10 air changes/hr). However the detailed
> results seem to show different patterns with temperature arbitrarily
> blocked into a few ranges. There does not appear to be any great
> consistency or overall theory for this analysis. A better description
> of the data as a whole might be produced by a more extensive
> regression study using temperature or some transform of temperature as
> a continuous variable and either fixed-effect or mixed effects
> modeling of differences among cities and for the air conditioner
> presence variable.
>
> [NO2 removal rate distribution\--p. 101]{.underline}. At least one
> panelist expressed an objection to the narrow fixed limits used for
> the removal rate distribution based on six values from Spicer et al
> (1993). The abstract to the Spicer paper makes it clear that all six
> observations were made in a single house, and that there are
> additional complications from the presence of HONO, an apparently
> longer-lived NOx species:
>
> p\. 101---The same panelist also objected to the fixed limits used for
> the removal rate distribution based on six values from Spicer et al
> (1993). The abstract reads.
>
> Transformations, lifetimes, and sources of NO2, HONO, and HNO3 in
> indoor environments.
>
> Spicer CW, Kenny DV, Ward GF, Billick IH.
>
> Air Waste. 1993 Nov;43(11):1479-85.
>
> Battelle, Columbus, OH 43201-2693.
>
> Recent research has demonstrated that nitrogen oxides are transformed
> to nitrogen acids in indoor environments, and that significant
> concentrations of nitrous acid are present in indoor air. The purpose
> of the study reported in this paper has been to investigate the
> sources, chemical transformations and lifetimes of nitrogen oxides and
> nitrogen acids under the conditions existing in buildings. An
> unoccupied single family residence was instrumented for monitoring of
> NO, NO2, NOy, HONO, HNO3, CO, temperature, relative humidity, and air
> exchange rate. For some experiments, NO2 and HONO were injected into
> the house to determine their removal rates and lifetimes. Other
> experiments investigated the emissions and transformations of nitrogen
> species from unvented natural gas appliances. We determined that HONO
> is formed by both direct emissions from combustion processes and
> reaction of NO2 with surfaces present indoors. Equilibrium
> considerations influence the relative contributions of these two
> sources to the indoor burden of HONO. We determined that the lifetimes
> of trace nitrogen species varied in the order NO approximately HONO \>
> NO2 \> HNO3. The lifetimes with respect to reactive processes are on
> the order of hours for NO and HONO, about an hour for NO2, and 30
> minutes or less for HNO3. The rapid removal of NO2 and long lifetime
> of HONO suggest that HONO may represent a significant fraction of the
> oxidized nitrogen burden in indoor air.
>
> The uniform distribution with its fixed boundaries (0% probability
> assumed for values outside of the defined limits) is particularly
> inappropriate when the data are limited, as in this case. Use of the
> uniform distribution artificially reduces the likelihood of more
> extreme values of the modeled parameter than happen to be present in
> the limited available data. This in turn limits the model-predicted
> variability of NO2 concentrations, which critically determines the
> number of exceedances of the high hourly NO2 levels that are the focus
> of the risk assessment modeling. It would likely be far better to use
> a lognormal here as an initial hypothesis, but in the light of the
> fact that different houses with different internal materials might
> well destroy NO2 at different rates, expert judgment might well be
> needed to expand the likely distribution beyond what can be derived
> from a simple data fit.
>
> The same panelist also strongly objected to the use of uniform
> distribution of concentrations of NO2 from use of gas stoves (p. 101).
> The very breadth of the bounds derived (4 -- 188) ppb argues against a
> uniform distribution and in favor of something more skewed, such as a
> lognormal. The lognormal guarantees a positive contribution, and
> doesn't have the unfortunate property of implying zero chance that the
> indoor contribution will be above the derived maximum. Moreover, if a
> mass balance approach is being used to model indoor NO2, then the
> input per cooking event should be in terms of mass units of NO2, not
> concentration. Concentration will depend on house- and temperature
> specific factors such as air exchange rates, NO2 removal rates and
> residual contributions from HONO, among other things. Because these
> observations were from a single house in California, there must be
> extra allowance for variability and uncertainty in these estimates
> that must clearly extend beyond the mass equivalent of the
> concentration range quoted.
>
> Finally the assumption that all cooking events contributing to indoor
> NO2 last exactly one hour also artificially limits the variability in
> NO2 inputs and therefore exposures represented in the model."

In addition to these suggestions from the last meeting, earlier
suggestions on the planning document for the REA included explicit
modeling to deal with the fact that the air quality monitors are
generally greater than people's breathing zones, leading to a systematic
underestimation of air concentrations to which people are exposed in the
air quality characterizations.

My reading of the current chapter 8 is that the EPA authors have
completely blown off all of these suggestions for revision of the
uncertainty/variability analysis using the APEX modeling tool and the
air quality modeling system. Not only that, the most recent version of
the REA appears to omit sections of the previous exposure analysis draft
that gave the reader a clue that improved quantitative characterization
of the uncertainty and variability was possible. By not acknowledging
these sources of uncertainty, and discussing the previous suggestions
for improvement it seems to me they have not just been neglectful. It is
fine if EPA wants to disagree with my/our previous review of their
analysis on technical grounds. It is also fine if they want to say that
they cannot make the improvements in their analyses that we suggested
because of time and resource constraints. But to completely omit
discussion of the previous critiques and suggestions for improvement of
the analysis is unacceptable and could be interpreted as actively
misleading. At the very minimum, the REA authors need to provide
quantitative documentation and discussion of:

-   The height distributions for the air quality monitors, both
    nationally (as in earlier drafts of the REA and/or ISA appendices)
    and specifically for the case study cities (Philadelphia and
    Atlanta),

-   The 30-odd values of the on-road/off-road ratios that the authors
    used for their empirical distribution of this key parameter,

-   The six values for the indoor NO2 removal rate distribution that
    underlies the calculations of indoor air concentrations.

Documentation of the individual data are needed for these parameters in
order for the committee to adequately review the conclusions of the
final REA and the eventual comparative evaluations of alternative
standards. The "margin of safety" language of the Clean Air Act, it
seems to me, requires the Administrator of EPA to consider in some
detail the degree of confidence that can be achieved in conclusions that
alternative NAAQS adequately protect public health. Therefore, absent a
fair analysis and discussion of the likely quantitative effects of
uncertainty and variability of these parameters on the estimates of
population exposure under different regulatory scenarios, I would have
the committee advise the Administrator that the current document is so
deficient that it cannot be a reasonable basis for informed
decision-making.

[]{#__RefHeading___Toc212907972 .anchor}[Comments from Dr. Rogene
Henderson]{.underline}

Preliminary Comments on Chapter 8 of the NOx REA

Submitted by Rogene Henderson, PhD

October 16, 2008

My comments are given from the viewpoint a non-modeler.

1.The details of the models for exposure were fully described and
discussed. I think it is important to compare the modeling values with
actual measurements to see how well the models reflect reality. This was
described well in Chapter 8. I am concerned that the diurnal comparisons
showed the models overestimated NO2 exposures at the beginning and
ending of the day. Models can be useful when they are wrong because they
tell us that we are not taking something(s) into account that
influences, in this case, the diurnal variation in NO2. Has any thought
been given to what that might be?

2\. Using only one city as the basis for the exposure analysis has the
potential for not being representative of the total population of the
US. However, from a practical viewpoint, limited resources prevent one
from analyzing more cities in depth. I think the document describes the
limitations of the approach well and I agree with it.

3.The limitations of the models for exposure are discussed in great
detail.

4\. I think of \"health risk characterization\" as comparing what we
know about the toxicity or hazard potential of the pollutant with what
we know about exposures to come up with the potential health risk. But
in this chapter I only see the exposure portion of the characterization
discussed. Can you add a discussion of the health effects you might
expect from the described exposures?

[]{#__RefHeading___Toc212907973 .anchor}[Comments from Dr. Timothy
Larson]{.underline}

Comments by Tim Larson on Chapter 8 of the NO~2~ REA

General Comments

EPA is to be commended for the framework outlined in Chapter 8. The
inclusion of deterministic approaches to spatial extrapolation of
measurements from sparse network for purposes of exposure estimation is
a welcome addition. It is an improvement on simple interpolation schemes
that ignore the spatial distribution of important sources.

My main concern with this analysis centers around the assumption that
the census block centroids describe the location of the exposed
population. My reading of the chapter says that only 1% of the exposed
population is assumed to be living within 100 meters of relatively small
roadways (\>15,000 vpd). If this is the underlying assumption in the
final assignment of ambient NO~2~ by AERMOD, then it does not reflect
the reality in most urban areas. In fact, if this assumption is what was
made, the exposed population in the Atlanta assessment could
underestimated that in most urban areas by at least an order of
magnitude. While I agree that the AERMOD model probably overpredicts the
on-road exposures by up to a factor of 2, this is outweighed by the
exposure location assignment bias.

Something should also be mentioned in the uncertainty section regarding
the roll-up methods. There is an apparent correlation between the
average NO2 concentrations at EPA sites and the peak to mean ratio. In
the roll-up method used by EPA, the assumption is that these two
quantities are not correlated and therefore the peak to mean ratio
remains constant during roll-up. The data suggest otherwise. Therefore
the peak values may be overpredicted for a scenario just meeting the
current annual standard. This is important when drawing conclusions
about how well the current standard protects against the exceedance of
peak hourly benchmark values. The analysis presented by Paine in the
September 26 public comments suggests that the EPA approach is very
conservative. There needs to be acknowledgement of this issue and/or a
rebuttal argument.

Specific Comments

P6 line 19 Strictly speaking, AERMOD is a bi-Gaussian model

P 7 line 8 clarify wording

P23 what about smaller NOx point soujrces with short stacks? Is emission
rate the only criterion?

P25 only 1% near roadways?

Fig 8-7 The overprediction of AERMOD could be the effect of ignoring
vehicle induced turbulence as well as including too much fresh ozone.
Comparison of NOx predictions would be useful in assessing the pure
dispersion estimates.

P31 locally high values? Do you mean within 100 meters of the road?

Table 8-8 How do you get asthma prevalence rate for a 0-1 year old?

P53 line 16 should read "spend more time"

Fig 8-16 the in-transit estimates are probably generous given the
assumption of pen=1. Also, given that people spend most time in their
homes, even if the pen=0.5 for homes near roads, this could be
important. In Atlanta, only 1% of the population lives within 50 m of a
major road??

P55 line 2 should read figure 8-19

P61 line 13 in travel exposures may be underestimated due to ignoring
the communting route on-road levels

P63 line 24 a bit optimistic- e.g. compare with line 20 on p64

P64 line 27 the vehicle wake effect is most pronounced for wind
directions parallel to the road, where the wakes of multiple vehicles
can reduced on-road concentrations more than a simple one-time
adjustment factor (e.g. 1.7) applied to all wind directions (including
cross-winds).

P66-67 discussion of between city variation in APEX results for ozone
not necessarily applicable to NO2, given the proximity to road effect
for NO2.

Other comments (including material in Appendix B)

Discuss whether there any visual check on the conflation of TDM links
with the actual street map. Using an algorithm that is based on closest
road to the TDM node will not always put the link on the correct road
(case of nearly parallel roads and also underpass links).

Add a few sentences about the use of segmented lengths- discuss how much
mismatch there is with actual road locations. Could be large for curved
roads and may affect both on-road and off-road estimates from AERMOD.

Initial NO2 to NOx ratio from tailpipe emissions- include a few
sentences about the effect of catalytic converters on newer heavy duty
vehicles.

Add to the discussion of vehicle induced turbulence- initial sigma z
values are not adjusted for wind path length along the roadway, an
effect most pronounced for near parallel wind directions.

Discuss the uncertainties of assuming a superimposed, steady state area
source plume model for aggregated small roadway area sources- if travel
times are greater than one hour, then the steady state assumption breaks
down.

[]{#__RefHeading___Toc212907974 .anchor}[Comments from Dr. Kent
Pinkerton]{.underline}

COMMENTS: Kent E. Pinkerton, University of California, Davis

CHARGE QUESTION #1: To what extent is the assessment, interpretation,
and presentation of the results of the exposure analysis technically
sound, clearly communicated, and appropriately characterized?

REPLY: The authors of chapter 8 have done an incredible job to organize
this chapter. The presentation of information within the chapter is
extremely thorough and carefully described. The methodology and data
used in inhalation exposure assessment and health risk characterization
for NO~2~ as a criteria pollutant is up to date and highly innovative in
its application as presented in Chapter 8. The estimation of temporal
and spatial variation of NO~2~ concentrations appears to be a highly
complicated and complex process, but the authors have done an reasonable
job to provide a sound description and rationale. The authors describe
approaches taken using the Air Pollutants Exposure Model (APEX) to
estimate human population exposure, while AERMOD is used to characterize
ambient air quality. The APEX model simulates exposure using 1)
characterization of the study area, 2) generation of simulated
individuals, 3) construction of a sequence of activities, 4) calculation
of hourly concentrations in mcroenvironments and 5) estimation of
exposure. Each step in Chapter 8 is systematically described with
potential concerns for potential error described. It is my evaluation
that the assessment, interpretation, and presentation of the results of
the exposure analysis in Chapter 8 have been done in a technically
sound, clearly communicated, and appropriately characterized fashion.

CHARGE QUESTION #2: The draft risk and exposure assessment document
evaluates exposures in Atlanta. What are the views of the Panel on the
approach taken and on the interpretation of the results of this
analysis?

REPLY: Atlanta appears to be an ideal location for crafting this risk
and exposure assessment, based on the historical richness for NO~2~
concentrations in this region and the availability of an excellent
database of information to use in creating this model. The inclusion of
four counties covering much of urban and suburban Atlanta seems to
provide a reasonable base to model and to make predictions for risk and
exposure assessment. However, it is critical that the authors fully
justify the selection of a single city or region for making predictions
for the entire nation in the assessment of NO~2~ for the REA for NOx.

CHARGE QUESTION #3: What are the views of the Panel regarding the
adequacy of the assessment of uncertainty and variability with respect
to characterization of exposures and health risks associated with those
exposures?

REPLY: The authors have carefully laid out the parameters used for the
modeling to estimate NO~2~ exposure risk to individuals. The authors
acknowledge the limitations of their model that must use assumptions,
but have applied APEX which from the authors' point of view provides the
most power and flexibility to estimate exposure assessment and risk
characterization. The authors acknowledge CHAD (Consolidated Human
Activity Database) used in their model is likely to produce the greatest
uncertainty. This concern is based on 1) the variability of human
activity patterns and 2) much of the information used is over 20 years
old. The authors have attempted to correct for some of these
uncertainties, but there appears to still remain potential errors and/or
factors that may not be able to be accounted for and/or are unknown.
Population and commuting data, meterological data, and air quality data
are only a few of the many parameters that must be factored into this
model. Based on these parameters as well as indoor and outdoor factors,
air exchange rates and others, it is impressive that the authors can
make any predictions at all with any degree of certainty!

CHARGE QUESTION #4: To what extent is the assessment, interpretation,
and presentation of health risk characterization included in Chapter 8
technically sound, clearly communicated, and appropriately
characterized?

REPLY: The implementation of the exposure modeling and health risk
characterization is very thorough, well documented and reasonably
explained. However, concern remains when multiple assumptions must be
made in a model where minor errors or large variations in values are
possible, can easily be compounded, thus leading to a highly misleading
exposure model or health risk characterization. Again, the authors
simply need to make these limitations of the model clearly evident for
the reader.

[]{#__RefHeading___Toc212907975 .anchor}[Comments from Dr. Armistead
Russell]{.underline}

Review of Chapter 8:

EPA Staff and their contractor are to be complimented on the amount of
work conducted, and how quickly, to address concerns discussed at the
last NOx-primary meeting. While they have reduced some of the largest
differences between the modeled and observed NO2, some remain. Those,
for the most part, can be explained by some of the assumptions made:

1.  10% NO2 in exhaust. This could be lower, and would, in particular,
    lower the early morning NO2 peak. It was chosen to be conservative.
    This is fine, and should be recognized as leading to a possible
    bias, and the issue observed.

2.  The temporal NOx emissions pattern would show less hourly variation
    than is likely, and, in particular, the smoothing used will put more
    emissions before 6 am than is likely the case. I drive here. There
    are few on the road before 6 am, and it is ghostish before 5 am.
    They would have done better to use the hourly temporal distributions
    available in SMOKE. This, alone, might get rid of the aberrant peak
    at 5 am.

3.  They could choose greater initial mixing values, though this would
    be small except for on-road estimates.

On the other hand, I would actually think the way they did the NOx
emissions estimation would be more accurate than the NEI, and the 28% is
too big of an expected change (and would go in the other direction).

There is a major bias in the AERMOD results that can not be readily
dismissed, that being that the on-road/non-road ratio gets much higher
for AERMOD. There is a consistent tendency, in all of the approaches,
for AERMOD to have higher on-road than might be indicated by the
observations. This is discussed, but should be highlighted more,
particularly later on when interpreting the APEX results. As you will
note, most of the exposures to high levels come from on- and near-road
activities.

The application of APEX appears solid, though the evaluation and
interpretation is flawed. While I am glad to see the evaluation
performed, it is misleading. In particular, Fig. 8-11 should show
maximums, not P97.5 for comparison with Fig. 8-10. Indeed, I suspect we
will see that the maximums predicted by APEX (which drive the later
analysis) are much higher than observed. I would actually like to see a
figure where they bin the APEX results in deciles (e.g., cumulative %ile
deciles), and plot the min, median, 97.5%ile and max. They also might
estimate the 97.5%ile of the observations from Suh et al., for
comparison. Call Helen.

The reason all of the above is important (at least to me) is that we are
compounding some apparently large biases at the upper end of the
distribution to the point that I think we are well overestimating the
population that is being exposed to greater than the benchmarks chosen.
Remember, there has been one observation of over 200 ppb of NO2 over the
three year period. What was the second high? Does that support the
distribution found?

In the end, I am uncomfortable with the compounding of biases for
factors leading to the upper tail of the NO2 exposure distribution.
While they may have to work with this, they should be extremely cautious
about how the results are communicated.

[]{#__RefHeading___Toc212907976 .anchor}[Comments from Dr. Jonathan
Samet]{.underline}

Comments: Draft Chapter 8 of EPA's *Risk and Exposure Assessment to
Support the Review of the NO~2~ Primary National Ambient Air Quality
Standard*

General Comments:

The draft Chapter 8 on Exposure Assessment and Health Risk
Characterization is improved and now complete. The Staff made
substantial progress in developing the chapter and it will prove useful
as the basis for health risk characterization. As far as major concerns,
I offer the following:

-   The document is detailed and lengthy, and does not offer a
    sufficiently informative overview of the approach that would be
    taken. It does have an introductory section, but the description is
    inadequate. I recommend the incorporation of appropriate diagrams
    that would lay out the various models that would be used and the
    data inputs for the modeling. This diagram would be useful for
    describing what has been done and also for describing the
    uncertainties that arise.

-   At a number of points, judgments are made with regard to what
    evidence shows and what determinations can be made. However, I find
    that the criteria for these determinations are lacking. For example,
    on page 27, lines 7-9, the conclusion is reached that adjustment of
    model concentrations to the ambient monitors was not necessary. I
    cannot find a clear rationale. Similarly, on page 24, lines 1-3,
    "general agreement" is said to be found between observed and modeled
    values. Again, what is "general agreement?"

    []{#__RefHeading___Toc212907977 .anchor}[Comments from Dr. Richard
    Schlesinger]{.underline}

I do not have expertise in the area of exposure assessment. However,
that said, the revised chapter did present material in a coherent and
understandable manner. As I have noted in prior types of analyses such
as these, I am concerned about having one site become the model site for
all of the analyses. Thus, there should be a more comprehensive
justification for use of this one site.

[]{#__RefHeading___Toc212907978 .anchor}[Comments from Dr. Christian
Seigneur]{.underline}

Comments on Chapter 8 of the 2^nd^ draft of the REA

Christian Seigneur

CEREA, Université Paris-Est

OAQPS has accomplished a large amount of work in a short time. Their
overall methodology to calculate exposure and risk for Atlanta seems
sound. However, I have some questions regarding the results of the air
quality modeling.

Section 8.4.2.4: It seems that the ozone limitation techniques were
applied to AERMOD by source category ((mobile sources, airport, etc.).
Thus, if two source categories were to impact significantly a same
location, would it be possible to double-count the ozone available for
oxidizing NO to NO2?

Section 8.4.8: The report presents a thorough comparison of modeled and
measured NO2 concentrations. Both the cumulative distributions and the
diurnal profiles show that AERMOD tends to overestimate the measured
concentrations. Such results suggest that some adjustment of the modeled
values is warranted to better reflect the actual concentrations.
However, the document states that "it was determined that adjustment of
the modeled air quality based on the three monitors was not necessary".
Such a statement could apply if the model showed error with no or
negligible bias. However, the model shows a systematic bias
(overestimation) for concentrations above the 60^th^ percentile of the
observed values (i.e., those concentrations of most interest for health
effects). Therefore, it seems that some adjustment is warranted to scale
the modeled values down toward the values observed at the monitors.

Section 8.10.4: The discussion of the uncertainties associated with the
air quality values does not highlight the significant bias that was
shown in the AERMOD evaluation section.

[]{#__RefHeading___Toc212907979 .anchor}[Comments from Dr. Elizabeth
"Lianne" Sheppard]{.underline}

*General comments*:

I wish to complement EPA staff for completing so much work in a short
time. The APEX model for NO2 is complex and requires more complex inputs
than some previous APEX applications, particularly given the 1-hour
averaging time and the need to capture strong spatial gradients in the
ambient exposure. Adequate spatio-temporal modeling of ambient
pollutants is a new focus area for the ERA, and a strong effort has been
made to apply available tools appropriately. While further improvements
to the modeling and its assessment are desirable (and possibly
necessary), significant revisions since the first draft REA have
resulted in 1) better predictions and 2) a model assessment that is much
better aligned with the modeling goals.

*Important issues that need additional attention before the REA is
finalized*:

-   [Philadelphia vs. Atlanta]{.underline}. Make sure to bring out in
    several places in the report (intro to Appendix B (B-1), Appendix B
    case study descriptions (B-3 and B-4), intro to Chapter 8, and as
    added comments in conjunction with text references to the
    Philadelphia analysis (e.g. p 4, line 21)) the fact that these two
    modeling efforts were sequential and the improvements in Atlanta
    were not applied to Philadelphia. While it is reasonable to include
    the Philadelphia analysis in the appendix, the presence of the two
    will mean some parties will make an effort to compare the results.
    Since the modeling efforts were significantly different, this is not
    appropriate. Readers need to be told this.

-   [AERMOD application]{.underline}:

    -   [On-road model assumptions]{.underline}: The strong discrepancy
        between the data and the model predictions plotted in Figure 8-7
        suggest much more attention is needed to this model feature,
        particularly since it is an important driver of exceedances. Do
        the existing published data provide an adequate foundation for
        this direct comparison? Is the comparison of ratios of AERMOD
        receptors fair (since there may be a large amount of variability
        in locations of AERMOD receptors relative to the empirical
        studies)?

    -   [Census block centroids and the consequent impact on near-road
        exposures]{.underline}: The assumption that the Atlanta
        population resides at census block centroids must be evaluated
        more carefully since this could have a huge impact on estimation
        of exceedances. Exposure to high pollution falls off rapidly in
        the first 100 m from a road. The data show 1% of the simulated
        population (located at block centriods) is within 50m of a road.
        Since census block boundaries may not cross busy roads, the
        fraction of population living near busy roads as defined by
        census block centroids is most likely underestimated. For
        comparison, here are data from the 6 cities that are a part of
        the MESA air study. The percentages of 6,014 MESA Air study
        subjects living within 50 meters of a busy road (census class
        A1-A3) are approximately 16% (Winston-Salem), 18% (Baltimore,
        Los Angeles, Minneapolis-St Paul), 26% (Chicago), and 48% (New
        York City).

-   [Discussion and interpretation]{.underline}: Additional text is
    needed to discuss the results, particularly in terms of the large
    fraction of the simuated population experiencing exceedances and the
    dominant locations/activities where exceedances occur (which appear
    to me to be predominantly driven by near-road and on-road
    exposures). The main locations for exceedances are not only
    important from a policy perspective, but should also guide the
    discussion of uncertainty and variability.

-   [Uncertainty and variability]{.underline}: The key questions to
    address with respect to uncertainty and variability are 1) what
    features are most influential in modeling the tails of the
    distributions, and 2) are there major sources of bias and
    uncertainty in these features. Both aspects need more attention.
    Make sure concerns about on-road and near-road exposures are
    addressed. One approach for structuring a revision may be to list
    every single assumption and model input (e.g. receptor height of
    0 m) and discuss its possible contribution to bias and uncertainty.
    If more extensive analytical work were to be done in support of this
    REA, better quantification of bias and variability is most likely
    where the bulk of the attention should be focused.

*Specific comments*:

-   Appendix B-4.14: It is good to summarize the receptor location
    distribution. This should be compared to information about the
    Atlanta population residence distribution.

-   p 1 l 7: insert "ambient" before NO2.

-   p 21 l19-21: Fix wording

-   p 25 l 5: Replace "the same seasonal pattern" with "similar seasonal
    relationships".

-   p 25 l 16-19: This statement needs more analysis to support it.

-   Section 8.4.8.3: The value-laden statements in this section should
    be moved to the uncertainty and variability section.

-   p 37 l 11 and others following: Improve notation. Why define b if
    b=1-a? Include indexes, at least for individuals and time, to
    reflect that the predictions are across individuals and hours.

-   p 47, 52: The fraction of the population exposed to exceedances is
    huge and deserves further comment (most likely in the discussion
    section to be added).

-   p 49: The evaluation of microenvironments to reflect locations of
    exceedances is valuable.

    []{#__RefHeading___Toc212907980 .anchor}[Comments from Dr. Frank
    Speizer]{.underline}

Comments on Chapter 8, REA NO2 Document, dated October 2008

Submitted by : Frank E. Speizer, MD

Pre Conference Call submission. 10/15/08

In general I thought this chapter reads rather well. Most of my comments
are minor.

Charge Question 1: The first part of chapter 8 does an excellent job of
describing what was done to make the exposure estimates. The only
question would be whether it would be worth reviewing and expanding on
both the selection of Atlanta and the reasonableness of Atlanta as a
place to derive estimates for the whole country. As I recall some of
this is discussed in previous chapter and there may be no need to repeat
here except to reinforce the selection.

Page 10-11, Tables 8-1 and 8-2: It is not clear how Central Business
District (CBD) differs from Urban. May need an expanded footnote.

Charge Question 2: The logic used in developing the assessment is well
laid out. In table 8.12 it might be worth constructing an addition set
of figures for the 24 hour measure (or 8 hour), since in the end we will
need to advise on one number and it would be good to be able to justify
how that one number differs from or covers the same at risk groupings as
other time periods. At 100ppb there appears to be little difference in
the 98^th^ vs 99^th^ percentiles. Only be dropping the level to 50ppb
does the form appear to have an effect. Without additional calculations,
it would appear that this argues strongly for a margin of safety for
safety sake rather than as an estimate of uncertainty (see below).

Page 54, Figure 8-17: the labels on the figures need to be corrected.
(a-a-a, should be a-b-c).

Charge Question 3: Although there is an excellent description of the
uncertainties that are associated with the various analyses presented,
there appears to be no comment in either direction of the magnitudes of
what these uncertainties do to the estimates provided. It seems clear
that in some cases they would actually reduce the variance of the
estimates and in other cases may increase it. As indicated on the top of
page 66, by analogy with O3 may not be adequate, and therefore some
sensitivity analyses with NO2 need to be performed. By example, the
sensitivity test performed that generally showed low variation helped to
reinforce the findings as presented, and thus reduce the potential
importance of some of the issues mentioned.

Page 68, section 8.10.8 is useful but raises an issue that is generic to
the consideration of Atlanta. This section states that almost 100 %
households are air conditioned. Thus understanding both penetration from
outdoors as well as indoor source contribution, although understood for
this region, may not be adequate for generalizing beyond the Atlanta
region, particularly to regions that are not so saturated by air
conditioning.

Charge Question 4:Although the chapter does a reasonable job in
characterizing the exposure of the at risk population, it seems to fall
short in estimation of outcomes from those exposures. For example it
never goes on to provide estimates of asthma exacerbations, or
hospitalizations based on the estimates of exposure. This is
particularly troublesome in that the lowest estimated exposure result in
substantial numbers of at risk subjects. It would therefore be useful to
indicate what those at risk subject will suffer from, and the degree
that true health impacts will occur from those exposures. I would think
this next step is necessary and look forward to hearing it presented in
the conference call (or seeing a few charts or tables before the
conference call).

Section B-4.2.6. Supplemental Exposure Results

It appears that there is little impact on the exposure results until the
Air Quality Adjustment gets to 50 ppb for the form (98 vs 99 percentile)
to make any difference. This would suggest that estimates based on a
lower bound of 100ppb are too high, and that they should be recalculated
and presented as low as 50 ppb, if not lower.

In reviewing the rest of **[Appendix B]{.underline}** it is clear that
there is a repeat of the material from the first draft on Philadelphia,
but because of the way that analysis was done and the way the current
analysis of Atlanta was changed, they really are not comparable. I
therefore am concerned that these two very different urban centers (eg.
One warm and almost 100% air conditioned, and the other more typical of
urban Northeast) can't be judged side by side and therefore cannot
figure out how comfortable to be with just using Atlanta. Right now it
looks as if the estimates for Phili are done to a level of 200 ppb
whereas for Atlanta they go down to 50 ppb. I would like to see similar
analysis of what would be predicted exposures in Phili at the same
levels seen for example in table B-50 on page B-114. (There may be a
reason why this can't be done\-\--as long as it is not that there is
just not enough time to do it, and if so, so be it.)

[]{#__RefHeading___Toc212907981 .anchor}[Comments from Dr. George
Thurston]{.underline}

Prof. George Thurston's Initial Comments for CASAC on Chpt. 8 of the
Second Draft of:

*Risk and Exposure Assessment to Support the Review of the NO2 Primary
National Ambient Air Quality Standard*

1\. To what extent is the assessment, interpretation, and presentation
of the results of the exposure analysis technically sound, clearly
communicated, and appropriately characterized?

The EPA staff should be congratulated for their arduous effort at a
formidable task to reach the ultimate goal of the chapter: to conduct a
case study of an exposure and health risk characterization for various
alternative NO2 standards. While the approach is basically technically
sound, it involved numerous challenging modeling steps, each with their
own uncertainties and possible biases that are potentially compounded
from step to step. The populations considered and the 3 receptor
classifications considered in these analyses seem very appropriate, and
the modeling applied is state-of-the art. One major concern I have with
regard to the modeling, though, is the continuing overestimation of peak
values, as displayed (for example) in Figures 8-5 and 8-6. I wonder if
it is the fact that the roadway emissions are reportedly modeled here
using the Ozone Limiting Method (as noted on page 8, lines 10-12), an
approach which I believe tends to give higher (more conservative) NO2
level estimates, since it is my understanding that this model assumes
all available ozone is reacted with NO to form NO2, which will not
likely be met at peak times of the day, seemingly consistent with the
results in Figure 8-6. Is this the cause of these overestimates? If so,
this assumption is fine for screening analysis, but may not be
satisfactory for this application.

2\. The draft risk and exposure assessment document evaluates exposures
in Atlanta. What are the views of the Panel on the approach taken and on
the interpretation of the results of this analysis?

The choice of Atlanta seems appropriate, given the existing pollution
conditions and availability of modeling input information, but more
discussion as to how these results might vary in other cities and
regions would seem appropriate. Section 8.10.7.1 needs much further
elaboration (e.g., sensitivity analysis of potentially varying
conditions such as %AC, and discussion of the potential health
effects/policy implications of this variability).

3\. What are the views of the Panel regarding the adequacy of the
assessment of uncertainty and variability with respect to
characterization of exposures and health risks associated with those
exposures?

To me, this is the greatest weakness of the chapter as it is now
written: no quantitative or qualitative propagation of errors by the
sequential modeling steps is provided in Section 8.10. It is my belief
that such a comprehensive uncertainty analysis would indicate large
uncertainties in this approach to achieve the final estimates. A
consideration of these uncertainties would, I think, indicate that,
while a useful exercise, such a human exposure-health effects analysis
is not sufficient to alone assess the health implications of alternative
standards, and that a consideration of the results of an application of
the NO~2~-health effects epidemiology are also required in each such REA
(as *is* the case for this particular document) to obtain a fuller
appreciation of the health implications of various alternative
standards.

4\. To what extent is the assessment, interpretation, and presentation
of health risk characterization included in Chapter 8 technically sound,
clearly communicated, and appropriately characterized?

*Very little is provided in the way of an interpretation of the results
(e.g., putting the numbers into a context useful for regulatory
assessment). I suggest an additional Section 8.11 to provide just such
information.*

[]{#__RefHeading___Toc212907982 .anchor}[Comments from Dr. James
Ultman]{.underline}

**Comments on Second Draft NOx REA, Chapter 8**

**James Ultman**

[General Comments]{.underline}

I commend staff on completing an extensive body of work in such a short
time. The document is clearly written. It conveys the methodology
(including the description of computer models, sources of input data and
assumptions) in as detailed a manner as is possible within the confines
of a single chapter. The results are also summarized in a logical
fashion.

I do, however, feel somewhat overwhelmed by the complexity that emerges
from the layering of different models, input data sets and accompanying
assumptions that I feel are not always justified or validated. Moreover,
lacking an integrated summary at the end of the chapter, it is not clear
what key conclusions should be drawn.

[Specific Comments]{.underline}

I am skeptical that rolling back the health benchmark values in the
exposure analysis is equivalent to rolling up of ambient concentrations
(and visa versa). The document does not present clear evidence---either
numerical or mathematical---that this is the case. Moreover, the
equation that is used to justify a parallel roll-back in indoor source
concentration (pg 38, line 5) fails to recognize that C~indoor~ is
determined, in part, by C~ambient~. In particular, if C~ambient~ is
rolled up by a factor f (pg. 38, line 3), C~indoor~ must also increase.
Thus, its original value cannot be used to compute C~thresh~ (see
equation on pg. 38, line 3). Moreover, the relationship between
C~indoor~ and C~ambient~ is algebraically non-linear when the mass
balance model is used to determine C~indoor~.

Given the importance of on-road exposures, I am also concerned with the
discrepancy between the AERMOD-based and data-based determination of the
on-road/non-road concentration ratio. On the other hand, this is not
surprising given that the data was measured in a number of locales where
meteorology and emission patterns are different than in Atlanta.

Finally, I believe that the document provides an extensive discussion of
areas of uncertainty, but it does not attempt to quantify the
uncertainty. In retrospect, I feel that it would have been wiser to
spend less time in the modeling/simulation effort and more time in
analyzing those uncertainties that are judged to be the most important.

[^1]: See, for example, U.S. EPA. *Limitations in the 1996
    National-Scale Air Toxics Assessment* 1996 \[cited 10/23/08.
    Available from http://www.epa.gov/ttn/atw/nata/natsalim2.html\] and
    World Health Organization International Programme on Chemical
    Safety. *Part 1: guidance document on characterizing and
    Communicating uncertainty in exposure assessment. World Health
    Organization* In Press. \[cited 10/23/08. Available from
    http://www.who.int/ipcs/methods/harmonization/areas/uncertainty%20.pdf\].