UNITED STATES ENVIRONMENTAL PROTECTION AGENCY

WASHINGTON D.C. 20460

OFFICE OF THE ADMINISTRATOR

SCIENCE ADVISORY BOARD April 15, 2010

EPA-CASAC-10-008

The Honorable Lisa P. Jackson

Administrator U.S. Environmental Protection Agency 1200 Pennsylvania Avenue, N.W. Washington, D.C. 20460

Subject: CASAC Review of Quantitative Health Risk Assessme nt for Particulate Matter – Second External Review Draft (February 2010)

Dear Administrator Jackson,

The Clean Air Scientific Advisory Comm ittee (CASAC) Particulate Matter (PM) Review Panel met on March 10–11, 2010 to review the Quantitative Health Risk Assessment for Particulate Matter – Second External Review Draft (February 2010) and Particulate Matter Urban-Focused Visibility Assessment – Second External Review Draft (January 2010). This letter provid es our main comments concerning the Quantitative Health Risk Assessment for Particulate Matter (RA). CASAC’s responses to the charge questions, along with comments from i ndividual Panel members follow.

The Second Draft RA presents quantitative assessments of PM-associated risks to support judgments that will be made in the Policy Assessment (PA) with regard to

recommendations for the PM National Ambi ent Air Quality Standard (NAAQS). The RA provides estimates of the degree and nature of risk reductions asso ciated with various population exposure scenarios, corresponding to alternative suites of annual and 24-hour

PM 2.5 standards for the 15 selected urban case study areas. Toward that end, the Second Draft has largely accomplished its objective of providing input to the PA. The Second Draft has been substantially improved in response to comments provided by CASAC following our review of the first draft. The rationale, methods and results for the

assessment are well described. The discussi on and clarifications regarding uncertainty and variability and the characte rization of confidence in the co re risk estimates markedly

enhance the document. The enhanced discussi on of the interplay of annual and 24-hour design values together with patterns in PM 2.5 monitoring helps to interpret patterns of risk reduction in study areas. With some addi tional changes recommended here, the RA will

be a solid foundation for the PA.

While the Second Draft has appropriately justified EPA’s decision not to estimate risks for thoracic coarse PM (PM 10-2.5 ) based on limitations in th e health effects data and

the PM 10-2.5 monitoring network, we encourage EPA to remain focused on pursuing the

1research needed for a quantitative risk assessm ent of thoracic coarse PM. As mentioned in our letter of November 24, 2009 reviewing the first draft RA, we also encourage the Agency to develop the capacity to conduct a formal quantitative exposure assessment for future revisions of the NAAQS.

As part of simulating air quality stan dards in the chapter on Urban Case Study Analysis Methods (Chapter 3), EPA uses thr ee different approaches for simulating just meeting the current and alternative suites of PM 2.5 standards: the proportional approach, the hybrid approach, and th e peak-shaving approach. CASAC recommends sharpening the descriptions of these three alternatives. We suggest a graphical and/or mathematical approach, along with the inclusion of exampl es based on the 15 urban study areas. EPA might also consider discussing how these th ree approaches relate to potential control scenarios. The hybrid approach merits the greatest emphasis, while the proportional and peak-shaving approaches represent b ounding scenarios. As a suggestion on nomenclature, an alternative label for the "p eak shaving" approach would be preferable, since this term does not adequa tely describe the actual method.

The Second Draft RA includes a new chap ter which provides a national scale assessment of long-term mortality related to PM 2.5 exposure (Chapter 5). In addition to providing a national estimate, th is chapter identifies where the urban case study areas fall along the distribution of nationa l risk. While this chapter is useful for assessing the generalizability of the findings in the 15 urban study areas, its findings are otherwise not

central to the risk assessme nt. Consequently, CASAC recommends that the chapter be moved to an appendix and that discussion of the key findings, summarized in Figure 5-4, be placed at the appropriate points in Chapte r 4 (Urban Case Study Results) and in the current Chapter 6 (Integrative Discussi on of Urban Case Study Analysis of PM 2.5 -related Risks).

The Second Draft estimates risk reducti ons for different air quality scenarios involving specified valu es of 24-hr and annual standards. Five alternative sets of standards are considered, with the lowest scenario being an annual standard of 12 μg/m 3
combined with a 24-hour standard of 25 μg/m 3 . Two additional scenarios were presented at the March 2010 CASAC meeting with pairings of 10/35 and 10/25 μg/m 3 . The reduction of the annual standard to10 μg/m 3 showed additional benefits beyond those

estimated for the scenarios in the RA. Howeve r, it was not clear whether these additional scenarios represented potential revisions to the NAAQS and will be added to the final RA. With the assumption of a linear, no-thres hold risk model, further reductions would be anticipated as the annual standard is pr ogressively lowered. We recommend that EPA develop and apply specific criteria for determining the lower-bound exposure

concentrations to be considered in the risk assessment. Mounting uncertainty at lower

concentrations would be one such reasonable basis. Other relevant considerations include the range of concentrations at wh ich the epidemiological studies have been

carried out and the need for consideration of the degree of protection afforded to susceptible populations under various scenarios. The interpretation of the findings should also acknowledge the controlling influence of the annual average concentration as the proposed scenario values are progressively lowered.

2On a related matter, CASAC recommends greater clarity as to the basis for selecting the exposure scenarios included in th e RA. Are the potential suites of standards or exposure concentration scenarios inte nded to guide selection among alternative revisions to the NAAQS? As the new ove rall approach to NAAQS revision is implemented, CASAC and the EPA need to address the extent to which the RA and the

PA should be linked.

In response to CASAC’s previous advice, EPA provided an integrated discussion of risk-related analyses that draws on uncertainty and variab ility analyses as well as the national-scale analyses completed as part of this assessment. This integrated discussion is presented in Chapter 6, arguably the most critical chapter of the document. EPA has captured the key policy-relevant questions and appropriately characterized the uncertainty and variability associated with its core risk estimates. While we are in general agreement with EPA’s interpretations of urban study area re sults as well as its observations on the degree of risk reduction likel y to be associated with alternative suites of standards, given its importance, we r ecommend a careful rewr iting and editing of Chapter 6 to ensure that its findings are clearly presente d and that unneeded detail is

removed.

CASAC thanks the Agency for the oppor tunity to provide advice on the Risk

Assessment. We look forward to continued di scussions with the Agency as we review

the Policy Assessment for the Review of the PM NAAQS - First External Review Draft (March 2010).

Sincerely,

/Signed/

Dr. Jonathan M. Samet

Chair Clean Air Scientific Advisory Committee

Enclosures A: CASA C Particulate Matter Review Panel Roster

B: CASAC Responses to EP A’s Charge Questions

C: Individual Panelist s’ Responses to Charge Questions

3NOTICE

This report has been written as part of the activities of the EPA's Clean Air Scientific Advisory Committee (CASAC), a federal advi sory committee independently chartered to provide extramural scientific information and advice to the Administrator and other officials of the EPA. CASAC provides balanced , expert assessment of scientific matters related to issues and problems facing the Ag ency. This report has not been reviewed for approval by the Agency and, hence, the cont ents of this report do not necessarily represent the views and policies of the EPA, nor of other ag encies within the Executive Branch of the federal government. In addition, any mention of trade names or commercial products does not constitute a recommendation for use. CASAC reports are posted on the EPA Web site at: http://www.epa.gov/casac.

4Enclosure A Clean Air Scientific Advisory Committee

Particulate Matter Review Panel

CHAIR Dr. Jonathan M. Samet, Professor and Chai r, Department of Pr eventive Medicine, University of Southern California, Los Angeles, CA

CASAC MEMBERS Dr. Joseph Brain, Philip Drinker Professor of Environmental Physiology, Department of Environmental Health, Harvard School of Public Health, Harvard University, Boston,

MA

Dr. H. Christopher Frey, Profe ssor, Department of Civil, Construction and Environmental Engineering, College of Engineering, No rth Carolina State University, Raleigh, NC

Dr. Donna Kenski, Data Analys is Director, Lake Michigan Air Directors Consortium,
Rosemont, IL

Dr. Armistead (Ted) Russell, Professor, Department of Civil and Environmental Engineering, Georgia Institute of Technology, Atlanta, GA

Dr. Helen Suh, Associate Professor, Harv ard School of Public Health, Harvard University, Boston, MA

Dr. Kathy Weathers, Senior Scientist, Cary Institute of Ecosystem Studies, Millbrook,

NY

CONSULTANTS Dr. Lowell Ashbaugh, Associate Research Ecologi st, Crocker Nuclear Lab, University of California, Davis, Davis, CA

Prof. Ed Avol, Professor, Prev entive Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA

Dr. Wayne Cascio, Professor, Medicine, Cardiology, Brody Sc hool of Medicine at East Carolina University, Greenville, NC

Dr. David Grantz, Director, Botany and Pl ant Sciences and Air Pollution Research Center, Riverside Campus and Kearney Agricu ltural Center, Univer sity of California, Parlier, CA

Dr. Joseph Helble, Dean and Professor, Thayer School of Engineering, Dartmouth College, Hanover, NH

5Dr. Rogene Henderson, Senior Scientist Em eritus, Lovelace Respiratory Research Institute, Albuquerque, NM

Dr. Philip Hopke, Bayard D. Clarkson Distingu ished Professor, Department of Chemical Engineering, Clarkson University, Potsdam, NY

Dr. Morton Lippmann, Professor, Nelson Inst itute of Environmental Medicine, New York University School of Medicine, Tuxedo, NY

Dr. William Malm, Research Physicist, Natio nal Park Service Ai r Resources Division, Cooperative Institute for Research in the At mosphere, Colorado State University, Fort Collins, CO

Mr. Charles Thomas (Tom) Moore, Jr., Air Quality Program Manager, Western Governors' Association, Coope rative Institute for Research in the Atmosphere, Colorado State University, Fort Collins, CO

Dr. Robert F. Phalen, Professor, Departme nt of Community & Environmental Medicine; Director, Air Pollution Health Effects La boratory; Professor of Occupational & Environmental Health, Center for Occupa tion & Environment Health, College of Medicine, University of Ca lifornia Irvine, Irvine, CA

Dr. Kent Pinkerton, Professor, Regents of the Un iversity of California, Center for Health and the Environment, University of California, Davis, CA

Mr. Richard L. Poirot, Environmental Analyst, Air Pollution Control Division, Department of Environmental Conservati on, Vermont Agency of Natural Resources, Waterbury, VT

Dr. Frank Speizer, Edward Kass Professor of Medicine, Channing Laboratory, Harvard Medical School, Boston, MA

Dr. Sverre Vedal, Professor, Department of Environmental and Occupational Health Sciences, School of Public Health and Comm unity Medicine, University of Washington, Seattle, WA

SCIENCE ADVISORY BOARD STAFF Dr. Holly Stallworth, Designated Federal O fficer, EPA Science Advisory Board Staff

Office

6Enclosure B

CASAC Responses to EPA’s Charge Questions

Chapter 3 – Urban Case Study Analysis Methods

Charge Question 1: Air quality inputs (section 3.2): We have expanded the consideration of alternative approaches to simulating just meeting the current and

alternative suites of PM 2.5 standards (i.e., rollback a pproaches) to include a peak shaving approach, in addition to the hybrid and proportional approaches

considered in the first draft assessment. This peak shaving approach is intended to represent more localized, rather than regional, patterns of PM 2.5 reductions (discussed in section 3.2.3.3).

a) To what extent does the Panel believe that the use of the peak shaving approach provides useful additional exploration of variability associated with how ambient

PM 2.5 concentrations are simulated to ch ange upon just meeting the current and

alternative suites of standards?

As noted in our overall comments, CASA C recommends that the "peak shaving" approach be more clearly defi ned in a graphical and/or mathematical fashion. We also recommend that it be renamed, gi ven the potential to misinterpret its actual application. The “peak shaving” approach does provide usef ul information, but specific examples are needed as to the situation(s) and potential c ontrol approach(es) it is meant to simulate. Appendix B should also provide the specific mathematical formulation. It would be instructive to provide an example of the three rollback approaches as applied to data for several cities in section 3.2, or in Appendix B.

b) We have used comparisons of comp osite monitor annual averages generated using the different rollback approaches as a surrogate for diffe rences in long-term exposure-related mortality in looking acro ss all three rollback approaches. To what

extent does the Panel believe that this is a reasonable approach for assessing the impact of variability associated with simulating changes in air quality patterns on estimates of long-term exposure-related mortality?

A very brief description of how annual design values are cu rrently calculated should be provided to help motivate the procedure developed here. CA SAC is uncomfortable with the approach used for imputing missing valu es, and has discussed alternatives in the

members' individual comments. Whatever met hod is chosen, the impact of the approach taken can be characterized using data from monitors where there is little missing data, then removing some data, applying the imput ation approach, and testing to see how closely the method reproduces the original annual average. The approaches to compositing and imputing missing values would be more easily followed if equations were provided.

7Charge Question 2: Selection of model inputs (section 3.3): We have expanded and

clarified the discussion of our rationale for identifying modeling choices comprising the core risk model, focusing in particul ar on selection of C-R functions (section 3.3.3). To what extent does the Panel consider this discussion to be clear and the model selections appropriate?

CASAC commends the authors for expanding an d clarifying the rati onale for identifying modeling choices comprising the core risk m odel in a logical and satisfactory manner. Their model selections were appropriate for this review cycle, which is focused on PM 2.5 exposure and cardiovascular responses. The st ated rationale provides a solid foundation for the selection of the epidemiological studi es that were utilized to establish C-R functions. The expansion of the discussion and integration of the ISA was useful in reinforcing gaps in knowledge, as in the text on p. 3-20 stating that there were no multicity studies for the category of short-term exposure to PM 2.5 and emergency department visits for cardiovascular and/or respiratory illnesses. The summary tables (Tables 3-5 through 3-8) provide a useful synopsis of the model inputs for the co re risk models and sensitivity analyses.

Charge Question 3: Addressing uncertaint y and variability (section 3.5): We have clarified the process used to evaluate sou rces of variability and added coverage for specific sources of variability (section 3.5.2); expanded our discussion of the qualitative analysis of uncertainty (section 3.5.3); and included analyses of pair-wise

interactions of sources of uncertainty (section 3.5.4). To what extent does the Panel consider these discussions to be clear and appropriate?

In general, the second draft RA appropriate ly identifies and discusses key sources of variability and uncertainty, and includes sensitivity analyses that provide insight regarding the impact of some sources of uncertainty on the core risk estimates. The authors have provided a footnote explaining th e rationale for identifying “key” sources of variability. The document should indicate if the same process was used to identify “key” sources of uncertainty. We note that EP A added material regarding co-pollutant concentrations and demographic and socio economic status, as requested in CASAC

comments on the first draft of the RA.

EPA did not address CASAC's recommendation on the first draft RA suggesting that “exposure modeling should be included in th e REA. A probabilistic Tier 3 approach should be used for the exposure assessment.” While we understand that timing may have precluded adequate treatment of this topic, we urge EPA to develop the capacity to do so for future revisions of the NAAQS. CASAC made the same recommendation at the time

of the last review of the PM NAAQS. With regard to uncertainties, in response to

CASAC comments on the first draft of the RE A, EPA has included uncertainty in the CR function itself, which was developed from si ngle studies. EPA has appropriately taken

into account differences in C-R functional fo rm associated with studies that addressed long-term or short-term effects for single or multi-city studies even if they were not the

basis for the final set of C-R functions used in the RA.

8The RA now provides adequate explanation of the uncertainty categories of “low”, “medium”, and “high.” However, there is a c onfusing statement to th e effect that “high” sources of uncertainty “are li kely to influence the interp retation of risk...” “if those sources of uncertainty are reduced or more fu lly characterized.” The parenthetical “if”

clause seems to confuse the i ssue and should be deleted. Th e RA needs to describe the process by which "staff consensus" was achie ved (see lines 6-20 of p. 3-63). If staff consensus is an element of decision-making in carrying out the RA, then this aspect of the process needs to be described.

EPA has adequately commented on the extent to which there are dependencies among pairwise combinations of s ources of uncertainty, and whet her these dependencies would

tend to offset or to increase the overall range and direction of uncertainty in the assessment results. For example, the statistica l fit of the C-R functions, and the shape of the functions, are inter-related.

Based on quantifiable sensitivity analysis, the report generally clearly conveys that the “core” estimates appear to be at the low e nd of alternative “plausible” estimates. However, particularly in Chapter 6, the role of sources of uncertainty treated qualitatively

should also be addressed. In particular, given exposure misc lassification, it is likely that the core estimates are biased low and hence conservative, an importa nt point to convey consistently.

Chapter 4 – Urban Case Study Results

Charge Question 4: Sensitivity analysis re sults (section 4.3): We have included a

discussion of how the results of the sensitiv ity analysis can be used as an additional

set of reasonable risk estimates to info rm consideration of uncertainty in the core risk estimates (see section 4.3.2). What are the Panel’s views on how we have used the sensitivity analysis results to support consideration of uncertainty in the core

risk estimates?

Overall, the sensitivity analysis section 4.3 is very good and nicely covers a complex topic. Table 4-3 is a useful summary. Th e classification of descriptive categories for small, moderate, moderate-large, and large cont ributions is useful. However, it would be more appropriate to refer to these as c ontributions to “sensi tivity” rather than “uncertainty.” As noted in several place, the sensitivity analyses represent plausible

alternatives to the core estim ate, but are not probabilistic. The language of the RA is appropriate on this point and ha s clearly articulated that the sensitivity analyses results represent plausible and scientif ically defensible estimates. The range of these estimates provides an indication of the im plications of uncertainty.

The evaluation of alternative model struct ure is critically important, because model structure can potentially be a la rger source of uncerta inty than the range of values for an input to a given model. The results in Tabl e 4-3 indicate, for example, that the random effects log-log model provides larger risk es timates than the fixe d effects log-linear

9model used for the core estimates. This info rmation is very useful and is an excellent addition to the RA. The more thorough trea tment of model choices and alternative C-R functions provides plausible alternativ e estimates to the core estimate.

Per CASAC’s comments on the first draft RA, EPA indicates the direction of the percent changes in risk. In addition to the percent difference, the actual difference in risk should be reported to provide further context. Th e second draft RA seems to put emphasis on relative changes in risk. However, the NAAQS are intended to be protective of public health, and therefore the magn itude of the risk estimates is ultimately a more useful policy-relevant metric.

The sensitivity analysis related to peak shav ing and “peakiness” was not very clear in its interpretation. What are the main point s to take away from these analyses?

Charge Question 5: Consideration of design values and patterns of PM2.5 monitoring data in interpreting core risk estimates (section 4.5): To enhance our interpretation of the patterns of core risk estimates generated for both the current and alternative suites of standards, we have included analyses of 24-hour and annual design values together with pattern s of PM2.5 monitoring data for the 15 urban study areas. This reflects the fact that these two factors play a key role in determining the degree of risk reduction estimated upon just meeting the current and alternative suites of standards under alternative rollback approaches. As part of the consideration of design values, we have also contrasted the 15 urban study areas with patterns of design values seen for the broader set of urban areas in the U.S. in order to help place the urban study area in a broader national context.

a) To what extent is the Panel support ive of these addition al assessments?

b) Does the Panel have any recommendati ons for additional insights based on consideration of patterns in design values and PM2.5 monitoring data across the 15 urban study areas and at the national level?

The graphical presentations depicting the 24-hour and annual average design values for US urban areas and the 15 urban study areas used in the RA were very helpful for understanding the concept of the controlling st andard, the implications of reducing either

the annual or the 24-hour st andard, or both, and the repres entativeness of the 15 urban study areas. To further enhance the value of these graphical presentations, more complex color coding could be used to provide info rmation on the US region of each urban study area. In these plots, unsupported conclusions were drawn for cities lying on the border between zones. Graphical presentation of the design values by mon itoring site for the 15 urban study areas provides valuab le insights into the role of patterns of PM monitoring data in different cities in determining cons equences of various control strategies. We recommend that the term “peaky” as used to describe PM pattern s be better defined and applied consistently. The main observations from both sets of plots should be summarized, especially as regards impacts of the alternative rollback procedures.

10Chapter 5 of the revised RA provides estimates of the numbers of deaths attributable to long-term PM 2.5 exposure, based on air quality estim ates from the Community Model for Air Quality (CMAQ) and the environmental Benefits Mapping and Analysis Program (BenMap), and uses the risk estimates deri ved for the Krewski, 2009 assessment of the ACS data with a LML of 5.8 ug/m 3 . A principal purpose for incl usion of this chapter is to place the PM 2.5 -associated risks for the 15 urban st udy areas within the distribution of risks nationally. Figure 5-4 provides the key findings in regard to this purpose and

indicates that the selected ur ban study areas in large part fall in the highest 20% of the

distribution of sites. .

We recommend that Chapter 5 be moved to an Appendix with inclusion of Figure 5-4 at appropriate points within Chapter 4 and the current Chapter 6. The figure provides information relevant to the generalizability of findings from the 15 areas to the entire United States. However, the estimates themselves are not directly relevant to the overall purpose of the RA; the estimation approach diffe rs from that used for the 15 urban study areas; and the chapter is brief and does not ad equately set out sources of uncertainty and variability. By placing the chapter's cont ents into an appendix and specifically acknowledging its purpose, it will not distract from the flow of the RA and the major objective of the analysis will be met by inclusion of Figure 5-4 in Chapter 4 and the revised Chapter 5.

Chapter 6 – Integrative Discu ssion of PM2.5-related Risks

Charge Question 6: We have developed an integrated discussion of the PM2.5-

related risk estimates which considers the results of the qualitative and quantitative treatment of uncertainty and variability t ogether with the various national-scale assessments completed for the analysis to support interpretation of the core risk estimates. As part of the integrative di scussion, we also provide key observations that bear on policy-relevant risk-based questions.

CASAC was unanimously pleased by the additio n of this chapter to the document. The chapter summarizes the many detailed analyses carried out in Chapter 4 and extended in the Appendices. However, the results presen ted could be summarized more effectively and rather than being presented in great detail, there should be a more integrative discussion. The choice of the 15 urban study areas was previously discussed in Chapter 4. Similarly, the choices of endpoints were al ready documented. Clearly, the use of IHD mortality (as opposed to all- causes of car diovascular mortality) represents an upper bound of effects. This outcome was apparently selected because of its availability in the Krewski et al. 2009 analysis.

CASAC discussed the selection of alternative scenarios under the assumption of a linear,

no-threshold C-R function. In the current draf t RA, the lowest levels assessed were 12/25 μg/m 3 ; however, preliminary analyses at 10/25 μg/m 3 were presented, indicating a variable effect across the 15 urban study areas, resulting from different characteristics of some of the areas. As mentioned previously, specific criteria need to be developed and applied in selecting the lo wer bound scenarios. These scenarios will be particularly

11informative for considering pr otection of susceptible populati ons as uncertainty increases

at lower bound concentrations. There are a numbe r of potential bases for the specification of these scenarios including the concentratio ns at which observations have been made, the level of uncertainty, and the exte nt of protection to be achieved.

a) To what extent does the Panel believe that we have captured the key policyrelevant questions that can be add ressed by this risk assessment?

EPA has presented and captured the key-pol icy relevant questions. However, CASAC

continues to recommend a qualitative discussion on PM 10-2.5 and on those effects of PM 2.5 for which the evidence was found to be “suggestive.”

b) We provide a set of key ob servations related to estimates of risk associated with simulations of just meeting the current and alternative suites of standards. These observations are based not only on considera tion of trends in risk reduction across

alternative suites of standards and residual risk remaining after simulation of just meeting specific suites of standards, but al so on additional factors that can impact risk (e.g., the role of annual and 24-hour design values, the peakiness of PM 2.5 distributions within a study area, and app lication of different rollback approaches).

To what extent do the Panel members beli eve that the observations presented in section 6.2 are well supported by the resul ts of the analyses? Are there other observations that might be made that would help to address the policy-relevant questions identified at the beginning of the chapter?

Section 6.2 summarizes the findings of key anal yses. The sensitivity analyses presented explore multiple factors including the roll b ack approach, concentration scenarios, and location, the latter influenc ing findings through the the PM 2.5 concentration profile and mortality rate. The text describing these key observations is too l ong and not sufficiently structured. While the analys es do support the observations made, the text does not make these linkages with sufficient clarity. In fact, much of the discussion becomes too

anecdotal as individual cities and scenarios are reviewed.

Key observations are presented and adequately discussed with rega rd to the roles of annual and 24-hour design values in determ ining population risk and of the role of “peakiness” of distributions. The text should more sharpl y characterize the critical finding of the cross-area vari ation in the extent of risk reduction under the various scenarios. This variation needs emphasis as extension of the findings from the 15 urban study areas to the entire country is considered.

c) Part of our interpretation of the core ri sk estimates presented in section 6.2 is our

characterization of confidence in the core risk estimates and in observations made

based on those estimates. These a ssessments of confidence are based on

consideration of the results of the sensitivity analysis as well as on the qualitative assessment of uncertainty and variability. To what extent does the Panel believe

that the characterizations of confidence in the core risk estima tes and associated policy-related observations are reasonable given available information?

1213 See above. The RA reasonably and appropriate ly describes the level of confidence that can be given to its analyses, given sour ces of uncertainty and variability.

d) As part of the integrative discussion, we use the results of several national-scale analyses (i.e., the national scale PM2.5 mortality analysis, the representativeness analysis, and the new exploration of design values and patterns of PM2.5 monitoring data presented in section 4.5) to place the results of the risk assessment

in a broader national-context. What are the Panel members’ views on appropriateness of this effort to place resul ts of the analysis in a national context?

The RA presents multiple analyses that explore issues related to the extension of the findings from the 15 urban study areas to a national-scale. These areas were admittedly not chosen as a representative sample per se, but to be illustrative of locations with different patterns of PM 2.5 concentrations. The analyses presented in sections 4.4 and 4.5 and in Chapter 5 satisfactorily set out a ba sis for placing the RA results in a national context. Figure 5.4 is also valuable, and s hould be included in the revised Chapter 6.

e) We conclude chapter 6 with a list of key observations. Does the Panel believe that we have appropriately high lighted key findings of the risk assessment in these observations? Of particular note is the observation that, while alternative 24-hour standard levels can be used to reduce annual-average PM2.5 concentrations and thus to reduce estimated risk, the results are likely to be highly variable across urban areas. More consistent lowering of annual-average PM2.5 concentrations across study areas, and thus more consistent reductions in estimated risk, may result from application of alternative annua l standard levels. We also note that

simulation of the alternative 24-hour standard level of 25 μ g/m3 resulted in reductions in annual-average PM2.5 levels for some study areas that were well

below the lowest annual standard level assessed (i.e., below 12 μ g/m3). As a consequence, we observed risk reducti ons reflecting these changes in annualaverage PM2.5 levels below 12 μ g/m3. Given these results, does the Panel believe that there is utility in estimating risks for alternativ e annual standard levels below

12 μ g/m3?

Key observations are presented in a balanced and fair way. Sta ff acknowledges that the ranges of effects are in two cat egories: 3-9% and 0-3% in two halves of the country. This may be less precise than what the data indicate in that it w ould appear from their own estimates that the bulk of the effect comes from the upper end of the exposure in the counties (pg 5.8, line 12-15).Enclosure C

Compendium of Individual Pr eliminary Panelists Comments Quantitative Health Risk Assessment fo r Particulate Matter (February 2010)

Comments from Mr. Ed Avol .......................................................................................... 2

Comments from Dr. Joe Brain ........................................................................................ 9 Comments from Dr. Wayne Cascio ............................................................................... 12 Comments from Dr. Christopher Frey ......................................................................... 13 Comments from Dr. Joseph Helble ............................................................................... 16 Comments from Dr. Rogene Henderson ....................................................................... 18 Comments from Dr. Philip Hopke ................................................................................ 21 Comments from Dr. Morton Lippmann ....................................................................... 23

Comments from Dr. Robert Phalen .............................................................................. 28

Comments from Dr. Ted Russell ................................................................................... 29 Comments from Dr. Frank E. Speizer .......................................................................... 32

Comments from Dr. Helen Suh ..................................................................................... 36

Comments from Dr. Sverre Vedal ................................................................................. 37

1Comments from Mr. Ed Avol

General:

The second external review draft of the Quantitative H ealth Risk Assessment for

Particulate Matter is a dramatic and substa ntial improvement over pr evious versions, and sets a new thoughtful foundation and prototyp e for subsequent risk assessments to generally follow. Improved presentations, superior figures, and useful tables and footnotes are present to guide the reader through the pr ocess by which EPA staff and consultants have reviewed available data, pe rformed additional analyses, and interpreted the results of those analyses. The NAAQS h ealth risk assessment process is becoming increasingly transparent and tr ackable (to the benefit of th e reader and the public), but some convolutions and meanderings still pe rsist to prevent co mplete clarity of presentation and focused presentations of conclusions, decisions, and positions. Chapter 4 provided a detailed step-by-step examina tion of the approach, the analyses, and the results in appropriate depth. Chapter 5 seemed minimal and uncertain as to whether the authors wanted to present it in the main body of the risk assessment or in an appendix. The addition of Chapter 6 as an opportunity to present an integrated view of the work performed is much-appreciated a nd was generally well-presented.

Charge Questions to CASAC: 1a) Utility of the peak-shaving approach: The peak shaving approach seems useful a nd warranted, and aids in understanding the impact and effects of various reduction a pproaches for a range of possible 24hr/annual

standards.

1b) Use of the composite monitor approach: This may well be the appropriate approach to take, but it would be more convincing if some supporting data for this approach were provided in the appendix or main body of the document. This could readily be done us ing sites in several of the urban study areas with complete or near-complete data, and dr opping out one or more data points, creating the “need” for fill-in strategies and a ready co mparison with the actual (observed) data.

2) Selection of model inputs: This seemed generally clear and s upported. The justified emphasis on PM 2.5 and cardiovascular endpoints may be appropriate at this point in the NAAQS review cycle, but ongoing concern about other particle sizes and constituents, as well other health endpoints, makes the current specific approach time-limited and in need of likely review in successive cycles.

3) Addressing uncertainty and variability: The discussion and clarifications regard ing uncertainty and variability are muchimproved, much-appreciated, and markedly enha nce the credibility and stature of the entire document. The presentations are de finitely appropriate and mostly clear.

4) In general, the approach seemed appr opriate and well-documented. Some assumptions appear to have been made (or at least, not justified in the main body of the presentation),

2such as the consideration of Philadelphia a nd Los Angeles as “representative” East and West Coast urban areas, respectiv ely, for sensitivity analyses.

5a) The approach to include both 24hr and a nnual design values in assessing core risk estimates is a reasonable one that helps to cl arify differences between reduction strategies that may not otherwise have been apparent . Expansion of the discussion to 15 urban study areas is a worthwhile endeavor that yiel ds a number of insight s and benefits (not the least of which is added credibility to the calim of national representativeness). The approach was generally well-done , added to the discussion, and enhanced the validity of the presentation. Several specifi c questions remain (see deta iled queries below) but in general, I am wholly supportive of the pres ented approach and applaud the staff and

consultants for their work.

5b) (No additional insights or r ecommendations at this time)

6a) The key policy-relevant questions ha ve been captured in the presentation.

6b) The observations in Secti on 6.2 are well-supported but not always clearly articulated.

6c) The characterization of confidence in the core risk estimates is a marked improvement over previous versions of th e document, and helps to establish the credibility of the overall approach. Given the available information, the presentation appears appropriate.

6d) The presentation of the placement of the risk assessment results in to a national context appeared a bit tentativ e and unsure. A more forcef ul presentation and assertion about representativeness c ould and should be made.

6e) The key observations listed at the end of Chapter 6 are generally present, but not presented in as clear or concise a manner as could be done. I believe it is useful to present estimated risks for a range of standard s and levels, so that the Administrator has a more complete perspective of the possibl e implications of the various reduction strategies. However, I don’t think it is worth a great deal of effort or time to estimate risk reduction at 6 or 8 or 10 ug/m 3 annual average. Practically speaking, there are US urban areas unlikely to ever get those low levels, so with a uniform national standard, it seems

more of an academic exercise.

Questions: P3-11, lines 30 on (the composite monitor a pproach) – The procedure is described for filling in missing data, but there is not presentation or reference to any validation of how precise, accurate, or appropriate the fill-in a pproach is, based on more complete data at sites within the respective urban study areas. Was any assessment of the efficacy of this approach done? The comment in the text (p3-13, lines 7-11 that in New York, for example, 2/3 of the data was interpolated ma y raise some cause for concern that could be allayed by providing some sample analyses with existing and retrievable data.

3P4-28, line 18, sensitivity analysis modeli ng using Philadelphia and Los Angeles as “representative” East and West Coast urba n areas...how was this decision made? On what basis or comparison criteria? How ”represe ntative” are either of these urban areas of their respective coasta l counterparts? Some explana tion or justification should be provided for these selections, even if it is only to refer to an Appendix or footnote.

P4-49, line2 – the explanation behind th is discussion of determinants of PM 2.5 health estimates is confusing. On p4-48, lines 21-24, the estimates of risk in the risk assessment

framework are claimed to be based on four elements: “...populati on, baseline incidence rates, air quality, and the coefficient rating air quality and the health outcome (i.e, the

PM 2.5 effect estimates)”. On the next page, the determinants of the PM 2.5 effect estimates are claimed to”... be grouped into three areas: demographics, baseline health conditions, and climate and air qualit y” (which are actually four areas). However, it would appear that these statements argue that the fourth el ement of the basis for the risk estimates is composed of the first three elements (if population = demographics, baseline incidence

rates = baseline health cond itions, and air quality = air quality), which seems a bit circular in descrip tion and confusing.

P4-49, line 14 bullet – It is not clear why clim ate and air quality are included in the same bullet, since the references and variables list ed seem to cover each of the climate and air quality categories separate ly and without overlap.

P4-51, Table 4-5, Co-Pollutant Levels row – in the text discussion of PM 2.5 and relevant risk estimates, SO 2 is specifically identified and include d in the figures and tables, yet in the table presented national-level data for us e in developing risk determinants, ozone is the only co-pollutant presented. It would th erefore seem appropriate to note why Ozone is listed (cardiovascular mo rtality reference?) and why SO 2 is not.

P4-54, line27 – It is a bit confusing for the r eader to follow the discussion from text to table to figures, sin ce the number of urban case study areas plotted vary in the table (4-6) by category, are not presented in any key w ith the appropriate figures, and don’t appear

to be consistent between the table listing and the related figure.

P4-57, Figure 4-9 through 4-16: (a) It would make it easier for the reader to follow the discussion and the illustrated material in the figures if some additional information were provided in a key or caption with the figures, such as the number (n) for each of the three data categories plotted (all counties CDF, Case study count ies CDF, case study counties) and some comment about the splin es or plots and inflection poin ts displayed; (b) there is an inconsistency in the presen tation of these figures, sin ce some specifically comment on the number of urban case study areas above or below a given percentile (Figures 4-9, 4- 11,4-14, 4-15, 4-16), while others have no comment (Figure 4-10, 4-12, 4-13); A summary comment for these three figures woul d help guide the reader as to the key implications of the provided figure; (c) Woul d there be additional clarity or power of argument gained by somehow highlighting or identifying the plotted values of the 15 urban study areas from amongst the larger number shown?

4P4-66, lines 10-12 and P4-67 (Figure 4-17) – Th e text and figure are inconsistent, since the text says (with respect to the 35/15 plot in Figure 4-17) “...Atlanta, Birmingham, and Houston fall into this zone...” (Zone C), but Figure 4-17 shows Birmingham in Zone B2, Atlanta arguably on the cusp between C and D, and Houston well into Zone D...and in fact, the next bullet in the text (P4-66, lines 13-16) discusses Houston’s presence in Zone

D.

Chapter 5 (National-Scale Assessment of L ong-Term Mortality) – This chapter seems a bit tentative. The data pres ented is critical to the overall discussion, but the manner in which it is presented doesn’t fit with the previous chapter or document flow. The introductory sections are fine and lay the ground for presentation of the results, but the methods section seemed spotty and uncertain in tone as to whether this should be better placed entirely in Appendix G with more deta ils, or presented more conceptually here, with reference to the appropriate appendix fo r details. The two figures presented are the key information for the chapter. The text provides a description of what is literally presented in the figures, but provi des little in the way of interpretation (see text related to Figure 5-3, for example). What is the intende d message associated with Figure 5-3? A sentence or two would help to direct readers’ interpretati on or consideration of this.

P5-2, lines14 thru 21 (interplay between re gional-scale reductions in adjacent urban areas) – the issue under discus sion and the key problem bei ng identified still is not

clear...

P5-5, line 15 - The reference to “...7 Health Effects Institute PM regions...” seems unnecessary and distracting; th is discussion of health im pacts relates to geographic regions of the country (southwes t, southeast, etc, as pres ented in Table 3.2, p3-14), and

how the Health Effects Institute identifies these areas does not seem germane to the central argument.

Chapter 6 (Integrative Discussion) offers the promise of pulling together the previous chapters’ presentations into one focused summary for reader review, but up through Section 6.3, too often self-destructs by using meandering phrasing and including extraneous comments in conclusionary statements. The net result of the writing approach employed is to obscure the message and frustr ate the reader. There are some excellent, insightful, and important points to be made in this presentation, but they are often hidden in multiple qualifying commentaries when the main point is being presented. Use of bullets in the commentary can be an effec tive way of focusing the reader on the main persuasive points of the presentation, but in th is chapter, their use suffers from a lack of clarity. With each bullet, make the statement, and then provide a terse justification in the ensuing paragraph. For example, the bullet might be “Dramatically different rates of mortality risk reduction are obs erved in areas with peaky PM 2.5 distributions, when peakshaving, compared to proportional rollback , ap proaches are used.” A brief supporting paragraph could use Salt Lake City or Los A ngeles data as an example to make the point.

5P6-7, line 21-22 – This comment is self- obvious and not necessary; of course the variability in incidence estimat es is driven by differences in the study population (if the analyses are correctly performed and identifia ble confounders are adjusted for). Remove this opening sentence and begin this para graph with something like, “Substantial variability in incidence estimates wa s observed for the 15 urban study cases. Substantially less variability woul d be expected in estimates...”

P6-17, lines 26 thru 29 – This seems like a ve ry odd disclosure comment to make as a closing comment. Under what set of circ umstances does EPA believe someone reading this document would interpret the 15 urba n areas under discussion being the ONLY 15 areas in the country having health risks associat ed with PM2.5? This last sentence in the paragraph seems entirely unnecessar y, but if there is a sense that this point is worthy of a comment, I would suggest that the last two sent ences in this section of the text (lines 24- 29) be deleted and replaced with somethi ng like the following: “The results of the national-scale mortality analysis suggests that the 15 urban study areas selected for presentation in this document are represen tative of a continuum of urban area results towards the upper end of the cumu lative mortality distribution.”

P6-19, lines 16 thru 23 (the final bullet on th e page) – This comment, which establishes the credibility for evaluating the 15 urban st udy areas chosen, does not directly address the policy –relevant question posed under which it is listed, but does go to a central issue in the considering the releva nce of the risk assessments undertaken. Accordingly, this bullet should be moved up to become the fi rst bullet in Section 6.4, establishing the credibility of the study areas used in the analyses, before the results of those analyses and

their relevance to policy are presented.

Editing Details: P2-6, line 28 – delete one of the periods at the end of the sentence P2-8, line 27 – something is missing where the “(chapter 5)” placeholder is in the sentence; perhaps “long-term mortality”? P3-7, line 34 – need hyphen between “empirically estimated” P3-13, line 6 – “synced” is mis-spelled; shou ldn’t this be “synched” or “synchronized”? P4-17, line 22 – remove underline for “of total incidence” P4-17, line 28 – change “these head negative point estimates” to “these had negative...” P.4-23, line 13 – change “...2-4hour average risk...” to “24-hour average risk” P4-27, line 26 – capitalize “elev en” to become “Eleven” P4-38, line 2 – “...morality...” should be “....mortality...” P4-40, line 26 - “...at the same urban study ar e...” should be “...at the same urban study

area...” P4-41, line 25 - This first sentence anticipates a listing of several factors that are then never presented, so the sentence should be re -structured, perhaps as, “There are several observations regarding key sources of ... that can be made.” P4-41, line 28 – change “factor” to “factors” P4-42, line 1 – what does “...the highest sensitivity analysis results...” mean? Does it mean the biggest changes, or the best outcome? A little re-phrasing would help.

6P4-42, line 25 – for consistency with previous line, probably better to say “...log-linear

model with fixed effects...” P4-42, line 28 – remove hyphen from local-sources P4-42, line 34 – should be re-phrased to read “...in the area of the curve between the LML and the PRB” rather than “...as you move below the LML” P4-42, lines 36 to 38 – this is an important comment that, as written, is difficult to understand. I would offer the following: “ Due to large uncertainties associated with C-R functions in the range between the LML a nd PRB, such estimates should be excluded when considering reasonable alternative risks associated with core risk estimates.” P4-45, footnote refers to open circles in Figures 4-6 and 4- 7, but neither has open circles nor displays SO 2 risk estimates; I believe Figure 4-8 is the correct reference. P4-46, Figure 4-8 key – add a “g” to “modelin” P4-47, line 13 – change phrasing from “...a ttributable to secondarily formed PM 2.5 ” to “...partially attributable to secondary formation of PM 2.5 ” P4-47, line 18 – this statement can be more definitive, based on Figure 4-8; Instead of “Most if not all”, change to r ead “Virtually all of the alternative model specifications...” P4-47, lines 29, 31, and 32 – Figure references should be to Figures 4-7 and 4-8, not 4-6

and 4-7. P4-49, line2 – shouldn’t this be four areas, not three (demographics, baseline health, climate, and air quality)? P4-49, line3 – change “..., and climate and air quality...” to “..., climate, and air quality...” P4-49, line31 – Incorrect table re ferenced; should be Table 4- 5 in current draft version. P4-54, line 5 –Kolmogorov- Smirnov test results appear in Table 4-7, not in Table 4-4. P4-54, line21 – Figure number s are incorrect in text. P4-70, Table 4-8 – the entry for Urban study area Pittsburgh PA has a superscript “5” following it, but no footnote or ke y to what this refers to. P4-71, line 5 – should be Figure 4-19, not F-19. P4-75, footnote 61 – second line should read “...related mortality – see section...” P4-75, lines 22 thru 27 – this sentence can and should be divided into several shorter

sentences. P5-5, lines 27 thru 29 – this looks like the title of a table or figure, not part of the

text...??? P5-6, footnote, second-to-last line – should read “...are subj ect to greater uncertainty...” P6-1, line 31 – remove one of the two periods at the end of the sentence. P6-4, line 23 – “INTERPRETATION” is mis-spelled (missing a T). P6-6, line 2 – it would be clearer to re-phras e this to read: “...rela tively high 24-hr design values compared to the respective area’s corresponding annual aver age design values...” P6-6, line 6 – the term “composite monitor” should be defined or explained in a parenthetical comment or footnote in this in tegrative summary chapter, to avoid having readers backtrack to previous chapters to try and find the intended meaning. P6-6, line 18 – the correct figure reference (f or the current standard values) is Figure 4-

17. P6-7, lines 17 & 18 – Actual IHD mortality values in Table 4-1 are 15-19 for Salt Lake City and 1755-2222 for New York City. P6-7, line 25 – should read “...current suite of standards...”, not “...if standards...”

7P6-9, line 27 – change “that” to “than” P6-11, line 1 – change “...just meeting the a given...” to “...just meeting a given...” P6-14, footnotes 78 and 79 – referen ces to Figure 4-20 should be 4-19 P6-15, lines 7-9 – This sentence is garble d and confusing, and needs re-writing and

clarification. P6-15, footnote 81 – reference Figure 4-18, not 4-20. P6-18, lines 11-12 – delete the rest of th e sentence and end after “...2,000 cases per year.” (This is repetitive and not n eeded in this summary statement). P6-18, lines 14-15 – delete the rest of the sent ence after “...in a given year for the urban study areas.” P6-18, line 17 – Delete “Generally comparab le” and begin the sentence “Estimates of CV-related mortality...” P6-18, lines 24-25 – Correct first sentence to r ead, “A broader array of health effects has

also been associated with PM 2.5 exposures, including reproductive effects and developmental effects in children.” P6-18, lines 29-32 – Change first sentence to read:”Given the quantit ative and qualitative assessments of uncertainty and variability performed as part of the risk assessment, it is unlikely that the degree of risk remaini ng upon the simulated meeting of the current (15/35) suite of standards has been over-stated.” P6-18, line 41- Re-write to read: “T his variability in annual-average PM 2.5 concentrations is most prominent in study areas where the 24-hour standard is “controlling”.” P6-19, line 14 – Delete the word “well”; not clear what the definition of “well below a value” is, if some of the values are said to be approximately equal to it). In this context,

it seems sufficient to state th at the annual average concen trations under discussion are

below the level of the current annual standard. P6-19, line 19 – delete the phrase “in most such areas”; it is redundant, given the beginning of the sentence and the sentence structure. P6-19, line 21 – replace “ref lective of” with “as do” P6-19, line 36 – add “%’ after “11” P6-21, line 2 – delete comma after “... with a” P6-21, line 3 – insert “U.S.” before “...counties” P6-21, line 5 – replace the phrase “...likely capture well...” with “do represent” P6-21, lines 7-9 – why is this comment here ? Isn’t it self-evident in selecting a representative sub-sample? As I have comm ented previously, this is unnecessary and can

be deleted.

8Comments from Dr. Joe Brain

Charge Question #6a-e

6) We have developed an integrated discussion of the PM 2.5 -related risk estimates which considers the results of the qualitative and quantitative treatment of uncertainty and variability together with the various national-scale a ssessments completed for the analysis to support interpretation of the core risk estimates. As par t of the integrative discussion, we also provide key observations that bear on policy-relevant risk-based
questions.

a) To what extent does the Panel believe that we have captured the key policy-relevant questions that can be addressed by this risk assessment?

Response : Chapter 6 is the culmination a rationa le developed throughout the document. It needs to pull all the pieces together and cl early integrate them in a form that provides the basis for decisions. The panel responds pos itively to Question 6a. We believe that

this second draft does effectiv ely capture the key policy-relevant ques tions that can be addressed by the preceeding risk assessment.

b) We provide a set of key observations rela ted to estimates of risk associated with simulations of just meeti ng the current and alternative suites of standards. These observations are based not only on considerat ion of trends in risk reduction across alternative suites of standards and residual ri sk remaining after simulation of just meeting specific suites of standards, but also on additional factors that can impact risk (e.g., the role of annual and 24-hour de sign values, the peakiness of PM 2.5 distributions within a study area, and application of diff erent rollback approache s). To what extent do

the Panel members believe that the observ ations presented in section 6.2 are well supported by the results of the analyses? Are there other observations that might be made that would help to address the policy-relevant questions identified at the beginning of the chapter?

Response : The panel believes that Section 6.2, Interpretation of Urban Study Area Results, is well supported by the results of the previous analyses. We are not aware of other observations that might affect the answers to th e policy-relevant questions identified at the beginning of Chapter 6.

c) Part of our interp retation of the core risk estimate s presented in section 6.2 is our characterization of confidence in the core risk estimates and in observations made based

on those estimates. These asse ssments of confidence are based on consideration of the results of the sensitivity anal ysis as well as on the qualitat ive assessment of uncertainty and variability. To what extent does the Pane l believe that the characterizations of confidence in the core risk estimates and associated policy-related observations are reasonable given ava ilable information?

9Response : The authors have done an appropriate se nsitivity analysis and have adequately characterized both uncertainly and variabilit y. The panel believes that the degree of confidence expressed in the core risk estimat es is appropriate. We believe that the adjacent policy-related observations are reasona ble given the information available to the

EPA and summarized in the final version of the PM ISA.

d) As part of the integrative discussion, we use the results of several national-scale analyses (i.e., the national scale PM 2.5 mortality analysis, the representativeness analysis, and the new exploration of desi gn values and patterns of PM 2.5 monitoring data presented in section 4.5) to place the results of th e risk assessment in a broader national-context. What are the Panel members’ views on appropriate ness of this effort to place results of the analysis in a national context?

Response : One could always suggest additional cities or parts of the country where PM data could be applied. The panel believes that the national-scale analyses which were carried out are appropriate and are reasonably varied and thus they do provide the context for placing the results of the policy analysis in a reasonably representative national

context.

e) We conclude chapter 6 with a list of key observations. Does the Panel believe that we have appropriately highlighted k ey findings of the risk assess ment in these observations? Of particular note is the obser vation that, while a lternative 24-hour st andard levels can be used to reduce annual-average PM 2.5 concentrations and thus to reduce estimated risk, the results are likely to be highly variable across urban areas. More consistent lowering of annual-average PM 2.5 concentrations across study area s, and thus more consistent reductions in estimated risk, may result from application of altern ative annual standard levels. We also note that simulation of the alternative 24-hour standard level of 25 μ g/m 3 resulted in reductions in annual-average PM 2.5 levels for some study areas that were well

below the lowest annual standard l evel assessed (i.e., below 12 μ g/m 3 ). As a consequence, we observed risk reductions reflecting these cha nges in annual-average

PM 2.5 levels below 12 μ g/m 3 . Given these results, does the Panel believe that there is utility in estimating risks for alte rnative annual standard levels below 12 μ g/m 3 ?

Response : Section 6.4, Key Observations, is a valuab le part of this chapter and of the document as a whole. The three questions asked are important and the answers provided are well founded and useful. They get to the heart of the matter. What are the consequences of maintaining the current st andard? What are the likely benefits of lowering the standards? To what extent are policies abou t PM2.5 applicable throughout the country? The bullets follo wing the three questions succinc tly address these questions, and will be useful to the administrator in forming the basis for her decision.

Consistent with discussions at the last meeting of the PM advisory committee, we are comfortable with not estimating risks for a lternative annual standard levels below 12

μ g/meter 3 . At these concentrations, data is in creasingly less common and less reliable. Therefore, estimates of risk will be subject to greater and greater uncertainty. We believe

10that the risk estimates at levels higher than this provide an adequate basis for decision making.

11Comments from Dr. Wayne Cascio

General Comments:

With the second draft assessment document the staff of the EPA’s Office of Air Quality Planning and Standards (OAQPS) has produces an excellent document that largely addresses concerns raised by members of CASAC and the public . Specifically, the

second external review draft now contains a more comprehensive discussion of the core risk estimate through the integrat ion of uncertainty and variab ility analyses from 15 urban centers, and national-scale long-term exposur e PM2.5 mortality assessment. The rationale for not including a quantitative assessment of risk associated with PM10-2.5 presented in section 2.3 and appendix H is justified based on the limitations in the data available for characterizing PM10-2.5 exposure and risk. Th e overall presentati on and discussion of the consideration of variability a nd uncertainty is much improved.

Charge Question 2. Selection of model inpu ts (section 3.3): We have expanded and

clarified the discussion of our rationale fo r identifying modeling choices comprising the core risk model, focusing in particular on selection of C-R func tions (section 3.3.3). To

what extent does the Panel consider this disc ussion to be clear and the model selections appropriate?

Response. The expanded discussion provided in Section 3.3.3 Selection of Epidemiological Studies and Concentration- response (C-R) Function within those Studies conveys the rationale for identifying mode ling choices and adequa tely justifies the selection of specific epidemiological studies utilized to establish the C-R functions. The rules employed for study selection as described on 3.27 and 3.28 provided a strong foundation to identify studies that provide the most accurate data to derive the C-R relationship. The expansion of the discussion and integration of the ISA was also very useful in that it provides an opportunity to reinforce the gaps in knowledge, for example on 3-30 where it is stated that, “There were no multi-city studies for this [short-term exposure to PM2.5 and emergency department vis its for cardiovascular and/or respiratory illnesses] category of health endpoint.”

12Comments from Dr. Christopher Frey

Charge Question 3:

In general, the second draft REA appropriate ly identifies and disc usses key sources of variability and uncertainty, and includes se nsitivity analysis th at provides insight regarding the impact of some sources of uncertainty on the core risk estimates.

EPA provided a footnote explaining the rati onale for identifying “key” sources of variability. The document should indicate if the same process was used to identify “key” sources of uncertainty.

In the discussion of key sources of variability, EPA added ma terial regarding copollutant concentrations and on demographic and soci oeconomic status, as requested in CASAC

comments on the first draft of the REA. As a minor comment, it is not entirely clear that age of housing only affects air exchange rate because of air conditioning use. Newer homes are typically “tighter” than older homes , and thus have lower infiltration rates.

Climate zones are another factor in infiltra tion. For example, northeastern homes do not have as high a proportion of central air c onditioning as southeastern homes. Given its effect on particle composition, concentrations, ventilation and activity patterns, it would make sense to also include seasonality in the list of va riability sources.

We note that EPA did not address a comment on the first draft REA to the effect that “exposure modeling should be included in th e REA. A probabilistic Tier 3 approach should be used for the exposure assessment.” While we understand that timing may have precluded adequate treatment of this topic, we expect that EPA will develop this capacity

for future revisions of the st andard. CASAC asked for this five years ago, and would

like to see this in the next revision.

With regard to uncertainties, in response to CASAC comments on the first draft of the REA, EPA has included uncertainty in the C- R function itself, which was developed from single studies. EPA has appropriately taken into account differences in C-R functional

form associated with studies that addressed long-term or short-term effects for single or multi-city studies even if they were not the basis for the final set of C-R functions used in

the REA.

EPA has provide some explanation of the m eaning of uncertainty categories of “low”, “medium”, and “high.” This discussion is adequate. However, there is a confusing statement to the effect that “high” sources of uncertainty “are likely to influence the interpretation of risk...” “if those sources of uncertainty are reduced or more fully characterized.” The parenthetical “if” clause seems to confuse the issue, and should be deleted. A reader of the paragraph on lines 6-20 of p. 3-63 mi ght wonder how “staff consensus” was achieved, and whether c onsensus is an appropriate goal when characterizing uncertainty. A potential concer n is that achievement of “consensus” might mean that some opinions over-ride others a nd that the resulting characterization of

13uncertainty might be biased. There are short-comings of group-based elicitation processes, such as dominance by strong pers onalities or a tendency to provide opinions about goals rather than state of knowledge. It would be usef ul to explain the process by

which “consensus” was achieved.

EPA has done a nice job on commenting on the extent to which there are dependencies among pairwise combinations of sources of uncertainty, and whethe r these dependencies

would tend to offset or to increase the overa ll range and direction of uncertainty in the assessment results. For example, the statistica l fit of the C-R functions, and the shape of the functions, are inter-related. EPA has provi ded a nice treatment of this on page 3-71.

Based on quantifiable sensitivity analysis, the report generally clearly conveys that the “core” estimates appear to be at the low e nd of alternative “plausible” estimates. However, particularly in Chapter 6, the role of sources of uncertainty treated qualitatively

should also be addressed. In particular, given exposure misc lassification, it is likely that

the core estimates are biased low. This is an important point to c onvey consistently. The

core estimates seem to be conservative in the sense of being underestimated, which is not typical practice for public health endpoints, given that virtually al l of the sensitivity analyses result in higher risk estimates as compared to the core.

Charge Question 4:

Overall, the sensitivity analysis section 4.3 is very good and nicely covers a complex topic. Table 4-3 is a useful summary. Th e classification of descriptive categories for small, moderate, moderate-large, and large cont ributions is useful. However, it would be more appropriate to refer to these as c ontributions to “sensi tivity” rather than “uncertainty.” As noted in several place, the sensitivity analyses represent plausible alternatives to the core estimate, but are not a probability sample. Thus, there is not a probabilistic interpretation to the sensitivity analysis results. EPA has appropriately addressed this point and has clearly articulated, quite reas onably, that the sensitivity analysis results represent plau sible and scientifically defens ible estimates. The range of these estimates provides an indication of the implications of uncertainty.

The evaluation of alternative model struct ure is critically important, because model structure can potentially be a la rger source of uncerta inty than the range of values for an input to a given model. The results in Tabl e 4-3 indicate, for example, that the random effects log-log model provides la rger risk estimates than the fixed effects log-liner model

used for the core estimates. This information is very useful and is an excellent addition to the REA. The more thorough treatment of model choices and alternative C-R functions provides plausible alternativ e estimates to the core estimate.

It was not apparent that EP A responded to this comment on the first draft REA: “The range of uncertainty a ssociated with confidence interval s for a given C-R function (which is an example of a Tier 3 assessment, which should be mentioned) should be compared to the range of estimates obtained by comparing alternative functional forms. This would provide insight as to whether model structur e, or random error for a given model, is a

14more important source of uncertainty.” It would be useful to make this comparison, which can be discussed qualitatively.

Per CASAC’s comments on the first draft RE A, EPA indicates the direction of the percent changes in risk. In addition to the pe rcent difference, the actual difference in risk should be reported to provide further cont ext. The second draft REA seems to put emphasis on relative changes in risk. However, the NAAQS are intended to be protective of public health, and therefore the magnitude of the risk estimates is ultimately a more useful policy-relevant metric.

The sensitivity analysis related to peak shaving and “peakiness” was not very clear. What are the main points to take away from th ese analysis could be clarified. Is there an implication of some sort of risk trad e-off between the cities with and without “peakiness”, as shown in the comparison of re sults for the different roll-back approaches?

What is the premise of the overarching conclu sions – e.g., that there are strong regional or inter-city effects? Woul d this carry forth to the policy analysis in some way?

15Comments from Dr. Joseph Helble

1) Air quality inputs (section 3.2): We have exp anded the considerat ion of alternative approaches to simulating just meeting th e current and alternative suites of PM2.5 standards (i.e., rollback approaches) to incl ude a peak shaving approach, in addition to the hybrid and proportional approaches consider ed in the first draft assessment. This peak shaving approach is intended to repres ent more localized, rather than regional, patterns of PM2.5 reductions (discussed in section 3.2.3.3).

a) To what extent does the Panel believe that the use of the peak shaving approach provides useful additional exploration of vari ability associated w ith how ambient PM2.5 concentrations are simulated to change upon just meeting the curre nt and alternative suites of standards?

It is certainly reasonable to explore alternativ e approaches to just-meeting standards. The traditional proportional rollback approach was au gmented in an earlier draft of the current PM risk assessment by including “hybrid rollbac k,” in which localized monitor-specific reductions are imposed and then allowed to propagate through an air quality study area through imposition of a distance-decay function. Area-wide proportional rollback is then applied as needed. As noted previously, th is seems a reasonable approach to estimating the potential effect of local controls on PM levels for comparison with the region-wide effects approximated by proportional rollback.

The current question regards the benefits of adding a third approach that involves peakshaving of concentrations at sp ecific monitors within an air quality study area in an effort to assess the potential effects of highly local ized controls. In this approach, only those monitors exceeding the 24 hour standard are rolle d back. The use of this alternative to examine hypothetical “what-if”s cenarios seems reasonable if the goal is to understand whether reducing concentrations at only the peak monito ring locations would be sufficient to meet overall air quality target s in a given study area. Beyond this, however,

the value of the exercise is not clear. It is difficult to imagine a scenario in which such targeted localized controls could be effectively deploye d for PM2.5. Given this, the hybrid approach seems more appropriate.

b) We have used comparisons of composite monitor annual averages generated using the different rollback approaches as a surrogate for differen ces in long-term exposurerelated mortality in looking across all thre e rollback approaches. To what extent does the

Panel believe that this is a reasonable approach for assessin g the impact of variability associated with simulating changes in air quality patterns on estim ates of long-term exposure-related mortality?

Given the uncertainty in the data, particularly due to incomplete monitoring datasets or differences in data collection patterns from re gion to region, this approach is reasonable.

Related to this, regarding the approach to generating composite data sets in locations where there are gaps in indivi dual monitor datasets, was the accuracy of the interpolation

16routines tested? If at least one urban study area has a complete data set, the accuracy of the interpolation approach could be assessed by artificially (randoml y) deleting specific values from the monitoring dataset, interp olating as before, and then comparing the

estimated values with actual.

17Comments from Dr. Rogene Henderson

Comments on Assigned Charge Qu estions for 2nd Draft of PM-RA

Charge Question 5: Consideration of desi gn values and patterns of PM2.5 monitoring data in interpreting core risk estimates (sec tion 4.5): To enhance our interpretation of the patterns of core risk estimates generated fo r both the current and alternative suites of standards, we have included analyses of 24-hour and annual design values together with patterns of PM2.5 monitoring data for the 15 urba n study areas. This reflects the fact that these two factors play a key role in determ ining the degree of risk reduction estimated upon just meeting the current and alternativ e suites of standa rds under alternative rollback approaches. As part of the consid eration of design values, we have also contrasted the 15 urban study area s with patterns of design values seen for the broader set of urban areas in the U.S. in order to help place the urban study ar ea in a broader national

context

a) To what extent is the Panel suppo rtive of these additional assessments? I found Figures 17-21 quite helpful in putting the 15 urban areas into context with the monitoring values from other cities in the US . The figures also helped me to visualize what the overall picture looked like a nd what the controlling values are.

I would like to have more discussion a bout how the peak shaving rollback method

was used. When the short-term values are controlling, it does not make sense to me to use the same percentage rollb ack for the annual values as is required by the short-term

values. This results in non-feasible annual va lues (e.g., 7 ug/m3) for some cities such as

SLC. Do we have actual evidence to indicate that if the short-term value were reduced

55% that the annual value would also be re duced 55%? I do not s ee the logic to that assumption. Why not consider the risk of mortality from s hort-term exposures separately from the risk of mortality from the average a nnual exposures. It is stated in the beginning of Chapter 6 (page 6-1) that the primary focus will be based on risk associated with long-term exposure to PM, because long-term exposure to PM2.5 has been shown to produce substantially larger mortalit y risk compared to short-te rm PM2.5 exposure. Based on that, one might expect the calculated risks from the annual exposures to be the most critical consideration to protec t public health. I do not think it is appropriate to mix the two design values in th e rollback procedures. But perhaps I do not understand how the p eak shaving rollback method was used. It states on page 3-15, lines 20-23, that the proportional rollback method was the only one used to generate core risk estimates and th e other two rollback methods were only used for sensitivity analysis. I would appreciat e more discussion of this at our meeting. I suggest adding the term "design value" to the list of terms on page viii, with a reference to page 3-16 for a definition.

b) Does the Panel have any recomme ndations for additional insights based on consideration of patterns in design values and PM2.5 monitoring data across the 15 urban study areas and at th e national level?

18It would be nice to have Figur es 17-19 in three dimensions with some measure of health

effects for the 15 cities on the third axis. Can that be done? It stated that the C-R functions are fairly linear and it would be nice to see that.

Chapter 6 – Integrative Discu ssion of PM2.5-related Risks

6) We have developed an integrated discus sion of the PM2.5-related risk estimates which considers the results of the qualitative a nd quantitative treatment of uncertainty and variability together with the various national-scale assessments completed for the analysis

to support interpretation of the core risk estimates. As part of the integrative discussion, we also provide key observations that bear on policy-relevant risk-based questions.

a) To what extent does the Panel believe th at we have captured the key policy-relevant questions that can be addresse d by this risk assessment? I thought the policy-relevant questions were well stated.

b) We provide a set of key observations rela ted to estimates of risk associated with simulations of just meeting the current a nd alternative suites of standards. These observations are based not only on considera tion of trends in risk reduction across

alternative suites of standards and residual risk remaining afte r simulation of just meeting specific suites of standards, but also on addi tional factors that can impact risk (e.g., the role of annual and 24-hour design values, th e peakiness of PM2.5 distributions within a study area, and application of different rollback approaches). To what extent do the Panel

members believe that the obser vations presented in section 6.2 are well supported by the results of the analyses? Are there other observa tions that might be made that would help to address the policy-re levant questions identified at the beginning of the chapter? I thought the list of obser vations was appropriate.

c) Part of our interpretation of the core ri sk estimates presented in section 6.2 is our

characterization of confidence in the core risk estimates a nd in observations made based

on those estimates. These assessments of c onfidence are based on consideration of the results of the sensitivity analysis as well as on the qualitative asse ssment of uncertainty and variability. To what extent does the Pa nel believe that the characterizations of

confidence in the core risk estimates and associated policy-related observations are reasonable given available information? The confidence statements were well presented.

d) As part of the integrative discussion, we use the results of several national-scale analyses (i.e., the national sc ale PM2.5 mortality analysis, th e representativeness analysis, and the new exploration of design values and patterns of PM2.5 monitoring data presented in section 4.5) to place the results of the risk assessment in a broader national-context. What are the Panel members’ views on appropriateness of this effort to place results of the analysis in a national context? I thought it was a good idea.

19e) We conclude chapter 6 with a list of ke y observations. Does the Panel believe that we have appropriately highlighted key findings of the risk asse ssment in these observations? Of particular note is the observa tion that, while alternative 24- hour standard levels can be used to reduce annual-average PM2.5 concentrations and thus to reduce estimated risk, the results are likely to be hi ghly variable across urban areas. More consistent lowering of annual-average PM2.5 concentrations across st udy areas, and thus more consistent reductions in estimated risk, may result from application of altern ative annual standard

levels. We also note that simulation of the a lternative 24-hour standa rd level of 25 μg/m3 resulted in reductions in annual-average PM2.5 levels for some study areas that were well

below the lowest annual standard level assessed (i.e., below 12 μg/m3). As a consequence, we observed risk reductions re flecting these changes in annual-average PM2.5 levels below 12 μg/m3. Given these result s, does the Panel beli eve that there is utility in estimating risks for alternative annual standard levels below 12 μg/m3? Definitely not. I think there is too much uncertainty to do that with much

confidence.

20Comments from Dr. Philip Hopke

In general, there are no major problems with this assessment. It is generally following the approaches that we have now seen in multiple rounds of review.

1) Air quality inputs (section 3.2): We have expanded the consideration of alternative approaches to simulating just meeting th e current and alternative suites of PM 2.5 standards (i.e., rollback approaches) to include a peak shaving appro ach, in addition to the hybrid and proportional approaches considered in the first draft assessment. This peak shaving approach is intended to represent more loca lized, rather than regional, patterns of PM 2.5 reductions (discussed in section 3.2.3.3). a) To what extent does the Panel believe th at the use of the peak shaving approach provides useful additional exploration of variability associated with how ambient PM 2.5 concentrations are simulated to change upon just meeting the current and alternative

suites of standards?

It does not seem useful to me since it is un likely that there are obvi ous sources that could

be controlled that would let one shave peaks in practice. Thus, it provides unrealistic scenarios that really do not c ontribute useful information to the assessment. The other rollback approaches seem fine and it is hard to see a role for the peak shaving approach.

b) We have used comparisons of composite monitor annual averages generated using the different rollback approaches as a surrogate for differences in long-term exposure-related mortality in looking across all three rollback approaches. To what extent does the Panel believe that this is a reasonable approach for assessing the impact of variability associated with simulating changes in air quality patterns on estim ates of long-term exposure-related mortality?

It would make the compositing easier to follow if equations were provided. If you do not wish to put them into the main chapter, then an appendix can be added to guide the

interested reader th rough exactly how all of the cal culations were performed.

I agree with the approach to removing quarter s in which too few values are reported to provide a reliable estimate of the average concentration during that period. However, it makes little sense to replace missing values wi th a mean value. If you are going to attempt to impute missing values, then a much more sophisticated approach should be employed using other monitors in the area as well as historic data for similar meteorological conditions. I would suggest that for those quarters where there are missing values, but a sufficient number of values to provide a valid mean value, then that

should be the value used in the health analyses.

2) Selection of model inputs (section 3.3): We have expanded and clarified the discussion of our rationale for identifying modeling choices comprising the core risk model, focusing in particular on selection of C-R functions (section 3.3.3). To what extent does

the Panel consider this discussion to be clear and the model se lections appropriate?

213) Addressing uncertainty and variability (s ection 3.5): We have clarified the process used to evaluate sources of variability and added coverage for specific sources of variability (section 3.5.2); expanded our disc ussion of the qual itative analysis of uncertainty (section 3.5.3); and in cluded analyses of pair-wise interactions of sources of uncertainty (section 3.5.4). To what extent does the Panel consider these discussions to be clear and appropriate?

It is time to move toward stochastic uncer tainty analysis. Every time we get a risk assessment, we continue to get the qualitat ive review of uncertainties and some hand waving. If there are insufficient resources available to do the full stochastic risk assessment, then the Agency should state this clearly so that it can be made clear why they have chosen not to move to a more comp lete analysis. Otherw ise, we should not see them continue to duck this approach.

22Comments from Dr. Morton Lippmann

Overall Comments: The authors have been very responsive to the prior CASAC PM Panel comments and suggestions, and the 2 nd draft provides a thorough and well-prepared presentation of realistic quantitative estimates of th e effects of long-term ambient air PM 2.5 exposures on premature mortality in 15 US urban areas, th eir variability and uncertainties, and their representativeness of the US population as a whole. The alte rnate risk assessments (RAs) for recent years’ concentrations, meeting the current suite of PM 2.5 NAAQS, and alternate NAAQS (14, 13, and 12 u g/m 3 annual and 30 and 25 u g/m 3 daily, and combinations thereof) is an appropriate way to tee up th e selection options for the Policy Assessment (PA).

Having offered a strong endorsement of the overall work of the OAQP S in preparing the RA document, I need to raise two issues that I have raised before that have not been properly addressed. These are:

1) The fact that the ACS cohort is not a representa tive US population, but rather is of higher SES. Within this cohort, the mort ality risk is lower for those of higher SES. Thus, the risk coefficient for a repr esentative population would be higher, as

it is for the 6-cities cohort that was selected to be more representative (at least for

cities in the eastern half of the US). 2) The description and use of the data from the Ito et al. (2007) study has not been corrected. As noted below, this was a study covering all of New York City (Kings, Queens, New York, Bronx, and Richmond Counties), not just New York County (Borough of Manhattan).

Specific Comments on Text Entries:

Page Line Comment

3-13 Table 3.1. The entry for New York City is incorrect. Change New York City (Manhattan)” to “New York County (Manhattan)”.

3-13 6 Change “New York (Manhattan)” to “New York City ”.

3-30 11 Delete “(Manhattan)”.

3-31 7 The number of members of the ACS cohort in the 156 MSAs with air quality data is much smaller than 1.2 million.

3-40 Counties Column for “New York”. Change New York City (Manhattan)” to “New York County (Manhattan)”, and delete “New York City (Manhattan)” where it precedes “Ito et al. (2007)”. The Ito et al. (2007) study covered a ll of NYC, not just Manhattan!

233-50 Table 3-10. There should not be two en tries for New York, NY. As noted above, the Ito et al. (2007) study c overed all of NYC, not just Manhattan! The second New York entry, which shows only the population of New York County (Manhattan), should be deleted, and the calculations for the Ito et al. study need to be revised to reflect the whole NYC population a nd all of the NYC PM 2.5 monitors.

3-77 Section 3.54.3. This section refers to si ngle- and multiple-elements, as well as to single- and multiple-factors, in terms of sensitivity analyses. There is no definition provided as to what constitutes an element or a factor. I suggest that element is a poor choice in terms of terminology, si nce it implies, at least to me , a chemical element. This will become important in the next round of PM NAAQS review. [On page 4-42, there is a clarification indicating that elements refers to “modeling elements”]

4-19 10 Change “fig” to “fit”.

4-19 11 Change “greater” to “great”.

4-36 12 Change “CPD” to “CVD”.

4-45 Note at bottom of the page: Change “Figures 4-6 and 4-7” to “Figure 4-8”.

4-47 29 Change “Figures 4-6 and 4- 7” to “Figures 4-7 and 4-8”.

4-48 para. 2 : Once again, there are elements and factors with no definitions. Here the elements are not modeling elements, but some thing else. [On page 4-54, line 21, I learned

that these were “critical risk function elements”]. Please reword to avoid confusion!

4-55 23 Change “population” to “populated”.

6-1 31 Delete extra period at the end.

6-14 25 Change “Pitts” to “Pittsburgh”.

6-18 15,16 Where can we find these estimates?

6-18 21-23 Where can we find these estimates?

6-21 8 Change “PM2.5” to “PM 2.5 ”.

Charge Questions to the CASAC PM Review Panel - focus on the charge questions

listed below in review of the Quantitative Health Risk Asse ssment for Particulate Matter

Chapter 3 – Urban Case Study Analysis Methods

24Charge Question #2: Selection of model inputs (section 3.3):

We have expanded and clarified the discussi on of our rationale for identifying modeling choices comprising the core risk model, fo cusing in particular on selection of C-R functions (section 3.3.3). To what extent does the Panel consider this discussion to be

clear and the model sele ctions appropriate?

Response: The Panel commends the authors for expa nding and clarifying their rationale for identifying modeling choices comprising the core risk model in a logical and satisfactory manner. Their model selections were appropriate

Chapter 6 – Integrative Discussion of PM 2.5 -related Risks

Charge Questions #6a-e: We have developed an inte grated discussion of the PM 2.5 -related risk estimates which considers the results of the qualitative a nd quantitative treatment of uncertainty and variability together with the various national-scale assessments completed for the analysis to support interpretation of the core risk estimates. As part of the integrative discussion, we also provide key observations that bear on policy-relevant risk-based questions.

Charge Questions #6a : To what extent does the Panel be lieve that we have captured the key policy-relevant questions that can be addressed by this risk assessment?

Response: The Panel considers that the authors have cap tured the key policy-rele vant questions that can be addressed by this risk asse ssment that is focused solely on PM 2.5

.

Charge Questions #6b: We provide a set of key obser vations related to estimates of risk associated with simulations of just meeting the current a nd alternative suites of standards. These observations are based not only on considera tion of trends in risk reduction across

alternative suites of standards and residual risk remaining afte r simulation of just meeting specific suites of standards, but also on addi tional factors that can impact risk (e.g., the

role of annual and 24-hour de sign values, the peakiness of PM 2.5 distributions within a study area, and application of di fferent rollback approaches).

To what extent do the Panel members believe that the observations presented in section 6.2 are well supported by the results of the analyses? Are there ot her observations that might be made that would help to address th e policy-relevant questi ons identified at the beginning of the chapter?

Response: The Panel considers that the observations pres ented in section 6.2 are consistent with the analytical results, and provide all of the info rmation needed in the development of the PA

document.

25Charge Questions #6c: Part of our interpretation of the core risk estimates presented in

section 6.2 is our characteriza tion of confidence in the core risk estimates and in

observations made based on those estimates. These assessments of confidence are based

on consideration of the results of the sensitivity analysis as well as on the qualitative

assessment of uncertain ty and variability.

To what extent does the Panel believe that the characterizatio ns of confidence in the core risk estimates and associated policy-relate d observations are reas onable given available

information?

Response:

The Panel considers that the characterizations of confidence in the core risk estimates and associated policy-related observations are reasonable.

Charge Questions #6d: As part of the integrative discussion, we use the results of several national-scale analyses (i.e., the national scale PM 2.5 mortality analysis, the representativeness analysis, a nd the new exploration of desi gn values and patterns of

PM 2.5 monitoring data presented in section 4.5) to place the results of the risk assessment

in a broader national-context.

What are the Panel members’ views on appropria teness of this effort to place results of the analysis in a national context?

Response: The Panel considers that the effort to place re sults of the analysis in a national context was appropriate.

Charge Questions #6e: We conclude chapter 6 with a list of key observations. Does the Panel believe that we have appropr iately highlighted key findings of the risk

assessment in these observations?

Response: The Panel considers that Staff has appropria tely highlighted key findings of the risk

assessment in these observations

Of particular note is the obser vation that, while alternative 24 -hour standard levels can be used to reduce annual-average PM 2.5 concentrations and thus to reduce estimated risk, the results are likely to be highly variable acro ss urban areas. More c onsistent lowering of annual-average PM 2.5 concentrations across study areas , and thus more consistent reductions in estimated risk, may result from application of altern ative annual standard

levels. We also note that simulation of th e alternative 24-hour standard level of 25 μ g/m 3 resulted in reductions in annual-average PM 2.5 levels for some study areas that were well

below the lowest annual standard level assessed (i.e., below 12 μ g/m 3 ). As a consequence, we observed risk reductions re flecting these changes in annual-average

PM 2.5 levels below 12 μ g/m 3

.

26Given these results, does the Panel believe th at there is utility in estimating risks for

alternative annual sta ndard levels below 12 μ g/m 3 ?

Response: The Panel does not consider that estimating risks for alternative annual standard levels

below 12 μ g/m 3 is a worthwhile endeavor on severa l grounds. One, the extrapolation of

the risk coefficients at such concentrati ons becomes increasingly uncertain; and two, the prospects of achieving compliance with such levels in the foreseeable future is vanishingly small.

27Comments from Dr. Robert Phalen

General Comments: The second draft of “Quantitative Health Ri sk Assessment for Part iculate Matter” is clearly-written and the logical development is well-described. The staff has done an excellent job. My enthusiasm for the entire document is diminished by factors that are apparently outside of policy-driven limitati ons imposed on EPA staff. These limitations

include:

• PM2.5 is a mass-based metric (indicator) fo r assessing health-effect s. It is likely that specific components, such as vanadium , nickel, and elemental carbon, are actually driving the health effects. The use of a mass-based indicator can lead to air-quality standards that do not permit appropriate abatements.

• Secondary health effects, e.g. as generate d by abatements that adversely affect the economy (and the many associated health eff ects), are not considered. Such indirect adverse health effects are real, and the a ffected populations must face them along with

the direct effects.

• The current risk assessment does not c onform to the recommendations made by the National Research Council of the National Academies (Science and Decisions: Advancing Risk Assessment, the National A cademies Press, Washington D.C., 2008). Specifically, ... “that risk assessment should be viewed as a method for evaluating the relative merits of various options for managing risk rather than as an end in itself.” The concept of managing risk must include all of the significant risks associated with a particular decision, not just some selected direct effects associat ed with the decision.

• National air quality standards have th e disadvantage of forcing abatements on regions of the U.S. for which the abatements may harm health to a greater extent than they improve health. As an example consider restrictions on diesel -emissions to try to meet standards in areas where a significant portion of PM2.5 can be found in soil fineparticle mass. Control of soil aerosols is no t practical, so the dies els must be targeted.

Specific Comments on Section 3.3

This reviewer found only minor errors.

Table 3-4, pg. 3-24: Replace “LA” with “Los An geles”, as LA is also the designator for

Louisiana.

Line 34, pg. 3-31: Drop either “the” or “our” for clarity.

28Comments from Dr. Ted Russell

Overall, I continue to be pl eased with the amount of analys is conducted in this PM Risk Assessment (RA), though I am still disappointed that a more comprehensive exposure modeling effort was not made. The document provides the type of in formation needed to inform the review of the primary NAAQS, and does s good job of providing a quantitative assessment of the potential ri sks, and has done a commendable sensitivity analysis.

While I appreciate the addition of Chapter 6, it was not as effective as it might be. First, I was looking for Chapter 6 to be more integrativ e of the results from Chapter 4 and 5 as it is now more focused on Chapter 4. Second, I was looking for it to also integrate uncertainties in to the discussion to a greater degree. This is the point that they could bring in the issue of using the LML as the ze ro risk level, and how that might influence the overall interpretation of the results. Third, it was a bit repetitive, bringing up the issue of “peaky nature” and explaining it more often than necessary. It is an important concept, but it was overdone in Chapter 6. St ill, Chapter 6 is a good addition, but it could be made stronger in response to the above comments.

Chapter 3 Charge Questions:

1) Air quality inputs (section 3.2): We have exp anded the considerat ion of alternative approaches to simulating just meeting th e current and alternative suites of PM 2.5 standards 5 (i.e., rollback approaches) to in clude a peak shaving approach, in addition to the hybrid and proportional approaches consid ered in the first dr aft assessment. This peak shaving approach is intended to repres ent more localized, rather than regional, patterns of PM 2.5 reductions (discussed in section 3.2.3.3).

a) To what extent does the Panel believe that the use of the peak shaving approach provides useful additional exploration of va riability associated with how ambient PM 2.5 concentrations are simulated to change upon just meeting the curre nt and alternative suites of standards?

Response: This approach, along with the hybrid approach, provide a reasonable method to get an estimate of the lower bound impact of what would happen when an urban area attains a specific standard. This approach is specifically of interest in areas where the 24- hour standard will be the driving standard, a nd the proportional roll-back would lead to reductions in the annual level beyond that wh ich might be viewed as likely. The method, either in the report or in the appendices, should be specified mathematically, as well as in words. They might present a specific set of example applications of the approaches, e.g., for three different cities, at this point.

b) We have used comparisons of composite monitor annual averages generated using the different rollback approaches as a surrogate for differen ces in long-term exposure29related mortality in looking across all thre e rollback approaches. To what extent does the

Panel believe that this is a reasonable approach for assessin g the impact of variability associated with simulating changes in air quality patterns on estim ates of long-term exposure-related mortality?

Response: First, the approach to compositi ng should be better explained. A critical question is how well it aligns with what is act ually done in reporting fo r the calculation of design values. Indeed, the approach they used should be motivated by first explaining how the annual average PM is found (provide specific equations as needed), and then showing that the approach taken is in line with how the annual average PM is now found. Also, they need to explain how and when compositing is used in calculating design values. This will explain why quarterly averag es are first calculated, and may impact my thoughts on how the current approach to compositing should be altered.

I do not like how they replace missing values , i.e., using the average value to replace missing values when a certain number of samp les are missing. It seems to be a bit arbitrary, and could lead to a bi as. Is this what is done in practice (i.e., specified)? In terms of compositing, the values could each be adjusted using a centering approach. In this case, the annual average from each monitor being used in the composite is subtracted from the daily value from that monitor, leadi ng to a string of values that have a mean of zero. These annual averages are also used to calculate the composite annual average of the stations being used in the composite. The daily values for each monitor (after subtracting the mean of that st ation) are then averaged as available. This leads to the average variation from the mean for that day. The annual average composite value is then added back to get the daily composite valu e. This is relatively insensitive to stations dropping out. Further, it should exactly give the observed annual mean at each station, and well as the composite mean. The curren t approach for imputing missing days can lead to an average that would not agree with the reported value. The same approach for calculating quarterly averages, that are then used to calcu late the annual average and design value, can be used.

Like my response to part (a), the mathematical equations should be provided here or in the appendix.

5) Consideration of design values and patterns of PM 2.5 monitoring data in interpreting core risk estimates (section 4.5): To enhance our interpretation of th e patterns of core risk estimates generated for both the current and alternative suites of standards, we have included analyses of 24-hour and annual design values together with patterns of PM 2.5 monitoring data for the 15 urban study areas. This reflects the fact t hat these two factors play a key role in determining the degree of risk reduction estimated upon just meeting the current and alternative suites of standards under alternative rollback approaches. As part of the consideration of design values, we have also contrasted the 15 urban study areas with patterns of design values seen fo r the broader set of urban areas in the U.S. in order to help place the urban study area in a broader national context

a) To what extent is the Panel suppo rtive of these additi onal assessments?

30Response: These are beneficial. Figure s 4-17 to 4-19, and Tabl e 4-8 do a good job of identifying which are the controlling standard s for each of the 15 cities, and also show that the cities examined do a good job of sp anning the space of conditions for cities around the US. It might have been good to al so identify some of the major outliers on Fig. 4-17 (there really is only one to identif y). Also, you could consider color coding the dots on the figure, such that each portion of the US is assigned a color, so it would be apparent if there were specific areas where an issue might identified. This would also help address why no Upper Midwest city is targeted. Without this, I don’t think this section really does as good job of contras ting the 15 urban study areas with other areas, that they are supposed to represent, particular ly at a regional level. Section 4.4 presents additional information putting othe r areas in to perspective, bu t what would be valuable is to show that all regions and sizes of cities are addressed adequately.

b) Does the Panel have any recommenda tions for additional insights based on consideration of patterns in design values and PM 2.5 monitoring data across the 15 urban study areas and at the national level?

Response: See above. I think this section could be strength ened by being able to have a concluding statement somewhat along the li nes of “We have captured the range of

conditions found for cities that would be in non-attainment of the various combinations

of standards in each of the six of the seven (I would have preferred all seven) regions of the country, and the 15 cities capture X% of the population. The analysis has identified cities that span the range of non-attainment levels, and whic h standard would require the greatest level of control based on th is analysis.” It is close to being able to say this.

Other details:

Page 3-8, Footnote: This f ootnote is not needed and I fi nd it confusing. Also, while it need not be added at this point, unless βΔ x is above about 0.2, the error in linearizing (3) and (4) is rather small. This might simp lify interpretation of later analyses since it makes the response to concen tration changes linear.

In Chapter 3, no city in the Upper Midwest was identified. Might this be alleviated?

31Comments from Dr. Frank E. Speizer

Pre-meeting Comments: 3/5/2010

General Comment: Staff has done an excellent job of using the av ailable literature and assessments from the ISA to generate a series of outcome assessments over an appropriate range of alternative levels of both 24 hour and annual averages of PM2.5 that are justified by the available data. They appear to have been response to our previous comments on the first draft, and where they have not followed our sugges tions have presented evidence that they considered our requests and accepted or re jected or modified with appropriate justification. The one area wh ere I would have like to have seen more analyses is with regard to PM10-2.5, simply to have demonstrat ed the “:inappropriate ness” of trying to rely on such an analysis would have emphasi zed the need for more data related to the

course fraction effects.

Specific Comments and Charge Questions Chapter 2 Page2.4, line 14. Typo: PM2.5

Page 2.6, line 2-4 Query the continuing de velopmental work on population exposure analysis methodology. It would have been useful to read or hear further what this plan would be. One would have thought that over th e years much of this would have already

been worked out.

Page 2.6, line 22-24. Appendix H provides a well reasoned argument for Staff concluding that they could not do a quantitative risk assessment for PM10-2.5. From my perspective

it would seem to me that some estimate of risk should be made if for no other reason than

to document wide confidence intervals a nd thus reason for not including it in the quantitative assessment.

Page 2.7, line 10-11. Perhaps at the end of this process a note could be sent to CASAC to indicate what the plan might be for the future to prepare for the next round by carrying out the methods development necessary to use “s pecialized analysis of risk...” Is this important enough for CASAC to include such a re quest in our letter to the administrator?

Page 2.12, line 13-20. The full set of model choices offered seems appropriate, given the plan is to maintain an annual as well as a 24 hour standard..

Chapter 3 Page 3.19-3.20 and Charge Question 1a and 1b. Use of peak shaving rollback approach as discussed is confusing. Staff indicat es (at top of page 3.20) that because of time

constraints they did not calculate health risks usi ng this method. They go on to indicate what the method does. The appendix that di scusses the method more fully refers to Tables F-49 and F-50. These tables present the curious finding that except for Fresno (where there is virtually no change) and Tacoma, all of the other cities show a progressive increase in maximum values go ing from the “proportional” to “hybrid” to

32“shaving”. This certainly does not seem to indicate that the shaving analyses were not

done. I t also raises for me concern that more effort is needed to understand which is a better technique or more repr esentative of “truth”. T hus I am worried that “time constraints may be once again getting in the way of what the proper analysis should be.

Section 3.3., Charge Question 2 asks about the discussion of rationale for modeling

choices and selection of CR function. The section follows a logical rationale and provides appropriate documentation of both model and site selection.

Page 3.49, Section 3.4.1.3. It might be worth adding a sentence that approximately 52 x

10 6 people are represented in the 15 cities or about 20% of the to tal population of US (even if not a representative sample).

Page 3.53-54, Table 3.11. there is far too much missingness in this table to make it at all useful. It raises more questions about the quality of the health input than would be justified. The lack of COPD for all site s but LA just doesn’t compute for the 4 th major

cause of death in the US. W hy are there not values for All Ages for All Causes? The data are presented for some but not all categories of disease. Someone needs to take the time to get all these cells filled in. These data a ll exist at the Federal level in one book o(or on one web site)!

Charge Question 3, Uncertainty and Variability. The discussion is quite complete dealing with a number of issues. What is not fully c onsidered is the role of the potential for nonrandom missingness in both site se lection within the 15 cities a nd thus the selection of the

cities themselves.

Chapter 4 Charge Question 4-Sensitivity Analysis Section 4.3, Table 4.1, page 4.6. This table need s to define denominators for Incidence (in title). Assuming both are the same then in general hospitalization considerably more frequent than mortality. (My concern is mortality may be x/100,000 and morbidity might be y/10,000 and if this is the cas e there are strikingly different numbers that need to be discussed more fully).

Page 4.17, line 6. Need to indicate de nominators for these incidence rates.

Page4.46 Figures 4.7 &4.8. I have difficulty in interpreting these fi gures. For HD total incidence for LA go from 5-10% to as high as 19%. For Philadelphia from10-15%

to14-15% with outlier at 23%. For total mortal ity the variation are separately 2-6% and

3-4 to 8%. For both these cases the core an alysis seems too high and the delta seems too big, as tests of sensitivity. Is this the wrong interpretation?

Page 4.47, line 29. Should this be figu res 4.7 and 4.8 rather than 4.6 and 4.7? Page 4.54, line 5. This should be changed to Table 4.7 (page 4.56). Line 21. Figure numbers seem not to matc h up with text being discussed. Please

check.

33Page 4.67-4.69, Figures 4.17-4.19. These are ve ry effective presentations of design values. I suggest either change titles or text numbers so that 24 hour/annua l are presented consistently in both.

Charge Question 5a. Design values analysis and assessments. These analyses are quite useful, as they are described. They provide vi sual interpretations dire ctly of the range of

effects related to the alternative standards. Notably in Chapter 3 the suggestion is made tht the use of peak rollback shaving is not to be considered; however, it looks like it is being used here.

Charge Question 5b. Additional suggestions. I think the ranges of outcomes are well demonstrated in figures 4.20 and 4.21. Rather th an simply ending with the description of how they were constructed it might be wort h adding a paragraph or two that summarize the degree to which the various alternative w ould provide changes in some fraction or all of the cities, rather than just ending with selected examples (maybe this will happen in Chapter 6).

Chapter 5 This chapter leaves me uneasy and I would like to suggest we spend some time discussing it. It look to me th at it is a straight forward extr apolation of results from the two large cohort studies (ACS and 6 Cities). Th e level of exposure are extracted from the 15 urban areas and then scaled up to the US. This leads to ~88,000 (4-8% of total mortality—a figure repeated in Chapter 6). However, this seems too high. At one point there is a suggestion that the figures come from upper end of urban mortality risk, but

isn’t the calculation for the whole nation where exposure levels must be a lot lower? Lets

discuss!

Chapter 6 Page 6;.7, line 25. Typo change “if “to “of”

General Comment: The integrated discussion pulls together and highlights some of the specific details presented in Chapter 4 and the Appendices. It reflects the obvious and

not so obvious conclusions that results fr om manipulating the various alternative scenarios between current existing exposure, estimates to current NAAQS levels and the various alternatives proposed. In spite of th e evidence that there does not appear to be a

threshold the lowest level assessed 12/25 seem s appropriate in that the evidence of an effect below those levels is simply too un certain to evaluate. On the other hand the question of margin of safety remains and it wi ll need to be argued th at any level chosen (above that level) will need to be defended as to whether there is adequate margin of safety.

With regard to the Charge Questions:

a. Staff has done an excellent job in presen ting and capturing the key-policy relevant questions. However, as indicated early on in this document there was to be a qualitative discussion on PM10-2.5 and on those effects that were deemed only

34“suggestive” but might have important public health implications (e.g. lung cancer, reproductive effects), but for whic h quantitative risk assessment was not thought warranted but that would appear in the PA (page 2.6). I would have thought that some remarks in this chapter would be necessary to assure that the

PA would discuss the issues.

b. Key observations are presented and discu ssed with adequate discussion of the

relevant contribution of the role of an nual and 24 hour design values and the role of “peakiness” of distributions. One obs ervation that appears to be focused upon and may be a driving force is the uneve n distribution among the 15 urban sites on the impact of the various scenarios and whet her this fact is sufficiently taken into account in scaling up for the national estimat es. More discussion and or analyses on this point may be warranted. For exam ple what role does the actual estimates from these 15 sites play is coming to the estimates of 3-9% excess mortality? It may be too much to expect (in sp ite of the statistics) that 63,000-88,000 premature deaths would be prevented. Part of the country is already well below the proposed alternative levels and thus would not contribute to lives saved. Are

there additional alternative sensitivit y analyses that would provide either alternative estimates or put more confidence in these estimates by taking into account better population we ighted C-R analyses?

c. See above. In spite of the last comment, the uncertainties and variability of the core assessments seems to be as good as it can be.

d. Evaluation of the several national scale an alyses, as indicated above is of some

concern. If I read the Tabl es in Appendix E correctly, th e effect of moving to the lowest alternative (25/12) in some cas es within the 15 urban sites produces a range of 32-67% (with one outlier at 11% and one at 100%) reduction in the IHD compared to the current standard. The quest ion is, is this the best baseline for the comparison or should it be the current recent measurements, which would drop the percent changes considerably (and pe rhaps provide a more realistic estimated of the potential benefits from implementing changes). Obviously, the proportional ranking and changes would be the same, but the impact on “lives saved” on a national scale might be cons iderably less and more realistic.

e. Key observations seem to be presented in a balanced and fair way. Although the national assessment suggests a range of 63,000-88,000 premature deaths per year attributable to PM2.5 does not jive with a fairly often quoted figure from 2006 that moving the annual standard from 15 to 14 ppm would result is “more lives than perished in 9/11”. (That figure tr anslated into about 3000 lives.) Staff acknowledges that the rage of effects are in two categor ies: 3-9% and 0-3% in two halves of the country. This may be le ss precise than what the data indicate in that it would appear from their own estimat es that the bulk of the effect comes fro the upper end of the exposure in the counties (pg 5.8, line 12-15).

35Comments from Dr. Helen Suh

The Second Draft of the Risk Assessment is a clear and comprehensive presentation of the rationale, methods and results for the assessment th e acute and chronic PM-mediated health risks. In this draft, the authors ha ve thoughtfully considered and incorporated the comments and suggestions from the CASAC panel. As an overall comment, the rationale that was used in the decision to forego a risk assessment for PM 10-2.5 made logical sense. Further, inclusion of a discussion of PM 10-2.5 exposure and risk in the upcoming draft Policy Assessment document is welcomed. However, it still seems that a qualitative and

indirect assessment of PM 10-2.5 risks could be made based on PM 10 monitoring and health data for metropolitan areas where PM 10-2.5 concentrations comprise a large fraction of PM 10

. While certainly not ideal, this qualitative assess ment may provide valuable information about

PM 10-2.5 risks

.

Charge Question 2 : Selection of model inputs (section 3.3): We have expanded and clarified the discussion of our rationale fo r identifying modeling choices comprising the core risk model, focusing in particular on selection of C- R functions (section 3.3.3). To

what extent does the Panel cons ider this discussion to be clear and the model selections appropriate?

The revisions to section 3.3 re present a significant improveme nt over the previous draft,

with the selection rationale for the core ri sk model clearly and cogently presented and previous concerns addressed. The rationale provided is a thoughtful and sensible approach to assess particle-mediated health ri sks. Further, the su mmary tables (Table 3.5 - 3.8) provide a useful and nice synopsis of the model inputs fo r the core risk models and sensitivity analyses. As a very minor comme nt, it might be possible to condense Table 3.7 somewhat by replacing certain columns with check boxes instead of text (for example to indicate short-term or long-term).

Charge Question 3: Addressing uncertainty and variability (section 3.5): We have clarified the process used to evaluate s ources of variability and added coverage for specific sources of variability (section 3.5.2 ); expanded our discussion of the qualitative analysis of uncertainty (section 3.5.3); and includ ed analyses of pair-wi se interactions of sources of uncertainty (section 3.5.4). To what extent does the Panel consider these discussions to be clear and appropriate?

The discussions of uncertainty and variabil ity were clear and t houghtful, representing a substantial improvement over the previous draft. The sources of variability and uncertainty were well represented, although so me explicit discussion of the impact of seasonality on risks should be included give n its effects on each of the other discussed

sources.

36Comments from Dr. Sverre Vedal

PM Risk Assessment, Charge Question 6.

General:

1. There is a too much formulaic and detail ed presentation of results, for example when presenting findings of just meeting the curr ent standards and just meeting alternative

standards. This seems out of place in an integrative discussion. 2. While there is some opinion that effect s of long-term exposure on IHD mortality (as opposed to all-cause cardiovascular mortalit y) are especially st rong, this is based on relatively weak evidence from the ACS cohort (Pope 2004), in my opinion. It is therefore not clear why it was elected to pres ent risk assessment resu lts largely for IHD in this discussion (p. 6-6) and in chapter 4 of the RA. On second thought, it is clear because all-cause CVD mortality effects were not presented in Krewski 2009, unfortunately, and Krewski 2009 was chosen to provide the most defensible effect estimates. 3. The remaining percent PM-attributa ble effect of long-term exposure on total mortality is presented (p. 6-7, line 18; p. 6-8, line 8 and lines 14 & 15). These are ridiculously high. Presumably this should be on IHD mortality. The first bullet under the first Key Observation (p. 6-18) gets it right, I believe. 4. I question whether short-term exposure-rela ted risk (p. 6-5, line 3) is also driven by changes in long-term average PM concentr ations. Short-term effects are observed independent of long-term PM concentra tion, ie, down to the lowest baseline

concentrations. Therefore the argument in this paragraph fo r motivating attention to the annual average is not sound.

6.a. Key questions captured?

Yes

6.b. Role of additional factors? Observations in 6.2 are well-supported. No other observations are apparent.

6.c. Characterization of confidence. Again (see above), the emphasis on IHD morta lity as opposed to all-cause cardiovascular mortality reduces our confidence in eff ect estimates; unfortunately, all-cause cardiovascular effect estimates are not provi ded in Krewski 2009. IHD effect estimates are highest in the ACS cohort. Otherw ise, estimates are conservative.

6.d. National context. I’m not sure I agree that the 15 cities capture “the overall distribution of risk for the nation,” (p.6-16, line 36) given the fact that these are the more polluted cities in the

nation. More correct is the c ontention that these cities refl ect the experience of cities with relatively elevated levels of PM-related risk and attrib utable mortality. Both can’t

be true.

3738 6.e. Key observations.

These observations are fine. I se e little utility in estimating risk below an annual standard of 12 mcg/m 3

.