EPA is required by the Clean Air Act to conduct reviews of the National Ambient Air Quality Standards (NAAQS) for the six criteria pollutants, including particulate matter, every 5 years. The overarching purpose of such reviews is to determine whether the current standards are sufficient to protect public health and welfare at large, with an adequate margin of safety, given the latest scientific information available at the time of the review. Major steps in the NAAQS process include the following: developing a criteria document that synthesizes new research on health preparing a staff paper that assesses the policy implications of the scientific information in the criteria document, which also discusses possible ranges for air quality standards; and determining whether and how EPA should revise the NAAQS. If EPA decides to revise the NAAQS, the agency proposes the changes in the Federal Register. As part of the federal rule-making process, EPA is to comply with Executive Order 12866, which directs federal agencies to analyze the costs and benefits of proposed and final rules expected to affect the economy by $100 million or more per year. In September 2003, the Office of Management and Budget (OMB) issued its Circular A-4, which presents guidance and best practices and states that agencies should analyze the costs and benefits in accordance with the principles of full disclosure and transparency. Further, in cases such as the particulate matter rule, where expected economic impacts exceed $1 billion annually, Circular A-4 also states that agencies should conduct a comprehensive assessment of key uncertainties in their analyses of costs and benefits, which EPA also refers to as regulatory impact analyses. EPA’s January 2006 regulatory impact analysis presents estimates of the costs and benefits for the proposed particulate matter rule. The focus of the National Academies’ 2002 report was on how EPA estimates the health benefits of its proposed air regulations. To develop such estimates, EPA conducts analyses to quantify the expected changes in the number of deaths and illnesses that are likely to result from proposed regulations. The regulatory impact analyses also estimate the costs associated with implementing proposed air regulations, although, under the Clean Air Act, EPA is not permitted to consider costs in setting health- based standards for the criteria air pollutants, such as particulate matter. Soon after the National Academies issued its report in 2002, EPA staff identified key recommendations and developed a strategy, in consultation with OMB, to apply some of the recommendations to benefit analyses for air pollution regulations under consideration at the time. EPA roughly approximated the time and resource requirements to respond to the recommendations, identifying those the agency could address within 2 or 3 years and those that would take longer. According to EPA officials, the agency focused primarily on the numerous recommendations related to analyzing uncertainty. Both the National Academies’ report and the OMB guidance emphasize the need for agencies to account for uncertainties and to maintain transparency in the course of conducting benefit analyses. Identifying and accounting for uncertainties in these analyses can help decision makers evaluate the likelihood that certain regulatory decisions will achieve the estimated benefits. Transparency is important because it enables the public and relevant decision makers to see clearly how EPA arrived at its estimates and conclusions. In prior work on regulatory impact analyses, we have found shortcomings in EPA’s analyses of uncertainty and the information the agency provides with its estimates of costs and benefits. EPA applied—either wholly or in part—approximately two-thirds of the Academies’ recommendations to its January 2006 regulatory impact analysis and continues to address the recommendations through ongoing research and development. The January 2006 regulatory impact analysis demonstrated progress toward an expanded analysis of uncertainty and consideration of different assumptions. EPA officials cited time and resource constraints, as well as the need to mitigate complex technical challenges, as the basis for not applying other recommendations. According to EPA officials, the agency did not apply some of the more complex recommendations because it had not achieved sufficient progress in the research and development projects under way. The January 2006 regulatory impact analysis on particulate matter represents a snapshot of an ongoing EPA effort to respond to the National Academies’ recommendations on developing estimates of health benefits for air pollution regulations. Specifically, the agency applied, at least in part, approximately two-thirds of the recommendations—8 were applied and 14 were partially applied—by taking steps toward conducting a more rigorous assessment of uncertainty for proposed air pollution regulations by, for example, evaluating the different assumptions about the link between human exposure to particulate matter and health effects and discussing sources of uncertainty not included in the benefit estimates. According to EPA officials, the agency focused much of its time and resources on the recommendations related to uncertainty. In particular, one overarching recommendation suggests that EPA take steps toward conducting a formal, comprehensive uncertainty analysis—the systematic application of mathematical techniques, such as Monte Carlo simulation— and include the uncertainty analysis in the regulatory impact analysis to provide a “more realistic depiction of the overall uncertainty” in EPA’s estimates of the benefits. A number of the other recommendations regarding uncertainty are aimed at EPA’s developing the information and methodologies needed to carry out a comprehensive uncertainty analysis. Overall, the uncertainty recommendations suggest that EPA should determine (1) which sources of uncertainties have the greatest effect on benefit estimates and (2) the degree to which the uncertainties affect the estimates by specifying a range of estimates and the likelihood of attaining them. In response, EPA devoted significant resources to applying an alternative technique called expert elicitation in a multiphased pilot project. The pilot project was designed to systematically obtain expert advice to begin to better incorporate in its health benefit analysis the uncertainty underlying the causal link between exposure to particulate matter and premature death. EPA used the expert elicitation process to help it more definitively evaluate the uncertainty associated with estimated reductions in premature death—estimates that composed 85 percent to 95 percent of EPA’s total health benefit estimates for air pollution regulations in the past 5 years, according to the agency. EPA developed a range of expected reductions in death rates based on expert opinion systematically gathered in its pilot expert elicitation project and provided the results of this supplemental analysis in an appendix to the regulatory impact analysis. However, the National Academies had recommended that EPA merge such supplemental analyses into the main benefit analysis. Moreover, the Academies recommended that EPA’s main benefit analysis reflect how the benefit estimates would vary in light of uncertainties. In addition to the uncertainty underlying the causal link between exposure and premature death that EPA analyzed, other key uncertainties can influence the estimates. For example, there is uncertainty about the effects of the age and health status of people exposed to particulate matter, the varying composition of particulate matter, and the measurements of actual exposure to particulate matter. EPA’s health benefit analysis, however, does not account for these key uncertainties by specifying a range of estimates and the likelihood of attaining them, similar to estimates derived from the expert elicitation addressing causal uncertainty. For these reasons, EPA’s responses reflect a partial application of the Academies’ recommendation. In addition, the Academies recommended that EPA both continue to conduct sensitivity analyses on sources of uncertainty and expand these analyses. In the particulate matter regulatory impact analysis, EPA included a new sensitivity analysis regarding assumptions about thresholds, or levels below which those exposed to particulate matter are not at risk of experiencing harmful effects. EPA has assumed no threshold level exists—that is, any exposure poses potential health risks. Some experts have suggested that different thresholds may exist and the National Academies recommended that EPA determine how changing its assumption—that no threshold exists—would influence the estimates. The sensitivity analysis EPA provided in the regulatory impact analysis examined how its estimates of expected health benefits would change assuming varying thresholds. Another recommendation that EPA is researching and partially applied to the draft regulatory impact analysis concerns alternative assumptions about cessation lags—the time between reductions in exposure to particulate matter and the health response. The National Academies made several recommendations on this topic, including one that EPA incorporate alternative assumptions about lags into a formal uncertainty analysis to estimate benefits that account for the likelihood of different lag durations. In response, EPA has sought advice from its Advisory Council on Clean Air Compliance Analysis on how to address this recommendation and has conducted a series of sensitivity analyses related to cessation lags. EPA is also funding research to explore ways to address lag effects in its uncertainty analysis. According to an EPA official, specifying the probability of different lag effects is computationally complex, and the agency is working to resolve this challenge. In response to another recommendation by the National Academies, EPA identified some of the sources of uncertainty that are not reflected in its benefit estimates. For example, EPA’s regulatory impact analysis disclosed that its benefit estimates do not reflect the uncertainty associated with future year projections of particulate matter emissions. EPA presented a qualitative description about emissions uncertainty, elaborating on technical reasons—such as the limited information about the effectiveness of particulate matter control programs—why the analysis likely underestimates future emissions levels. EPA also applied the Academies’ recommendation on the presentation of uncertainty, which encouraged the agency to present the results of its health benefit analyses in ways that convey the estimated benefits more realistically by, for example, placing less emphasis on single estimates and rounding the numbers. EPA’s regulatory impact analysis presented ranges for some of the benefit estimates. Also, EPA sought to convey the overall uncertainty of its benefit estimates in a qualitative manner by clearly stating that decision makers and the public should not place significant weight on the quantified benefit estimates in the regulatory impact analysis because of data limitations and uncertainties. Another example of EPA’s response to the National Academies’ recommendations involves exploring the various regulatory choices available to decision makers. The Academies recommended that EPA estimate the health benefits representing the full range of regulatory choices available to decision makers. In the particulate matter analysis, EPA presented health benefits expected under several regulatory options targeting fine particulate matter. Citing a lack of data and tools needed to conduct an accurate analysis, EPA did not estimate the benefits expected under the proposed regulatory options for coarse particulate matter but, consistent with the National Academies’ recommendation, presented its rationale for not doing so. Overall, we considered this a partial application of the recommendation. (See app.