Particulate Matter: EPA Has Started to Address the National Academies' Recommendations on Estimating Health Benefits, but More Progress Is Needed
Published: 2006-07-17T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: A large body of scientific evidence links exposure to particulate matter--a widespread form of air pollution--to serious health problems, including asthma and premature death. Under the Clean Air Act, the Environmental Protection Agency (EPA) periodically reviews the appropriate air quality level at which to set national standards to protect the public against the health effects of particulate matter. EPA proposed revisions to these standards in January 2006 and issued a draft regulatory impact analysis of the revisions' expected costs and benefits. The estimated benefits of air pollution regulations have been controversial in the past. A 2002 National Academies report generally supported EPA's approach but made 34 recommendations to improve how EPA implements its approach. GAO was asked to determine whether and how EPA applied the Academies' recommendations in its estimates of the health benefits expected from the January 2006 proposed revisions to the particulate matter standards. GAO examined the draft analysis, met with EPA officials, and interviewed members of the National Academies' committee. In providing technical comments on the report, EPA officials said it was fair and balanced and noted the agency's progress in addressing recommendations via research and development and other analyses.
section_title: What GAO Found
paragraphs: EPA has begun to change the way it conducts and presents its analyses of health benefits in response to recommendations from the National Academies. Specifically, EPA applied, at least in part, 22--or about two-thirds--of the Academies' recommendations to its health benefit analysis of proposed revisions to particulate matter standards. For example, in response to some of the recommendations, EPA took steps toward conducting a more rigorous assessment of uncertainty by, for instance, evaluating how benefits could change under different assumptions and discussing sources of uncertainty not included in the benefit estimates. In one case, EPA applied an alternative technique, called expert elicitation, for evaluating uncertainty by systematically gathering expert opinion about the uncertainty underlying the causal link between exposure to particulate matter and premature death. Consistent with the National Academies' recommendation to assess uncertainty by developing ranges of estimates and specifying the likelihood of attaining them, EPA used expert elicitation to develop ranges of reductions in premature death expected from the proposed revisions. EPA officials said that ongoing research and development efforts will allow the agency to gradually achieve more progress in applying the recommendations. We note that robust uncertainty analysis is important because estimates of health benefits can be highly uncertain, as the draft regulatory impact analysis for particulate matter illustrates. EPA viewed the estimates in this analysis as so uncertain that it chose not to present them in the executive summary. For various reasons, EPA has not applied the remaining 12 recommendations to the analysis, such as the recommendation to evaluate the impact of using the simplifying assumption that each component of particulate matter is equally toxic. EPA officials viewed most of these recommendations as relevant to its health benefit analyses and, citing the need for additional research and development, emphasized the agency's commitment to continue to respond to the recommendations. For example, EPA did not believe that the state of scientific knowledge on the relative toxicity of particulate matter components was sufficiently developed to include in the January 2006 regulatory impact analysis, and the agency is currently sponsoring research on this issue. In addition, a senior EPA official said that insufficient resources impeded the agency's progress in applying the recommendations, citing, in particular, the limited availability of skilled staff, time, and other resources to conduct the required analyses and research and development. EPA officials also said that some of the recommendations the agency did not apply to the draft analysis, such as one calling for a summary table describing key analytical information to enhance transparency, will be applied to the analysis supporting the final rule. To the extent that EPA continues to make progress addressing the Academies' recommendations, decision makers and the public will be better able to evaluate the basis for EPA's air regulations.

section_title: Letter
paragraphs: A large body of scientific evidence links exposure to particulate matter—a  ubiquitous form of air pollution commonly referred to as soot—to serious  health problems, including asthma, chronic bronchitis, heart attack, and  premature death. The many sources releasing particulate matter into the  air include cars, trucks, power plants, industrial processes, forest fires, and  waste incinerators. In 1971, the Environmental Protection Agency (EPA)  first established national air quality standards to protect the public against  the health effects of particulate matter, one of the six widespread criteria  pollutants considered harmful to public health. Under the Clean Air Act,  EPA determines the appropriate level at which to set national air quality  standards, and the states must develop programs to achieve and maintain  compliance with them. Further, EPA must review the standards every 5  years to determine whether they adequately protect human health and  welfare, given the latest scientific information available, and revise them if  they do not.
In January 2006, after its most recent review of the national air quality  particulate matter standards, EPA proposed revisions to the standards and  issued a draft regulatory impact analysis. Overall, the draft regulatory  impact analysis discussed the scope and magnitude of the particulate  matter problem, the likely benefits of the proposed revisions for public  health and the environment, and the expected costs of implementing the  standards. Regarding public health, the analysis presented estimates of  expected health benefits for the particulate matter revisions in five major  urban areas, including reductions in the number of premature deaths and  emergency room visits for asthma. Among the changes EPA said it plans to  make in its final regulatory impact analysis is providing national estimates  of expected health benefits.  EPA is required under a court order to issue  the final rule on particulate matter standards by September 2006.
In 2000, at the direction of the Senate Appropriations Committee, EPA  asked the National Academies (Academies) to evaluate EPA’s methodology  for estimating the health benefits of proposed air pollution regulations.   According to the National Academies, these estimates have often been  controversial, and the methods EPA has used to prepare them have been  questioned. For example, some observers, such as researchers and  industry groups, have expressed concerns that EPA does not adequately  factor uncertainty into its estimates of health benefits. Some level of  uncertainty is unavoidable, in part because the scientific information used  to develop estimates, such as the inventory of particulate matter emissions,  will never be perfect or complete. However, according to some observers,  EPA’s estimates of benefits appear more definitive than they really are  because the agency does not adequately account for uncertainty in its  analyses or in its reporting of health benefit estimates.
The National Academies’ 2002 report on this subject generally supported  EPA’s approach to estimating health benefits but, nevertheless, made 34  detailed recommendations to improve how EPA implements its approach.   Overall, these recommendations focus on conducting more rigorous  assessments of uncertainty, increasing the transparency of how EPA  estimates benefits, conducting more detailed analyses of exposure, and  estimating the benefits of each regulatory option under consideration.  Many of the recommendations include qualifying language indicating that it  is reasonable to expect that they can be applied in stages, over time; and a  number of the recommendations are interrelated and, in some cases,  overlapping.
You asked us to determine whether and how EPA applied the National  Academies’ recommendations in its estimates of the health benefits  expected from the January 2006 proposed revisions to the particulate  matter standards. To respond to this objective, we reviewed EPA’s draft  regulatory impact analysis presenting the costs and benefits of the  proposed rule and met with senior officials from EPA’s Office of Air and  Radiation, which was responsible for developing the proposed rule and  analyzing its costs and benefits, and with officials from EPA’s Office of  Policy, Economics, and Innovation. We also reviewed EPA’s and the Office  of Management and Budget’s (OMB) guidance on conducting economic  analyses, prior GAO reports on EPA’s regulatory impact analyses, and other  relevant reports. As requested, our work addressed the application of the  National Academies’ recommendations to EPA’s draft regulatory impact  analysis supporting the 2006 proposed particulate matter rule; thus, we did  not examine how EPA applied the recommendations to other recent air  rules. Our work focused on broadly characterizing EPA’s progress toward  applying the recommendations; we did not evaluate the effectiveness and  quality of the scientific and technical actions the agency has taken to apply  the recommendations. See appendix I for a more detailed description of the  scope and methodology of our review. We performed our work from  January 2006 to July 2006 in accordance with generally accepted  government auditing standards.
subsections: section_title: Background
paragraphs: EPA is required by the Clean Air Act to conduct reviews of the National  Ambient Air Quality Standards (NAAQS) for the six criteria pollutants,  including particulate matter, every 5 years. The overarching purpose of  such reviews is to determine whether the current standards are sufficient  to protect public health and welfare at large, with an adequate margin of  safety, given the latest scientific information available at the time of the  review.  Major steps in the NAAQS process include the following:   developing a criteria document that synthesizes new research on health   preparing a staff paper that assesses the policy implications of the  scientific information in the criteria document, which also discusses  possible ranges for air quality standards; and  determining whether and how EPA should revise the NAAQS.
If EPA decides to revise the NAAQS, the agency proposes the changes in  the Federal Register. As part of the federal rule-making process, EPA is to  comply with Executive Order 12866, which directs federal agencies to  analyze the costs and benefits of proposed and final rules expected to  affect the economy by $100 million or more per year. In September 2003,  the Office of Management and Budget (OMB) issued its Circular A-4, which  presents guidance and best practices and states that agencies should  analyze the costs and benefits in accordance with the principles of full  disclosure and transparency. Further, in cases such as the particulate  matter rule, where expected economic impacts exceed $1 billion annually,  Circular A-4 also states that agencies should conduct a comprehensive  assessment of key uncertainties in their analyses of costs and benefits,  which EPA also refers to as regulatory impact analyses. EPA’s January  2006 regulatory impact analysis presents estimates of the costs and  benefits for the proposed particulate matter rule.
The focus of the National Academies’ 2002 report was on how EPA  estimates the health benefits of its proposed air regulations.  To develop  such estimates, EPA conducts analyses to quantify the expected changes in  the number of deaths and illnesses that are likely to result from proposed  regulations. The regulatory impact analyses also estimate the costs  associated with implementing proposed air regulations, although, under  the Clean Air Act, EPA is not permitted to consider costs in setting health- based standards for the criteria air pollutants, such as particulate matter.
Soon after the National Academies issued its report in 2002, EPA staff  identified key recommendations and developed a strategy, in consultation  with OMB, to apply some of the recommendations to benefit analyses for  air pollution regulations under consideration at the time. EPA roughly  approximated the time and resource requirements to respond to the  recommendations, identifying those the agency could address within 2 or 3  years and those that would take longer. According to EPA officials, the  agency focused primarily on the numerous recommendations related to  analyzing uncertainty.
Both the National Academies’ report and the OMB guidance emphasize the  need for agencies to account for uncertainties and to maintain  transparency in the course of conducting benefit analyses. Identifying and  accounting for uncertainties in these analyses can help decision makers  evaluate the likelihood that certain regulatory decisions will achieve the  estimated benefits. Transparency is important because it enables the public  and relevant decision makers to see clearly how EPA arrived at its  estimates and conclusions.  In prior work on regulatory impact analyses,  we have found shortcomings in EPA’s analyses of uncertainty and the  information the agency provides with its estimates of costs and benefits.
section_title: EPA Is in the Process of Addressing Many of the Academies’ Recommendations
paragraphs: EPA applied—either wholly or in part—approximately two-thirds of the  Academies’ recommendations to its January 2006 regulatory impact  analysis and continues to address the recommendations through ongoing  research and development.  The January 2006 regulatory impact analysis  demonstrated progress toward an expanded analysis of uncertainty and  consideration of different assumptions. EPA officials cited time and  resource constraints, as well as the need to mitigate complex technical  challenges, as the basis for not applying other recommendations.  According to EPA officials, the agency did not apply some of the more  complex recommendations because it had not achieved sufficient progress  in the research and development projects under way.
subsections: section_title: EPA Applied, at Least in Part, about Two-thirds of the Recommendations to Its Particulate Matter Health Benefit Analysis in the Proposed Rule
paragraphs: The January 2006 regulatory impact analysis on particulate matter  represents a snapshot of an ongoing EPA effort to respond to the National  Academies’ recommendations on developing estimates of health benefits  for air pollution regulations. Specifically, the agency applied, at least in  part, approximately two-thirds of the recommendations—8 were applied  and 14 were partially applied—by taking steps toward conducting a more  rigorous assessment of uncertainty for proposed air pollution regulations  by, for example, evaluating the different assumptions about the link  between human exposure to particulate matter and health effects and  discussing sources of uncertainty not included in the benefit estimates.  According to EPA officials, the agency focused much of its time and  resources on the recommendations related to uncertainty. In particular,  one overarching recommendation suggests that EPA take steps toward  conducting a formal, comprehensive uncertainty analysis—the systematic  application of mathematical techniques, such as Monte Carlo simulation— and include the uncertainty analysis in the regulatory impact analysis to  provide a “more realistic depiction of the overall uncertainty” in EPA’s  estimates of the benefits. A number of the other recommendations  regarding uncertainty are aimed at EPA’s developing the information and  methodologies needed to carry out a comprehensive uncertainty analysis.
Overall, the uncertainty recommendations suggest that EPA should  determine (1) which sources of uncertainties have the greatest effect on  benefit estimates and (2) the degree to which the uncertainties affect the  estimates by specifying a range of estimates and the likelihood of attaining  them. In response, EPA devoted significant resources to applying an  alternative technique called expert elicitation in a multiphased pilot  project. The pilot project was designed to systematically obtain expert  advice to begin to better incorporate in its health benefit analysis the  uncertainty underlying the causal link between exposure to particulate  matter and premature death. EPA used the expert elicitation process to  help it more definitively evaluate the uncertainty associated with estimated  reductions in premature death—estimates that composed 85 percent to 95  percent of EPA’s total health benefit estimates for air pollution regulations  in the past 5 years, according to the agency. EPA developed a range of  expected reductions in death rates based on expert opinion systematically  gathered in its pilot expert elicitation project and provided the results of  this supplemental analysis in an appendix to the regulatory impact  analysis. However, the National Academies had recommended that EPA  merge such supplemental analyses into the main benefit analysis.
Moreover, the Academies recommended that EPA’s main benefit analysis  reflect how the benefit estimates would vary in light of uncertainties. In  addition to the uncertainty underlying the causal link between exposure  and premature death that EPA analyzed, other key uncertainties can  influence the estimates. For example, there is uncertainty about the effects  of the age and health status of people exposed to particulate matter, the  varying composition of particulate matter, and the measurements of actual  exposure to particulate matter. EPA’s health benefit analysis, however, does  not account for these key uncertainties by specifying a range of estimates  and the likelihood of attaining them, similar to estimates derived from the  expert elicitation addressing causal uncertainty. For these reasons, EPA’s  responses reflect a partial application of the Academies’ recommendation.
In addition, the Academies recommended that EPA both continue to  conduct sensitivity analyses on sources of uncertainty and expand these  analyses. In the particulate matter regulatory impact analysis, EPA  included a new sensitivity analysis regarding assumptions about  thresholds, or levels below which those exposed to particulate matter are  not at risk of experiencing harmful effects. EPA has assumed no threshold  level exists—that is, any exposure poses potential health risks. Some  experts have suggested that different thresholds may exist and the National  Academies recommended that EPA determine how changing its  assumption—that no threshold exists—would influence the estimates. The  sensitivity analysis EPA provided in the regulatory impact analysis  examined how its estimates of expected health benefits would change  assuming varying thresholds.
Another recommendation that EPA is researching and partially applied to  the draft regulatory impact analysis concerns alternative assumptions  about cessation lags—the time between reductions in exposure to  particulate matter and the health response. The National Academies made  several recommendations on this topic, including one that EPA incorporate  alternative assumptions about lags into a formal uncertainty analysis to  estimate benefits that account for the likelihood of different lag durations.  In response, EPA has sought advice from its Advisory Council on Clean Air  Compliance Analysis on how to address this recommendation and has  conducted a series of sensitivity analyses related to cessation lags. EPA is  also funding research to explore ways to address lag effects in its  uncertainty analysis. According to an EPA official, specifying the  probability of different lag effects is computationally complex, and the  agency is working to resolve this challenge.
In response to another recommendation by the National Academies, EPA  identified some of the sources of uncertainty that are not reflected in its  benefit estimates. For example, EPA’s regulatory impact analysis disclosed  that its benefit estimates do not reflect the uncertainty associated with  future year projections of particulate matter emissions. EPA presented a  qualitative description about emissions uncertainty, elaborating on  technical reasons—such as the limited information about the effectiveness  of particulate matter control programs—why the analysis likely  underestimates future emissions levels. EPA also applied the Academies’  recommendation on the presentation of uncertainty, which encouraged the  agency to present the results of its health benefit analyses in ways that  convey the estimated benefits more realistically by, for example, placing  less emphasis on single estimates and rounding the numbers. EPA’s  regulatory impact analysis presented ranges for some of the benefit  estimates. Also, EPA sought to convey the overall uncertainty of its benefit  estimates in a qualitative manner by clearly stating that decision makers  and the public should not place significant weight on the quantified benefit  estimates in the regulatory impact analysis because of data limitations and  uncertainties.
Another example of EPA’s response to the National Academies’  recommendations involves exploring the various regulatory choices  available to decision makers. The Academies recommended that EPA  estimate the health benefits representing the full range of regulatory  choices available to decision makers. In the particulate matter analysis,  EPA presented health benefits expected under several regulatory options  targeting fine particulate matter. Citing a lack of data and tools needed to  conduct an accurate analysis, EPA did not estimate the benefits expected  under the proposed regulatory options for coarse particulate matter but,  consistent with the National Academies’ recommendation, presented its  rationale for not doing so. Overall, we considered this a partial application  of the recommendation. (See app. II for more detail on the  recommendations that EPA has applied or partially applied to the draft  particulate matter regulatory impact analysis.)
section_title: EPA Plans to Address Some of the Remaining Recommendations in the Final Rule and Has Research and Development Under Way to Address Others
paragraphs: EPA did not apply the remaining 12 recommendations to the analysis for  various reasons. While EPA applied some recommendations—either  wholly or in part—that require additional studies, methodologies, or data to  its particulate matter analysis, the agency had not made sufficient progress  in addressing others and therefore did not apply them to the analysis. EPA  officials viewed most of these recommendations as relevant to its health  benefit analyses and, citing the need for additional research and  development, emphasized the agency’s commitment to continue to respond  to the recommendations. According to a senior EPA official, insufficient  resources impeded the agency’s progress in applying the  recommendations. This official cited limited availability of skilled staff,  time, and other resources to conduct the required analyses and research  and development. According to EPA, some of the more complex, long-term  recommendations include the following: relying less on simplifying  assumptions, such as the assumption that the various components of  particulate matter have equal toxicity; conducting a formal assessment of  the uncertainty of particulate matter emissions; and assessing the expected  reduction of any harmful effects other than air pollution or human health  problems.
For example, EPA is in the process of responding to a recommendation  involving the relative toxicity of components of particulate matter, an  emerging area of research that has the potential to influence EPA’s  regulatory decisions in the future. Specifically, the agency could,  hypothetically, refine national air quality standards to address the  potentially varying health consequences associated with different  components of particulate matter. The National Academies recommended  that EPA strengthen its benefit analyses by evaluating a range of alternative  assumptions regarding relative toxicity and incorporate these assumptions  into sensitivity or uncertainty analyses as more data become available.   EPA did not believe the state of scientific knowledge on relative toxicity  was sufficiently developed at the time it prepared the draft regulatory impact analysis to include this kind of analysis. However, EPA is  sponsoring research on this issue. For example, EPA is supporting long- term research on the relative toxicity of particulate matter components  being conducted by EPA’s intramural research program, its five Particulate  Matter Research Centers, and the Health Effects Institute, an organization  funded in part by EPA. In addition, an EPA contractor has begun to  investigate methods for conducting a formal analysis that would consider  sources of uncertainty, including relative toxicity and lag effects. To date,  the contractor has created a model to assess whether and how much these  sources of uncertainty may affect benefit estimates in one urban area.
The National Academies also recommended that EPA incorporate an  assessment of uncertainty into the early stages of its benefit analyses by  characterizing the uncertainty of its emissions estimates on which the  agency is going to base its benefit estimates. While the agency is  investigating ways to assess or characterize this uncertainty, EPA did not  conduct a formal uncertainty analysis for particulate matter emissions for  the draft regulatory impact analysis because of data limitations. These  limitations stem largely from the source of emissions data, the National  Emissions Inventory, an amalgamation of data from a variety of entities,  including state and local air agencies, tribes, and industry. According to  EPA, these entities use different methods to collect data, which have  different implications for how to characterize the uncertainty.  Furthermore, the uncertainty associated with emissions varies by the  source of emissions. For example, the analytical methods for evaluating the  uncertainty of estimates of emissions from utilities would differ from those  for car and truck emissions because the nature of these emissions and the  data collection methods differ. In sum, to apply this recommendation, EPA  must determine how to characterize the uncertainty of the estimates for  each source of emissions before aggregating the uncertainty to a national  level and then factoring that aggregation into its benefit estimates.  According to EPA officials, the agency needs much more time to resolve  the complex technical challenges of such an analysis. EPA officials also  noted that the final particulate matter analysis will demonstrate steps  toward this recommendation by presenting emissions data according to the  level emitted by the different kinds of sources, such as utilities, cars, and  trucks.
Another recommendation that EPA is researching but did not apply to the  draft regulatory impact analysis concerns whether the proposed revisions  to the particulate matter standards would have important indirect impacts  on human health and the environment. According to an EPA official, the  agency could not rule out the possibility that the revisions could have  indirect impacts on the environment, such as whether reductions to  particulate matter emissions would reduce the amount of particulate  matter deposited in water bodies, thereby decreasing water pollution. EPA  has considered indirect impacts of air pollution regulations on sensitive  water bodies in the past and plans to include a similar analysis in the final  particulate matter rule. An agency official further noted that ongoing  research about environmental impacts could reveal additional indirect  impacts for future analyses.
Other recommendations that EPA did not apply to its benefit estimates in  the regulatory impact analysis concern issues such as transparency and  external review of EPA’s benefit estimation process. For example, the  National Academies recommended that EPA clearly summarize the key  elements of the benefit analysis in an executive summary that includes a  table that lists and briefly describes the regulatory options for which EPA  estimated the benefits, the assumptions that had a substantial impact on  the benefit estimates, and the health benefits evaluated. EPA did not,  however, present a summary table as called for by the recommendation or  summarize the benefits in the executive summary.  As EPA stated in the  particulate matter analysis, the agency decided not to present the benefit  estimates in the executive summary because they were too uncertain.  Specifically, officials said the agency was not able to resolve some  significant data limitations before issuing the draft regulatory impact  analysis in January 2006—a deadline driven by the need to meet the court- ordered issue date for the final rule in September 2006.  According to EPA  officials, EPA has resolved some of these data challenges by, for example,  obtaining more robust data on anticipated strategies for reducing  emissions, which will affect the estimates of benefits. The officials also said  that EPA intends to include in the executive summary of the regulatory  impact analysis supporting the final rule a summary table that describes  key analytical information. EPA officials also acknowledged other  presentation shortcomings, including references to key analytical elements  that were insufficiently specific, that officials attributed to tight time  frames and the demands of working on other regulatory analyses  concurrently. They said they plan to address these shortcomings in the final  regulatory impact analysis.
Regarding external review, the National Academies recommended that EPA  establish an independent review panel, supported by permanent technical  staff, to bolster EPA’s quality control measures for its regulatory impact  analyses, such as the one for particulate matter. The National Academies  noted that peer review of EPA’s regulatory impact analyses would be  advantageous when the agency designs and conducts its economic  analysis. EPA has not directly addressed this recommendation. According  to the Director of the Office of Policy Analysis and Review in EPA’s Office  of Air and Radiation, establishing and supporting independent committees  is costly, making it important for EPA to take advantage of existing panels  rather than set up new ones. Further, an official in the Office of Air and  Radiation who oversees the development of regulatory impact analyses  said that the cost of reviewing all regulatory impact analyses would be  substantial. In this regard, EPA officials identified peer reviews the agency  received from its existing independent committees, such as the Clean Air  Scientific Advisory Committee and the Advisory Council on Clean Air  Compliance. For example, to respond to the Academies’  recommendations about lag effects, EPA sought independent advice on the  assumptions it was developing regarding the time between reduced  exposure to particulate matter and reductions in incidences of health  effects. Finally, EPA officials noted that although the agency does not  have each regulatory impact analysis peer reviewed, EPA typically does  have the methodologies that will be applied to regulatory impact analyses  peer reviewed. (See app. III for more detail on these recommendations and  others that EPA did not apply to the draft particulate matter regulatory  impact analysis.)
section_title: Concluding Observations
paragraphs: While EPA has taken a number of steps to respond to the Academies’  recommendations on estimating health benefits, continued commitment  and dedication of resources will be needed if EPA is to fully implement the  improvements endorsed by the National Academies.  In particular, the  agency will need to ensure that it allocates resources to needed research on  emerging issues, such as the relative toxicity of particulate matter  components; assessing which sources of uncertainty have the greatest  influence on benefit estimates; and estimating other benefits, such as  environmental improvements. In addition, it is important for EPA to  continue to improve its uncertainty analysis in accordance with the  Academies’ recommendations. The agency’s draft regulatory impact  analysis illustrates that estimates of health benefits can be highly  uncertain. In fact, EPA officials viewed these estimates as so uncertain that  they chose to not present them in the executive summary of the regulatory  impact analysis. While EPA officials said they expect to reduce the  uncertainties associated with the health benefit estimates in the final  particulate matter analysis, robust uncertainty analysis will nonetheless be  important for decision makers and the public to understand the likelihood  of attaining the estimated health benefits. According to EPA officials, the  final regulatory impact analysis on particulate matter will reflect further  responsiveness to the Academies’ recommendations by, for example,  providing additional sensitivity analysis and improving the transparency of  the regulatory impact analysis by highlighting key data and assumptions in  the executive summary. Moreover, these officials emphasized the agency’s  commitment to further enhancing the transparency of the analysis by  presenting clear and accurate references to the supporting technical  documents, which detail the analytical assumptions and describe the data  supporting the estimates. To the extent EPA continues to make progress  addressing the Academies’ recommendations, decision makers and the  public will be able to better evaluate the basis for EPA’s air regulations.
section_title: Agency Comments
paragraphs: We provided a draft of this report to EPA for review. EPA provided  technical comments that we incorporated, as appropriate. Officials from  the Office of Policy Analysis and Review within EPA’s Office of Air and  Radiation noted in their technical comments that the report provides a fair  and balanced representation of EPA’s efforts to apply the National  Academies’ recommendations to the draft particulate matter regulatory  impact analysis. However, these officials also cited progress made in  applying the National Academies’ recommendations through analyses of  other air programs and through research and development efforts. We note  that this report does identify, as appropriate, EPA’s research and  development efforts for recommendations EPA did not apply to the draft  particulate matter analysis, its plans to apply some additional  recommendations to the final particulate matter regulatory impact  analysis, and the agency’s responses to recommendations in prior rule- making analyses of air programs.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the EPA Administrator and  other interested parties. We will also make copies available to others upon  request. In addition, the report will be available at no charge on the GAO  Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-3841 or stephensonj@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. Key contributors to this report are listed in appendix IV.
section_title: Scope and Methodology
paragraphs: We were asked to determine whether and how the Environmental  Protection Agency (EPA) applied the National Academies (Academies)  recommendations in its estimates of the health benefits expected from the  January 2006 proposed revisions to the particulate matter national ambient  air quality standards. In response to this objective, we assessed EPA’s  response to the Academies’ recommendations and present an overview of  the agency’s completed, ongoing, and planned actions addressing the  recommendations. To develop this overview, we reviewed EPA’s particulate  matter regulatory impact analysis, EPA’s economic analysis guidelines, and  Office of Management and Budget (OMB) guidance on regulatory impact  analysis. We also analyzed documentation addressing current and future  agency efforts to address the recommendations, such as project planning  memorandums and technical support documents discussing the  application of economic techniques. In addition, we met with senior  officials from EPA’s Office of Air and Radiation, which was responsible for  developing the proposed rule and analyzing its economic effects, and with  officials from EPA’s Office of Policy, Economics, and Innovation to discuss  the agency’s responses to the recommendations. We interviewed several  experts outside EPA, including (1) the Chair and other members of the  National Academies’ Committee on Estimating the Health-Risk-Reduction  Benefits of Proposed Air Pollution Regulations, to clarify the basis for their  recommendations; and (2) economists at Resources for the Future, to  discuss the technical issues underlying the recommendations on  uncertainty analysis.
While the 2002 National Academies’ report is generally applicable to EPA  air pollution regulations, our review focused on the application of the  recommendations to the proposed revisions to the particulate matter  standards, as requested. Our work focused on broadly characterizing EPA’s  progress toward applying the recommendations; we did not evaluate the  effectiveness or quality of the scientific and technical actions the agency  has taken to apply them. To assess whether and how EPA has made  progress in responding to the recommendations, we developed the  following recommendation classification continuum: applied, partially  applied, and not applied. The applied and partially applied categories refer  to completed and initiated actions in EPA’s health benefit analysis of  particulate matter that corresponds to components of the National  Academies’ recommendations. The not applied category includes  recommendations that EPA did not apply when conducting the analysis for  the January 2006 particulate matter regulatory impact analysis and  identifies those for which ongoing research and development efforts were  not far enough along to apply to the particulate matter analysis. We  performed our work from January 2006 to July 2006 in accordance with  generally accepted government auditing standards.
section_title: Recommendations Applied or Partially Applied to the Draft Particulate Matter Regulatory Impact Analysis
paragraphs: Table 1 provides a summary of the National Academies’ recommendations  that EPA has applied or partially applied to its draft regulatory impact  analysis (RIA) for particulate matter (PM). This table also provides GAO’s  assessment of EPA’s progress in applying each recommendation, in terms  of steps EPA has taken thus far to address issues highlighted in the  National Academies’ report. The final column characterizes EPA’s  comments regarding each recommendation, including, as pertinent,  contextual information, potential impediments to application, and intended  next steps.
section_title: Recommendations Not Applied to the Draft Particulate Matter Regulatory Impact Analysis
paragraphs: Table 2 provides a summary of the National Academies’ recommendations  that EPA has not applied to its draft regulatory impact analysis (RIA) for  particulate matter (PM). This table provides GAO’s assessment of EPA's  progress to date regarding recommendations that required additional  research and development, were deemed as not relevant to the PM  National Ambient Air Quality Standards (NAAQS) by the agency, or were  not included in the draft PM RIA due to time and resource constraints. The  final column characterizes EPA’s comments regarding each  recommendation, including contextual information, potential impediments  to application, justification for not addressing the recommendation, and  intended next steps, if applicable.
section_title: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Christine Fishkin, Assistant  Director; Kate Cardamone; Nancy Crothers; Cindy Gilbert; Tim Guinane;  Jessica Lemke; and Meaghan K. Marshall made key contributions to this  report. Timothy Bober, Marcia Crosse, and Karen Keegan also made  important contributions.