Clean Air Act: Observations on EPA's Cost-Benefit Analysis of Its Mercury Control Options
Released: 2005-03-07T00:00:00

Letter

Background

Mercury enters the environment through natural and man-made sources, including volcanoes, chemical manufacturing, and coal combustion, and poses ecological threats when it enters water bodies, where small aquatic organisms convert it into its highly toxic form—methylmercury. This form of mercury may then migrate up the food chain as predator species consume the smaller organisms. Through a process known as bio-accumulation, predator species may develop high mercury concentrations in their tissue as they take in more mercury than they can metabolize or excrete.

Fish contaminated with methylmercury may pose health threats to those that rely on fish as part of their diet. According to EPA, mercury harms fetuses and can cause neurological disorders in children, including poor performance on behavioral tests, such as those measuring attention, motor and language skills, and visual-spatial abilities (such as drawing). In addition, populations that consume larger amounts of fish than the general population—including subsistence fishers, as well as certain Native Americans and Southeast Asian Americans—may face higher risk of exposure to contaminated fish, according to EPA. The Food and Drug Administration (FDA) and EPA recommend that expectant mothers, young children, and those nursing children avoid eating swordfish, king mackerel, shark, and tilefish and limit consumption of other potentially contaminated fish, such as tuna. These agencies also recommend checking local advisories for recreationally caught freshwater and saltwater fish. According to EPA, 45 states issued mercury advisories in 2003 (the most recent data available).

Because mercury released to the atmosphere can circulate for long periods of time and be transported thousands of miles before it gets deposited, it is difficult to link mercury accumulation in the food chain with sources of mercury emissions. EPA estimates that about half of the mercury deposited in the United States is emitted by sources within this country. In 1999, the most recent year for which data were available, EPA estimated that man-made sources within the United States emitted about 115 tons of mercury. Of these emissions, the agency estimates that about 48 tons, 42 percent of the total, came from coal-fired power plants. While power plants are not required to limit their mercury emissions, EPA estimates that the plants currently capture about 27 tons of mercury each year, primarily through the use of controls for other pollutants, such as those used to control nitrogen oxides, particles, and sulfur dioxide. EPA estimates that power plants would otherwise emit about 75 tons of mercury per year.

The Clean Air Act (CAA) Amendments of 1990 required EPA to study the environmental and health effects of hazardous air pollutants from coal-fired power plants and determine whether it was “appropriate and necessary” to regulate these pollutants. In 2000, EPA determined that mercury was a hazardous air pollutant and that it was appropriate and necessary to regulate mercury using the technology-based option. Under this section of the act, the emissions limit had to be at least as strict as the average emissions of the facilities with the best-controlled emissions. Because power plants did not already use controls specifically intended to control mercury, EPA analyzed the effectiveness of controls for other pollutants that capture mercury as a side benefit.

This effort culminated in EPA’s January 2004 proposal for a technology-based option that would reduce mercury emissions from a current level of 48 tons per year to a projected 34 tons per year (a 29 percent reduction) by 2008. At the same time, however, EPA proposed an alternate policy option that would limit mercury emissions in two phases: to 15 tons in 2018 (a 69 percent reduction from current levels), preceded by an as-yet-unspecified interim cap starting in 2010. The alternate policy option, which would rely on a cap-and-trade system similar to that currently used to control emissions that cause acid rain, differs from the technology-based option in that it would not require each facility to meet emission standards based on control technology. Instead, EPA would set a nationwide “cap” for mercury emissions from coal-fired power plants and then distribute tradable emissions allowances that represent a certain amount of the total cap. At the end of each year, each power plant would have to hold sufficient allowances for the mercury it emitted that year. Plants that reduced their emissions below the levels represented by their allowances could sell their extra allowances to other plants.

In addition to its proposed mercury rule, EPA has proposed another rule for power plants, the Clean Air Interstate Rule, which is intended to reduce emissions of nitrogen oxides and sulfur dioxide beginning in 2010. EPA expects that this proposed rule would result in the installation of pollution controls that capture mercury as a side benefit, and thereby decrease mercury emissions to 34 tons per year by 2010, the same level of reduction as the technology-based option. Under the cap-and-trade option, EPA has indicated that it may establish a mercury cap for 2010 equal to the control level expected through the interstate rule. EPA postponed its decision on finalizing the interstate rule until March 2005 while the agency awaits congressional action on pending legislation, known as the Clear Skies Act, that would establish emissions caps and an allowance system similar to those in the interstate rule and the cap-and-trade mercury control option. EPA has stated a preference for achieving reductions of mercury, nitrogen oxides, and sulfur dioxide simultaneously through legislation rather than regulations.

Responsibility for analyzing the economic impacts—including costs to industry and expected public health effects—of air pollution control policies rests with EPA’s Office of Air and Radiation. EPA provided documentation of its economic analysis for the proposed mercury rule in three primary documents, some of which refer readers to additional documentation on the agency’s Web site or in the public rule-making docket. According to EPA, the agency did not have time to assemble its economic assessment of the proposed rule in a single document prior to issuing the proposed rule. To assist in estimating costs that air quality regulations will impose on the power industry, EPA uses the Integrated Planning Model (IPM), which estimates how power plants would respond to various environmental policies. The assumptions underlying this model, such as those regarding fuel costs, the costs of pollution controls, and future electricity demand, can affect the modeling results, according to EPA officials responsible for the modeling.

EPA’s Economic Analysis Is of Limited Use for Informing Decision Makers about the Economic Trade-offs of Different Policy Options

We identified four major shortcomings in the economic analysis underlying EPA’s proposed mercury rule that limit its usefulness for informing decision makers and the public about the economic trade-offs of the two options. First, EPA did not consistently analyze each of its two mercury policy options or provide estimates of the total costs and benefits of the two options, making it difficult to ascertain which policy option would provide the greatest net benefits. Second, EPA did not document some of its analysis or provide consistent information on the anticipated economic effects of different mercury control levels under the two options. Third, the agency did not estimate the economic benefits directly related to decreased mercury emissions. Finally, the agency did not analyze some of the key uncertainties underlying its cost-and-benefit estimates.

EPA Did Not Consistently Analyze Each Policy Option or Provide a Complete Accounting of Costs and Benefits

EPA’s estimates of the costs and benefits of its two proposed policy options are not comparable because the agency used inconsistent approaches in analyzing the two options. As shown in table 1, EPA analyzed the technology-based option alone, while it analyzed the cap-and-trade option in combination with the interstate rule. In analyzing the technology-based option by itself, EPA estimated the rule would cost about $2 billion annually, and achieve benefits of $15 billion or more annually, yielding net benefits (benefits minus costs) of $13 billion or more annually. In contrast, EPA analyzed the effects of the cap-and-trade option in combination with the proposed interstate rule by combining the costs and benefits of the two proposed rules without separately identifying and documenting those associated with the cap-and-trade option alone. This analysis found that the two proposed rules together would impose costs of $3 billion to $5 billion or more annually, while generating annual benefits of $58 billion to $73 billion or more and annual net benefits of $55 billion to $68 billion or more.

Because the estimates for the two options are not comparable, however, it is not clear which option would provide the greatest net benefits. This is particularly important in light of EPA’s decision to delay finalization of the interstate rule. EPA officials responsible for the rule acknowledged the lack of comparability with its analysis of the two proposed options. These officials said the agency analyzed the cap-and-trade option alongside the interstate rule because it viewed these two proposed policies as complementary. They also said it would have been useful to analyze the technology-based option alongside the interstate rule, but the agency did not do so because of time constraints. Nonetheless, it is important for EPA to consistently analyze each policy option and provide decision makers with comparable estimates of net economic benefits.

The comparability of EPA’s analysis is further limited because the agency did not provide consistent information on the total costs and benefits of the two options over their entire implementation periods. Specifically, EPA provided cost-and-benefit estimates for 2010, rather than estimates of the total costs and benefits over the entire implementation period. This is important because the economic impact of the policy options could vary from year to year and because the two options have different implementation timelines. For example, under the proposed cap-and-trade option, a second level of mercury reductions would take effect in 2018, which would likely generate additional costs and benefits at that time. Thus, the estimates EPA provided for 2010 did not fully account for the expected costs and benefits over the implementation period for this option. In contrast, EPA officials said that its estimate of the technology-based option in 2010 reflects the full implementation cost because its analysis assumes that power plants would achieve compliance with the technology-based option by that date. However, without estimates of the total value of benefits and costs of each option over the entire implementation period, it is difficult to ascertain which option would generate the greatest net benefits.

EPA Did Not Document Some of Its Analysis Supporting the Policy Options or Provide Consistent Information on the Economic Impacts of Different Control Levels

The economic analysis underlying the proposed mercury rule does not consistently reflect OMB’s guidance to agencies in terms of adhering to the principles of full disclosure and transparency when analyzing the economic effects of regulations. Specifically, we identified two primary cases where EPA’s analysis does not adhere to these principles, further limiting the usefulness of the agency’s analysis in decision making and diminishing the transparency of the analysis to the public.

First, while EPA provides substantial information on its analysis of the technology-based option in the documents supporting its economic analysis of the proposed rule, the agency does not do so for the cap-and-trade option. For the technology-based option, EPA provides documents that describe its findings. In contrast, the agency provides only a summary of its findings for the cap-and-trade option in the rule’s preamble and refers to its findings as “rough estimates” that are based on consideration of available analysis of the interstate rule, the technology-based option, and the proposed Clear Skies legislation. EPA does not describe specifically how the agency used this analysis of other proposed rules and legislation to estimate the costs and benefits of the cap-and-trade option, and it does not identify the key analytical assumptions underlying its cost-and-benefit estimates. This lack of documentation and transparency leaves decision makers and the public with limited information on EPA’s analysis of the cap-and-trade option.

Second, EPA officials responsible for the economic analysis told us that they analyzed two variations of the proposed technology-based option with more stringent mercury limits than the option included in the proposal, but the agency did not include this analysis in the documents supporting its economic analysis or in the public rule-making docket. This is inconsistent with EPA’s analysis of the cap-and-trade option, in which it provided a range of costs and benefits associated with different levels of stringency. This omission is also at odds with OMB guidance directing agencies to conduct their economic analysis in accordance with the principles of full disclosure and transparency.

With respect to the analysis of the technology-based scenarios that the agency did not make publicly available, EPA officials said the additional modeling showed that the more stringent scenarios were not as cost-effective as the proposed technology-based option. However, EPA did not estimate the benefits of these two scenarios, thereby precluding a comparison of the net economic benefits under the proposed mercury policy options. As a result, it is unclear whether the reduction levels and implementation timelines under either proposed option represent the regulatory scenario that would provide the greatest net benefits.

In January 2005, EPA officials responsible for the mercury rule said the agency does not have an obligation to analyze and document every control scenario. We recognize that OMB guidance gives agencies latitude in determining the number of regulatory alternatives to consider and that agencies must balance the thoroughness of their analysis with the practical limits of their ability to carry out analysis. Nonetheless, providing information on the costs and benefits of even a limited range of control scenarios under both proposed options would help decision makers and the public in assessing how different levels of stringency would affect overall estimates of costs and benefits. In December 2004, EPA solicited public comment on additional economic analyses the agency received from commenters on the January 2004 proposed rule, including some that relied on models, assumptions, and levels of stringency that were different from the scenarios EPA analyzed.

EPA Did Not Estimate the Human Health Benefits of Mercury Reductions

Although EPA’s analysis states that a mercury regulation would generate a variety of benefits, the agency did not estimate in monetary terms all of the benefits expected from reducing mercury emissions. Most notably, EPA did not quantify the human health benefits of decreased exposure to mercury, such as reduced incidence of developmental delays, learning disabilities, and neurological disorders. Instead, EPA estimated only some of the health benefits it anticipates would occur from decreased exposure to fine particles and discussed other impacts qualitatively. Because the two options in the proposed rule differed significantly in both the amount of mercury emission reductions and the time frames in which these reductions would occur, the lack of estimates of the mercury-specific benefits of each policy option represents a significant limitation of EPA’s economic analysis. That is, to the extent that each proposed option would yield measurable mercury-specific health benefits, EPA’s analysis may underestimate the total expected benefits of both options. Moreover, because the options may yield different mercury-related health benefits, the lack of estimates of these benefits makes it difficult to weigh the relative merits of the two proposed options.

According to EPA, its analysis did not estimate key mercury-related health benefits because of technical, time, and resource limitations. Specifically, agency officials responsible for the analysis said the agency did not have a method for determining the extent to which mercury reductions from power plants would translate into decreased incidence of mercury-related health problems. According to EPA, estimating these benefits involves a number of complex chemical, physical, and biological processes, as well as a wide variety of human behaviors, such as fish consumption practices.

Although EPA did not estimate the expected human health and other benefits of decreased exposure to mercury emissions in the analysis supporting the proposed rule, the agency did list the various human health and other benefits it expects would stem from a mercury rule. Importantly, in December 2004, the agency announced that it was revising its benefit estimates and solicited public comment on a proposed method for estimating mercury-specific benefits. According to EPA, this method would focus on (1) quantifying projected emissions from coal-fired power plants relative to other sources, (2) modeling the dispersion and deposition of mercury, (3) modeling the link between changes in mercury deposition and changes in the methylmercury concentrations in fish, (4) assessing the methylmercury exposure from consuming fish, and (5) assessing how reductions in methylmercury exposure affect human health. According to EPA officials responsible for analyzing the proposed rule’s effects, the agency will consider public comments on this approach and revise its analysis before finalizing a rule. In January 2005, EPA officials responsible for the analysis agreed that providing monetary estimates of mercury-specific benefits would enhance their analysis, and said that the agency might have sufficient information to estimate some, but not all, of the expected human health benefits of reducing mercury emissions.

EPA Did Not Assess Key Analytical Uncertainties That Could Affect Its Cost-and-Benefit Estimates

OMB guidance under Executive Order 12866 stipulates that agencies should analyze and present information on uncertainties with their cost-and-benefit estimates. According to EPA officials responsible for the economic analysis, the agency’s cost model is generally sensitive to assumptions about future electricity demand and fuel prices, as well as the availability, cost, and performance of pollution controls. Because these assumptions involve long-term projections, they also involve a substantial amount of uncertainty. EPA conducted a limited uncertainty analysis of natural gas prices and electricity demand growth on the cost estimates by examining the impact of alternative projections and concluded that its cost estimates were not particularly sensitive to changes in these variables. However, EPA did not assess how the distribution of estimated benefits and costs would differ given changes in its assumptions about the availability, cost, and performance of mercury control technologies, even though the agency believes that these assumptions could affect its economic modeling.

Furthermore, EPA’s December 2004 notice for additional public comment on the mercury proposal highlighted the uncertainty surrounding the ability of its computer model to estimate mercury control costs, primarily because of the power industry’s limited experience with implementing mercury controls. This notice solicited public comment on, among other things, the assumptions in its economic modeling related to the cost, availability, and performance of mercury control technologies. According to senior EPA officials responsible for analyzing the mercury proposal, changes in these assumptions could have a sizable impact on the agency’s cost-and-benefit estimates. This acknowledgment of key uncertainties in its economic modeling highlights the need to determine how they could affect the overall cost-and-benefit estimates for each proposed option.

In addition, EPA did not analyze the key uncertainties surrounding its benefit estimates. For example, EPA used economic data from its earlier assessment of the proposed Clear Skies legislation to approximate the impact of emissions reductions that would be expected under the mercury rule. According to EPA, the agency used this approach—referred to as a “benefits-transfer approach”—because time and resource constraints prevented it from performing new research to measure the value of health impacts under a mercury rule. OMB’s September 2003 guidance, which applies to economically significant final rules issued after January 1, 2005, states that although such an approach can provide a quick and low-cost means of obtaining monetary values, the method may be characterized by uncertainty and potential biases of unknown magnitude and should be treated as a last-resort option. Furthermore, EPA’s economic analysis states that the benefits analysis has many sources of uncertainty, including those associated with emissions data, air quality modeling, and the effect of emissions on human health. The agency did not, however, formally assess the impact of these uncertainties.

In January 2005, EPA officials responsible for the proposed mercury rule acknowledged this limited analysis of key uncertainties and said that the agency plans to conduct a more formal assessment of these uncertainties prior to issuing a final rule, as directed by OMB’s September 2003 guidance. This guidance directs agencies to assess the sources of uncertainty in their regulatory analyses and the way in which cost-and-benefit estimates may be affected under plausible assumptions. Furthermore, in cases where the annual economic effects total $1 billion or more, the guidance states that agencies should provide a formal quantitative assessment of the key uncertainties about costs and benefits.

Conclusions

Because EPA estimates that regulating mercury emissions would have significant economic impacts totaling billions of dollars per year, it is important for the agency to have a credible basis for selecting a policy that will maximize the return on this investment. However, EPA’s initial economic analysis of the two policies it is considering has a number of shortcomings. Specifically, because EPA did not analyze and document the economic effects of each policy option by itself—as well as in combination with the interstate rule—over their varying full implementation periods, the results cannot be meaningfully compared. In addition, EPA did not document the analysis supporting the cap-and-trade option or provide consistent information on the economic impacts of different mercury control levels for the two options, limiting the transparency and usefulness of the analysis. Further, without monetary estimates of the human health benefits of mercury emissions reductions—a primary purpose of a mercury regulation—over the full implementation period of each option or, at a minimum, a qualitative comparison of these benefits, EPA’s analysis does not provide decision makers with a strong basis for comparing the net benefits under each option. Finally, because EPA did not analyze some of the key analytical uncertainties that could affect its estimates of net benefits, the agency could enhance its economic analysis by further evaluating these uncertainties and how they could affect its overall findings. Unless EPA conducts and documents further economic analysis, decision makers and the public may lack assurance that the agency has evaluated the economic trade-offs of each option and taken the appropriate steps to identify which mercury control option would provide the greatest net benefits.

Recommendations for Executive Action

To improve the usefulness of the agency’s economic analysis for informing decision makers and the public, and to help ensure consistency with OMB guidance for economic analysis, we recommend that, as the agency revises its economic analysis prior to selecting a mercury control option, the EPA Administrator take the following four actions: Analyze and fully document the economic effects of each policy option by itself, as well as in combination with the interstate rule, over their full implementation periods.

Ensure that the agency documents its analysis supporting the final rule and consistently analyzes the effect that different levels of mercury control would have on cost-and-benefit estimates under each policy option.

Include monetary estimates, where possible, of the human health benefits of reductions in mercury emissions from power plants or, at a minimum, provide qualitative information on how these benefits are likely to compare under the two options over a consistent time frame, reflecting full implementation of both options.

Further analyze uncertainties surrounding estimates of costs and benefits, as directed by OMB guidance, and evaluate how these uncertainties could affect overall estimates of the rule’s impacts.

Agency Comments

We provided EPA with a draft of this report for review and comment. In commenting on the draft report, the Assistant Administrator for Air and Radiation said that, prior to issuing a final mercury regulation by March 15, 2005, EPA will conduct additional analysis that will largely address the findings and recommendations identified in our report. EPA’s letter is included as appendix II.

As agreed with your offices, unless you publicly announce the contents of this letter earlier, we plan no further distribution until 7 days from the report date. At that time, we will send copies of the report to the EPA Administrator and other interested parties. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you have any questions about this report, please contact me at (202) 512-3841 or stephensonj@gao.gov. Key contributors to this report are listed in appendix III.

Objectives, Scope, and Methodology

Congressional requesters asked us to assess the usefulness of the economic analysis underlying EPA’s proposed mercury rule for decision making. To respond to this objective, we, among other things, reviewed EPA’s analysis of the proposed rule’s economic effects using standard economic principles, OMB guidance, Executive Order 12866, and the Unfunded Mandates Reform Act of 1995. We also discussed the analysis with senior officials within EPA’s Office of Air and Radiation responsible for developing the proposed rule and analyzing its economic effects. In doing this work, we did not independently estimate the costs or benefits of the mercury control options, evaluate EPA’s process for developing the options, or assess legal issues surrounding the extent to which the options comply with the provisions of the Clean Air Act or its amendments.

We took several steps to assess the validity and reliability of computer data underlying EPA’s estimates of economic impacts discussed in our findings, including reviewing the documentation and assumptions underlying EPA’s economic model and assessing the agency’s process for ensuring that the model data are sufficient, competent, and relevant. We also discussed these assumptions and procedures with agency officials responsible for the modeling data. (For the background section of this report, we obtained data on mercury emissions. Because they are used for background purposes only, we did not assess their reliability.) We assessed compliance with internal controls related to the availability of timely, relevant, and reliable information. Our concerns about EPA data and analysis are discussed in the body of this report.

We performed our work between May 2004 and February 2005 in accordance with generally accepted government auditing standards.

Comments from the Environmental Protection Agency

GAO Contacts and Staff Acknowledgments

GAO Contacts

Staff Acknowledgments

In addition to the individuals named above, Tim Guinane and Michael Hix made key contributions to this report. Kate Cardamone, Jessica Fast, Cynthia Norris, Judy Pagano, Janice Poling, and Amy Webbink also made important contributions.