Air Pollution: Information Contained in EPA's Regulatory Impact Analyses Can Be Made Clearer
Published: 1997-04-14T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: GAO reviewed the Environmental Protection Agency's (EPA) 23 regulatory impact analyses (RIA) supporting air quality regulations, focusing on whether the RIAs clearly describe: (1) key economic assumptions subject to uncertainty and the sensitivity of the results to these assumptions; (2) the extent to which benefits and costs were quantified for the proposed regulatory action; and (3) the extent to which alternative approaches were considered.
section_title: What GAO Found
paragraphs: GAO noted that: (1) while certain key economic assumptions, such as the discount rate and the value of human life, can have a significant impact on the results of benefit-cost analyses and are important to the regulations being proposed, eight of the RIAs did not identify one or more of these assumptions; (2) furthermore, in the RIAs that identified key economic assumptions, the rationale for the values used was not always explained; (3) for example, one RIA assumed a value of life that ranged from $1.6 million to $8.5 million and another, prepared in the same year, assumed a value of life that ranged from $3 million to $12 million; (4) in neither instance did the RIAs provide a clear explanation of the rationale for the values that were selected; (5) even though EPA's guidance suggests that RIAs account for any uncertainties in the values of key assumptions by conducting sensitivity analyses, which show how benefit and cost estimates vary depending on what values are assumed, 13 RIAs used only a single discount rate; (6) all 23 RIAs assigned dollar values to the estimated costs of proposed regulations, however, 11 of the RIAs assigned dollar values to the estimated benefits; (7) according to EPA officials, assigning dollar values to potential benefits is difficult because of the uncertainty of scientific data and the lack of market data on some of these effects; (8) all of the RIAs contained some quantitative or qualitative information on the expected benefits, such as a reduced incidence of mortality and illness; (9) while the number and the types of alternatives considered in the 23 RIAs were not always clear, GAO's examination indicated that six of the RIAs compared a single alternative, which was the regulatory action being proposed, to the baseline, which was the situation likely to occur in the absence of regulation, the status quo; (10) the remainder compared two or more alternatives to the baseline; (11) resource constraints and the specific requirements of authorizing legislation, which sometimes limits EPA's options, were factors influencing the extent to which alternatives were considered; (12) ten of the RIAs GAO examined did not include executive summaries, even though these summaries can be a significant benefit to decisionmakers; and (13) EPA officials acknowledged that some of the RIAs did not include executive summaries and agreed that executive summaries, by providing easily accessible information, can be useful to decisionmakers.

section_title: Letter
paragraphs: The Environmental Protection Agency (EPA) has been required to perform benefit-cost analyses, or regulatory impact analyses (RIA), to support its regulatory actions since 1971. The analyses are expected to conform to guidelines developed by the Office of Management and Budget (OMB) and EPA to implement executive orders requiring them. Generally, the guidelines describe the major components that should be contained in the analyses, such as a statement of the need for a regulation and a description and estimation of the benefits and costs for regulatory alternatives, including the results of sensitivity analyses to characterize the effects of uncertainties. The guidelines allow EPA considerable flexibility in preparing RIAs. Specifically, the guidelines stipulate that the scope and precision of analysis depend on the quality of underlying data, the scientific understanding of the problems to be addressed through regulation, resource constraints at EPA, and the specific requirements of the authorizing legislation. The guidelines also recommend that the scope and precision of the analysis should be proportionate to the importance and complexity of the issues being addressed.
This report, which describes the results of our review of 23 RIAs supporting air quality regulations, is addressed to you because of your jurisdiction for the Clean Air Act. These RIAs were prepared by EPA’s Office of Air and Radiation and issued between November 1990, the effective date of the Clean Air Act Amendments of 1990, and December 1995. Specifically, our review focused on whether the RIAs clearly describe (1) key economic assumptions subject to uncertainty and the sensitivity of the results to these assumptions, (2) the extent to which benefits and costs were quantified for the proposed regulatory action, and (3) the extent to which alternative approaches were considered. This report also follows up on our previous recommendation that RIAs include executive summaries highlighting the analytical results important to decisionmakers.
subsections: section_title: Background
paragraphs: The Clean Air Act Amendments of 1990 required EPA to issue a series of new or stricter regulations to address some of the more serious air pollution problems, including acid rain, toxic air pollutants, motor vehicle emissions, and stratospheric ozone depletion. In view of the estimated billions of dollars in annual costs to implement these and other requirements, the Congress required EPA to report on the benefits and costs of the agency’s regulatory actions under the 1990 amendments, as well as under previous amendments to the act. Specifically, section 812 of the 1990 amendments required EPA to (1) conduct an analysis of the overall impacts of the Clean Air Act on public health, the economy, and the environment, (2) report on the estimated benefits and costs of the regulations implemented under clean air legislation enacted prior to 1990; and (3) biennially update its estimates of the benefits and costs of the Clean Air Act, beginning in November 1992. In May 1996, EPA drafted a report that examined the benefits and costs of the 1970 and 1977 amendments to the act. EPA is currently in the process of compiling its first prospective study evaluating the benefits and costs of the 1990 amendments.
Section 812 also required GAO to report on the benefits and costs of the regulations issued to meet the requirements of the 1990 amendments. In a February 1994 report, we described the methodologies that EPA had used and the progress that the agency was making.
In addition, since 1971 a series of executive orders and directives by OMB have required EPA and other federal agencies to consider the benefits and costs associated with individual regulations. In February 1981, President Reagan issued Executive Order 12291, which required federal agencies, including EPA, to prepare RIAs that identify the benefits, costs, and alternatives for all proposed and final major rules that the agencies issued. Subsequently, in September 1993, President Clinton issued Executive Order 12866 replacing Executive Order 12291 and directing federal agencies, including EPA, to assess benefits, costs, and alternatives for all economically significant regulatory actions.
OMB and EPA have developed guidelines for conducting the benefit-cost analyses required by these executive orders. While describing the components to be included in these analyses, the guidance affords EPA’s program offices considerable flexibility in preparing RIAs. Specifically, EPA’s guidance stipulates that the level and precision of analyses in RIAs depend on the quality of the data, scientific understanding of the problem to be addressed through regulation, resource constraints, and the specific requirements of the authorizing legislation. This guidance also states that the amount of information and sophistication required in benefit-cost analyses depend on the importance and complexity of the issue being considered.
The recently enacted Small Business Regulatory Enforcement Fairness Act of 1996 provides that before a rule can take effect, the agency preparing it must submit to GAO and make available to the Congress, among other things, a complete copy of any cost-benefit analysis of the rule. This act also provides for congressional review of major rules issued by federal agencies, including EPA, and the potential disapproval of such rules by the enactment of a joint resolution.
section_title: Extent to Which Economic Assumptions Were Identified, Explained, and Subjected to Sensitivity Analysis
paragraphs: Eight of the RIAs that we examined did not clearly identify the values assigned for key economic assumptions, such as the discount rate and value of human life, used to assess the economic viability of the regulations. Furthermore, we found that in the RIAs that identified key economic assumptions the rationale for the values used was not always explained. While EPA’s guidance suggests that RIAs account for uncertainties in such values by conducting sensitivity analyses that show how benefit-cost estimates vary depending on what values are assumed, many RIAs used only a single value and did not always provide a clear explanation. Appendix I summarizes the results of our examination of the economic assumptions used in the 23 RIAs.
Five of the 23 RIAs did not indicate whether the estimated future benefits and costs were discounted. The discount rate can have a significant effect on the estimated impact of an environmental regulation. For example, most environmental regulations impose immediate costs, while the benefits are realized in the future. In such a case, a lower discount rate has a more positive effect on future benefits, thus enhancing the regulation’s perceived value. Conversely, using a higher discount rate makes benefits that occur in the future appear less valuable. Not clearly indicating the discount rate used in benefit-cost analyses makes it more difficult for decisionmakers to assess the desirability of a proposed regulation.
EPA’s guidelines recognize that there may be uncertainties about which discount rates should be used. Moreover, EPA’s Director of the Office of Economy and Environment stated that there are uncertainties associated with choosing discount rates for conducting benefit-cost analyses. As a result, EPA’s guidance suggests the use of sensitivity analyses to show how benefit and cost estimates are affected by different discount rates. Of the 18 RIAs that clearly identified the discount rates used, 5 showed the sensitivity of their estimates to different rates ranging from 2 to 10 percent. Thirteen of the RIAs used a single rate.
Although 14 RIAs indicated that the reduction in mortality was an expected benefit, five did not indicate the value placed on a human life. Of the nine RIAs that indicated the value placed on a human life, eight included sensitivity analyses to indicate how their benefit estimates were affected by different values assumed for a life. Assigning a relatively high value for human life can have a significant positive effect on estimated benefits. However, for the nine RIAs that assumed a value for a human life, the ranges used were not always explained. For example, one RIA assumed a value of human life that ranged from $1.6 million to $8.5 million, and another, prepared in the same year, assumed a value of human life that ranged from $3 million to $12 million. In both instances, the RIAs did not provide a clear explanation of the rationale for the values that were used.
subsections: section_title: EPA’s Views on Economic Assumptions
paragraphs: Because of the agency’s concern about the use of different values for key assumptions and the extent to which sensitivity analyses were used to account for uncertainties about the appropriate values for these assumptions, EPA recently formed an Economic Analysis Consistency Task Group under the direction of the Regulatory Policy Council to develop information on the causes of inconsistencies in the agency’s RIAs. The Council is chaired by EPA’s Deputy Administrator.
In addition, EPA officials explained that the authorizing legislation for some environmental regulations is often a key determinant in the thoroughness of the agency’s benefit-cost analyses. For example, they said that health-based national ambient air quality standards issued by the agency are not based on costs or other economic considerations. However, costs may be considered when developing and implementing control strategies for these standards. Although benefit-cost analyses are completed for these regulations, they do not directly impact the regulatory decision-making process. Therefore, the level of analysis and the number of alternatives analyzed could be more limited.
Time constraints imposed by statutory and court-ordered deadlines and shortages of resources and staff also restrict EPA’s ability to conduct comprehensive benefit-cost analyses. Given the limited resources and staff available for completing economic analyses, EPA officials stated that they assign a higher priority to benefit-cost analyses supporting regulations facing imminent deadlines, regulations expected to have greater economic impacts on society, and those for which the economic analysis has the highest potential to affect the regulatory alternative selected.
section_title: Extent to Which Benefits and Costs Were Quantified
paragraphs: OMB’s and EPA’s guidelines encourage EPA to quantify, to the extent feasible, all potential regulatory benefits and costs in monetary terms, but the guidance recognizes that assigning reliable monetary values to some benefits may be difficult, if not impossible. When benefits and costs cannot be described in monetary terms, the guidance recommends that RIAs include quantitative and qualitative information on the benefits and costs associated with the proposed regulations. The benefits mentioned in the guidance include reduced mortality, reduced morbidity, improved agricultural production, reduced damage to buildings and structures, improved recreational environments, improved aesthetics, and improvements in ecosystems.
Our review of the 23 RIAs indicated that while all of them assigned dollar values to the costs of proposed regulations, 11 assigned dollar values to estimated benefits. EPA acknowledges that assigning monetary values to projected benefits is more difficult than assigning values to the costs of regulatory actions. According to EPA officials, the uncertainty of the science and inadequacy of other data often prevent the agency from estimating dollar benefits. For example, EPA’s guidance recognizes that assigning a monetary value to reduced health risks, a potentially significant benefit, is difficult because of uncertainties about the precise relationship between different pollution levels and corresponding health effects and the appropriate monetary values to be assigned to reductions in mortality and reduced risks of individuals’ experiencing serious illnesses. Estimating the monetary value of improvements in ecosystems, another potentially significant benefit, is even more complex.
Although some RIAs did not assign dollar values to benefits, all 23 of the RIAs we examined contained other quantitative or qualitative information on the benefits of the proposed regulations. When benefits cannot be assigned dollar values, quantifying the benefits, such as a reduced incidence of deaths and illnesses, helps clarify the impact of proposed regulations. For example, an RIA for the National Recycling and Emissions Reduction Program’s regulation estimated that 76,500 fewer cases of skin cancer and 1,400 fewer deaths from skin cancer would occur because of the regulation. Qualitative information is also helpful to decisionmakers because it gives them a more complete understanding of the overall benefits of regulations. Nineteen of the RIAs discussed qualitative benefits, such as increased crop yields, improvements in ecosystems, and reduced damage to buildings and other structures.
Recognizing the difficulties associated with assigning dollar values to benefits, EPA’s guidelines state that cost-effectiveness analyses can assist decisionmakers in comparing the desirability of various regulatory alternatives. We found that 20 of the RIAs we examined included the results of cost-effectiveness analyses, such as the cost per ton of reduced emissions.
section_title: Extent to Which Alternative Approaches Were Considered
paragraphs: OMB’s and EPA’s guidelines require EPA to identify and discuss in RIAs the regulatory and nonregulatory alternatives for mitigating or eliminating the environmental problems being addressed and to provide the reasoning for selecting the proposed regulatory action over other alternatives. While EPA’s guidance recommends that RIAs consider four major types of alternatives—voluntary actions, market-oriented approaches, regulatory approaches within the scope of the authorizing legislation, and regulatory actions initiated through other legislative authority—it states that the number and choice of alternatives to be selected for detailed benefit-cost analyses is a matter of judgment. While it was not always clear how many alternatives or what types of alternatives were considered, our examination of the 23 RIAs indicated that 6 of them compared a single alternative, which was the regulatory action being proposed, to the baseline, which was the situation likely to occur in the absence of the regulation—the status quo. All other RIAs compared two or more alternatives to the baseline. Figure 1 shows the results of our examination of the number of alternatives that EPA considered in the 23 RIAs.
A major goal of RIAs is to develop and organize information on benefits and costs to clarify trade-offs among alternatives. EPA’s guidance states that RIAs should provide decisionmakers with a comprehensive assessment of the implications of alternatives. EPA officials acknowledged that it is not always clear in the RIAs which alternatives were actually analyzed. They stated that some alternatives are excluded before the benefit-cost analyses are performed because of noneconomic reasons, such as statutory language that precludes EPA from using certain approaches.
section_title: Extent to Which Executive Summaries Were Used
paragraphs: In our 1984 report, we recommended that future RIAs prominently include executive summaries that (1) clearly recognize all benefits and costs, even those that cannot be quantified; (2) identify a range of values for benefits and costs subject to uncertainty, as well as the sources of uncertainty; and (3) compare all feasible alternatives. While 13 of the 23 RIAs that we examined included executive summaries, some of these RIAs only briefly discussed the types of information that we recommended they contain. For example, the executive summary for the RIA on the regulation for national emissions standards for coke ovens contained a limited discussion of the uncertainties underlying the analysis, and the executive summary for the RIA on the operating permits program’s regulation included only two sentences on the three alternatives that EPA considered.
In contrast, the executive summary for the RIA supporting the regulation on phasing out ozone-depleting chemicals presented a relatively thorough discussion of the results of the benefit-cost analysis. For example, it included a range of cost estimates, qualitative and quantitative benefit estimates, discussions of scientific and economic uncertainties, and estimated benefits and costs for baseline conditions and three alternatives. The prominent display of this type of information in the executive summary makes it easier for decisionmakers to locate the information they need without searching through hundreds of pages in the body of the RIAs.
EPA officials acknowledged that some of the RIAs did not include executive summaries and agreed that executive summaries that include information such as descriptions of the difficulties in assigning dollar values to benefits, uncertainties of the data, and regulatory alternatives are useful. However, they stated that time constraints and limited resources and staff often determine whether they prepare executive summaries and the amount of detail that is included when summaries are done.
section_title: Conclusions
paragraphs: We believe that improvements in the presentation and clarity of information contained in EPA’s RIAs would enhance their value to both agency decisionmakers and the Congress in assessing the benefits and costs of proposed regulations. EPA’s guidelines state that the goal of RIAs is to provide decisionmakers with well-organized, easily-understood information on the benefits and costs of major regulations and to provide decisionmakers with a comprehensive assessment of the implications of alternative regulatory actions. However, many of the RIAs we reviewed did not clearly identify key economic assumptions, the rationale for using these assumptions, the degree of uncertainty associated with both the data and the assumptions used, or the alternatives considered. Not clearly displaying this information makes it difficult for decisionmakers and the Congress to appreciate the range and significance of the benefit and cost estimates presented in these documents.
section_title: Recommendations
paragraphs: To help EPA decisionmakers and the Congress better understand the implications of proposed regulatory actions, we recommend that the EPA Administrator, ensure that RIAs identify the (1) value, or range of values, assigned to key assumptions, along with the rationale for the values selected; (2) sensitivity of benefit and cost estimates when there are major sources of uncertainty; and (3) alternatives considered, including those not subjected to benefit-cost analyses.
section_title: Agency Comments
paragraphs: We provided a draft of this report to EPA and OMB for review and comment. We obtained comments from EPA officials, including the Director, Office of Economy and Environment, and representatives of the Office of Air and Radiation. EPA officials stated that the information in the report was accurate and agreed with the recommendations in the report. They provided specific comments on a number of issues, which we have incorporated into the report, including a clarification of the objectives of the Economic Analysis Consistency Task Group. According to EPA officials, this group is in the process of identifying key issues associated with benefit-cost analyses that offer the potential for greater consistency in the agency’s RIAs. Among the issues being considered are the valuation of reductions in the risk of mortality, discount rates and baselines, intergenerational issues, and distribution effects. Additionally, they emphasized that greater consistency in addressing key issues in the RIAs would enhance their usefulness for EPA’s decisionmakers. EPA views this as an ongoing process and anticipates that it will result in revisions to the agency’s guidelines for preparing economic analyses. OMB did not provide comments on the draft report.
We conducted our work from February 1996 through February 1997 in accordance with generally accepted government auditing standards. A detailed discussion of our scope and methodology is contained in appendix II. We are sending copies of this report to the Administrator, EPA; the Director, Office of Management and Budget; and other interested parties. Copies are also available to others on request.
Please call me at (202) 512-4907 if you or your staff have any questions. Major contributors to this report are listed in appendix III.
section_title: Economic Assumptions Used in the Environmental Protection Agency’s Regulatory Impact Analyses
paragraphs: Discount rates (percent)
Value of life (dollars in millions)
RIA for the National Recycling and Emission Reduction Program (1)
RIA for the National Recycling and Emission Reduction Program (2)
RIA of Nitrogen Oxides Regulations—1993 (continued)
Discount rates (percent)
Value of life (dollars in millions)
These are real discount rates, which exclude the effects of inflation.
Nine of these RIAs did not identify reduced mortality as a benefit associated with a proposed regulation. Therefore, assigning a monetary value for a human life was not applicable.
section_title: Scope and Methodology
paragraphs: We examined 23 RIAs issued by the Office of Air and Radiation between November 1990, the effective date of the Clean Air Act Amendments of 1990, and December 1995. Eighteen of these RIAs supported regulations that were estimated to cost $100 million or more annually and therefore were considered economically significant. Five RIAs supported regulations that were considered major or significant by the Environmental Protection Agency (EPA) because of their potential impact on costs and prices for consumers, the international competitive position of U.S. firms, or the national energy strategy or because they were statutorily required by the 1990 amendments. To determine the number of the RIAs, we interviewed officials from EPA’s Office of Policy, Planning, and Evaluation and Office of Air and Radiation, which has four program offices—the offices of Air Quality Planning and Standards, Mobile Sources, Atmospheric Programs, and Radiation and Indoor Air—and examined EPA’s database of completed RIAs. Although EPA’s other program offices are also responsible for preparing RIAs, we limited our review to the RIAs prepared by the Office of Air and Radiation because this office is primarily responsible for implementing the requirements of the 1990 amendments.
We reviewed Executive Orders 12866 and 12291 and EPA’s and the Office of Management and Budget’s guidance on the preparation of RIAs under these executive orders. From those documents, we identified the key components of RIAs and reviewed the 23 selected RIAs for their handling of these components. We also discussed issues affecting the clarity of RIAs with officials of the Office of Air and Radiation and Office of Policy, Planning, and Evaluation.
section_title: Major Contributors to This Report
subsections: section_title: Resources, Community, and Economic Development Division
paragraphs: William F. McGee, Assistant Director Charles W. Bausell, Jr., Adviser Harry C. Everett, Evaluator-in-Charge Kellie O. Schachle, Evaluator Kathryn D. Snavely, Evaluator Joseph L. Turlington, Evaluator The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000  or by using fax number (301) 258-4066, or TDD (301) 413-0006.
Each day, GAO issues a list of newly available reports and testimony.  To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone.  A recorded menu will provide information on how to obtain these lists.