metadata: authors: 0: author: Harrington, Winston
1: author: Morgenstern, Richard
2: author: Velez-Lopez. Daniel
count: Mexico,United States
docna: 0: docna: Tools for assessing the costs and benefits
            of green growth : the U.S. and Mexico
docty: Policy Research Working Paper
subtopic: Health Care Services Industry,Employment and Unemployment,Energy Demand,Energy and Environment,Energy and Mining,Green Issues,Global Environment
teratopic: Environment,Transport,Public Sector Development,Macroeconomics and Economic Growth
abstracts: cdata!: This paper examines the processes used
            in the United States and Mexico to assess the economic costs
            and benefits of environmental improvement, the kinds of
            information obtained from these procedures, and the
            additional knowledge that is needed about both elements to
            improve understanding of the problems and prospects of
            advancing a green growth agenda. Because environmental and
            other development needs are large and resources are limited,
            it is important to choose the best projects, those with the
            highest returns on both public investments and private
            resources harnessed by regulation. The United States is
            well-established as a world leader in the use of
            quantitative methods to evaluate options for environmental
            regulation and policy. Mexico represents a case where a
            developing country has made clear advances in reforming its
            economy and in introducing transparency in its regulatory
            processes for environmental and other policy areas.
geo_reg_key: 80475,80450,80465
repnme: repnme: Tools for assessing the costs and benefits
            of green growth : the U.S. and Mexico
display_title: Tools for assessing the costs and
            benefits of green growth : the U.S. and Mexico
listing_relative_url: /research/2012/10/16840589/tools-assessing-costs-benefits-green-growth-mexico
topicv3: Green Growth m1326430 476,Energy and Extractives m1326595 507,Environment, Natural Resources and Blue
            Economy m1327391 566,Unemployment m1504051 2747,Health m1504661 3480
publishtoextweb_dt: 2010-10-01T00:00:00Z
historic_topic: Environment,Transport,Public Sector Development,Macroeconomics and Economic Growth
no_of_pages: 36
model_extractions: WPS6242 Policy Research Working Paper 6242 Tools for Assessing the Costs and Benefits of Green Growth The U.S. and Mexico _Winston Harrington Richard Morgenstern Daniel Velez-Lopez_ The World Bank Development Research Group Environment and Energy Team & Sustainable Development Network Office of the Chief Economist October 2012
Policy Research Working Paper 6242

# **Abstract**

This paper examines the processes used in the United States and Mexico to assess the economic costs and benefits of environmental improvement, the kinds of information obtained from these procedures, and the additional knowledge that is needed about both elements to improve understanding of the problems and prospects of advancing a green growth agenda. Because environmental and other development needs are large and resources are limited, it is important to choose the best projects, those with the highest returns on both public investments and private resources harnessed by regulation. The United States is well-established as a world leader in the use of quantitative methods to evaluate options for environmental regulation and policy. Mexico represents a case where a developing country has made clear advances in reforming its economy and in introducing transparency in its regulatory processes for environmental and other policy areas.

This paper is a product of the Environment and Energy Team, Development Research Group, and the Office of the Chief Economist, Sustainable Development Network, in the World Bank. It was produced for the Green Growth Knowledge Platform (www.greengrowthknowledge.org), a joint initiative of the Global Green Growth Institute, Organisation for Economic Co-operation and Development, United Nations Environment Programme, and the World Bank. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are posted on the Web at http:// econ.worldbank.org. The authors, Winston Harrington, Richard Morgenstern and Daniel Velez-Lopez may be contacted at harrington@rff.org, Morgenst@rff.org and velez@rff.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Tools for Assessing the Costs and Benefits of Green Growth: The U.S. and Mexico**

Winston Harrington, Richard Morgenstern and Daniel Velez-Lopez Resources for the Future October 2012

# **Abstract**

This paper examines the processes used in the United States and Mexico to assess the economic costs and benefits of environmental improvement, the kinds of information obtained from these procedures, and the additional knowledge that is needed about both elements to improve understanding of the problems and prospects of advancing a green growth agenda. Because environmental and other development needs are large and resources are limited, it is important to choose the best projects, those with the highest returns on both public investments and private resources harnessed by regulation. The United States is well-established as a world leader in the use of quantitative methods to evaluate options for environmental regulation and policy. Mexico represents a case where a developing country has made clear advances in reforming its economy and in introducing transparency in its regulatory processes for environmental and other policy areas.

Key words: cost-benefit analysis. Green growth.

JEL codes: D61. O44. Q52.

Sectors: Environment, Public Sector Governance This paper is a product of the Environment and Energy Team, Development Research Group, and the Office of the Chief Economist, Sustainable Development Network, in the World Bank. It was produced for the Green Growth Knowledge Platform (www.greengrowthknowledge.org), a joint initiative of the Global Green Growth Institute, Organisation for Economic Co-operation and Development, United Nations Environment Programme, and the World Bank. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are posted on the Web at http:// econ.worldbank.org. The authors, Winston Harrington, Richard Morgenstern and Daniel Velez-Lopez may be contacted at harrington@rff.org, <u>Morgenst@rff.org</u> and velez@ **rff** .org.
2
# **I.  Introduction**

This paper examines the processes used in the United States and México to assess the economic costs and benefits of environmental improvement, the kinds of information obtained from these procedures, and what more we need to know about both elements to improve understanding of the problems and prospects of advancing a green growth agenda. Because environmental and other development needs are large and resources are limited, it is important to choose the best projects, those with the highest returns on both public investments and private resources harnessed by regulation. The U.S. is an obvious choice for study since it is well established as a world leader in the use of quantitative methods to evaluate options for environmental regulation and policy. At the same time, Mexico represents a case where a developing country has made clear advances in reforming its economy and in introducing transparency in its regulatory processes for environmental and other policy areas.

The strategy of this paper is straightforward. Following this brief introduction, Section II discusses the background and context. Section III examines the U.S. case, with an emphasis on the history, rationale and processes associated with the use of benefit-cost analysis (BCA) of environmental regulation. Based on three decades of experience, the literature on _ex post_ analyses of the costs of environmental regulation is reviewed in some detail, along with comparisons to the _ex ante_ studies conducted as part of the now standard regulatory process. There is also consideration of the potential employment impacts of new regulations, including a description of different methodologies used. Section IV focuses on Mexico. It examines the recent advances in regulatory analysis, including an assessment of the requirements for _ex ante_ BCA of regulations. It also includes a critical review of several recent BCAs and a glimpse at some limited _ex post_ analyses that the Mexican government has undertaken. Section V offers concluding remarks, and a series of recommendations for future study of environmental regulation in support of an international green growth agenda.

# **II.  Background and Context**

The costs of environmental policies are generally thought to be easier to measure than the benefits, and while that may be true, it in no way implies that cost estimation is easy or without controversy. What is supposed to make cost estimation easier is that one can rely on market prices to estimate the costs of the resources employed. But in the wake of the worst global economic crisis in 80 years, one that has idled thousands of businesses and thrown millions out of work, even that assumption is unreliable. When there is high unemployment, market prices may be poor measures of opportunity costs. Indeed, it is in part the prospect of making these resources productive again that makes the green growth story so compelling. Environmental policies that cost jobs in a high-unemployment 3
environment, on the other hand, can have outsized effects on social welfare, especially if prolonged or concentrated in pockets of unemployment (Masur and Posner 2011).

Thus, living in a world of high unemployment raises both the difficulty and the value of policy analysis in all areas of policy, not excepting green growth. Though what is meant by “green growth” is not always clear and may vary from speaker to speaker, we mean a hierarchy of activities that put a premium on the simultaneous advancement of economic and environmental goals. The gold standard is achieved by those projects in which economic growth and improved environmental quality are complements, not substitutes. Beyond that, we would look for projects for which the benefits, public and private, exceed the costs, where the latter are defined in economic rather than accounting terms and fully reflect environmental and social costs.

In the wake of efforts to produce economic assessments of initiatives to achieve environmental quality and other policy goals, researchers and policy analysts have moved to examine the results after implementation, so-called _ex post_ analysis, to determine whether and to what extent the initial policy assessment was borne out in practice. These efforts have been especially prominent in reference to social welfare and poverty programs. In the United States, these efforts go back to the 1960s, especially regarding the War on Poverty. Soon afterwards, every major welfare program was likely to be subjected to some kind of _ex post_ evaluation, and the requirement to do so was written into major federal statutes (Manski and Garfinkel 1992).

These efforts have encouraged adoption of methodological innovations in medicine and other fields for the study of major interventions. Most notable are the use of randomized methods to increase variation and control for selection bias. While such randomized experiments have some limitations (Heckman 1991), they have achieved a good deal of credibility in examining a variety of interventions in multiple settings.

Recent years have seen a big increase in such empirical work (both experimental and non-experimental) in developing countries, making development economics one of the most dynamic subject areas in economics. Much of this work has been directed at trying to understand the puzzles of economic growth and development: why certain interventions work in some countries or regions but not others, why interest rates remain so persistently high in the developing world, etc. Some of this work has also touched on public health and environmental quality. For example, Cohen and Dupas (2007) use a randomized trial to investigate whether giving away treated bed nets causes them to be undervalued and unused by recipients. In a non-randomized setting, Duflo and Pande (2007) examine the results of dam construction on income and its distribution in India. Because the number of dams in a region is likely to be endogenous, the authors develop an instrument for the number of irrigation dams based on the average regional gradients, which are assumed to be exogenous. This allows the investigators to develop robust 4
estimates of the effects of dams on income and income distribution. The results are striking and suggest that dams are disappointing investments, with an ROI of about 1 percent. They also find that the benefits primarily accrue to those inhabitants living downstream, while those living upstream suffer losses.

In Mexico, _Oportunidades_ is the principal anti-poverty program focusing on improving the education, health, and nutrition of children in rural and urban communities. _Oportunidades_ was the first social program in Mexico to carry out a rigorous independent evaluation of program impacts that included randomly assigned treatment and control groups. Given the size and ambitions of the program, officials emphasized the importance of accurate, credible data as a strategy to measure effectiveness. The results of an early evaluation focusing on the rural program showed important positive impacts on school enrollment, health clinic attendance, and nutrition (Hoddinott, et al., 2000). A 2008 study found that urban children in intervention families younger than 6 months of age at baseline grew more in both height and weight than children in comparison families. Overall, they found that _Opportunidades_ is “an effective tool to improve the growth of infants in poor urban households.” (Leroy, et al., 2008) Unfortunately, these experimental methods have led to only limited improvements in understanding the performance and outcomes of environmental regulations, either in the developed or the developing world. Experimental determination of the effects of regulation necessarily requires a situation where some regions are subject to the regulation and some are not. Despite some differences across states in the U.S., it is almost never the case that the division between the regulated and unregulated regions is random. That is not to say that there will be no influence of empirical studies on understanding the effects and costs of regulation, but that influence will have to be indirect, through a better understanding of firm and individual behaviors. For now, though, _ex post_ examination of regulations will be primarily limited to relatively simple estimations of the direct cost and benefits of regulations, based on surveys or observed price changes and measured changes in environmental quality.

As discussed further below, benefit–cost analysis (BCA) and regulatory impact analysis (RIA) are systematic processes for calculating and comparing benefits and costs of a project or regulation. To its proponents, the principal advantages of a well-executed BCA are the incentives it provides to regulators to think in quantitative terms about regulatory outcomes and to consider whether and how much those changes matter, as expressed in monetary terms. BCA also increases transparency and accountability; provides a systematic framework for consistent data collection and the identification of gaps and uncertainty in knowledge; encourages the development of metrics for both the beneficial and adverse consequences of alternative regulatory approaches, allowing for comparison; and provides the ability to aggregate dissimilar effects into one measure of net benefits based on the monetary metric.

5
To its critics, BCA is a flawed technique that excessively emphasizes the quantification and monetization of health and safety risks and environmental amenities, trivializes the future through discounting, fails to meaningfully assess the value of avoiding catastrophe, and ignores distributional concerns. For a more detailed discussion of the advantages and disadvantages of BCA, we direct the reader to Harrington, et al. 2009.

# **III** . **The U.S. Regulatory System**

Requirements for economic analysis have been a part of the U.S. regulatory system for at least three decades. Although the roots of U.S. efforts to evaluate the benefits and costs of new regulations can be traced to Executive Orders (E.O.s) issued during the 1970s governing “Quality of Life Reviews” and “Inflation Alerts”, formal procedures for the conduct of benefit-cost analysis of environmental, health and safety regulations were first established in 1981, in E.O.12291. That Order required that major regulations, defined as those with an effect on the economy of $100 million or more, undergo a Regulatory Impact Analysis (RIAs) prior to issuance in the _Federal Register_ . Interestingly, the $100 million cutoff was set in nominal as opposed to real, inflation-adjusted terms. Thus, given the virtual doubling of price levels over the past thirty years, the real cutoff is currently about $50 million in 1981$.

E.O. 12866, issued in 1993, and E.O. 13563, issued in 2011, reaffirmed the key components of regulatory benefit and cost estimation, as well as the review process by the Office of Management and Budget (OMB), and placed emphasis on the transparency of the review. E.O.s 12866 and 13563 also put a heightened focus on the non-quantified consequences of major federal rules.

Regulations aim to establish specific requirements about which types of behaviors are legal and which are not. For example, a regulation issued by EPA to implement the Clean Air Act might determine what levels of a pollutant - such as sulfur dioxide - adequately protect human health and the environment. It also tells firms how much sulfur dioxide they can legally emit into the air, and what the penalty will be if they emit too much. BCAs and RIAs are systematic processes for calculating and comparing benefits and costs of a project or regulation. The results of such analyses can help identify whether a regulatory option is expected to have net benefits and also whether it may be net beneficial compared to other regulatory options. More generally, RIAs serve to inform decision-makers in regulatory agencies, Congress, and the public about the expected outcomes of regulatory decisions.

BCA compares the expected costs and benefits (expressed in money terms) of each option in a "present value" framework. Individual well-being is assumed to depend on the satisfaction of individual preferences, and monetary measures of welfare change are derived by observing how much individuals are willing to pay or give up in terms of 6
other consumption opportunities. This approach can be applied to nonmarket public goods such as environmental quality or risk reduction as well as to market goods and services, although the measurement of nonmarket values is more challenging. When measurement of nonmarket values is not feasible, a _cost-effectiveness_ framework is often used _,_ in which a policy goal or outcome is taken as given, and the analysis seeks to identify the least-cost means for achieving the goal.

The key elements of a high-quality benefit-cost analysis are:

- identification of a market failure,

- establishment of a credible baseline,

- description of the alternatives considered,

- identification and discussion of possible unanticipated consequences,

- the scope of the costs addressed,

- the scope and nature of the benefits considered,

- appropriate comparisons of regulatory outcomes occurring at different times, and

- the treatment of uncertainty.

Beyond BCA, a complete RIA includes an examination of the distributional consequences of the regulation, and a description of the key non-quantified, nonmonetized costs and benefits. Formal guidelines for the preparation of RIAs have been issued by the OMB, the EPA, and other regulatory agencies.<sup>1</sup> Under U.S. law, BCA can play several different roles in the regulatory process. Some U.S. statutes, for example, the Safe Drinking Water Act Amendments of 1996 explicitly call for a formal BCA in deciding on the scope and stringency of regulation. Other statutes, such as the Toxic Substances Control Act, require agencies to conduct a generalized balancing of costs and benefits in making regulatory decisions. In contrast, sections of the Clean Air and Water Acts, and others limit or restrict consideration of costs, benefits, or net benefits in choosing a regulatory approach.

Individual regulations and the accompanying RIAs are prepared by the EPA or other relevant agencies. The Office of Information and Regulatory Affairs (OIRA) in the OMB is responsible for the oversight and coordination of regulations across the federal government, including agriculture, energy, transportation, information technology, housing, manufacturing, immigration, food safety, health care, public health, occupational safety and health, environmental protection and criminal justice. The only regulations not overseen by OIRA are those which, as a matter of law, are issued by

> 1 For example see USEPA (2010).

7
independent regulatory commissions, not subject to Presidential oversight, such as those involving nuclear plant safety, consumer product safety, and most financial regulations.<sup>2</sup> Before a new regulation in the U.S. is adopted, it must be published in the _Federal Register_ in proposed form, with an opportunity for public comment and, in some cases, a formal public hearing. When it is published in the _Federal Register_ as a final rule, written explanations of any important revisions and responses to public comments are included. Major regulations must be cleared by OIRA.

Figure 1 categorizes the major rules submitted to OMB by agency over the period 20002010. As shown, the EPA accounts for the most rules issued by any single agency, or about 30 percent of the total. In terms of economic impact, EPA rules account for about half the total costs and 60-80 percent of the total benefits. (OMB 2011, page 13).

> 2 For a discussion of rules excluded from RIA requirements, see Fraas and Lutter 2011).

8
# **Figure 1**

<!-- Start of picture text --> Major Rules and Regulations Submitted to OMB<br><!-- End of picture text --> <!-- Start of picture text --> Department of Agriculture<br>Department of Energy<br>Department of Health and Human Services<br>Department of Homeland Security<br>Department of Housing and Urban<br>Development<br>Department of Justice<br>Department of Labor<br>Department of Transportation<br>Environmental Protection Agency<br><!-- End of picture text --> Not surprisingly, a great deal has been written about the techniques use by the EPA and other U.S. regulatory agencies, as well as about the results of the analyses performed on regulations _before_ they are issued in the _Federal Register._ For a discussion of these issues, we refer the reader to Harrington et al. (2009). In contrast, there is a relatively small literature on regulatory studies performed _after_ the rules have been implemented. In the next section we present recent information on _ex post_ assessments and, in particular, we compare the _ex ante_ and _ex post_ analyses.

# **_Ex post_ v.** **_ex ante_ analysis of regulation**

_Ex post_ analysis of the costs (and other consequences) of government regulations is relatively uncommon, especially in comparison to the large number of _ex ante_ studies. Harrington, Morgenstern and Nelson (2000, hereafter HMN) surveyed the peer-reviewed and gray literature for published comparisons of _ex ante_ and _ex post_ cost estimates of specific social regulations.<sup>3</sup> The focus was on environmental and workplace safety regulations in the U.S., but all case studies that became known to the authors were

> 3 Also see earlier studies by Putnam, Hayes and Bartlett (1980), the U.S. Office of Technology Assessment (1995), and by Goodstein (1997).

9
considered, regardless of country or subject area. The researchers relied primarily on colleagues for help and also posted requests on various academic list servers. Several replies were received to the effect that there are “plenty of studies out there; you shouldn’t have any trouble”. However, on checking these leads, the authors only found a handful that were genuine before-and-after comparisons of regulatory costs. In the end, HMN found only 25 cases, even after broadening the search to state-level rules and those from other countries. In almost all cases, the constraint on case selection was the unavailability of an _ex post_ study, or at least an _ex post_ study that estimated regulatory costs. However, there were a few cases where it was the _ex ante_ study that was missing, in part because HMN only selected cases where the _ex ante_ estimate was prepared by the government, or at least used by the government in the preparation of the RIA or environmental impact assessment (EIA) accompanying the proposed regulation.

Tables 1 and 2 summarize the HMN results. Three metrics were examined: effectiveness, total costs and unit costs. Effectiveness refers to the measured improvement, e.g. changes in air quality for an air pollutant regulation or reductions in workplace accidents for a safety regulation issued by the Occupational Health and Safety Administration (OSHA). The distinction between unit costs and total costs allows for the observation of those cases where regulations were not completely implemented. For EPA’s vehicle inspection and maintenance program, for example, it found total costs to be overestimated and unit costs to be underestimated. In fact, it would not be unfair to say that total costs (and effects) were overestimated precisely _because_ unit costs were underestimated. Because of unexpected difficulties and disappointing results this program was only implanted in a few metropolitan areas, rather than in most nonattainment areas as EPA had originally envisioned.

HMN defined the estimates to be “accurate” if the _ex ante_ estimate was within 25 percent of the _ex post_ estimate. As shown in tables 1 and 2, overestimation of effectiveness and of both cost metrics was the most common outcome, but there was no apparent bias in the estimate of unit costs. Overestimation of effectiveness was particularly dominant, perhaps a reflection of the relative ease by which interested parties in the U.S. regulatory system can challenge rulemakings and obtain exemptions of different types. Also, estimation of total cost was more common than estimation of unit cost, possibly a reflection of the difficulty of estimating effectiveness. Interestingly, although incentivebased rules constituted less than a third of the total sample (8 out of 28), both the total and unit costs of rules employing economic incentives rather than direct regulation were much more likely to be overestimated.

10
# **Table 1: Case Study Results, Regulation Type**

||Accurate|Overestimate|Underestimate|Unable to|
|---|---|---|---|---|
|||||Determine|
|**All Regulations**|||||
|Quantity Reduction|13|9|4|2|
|Unit Cost|8|14|6|0|
|Total Cost|6|14|3|5|
|**U.S. Federal**<br>**regulations**|||||
|Quantity Reduction|11|9|1|0|
|Unit Cost|7|8|6|0|
|Total Cost|5|12|3|1|
|**EPA Regulations**|||||
|Quantity Reduction|8|4|1|0|
|Unit Cost|4|5|4|0|
|Total Cost|4|6|2|1|
|**U.S. Federal Rules**<br>**Excluding Pesticides**|||||
|Quantity Reduction|6|8|1|0|
|Unit Cost|6|6|3|0|
|Total Cost|4|9|1|1|
|**Pesticide Bans**|||||
|Quantity Reduction|5|1|0|0|
|Unit Cost|1|2|3|0|
|Total Cost|1|3|2|0|
|**OSHA Regulations**|||||
|Quantity Reduction|3|5|0|0|
|Unit Cost|3|3|2|0|
|Total Cost|1|6|1|0|

11
# **Table 2: Case Study Results, Characteristics of Federal Regulations**

Accurate Overestimate Underestimate Unable to Determine

|**Pre-1981**|||||
|---|---|---|---|---|
|Quantity Reduction|1|4|0|0|
|Unit Cost|1|2|2|0|
|Total Cost|0|5|0|0|
|**Post-1981**|||||
|Quantity Reduction|10|5|1|0|
|Unit Cost|6|6|4|0|
|Total Cost|7|5|3|1|
|**>$100M/yr**|||||
|Quantity Reduction|5|7|1|0|
|Unit Cost|4|5|4|0|
|Total Cost|3|8|1|1|
|**<$100M/yr**|||||
|Quantity Reduction|6|2|0|0|
|Unit Cost|3|3|2|0|
|Total Cost|2|4|2|0|
|**Regulations using**|||||
|**Economic Incentives**|||||
|Quantity Reduction|3|1|4|0|
|Unit Cost|1|7|0|0|
|Total Cost|2|4|0|2|

12
# **_Ex post –ex ante comparison studies since 2000_**

In 2006 OIRA updated and extended the HMN analysis as part of its Annual _Report to Congress_ (OMB 2006). This study includes a few analyses overlooked by HMN as well as work that appeared after its publication, nearly doubling the sample size to 47 case studies, including studies of regulations by agencies that were not part of the original HMN tally. The new analyses included studies of vehicle safety rules promulgated by the National Highway Traffic Safety Administration (NHTSA), none of which make it into HMN; results of a study of OSHA workplace safety rules by Seong and Mendeloff (2004); a study of vehicle emission rules by Anderson and Sherwood (2002); and a study of EPA pesticide rules (Gianessi 1999)<sup>4</sup> ; a study of Department of Energy (DOE) rules for energy efficiency in household appliances (Dale et al. 2002) and two nuclear plant safety rules issued by the Nuclear Regulatory Commission (NRC).

For some rules in the OMB sample, either benefits or costs are not quantified. For a number of OSHA safety rules there are no cost estimates, because the source of information is a paper by Seong and Mendeloff (2004), which focuses on comparing expected change in fatalities estimated in the RIAs with estimates of fatality reductions following implementation. For Department of Energy (DOE) appliance efficiency standards no benefit estimates are cited. Again, in the source document (Dale et al. 2002), the focus is on the estimates of the cost of energy efficiency. In principle, it should have been possible to determine the effects of the rule, because the change in the energy efficiency of the appliances should be known. In any event, for both these types of regulations it was assumed that the missing estimate (costs or benefits) were accurate for the purposes of calculating a change in benefit-cost ratio. Pesticides represent a slightly different case since the rules in question were bans on pesticide uses for certain crops in particular regions.

OMB scoring criteria are slightly different from HMN, especially for assessing the effectiveness of rules. Whereas HMN stopped at examining the “effects” of regulation in physical terms, OMB used a hierarchy. The most desirable benefit criterion is the total monetized effects of the regulation, if available in both the _ex ante_ and _ex post_ study. If this metric is not available, the benefits are the quantitative effects of the regulation, just as in HMN. OMB’s preferred metric for assessing accuracy is the change in benefit-cost ratio. If it is overestimated—that is, higher in the _ex ante_ versus the _ex post_ estimate, that means the performance of the regulation is not as good as was predicted in the _ex ante_

> 4 The Gianessi study of EPA pesticide cancellations (for particular crops in designated areas) was sponsored by RFF and was ongoing at the time the HMN manuscript was in preparation. We took results from the six finished case studies. In all, 31 case studies were completed, but after excluding pesticide rules for which farmers applied for and received emergency or permanent exemptions, 26 remain..

13
study. This rarely caused differences in results between HMN and OMB, since the valuation of a given level of benefits almost never changes between the _ex ante_ and _ex post_ estimates.

The OMB study concludes that both benefits and costs are more likely to be overestimated than underestimated, with the benefit overestimation substantially larger than the cost overestimation. As a result, the benefit-cost ratio is overestimated far more often than it is underestimated. In other words, the predicted performance of the regulation is better than its actual performance. Thus, the conclusions of the OMB study differ slightly from those of HMN.

However, the OMB assessment does not include all rules examined in the studies published since 2000. As shown in the Table 3, which compares the number of analyses used in the different studies, OMB did not include in its sample all of the rules for which new _ex post_ studies had been completed, nor did OMB make it clear what the basis of inclusion or exclusion was. In particular, the Dale et al. (2002) paper on the DOE energy efficiency rules has results for 11 appliance rules; OMB included only 6. Also, OMB included only 13 of 26 available pesticide rules.

When the studies omitted by OMB are included, we have a database of 74 studies. Table 4 shows the results, now expressed in terms of benefit-cost ratios to conform to the units used in the OMB tally. That is, “over” means the benefit-cost ratio is overestimated in the _ex ante_ study, so that regulatory performance is not as good as predicted in the _ex ante_ study. The top panel shows the detailed count used in the OMB study. The lower part of the table shows the studies excluded by OMB, followed by the addition of the excluded studies and the original OMB results. Final results are displayed with and without the pesticide results. Since the pesticide results are for particular kinds of rules, i.e., pesticide cancellations, it seemed appropriate to display the overall results both with and without them. As shown, OMB’s original analysis found that the most common result was for the BC ratio to be overestimated, i.e. the rule turned out to be not as favorable as predicted _ex ante_ . When the excluded rules are included, however, the OMB result is reversed, i.e., for a slight plurality of rules the BC ratio is underestimated. This result obtains whether the pesticide studies are included or not.

14
**Table 3. Available Regulatory Comparisons by Agency**

|Agency<br>(1)|HMN<br>(2)|OMB<br>(3)|Minimum<br>available<br>(4)|
|---|---|---|---|
|NHTSA|0|8|8|
|OSHA health|7|6|8|
|OSHA safety|1|7|7|
|NRC|0|2|2|
|DOE appliance|0|6|11|
|EPA mobile source|3|2|8|
|EPA pesticide|6|13|26|
|EPA other|5|3|5|
|Total|22|47|74|
|Total excluding EPA pesticide|16|34|48|

# **Table 4. Benefit-Cost Ratios: Summary of Revised OMB Results with New Cases Added**

||Accurate|Over|Under|
|---|---|---|---|
|OMB<br>||||
|In validation chapter|11|22|14|
|Excluding all Gianessi (1999) pesticide cases|2|4|6|
|Excluding remaining contested cases|0|1|0|
|Net OMB|9|17|8|
|Added cases||||
|OSHA health studies (asbestos and vinyl<br>chloride)|1|0|1|
|DOE appliance standards (Dale et al. 2002)|0|1|4|
|Mobile source fuel regulations (AS)|2|0|1|
|Mobile source vehicle emissions regulations|0|0|1|
|Unadjusted pesticide cases (Gianessi (1999))|4|3|9|
|Net added cases|7|4|16|
|Final tally|16|21|24|
|Excluding pesticide cases|12|18|15|

15
# **_Further observations on regulatory comparisons_**

Regardless of one’s take on the outcomes discussed in the preceding sections, the results demonstrate the value of _ex post_ analysis. Why, then, is it so rare, especially when so many close observers claim to be in favor of it? One answer is that it depends on what sort of _ex post_ analysis you mean. There are plenty of _ex post_ studies of policy effectiveness and overall progress at achieving policy goals. In the environmental sphere EPA closely monitors air and water quality, among other things. They also routinely conduct studies of the overall benefits and costs of programs, often retrospective studies like the benefits and costs of a clean environment, which EPA is mandated to prepare and send to Congress periodically. Some analyses are also carried out by academic researchers, such as an oft-cited paper by Greenstone (2002) on the effect of nonattainment designation on regional economic performance and air quality. Still other researchers carry out studies of the overall costs of regulations, attempting to assess the importance of regulation of various kinds relative to the economy as a whole. But these studies do not compare costs and benefits of individual regulations. Yet, it is the such uncommon _ex post_ studies that are particularly valuable for assessing the quality of regulatory processes.

One reason why there are so few _ex post_ studies of individual regulations is that there is no obvious or consistent source of funding. _Ex post_ studies are not routinely performed by regulatory agencies in the U.S. Rather, they generally appear for idiosyncratic reasons, largely dependent on the initiative of individual researchers. _Ex post_ studies also must contend, typically, with serious problems of data acquisition and interpretation. Compliance cost data are likely to be considered business confidential by regulated firms, whose participation in cost studies is often voluntary anyway. And even if the data are available, there are likely to be difficult issues of joint cost allocation. OMB’s work collecting the available _ex post_ studies is valuable, and it is to be hoped that OMB revisits the question periodically.

A related question is why certain kinds of regulations are more apt to be studied in retrospect than others. For example, in the earlier HMN survey of comparison studies (Harrington, et al. 2000, Harrington 2006), it was striking how such a large fraction of the _ex post_ studies identified were economic incentive regulations. In part, HMN interpreted this as an avuncular interest on the part of economists, who have chosen to examine these particular policies because they find them more interesting from a professional point of view. In addition, with economic incentive regulations it may be easier to get information on _ex post_ regulatory costs. For cap-and-trade programs, the marginal cost of compliance _ex post_ is equal to the permit price for every participant in the market. For emission-fee programs, the marginal compliance cost equals the unit fee, so it is the same _ex post_ as it is _ex ante_ . The total cost, of course is not quite so easy to get, since one must still estimate the cost of the infra-marginal units. Nonetheless, even total cost is much easier 16
to estimate for incentive-based rules than for direct regulatory programs, because of the anchoring provided by knowing the cost at one point. Economic incentive policies appear to be unusual in still another way. As noted, it is far more likely for the costs of economic incentive regulations to be overestimated than for other types of regulations. Presumably this is due to the wider range of choices that regulated entities have in responding and the difficulty for regulatory analysts to anticipate what those opportunities are.

Despite its value, it is important to keep in mind that this sort of analysis is far from perfect. There are no doubt significant costs and benefits of particular regulations that are not counted in either _ex ante_ or _ex post_ regulatory assessments. Some of the uncounted costs include the efforts that a firm subject to a new regulation must undertake to determine what is required and to figure out how to comply (probably small) , as well as the effect on rent seeking (which could be more substantial for some regulations). Benefits are often uncounted for effects that are real but difficult to monetize, such as the value of ecological preservation.

_Consideration of Employment Impacts._ Although regulatory-induced employment impacts are not typically included in RIAs, the employment issue is attracting increasing attention in policy circles, especially in the current period of high unemployment. Accordingly, the U.S. EPA has added a section entitled ‘Impacts on Employment’ to its most recent _Guidelines for Preparing Economic Analyses_ (EPA, 2010). In its new guidelines, EPA recognizes that the full employment assumption underlying most BCAs may not always be appropriate. It also recognizes that while higher manufacturing costs can lead to fewer sales and lower employment in a given industry, it is faulty to assume that employment losses in a given industry are proportional to output reductions, i.e., if production goes down by 1 percent, it is not necessarily the case that employment goes down by 1 percent.

In place of the proportionality approach, the EPA embraces a framework originally proposed by Morgenstern, Pizer and Shih (2002) which decomposes the labor consequences in an industry facing increased abatement costs into three separate components:

- _Demand effect:_ Higher production costs raise market prices.  Higher prices reduce consumption (and production) reducing demand for labor within the regulated industry;

- _Cost effect:_ As production costs increase, plants use more of all inputs including labor to produce the same level of output.  For example, pollution abatement activities require additional labor services to produce the same level of output; and

17
- _Factor-shift effect:_ Post-regulation production technologies may be more or less labor intensive, as more /less labor is required per dollar of output.

Morgenstern, Pizer and Shih (2002) empirically estimate a model incorporating these different effects of regulation on employment for four highly polluting/regulated industries using data for the period 1979-91. Importantly, they measure regulation by the plant’s expenditures on pollution controls, as self-reported in the Commerce Department’s survey on Pollution Abatement Control Expenditures (PACE).

While there are some differences across the four industries, Morgenstern et al. find that increased abatement expenditures generally do _not_ cause a significant change in employment at the industry level. Since these estimates do not consider the effect that regulation in one industry may have on other industries that make complementary or substitute products, the results should not be interpreted as relevant to the whole economy, but they do highlight the complex responses to the new regulations exhibited by plants in the four industries.

Interestingly, other studies have focused on air pollution control induced changes in employment, using measures of regulation tied to specific policy actions rather than to self-reported pollution control expenditures. Examples include the stringent air quality standards imposed in the South Coast Air Quality Management District in the 1990s, and the nationwide attainment/nonattainment designations used for criteria air pollutants, as well as the realignment of such designations following the 1990 Clean Air Act Amendments.

In an early study, Berman and Bui (1997) found small, statistically insignificant employment _gains_ associated with the more stringent South Coast air quality rules, consistent with the later findings of Morgenstern et al. In contrast, in the analyses of federal attainment/nonattainment designations, studies by Greenstone (2002) and Walker (2011) found adverse employment impacts associated with the regulations.

Importantly, all three of these studies use measures of regulation tied to specific policy actions rather than self-reported pollution control expenditures. The actual modeling is based on fixed effects, which only provides a relative indication of impact compared to the control group. Thus, even when employment is found to decline in a more stringently regulated area compared to a less stringently regulated one, it does not necessarily represent a net loss in jobs across all the areas. In fact, it may simply be a job _shift_ from one area to another. Without additional information on the nature of the impact of the regulation in the less stringently regulated area, it is not possible to determine if regulation caused a net loss or possibly even a net gain in employment across the different areas.

18
A further employment-related issue concerns the overall welfare impact of any regulation-induced job loss. As Masur and Posner (2011) note, the standard RIA assumption is that in a full employment world there are few transitional impacts associated with any individual job loss, as displaced workers can readily find comparable work elsewhere. Yet, as they note, especially in times of high unemployment, there are likely to be some lost wages, as well as the potential for depreciation in workers’ human capital and increased family instability. If properly monetized, such impacts could be substantial. Importantly, if the results were actually included in a formal BCA, it could influence the desired degree of stringency of a rule. Although the quantification and monetization of these effects is limited, further analysis may reveal this to be a nontrivial effect.

# **IV. The Mexican Regulatory System**

Mexico is a particularly interesting case to examine regulatory issues in the context of green growth in a developing nation. A recent World Bank study reviewed the Worldwide Governance Indicators for 212 countries and territories, measuring six dimensions of governance, 1996-2007 (Kaufman et al., 2008): Voice and Accountability, Political Stability and Absence of Violence/Terrorism, Government Efficiency, Regulatory Quality, Rule of Law, and Control of Corruption. The aggregate indicators are based on 35 data sources provided by 32 different organizations, reflecting the views of experts, as well as citizen and firm survey respondents worldwide. Despite its relatively low scores in the areas of Political Stability and Absence of Violence/Terrorism, Rule of Law, and Control of Corruption, Mexico scores reasonably well in the area of Regulatory Quality, ranking above the median among all 212 countries for this category (OECD). Mexico also shows modest improvements in Regulatory Quality for the three most recent years in the study. Interestingly, Mexico’s scores for Regulatory Quality are significantly higher than for other large, industrial countries in Latin America (with the exception of Chile, which scores higher than all other countries in the region). This section describes various aspects of regulation in Mexico, with particular emphasis on green growth issues. Unsurprisingly, the focus is on _ex ante_ analysis. There are, however, a limited number of evaluations of existing regulations that were recently published in a government report that provide some useful information about regulatory performance.

Starting in the year 2000, Mexico introduced various legal, institutional and policy changes to its regulatory system, including the creation of the Federal Regulatory Improvement Commission ( _Comisión Federal de Mejora Regulatoria_ , COFEMER), a commission charged with the role of ensuring regulatory quality. As an independent body of the Ministry of the Economy, COFEMER promotes reforms across all agencies of 19
government, including the Ministry of the Environment and Natural Resources ( _Secretaría de Medio Ambiente y Recursos_ Naturales, SEMARNAT).

COFEMER is responsible for overseeing the process for preparing federal regulations and for promoting and developing cost effective rules that produce net benefits. Its principal activities are:

- elimination and simplification of business and citizens formalities;

- transparent and analytical review of all draft regulations and their RIAs;

- diagnosis of and proposals to reform existing laws and regulations in specific areas or economic sectors; and

- support for state and municipal regulatory improvement programs.

Among its accomplishments is the establishment of the Rapid Businesses Start-up System ( _Sistema de Apertura Rápido de Empresas_ , SARE), which provides support to firms in complying with federal, state and municipal regulations. With respect to environmental regulation _per se,_ the legal basis originates from the General Law of Ecological Equilibrium and the Protection of the Environment ( _Ley General de Equilibrio Ecologico y Proteccion del Ambiente_ , henceforth referred to as LGEEPA), initially adopted in 1988, with multiple reforms (the latest in 2011). As in the case of most U.S. environmental statutes, LGEEPA provides very general guidelines for environmental policy, as well as the administrative tools for the execution of this policy. The preliminary norms of this law are quite broad, such as:

- “To guarantee the right of every person to live in an environment that is appropriate for their development, health, and well-being.”

- “Preservation, restoration, and improvement of the environment.”

- “The sustainable use, preservation, and when necessary, the restoration of the ground, water and additional natural recourses, so that the acquisition of economic benefits and society’s activities can be compatible with the preservation of the ecosystems.”

- “The prevention and control of air, water, and ground pollution.”

SEMARNAT, which can most readily be compared to the U.S. EPA, is authorized under the LGEEPA to enact specific regulations that require compliance from state, local, and private actors. These regulations, known as NOMs (Norma Oficial Mexicana), allow SEMARNAT to establish enforceable requirements for pollution levels as well as for the use of natural resources. While adoption of NOMs is the most common method that SEMARNAT uses for regulation, other judicial vehicles can also be used (e.g., Reglamentos, which typically do not contain specifics about enforcement).

20
The LGEEPA gives SEMARNAT the authority to determine levels of pollution necessary to protect health and well-being, as well as to provide for the preservation and/or restoration of natural resources and the protection of the environment. NOMs can be used to encourage economic agents to adopt specific technologies. They can also help provide long-term certainty about the costs that economic agents will have to bear due to environmental regulation. Although NOMs can specify a particular technology or process, most NOMs are issued in the form of performance standards which allow some flexibility for the regulated entities.

# **_Benefit-Cost Analysis and Regulation in Mexico_**

The implementation and amendment of NOM’s and other mechanisms are subject to the Federal Law of Metrology and Normalization (Ley Federal Sobre Metrología y Normalización, hereupon referred to as LFMN). LFMN covers issues unrelated to NOM’s, but it also establishes the format, specifications, and the process for implementation of NOM’s, as well as the administrative aspects necessary for enacting environmental regulation.

LFMN also sets out the process for submitting proposed regulations to COFEMER. This process includes a public discussion of the regulation before it is adopted. When proposing a regulation, SEMARNAT must develop an RIA, which includes the justification for the new regulation. RIAs should include a listing of the advantages (benefits) and disadvantages (costs) of the new regulation compared to alternative approaches, as well as the procedures for monitoring and compliance. In the case of high impact regulations, the RIA must also include a separate, more extensive BCA.

COFEMER plays a similar role in the Mexican system as OMB does in the U.S. It reviews both the draft and the RIA and asks for corrections or additional information to the RIA, if it lacks substantial information regarding market conditions, legal issues, unjustified obligations, potential effects to the proposal, formalities, etc. During the designated review period, the proposed regulation and its corresponding RIA are made available to the public for comment and discussion, much like in the U. S.. In our review of RIAs submitted to COFEMER 2000-2010, we found that only about 7 percent of submissions came from SEMARNAT as opposed to other Mexican agencies. Across all agencies, about 11 percent of the RIAs submitted to COFEMER during this period were classified as high impact. For SEMARNAT’s submissions to COFEMER, the proportion deemed high impact was slightly more than 2 percent. As shown in figure 2, final approval of environmental rules was typically granted in about five weeks in 2009.

21
# **Figure 2:**

<!-- Start of picture text --> Duration of regulatory improvement<br>process<br>60<br>50<br>40<br>30 2008<br>2009<br>20<br>10<br>0<br>A B C D<br>Days<br><!-- End of picture text --> Key: Route A: Total approval with final effects Route B: Preliminary approval + Final approval Route C: Adjustments and corrections + Total approval with final effects Route D: Adjustments and corrections + Preliminary approval + Final approval Source: “Redesign of the Regulatory Improvement Process”, COFEMER, July 26, 2010.

As in the U.S., the RIA process establishes a set of standard criteria to determine if the regulation is justified:

- an explanation of the underlying situation and why the government needs to be involved;

- a justification of the specific obligations imposed on private agents; an analysis of the potential effects of the regulations; and

- the identification of requirements created, modified or eliminated by the rule.

Until recently, any regulation that had expected annual costs greater than MEX$800 million was routinely considered high impact. If a regulation specifically affected a particular industry, area of commerce, or geographic population it also may be considered 22
high impact. In late 2010, COFEMER adopted a multidimensional scoring system for determining whether or not a regulation is high impact. This scoring system is based on an online calculator keyed to both quantitative and qualitative measures. The calculator includes questions on the industries that are affected by the regulation, the cost of the regulation, the number of consumers affected, expected compliance problems, etc. In conversations with officials we have been told that there are a few variables that often trigger a high impact designation, including the number of consumers, and expected compliance problems. Particular attention is also paid to strategic industries like petroleum, electricity, and transportation. As in the U.S., while the calculator and the previous cutoff are explicit determinants for being a high impact classification, COFEMER has discretionary authority to request that any regulation be submitted as high impact if they consider it appropriate.

The content of the BCA varies across regulations. Detail and depth can be quite different between high and moderate impact rules, but there are also differences within categories. The RIA form (which was altered after the calculator was adopted in 2010) requires enumeration of all the groups that will be affected by the regulation and some estimate of the magnitude of the impacts. Both quantifiable and non-quantifiable costs and benefits are considered. If COFEMER finds the analysis unsatisfactory, the law permits COFEMER to request that an expert be hired to execute a new version of the BCA.

Both the U.S. and Mexico have criteria to determine whether a proposed rule is “major.” In each country these criteria consists of a numerical size limit that must be satisfied, plus a set of additional considerations that may trigger a major analysis even if it does not meet the numerical limit. In the U.S., as noted, the size limit is $100 million in projected annual benefits or costs. In addition, rules with costs concentrated by region or industry are RIA-worthy, as are rules that affect unique or especially valued resources. Prior to the 2010 changes in procedures, a rule was subjected to major analysis in Mexico if it had projected annual cost of at least MEX$800 million, or if its costs were concentrated in a particular region or industrial sector. Unlike the U.S., no level of estimated benefits was by itself sufficient to trigger a major analysis.

MEX$800 million pesos is about $60 million at current exchange rates or $100 million in terms of purchasing power parity. Either way, the size criteria are roughly comparable in dollar terms in the two countries. However, the U.S. economy is about ten times the size of the Mexican economy. Thus, unless Mexican regulations are considerably more stringent on a per ton basis than comparable U.S. rules, it seems that the more detailed economic analysis is considerably less likely to be required in Mexico than in the U.S. Perhaps as a response to such a high cut-off, COFEMER has changed its scoring system for high and moderate impact regulations. Unsurprisingly, since the new classification system was introduced the number of SEMARNAT submissions to COFEMER 23
designated high impact has almost quadrupled as a percent of the total, from 1.7 percent to 6.6 percent.

In examining the conduct of BCA in Mexico, we have reviewed the analyses for the two most recent NOMs designated by SEMARNAT as ‘high impact’: NOM-085, which involves emissions from indirect heating equipment used in power generation (2008); and NOM-044 for emissions from diesel engines used in large vehicles (2006). We also reviewed the BCA for the Reglamento on highly risky activities, also designated as high impact (2011). We have characterized these analyses in terms of the cost and benefit calculations performed, the methods used, and whether or not alternatives were considered.

As shown in table 5, a somewhat mixed picture emerges from this review. All three analyses developed quantitative estimates of both costs and benefits. In one of the analyses, they addressed the issue of unquantified costs, and in two of the analyses they considered unquantified benefits. At the same time, the methods used in the three BCAs differed markedly. One analysis, for NOM-085, was based on the costs of adopting specific technologies. The emission reductions, in turn, also assumed use of the same technologies. Another regulation, NOM-044, relied principally on previous studies conducted by SEMARNAT and by the World Bank to make extrapolations about both the costs and benefits of the proposed regulation. The third, the Reglamento for highly risky activities, focused on the procedural costs involved in undertaking the required studies to gain approval by SEMARNAT. As regards alternative options, we see that two of the three BCAs did consider some alternatives, while the third (NOM-085) analyzed the possibility of regulating additional pollutants but did not consider different stringency levels for the initial pollutants.

In still another interesting case, regulation NOM-086-SEMARNAT-SCIFI-2005, a relatively high cost rule, an exemption from the RIA requirements was requested. This rule required a large investment on the part of state-owned PEMEX to reduce the sulfur content of gasoline and diesel at its refineries. Ultimately, an exhaustive BCA was performed for the project for “fuel quality improvement” and is perhaps the most complete BCA that we encountered in our review of Mexican regulations. However, this BCA was not part of any formal RIA submitted to COFEMER. Rather, it was conducted independently through the National Institue of Ecology ( _Instituto Nacional de Ecología_ , INE) as a BCA for a project, not a regulation.. Possibly the rule was not submitted to COFEMER because PEMEX is a public entity, and regulations going through COFEMER should pertain only to private costs. Or, perhaps PEMEX was already making the investment to improve productivity and the environmental benefits were achieved at zero marginal costs.

24
At first blush, this rather mixed set of analyses suggests a lack of uniformity in compliance with the BCA requirements. This result, impressionistic as it is, however, based on a sample of three, is generally consistent with the results of an often-cited, quite rigorous, review of U.S. RIAs. Work by Robert Hahn and colleagues (2000, 2002) examined 48 RIAs in federal agencies in the United States prepared between 1996 and 1999, paying particular attention to two aspects: whether the RIAs met the requirements of executive order 12866, and whether they satisfied the guidelines produced by the OMB. Their conclusion: RIAs often do not contain all the required items. Whereas 90 percent of the RIAs examined monetized costs, only 50 percent monetized benefits and 29 percent calculated net benefits. Just two-thirds of the RIAs discussed alternatives to the regulation, and only 25 percent of the cases calculated benefits and costs of alternatives.

A follow-up study by Hahn and Dudley (2007) focused exclusively on the EPA, based on a sample of 74 RIAs issued by the agency between 1982 and 1999, roughly balanced among the Reagan, Bush, and Clinton administrations. Using a scorecard similar to the earlier studies, the authors found that while all the RIAs monetized at least some costs, not all provided estimates of total costs. Beyond that, a significant percentage of the analyses done by the EPA did not report the benefits, consider alternatives to the selected policy option, or adequately discount costs and/or benefits. Further, Hahn and Dudley reported no statistically significant changes in the observed quality of RIAs over the 17year sample period or across the three presidential administrations studied. Overall, they concluded that the EPA RIAs fall far short of the formal requirements, although they also observed a great deal of variation among individual RIAs.

25
**Table 5:  Review of Three Recent Mexican BCAs**

|Regulation|Quantified<br>Costs|Unquantified<br>Costs|Monetized<br>Benefits|Unquantified<br>Benefits|Alternatives<br>Considered?|Methodology|
|---|---|---|---|---|---|---|
|NOM-085<br>for<br>emissions<br>from<br>indirect<br>heating<br>equipment<br>used in<br>power<br>generation|Yes|No|Yes|No|No (only<br>considered<br>regulating<br>additional<br>pollutants)|Calculated<br>cost of<br>technology<br>adoption and<br>corresponding<br>emissions<br>reduction for<br>each unit|
|NOM-044<br>for<br>emissions<br>from diesel<br>engines used<br>in large<br>vehicles|Yes|Yes|Yes|Yes|Yes<br>(considered<br>a voluntary<br>program)|Extraction<br>from other<br>study|
|Rules for<br>regulating<br>highly risky<br>activities|Yes|No|Yes|Yes|Yes<br>(considered<br>existing<br>regulation|Quantified<br>procedural<br>costs of<br>undertaking<br>studies to<br>receive<br>approval|

# **_Ex Post Analysis of Regulation_**

As part of their efforts to improve the regulatory system, the LMFN requires that regulations be revised every five years. This legal requirement provides regulators with an opportunity to conduct at least a limited _ex post_ analysis to determine how effective individual regulations have been in achieving their environmental goals, while assessing other factors including compliance levels, costs, and enforcement.

26
The newest set of evaluations conducted by SEMARNAT, through it Sub-Ministry of Promotion and Regulation ( _Subsecretaría de Fomento y Normatividad_ ), does not provide specific cost or benefit estimates of the selected regulation, nor does it provide any comparison between the evaluations and the _ex ante_ estimates of the type that the OMB does in the U.S. Rather, these evaluations use a rudimentary grading system for assessing regulations based on four metrics: effect or impact, effectiveness, efficacy, and efficiency.

- Effect or impact measures how successful the regulation has been at achieving the environmental goals for which it was intended.

- Effectiveness measures how successful the regulation has been at forcing the regulated to alter their behavior in order to comply with said regulation.

- Efficacy is based on the viability of enforcing said regulation. This can be based on technical, legal and economic questions that determine whether enforcing the regulation is possible.

- Efficiency determines whether the benefits of a regulation outweigh the costs.

The recent study evaluates sixteen specific NOM’s that were enacted between 1996 and 2006, as displayed in table 6. Because of obstacles such as the lack of data for evaluating outcomes (much less levels and costs of compliance), the study is limited to a qualitative rating system. Values for each variable range from 0 to 4 where 0 represents “none” and 4 “very high”. Despite the lack of detail in these evaluations, the combination of the grades for each variable and the brief descriptions that are provided can be valuable in assessing and improving regulation.

27
**Table 6:  Ex Post Evaluation of Mexico’s Environmental Regulation**

|Regulation|Regulation Description|Effect or<br>Impact|<br>Effectivenes|s Efficacy|Efficiency|Average<br>Rating|<br>Observations|
|---|---|---|---|---|---|---|---|
|NOM-062-SEMARNAT-1994|Specifications for mitigating the adverse effects of the<br>change in forest and farmland use on biodiversity|0|0|0|0|0.00|Has zero effect, because there are other<br>policyinstruments withgreater authority|
|NOM-001-SEMARNAT-1996|Maximum allowed levels of contamination for waste water<br>discharges|0|1|2|0|0.75|Little to no compliance or changes in<br>polluter behavior|
|NOM-027-SEMARNAT-1996|Procedures, criteria, and specifications for the use,<br>transport,and storage of mountain earth|1|1|2|2|1.50|Mixed knowledge and compliance due to<br>lack of supervision|
|NOM-003-SEMARNAT-1997|Maximum allowed levels of contamination for wasted water<br>for treatment|2|2|1|4|2.25|Compliance is economically beneficial rather<br>than forced onpolluters|
|NOM-120-SEMARNAT-1997|Environmental protection specifications for direct<br>exploratory mining in climates with specific flora|0|0|0|0|0.00|There is confusion about application of<br>regulation. Therefore, it is not applied.|
|NOM-047-SEMARNAT-1999|Characteristics of the measurement equipment and<br>procedure for verifying the emissions limits for pollutants<br>for motor vehicles in circulation|2|2|3|3|2.50|In complying states, there is a drop in air<br>pollution. Fines are the main enforcement<br>tool.(Applied with 041)|
|NOM-133-SEMARNAT-2000|Specifications for dealing with polychlorinated biphenyls<br>contamination|1|1|2|1|1.25|Competes with an international agreement<br>for authority.|
|NOM-040-SEMARNAT-2002|Maximum levels of atmospheric emissions for cement<br>production|3|3|3|4|3.25|Low compliance costs because no<br>equipment changes are necessary|
|NOM-098-SEMARNAT-2002|Operation specifications and pollutant limits for waste<br>incineration|2|4|3|4|3.25|While compliance is expensive, incineration<br>operations are still economically viable.|
|NOM-055-SEMARNAT-2003|Requirements for places designated for controlled storage<br>of dangerous, previouslystabilized,waste|0|0|3|0|0.75|Conflicts with legislation regulating this<br>activity. No regulatedparties|
|NOM-083-SEMARNAT-2003|Requirements for places designated for solid municipal<br>waste as a function of daily tonnage|1|1|3|1|1.50|It is more expensive to comply than to pay<br>the fines. There are higher levels of<br>compliance among private firms than<br>|
||||||||municipalities(57% overall)|
|NOM-115-SEMARNAT-2003|Environmental protection specifications that must be met<br>for perforation and maintenance of land petroleum wells|3|3|4|4|3.50|Cost of compliance are very low for the<br>single regulated agent, leading to high<br>compliance rates.|
|NOM-137-SEMARNAT-2003|Sulfur emissions controls for desulfurization plants for gas<br>and condensate|4|2|4|3|3.25|The only agent subject to the regulation<br>changed its technology base to comply<br>before the regulation was enacted.|
|NOM-141-SEMARNAT-2003|Procedure for the development of talings dams|2|3|2|3|2.50|Mixed compliance and very moderate<br>environmental effects due to the nature of<br>dam lifetime and use.|
|NOM-041-SEMARNAT-2006|Maximum allowed levels of emissions of polluting gases<br>from the exhaust of gasoline vehicles|2|2|3|3|2.50|In complying states, there is a drop in air<br>pollution. Fines are the main enforcement<br>tool.(Applied with 047)|
|NOM-045-SEMARNAT-2006|Regulates exhaust emissions of diesel vehicles through<br>exhaust opacitymeasurements|1|1|0|1|0.75|Only 15 states apply tests. Compliance has<br>led to minimal environmental effect|
Our review of these _ex post_ evaluations yields some interesting observations. The presence of four regulations that have no environmental effect at all is particularly striking. Whether this is due to noncompliance or other factors in unknown. We also see that regulations with higher efficiency grades tend to have a higher effect or impact grade. In fact, regulations with an effect or impact grade of 3 or 4 seemed to have relatively high grades across the board. Further, we observe regulations for a single party (PEMEX), which tend to have high grades for all categories. This probably happens because of the large amount of consultation that takes place when planning regulations that will impose high compliance costs on PEMEX and other government controlled enterprises. Finally, despite the small sample size (16), we looked without success for evidence of a trend in quality over time. However, there is no evidence of either an increase or a decline in quality of the 10 year period, 1996-2006.

# **V. Conclusion**

The use of quantitative assessments of both the costs and benefits of new regulation to advance the international green growth agenda is clearly a work in progress. Especially in the case of new rules that could impose significant costs on public and/or private sectors, there is a strong impetus to develop an in-depth understanding of the expected outcomes before the rules are put in place. Yet, unlike the burgeoning efforts to evaluate anti-poverty efforts in developing countries, randomized experiments are not typically suitable to environmental/green growth initiatives. Thus, decision-makers are dependent on relatively simple estimates of the direct costs and benefits of proposed actions.

In our review of progress in the use of quantitative BCA assessments in the U.S. and Mexico, we observe a number of similarities and differences. Notwithstanding the relatively long U.S. track record in this area, both nations are seen as having embraced the BCA approach in support of the green growth agenda. Both nations have developed specific requirements and programs to advance their efforts, including the institutionalization of processes enforcing and managing the systems. Compared to most developing nations, Mexico has taken clear steps to strengthen its regulatory system.

Unsurprisingly, there are also important differences between the BCA activities in the two countries. With its advanced and internationally respected research base in both natural and social sciences, the U.S. is widely seen as a world leader in the quantification and monetization of both benefits and costs of environmental, health and safety regulations. It is also a leader in promoting the evolving work on retrospective analyses of both costs and benefits. It is, perhaps, comforting to know that the initial results of _ex post_ studies from the U.S. suggest that while the total costs of new regulations are often overestimated, unit costs tend to be generally accurate. That is, while regulators often anticipate that greater reductions will be achieved than actually occurs –the principal
source of the cost overestimates – they tend to be relatively accurate in their calculation of the per ton cost burdens imposed.

Of interest is the fact that both nations have established clear numerical criterion for conducting a rigorous BCA, although the Mexican criteria seem less stringent. Also of interest is the incomplete nature of the analyses conducted in both nations. Based on a quite limited assessment of the Mexican BCAs, we find quite significant gaps in the analyses conducted. Unsurprisingly, these gaps occur in some of the very same areas that have been identified as problematic for the U.S. Interestingly, the U.S. is expanding the scope of its analyses to consider employment impacts of new regulations, a topic not previously emphasized in RIAs.

In terms of recommendations for expanding the role of quantitative assessments to advance the green growth agenda, we highlight the need for expanded efforts in both the U.S. and Mexico. Notwithstanding its early efforts in the area of _ex post_ analyses, a key to validating and improving the quality and credibility of the BCAs, the U.S. has just scratched the surface on this topic. There is much more to do. As noted, the critical issue here is the lack of resources available to conduct impartial _ex post_ studies. A further area for the U.S. is to strive for greater consistency in the methods used, and for more complete RIAs, including the consideration of alternative policy options.

Mexico faces serious challenges in a number of areas. Even though its regulatory program gets relatively high marks compared to those in other developing nations, more rigorous and more complete RIAs are clearly a high priority. Only limited work has been done on retrospective analyses in Mexico, an effort that would also advance both the quality and credibility of the Mexican BCAs.

Finally, we return to the increasingly discussed question of how to make the RIA process and the green growth agenda more relevant to the slow growth/high unemployment economies all too common in today’s world. A research agenda in this area might include a number of questions:

- How do regulations affect employment levels in the industries subject to regulation?

- How do regulations affect employment levels in the overall economy?

- For workers who are displaced by regulation, what are the economic impacts on the displaced individuals and, more broadly, on society at large?

- How is the estimate of the economic costs of job loss influenced by the state of the national economy, i.e., low unemployment vs. high unemployment?

30