Methods of Estimating the Total Cost of Federal Regulations
Released: 2016-01-21T00:00:00

Summary:
Federal agencies issue thousands of regulations each year under delegated authority from Congress. Over the past 70 years, Congress and various Presidents have created a set of procedures agencies must follow to issue these regulations, some of which contain requirements for the calculation and consideration of costs, benefits, and other economic effects of regulations. In recent years, many Members of Congress have expressed an interest in various regulatory reform efforts that would change the current set of rulemaking requirements, including requirements to estimate costs and benefits of regulations. As part of this debate, it has become common for supporters of regulatory reform to comment on the total cost of federal regulation.
Estimating the total cost of regulations is inherently difficult. Current estimates of the cost of regulation should be viewed with a great deal of caution.
Scholars and governmental entities estimating the total cost of regulation use one of two methods, which are referred to as the "bottom-up" and the "top-down" approach. The bottom-up approach aggregates individual cost and benefit estimates produced by agencies, arriving at a government-wide total. In 2014, the annual report to Congress from the Office of Management and Budget estimated the total cost of federal regulations to range between $68.5 and $101.8 billion and the total benefits to be between $261.7 billion and $1,042.1 billion. The top-down approach estimates the total cost of regulation by looking at the relationship of certain macroeconomic factors, including the size of a country's economy and a proxy measure of how much regulation the country has. This method estimates the economic effect that a hypothetical change in the amount of regulation in the United States might have, considering that economic effect to represent the cost of regulation. One frequently cited study estimated the total cost of regulation in 2014 to be $2.028 trillion, $1.439 trillion of which was calculated using this top-down approach.
Each approach has inherent advantages and disadvantages. The bottom-up approach relies on agency estimates of the effects of specific regulations and can also be used to estimate benefits, because agencies typically estimate both costs and benefits under current requirements so that they may be compared and evaluated against alternatives. The bottom-up approach does not, however, include estimates of costs and benefits of all rules, nor does it include costs and benefits of regulations that are not monetized—meaning that the bottom-up approach is likely an underestimate of the total cost of regulation. Furthermore, the individual estimates produced by agencies and used in the bottom-up approach may not always be accurate.
The top-down approach can be used to estimate effects of rules that are not captured by the bottom-up approach—such as indirect costs and costs of rules issued by independent regulatory agencies, which are not included in the bottom-up approach—thus theoretically capturing the whole universe of regulatory costs. Its results are, however, entirely reliant upon a number of methodological challenges that are difficult, if not impossible, to overcome. The biggest challenge may be finding a valid proxy measure for regulation: proxy measures of the total amount of regulation in a country are inherently imprecise and cannot be reliably used to estimate macroeconomic outcomes. Because of this difficulty in identifying a suitable proxy measure of regulation, even if the total cost of regulation is substantial, it cannot be estimated with any precision. The top-down method is intended to measure only costs; measuring costs without also considering benefits does not provide the complete context for evaluating the appropriateness of a country's amount of regulation.
For these and other reasons, both approaches to estimating the total cost of regulation have inherent—and potentially insurmountable—flaws. The discrepancy between the two approaches and their associated estimates raises the question of the utility of using such figures in the regulatory reform debate.

subsections: section_title: Estimating the Total Costs and Benefits of Federal Regulation
paragraphs: Federal agencies issue thousands of regulations every year. These regulations are often the means through which various government policies and programs are implemented. Many of these regulations are administrative or routine in nature and have little or no compliance cost associated with them. A number of these regulations can, however, have a substantial effect on the economy in the form of costs, benefits, and transfer payments. Over the past 70 years, Congress and various Presidents have created a set of procedures agencies must follow to issue these regulations, some of which contain requirements for the calculation and consideration of costs, benefits, and other economic effects of regulations. 
Federal regulations are the product of delegated legislative authority from Congress—agencies may promulgate regulations only with the authority from Congress to do so. As such, Congress has shown an interest in conducting oversight of those regulations, both on the individual level for particular regulations and also for the regulatory system as a whole. One way for Congress to conduct oversight of the regulatory system as a whole, some say, is to monitor the total cost and benefits of federal regulation. Comparing the estimated costs against the benefits would provide some insight into the potential tradeoffs of regulation. 
In recent years, many Members of Congress have expressed interest in various regulatory reform efforts that would change the rulemaking process. Proponents of these efforts argue that the system under which agencies currently issue federal regulations is outdated, and that federal agencies should be required to conduct more rigorous economic analysis of their regulations. Opponents of these regulatory reform efforts argue that adding to federal rulemaking requirements could cause fewer regulations to be issued by federal agencies or could create delays in the issuance of federal regulations. As part of this debate, it has become common for supporters of regulatory reform to comment on the total cost of federal regulation.
Estimating the total cost of regulations is inherently difficult. Current estimates of the cost of regulation should be viewed with a great deal of caution.
Scholars attempting to identify an estimate for the total cost of regulation have taken two primary approaches that lead to radically different conclusions about the total economic effect of regulation. In this report, these two approaches are referred to as the "bottom-up" and the "top-down" approaches or methods. In short, the bottom-up approach aggregates individual cost estimates produced by federal agencies. The top-down approach relies on macroeconomic modeling to find a causal relationship between larger economic factors, such as gross domestic product (GDP), and a proxy measure intended to represent the overall amount of regulation.
The two approaches use entirely different methods and produce radically different results. In 2014, the most frequently cited cost estimates resulting from each of these studies ranged from $57-$87 billion (using the bottom-up method) to $2 trillion (the majority—almost $1.5 trillion—of which was arrived at using the top-down method). 
This report analyzes these two approaches for estimating the total cost of federal regulations. In discussing each approach, the report provides an overview of the advantages, a brief case study, and an analysis of the potential issues or inherent problems using the case study to illustrate the concepts.
The objective of this report is not to provide an estimate of the total costs and benefits of federal regulations, but rather, to inform the broader regulatory reform debate by identifying the difficulties in providing such estimates and potential problems inherent in the methods that exist.
section_title: "Bottom-Up" Method: Aggregating Existing Cost Estimates
paragraphs: The first approach to aggregating the total costs and benefits of federal rules is generally referred to as a "bottom-up" approach. 
subsections: section_title: What Is the Bottom-Up Approach?
paragraphs: This method relies on estimates of costs and benefits that agencies produce during the rulemaking process, pursuant to several requirements. The bottom-up approach aggregates these estimates of costs and benefits that agencies calculate in individual rulemakings, using the sum as a government-wide total. Understanding the requirements under which agencies conduct these estimates—specifically, knowing when agencies are required to estimate costs and benefits, and when they are not—is important for understanding the advantages and disadvantages of the bottom-up approach.
The primary requirement for most agencies to calculate estimates of costs and benefits when issuing rules is under Executive Order 12866. That executive order requires covered agencies to assess costs and benefits for "economically significant" rules at the proposed and final rule stage. Economically significant rules are defined in the executive order as those that may "have an annual effect on the economy of $100 million or more or adversely affect in a material way the economy, a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, local, or tribal governments or communities." The term "effect on the economy" means that a rule may be considered economically significant if it has costs or benefits of over $100 million. Other provisions of Executive Order 12866 encourage agencies to consider costs and benefits during the rulemaking process for all rules, although those other provisions do not require a complete, detailed cost-benefit analysis for non-economically significant rules. Executive Order 12866, issued by President Clinton, has remained in effect since 1993, and it was reaffirmed in 2011 by President Obama in Executive Order 13563. The estimates that agencies produce under Executive Order 12866 are subject to review by the Office of Management and Budget (OMB). Specifically, agencies submit their rules and cost-benefit analyses to OMB's Office of Information and Regulatory Affairs (OIRA), the agency within OMB responsible for reviewing regulations and cost-benefit analyses. OMB has issued a number of guidance documents agencies are required to follow when estimating costs and benefits of regulations.
In addition to these executive order requirements, certain statutory requirements for cost-benefit analysis, or other types of regulatory impact analysis, sometimes require agencies to calculate costs, benefits, and other economic effects of rules. The Regulatory Flexibility Act (RFA) requires regulatory impact analyses for proposed and final rules that will have a "significant economic impact on a substantial number of small entities." Title II of the Unfunded Mandates Reform Act (UMRA) requires agencies to analyze and reduce costs associated with federal mandates upon state, local, and tribal governments and the private sector. However, in practice, the RFA and UMRA apply to a fairly small number of rules. Finally, agencies may be required under their own authorizing statutes to calculate and/or consider the costs and benefits of their rules.
The bottom-up approach for estimating costs and benefits aggregates the estimates produced under these requirements, producing a total, government-wide figure for the costs and benefits of regulation. 
section_title: Why Use the Bottom-Up Approach?
paragraphs: The bottom-up approach to estimating the costs and benefits of federal regulation has several potential benefits. First, this approach sums up actual estimates of costs and benefits that agencies have calculated for individual regulations, and, as described above, most of these estimates have undergone review from OMB. 
Second, under the requirements discussed above, agencies estimate both costs and benefits, which allows the bottom-up approach to compare total estimated costs to total estimated benefits. Such information can be valuable for evaluation of cost-effectiveness of regulation generally (i.e., what the benefits received are for the costs invested) and it allows for calculation and evaluation of a ratio of costs to benefits. 
Third, the components of the bottom-up approach to measuring costs and benefits could be validated by conducting analysis ex post , or after the fact, of what the costs and benefits of specific regulations actually turned out to be. This could be done, for example, as part of agencies' retrospective review process, in which agencies reanalyze existing rules and may consider making amendments to those rules in light of the ex post analysis. Agencies do not always conduct retrospective review of their regulations, however, and a retrospective review does not necessarily include a reevaluation of the initial cost-benefit analysis to test its accuracy. But some such studies have been conducted in recent years by scholars and observers of the regulatory process. 
Despite these and other potential advantages of using a bottom-up approach to aggregating costs and benefits, certain issues—and potential caveats—should be taken into consideration. To help illustrate some of these issues, this report first introduces a case study: the most well-known and widely cited bottom-up study of the total costs and benefits of regulation. The report then uses the case study to analyze and discuss some of the problems with a bottom-up study. 
section_title: Case Study: The Annual OMB Report on the Total Costs and Benefits of Federal Rules
paragraphs: The most well-known bottom-up study is the report to Congress on the benefits and costs of federal rules, which OMB compiles annually. 
subsections: section_title: Background on the OMB Report to Congress
paragraphs: Since the 1990s, OMB has estimated the total costs and benefits of federal regulations pursuant to various federal requirements. The initial requirement was in Section 645 of the Treasury, Postal Service and General Government Appropriations Act, 1997, which required the Director of OMB to submit a report by September 30, 1997, that provided—among other things—"estimates of the total annual costs and benefits of federal regulatory programs, including quantitative and nonquantitative measures of regulatory costs and benefits." Similar requirements were contained in other appropriations bills in subsequent years; as of 2015, the current requirement for OMB to report on the total annual costs and benefits of federal regulations is under the Regulatory Right-to-Know Act, which was enacted in 2000 as part of the Treasury and General Government Appropriations Act for FY2001. That provision required OMB to submit to Congress each year, along with the President's budget,
An accounting statement and associated report containing- 
(1) an estimate of the total annual costs and benefits (including quantifiable and nonquantifiable effects) of Federal rules and paperwork, to the extent feasible- (A) in the aggregate; (B) by agency and agency program; and (C) by major rule; 
(2) an analysis of impacts of Federal regulation on State, local, and tribal government, small business, wages, and economic growth; and 
(3) recommendations for reform.
OMB has submitted a report to Congress each year with a total of the costs and benefits produced by federal agencies pursuant to the requirements discussed above.
section_title: Summary of 2014 OMB Report on Costs and Benefits
paragraphs: The 2014 report to Congress on the costs and benefits of federal regulations was published in June 15, 2015. The principal findings of the 2014 report were as follows: 
The estimated annual benefits of major federal regulations reviewed by OMB from October 1, 2003, to September 30, 2013, for which agencies estimated and monetized both bene fits and costs , are in the aggregate between $281.0 billion and $1,119.0 billion, while the estimated annual costs are in the aggregate between $73.6 billion and $109.3 billion. These ranges reflect uncertainty in the benefits and costs of each rule at the time that it was evaluated (prior to promulgation).
The OMB reports demonstrate that agencies do not provide quantified and/or monetized information for every rule. For example, below is information on how frequently agencies provided quantified and/or monetized estimates of costs and benefits:
During FY2013, executive agencies promulgated 54 major rules, of which 30 were "transfer" rules. Transfer rules usually implement federal budgetary programs as required or authorized by Congress, such as rules associated with the Medicare Program and the Federal Pell Grant Program, and are categorized differently by OMB because they cause income transfers from federally collected tax dollars to program beneficiaries—meaning they "may not impose significant regulatory costs on the private sector." In all but one of the 30 transfer rules listed in the report, the issuing agenc ies quantified and monetized the transfer amounts. In 7 of the remaining 24 major rules issued in FY2013, the agencies quantified and monetized both benefits and costs. Those seven rules were estimated to result in a total of $33.2 billion to $87.4 billion in annual benefits, and $2.6 billion to $3.2 billion in annual costs. In two of the major rules, the agency was able to quantify and monetize only benefits. For these two rules, the agencies estimated annual benefits of $500 million to $655 million. In 11 major rules, the agencies quantified and monetized only costs, and in one case only partially. For these 11 rules, the agencies estimated total annual costs of about $1.6 billion to $2.3 billion. In four major rules, the agencies did not quantify or monetize costs or benefits.
section_title: Analysis of Bottom-Up Approach to Aggregating Costs and Benefits
paragraphs: A bottom-up approach, such as that taken by OMB, is likely to result in an underestimate of the total cost of federal regulations. Agencies are not required to conduct a cost-benefit analysis for every rule, and the bottom-up approach can only include costs and benefits that have actually been estimated. Furthermore, estimates of costs and benefits that agencies produce are primarily intended to inform decisionmakers about an individual rule, but aggregating the information causes the context and understanding of potential uncertainties in each individual estimate to be lost. Finally, a decision must be made about how many years' worth of rules to include in the aggregated estimate of costs and benefits, but this is likely to leave out costs and benefits of rules that were issued prior to the period included.
subsections: section_title: Costs Not Estimated for Every Rule
paragraphs: One of the main challenges to calculating accurately the total costs and benefits of federal regulations is that agencies are not required to estimate costs and benefits for all regulations. Without an estimated cost or benefit for every rule, it is impossible to arrive at a total dollar amount for all rules—the aggregated costs and benefits will only include those rules for which a monetized estimate exists. 
subsections: section_title: Independent Regulatory Agencies Are Often Not Required to Estimate Costs and Benefits
paragraphs: First, not all agencies are subject to cost-benefit analysis requirements. The cost-benefit analysis requirement does not apply to independent regulatory agencies, a class of agencies that were created by Congress to have various characteristics of independence from the President. 
Although some of the independent regulatory agencies have agency-specific instructions in statute to consider certain effects of their regulations, others are not specifically required to conduct cost-benefit analysis or monetize costs and benefits. According to OMB's 2014 report on costs and benefits, the independent regulatory agencies issued 18 major rules in FY2013. Of those 18 rules, the agencies appear to have provided some information on either costs or benefits in 12 rules. 
Because most cost-benefit analysis requirements do not extend to the independent regulatory agencies, the bottom-up approach to estimating the costs and benefits of regulation does not include costs and benefits of regulations issued by those agencies. This includes, for example, many of the financial regulations issued pursuant to the Dodd-Frank Wall Street Reform and Consumer Protection Act of 2010, which are primarily issued by the financial regulators—most of which are excluded from the cost-benefit requirement of Executive Order 12866. The exclusion of certain types of agencies from cost-benefit analysis requirements is a weakness in the annual OMB reports and presents a challenge to bottom-up approaches of estimating costs and benefits. The OMB reports generally identify the number of rules issued by independent regulatory agencies that provide some information on some costs and/or benefits, but they do not typically have information about the magnitude of those costs or benefits. 
section_title: Costs and Benefit Estimates Only Required for Economically Significant Rules
paragraphs: Second, the agencies that are subject to cost-benefit analysis requirements are not required to conduct a cost-benefit analysis for every rule—only those rules that are deemed economically significant, as defined by Executive Order 12866. Specifically, the order defines economically significant rules as those that have an annual effect on the economy of at least $100 million. Rules that are significant, but not economically significant, are subject to requirements for OIRA review and may have a less formal assessment of costs and benefits, but agencies are not generally required to conduct a complete cost-benefit assessment for those rules. As such, the rules included in the OMB reports' total estimates of costs and benefits are only economically significant rules.
section_title: Monetizing Costs Can Be Challenging
paragraphs: Third, quantifying and monetizing certain costs and benefits can be very difficult, and agencies do not often monetize all of the expected effects of their regulations. The bottom-up approach, however, is limited to totaling only those costs and benefits that are actually monetized. Under current rulemaking requirements, agencies are encouraged, but not necessarily required, to monetize costs and benefits. For example, Executive Order 12866 states that agencies should provide a quantification of costs and benefits "to the extent feasible."
Monetizing the effects of regulations involves converting expected effects, such as costs to consumers or changes to a population's health or behavior, into dollar terms. Monetizing the effects of regulations involves turning costs and benefits into a common unit—dollars—so that they can be compared against one another. This can allow for an evaluation of the cost-effectiveness of a rule, such as a calculation of the cost of each life expected to be saved by a rule, or the cost-benefit ratio of a rule. 
Although some of the effects of a rule can be measured fairly easily in dollar terms, such as certain types of equipment or technologies for which a market value can be easily identified, other effects are more difficult to monetize. In cases in which dollar amounts are not readily available, agencies often rely on economic techniques that attempt to simulate market exchanges. One method agencies use to monetize certain concepts is based on a formula that includes monetized values known as "willingness-to-pay" or "willingness-to-accept" to measure the value that individuals place on the change resulting from a particular regulation. This allows an agency to assign a dollar value to a regulatory outcome that may not otherwise have an easily identifiable value, and then the estimate can be compared against the costs of obtaining that benefit. For example, agencies sometimes use the "value of a statistical life" or "VSL" to assign a monetized amount to the benefits per life saved from certain types of regulations. The agency can then compare this monetized estimate of benefits against the costs of the rule to see whether the rule's costs were justified by its benefits. 
The executive orders and OMB guidance documents recognize that quantification and monetization can be difficult in some cases and allow agencies some flexibility in determining when effects can be quantified and monetized:
Executive Order 12866 instructs agencies to provide information on the quantified costs of the anticipated costs and benefits of regulations "to the extent feasible." OMB Circular A-4, which OMB issued in 2003 to provide guidance to agencies on how to conduct cost-benefit analysis, states that agencies "should develop quantitative estimates and convert them to dollar amounts if possible. In many cases, quantified estimates are readily convertible, with a little effort, into dollar equivalents." Executive Order 13563, which President Obama issued in January 2011, stated that "Where appropriate and permitted by law, each agency may consider (and discuss qualitatively) values that are difficult or impossible to quantify, including equity, human dignity, fairness, and distributive impacts." OMB's 2011 guidance document, "Regulatory Impact Analysis: A Primer," instructs agencies to quantify the costs and benefits in terms of units—for example, the number of premature deaths avoided each year or the number of prevented nonfatal illnesses—as well as monetizing the costs and benefits associated with each of these effects, to the extent possible. 
As OMB explained in its 2014 report, 
When agencies have not quantified or monetized the primary benefits or costs of regulations, it is generally because of conceptual and empirical challenges, including an absence of relevant information. Many rules have benefits or costs that cannot be quantified or monetized with existing information, and the aggregate estimates presented here do not capture those non-monetized benefits and costs. In some cases, quantification of various effects is highly speculative. For example, it may not be possible to quantify the benefits of certain disclosure requirements, even if those benefits are likely to be large, simply because the impact of some of these requirements cannot be specified in advance.
Practitioners or observers of the rulemaking process do not necessarily agree on what effects are appropriate to monetize. For example, the Obama Administration has placed greater emphasis than earlier Administrations on the value of qualitative benefits such as equity and dignity. Debates in the literature have raised questions over when it may be appropriate to quantify or monetize certain qualitative effects of rules. For example, questions have been raised as to whether dignity and other psychological effects of rules, such as fear and anxiety, should be monetized in regulatory impact analyses.
In sum, to achieve an accurate assessment of the total costs and benefits of regulations in dollar terms using a bottom-up method, one would have to monetize all possible effects of regulation, and such an undertaking is not consistent with current rulemaking requirements or practice. OMB did state in its 2014 report, however, that OMB believes "the benefits and costs of major rules, which have the largest economic effects, account for the majority of the total benefits and costs of all rules subject to OMB review." 
Given that agencies do not estimate cost and benefits for every rule issued in every year, estimates that use the bottom-up approach are likely under-reporting costs and benefits. On the other hand, as discussed more below, some of the literature evaluating the quality and accuracy of cost-benefit analyses has suggested that costs and benefits for rules evaluated are overestimated by agencies. 
section_title: Potential Uncertainties in Individual Cost-Benefit Analyses
paragraphs: Cost-benefit estimates are produced by agencies to assist with decisionmaking in individual rules, and the estimates often contain some uncertainty from not knowing precisely the potential effects of rules. For example, consider a rule in which an agency conducts a risk assessment, such as an airline safety rule intended to lower the risk of a terrorist attack on the United States. The estimate of costs and benefits for this rule would contain a great deal of underlying uncertainty, because precisely estimating the projected decrease in the risk of a terrorist attack and the precise cost of the rule is likely impossible. On the other hand, a rule that would result in changes that can be more easily predicted and measured, such as a rule requiring the purchase of new safety equipment for a vehicle fleet, is likely to contain less uncertainty. 
To attempt to reduce uncertainty and produce high-quality, robust estimates, current requirements and guidance documents encourage agencies to base their estimates on the best reasonable obtainable scientific, technical, and economic information. With regard to uncertainties, agencies are instructed to analyze and present them as part of the overall regulatory analysis. This generally results in the agencies producing a range of estimates of costs and benefits and using certain types of analytical techniques (e.g., sensitivity analysis) to identify how benefits and costs of a rule would change due to changes in key variables.
As stated in a 1996 article outlining the challenges of uncertainty in cost-benefit analysis (also referred to as benefit-cost analysis), 
Benefit-cost analysis can help decision-makers better understand the implications of decisions by identifying and, where appropriate, quantifying the favorable and unfavorable consequences of a proposed policy change, even when information on benefits and costs, is highly uncertain. In some cases, however, benefit-cost analysis cannot be used to conclude that the economic benefits of a decision will exceed or fall short of its costs, because there is simply too much uncertainty.
In adding the results of all the individual studies together, however, the context and uncertainties of each individual study may be lost. As OMB stated in its first report on costs and benefits, 
Studies that have attempted to total up the total costs and benefits of Federal regulations have basically added together a diverse set of individual studies. Unfortunately, these individual studies vary in quality, methodology, and type of regulatory costs included. Thus we have an apples and oranges problem, or, more aptly, an apples, oranges, kiwis, grapefruit, etc., problem.
Adding together existing estimates of costs and benefits treats the individual estimates as though they are precise estimates, but they may be imprecise estimates. "CBA can sometimes produce an illusion of certainty," as one former OIRA Administrator stated. This illusion of certainty can be misleading in individual cost and benefits estimates, and it becomes even more so when such estimates are aggregated. In any individual estimate of costs and benefits, "numerous technical judgments must be made, and technical analysts might well disagree." Aggregating all of the bottom-up estimates that agencies have produced can cause the context and any important caveats or reflections of uncertainty in each regulatory impact analysis to be lost. 
section_title: Questions Over Accuracy of Individual Cost and Benefit Estimates
paragraphs: Uncertainty in how best to estimate costs and benefits leads to the question of whether the results are accurate —i.e., whether agencies overestimate or underestimate costs and benefits. Proponents of regulation tend to argue that agencies overestimate costs of regulation. On the other hand, opponents of regulations tend to argue that agencies underestimate costs—agencies may have incentives to underestimate the costs of regulations and overestimate the benefits, which would help to make the case for promulgating a regulation because of its positive net benefits. To address this important question, a number of studies have examined the accuracy of agency estimates of costs and benefits. One academic study published in 2000 compared ex ante studies to e x post studies and found that agencies frequently overestimated both costs and benefits of regulations. In 2010, the same authors performed a similar ex post study of several cost-benefit estimates that had been produced by agencies subsequent to their 2000 study. Again, they concluded that regulatory agencies tended to overestimate the total costs of regulations, explaining that "a variety of factors contribute to initial government agency cost estimates that may differ from the realized results, although in some cases this is coincident with differences in benefits produced by regulations."
In its 2005 report to Congress on the costs and benefits of regulations, OMB included a chapter on "validation" of cost estimates. OMB examined a number of ex ante cost-benefit estimates and compared them with ex post estimates, when they were available. OMB's conclusions were that the costs of regulations were more often overestimated by the agency, but that the benefits were sometimes overestimated as well. 
A primary reason observers have given for the overestimation of costs is that agencies tend to underestimate industry's ability to innovate, and therefore compliance with regulations sometimes turns out to be less costly than expected. The ability to adapt to regulatory requirements and identify more cost-effective methods of meeting compliance targets can result in lower compliance costs than initially anticipated by the agency. For example, consider the Environmental Protection Agency's acid rain (sulfur dioxide) program in the 1990s. Mandated under the Clean Air Act Amendments of 1990, the EPA issued regulations aimed at reducing sulfur dioxide emissions. Studies suggest that the EPA's initial cost estimates for these regulations were too high due to EPA's underestimation of how industry would adapt and find less costly means of achieving the reduction targets. In sum, as stated in one retrospective study, "When forecasting the costs of new environmental regulations, economic analysts have routinely ignored a primary economic lesson: Markets will cut costs through innovation." 
A number of other reasons may contribute to inaccuracies in agency estimates of costs. For example, delays in implementation of regulations can help lower the compliance costs as it can allow industry more time to identify cost-effective solutions. Over- or under-estimating certain effects of regulations can also result in inaccuracies, such as compliance rates among regulated entities. Finally, agencies first estimate costs and benefits while writing the proposed regulation, but the regulation may change in response to comments received during the public comment period. As regulations are revised in response to comments, cost-benefit estimates are not always updated. Changes made in the rule after it has been proposed and before it is finalized could affect the likely costs and benefits of the rule—for example, if the final rule sets a standard that is less stringent as compared to the proposed rule, it will probably be less costly to comply.
In short, the accuracy of the bottom-up approach relies heavily on the precision of the individual cost and benefit estimates, but the accuracy of these estimates is likely to vary. 
section_title: Measuring Costs over Time
paragraphs: Another issue in aggregating the costs and benefits of individual regulations is identification of the appropriate time period. In other words, how many years' worth of regulations should be included in the total? Regulations issued many years ago typically still have some compliance costs, although the cost of complying with regulations is generally thought to decrease over time. For example, if compliance with a new regulation requires industry to invest in new types of technology, once these investments have been made, the majority of the costs may already have been incurred, and the ongoing costs will be less. The time frame under which these compliance costs are distributed varies among rules. Similarly, the distribution of benefits can vary widely as well, and benefits of regulations often are not realized until the regulation has been in place for some time. For example, this is typically the case with regulations that have health benefits, such as environmental regulations that are intended to improve air quality. 
Individual cost estimates of regulations are calculated by comparing the anticipated effects of a regulation against what would be expected to happen in a world without the regulation. These ex ante estimates that are conducted prior to the issuance of the regulation are usually the sole source of information on regulations' costs. Under current requirements and practice, an ongoing monitoring of the costs and benefits of rules is not required once a regulation has been issued. 
The annual OMB reports have typically included 10 years' worth of cost and benefit estimates. According to the 2014 report, "OMB chose a ten-year period for aggregation because pre-regulation estimates prepared for rules adopted more than ten years ago are of questionable relevance today." Similar reasons are cited in earlier versions of the OMB reports. 
In its 2014 report, OMB acknowledges this weakness in its approach, stating that because the estimates do not include non-major rules or rules issued more than 10 years ago, the total costs and benefits currently in effect are like to be "significantly larger" than the totals in the report. Over time, however, measuring the effects of those rules issued more than 10 years ago will become increasingly difficult. As stated in a 2006 study that discussed OMB's practice of including 10 years' worth of costs and benefits in its annual reports, 
If the regulation only requires small changes in behavior over a small time interval, then analysts feel confident of estimating the cost, hypothetical or not. But as the size or time interval increases, the shadow of the hypothetical looms ever larger. To take an extreme example, how would be begin to estimate the cost of federal child labor laws that were enacted during the 1930s?
It is certainly the case that some rules that are more than 10 years old still impose a cost, and therefore this 10-year rule may result in a lower estimate of total costs over time because it excludes those rules. However, some cutoff probably should be made, as including estimates for costs of regulations made more than 10 years ago becomes less practicable.
These potential flaws have led some to attempt to use an entirely different approach to estimating the effects of regulations: the top-down method, which is discussed next.
section_title: "Top-Down" Method: Estimating Costs Using Macroeconomic Modeling
paragraphs: Some scholars have adopted an approach to estimating the costs of rules that is referred to as a top-down approach. This section explains and analyzes the top-down approach, using a case study as an illustration for several of the concepts discussed. 
subsections: section_title: What Is the Top-Down Approach?
paragraphs: The top-down approach uses macroeconomic variables and modeling techniques to measure the effect of regulation on the economy as a whole. Rather than aggregating existing cost estimates, the top-down method uses the results of an economic model that has been used to measure the relationship between the size of an economy, or economic growth, and some proxy of the level of regulation in a country, to measure the economic effects of regulation. This method typically compares the U.S. economy, as measured by some variable such as gross domestic product (GDP), to a hypothetical scenario in which the U.S. has less regulation. The approach takes the difference in the GDP under these two scenarios—the status quo and the scenario with less regulation—and calculates the change in GDP that might occur were the U.S. to reduce its overall amount of regulation. This potential change in GDP is considered to be the cost of regulation under the top-down approach and is explained in further detail below.
section_title: Why Use the Top-Down Approach?
paragraphs: The benefits of using a top-down approach, in many ways, are opposite from the bottom-up approach described above. Because the top-down approach uses measurements of various economic factors, it incorporates broader, more indirect effects that are not included in the bottom-up approach. This could include indirect economic effects, as well as direct effects that are not monetized. 
section_title: Case Study: Crain and Crain Report
paragraphs: A commonly cited study using the top-down approach to totaling the cost of regulation is entitled "The Cost of Federal Regulation to the U.S. Economy, Manufacturing and Small Business," by W. Mark Crain and Nicole V. Crain, and it was most recently released in September 2014.
The Crain and Crain report estimates the total cost of regulation and also explores the distribution of those regulatory costs among regulated entities. The component of the report that is most discussed in the debate over regulatory reform is the estimate of the total cost of regulation. Hence, this CRS report focuses primarily on the components of the studies that describe the total cost of regulation, and not on the portion of the report exploring the distribution of costs among different types of firms. 
The Crain and Crain estimate of the total cost of regulation is not a purely top-down measure of the cost of regulations. It combines a top-down estimate—their estimate of the cost of economic regulation—with a bottom-up estimate of environmental, tax compliance, occupational safety, and homeland security regulation. However, the estimate of the cost of economic regulation resulting from their top-down methodology is almost 75% of the total estimate, totaling $1.439 trillion out of $2.028 trillion in the 2014 study. This report only discusses that portion of their estimate, as it is the most widely cited top-down study in the regulatory reform debate, and therefore serves as a useful example for discussion.
subsections: section_title: Background on the Crain and Crain Report
paragraphs: The 2014 Crain and Crain report can be traced back to a report originally published in 1995 by Thomas D. Hopkins for the Small Business Administration's (SBA's) Office of Advocacy entitled "Profiles of Regulatory Costs." Each of the studies from 1995 through 2010 was prepared for the Office of Advocacy. 
In the 1995 study, Hopkins estimated annual federal regulatory costs to be in the range of $416 billion to $668 billion in 1995. Six years later, in 2001, W. Mark Crain and Hopkins issued a follow-up to the 1995 Hopkins study, estimating the total annual cost of regulations to be $843 billion in 2000. In 2005, Crain estimated annual regulatory costs to be about $1.1 trillion in 2004. In 2010, Nicole V. Crain and W. Mark Crain issued another version of the study, which estimated the total cost of regulation to be $1.75 trillion in 2008. They explained the increase from $1.1 trillion in the 2005 report to $1.75 trillion as being, in part, the result of "new methodological techniques," meaning that "direct comparisons to the results in their prior studies should be made with caution."
A number of concerns were raised about the methodology used in the 2010 report—and primarily the methods used to arrive at the $1.75 trillion of the total cost of regulation. Some of the entities raising concerns included the Office of Advocacy itself, the entity that had granted the contract under which the study was conducted. For example, the Office of Advocacy posted a number of caveats on its website where the report was linked, stating that "the findings of the study have been taken out of context and certain theoretical estimates of costs have been presented publicly as verifiable facts." A 2011 CRS report called into question the methods used in the report to arrive at the total estimate of the cost of federal regulation. A 2014 Government Accountability Office (GAO) report also raised issues with the study related to federal data quality standards, asserting that the Office of Advocacy failed to uphold those standards. 
section_title: Summary of the 2014 Crain and Crain Report
paragraphs: The 2014 Crain and Crain study estimated the total cost of federal regulation to be $2.028 trillion in 2012 (in 2014 dollars), an amount equal to 12% of U.S. GDP. This section summarizes the methodology used to arrive at this estimate, so that the estimate may be used as an illustration throughout the rest of the discussion of the top-down methodology. 
As mentioned above, economic regulation is the majority of the total cost of the Crain and Crain estimate: $1.448 trillion. According to Crain and Crain, economic regulations are those that "govern decision-making in market transactions. These include markets for final goods and services; markets for physical and human resources; credit markets; and markets for the transport and delivery of products and factors of production." 
To arrive at this estimate of the total cost of economic regulation, Crain and Crain conducted a cross-country comparison of 34 Organization for Economic Cooperation and Development (OECD) countries, including the United States. Specifically, they looked at the relationship between each country's economy and a proxy measure of the amount of regulation in each country over eight years. The data Crain and Crain used for this proxy measure were derived from the World Economic Forum's Global Competitiveness Report. 
Crain and Crain used three components of the Global Competitiveness Index, a component of the Global Competitiveness Report that measures various aspects of the institutions, policies, and factors that determine a country's level of productivity. These types of measures are sometimes referred to as governance indicators. The report uses an "Executive Opinion Survey," which captures the opinions of business leaders around the world to construct a number of its indicators. Crain and Crain selected three indicators, each of which was constructed using the survey, to represent the amount of economic regulation in each country. The three indicators were
burden of government regulation; efficiency of legal framework in challenging regulation; and regulation of securities exchanges. 
The survey respondents provided a value for each of these indicators ranging from one to seven. As Crain and Crain explained in their report, "higher values correspond to improvements in regulatory quality—that is, reductions in the regulatory burden on product, factor and credit markets." To construct their measure for each country and each year, Crain and Crain used the mean value of these three factors. The regulation index that Crain and Crain used combined these three factors into a single measure for each of the countries in each of the years for which the data were available—2006 to 2013. 
Using the mean value for the 34 countries over their eight-year period, Crain and Crain estimated a regression model in which they measured the effect of a number of variables—with their primary variable of interest being the measure of regulatory quality—on GDP per capita. The other variables, or "control" variables, were foreign trade as a share of GDP, population over 65 relative to population aged 19 to 65 (the "dependency ratio"), new capital investment as a share of GDP, size of the labor force, and tax revenues as a share of GDP. 
Based on the results of the regression model, Crain and Crain concluded that the effect of their purported proxy of economic regulation on GDP per capita was statistically significant. Further, they used the results of the regression model to estimate the total cost of the index on the GDP per capita in the United States. To do so, they compared the U.S. score on the regulation index with the average score of the five top performing (i.e., highest ranked) countries on the scale, which they referred to as the "benchmark countries." They used this "benchmark" measure as a hypothetical measurement of what a lower level of regulation in the United States could be. The difference between the U.S. score and the benchmark countries' average score was 26%. 
Crain and Crain concluded that this 26% difference in the regulation index in the United States "implies an impact on GDP equal to $1.439 trillion. In other words, if the burden of economic regulation in the United States matched the benchmark countries, U.S. GDP would be $1.439 trillion higher than it was in 2012." To the $1.439 trillion estimate, Crain and Crain added another $8.3 billion, which was the estimated cost of import restrictions from the U.S. International Trade Commission. The total estimate of the cost of economic regulation they arrived at using this methodology was $1.448 trillion. 
section_title: Analysis of Top-Down Approach to Estimating Costs and Benefits
paragraphs: A number of issues have been raised with the top-down approach, some of which are explained below using the Crain and Crain study as an illustration.
subsections: section_title: Importance of Accurate Measures of Regulation in Top-Down Approach
paragraphs: One challenge for the top-down approach to estimating the cost of regulation is that the accuracy of the findings is dependent on the validity of the proxy measure of regulation. The proxy is used to model the relationship between the size of the economy and amount of regulation, and then the parameter estimates resulting from the model are used to calculate the total cost of economic regulation. 
Identifying an accurate measure of regulation, however, is a challenge. As explained in a report by the Council on Foreign Relations (CFR) on federal rulemaking policy in the United States and other countries, as well as in previous CRS reports, quantifying the total amount of regulation is an inherently difficult task. As stated in the CFR report, 
Economists have not settled on a good way to measure overall regulatory burden … Because of these data limitations, the best empirical studies take on a specific regulation rather than the full stock of regulations. Largely unknown is how the average business is affected by the cumulative set of regulations, or whether certain regulations harm or help different kinds of business activity, such as innovation or entrepreneurship.
Obtaining an accurate proxy measure of regulation is key in any economic model that uses regulation as an explanatory variable, because an inaccurate measure can introduce serious uncertainty into the model's results. The term economists use to refer to whether a measure of a construct correctly represents what it purports to represent is "content validity." To illustrate the importance of content validity in the top-down approach of measuring the cost of regulation, this section will more closely examine the measure of "regulatory quality" used by Crain and Crain.
Crain and Crain referred to their measure of regulatory quality as the "Economic Regulation Index." They created the index from data from the World Economic Forum's (WEF's) annual Global Competitiveness Report—specifically, the report's Executive Opinion Survey. In that survey, the WEF captured the opinions of over 13,000 business leaders in 148 different economies during a five-month period. Most of the survey questions, including the three used in the Crain and Crain study to measure regulation, involved rating on a scale of one to seven a particular aspect of the operating environment in the respondent's country. The questions covered such topics as innovation and technology infrastructure, education and human capital, and tourism. To create a measure of regulation from this survey, Crain and Crain selected three questions, each of which included an explicit reference to regulation (which appears to be how they selected them): (1) burden of government regulation, (2) efficiency of legal framework in challenging regulation, and (3) regulation of securities exchanges. 
Are these three questions from the Executive Opinion Survey measuring this type of specific regulation—in other words, can the responses to these three questions be considered a useful proxy of the burden of economic regulation? This may be difficult to answer, but it is crucial to the validity of the study's conclusions. Crain and Crain state that "the reach of economic regulations is vast. This means that an encompassing methodology is required to derive an estimate of these costs." While their statement about the vastness of economic regulation is certainly true, whether their measurement and methodology measures it accurately is difficult to validate. 
Proxy measures of governance indicators are inherently imprecise, and they cannot be reliably used to estimate macroeconomic outcomes. A recent article examining a similar cross-country set of indicators illustrates the difficulty in identifying an accurate measure for certain inherently abstract concepts related to governance, such as the "rule of law," or, in this case, regulation. The article specifically focused on the World Bank's "Worldwide Governance Indicators" (WGI), though it argued that "the concerns raised here about the WGI apply equally to other current governance indicators." The article stated that "both researchers and policymakers should require evidence that governance indicators are valid before employing them. In the absence of such evidence, research results obtained using such indicators are uninterpretable and should not survive peer review. For policymakers, reliance on such indicators would be arbitrary." It could be argued that the use of such indicators in the top-down model of total cost of regulation may suffer from this problem of questionable measurement.
Specific concerns over this issue as it relates to top-down studies involving regulation were raised in 2010 over the Crain and Crain report and arguably still apply to the 2014 report as well, due to the similarities of their measures. For example, economist Art Kraay, one of the creators of the World Bank's Regulatory Quality Index, the measure of regulation that Crain and Crain used in their 2010 study, commented in response to their study that the measure of regulatory quality they created measured the perceptions of various regulatory environments, rather than the stringency of those environments. The index used in the 2014 study uses a different proxy, although its construction was similar—it measures business leaders' perceptions of the regulatory environment in various countries. Some research suggests that there may be key differences between perceptions of something and actual levels of it, however:
There is a substantial difference between measuring a thing and measuring perceptions of it. In the context of governance, for example, perceptions of crime risk have been shown to be quite different than actual crime levels (see, for example, Forgas, 1980; Pfeiffer, 2005); perceptions of corruption have been shown to differ from actual corruption levels (see, for example, Olken, 2006; Seligson, 2006); and trust in government has been shown to differ from administrative performance (Van de Walle and Bouckaert, 2007).
In sum, this question of whether a measurement based on survey responses of business leaders or other individuals, such as the Executive Opinion Survey, is measuring what it purports to measure, is an important one. With any top-down model of the economic effects of regulation, the validity of a proxy for regulation is essential—and measuring an inherently abstract concept like the stringency of regulation in a country is difficult. Without a valid proxy, which is difficult to identify for the reasons discussed above (and possibly others), the findings of any top-down study could be brought into question.
section_title: Questions About Proper Model Specification
paragraphs: Setting aside the question of how to undertake the difficult task of measuring regulation, and assuming that the measure of regulation is a valid measurement, another crucial question remains for the top-down approach to measuring the cost of regulation: how should one select the right model? 
In a regression model, such as the one used by Crain and Crain, a researcher is attempting to explain a relationship between variables by analyzing the extent to which the dependent variable can be explained by changes in the independent variables. Proper selection of all variables in the model is important, as discussed below.
subsections: section_title: Selecting Independent Variables
paragraphs: Selecting and including the proper independent and control variables in an econometric analysis is crucial to the validity of the model's results. For example, a researcher must include all variables that are thought to be theoretically relevant and take care not to omit variables that may help explain the outcome. Explaining and theoretically justifying the components of economic models, including all independent variables, is important: "It is critical that researchers explain and justify the assumptions that underlie their model, which are presumed to be informed by theory." 
Failure to include all relevant variables in an economic model can result in omitted variable bias. Omitted variable bias can occur when one of the explanatory variables—those variables in the model that are helping to explain changes in the dependent variable—has its effects overstated because a variable that is related to that variable has been left out of the model. In other words, the omitted variable's explanatory power will be attributed to the variable that is included in the model, as long as the two variables are correlated. As explained in one study,
Because top-down methods [of estimating the cost of regulation] associate indicators of regulatory activity with changes in macroeconomic variables, they risk attributing to regulation the effects of other variables that are not considered in the analysis but that may be correlated with regulatory activity. There is a strong chance of omitted variable bias, in other words.
In sum, were a top-down model of regulation to leave out some relevant variables, the effect of regulation could be overstated. 
section_title: Selecting a Dependent Variable
paragraphs: Furthermore, identifying a theoretically sound dependent variable is crucial to the validity of a model's results. Crain and Crain used GDP per capita as their dependent variable, rather than GDP growth rate. Measuring GDP per capita does provide an indication of the size of a country's economy. However, it can potentially be problematic because it ignores the historical circumstances that have led each country's economy to its current size, and therefore it presumes to explain the current size of each economy only based upon the factors in the model. Using the rate of GDP growth instead of GDP per capita would presume that the changes in the size of each country's economy are explained by the independent variables, not the actual size of the economy itself—which could be considered more likely.
section_title: Selecting the Correct Form
paragraphs: Another important issue related to model specification is whether the nature of the relationship between the dependent variable and the independent variables is properly specified: in other words, is the relationship linear or nonlinear? A linear relationship in expressed on a graph with a straight line and assumes that the rate of change in the dependent variable does not vary—hence the straight line on a graph. A non-linear relationship can also be expressed on a graph but does not have a straight line, and it assumes that the rate of change in the dependent variable can change. These concepts are discussed more below in the context of the top-down approach. Notably, however, identifying the form of the relationship between dependent and independent variables is important—"the consequences of an incorrect functional form for interpretation and forecasting can be severe."
Crain and Crain ran a linear regression, which is based upon the assumption that the relationship is of direct proportionality and is expressed on a graph with a straight line. Whether this is the type of relationship between regulation and size of a country's economy, or its economic growth generally, is an important question. Crain and Crain did not appear to provide a theoretical basis for assuming the relationship is linear, and some scholars have suggested that the relationship between regulation and economic growth is nonlinear. A 2012 working paper from the George Washington University Regulatory Studies Center suggested that "perhaps... up to a point, increasing the size of government may tend to increase GDP, but that the relationship reverses after a certain threshold." That paper examined whether a measure of the on-budget costs of federal regulation in the United States had an effect on GDP growth, which is a different objective than a top-down model of the cost of regulation, but it illustrates the important point that some effects on GDP can be nonlinear. If such model misspecification did occur in the Crain and Crain model, the results of their model could be held in question.
In sum, these three issues related to model specification—selection of independent variables, selection of dependent variables, and selection of the correct functional form for the model—are highly important to the reliability and validity of conclusions made based upon a top-down estimate of the cost of regulation.
section_title: Other Potential Methodological Issues
paragraphs: There are several additional methodological issues that may be of concern in the top-down approach, including unclear directions of causality and insufficient sample size in the model. 
subsections: section_title: Questions of Causality
paragraphs: The first issue, which is linked to the question of identification of a dependent variable, relates to questions of causality in a top-down approach that uses a measure of regulation to explain the health of a country's economy: does the amount of regulation affect the economy, or could the economy also have an effect on the amount of regulation? This uncertainty about the direction of the causality between different components of a model is referred to by economists as "endogeneity." The presence of endogeneity can cause bias in the parameter estimates resulting from the model. As mentioned earlier in this report, it is crucial that researchers using a top-down approach explain the theory behind all of the components of their models. Having a solid theoretical foundation related to the causality question can reduce or eliminate the likelihood that a model will suffer from endogeneity.
In the case of top-down studies of regulation like Crain and Crain, the parameter estimates are used to calculate the potential difference in GDP, which is the estimate that Crain and Crain provide for the cost of economic regulation. As explained in the section above summarizing their methodology, Crain and Crain assert that the GDP per capita is a direction function of the amount of regulation in a country. They do not explore the possibility that the causality could also go in the other direction. The presence of such endogeneity can cause the results of the model to be incorrect.
The issue of correctly identifying the causal relationship is a challenge for scholars examining the relationship between regulation and other macroeconomic factors. For example, a recent study of the "regulatory volume" in states across the United States "looked at the relationship between regulatory output and a series of indicators described above meant to represent the economic health of the state." The study concluded that "citizens in more prosperous states may very well demand more regulation than citizens in less prosperous states." A similar concern could potentially be raised about the direction of causality between the volume of regulation and the economy. 
section_title: Sample Size
paragraphs: The second is an issue of sample size, particularly for a top-down study that uses a cross-country comparison of industrialized economies. A small sample size can have detrimental effects on the precision of the model's results: a smaller sample size makes the estimate less precise and therefore less reliable. Researchers who examine cross-country comparisons of regulation face some potential problems, including the small number of fully industrialized countries and the fairly short length of time over which regulatory measures have been constructed.
The Crain and Crain study uses 34 countries and a time period of eight years: 2006 to 2013. This is a relatively small sample size, especially given that the measures of regulation and other variables in the model are unlikely to change very much from one year to the next in just an eight-year period. In addition, this specific eight-year period is not necessarily a typical one, as it is the period leading up to and immediately following the Great Recession. Therefore, conclusions made based upon this narrow time period may not be generalizable to other years or other periods. 
section_title: Potential for Double-Counting Costs
paragraphs: In the top-down approach, another potential problem is whether costs may be double-counted. Crain and Crain separate their total estimate of the cost of regulations into four categories, as described above, although it appears that this approach may be double-counting the cost of some regulations for at least two reasons. 
The first reason is an econometric one: as mentioned above, the potential for omitted variable bias exists in any econometric model. If Crain and Crain omitted a variable that is highly correlated with their measure of "economic" regulations—such as a measure of environmental or other types of regulations, which are almost undoubtedly correlated with their measure of economic regulation—then the explanatory power of the omitted variable becomes attributed to the coefficient on the economic regulation measure. In other words, the explanatory strength of that measure is inflated by the omission of other variables that are not included. By adding other measures of regulation, including environmental regulations, to the total cost they estimate for economic regulation, Crain and Crain may therefore be double-counting the effects of these other types of regulations. 
Second, the potential for double-counting exists because of the nature of the questions asked to business leaders that comprise the regulatory quality measure. The questions themselves do not necessarily measure only economic regulation, although Crain and Crain imply that to be the case. Rather, as discussed above, they measure potentially very broad effects of all types of regulation by measuring the overall regulatory environment. Because Crain and Crain is a hybrid study that also employs a bottom-up methodology to measure certain types of regulation, they add an estimate of the cost of other types of regulation to their estimate of the cost of economic regulation. As a result, their overall number of roughly $2 trillion could include some double counting. 
section_title: No Discussion of Benefits of Regulations
paragraphs: To date, it appears that the top-down approach has not been used to estimate benefits of regulation—only costs. It is not clear whether the method could be used to measure benefits, because the approach measures cost in terms of potential economic growth that has not occurred due to the amount of regulation. A parallel approach does not seem to be applicable to benefits, which, as discussed above, are often not easily measured in dollar amounts or economic effects, and therefore may not be able to be estimated in this same way. 
Although the top-down method does not appear to be intended to measure costs and benefits, having an estimate of costs without an estimate of benefits does not provide the complete context for evaluating whether a country's amount of regulation is appropriate. Such a comparison of costs against benefits has been institutionalized in the regulatory process in the United States since the early 1980s. Specifically, one of the underlying components of the current regulatory system, which was formally put into place by President Ronald Reagan in 1981 when he issued Executive Order 12291, is that agencies should consider the costs of individual regulations and compare them against the benefits, and that "regulatory action shall not be undertaken unless the potential benefits to society for the regulation outweigh the potential costs to society." This general approach of comparing costs against benefits has been upheld since the Reagan Administration formalized it in 1981, and the shift to become more reliant on estimates of costs and benefits in informing regulatory decisions over the past several decades has been well documented. 
In sum, the notion that costs and benefits must be compared to one another has remained a crucial component of the regulatory process in the United States over the past several decades. Under current rulemaking requirements, agencies are responsible for measuring costs against benefits in individual regulations, and because of the nature of the bottom-up method and its reliance on those estimates, it can be used to make overall comparisons between total costs and benefits. Such a comparison does not seem possible, however, at the macroeconomic level.
section_title: Summary: Advantages and Disadvantages of Two Approaches
paragraphs: Each of the two main approaches taken to estimate the total costs and benefits of regulations have pros and cons, which tend to mirror one another. This final section briefly compares the two approaches and provides some perspective on why this issue is of potential interest to Congress.
The bottom-up approach to estimating the total costs and benefits of regulations, such as the approach taken by OMB in its annual report to Congress, has several advantages. The bottom-up approach involves adding up actual cost estimates calculated by agencies pursuant to rulemaking requirements. These estimates are conducted on an individual basis for certain regulations, and, although they often contain some uncertainty, are based upon specialized information the agency has regarding expected costs and benefits. 
Notably, current requirements for agencies to estimate costs and benefits of their rules are intended to assist with agency decisionmaking in individual regulations by more fully informing decisionmakers of the likely outcomes and providing a means of evaluating and comparing regulatory alternatives. The potential utility of that information when taken out of the context of individual rules and aggregated, however, is greatly reduced.
An individual cost-benefit analysis conducted by an agency and included in the bottom-up aggregated estimate could also suffer from econometric problems similar to those described above in the context of the top-down approach. However, the individual estimates used in the bottom-up approach can be validated by comparing the ex ante estimates of costs and benefits to ex post estimates. Some such studies exist and are discussed above. In addition, agencies may—though they are not explicitly required to—revisit the original cost-benefit estimate when conducting a retrospective review of their regulations. This provides a potential means of validating the results of the bottom-up method, whereas the top-down method does not have a similar opportunity for ex post validation. 
The biggest potential problem with the bottom-up approach, however, is that the aggregated estimate is unlikely to represent the costs and benefits of all rules. Not all rules are included in the aggregate, because cost-benefit estimates are not currently conducted or required for all regulations. Furthermore, even for rules in which a cost-benefit analysis is required, monetizing certain types of costs and benefits can be challenging, and any effects of regulations that are not monetized are not able to be included in a bottom-up aggregate. 
Because of these limitations about what may be missing from the bottom-up estimate, a top-down approach may be more likely to capture fully the overall cost of regulations, as the top-down approach could conceivably provide a way to include in its estimate of indirect costs and effects of regulations that are not included in the bottom-up approach. However, the top-down approach has several potential problems when it comes to implementation, most of which are conceptual and methodological. Any estimates of the cost of regulation resulting from a top-down approach are entirely reliant on the validity of the model, identification of its components and structure, and the theory behind it. The validity of an estimate is especially reliant upon the validity of the proxy measure of regulation. In practice, overcoming these conceptual and methodological hurdles is difficult, if not impossible, meaning that the results of a top-down approach should be treated with a great deal of caution. 
Finally, the top-down approach does not make a comparison of costs to benefits. Such a comparison appears to be outside the purpose of the top-down approach, but having both estimates of costs and benefits can provide a more complete representation of the cost-effectiveness of a country's regulation.
subsections: section_title: Issues for Congress
paragraphs: Because of the role Congress plays in delegating legislative authority to federal agencies to issue regulations, Congress has shown an interest in conducting oversight of those regulations, both for individual regulations and the regulatory system generally. One way for Congress to conduct oversight of the regulatory system as a whole is to monitor the total cost and benefits of federal regulation. 
Inaccuracies in cost-benefit estimates conducted by agencies could have the effect of undermining public confidence in the regulatory process. So, too, could a misunderstanding or over-reliance on estimates of the total cost of regulation that are not intended to be considered precise findings. For the reasons discussed throughout this report, both approaches to estimating the total cost of regulation have inherent—and potentially insurmountable—flaws. The true cost of regulation is incredibly difficult to estimate for the many reasons discussed in this report, and perhaps others as well. The discrepancy between the two approaches and their associated estimates raises the question of the utility of using such figures in the regulatory reform debate.