Reexamining Regulations: Agencies Often Made Regulatory Changes, but Could Strengthen Linkages to Performance Goals
Released: 2014-05-12T00:00:00

Letter

Background

Retrospective Regulatory Analysis

Regulations are the means by which agencies establish legally binding requirements, codified in the Code of Federal Regulations (CFR). Typically, regulations require a desired action or prohibit certain actions by regulated parties. The regulatory process is governed by statutes, executive orders, and agencies’ policies and procedures that require agencies to evaluate the need for regulations, assess the potential effects of new regulations, and obtain public input (with certain exceptions) during their development. In addition, under various congressional and presidential directives, agencies are expected to systematically review regulations to ensure they do not become outdated or overly burdensome, among other things. Agencies also may choose to review regulations at their own discretion, in response to feedback from regulated parties and agency staff or changes in technology and market conditions, among other inputs. Therefore, the normal rulemaking process often involves agencies reviewing their existing regulations. Over many years, we have reported on agencies’ activities related to retrospective regulatory analysis and have demonstrated its usefulness to Congress, agencies, and the public while recognizing the potential difficulties, such as data limitations and overlapping or duplicative requirements.

In 2011 and 2012, the administration issued new directives to agencies on how they should plan and assess analyses of existing regulations, among other subjects.

Executive Order 13563, “Improving Regulation and Regulatory Review,” and related memorandums, directed each executive agency to develop and submit to OIRA a preliminary plan consistent with law and its resources and regulatory priorities under which the agency will periodically review its existing significant regulations to determine whether they should be modified, streamlined, expanded, or repealed, so as to make the agency’s regulatory program more effective or less burdensome in achieving the regulatory objectives.

Executive Order 13579, “Regulation and Independent Regulatory Agencies,” encouraged independent regulatory agencies to develop and release to the public plans for retrospective analyses of existing significant rules.

Executive Order 13610, “Identifying and Reducing Regulatory Burdens,” contained additional requirements and guidance regarding public participation in retrospective reviews, setting priorities in implementing retrospective review plans (for example, focusing on initiatives that would reduce regulatory burdens), and agencies’ reporting on the status of their retrospective review efforts.

OMB periodically issues guidance to agencies on implementing executive orders. Under Executive Order 12866, OMB’s OIRA is responsible for providing meaningful guidance and oversight so that each agency’s regulations are consistent with applicable law, the President’s priorities, and the principles set forth in executive orders. For example, on October 26, 2011, the Administrator of OIRA issued a memorandum on the implementation of the retrospective review plans called for by Executive Order 13563. The memorandum provided more detailed guidance to agencies on how to address elements of the order on prioritizing review plans, seeking public comment, and reporting on the status of retrospective review efforts (including a suggested template for agencies to use for their reports). According to Executive Order 12866, OIRA is to be the repository of expertise concerning regulatory issues.

As we reported in 2007, agencies generally assess their existing regulations for a variety of purposes, including to determine whether (1) the expected outcomes of the regulation have been achieved, (2) the agency should retain, amend, or rescind the regulation, or (3) the actual benefits and costs of the implemented regulation correspond with estimates prepared at the time the regulation was issued. However, OMB’s guidance on the 2011 and 2012 orders particularly advised agencies to identify in their final plans specific reforms and initiatives that will significantly reduce existing regulatory burdens (including paperwork and reporting burdens) and promote economic growth and job creation. OMB also directed agencies to provide specific timelines and deadlines for implementation and to quantify burden and cost reductions to the extent feasible. Among other components of guidance on the conduct of retrospective analyses, agencies are expected to conduct a quantifiable assessment of the current costs and benefits of any proposed changes to existing regulations to the extent possible, develop plans for how they will measure the performance of regulations in the future, seek the public’s views on retrospective review plans and related analysis, and coordinate the development of the new retrospective plans with other retrospective review requirements.

The executive orders and related guidance addressed all seven of the recommendations we made in our 2007 report to ensure that agencies conduct effective and transparent retrospective reviews. Among the elements that we recommended incorporating in policies, procedures, or guidance were: minimum standards for documenting and reporting completed review results; including public input as a factor in regulatory review decisions; and consideration of how agencies will measure the performance of new regulations. For more information on key elements of the executive orders and guidance, which we used as criteria during our review, see appendix I.

Agency Priority Goals

GPRAMA requires the 24 agencies identified in the Chief Financial Officers (CFO) Act, or as otherwise determined by OMB, to develop agency priority goals (APG) every 2 years. These goals, which are a subset of the agency’s performance goals, are to reflect the highest priorities of each agency, as identified by the head of the agency, and be informed by broad crosscutting federal government priority goals as well as input from relevant congressional committees. Agencies are to identify the various regulations, as well as federal organizations, program activities, policies and other activities (both within and external to the agency) that contribute to each of their APGs and review progress quarterly. This information is to be included in agencies’ performance plans which are submitted to OMB for publication on Performance.gov.

In April 2013, we reported that OMB’s review process does not systematically check whether agencies have identified all relevant contributors to APGs. In that report we therefore recommended that OMB work to ensure that agencies adhere to its guidance for website updates by providing complete information about regulations, among other things, that contributed to each APG. OMB staff agreed with this recommendation.

Almost All Completed Retrospective Analyses Resulted in Changes to Regulations or Other Actions, but Progress Updates Could Be Improved

The 22 executive agencies within the scope of our review identified 665 initiatives (planned retrospective regulatory analyses) in the final review plans and subsequent updates prepared in response to Executive Orders 13563 and 13610. Between January 2011 and the end of August 2013, the agencies completed and had taken at least some final action on 246 of those planned analyses (see table 1). Because agencies’ analyses are part of an ongoing process, the information in table 1 represents a snapshot of agencies’ progress reported as of August 31, 2013.

Executive Agencies Often Changed Regulations to Improve Effectiveness or Reduce Burden

Agencies’ reports on the progress of their planned retrospective analyses and related documentation showed that analyses completed by the end of August 2013 almost always resulted in the agencies taking additional specific actions, as shown in figure 1.

In 225 of the 246 completed analyses we examined (more than 90 percent), the reviews led to agencies amending sections of the CFR to revise, clarify, or eliminate regulatory text. For example, DHS amended regulations administered by U.S. Customs and Border Protection (CBP) to allow for the publication of seizure and intent-to-forfeit notices on an official government forfeiture website. Previously, notices were posted for three successive weeks in a newspaper circulated at the CBP port and in the judicial district where CBP seized the property. According to DHS, the changes are intended to reduce administrative costs and improve the effectiveness of CBP’s notice procedures as Internet publication will reach a broader range of the public and provide access to more parties who may have an interest in the seized property.

Another 18 of the 246 completed retrospective analyses resulted in agencies taking some other type of action, such as revising agency guidance or evaluation criteria, affecting implementation of the regulations that had been reviewed. For example, DOT conducted a public meeting in February 2012 concerning the criteria used when determining an applicant’s minimum level of fitness for special permits and approvals by the Pipeline and Hazardous Materials Safety Administration under hazardous materials regulations. Based on the comments DOT received during the meeting, the agency decided to undertake a major restructuring of the fitness determinations process and may consider rulemaking once the restructuring is complete. The remaining three completed analyses within our scope resulted in an agency determining that no further action was needed. For example, in one of its completed analyses, EPA sought public feedback on a 2003 water quality trading policy to determine whether revisions to the related policy could help increase the adoption of market-based approaches. But after holding public workshops and webinars on the subject, the agency decided that no revisions were necessary.

Agencies most commonly identified three categories of expected outcomes from the actions they took in response to completed retrospective analyses: (1) improving the effectiveness of regulations; (2) reducing the burden on regulated entities; and (3) providing clarity on regulations or making other administrative changes. In addition, agencies also amended regulations to address statutory changes. These outcomes are not mutually exclusive because agencies reported multiple categories of outcomes for many of the completed analyses. Figure 2 displays a breakdown of the expected outcomes that agencies reported for their completed analyses.

Agencies identified improving the effectiveness of regulations as an expected outcome of 112 completed retrospective analyses. For example, DOT revised its regulations related to Railroad Locomotive Safety Standards in 2012 to improve their effectiveness by incorporating current industry and engineering best practices related to locomotive electronics. The number of outcomes in this category underscores one of the findings in our 2007 report, when agencies said that their primary purpose for conducting most retrospective analyses is to examine the effectiveness of the implementation of regulations.

Agencies cited reducing burden on regulated parties as an anticipated outcome of 99 completed retrospective analyses. For example, in October 2012 the Department of Commerce’s (Commerce) National Oceanic and Atmospheric Administration removed income- qualification requirements related to renewing Gulf of Mexico commercial reef fish permits and increased the maximum crew size for vessels that possess both charter and commercial permits. The stated intent of this rule is to remove permit requirements the agency viewed as no longer applicable to current commercial fishing practices and to improve safety at sea. The prevalence of outcomes in this category is consistent with one of the central goals of the executive orders and OMB guidance on the retrospective analysis initiative— reducing unnecessary regulatory burdens and costs on individuals, businesses, and state, local, and tribal governments.

For 93 of the completed analyses, agencies indicated that the actions they took would clarify regulations or make another type of administrative change. For example, in response to one of its analyses, the Department of State (State) implemented new procedures for its Summer Work Travel Program in 2011. The agency clarified which U.S. host employers and third-party overseas agents or partners could assist in the administration of the core functions of exchange programs.

We found 44 instances of agencies amending existing regulations in response to statutory changes. For example, DOE published a final rule in July 2013 to implement provisions in the Energy Conservation and Production Act that require the department to update the baseline federal energy efficiency performance standards for the construction of new federal commercial and multi-family high-rise residential buildings.

Executive Agencies Largely Attributed Quantified Savings to Reducing Burden

Executive Order 13610 directed agencies to give priority to regulatory reforms that would produce significant quantifiable monetary savings or reductions in paperwork burdens, while protecting public health, welfare, safety, and the environment. OMB’s guidance further directed agencies to quantify burden and cost reductions to the extent feasible when reporting on the progress of their retrospective analyses. Agencies identified quantified cost savings in the published progress updates regarding 38 of the 246 completed analyses in our scope. Half of those 38 analyses focused on reducing burden, including reducing the costs of information collection burdens. For example, HHS issued final rules in May 2012 to remove unnecessary regulatory and reporting requirements for hospitals and other health care providers. The agency reduced the burden to providers and suppliers by modifying, removing, or streamlining regulations that the agency identified as excessively burdensome. For example, HHS eliminated a requirement for non-physician personnel to have special training in administering blood transfusions and intravenous medications. The agency expects these changes to yield more than $5 billion in savings over the next 5 years. In the 38 cases where agencies quantified savings estimates, the extent to which agencies reported information about the methodologies, assumptions, and time periods used to develop those estimates varied. For example, not all agencies identified in their progress reports the discount rate used to discount future savings. In addition, agencies sometimes reported savings estimates based on varying time frames. Without additional information on agencies’ methodologies and key assumptions used to estimate savings in their plan updates and progress reports, a reader cannot determine the total cumulative amount of savings that agencies expected to result from these retrospective analyses. Including more detailed information behind savings estimates would also be consistent with the internal control standard for information and communication, which states that agencies must have relevant, reliable, and timely information for both external and internal uses. Such information could also help Congress and the public to better understand the basis for agencies’ estimates and the extent to which estimates may be comparable.

In other cases, agencies identified but did not quantify expected savings. For example, HHS encouraged that all clinical quality measures be electronically reported beginning in 2014 as part of the Medicare and Medicaid Electronic Health Record (EHR) Incentive Program. Instead of entering data manually or assembling paperwork, providers would instead generate a file from the EHR system. HHS reported that it expects this change to result in savings, but the agency did not quantify such savings. Agency officials we spoke with described impediments to quantifying the costs and benefits of their regulations, such as obtaining sufficient data, which we will discuss further in a later section of this report.

Executive Agencies Increased the Transparency of Review Plans, but Could Improve the Usefulness of Progress Reports to the Public

The executive orders and related OMB guidance contain transparency requirements intended to better inform the public and solicit input about agencies’ retrospective analyses. The orders outline general principles, along with some basic requirements, while guidance memorandums provide more detailed explanations and instructions on how agencies should implement the orders. Among these requirements, agencies were directed to make their retrospective analyses available to the public within a reasonable time period and to release their analyses, including the supporting data, online whenever possible. OMB guidance further recommended that agencies post their analyses on their individual “Open Government” web pages.

Consistent with executive orders and OMB guidance, almost all of the agencies included in our review published retrospective review plans either on their agency websites or other locations. Also, 20 published drafts of their plans in the Federal Register for public comment. Overall, this represents greater transparency compared to conditions we found in our 2007 report. In 2007, we found limitations in agencies’ documentation and reporting of discretionary retrospective analyses. In addition, nonfederal parties cited this lack of transparency as a barrier to the usefulness of retrospective analyses to the public.

Although many agencies in our scope for this review posted their retrospective review plans online, the ease of finding review plans and the comprehensiveness of the plans varied across agencies. For example, agencies did not always post plans on their “Open Government” web pages as recommended in OMB guidance. In some cases, agencies directed readers to multiple plans to identify outcomes from past retrospective analyses, but did not always explicitly state whether the current plan included or excluded a comprehensive list of analyses and outcomes. The White House web page, which posts a compiled list of agency plans, did not always include agencies’ semiannual progress reports showing the outcomes and progress of analyses. The last updates posted on that web page were in May 2012. OMB OIRA staff provided us with updated information in April 2014 explaining that the agency made a deliberate decision to discontinue updating the list of progress updates on the White House web page in an effort to reinforce that the responsibility reside with the individual agencies to post all of their plans and updates on their “Open Government” web pages. During our review, agencies’ plan updates and progress reports provided only summary information about completed analyses. Agencies did not always provide citations or references in the progress reports that a reader could use to look up published rules that contain more detailed descriptions of agencies’ analyses and the underlying data. Without more consistent and comprehensive reporting, the public may not be fully aware of the scope, progress, and results of agencies’ planned analyses. Providing links or citations that point readers to documentation of the supporting analyses and data, as outlined in the executive orders and guidance, would help agencies to better adhere to executive directives, enhance the public transparency of retrospective analyses, and improve the usefulness of reporting.

Independent Regulatory Agencies Followed a Different Retrospective Analysis Planning Process than Executive Agencies

Independent regulatory agencies were not required to develop retrospective review plans, but were encouraged to do so in Executive Order 13579 and a related memorandum providing additional guidance. Both of the independent regulatory agencies included within our scope, FCC and FTC, posted their final retrospective review plans on their agency websites. FCC also posted its final retrospective review plans in the Federal Register whereas FTC did not. Neither agency provided updates to the plans that identified the results or progress of their reviews, but are also not required to do so. However, both agencies said they have well-established practices to regularly review regulations and report outcomes. For example, FTC has a long-standing practice of maintaining a schedule, published annually, to ensure that each rule is reviewed approximately every 10 years. The commission modifies the schedule to accelerate the review of rules when it observes economic or marketplace changes that necessitate a rule review. FCC provided additional explanation that it did not develop an updated plan partly due to transitions among the agency’s top leadership. Specifically, the chairman position for the commission was vacant until November 4, 2013.

We checked on planned analyses that FCC and FTC had identified in their plans and identified multiple completed analyses within the time period for our review. All of the two agencies’ completed retrospective analyses that we reviewed resulted in changes to regulations, most often to improve the effectiveness of the regulations or make administrative changes. Of the 32 completed FCC analyses in our scope, 30 resulted in administrative changes, in particular cumulatively eliminating about 190 regulations that were obsolete. For example, FCC eliminated outdated and unnecessary reporting requirements related to international telecommunications traffic. Twenty of FCC’s analyses focused on improving the effectiveness of the regulations, one focused on burden reduction, and two responded to statutory changes. Four of FTC’s six completed analyses resulted in changes to improve the effectiveness of the regulations, four reduced burden, three addressed administrative changes, and two responded to statutory changes. Similar to executive agencies, these numbers are not mutually exclusive as some analyses led to multiple reported outcomes. For example, as part of its ongoing regulatory analysis of the Appliance Labeling Rule, FTC streamlined data reporting requirements for manufacturers, clarified testing requirements and enforcement provisions, improved online energy label disclosures, and made several minor technical changes and corrections.

In a December 2013 report on Dodd-Frank Act regulations, we assessed financial regulatory agencies’ plans to conduct retrospective reviews of existing rules. We found that federal financial regulators vary in their approaches and progress in developing and implementing plans to conduct retrospective reviews of their existing Dodd-Frank and other rules in recognition of Executive Order 13579. One of the independent financial regulatory agencies covered in that report, the Commodity Futures Trading Commission (CFTC) developed a final retrospective review plan. CFTC outlined a two-phase plan to conduct periodic retrospective reviews of its existing regulations in a June 2011 Federal Register notice. Under the first phase, CFTC reported that it has examined and revised a number of its existing regulations as part of its implementation of the Dodd-Frank Act. Under the second phase, CFTC plans to conduct retrospective reviews of the remainder of its regulations after substantial completion of its Dodd-Frank Act rulemakings. CFTC provided OMB with periodic status reports on its retrospective review plan and reported in July 2013 that it is still in phase one. The Securities and Exchange Commission is in the process of developing a retrospective review plan. Other prudential regulators reported that they generally view their retrospective rule reviews conducted by statute or policy to be consistent with Executive Order 13579’s principles and objectives. Thus, they did not develop a plan directly in response to the order.

Agency Officials Identified Strategies and Barriers That Affected Their Implementation of Retrospective Analyses

Officials from the nine agencies that we selected to complete questionnaires and participate in our roundtable discussion cited many strategies and practices their agencies used as facilitators, or factors the agencies encountered as barriers, which affected their ability to implement key retrospective analysis requirements or guidance. The following sections highlight those facilitators and barriers that were most commonly reported by officials across the nine agencies.

Agencies Identified Three Key Strategies to Facilitate Retrospective Analyses

Agency officials identified multiple strategies or practices that helped to facilitate their ability to conduct productive retrospective analyses. They most commonly identified three strategies or practices as being most significant in their experience. Two-thirds or more of the agencies cited each of the following three strategies as facilitators.

Establish a Centrally Coordinated Review Process to Develop Review Plans

Six of the nine agencies reported that a standard centralized review process to develop retrospective review plans and track the progress of outcomes was a helpful strategy. Agencies often achieved this centralization through their policy analysis or general counsel’s offices. In some instances, agencies also formed intradepartmental working groups. For example, DHS, which manages the retrospective analysis process through its general counsel’s office, has an ongoing working group of representatives from CBP, Coast Guard, U.S. Citizenship and Immigration Services, Immigration and Customs Enforcement, the National Protection and Programs Directorate, Transportation Security Administration, and the Federal Emergency Management Agency. These representatives are responsible for conducting analyses within their component and building retrospective analysis into the agency culture. According to agency officials, the working group helps the department’s components share best practices and learn from each other. FCC officials also said that centralizing their retrospective review effort in the general counsel’s office, while seeking input from each bureau and office, is a key factor that is helping the agency to implement retrospective analyses.

Leverage Existing Regulatory Activities to Identify Needed Changes

Six of the nine agencies reported that leveraging existing regulatory activities—such as regulatory planning, rulemaking, and enforcement processes—assisted them in implementing the executive orders and related guidance. Officials from most agencies reported that, when developing new rules, they examine existing regulations related to the rule as a normal course of conducting business. Agency officials expressed frustration at the misperception that they are not reviewing existing regulations, when in fact most of their regulatory activities involve such reviews. Agencies identified many examples of how they leveraged existing regulatory activities to develop their retrospective review plans in response to the executive orders. For example, the day-to-day regulatory enforcement activities of law enforcement staff informed which regulations to review and provided valuable information for assessing regulations. DOT reported that one of its agencies identified 15 to 20 regulations to review based on inspectors who noted consistent problems in the field, repeated questions from the public seeking clarification on certain regulations, and petitions for regulatory changes. Similarly, FTC reported that its law enforcement staff saw how regulations were working in the marketplace and provided ideas on whether to accelerate a review of a rule to address changes in the marketplace. These staff members were heavily involved in the regulatory review process and used their knowledge to inform how rules were working in practice. To help develop FCC’s review plan, agency officials said that they used existing review processes, such as the statutorily required biennial review of regulations related to telecommunications regulations.

Use Existing Feedback Mechanisms to Identify and Evaluate Regulatory Reforms

All of the agencies said that they used existing external feedback mechanisms to identify and evaluate regulatory reforms. In general, agency officials said that such mechanisms, which include petitions, listening sessions, and meetings with informed stakeholders, were more effective and useful than general public comments obtained through Federal Register notices. For example, one DHS official said that when commenting on the rulemaking process, members of the public tended to voice their opinions about general policy topics unrelated to the regulation being reviewed and, often, did not provide useful suggestions for regulatory changes. Some agency officials noted that the feedback they received from informed stakeholders, such as regulated entities, and policy advocacy groups, was more targeted and therefore more helpful for retrospective review purposes.

Among other specific examples of useful feedback mechanisms, HHS officials with the Office of Policy and Planning within the Office of the National Coordinator for Health Information Technology said stakeholder feedback and emails requesting regulatory interpretations in a final rule were most useful because they helped them focus on areas of ambiguity or potential burden that could be reduced by altering or removing the regulatory requirement. One DOT official said the agency has advisory committees for almost all of its public safety regulatory areas. These committees include regulated entities and public safety advocates who regularly meet with DOT to recommend approaches to new, and revisions to, existing regulations at the front end of the process—usually before a proposed rule is published in the Federal Register. The DOT official said this helps the revision process run much more smoothly than it would otherwise. One FCC official said the agency has several active channels already in place to obtain public input in addition to the comments it receives from its commissioners, bureau chiefs, and advisory committees. Other agencies reported having similar feedback mechanisms that they considered to be most useful.

Agencies Also Identified Other Helpful Strategies

A few of the agencies identified other helpful strategies for facilitating retrospective reviews. For example, the State Department requires its staff to submit a checklist to track the retrospective reviews of existing rules and reviews of proposed and final rules. This checklist served as an official record of the systematic review and addressed the elements of each significant rule. DHS said that compiling information about its regulations (for example, in a database), such as the normalized costs and benefits or the small entity impacts of each regulation, has been a useful tool. This information assisted the agency in identifying major cost drivers and regulations that were candidates for retrospective review and analysis. Another strategy cited by DOE was interagency discussion to identify and implement best practices adopted by other agencies. To obtain useful information from stakeholders, FTC adopted a standardized series of questions that provided a starting point to facilitate public comment on regulations subject to retrospective review.

Persistent Barriers to Implementing Retrospective Analyses Remain

While agency officials representing the nine agencies participating in our roundtable identified multiple challenges or barriers to conducting retrospective reviews, the most commonly cited barriers identified as being most significant to their experience were (1) competing priorities and (2) difficulty obtaining sufficient data. The officials confirmed that these two factors have been persistent impediments to implementing retrospective analyses. This is consistent with barriers we found in our 2007 report. More than half of the agencies mentioned both of these factors as barriers to conducting retrospective reviews.

Competing Priorities Hinder Agencies’ Ability to Conduct Retrospective Analyses

Seven of the nine agencies cited competing priorities, particularly in a constrained budget environment, as a significant challenge to conducting retrospective regulatory analyses. Specifically, they referred to having a limited number of staff with the needed expertise to conduct retrospective analyses and other regulatory activities that are also among their agencies’ priorities. For example, Commerce, DHS, and DOT officials reported that responding to new regulatory mandates imposed by Congress has been the focus for their staff, thereby limiting the amount of time and resources available to conduct retrospective analyses. DHS officials, in particular, pointed out that their agency cannot reassign existing staff to focus solely on retrospective reviews. HHS officials cited staff turnover across program areas and having limited staff with the necessary expertise to conduct retrospective analyses. According to agency officials, this issue can be exacerbated by requirements to report the same information multiple times to accomplish agencies’ goals through duplicative processes. For example, in addition to conducting retrospective reviews, HHS’s Centers for Medicare and Medicaid Services considers the need to eliminate or revise old, obsolete, or duplicative regulations when it reviews its plans quarterly for regulatory revisions or the need for new regulations. The agencies’ feedback on this issue echoed the findings in our 2007 report. In that report, agencies cited lack of time and resources as the most critical barriers to their ability to conduct retrospective analyses because of the need to perform analyses simultaneously with other mission-critical activities. This challenge of effectively balancing agency efforts among competing priorities reinforces the importance of the processes agencies use to select and prioritize which regulations to target for retrospective analysis. The challenge is magnified when considered in the context of a very extensive, and growing, inventory of existing regulations from which agencies must choose regulations to retrospectively assess within the limits of available resources.

Agencies Reported Difficulty Obtaining Sufficient Data to Identify Improvements Attributed to Regulations and Other Barriers

Five of the nine agencies cited a lack of data to identify improvements attributed to regulations as a barrier to conducting reviews. In particular, agency officials cited challenges related to obtaining information from industry, the cost of data from outside sources, and the potential to transfer some data-collection burdens to the states, which face similar resource constraints. For example, DOT officials said that for their agency to have better data for some cost-benefit analyses, it would have to put an additional burden on states to collect it, such as with police officers filling out longer accident forms to more accurately code how someone was injured. In addition, a DOT official said the data may be unavailable or too expensive to obtain, and it often can be against a regulated entity’s interest to provide certain information. DHS said most of the data needed to conduct retrospective analyses of its regulations require original data collection efforts to obtain potentially business-sensitive information from industry. According to DHS officials, restrictions under the Paperwork Reduction Act make it difficult to obtain information from a sufficient number of regulated entities to produce meaningful and useful results. A DOE official also said that additional data are constantly sought by its scientists and economists, and many data sets are expensive and have strict user requirements. Some agencies we spoke with noted, however, that at times they have more readily accessible existing data that can help inform their efforts to assess the performance of regulations. For example, to aid in its enforcement responsibilities, Commerce’s Bureau of Industry and Security (BIS) uses U.S. export data already collected by the Census Bureau, which is another component within the department. One BIS official said these data help the agency conduct cost-benefit analyses and measure how their regulations are working. The barriers discussed above are consistent with findings from our 2007 report, in which agencies reported they lacked the information and data needed to conduct reviews, such as baseline data for assessing regulations that they promulgated many years ago.

Agencies also confirmed other barriers that affected their ability to implement retrospective analyses, such as difficulty in (1) distinguishing between which results were produced by the regulation versus other factors, (2) resolving differences in opinion among internal or external stakeholders on the results, and (3) using the results to make regulatory decisions. For example, FTC reported that its ability to use analytical results of retrospective reviews to make changes to regulations were affected to a moderate extent by the difficulty in distinguishing between which results were produced by the regulation and which stemmed from other factors. In addition, FCC said its ability to use analytical results of retrospective reviews for the same purpose was affected to a moderate extent by the difficulty in resolving differences of opinion among internal or external stakeholders. Few agencies identified strategies to overcome the barriers discussed above. For example, the United States Patent and Trademark Office, within the Department of Commerce, combined two processes to leverage limited resources. The agency implemented retrospective review practices at the same time they were issuing new rules related to the Leahy Smith America Invents Act. In general, there was ultimately no consensus among the roundtable participants about strategies that might be more widely employed to effectively implement retrospective review while overcoming these barriers. However, later in this report we will discuss a related recommendation that we made to OMB in June 2013 to help address this matter. We recommended that the agency develop a detailed approach to examine such difficulties.

Opportunities Exist to Improve Linkages between Regulatory Reviews and Priority Goals

Retrospective reviews are just one way, among others such as program evaluations, to help agencies evaluate their own performance. Given how important regulations are to the missions of regulatory agencies, the information generated from retrospective reviews can be leveraged to directly inform agency annual performance goals and measurement. Moreover, as discussed earlier, GPRAMA requires certain agencies, as determined by OMB, to develop APGs every 2 years. In addition to developing APGs, which are a subset of agencies’ broader performance goals, agencies are required to identify the various regulations, as well as federal organizations and other program activities (both within and external to the agency) that contribute to each of their APGs, and review progress on a quarterly basis. This information is to be published on a governmentwide performance website, which OMB has implemented as Performance.gov, as well as in agencies’ performance plans submitted to OMB. In April 2013, we concluded that OMB’s review process does not systematically check whether agencies have identified all relevant contributors to APGs. Thus, we recommended that the Director of OMB ensure that agencies adhere to OMB’s guidance for website updates by providing complete information about regulations, among other activities, that contribute to each APG. Our review of the criteria agencies used for retrospective reviews shows they used multiple criteria, none of which explicitly mention ties to performance goals. Although the executive orders and guidance did not specifically direct agencies to include their broader performance goals, such as APGs, as a criterion, agencies may be missing a key opportunity to link these two activities.

Agencies Reported Mixed Experiences Linking Retrospective Analyses to APGs

Agencies’ responses about linkages between retrospective analyses and APGs were mixed. Three agencies (DOE, DHS, and DOT) said their recently completed, current, or future retrospective analyses were linked to their APGs to either a moderate or great extent. To illustrate this linkage, DOE said one of its APGs includes energy and costs savings as well as greenhouse gas reduction through the issuance of new or revised energy efficiency standards. As part of the agency’s process for issuing revised standards, DOE said it considers retrospective analyses of the existing standards to determine if greater efficiency can be achieved at lower costs or with lower burden on the regulated community. Responses from the other agencies varied, generally stating that either (1) there was little to no connection between retrospective analyses and APGs or (2) agency officials did not know or had no basis to judge whether such linkages existed. In instances where multiple offices or bureaus within an agency completed the questionnaire, the responses also varied. For example, based on the responses we received from multiple offices within some agencies, it appears there is a mixed understanding or awareness of whether retrospective analyses are linked to APGs. This can be explained, in part, by the division of responsibilities within an agency where the office or staff responsible for carrying out retrospective analyses differs from the staff responsible for carrying out broader agency performance reviews.

Although only three agencies in our review reported moderate or great linkages between their retrospective analyses and APGs, agency officials in offices from seven of the nine agencies identified regulations as key contributors to APGs. For example, while State reported little to no linkage between its retrospective analyses and APGs, the agency identified categories of regulations that contribute to three APGs, including “strengthening diplomacy and development by leading through civilian power.” State’s Bureau of Consular Affairs administers regulations that directly support this goal. DHS identified specific categories of regulations that contribute to its fiscal year 2013 APGs. For example, security regulations directly support the agency’s APG to “strengthen aviation security counterterrorism capabilities by using intelligence-driven information and risk-based decisions.” Immigration and border management regulations directly support DHS’ APG to “improve the efficiency of the process to detain and remove criminal aliens.”

Most agencies cited linkages between retrospective analyses and agency goals other than APGs, such as agency strategic objectives or performance goals. However, the extent of these linkages varied. For example, Commerce’s BIS said its retrospective analyses are tied to other agency goals to a great extent. During our roundtable discussion, one BIS official said the bureau’s regulations directly support a department-level strategic objective. This linkage is documented in BIS’ fiscal year 2015 budget submission and Commerce’s fiscal year 2014- 2018 strategic plan, which shows how BIS regulations contribute toward Commerce’s strategic objective to increase U.S. exports by broadening and deepening the U.S. exporter base. In addition, FTC said its ongoing retrospective review program ensures that FTC’s rules target deceptive and unfair practices while also addressing changing market conditions and avoiding undue burdens on legitimate business activity. Although agencies identified examples where they established such linkages as described above, they may not be able to do so in all instances. Some agency officials we spoke with emphasized that regulations are only one contributor, among others, to broader agency performance goals being examined at higher levels within the agency. For example, one DOE official indicated there may not necessarily be a one-for-one linkage between an individual regulation and a single specific goal. Further, in a March 2004 report, we found other difficulties regulatory agencies face in measuring performance including (1) obtaining data to demonstrate results, (2) accounting for factors outside of the agency’s control that affect results, and (3) dealing with the long time periods often needed to see results.

We asked agencies the extent to which they would like to receive more assistance or guidance on incorporating retrospective analyses into measuring and achieving agency goals, including APGs. Agencies expressed varying degrees of interest in receiving such assistance. In the cases where agencies received additional guidance on this matter, it was provided either internally within the individual agency or from OMB’s OIRA. Only one of the nine agencies reported receiving additional assistance or guidance from other federal oversight offices and groups, such as the Performance Improvement Council (PIC). Given the common, long-standing difficulties agencies have continued to face in measuring the performance of various types of federal programs and activities, including regulations, we recommended in June 2013 that OMB work with the PIC to develop a detailed approach to examine these difficulties across agencies. This work would include identifying and sharing any promising practices from agencies that have overcome difficulties in measuring performance of these programs and activities.

OMB staff agreed with this recommendation. As of March 2014, PIC staff told us they have taken initial steps to implement this recommendation through a pilot effort focused on acquisition, which is related to one of the areas highlighted in our June 2013 report. They said they plan to expand the model to focus on other issues, such as regulations and grants. According to PIC officials, it is too early in the process to have any supporting documentation related to these plans. We will continue to monitor progress on this effort.

As noted earlier, one of the potential purposes for conducting retrospective analyses is to assess whether regulations, once implemented, achieved the expected benefits at the expected costs. However, the efforts of agencies included in our review to re-evaluate original cost-benefit analyses associated with their regulations varied. In response to our questionnaire, only three agencies reported that they often conduct reviews of regulations to determine whether the regulations are accomplishing the originally intended benefits at the expected cost. The other six agencies generally reported that they rarely, if ever, do this or did not know. Some agencies said they will sometimes revisit cost- benefit estimates to improve methods or models for conducting such analyses in the future. However, according to a few agency officials, they do not believe redoing past cost-benefit analyses is useful in the context of making decisions about individual regulations looking forward. In a broader context however, reexamining benefits and costs achieved after a regulation is implemented could provide data useful for performance reviews, including assessments of APGs, and is consistent with regulatory executive orders and guidance. One of the principles in Executive Order 13563 states that the regulatory system must measure, and seek to improve, the actual results of regulatory requirements. Subsequent guidance from the OIRA Administrator noted that this “points to the need for empirical assessment of the consequences of rules.” Broadening the focus to think about retrospective reviews as a contributor to agency performance metrics related to APGs, rather than only in the context of examining a single regulation, may encourage agencies to revisit opportunities for more of this type of analysis.

Opportunities Exist to Better Link Retrospective Analyses and Agency Priority Goals

The focus of retrospective analyses differs from agency performance reviews. Agencies’ retrospective analyses tended to examine regulations on a micro level rather than their contribution to broader APGs. Regulations are developed and implemented individually, and often re- examined individually. In addition, the impetus for reviews is often statutory requirements, changes in the regulatory environment that affect the rule, availability of agency resources, or executive order requirements. The selection of which rule to review is typically tied to criteria such as the number of complaints or comments from regulated parties and the public, whether there have been changes in technology or other conditions since the regulation was issued, the potential for burden reduction, or the number of years since a regulation was promulgated or last reviewed, rather than whether it will be subject to upcoming performance goal discussions. The regulatory process and executive orders outlining how to conduct retrospective reviews have helped to shape the focus and timing of reviews.

In contrast, agencies’ performance reviews are designed to examine measures that cut across programs and policy tools at a macro level. For example, reviews of agency priority goals typically examine how agencies are accomplishing broad mission goals that may be implemented by a wide range of regulations, programs, grants, or other tools. As an illustrative example, in DOT’s fiscal year 2014 performance plan, the agency organized the descriptions of its planned performance—including its agency priority goals—into broad themes under its strategic goals. As shown in figure 3, DOT identified the regulations and enabling legislation that contribute to its APG for aviation safety.

Given the specific focus of retrospective regulatory analyses, several agency officials said the staff conducting the analyses often oversee implementation of the regulation and are not involved in, or sometimes aware of, performance discussions that are held at higher levels of the agency. Similar to the earlier discussion about barriers, agency officials said it may sometimes be difficult to isolate the effect of the regulation versus other external factors contributing to the same outcome. Including consideration of APGs as another part of planning and reporting retrospective reviews could strengthen these linkages. Currently, the selection of regulations to review is generally driven by a bottom-up process, while performance reviews are driven by a top-down process. While both processes have a useful purpose and structure, retrospective reviews could help inform broader performance discussions if agencies were to better link (1) the selected regulations to review, (2) the timing of reviews, and (3) the information assessed. First, as previously noted, the selection of rules to review is typically tied to specific criteria. Including whether a regulation contributes to an APG expected to be reviewed by management as one of the criteria for prioritizing reviews could help retrospective analyses contribute useful information to the discussion of whether and how goals are being accomplished. Second, better alignment between the timing of when regulations are reviewed and when APGs related to those regulations are examined by agency leadership would help to ensure that evaluations are conducted when the information would be most useful. Third, examining regulations that collectively contribute to a specific APG could also improve the usefulness of retrospective reviews. As illustrated in figure 3, multiple DOT regulations contributed to the agency’s APG on aviation safety.

Greater attention to the cumulative effects of regulations would also help agencies to better address certain key analytical and management principles. First, this would be consistent with Executive Order 13610’s direction that agencies consider the cumulative effects of their regulations, among other priorities. In addition, as agencies implement their APGs under GPRAMA, they are required to identify and assess the various contributors to those goals, which can include multiple regulations. Information generated from retrospective analyses can serve as one source of valuable information about such cumulative effects. Finally, this would help agencies to meet the internal control standard for information and communications. Among other things, the standard states that program managers need operational data to determine whether the agency is meeting its strategic and annual performance goals for accountability and for the effective and efficient use of resources. Cumulative analysis could include joint reviews with other agencies as appropriate. Four agencies we spoke with reported conducting joint reviews with other agencies. For example, HHS officials said the Centers for Disease Control and Prevention, a division within HHS, co-regulates a shared list of select agents and toxins with the Animal Plant Health and Inspection Service (APHIS) within the U.S. Department of Agriculture. According to HHS, APHIS provides input during related retrospective reviews of these regulations. This type of joint review can provide more comprehensive and useful information about the collective impact of the rules on performance and on the regulated community.

To better position agencies to conduct future retrospective analyses that could contribute to performance reviews, it is important for agencies to have considered in advance how they will evaluate their regulations. In 2007, we recommended that during the promulgation of certain new rules (such as significant rules) agencies consider whether and how they will measure the performance of the regulation, including how and when they will collect, analyze, and report the data needed to conduct a retrospective review. Consistent with that recommendation, OMB guidance on implementing retrospective analysis states that future regulations should be designed and written in ways that facilitate evaluation of their consequences and thus promote retrospective analyses. OMB asked agencies to identify in their retrospective review plans what metrics the agencies will use to evaluate regulations after they have been implemented. We asked OIRA the extent to which agencies are implementing that guidance. One OIRA official said his office has seen some evidence of agencies writing rules in ways that promote future retrospective analyses. He cited a December 2013 final rule by Commerce’s National Oceanic and Atmospheric Administration regarding regulations to reduce the threat of ship collisions with North Atlantic right whales. The final rule removed an expiration date (or “sunset clause”) contained in the regulations but included provisions for the agency to review the costs and benefits of the rule on a periodic basis, as required by Executive Order 13563. The agency expects to conduct such a review no later than 5 years from the publication of the final rule. The rule also discussed data and metrics the agency could use as measures of effectiveness. Ensuring that agencies build in such performance metrics and a timeline for evaluating regulations after implementation would not only help facilitate retrospective analyses, but also help to lay a foundation to more closely tie retrospective analyses to reviews of broader agency priority goals. Moreover, GPRAMA’s requirements for agencies to identify and assess how their various programs and activities, including regulations, contribute to agency performance goals and APGs further underscore the need for agencies to take such action.

Conclusions

We have long advocated the potential usefulness to Congress, agencies, and the public of conducting retrospective regulatory analyses while recognizing the potential difficulties of doing so. From January 2011 through August 2013, executive agencies identified more than 650 initiatives (planned retrospective regulatory analyses). Of these initiatives, 246 resulted in agencies implementing some type of final action by August 31, 2013. The majority of the completed reviews have led to regulatory changes focused on outcomes such as reducing burden and improving the effectiveness of regulations. While OMB has provided a sample template for agencies to use for reporting on the progress and the results of retrospective analyses, it could do more to enhance the transparency and usefulness of the information provided to the public. Although agencies posted their retrospective review plans online, obtaining a comprehensive picture of agencies’ progress was difficult when results were spread across multiple plans. In addition, consistently providing links or citations to the supporting analyses and data, and including more detail on the methodologies and key assumptions used to estimate savings, would help Congress and the public to better understand the basis for projected results.

Given how important regulations are to the missions of regulatory agencies, the information generated from retrospective analyses can also be leveraged to inform APGs, which are a subset of agencies’ broader performance goals. The broader perspective and focus of performance management and reviews may also enhance evaluation of the effectiveness of regulations beyond what is possible through retrospective analysis alone. This is especially true given the practical limits to the number of analyses that could be done compared to the large and growing inventory of regulations. We recognize that agencies conduct retrospective analyses for many reasons. However, contributing to evaluation of agency priority goals could also be among those reasons. Therefore, agencies should at least consider whether opportunities exist to inform performance reviews when selecting which regulations to retrospectively review. Among the agencies that participated in our roundtable, few identified examples where their retrospective review analyses were linked to APGs. Steps that agencies can take to strengthen the linkages between retrospective reviews and performance reviews include identifying specific regulations that contribute to APGs and including whether a regulation contributes to a key performance goal expected to be reviewed by management as one of the criteria for prioritizing retrospective reviews. Doing so may also provide greater impetus for agencies to review the benefits and costs of existing regulations as part of their retrospective analyses. Many of the agencies we spoke with reported rarely doing so in the context of reviewing individual regulations, and some agencies saw little value to the investment. However, there would be more incentive to measure benefits and costs if retrospective analyses were viewed in the broader context of providing information on the actual, rather than projected, performance and results of regulatory programs and agencies. This would also reinforce existing guidance that agencies should identify how they will measure the performance of new significant regulations when originally published.

The executive branch has already incorporated many, but not all, of our prior relevant recommendations into current guidance. Some of the opportunities to improve reporting on outcomes of retrospective analyses and strengthen linkages between these analyses and APGs could be implemented through augmenting existing guidance. However, additional opportunities for improvement depend in part on efforts to ensure that agencies are consistently held accountable for implementing existing guidance.

Recommendations for Executive Action

To improve agencies’ retrospective regulatory review processes and reporting, and strengthen linkages between retrospective reviews and agency performance management, we recommend the Director of the Office of Management and Budget direct the Administrator of the Office of Information and Regulatory Affairs to take the following three actions: 1. Work with regulatory agencies to implement existing guidance, and update guidance where needed, to improve the reporting of outcomes in their retrospective regulatory review plans by taking actions such as: publishing a link to updated plans, which list recent results and anticipated outcomes, on the White House website; submitting evidence that agencies listed updates of their plans on their “Open Government” web pages; providing more comprehensive information on completed reviews in agencies’ most recent plans and progress reports by (1) ensuring the most recent published plan contains a complete accounting of all completed reviews rather than expecting readers to review multiple plans, and (2) including the supporting analysis and data for results by listing a link or citation to the related documentation. 2. Ensure that the contributions made by regulations toward the achievement of APGs are properly considered and improve how retrospective regulatory reviews can be used to help inform assessments of progress toward these APGs by directing in guidance that agencies take such actions as: identifying whether a regulation contributes to an APG expected to be reviewed by management as one of the criteria for prioritizing retrospective analyses and for the timing of these analyses; and once an agency prioritizes a retrospective analysis based, in part, on its support of an APG, improving the usefulness of that analysis by examining regulations that collectively contribute to the goal in the scope of the review as appropriate. 3. Ensure that OIRA, as part of its oversight role, monitor the extent to which agencies have implemented the guidance on retrospective regulatory review requirements outlined in the related executive orders and confirm that agencies have identified how they will assess the performance of regulations in the future.

Agency Comments and Our Evaluation

We provided a draft of this report to the Director of the Office of Management and Budget, the Secretaries of Commerce, Energy, Health and Human Services, Homeland Security, State, and Transportation, as well as the Administrator of the Environmental Protection Agency and the Commissioners of the Federal Communications Commission and Federal Trade Commission.

In oral comments received on April 3, 2014, staff from OMB’s Office of Information and Regulatory Affairs (OIRA) generally agreed with the recommendations in this report. In response to our first recommendation, OIRA staff generally agreed but said they were open to taking actions other than updating the White House website in light of a decision to discontinue posting agencies’ updates as individual agencies assume primary responsibility for posting updates on their “Open Government” web pages. OIRA staff were supportive of identifying ways to improve agencies’ posting and reporting of retrospective analyses and progress updates. In response to our second recommendation, OIRA staff agreed and said the agency is currently working on strategies to help facilitate agencies’ ability to consider and improve how retrospective reviews can be used to help inform APGs, and otherwise be better integrated into agency strategic decision making. In response to our third recommendation, OIRA staff agreed, emphasizing that this remains a priority and therefore they would continue to monitor the extent to which agencies implement the guidance on retrospective regulatory review requirements. Further, OIRA staff said that as part of its review of agencies’ rules, unified agendas, and regulatory plans, OIRA will continue to encourage agencies to identify beforehand how they will evaluate the effectiveness of a regulation after it has been put in place. OIRA staff also provided technical comments and clarifications, which we incorporated as appropriate.

The Departments of Commerce, Homeland Security, Health and Human Services, and State, the Environmental Protection Agency, and the Federal Trade Commission provided technical comments, which we incorporated as appropriate.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the Director of the Office of Management and Budget, the Secretaries of Commerce, Energy, Health and Human Services, Homeland Security, State, and Transportation, as well as the Administrator of the Environmental Protection Agency and the Commissioners of the Federal Communications Commission and Federal Trade Commission. We are also sending copies of this report to relevant congressional committees. In addition, the report is available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-6806 or sagerm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who have made contributions to this report are listed in appendix II.

Appendix I: Key Retrospective Analysis Requirements Listed in Executive Orders and Memorandums Issued Since January 2011

Table 2 provides a summary of the various executive orders and related memorandums that were issued since January 2011 related to agencies’ retrospective regulatory review plans and analyses. For presentation purposes, we highlighted the elements of these directives that relate to (1) planning retrospective analyses, (2) implementing these plans and analyses, and (3) reporting the results of completed analyses.

Appendix II: Staff Contact and Acknowledgments

GAO Contact

Michelle Sager, (202) 512-6806 or sagerm@gao.gov.

Staff Acknowledgments

In addition to the contact named above, Tim Bober, Assistant Director, Leah Q. Nash, Latesha Love, Lou V.B. Smith, and Wesley Sholtes made major contributions to this report. Tom Beall, Tim Guinane, Andrea Levine, Benjamin T. Licht, Donna Miller, Cindy Saunders, Stephanie Shipman, and Stewart Small also made key contributions to this report.