Reexamining Regulations: Agencies Often Made Regulatory Changes, but Could Strengthen Linkages to Performance Goals
Published: 2014-04-11T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Federal agencies issue thousands of regulations annually to address such national goals as public health and safety. Retrospective analysis can help agencies evaluate how existing regulations work in practice. GAO was asked to provide insights on agencies' retrospective analyses. This report identifies for selected agencies (1) the results and anticipated outcomes of retrospective analyses agencies completed, (2) strategies, practices, or factors that affected agencies' ability to implement these analyses, and (3) the extent to which agencies are incorporating the analyses into processes for measuring and achieving agency priority goals. Applying criteria from executive orders, GPRAMA, and related guidance, GAO analyzed documents from 22 executive agencies and 2 independent regulatory agencies that prepared final retrospective review plans. These agencies issued more than 96 percent of all final rules published between 2011 and 2013. GAO also obtained agency officials' views through questionnaires and a roundtable of 9 agencies selected primarily on numbers of completed analyses. The officials' views are not generalizable to all agencies. GAO also interviewed OMB staff.
section_title: What GAO Found
paragraphs: Agencies often made changes to regulations in response to completed retrospective regulatory analyses, but could improve the reporting of progress. Executive Orders and related implementation guidance from the Office of Management and Budget (OMB) require executive agencies, and encourage independent regulatory agencies, to develop and implement retrospective review plans. Agencies use semiannual updates to report on the progress and results of their analyses. The 22 executive agencies in GAO's scope identified more than 650 planned analyses and reported having completed and taken final actions on 246 of those analyses by August 31, 2013. The two independent regulatory agencies in GAO's scope each chose to develop a final retrospective review plan, although not required to do so. More than 90 percent of the completed analyses led executive agencies to revise, clarify, or eliminate regulatory text. Agencies also took other actions such as updating guidance to the public. Agencies most commonly reported three expected outcomes from actions they took: improving the effectiveness of regulations (112 of 246), reducing regulatory burden (99 of 246), and clarifying regulations or making other administrative changes (93 of 246), such as implementing new procedures. Agencies often reported more than one outcome. Agencies quantified expected savings for 38 of the 246 completed analyses, often attributing savings to reduced information collection burdens. However, agencies did not consistently include citations or links to the supporting analyses and data in their progress reports. While OMB guidance contains transparency requirements for agencies to inform the public, OMB could work with agencies to effectively implement the guidance to improve the usefulness of the information agencies report on the results of their analyses.
Officials from the 9 agencies that participated in GAO's roundtable identified three key strategies and two barriers that most often affected their implementation of retrospective analyses. Strategies that facilitated planning and conducting analyses included: (1) establishing a centrally coordinated review process, (2) leveraging existing regulatory activities such as rulemaking and enforcement processes, and (3) using existing external feedback mechanisms such as advisory committees. The most commonly cited barriers were competing priorities for available staff and difficulty obtaining sufficient data.
Retrospective analysis can also help inform agencies' priority goals (APG). The Government Performance and Results Act Modernization Act of 2010 (GPRAMA) requires agencies to assess whether regulations, among other activities, are contributing as planned to APGs. Agencies reported mixed experiences linking retrospective analyses to APGs. The seven roundtable agencies with APGs identified regulations contributing to their priority goals, but their retrospective analyses were only sometimes linked to APGs. In some cases, different offices in the same agency had mixed responses about whether such linkages existed. Several agency officials said staff conducting retrospective analyses were not involved in performance discussions at higher levels of the agency. To inform broader performance planning and reviews, retrospective analyses can be another potential data source for APGs. Agencies could strengthen that linkage by taking actions such as considering APGs, to the extent practicable, when planning retrospective analyses and identifying how they will measure the performance of significant new rules related to priority goals.
section_title: What GAO Recommends
paragraphs: GAO recommends that OMB work with agencies to improve reporting on results of retrospective analyses and strengthen links between those analyses and agencies' performance goals by considering APGs when planning retrospective analyses, among other actions. OMB staff generally agreed with the recommendations in this report.

section_title: Letter
paragraphs: Regulation is one of many tools of government that federal agencies use  to achieve such national goals as protecting the health and safety of the  public and the environment. Each year, agencies issue thousands of  rules, within the scope of their authorizing statutes, to address these  goals. In light of their substantial social and economic benefits and costs,  agencies must comply with multiple procedural and analytical  requirements before issuing significant regulations. However, without  careful oversight of those regulations once implemented, they might  prove to be less effective than expected in achieving their intended goals,  become outdated, or create unnecessary burdens. Regulations may also  change the behaviors of regulated entities and the public in ways that  cannot be predicted by prospective analysis before implementation. For  these reasons, retrospective analysis can be important in evaluating how  existing regulations are working in practice. Agencies could use  retrospective analysis to examine how existing regulations have  contributed to specific policy goals, to assess the effectiveness of their  implementation, or to reexamine their estimated benefits and costs based  on actual performance and experience.
For many years Congress and presidents have directed agencies to  periodically review existing regulations through various statutes and  executive orders. In addition, agencies often exercise their own  discretionary authorities to review existing regulations, as we reported in  2007. To further promote and institutionalize retrospective regulatory  analysis, among other things, in 2011 and 2012 the administration issued  Executive Orders 13563, 13579, and 13610, along with related Office of  Management and Budget (OMB) guidance. Among other provisions,  those executive orders and guidance documents require executive  agencies—and encourage independent regulatory agencies—to develop  and implement retrospective review plans. Subsequent updates of  agencies’ plans (now produced semiannually by executive agencies) also  serve as the primary vehicle for agencies to report on the progress and  results of their retrospective analyses. Furthermore, as a separate,  broader effort to improve the performance of government agencies and  provide greater accountability for results, Congress updated the statutory  framework for performance management in the federal government—the  Government Performance and Results Act of 1993 (GPRA)—with the  GPRA Modernization Act of 2010 (GPRAMA). Particularly relevant to this  report, GPRAMA requires agencies to assess whether regulations,  among many other tools and activities, are contributing as planned to  agency priority goals, and to review their progress on those goals  quarterly.
In this context, you asked us to provide insights concerning the scope,  effectiveness, and results of agencies’ retrospective regulatory analyses.  This report identifies for selected agencies (1) the results and anticipated  outcomes of completed retrospective analyses included in agencies’  review plans and progress reports, (2) strategies, practices, or factors that  facilitated or limited agencies’ ability to implement retrospective analyses,  and (3) the extent to which agencies are incorporating retrospective  analyses into their processes for measuring and achieving agency priority  goals.
To address our first objective, we reviewed relevant policies, executive  orders, and related guidance to identify criteria. We analyzed published  final retrospective regulatory review plans and progress reports from 22  executive agencies and 2 independent regulatory agencies that published  final and updated retrospective analyses between January 2011 and  August 2013. In addition, we analyzed in greater detail those completed  analyses that resulted in agencies implementing a final action by August  31, 2013. These 24 agencies issued more than 96 percent of all final  rules published within the time period covered in our review. The  agencies within our scope include the Departments of Agriculture,  Commerce, Defense, Education, Energy (DOE), Health and Human  Services (HHS), Homeland Security (DHS), Housing and Urban  Development, Interior, Justice, Labor, State, Transportation (DOT),  Treasury, and Veterans Affairs. We also included the Environmental  Protection Agency (EPA), General Services Administration, National  Aeronautics and Space Administration, Office of Personnel Management,  Pension Benefit Guaranty Corporation, Small Business Administration,  Social Security Administration, and, from the independent regulatory  agencies, the Federal Communications Commission (FCC) and Federal  Trade Commission (FTC). Using criteria from Executive Orders 13563,  13579, and 13610, and related guidance, we assessed information that  agencies reported on concerning the implementation of their plans. This  included the number of completed retrospective analyses, the results and  outcomes agencies reported, and whether agencies addressed other  elements identified by the guidance. We specifically focused on those  retrospective analyses that agencies had completed and for which they  had taken a final action (including a decision to take no further action).  We based our analysis of the results on the descriptions agencies  reported in their retrospective review plans and updates, including  anticipated outcomes agencies expect to realize in future years. Thus, we  did not assess the accuracy or achievement of agencies’ reported  outcomes.
To test the reliability of the data sources we used to identify agencies’  final and updated retrospective analyses, we corroborated key elements  from the agencies’ plans and updates, such as the reported completion  dates and expected outcomes of individual analyses. We did this by  reviewing related published documents such as Federal Register notices  and the Unified Agenda of Federal Regulatory and Deregulatory Actions  (Unified Agenda) and the Regulatory Plan. We also interviewed officials  within OMB’s Office of Information and Regulatory Affairs (OIRA) and  confirmed that we included in our scope the most up-to-date agency  review plans. We concluded that the data were sufficiently reliable for the  purpose of identifying the scope and results of agencies’ retrospective  analyses.
To address our second and third objectives, we selected nine agencies to  participate in a structured questionnaire about their implementation of  retrospective reviews, barriers and facilitators in using retrospective  analyses to make regulatory decisions, and incorporating analyses into  agency performance management. Subsequently, representatives from  the nine agencies participated in a more detailed roundtable discussion.  Our selection process was based primarily on those agencies that  completed the greatest number of retrospective analyses during the time  period covered in our review. In addition, we selected agencies whose  analyses reported a mix of anticipated outcomes (such as reduced  burden) and quantified costs and benefits. The selected agencies also  reflect a mix of executive and independent regulatory agencies’  experiences. The nine agencies are: the Departments of Commerce,  Energy, Health and Human Services, Homeland Security, State, and  Transportation, the Environmental Protection Agency, the Federal  Communications Commission, and the Federal Trade Commission. The  views of officials from these agencies are not generalizable to all  agencies, but provided insights from those agencies that most often met  our selection criteria. We also met with OMB staff to supplement  information obtained from the questionnaires and roundtable.
We conducted this performance audit from May 2013 to April 2014 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: Retrospective Regulatory Analysis
paragraphs: Regulations are the means by which agencies establish legally binding  requirements, codified in the Code of Federal Regulations (CFR).  Typically, regulations require a desired action or prohibit certain actions  by regulated parties. The regulatory process is governed by statutes,  executive orders, and agencies’ policies and procedures that require  agencies to evaluate the need for regulations, assess the potential effects  of new regulations, and obtain public input (with certain exceptions)  during their development. In addition, under various congressional and  presidential directives, agencies are expected to systematically review  regulations to ensure they do not become outdated or overly  burdensome, among other things. Agencies also may choose to review  regulations at their own discretion, in response to feedback from  regulated parties and agency staff or changes in technology and market  conditions, among other inputs. Therefore, the normal rulemaking  process often involves agencies reviewing their existing regulations.  Over many years, we have reported on agencies’ activities related to  retrospective regulatory analysis and have demonstrated its usefulness to  Congress, agencies, and the public while recognizing the potential  difficulties, such as data limitations and overlapping or duplicative  requirements.
In 2011 and 2012, the administration issued new directives to agencies  on how they should plan and assess analyses of existing regulations,  among other subjects.
Executive Order 13563, “Improving Regulation and Regulatory  Review,” and related memorandums, directed each executive agency  to develop and submit to OIRA a preliminary plan consistent with law  and its resources and regulatory priorities under which the agency will  periodically review its existing significant regulations to determine  whether they should be modified, streamlined, expanded, or repealed,  so as to make the agency’s regulatory program more effective or less  burdensome in achieving the regulatory objectives.
Executive Order 13579, “Regulation and Independent Regulatory  Agencies,” encouraged independent regulatory agencies to develop  and release to the public plans for retrospective analyses of existing  significant rules.
Executive Order 13610, “Identifying and Reducing Regulatory  Burdens,” contained additional requirements and guidance regarding  public participation in retrospective reviews, setting priorities in  implementing retrospective review plans (for example, focusing on  initiatives that would reduce regulatory burdens), and agencies’  reporting on the status of their retrospective review efforts.
OMB periodically issues guidance to agencies on implementing executive  orders. Under Executive Order 12866, OMB’s OIRA is responsible for  providing meaningful guidance and oversight so that each agency’s  regulations are consistent with applicable law, the President’s priorities,  and the principles set forth in executive orders. For example, on October  26, 2011, the Administrator of OIRA issued a memorandum on the  implementation of the retrospective review plans called for by Executive  Order 13563. The memorandum provided more detailed guidance to  agencies on how to address elements of the order on prioritizing review  plans, seeking public comment, and reporting on the status of  retrospective review efforts (including a suggested template for agencies  to use for their reports). According to Executive Order 12866, OIRA is to  be the repository of expertise concerning regulatory issues.
As we reported in 2007, agencies generally assess their existing  regulations for a variety of purposes, including to determine whether (1)  the expected outcomes of the regulation have been achieved, (2) the  agency should retain, amend, or rescind the regulation, or (3) the actual  benefits and costs of the implemented regulation correspond with  estimates prepared at the time the regulation was issued. However,  OMB’s guidance on the 2011 and 2012 orders particularly advised  agencies to identify in their final plans specific reforms and initiatives that  will significantly reduce existing regulatory burdens (including paperwork  and reporting burdens) and promote economic growth and job creation.  OMB also directed agencies to provide specific timelines and deadlines  for implementation and to quantify burden and cost reductions to the  extent feasible. Among other components of guidance on the conduct of  retrospective analyses, agencies are expected to  conduct a quantifiable assessment of the current costs and benefits of  any proposed changes to existing regulations to the extent possible, develop plans for how they will measure the performance of  regulations in the future,  seek the public’s views on retrospective review plans and related  analysis, and  coordinate the development of the new retrospective plans with other  retrospective review requirements.
The executive orders and related guidance addressed all seven of the  recommendations we made in our 2007 report to ensure that agencies  conduct effective and transparent retrospective reviews. Among the  elements that we recommended incorporating in policies, procedures, or  guidance were: minimum standards for documenting and reporting  completed review results; including public input as a factor in regulatory  review decisions; and consideration of how agencies will measure the  performance of new regulations. For more information on key elements of  the executive orders and guidance, which we used as criteria during our  review, see appendix I.
section_title: Agency Priority Goals
paragraphs: GPRAMA requires the 24 agencies identified in the Chief Financial  Officers (CFO) Act, or as otherwise determined by OMB, to develop  agency priority goals (APG) every 2 years. These goals, which are a  subset of the agency’s performance goals, are to reflect the highest  priorities of each agency, as identified by the head of the agency, and be  informed by broad crosscutting federal government priority goals as well  as input from relevant congressional committees. Agencies are to identify  the various regulations, as well as federal organizations, program  activities, policies and other activities (both within and external to the  agency) that contribute to each of their APGs and review progress  quarterly. This information is to be included in agencies’ performance  plans which are submitted to OMB for publication on Performance.gov.
In April 2013, we reported that OMB’s review process does not  systematically check whether agencies have identified all relevant  contributors to APGs. In that report we therefore recommended that  OMB work to ensure that agencies adhere to its guidance for website  updates by providing complete information about regulations, among  other things, that contributed to each APG. OMB staff agreed with this  recommendation.
section_title: Almost All Completed Retrospective Analyses Resulted in Changes to Regulations or Other Actions, but Progress Updates Could Be Improved
paragraphs: The 22 executive agencies within the scope of our review identified 665  initiatives (planned retrospective regulatory analyses) in the final review  plans and subsequent updates prepared in response to Executive Orders  13563 and 13610. Between January 2011 and the end of August 2013,  the agencies completed and had taken at least some final action on 246  of those planned analyses (see table 1). Because agencies’ analyses  are part of an ongoing process, the information in table 1 represents a  snapshot of agencies’ progress reported as of August 31, 2013.
subsections: section_title: Executive Agencies Often Changed Regulations to Improve Effectiveness or Reduce Burden
paragraphs: Agencies’ reports on the progress of their planned retrospective analyses  and related documentation showed that analyses completed by the end of  August 2013 almost always resulted in the agencies taking additional  specific actions, as shown in figure 1.
In 225 of the 246 completed analyses we examined (more than 90  percent), the reviews led to agencies amending sections of the CFR to  revise, clarify, or eliminate regulatory text. For example, DHS amended  regulations administered by U.S. Customs and Border Protection (CBP)  to allow for the publication of seizure and intent-to-forfeit notices on an  official government forfeiture website. Previously, notices were posted  for three successive weeks in a newspaper circulated at the CBP port and  in the judicial district where CBP seized the property. According to DHS,  the changes are intended to reduce administrative costs and improve the  effectiveness of CBP’s notice procedures as Internet publication will  reach a broader range of the public and provide access to more parties  who may have an interest in the seized property.
Another 18 of the 246 completed retrospective analyses resulted in  agencies taking some other type of action, such as revising agency  guidance or evaluation criteria, affecting implementation of the regulations  that had been reviewed. For example, DOT conducted a public meeting in  February 2012 concerning the criteria used when determining an  applicant’s minimum level of fitness for special permits and approvals by  the Pipeline and Hazardous Materials Safety Administration under  hazardous materials regulations. Based on the comments DOT received  during the meeting, the agency decided to undertake a major  restructuring of the fitness determinations process and may consider  rulemaking once the restructuring is complete. The remaining three  completed analyses within our scope resulted in an agency determining  that no further action was needed. For example, in one of its completed  analyses, EPA sought public feedback on a 2003 water quality trading  policy to determine whether revisions to the related policy could help  increase the adoption of market-based approaches. But after holding  public workshops and webinars on the subject, the agency decided that  no revisions were necessary.
Agencies most commonly identified three categories of expected  outcomes from the actions they took in response to completed  retrospective analyses: (1) improving the effectiveness of regulations; (2)  reducing the burden on regulated entities; and (3) providing clarity on  regulations or making other administrative changes. In addition, agencies  also amended regulations to address statutory changes. These outcomes  are not mutually exclusive because agencies reported multiple categories  of outcomes for many of the completed analyses. Figure 2 displays a  breakdown of the expected outcomes that agencies reported for their  completed analyses.
Agencies identified improving the effectiveness of regulations as an  expected outcome of 112 completed retrospective analyses. For  example, DOT revised its regulations related to Railroad Locomotive  Safety Standards in 2012 to improve their effectiveness by  incorporating current industry and engineering best practices related  to locomotive electronics. The number of outcomes in this category  underscores one of the findings in our 2007 report, when agencies  said that their primary purpose for conducting most retrospective  analyses is to examine the effectiveness of the implementation of  regulations.
Agencies cited reducing burden on regulated parties as an anticipated  outcome of 99 completed retrospective analyses. For example, in  October 2012 the Department of Commerce’s (Commerce) National  Oceanic and Atmospheric Administration removed income- qualification requirements related to renewing Gulf of Mexico  commercial reef fish permits and increased the maximum crew size  for vessels that possess both charter and commercial permits. The  stated intent of this rule is to remove permit requirements the agency  viewed as no longer applicable to current commercial fishing practices  and to improve safety at sea. The prevalence of outcomes in this  category is consistent with one of the central goals of the executive  orders and OMB guidance on the retrospective analysis initiative— reducing unnecessary regulatory burdens and costs on individuals,  businesses, and state, local, and tribal governments.
For 93 of the completed analyses, agencies indicated that the actions  they took would clarify regulations or make another type of  administrative change. For example, in response to one of its  analyses, the Department of State (State) implemented new  procedures for its Summer Work Travel Program in 2011. The agency  clarified which U.S. host employers and third-party overseas agents or  partners could assist in the administration of the core functions of  exchange programs.
We found 44 instances of agencies amending existing regulations in  response to statutory changes. For example, DOE published a final  rule in July 2013 to implement provisions in the Energy Conservation  and Production Act that require the department to update the baseline  federal energy efficiency performance standards for the construction  of new federal commercial and multi-family high-rise residential  buildings.
section_title: Executive Agencies Largely Attributed Quantified Savings to Reducing Burden
paragraphs: Executive Order 13610 directed agencies to give priority to regulatory  reforms that would produce significant quantifiable monetary savings or  reductions in paperwork burdens, while protecting public health, welfare,  safety, and the environment. OMB’s guidance further directed agencies  to quantify burden and cost reductions to the extent feasible when  reporting on the progress of their retrospective analyses. Agencies  identified quantified cost savings in the published progress updates  regarding 38 of the 246 completed analyses in our scope. Half of those  38 analyses focused on reducing burden, including reducing the costs of  information collection burdens. For example, HHS issued final rules in  May 2012 to remove unnecessary regulatory and reporting requirements  for hospitals and other health care providers. The agency reduced the  burden to providers and suppliers by modifying, removing, or streamlining  regulations that the agency identified as excessively burdensome. For  example, HHS eliminated a requirement for non-physician personnel to  have special training in administering blood transfusions and intravenous  medications. The agency expects these changes to yield more than $5  billion in savings over the next 5 years. In the 38 cases where agencies  quantified savings estimates, the extent to which agencies reported  information about the methodologies, assumptions, and time periods  used to develop those estimates varied. For example, not all agencies  identified in their progress reports the discount rate used to discount  future savings. In addition, agencies sometimes reported savings  estimates based on varying time frames. Without additional information  on agencies’ methodologies and key assumptions used to estimate  savings in their plan updates and progress reports, a reader cannot  determine the total cumulative amount of savings that agencies expected  to result from these retrospective analyses. Including more detailed  information behind savings estimates would also be consistent with the  internal control standard for information and communication, which states  that agencies must have relevant, reliable, and timely information for both  external and internal uses. Such information could also help Congress  and the public to better understand the basis for agencies’ estimates and  the extent to which estimates may be comparable.
In other cases, agencies identified but did not quantify expected savings.  For example, HHS encouraged that all clinical quality measures be  electronically reported beginning in 2014 as part of the Medicare and  Medicaid Electronic Health Record (EHR) Incentive Program. Instead of  entering data manually or assembling paperwork, providers would instead  generate a file from the EHR system. HHS reported that it expects this  change to result in savings, but the agency did not quantify such savings.  Agency officials we spoke with described impediments to quantifying the  costs and benefits of their regulations, such as obtaining sufficient data,  which we will discuss further in a later section of this report.
section_title: Executive Agencies Increased the Transparency of Review Plans, but Could Improve the Usefulness of Progress Reports to the Public
paragraphs: The executive orders and related OMB guidance contain transparency  requirements intended to better inform the public and solicit input about  agencies’ retrospective analyses. The orders outline general principles,  along with some basic requirements, while guidance memorandums  provide more detailed explanations and instructions on how agencies  should implement the orders. Among these requirements, agencies were  directed to make their retrospective analyses available to the public within  a reasonable time period and to release their analyses, including the  supporting data, online whenever possible. OMB guidance further  recommended that agencies post their analyses on their individual “Open  Government” web pages.
Consistent with executive orders and OMB guidance, almost all of the  agencies included in our review published retrospective review plans  either on their agency websites or other locations. Also, 20 published  drafts of their plans in the Federal Register for public comment. Overall,  this represents greater transparency compared to conditions we found in  our 2007 report. In 2007, we found limitations in agencies’ documentation  and reporting of discretionary retrospective analyses. In addition,  nonfederal parties cited this lack of transparency as a barrier to the  usefulness of retrospective analyses to the public.
Although many agencies in our scope for this review posted their  retrospective review plans online, the ease of finding review plans and the  comprehensiveness of the plans varied across agencies. For example,  agencies did not always post plans on their “Open Government” web  pages as recommended in OMB guidance. In some cases, agencies  directed readers to multiple plans to identify outcomes from past  retrospective analyses, but did not always explicitly state whether the  current plan included or excluded a comprehensive list of analyses and  outcomes. The White House web page, which posts a compiled list of  agency plans, did not always include agencies’ semiannual progress  reports showing the outcomes and progress of analyses. The last  updates posted on that web page were in May 2012. OMB OIRA staff  provided us with updated information in April 2014 explaining that the  agency made a deliberate decision to discontinue updating the list of  progress updates on the White House web page in an effort to reinforce  that the responsibility reside with the individual agencies to post all of  their plans and updates on their “Open Government” web pages. During  our review, agencies’ plan updates and progress reports provided only  summary information about completed analyses. Agencies did not always  provide citations or references in the progress reports that a reader could  use to look up published rules that contain more detailed descriptions of  agencies’ analyses and the underlying data. Without more consistent and  comprehensive reporting, the public may not be fully aware of the scope,  progress, and results of agencies’ planned analyses. Providing links or  citations that point readers to documentation of the supporting analyses  and data, as outlined in the executive orders and guidance, would help  agencies to better adhere to executive directives, enhance the public  transparency of retrospective analyses, and improve the usefulness of  reporting.
section_title: Independent Regulatory Agencies Followed a Different Retrospective Analysis Planning Process than Executive Agencies
paragraphs: Independent regulatory agencies were not required to develop  retrospective review plans, but were encouraged to do so in Executive  Order 13579 and a related memorandum providing additional guidance.  Both of the independent regulatory agencies included within our scope,  FCC and FTC, posted their final retrospective review plans on their  agency websites. FCC also posted its final retrospective review plans in  the Federal Register whereas FTC did not. Neither agency provided  updates to the plans that identified the results or progress of their  reviews, but are also not required to do so. However, both agencies said  they have well-established practices to regularly review regulations and  report outcomes. For example, FTC has a long-standing practice of  maintaining a schedule, published annually, to ensure that each rule is  reviewed approximately every 10 years. The commission modifies the  schedule to accelerate the review of rules when it observes economic or  marketplace changes that necessitate a rule review. FCC provided  additional explanation that it did not develop an updated plan partly due to  transitions among the agency’s top leadership. Specifically, the chairman  position for the commission was vacant until November 4, 2013.
We checked on planned analyses that FCC and FTC had identified in  their plans and identified multiple completed analyses within the time  period for our review. All of the two agencies’ completed retrospective  analyses that we reviewed resulted in changes to regulations, most often  to improve the effectiveness of the regulations or make administrative  changes. Of the 32 completed FCC analyses in our scope, 30 resulted in  administrative changes, in particular cumulatively eliminating about 190  regulations that were obsolete. For example, FCC eliminated outdated  and unnecessary reporting requirements related to international  telecommunications traffic. Twenty of FCC’s analyses focused on  improving the effectiveness of the regulations, one focused on burden  reduction, and two responded to statutory changes. Four of FTC’s six  completed analyses resulted in changes to improve the effectiveness of  the regulations, four reduced burden, three addressed administrative  changes, and two responded to statutory changes. Similar to executive  agencies, these numbers are not mutually exclusive as some analyses  led to multiple reported outcomes. For example, as part of its ongoing  regulatory analysis of the Appliance Labeling Rule, FTC streamlined data  reporting requirements for manufacturers, clarified testing requirements  and enforcement provisions, improved online energy label disclosures,  and made several minor technical changes and corrections.
In a December 2013 report on Dodd-Frank Act regulations, we assessed  financial regulatory agencies’ plans to conduct retrospective reviews of  existing rules. We found that federal financial regulators vary in their  approaches and progress in developing and implementing plans to  conduct retrospective reviews of their existing Dodd-Frank and other rules  in recognition of Executive Order 13579. One of the independent financial  regulatory agencies covered in that report, the Commodity Futures  Trading Commission (CFTC) developed a final retrospective review plan.  CFTC outlined a two-phase plan to conduct periodic retrospective reviews  of its existing regulations in a June 2011 Federal Register notice. Under  the first phase, CFTC reported that it has examined and revised a number  of its existing regulations as part of its implementation of the Dodd-Frank  Act. Under the second phase, CFTC plans to conduct retrospective  reviews of the remainder of its regulations after substantial completion of  its Dodd-Frank Act rulemakings. CFTC provided OMB with periodic status  reports on its retrospective review plan and reported in July 2013 that it is  still in phase one. The Securities and Exchange Commission is in the  process of developing a retrospective review plan. Other prudential  regulators reported that they generally view their retrospective rule  reviews conducted by statute or policy to be consistent with Executive  Order 13579’s principles and objectives. Thus, they did not develop a  plan directly in response to the order.
section_title: Agency Officials Identified Strategies and Barriers That Affected Their Implementation of Retrospective Analyses
paragraphs: Officials from the nine agencies that we selected to complete  questionnaires and participate in our roundtable discussion cited many  strategies and practices their agencies used as facilitators, or factors the  agencies encountered as barriers, which affected their ability to  implement key retrospective analysis requirements or guidance. The  following sections highlight those facilitators and barriers that were most  commonly reported by officials across the nine agencies.
subsections: section_title: Agencies Identified Three Key Strategies to Facilitate Retrospective Analyses
paragraphs: Agency officials identified multiple strategies or practices that helped to  facilitate their ability to conduct productive retrospective analyses. They  most commonly identified three strategies or practices as being most  significant in their experience. Two-thirds or more of the agencies cited  each of the following three strategies as facilitators.
subsections: section_title: Establish a Centrally Coordinated Review Process to Develop Review Plans
paragraphs: Six of the nine agencies reported that a standard centralized review  process to develop retrospective review plans and track the progress of  outcomes was a helpful strategy. Agencies often achieved this  centralization through their policy analysis or general counsel’s offices. In  some instances, agencies also formed intradepartmental working  groups. For example, DHS, which manages the retrospective analysis  process through its general counsel’s office, has an ongoing working  group of representatives from CBP, Coast Guard, U.S. Citizenship and  Immigration Services, Immigration and Customs Enforcement, the  National Protection and Programs Directorate, Transportation Security  Administration, and the Federal Emergency Management Agency. These  representatives are responsible for conducting analyses within their  component and building retrospective analysis into the agency culture.  According to agency officials, the working group helps the department’s  components share best practices and learn from each other. FCC officials  also said that centralizing their retrospective review effort in the general  counsel’s office, while seeking input from each bureau and office, is a key  factor that is helping the agency to implement retrospective analyses.
section_title: Leverage Existing Regulatory Activities to Identify Needed Changes
paragraphs: Six of the nine agencies reported that leveraging existing regulatory  activities—such as regulatory planning, rulemaking, and enforcement  processes—assisted them in implementing the executive orders and  related guidance. Officials from most agencies reported that, when  developing new rules, they examine existing regulations related to the  rule as a normal course of conducting business. Agency officials  expressed frustration at the misperception that they are not reviewing  existing regulations, when in fact most of their regulatory activities involve  such reviews. Agencies identified many examples of how they leveraged  existing regulatory activities to develop their retrospective review plans in  response to the executive orders. For example, the day-to-day regulatory  enforcement activities of law enforcement staff informed which  regulations to review and provided valuable information for assessing  regulations. DOT reported that one of its agencies identified 15 to 20  regulations to review based on inspectors who noted consistent problems  in the field, repeated questions from the public seeking clarification on  certain regulations, and petitions for regulatory changes. Similarly, FTC  reported that its law enforcement staff saw how regulations were working  in the marketplace and provided ideas on whether to accelerate a review  of a rule to address changes in the marketplace. These staff members  were heavily involved in the regulatory review process and used their  knowledge to inform how rules were working in practice. To help develop  FCC’s review plan, agency officials said that they used existing review  processes, such as the statutorily required biennial review of regulations  related to telecommunications regulations.
section_title: Use Existing Feedback Mechanisms to Identify and Evaluate Regulatory Reforms
paragraphs: All of the agencies said that they used existing external feedback  mechanisms to identify and evaluate regulatory reforms. In general,  agency officials said that such mechanisms, which include petitions,  listening sessions, and meetings with informed stakeholders, were more  effective and useful than general public comments obtained through  Federal Register notices. For example, one DHS official said that when  commenting on the rulemaking process, members of the public tended to  voice their opinions about general policy topics unrelated to the regulation  being reviewed and, often, did not provide useful suggestions for  regulatory changes. Some agency officials noted that the feedback they  received from informed stakeholders, such as regulated entities, and  policy advocacy groups, was more targeted and therefore more helpful for  retrospective review purposes.
Among other specific examples of useful feedback mechanisms, HHS  officials with the Office of Policy and Planning within the Office of the  National Coordinator for Health Information Technology said stakeholder  feedback and emails requesting regulatory interpretations in a final rule  were most useful because they helped them focus on areas of ambiguity  or potential burden that could be reduced by altering or removing the  regulatory requirement. One DOT official said the agency has advisory  committees for almost all of its public safety regulatory areas. These  committees include regulated entities and public safety advocates who  regularly meet with DOT to recommend approaches to new, and revisions  to, existing regulations at the front end of the process—usually before a  proposed rule is published in the Federal Register. The DOT official said  this helps the revision process run much more smoothly than it would  otherwise. One FCC official said the agency has several active channels  already in place to obtain public input in addition to the comments it  receives from its commissioners, bureau chiefs, and advisory committees.  Other agencies reported having similar feedback mechanisms that they  considered to be most useful.
section_title: Agencies Also Identified Other Helpful Strategies
paragraphs: A few of the agencies identified other helpful strategies for facilitating  retrospective reviews. For example, the State Department requires its  staff to submit a checklist to track the retrospective reviews of existing  rules and reviews of proposed and final rules. This checklist served as an  official record of the systematic review and addressed the elements of  each significant rule. DHS said that compiling information about its  regulations (for example, in a database), such as the normalized costs  and benefits or the small entity impacts of each regulation, has been a  useful tool. This information assisted the agency in identifying major cost  drivers and regulations that were candidates for retrospective review and  analysis. Another strategy cited by DOE was interagency discussion to  identify and implement best practices adopted by other agencies. To  obtain useful information from stakeholders, FTC adopted a standardized  series of questions that provided a starting point to facilitate public  comment on regulations subject to retrospective review.
section_title: Persistent Barriers to Implementing Retrospective Analyses Remain
paragraphs: While agency officials representing the nine agencies participating in our  roundtable identified multiple challenges or barriers to conducting  retrospective reviews, the most commonly cited barriers identified as  being most significant to their experience were (1) competing priorities  and (2) difficulty obtaining sufficient data. The officials confirmed that  these two factors have been persistent impediments to implementing  retrospective analyses. This is consistent with barriers we found in our  2007 report. More than half of the agencies mentioned both of these  factors as barriers to conducting retrospective reviews.
subsections: section_title: Competing Priorities Hinder Agencies’ Ability to Conduct Retrospective Analyses
paragraphs: Seven of the nine agencies cited competing priorities, particularly in a  constrained budget environment, as a significant challenge to conducting  retrospective regulatory analyses. Specifically, they referred to having a  limited number of staff with the needed expertise to conduct retrospective  analyses and other regulatory activities that are also among their  agencies’ priorities. For example, Commerce, DHS, and DOT officials  reported that responding to new regulatory mandates imposed by  Congress has been the focus for their staff, thereby limiting the amount of  time and resources available to conduct retrospective analyses. DHS  officials, in particular, pointed out that their agency cannot reassign  existing staff to focus solely on retrospective reviews. HHS officials cited  staff turnover across program areas and having limited staff with the  necessary expertise to conduct retrospective analyses. According to  agency officials, this issue can be exacerbated by requirements to report  the same information multiple times to accomplish agencies’ goals  through duplicative processes. For example, in addition to conducting  retrospective reviews, HHS’s Centers for Medicare and Medicaid  Services considers the need to eliminate or revise old, obsolete, or  duplicative regulations when it reviews its plans quarterly for regulatory  revisions or the need for new regulations. The agencies’ feedback on this  issue echoed the findings in our 2007 report. In that report, agencies cited  lack of time and resources as the most critical barriers to their ability to  conduct retrospective analyses because of the need to perform analyses  simultaneously with other mission-critical activities. This challenge of  effectively balancing agency efforts among competing priorities reinforces  the importance of the processes agencies use to select and prioritize  which regulations to target for retrospective analysis. The challenge is  magnified when considered in the context of a very extensive, and  growing, inventory of existing regulations from which agencies must  choose regulations to retrospectively assess within the limits of available  resources.
section_title: Agencies Reported Difficulty Obtaining Sufficient Data to Identify Improvements Attributed to Regulations and Other Barriers
paragraphs: Five of the nine agencies cited a lack of data to identify improvements  attributed to regulations as a barrier to conducting reviews. In particular,  agency officials cited challenges related to obtaining information from  industry, the cost of data from outside sources, and the potential to  transfer some data-collection burdens to the states, which face similar  resource constraints. For example, DOT officials said that for their agency  to have better data for some cost-benefit analyses, it would have to put  an additional burden on states to collect it, such as with police officers  filling out longer accident forms to more accurately code how someone  was injured. In addition, a DOT official said the data may be unavailable  or too expensive to obtain, and it often can be against a regulated entity’s  interest to provide certain information. DHS said most of the data needed  to conduct retrospective analyses of its regulations require original data  collection efforts to obtain potentially business-sensitive information from  industry. According to DHS officials, restrictions under the Paperwork  Reduction Act make it difficult to obtain information from a sufficient  number of regulated entities to produce meaningful and useful results. A  DOE official also said that additional data are constantly sought by its  scientists and economists, and many data sets are expensive and have  strict user requirements. Some agencies we spoke with noted, however,  that at times they have more readily accessible existing data that can help  inform their efforts to assess the performance of regulations. For  example, to aid in its enforcement responsibilities, Commerce’s Bureau of  Industry and Security (BIS) uses U.S. export data already collected by the  Census Bureau, which is another component within the department. One  BIS official said these data help the agency conduct cost-benefit analyses  and measure how their regulations are working. The barriers discussed  above are consistent with findings from our 2007 report, in which  agencies reported they lacked the information and data needed to  conduct reviews, such as baseline data for assessing regulations that  they promulgated many years ago.
Agencies also confirmed other barriers that affected their ability to  implement retrospective analyses, such as difficulty in (1) distinguishing  between which results were produced by the regulation versus other  factors, (2) resolving differences in opinion among internal or external  stakeholders on the results, and (3) using the results to make regulatory  decisions. For example, FTC reported that its ability to use analytical  results of retrospective reviews to make changes to regulations were  affected to a moderate extent by the difficulty in distinguishing between  which results were produced by the regulation and which stemmed from  other factors. In addition, FCC said its ability to use analytical results of  retrospective reviews for the same purpose was affected to a moderate  extent by the difficulty in resolving differences of opinion among internal  or external stakeholders. Few agencies identified strategies to overcome  the barriers discussed above. For example, the United States Patent and  Trademark Office, within the Department of Commerce, combined two  processes to leverage limited resources. The agency implemented  retrospective review practices at the same time they were issuing new  rules related to the Leahy Smith America Invents Act. In general, there  was ultimately no consensus among the roundtable participants about  strategies that might be more widely employed to effectively implement  retrospective review while overcoming these barriers. However, later in  this report we will discuss a related recommendation that we made to  OMB in June 2013 to help address this matter. We recommended that the  agency develop a detailed approach to examine such difficulties.
section_title: Opportunities Exist to Improve Linkages between Regulatory Reviews and Priority Goals
paragraphs: Retrospective reviews are just one way, among others such as program  evaluations, to help agencies evaluate their own performance. Given how  important regulations are to the missions of regulatory agencies, the  information generated from retrospective reviews can be leveraged to  directly inform agency annual performance goals and measurement.  Moreover, as discussed earlier, GPRAMA requires certain agencies, as  determined by OMB, to develop APGs every 2 years. In addition to  developing APGs, which are a subset of agencies’ broader performance  goals, agencies are required to identify the various regulations, as well as  federal organizations and other program activities (both within and  external to the agency) that contribute to each of their APGs, and review  progress on a quarterly basis. This information is to be published on a  governmentwide performance website, which OMB has implemented as  Performance.gov, as well as in agencies’ performance plans submitted to  OMB. In April 2013, we concluded that OMB’s review process does not  systematically check whether agencies have identified all relevant  contributors to APGs. Thus, we recommended that the Director of OMB  ensure that agencies adhere to OMB’s guidance for website updates by  providing complete information about regulations, among other activities,  that contribute to each APG. Our review of the criteria agencies used for  retrospective reviews shows they used multiple criteria, none of which  explicitly mention ties to performance goals. Although the executive  orders and guidance did not specifically direct agencies to include their  broader performance goals, such as APGs, as a criterion, agencies may  be missing a key opportunity to link these two activities.
subsections: section_title: Agencies Reported Mixed Experiences Linking Retrospective Analyses to APGs
paragraphs: Agencies’ responses about linkages between retrospective analyses and  APGs were mixed. Three agencies (DOE, DHS, and DOT) said their  recently completed, current, or future retrospective analyses were linked  to their APGs to either a moderate or great extent. To illustrate this  linkage, DOE said one of its APGs includes energy and costs savings as  well as greenhouse gas reduction through the issuance of new or revised  energy efficiency standards. As part of the agency’s process for issuing  revised standards, DOE said it considers retrospective analyses of the  existing standards to determine if greater efficiency can be achieved at  lower costs or with lower burden on the regulated community. Responses  from the other agencies varied, generally stating that either (1) there was  little to no connection between retrospective analyses and APGs or (2)  agency officials did not know or had no basis to judge whether such  linkages existed. In instances where multiple offices or bureaus within  an agency completed the questionnaire, the responses also varied. For  example, based on the responses we received from multiple offices within  some agencies, it appears there is a mixed understanding or awareness  of whether retrospective analyses are linked to APGs. This can be  explained, in part, by the division of responsibilities within an agency  where the office or staff responsible for carrying out retrospective  analyses differs from the staff responsible for carrying out broader agency  performance reviews.
Although only three agencies in our review reported moderate or great  linkages between their retrospective analyses and APGs, agency officials  in offices from seven of the nine agencies identified regulations as key  contributors to APGs. For example, while State reported little to no  linkage between its retrospective analyses and APGs, the agency  identified categories of regulations that contribute to three APGs,  including “strengthening diplomacy and development by leading through  civilian power.” State’s Bureau of Consular Affairs administers regulations  that directly support this goal. DHS identified specific categories of  regulations that contribute to its fiscal year 2013 APGs. For example,  security regulations directly support the agency’s APG to “strengthen  aviation security counterterrorism capabilities by using intelligence-driven  information and risk-based decisions.” Immigration and border  management regulations directly support DHS’ APG to “improve the  efficiency of the process to detain and remove criminal aliens.”
Most agencies cited linkages between retrospective analyses and agency  goals other than APGs, such as agency strategic objectives or  performance goals. However, the extent of these linkages varied. For  example, Commerce’s BIS said its retrospective analyses are tied to  other agency goals to a great extent. During our roundtable discussion,  one BIS official said the bureau’s regulations directly support a  department-level strategic objective. This linkage is documented in BIS’  fiscal year 2015 budget submission and Commerce’s fiscal year 2014- 2018 strategic plan, which shows how BIS regulations contribute toward  Commerce’s strategic objective to increase U.S. exports by broadening  and deepening the U.S. exporter base. In addition, FTC said its ongoing  retrospective review program ensures that FTC’s rules target deceptive  and unfair practices while also addressing changing market conditions  and avoiding undue burdens on legitimate business activity. Although  agencies identified examples where they established such linkages as  described above, they may not be able to do so in all instances. Some  agency officials we spoke with emphasized that regulations are only one  contributor, among others, to broader agency performance goals being  examined at higher levels within the agency. For example, one DOE  official indicated there may not necessarily be a one-for-one linkage  between an individual regulation and a single specific goal. Further, in a  March 2004 report, we found other difficulties regulatory agencies face in  measuring performance including (1) obtaining data to demonstrate  results, (2) accounting for factors outside of the agency’s control that  affect results, and (3) dealing with the long time periods often needed to  see results.
We asked agencies the extent to which they would like to receive more  assistance or guidance on incorporating retrospective analyses into  measuring and achieving agency goals, including APGs. Agencies  expressed varying degrees of interest in receiving such assistance. In the  cases where agencies received additional guidance on this matter, it was  provided either internally within the individual agency or from OMB’s  OIRA. Only one of the nine agencies reported receiving additional  assistance or guidance from other federal oversight offices and groups,  such as the Performance Improvement Council (PIC). Given the  common, long-standing difficulties agencies have continued to face in  measuring the performance of various types of federal programs and  activities, including regulations, we recommended in June 2013 that OMB  work with the PIC to develop a detailed approach to examine these  difficulties across agencies. This work would include identifying and  sharing any promising practices from agencies that have overcome  difficulties in measuring performance of these programs and activities.
OMB staff agreed with this recommendation. As of March 2014, PIC staff  told us they have taken initial steps to implement this recommendation  through a pilot effort focused on acquisition, which is related to one of the  areas highlighted in our June 2013 report. They said they plan to expand  the model to focus on other issues, such as regulations and grants.  According to PIC officials, it is too early in the process to have any  supporting documentation related to these plans. We will continue to  monitor progress on this effort.
As noted earlier, one of the potential purposes for conducting  retrospective analyses is to assess whether regulations, once  implemented, achieved the expected benefits at the expected costs.  However, the efforts of agencies included in our review to re-evaluate  original cost-benefit analyses associated with their regulations varied. In  response to our questionnaire, only three agencies reported that they  often conduct reviews of regulations to determine whether the regulations  are accomplishing the originally intended benefits at the expected cost.  The other six agencies generally reported that they rarely, if ever, do this  or did not know. Some agencies said they will sometimes revisit cost- benefit estimates to improve methods or models for conducting such  analyses in the future. However, according to a few agency officials,  they do not believe redoing past cost-benefit analyses is useful in the  context of making decisions about individual regulations looking forward.  In a broader context however, reexamining benefits and costs achieved  after a regulation is implemented could provide data useful for  performance reviews, including assessments of APGs, and is consistent  with regulatory executive orders and guidance. One of the principles in  Executive Order 13563 states that the regulatory system must measure,  and seek to improve, the actual results of regulatory requirements.  Subsequent guidance from the OIRA Administrator noted that this “points  to the need for empirical assessment of the consequences of rules.”  Broadening the focus to think about retrospective reviews as a contributor  to agency performance metrics related to APGs, rather than only in the  context of examining a single regulation, may encourage agencies to  revisit opportunities for more of this type of analysis.
section_title: Opportunities Exist to Better Link Retrospective Analyses and Agency Priority Goals
paragraphs: The focus of retrospective analyses differs from agency performance  reviews. Agencies’ retrospective analyses tended to examine regulations  on a micro level rather than their contribution to broader APGs.  Regulations are developed and implemented individually, and often re- examined individually. In addition, the impetus for reviews is often  statutory requirements, changes in the regulatory environment that affect  the rule, availability of agency resources, or executive order  requirements. The selection of which rule to review is typically tied to  criteria such as the number of complaints or comments from regulated  parties and the public, whether there have been changes in technology or  other conditions since the regulation was issued, the potential for burden  reduction, or the number of years since a regulation was promulgated or  last reviewed, rather than whether it will be subject to upcoming  performance goal discussions. The regulatory process and executive  orders outlining how to conduct retrospective reviews have helped to  shape the focus and timing of reviews.
In contrast, agencies’ performance reviews are designed to examine  measures that cut across programs and policy tools at a macro level. For  example, reviews of agency priority goals typically examine how agencies  are accomplishing broad mission goals that may be implemented by a  wide range of regulations, programs, grants, or other tools. As an  illustrative example, in DOT’s fiscal year 2014 performance plan, the  agency organized the descriptions of its planned performance—including  its agency priority goals—into broad themes under its strategic goals. As  shown in figure 3, DOT identified the regulations and enabling legislation  that contribute to its APG for aviation safety.
Given the specific focus of retrospective regulatory analyses, several  agency officials said the staff conducting the analyses often oversee  implementation of the regulation and are not involved in, or sometimes  aware of, performance discussions that are held at higher levels of the  agency. Similar to the earlier discussion about barriers, agency officials  said it may sometimes be difficult to isolate the effect of the regulation  versus other external factors contributing to the same outcome. Including  consideration of APGs as another part of planning and reporting  retrospective reviews could strengthen these linkages. Currently, the  selection of regulations to review is generally driven by a bottom-up  process, while performance reviews are driven by a top-down process.  While both processes have a useful purpose and structure, retrospective  reviews could help inform broader performance discussions if agencies  were to better link (1) the selected regulations to review, (2) the timing of  reviews, and (3) the information assessed. First, as previously noted, the  selection of rules to review is typically tied to specific criteria. Including  whether a regulation contributes to an APG expected to be reviewed by  management as one of the criteria for prioritizing reviews could help  retrospective analyses contribute useful information to the discussion of  whether and how goals are being accomplished. Second, better  alignment between the timing of when regulations are reviewed and when  APGs related to those regulations are examined by agency leadership  would help to ensure that evaluations are conducted when the information  would be most useful. Third, examining regulations that collectively  contribute to a specific APG could also improve the usefulness of  retrospective reviews. As illustrated in figure 3, multiple DOT regulations  contributed to the agency’s APG on aviation safety.
Greater attention to the cumulative effects of regulations would also help  agencies to better address certain key analytical and management  principles. First, this would be consistent with Executive Order 13610’s  direction that agencies consider the cumulative effects of their  regulations, among other priorities. In addition, as agencies implement  their APGs under GPRAMA, they are required to identify and assess the  various contributors to those goals, which can include multiple  regulations. Information generated from retrospective analyses can serve  as one source of valuable information about such cumulative effects.  Finally, this would help agencies to meet the internal control standard for  information and communications. Among other things, the standard states  that program managers need operational data to determine whether the  agency is meeting its strategic and annual performance goals for  accountability and for the effective and efficient use of resources.  Cumulative analysis could include joint reviews with other agencies as  appropriate. Four agencies we spoke with reported conducting joint  reviews with other agencies. For example, HHS officials said the Centers  for Disease Control and Prevention, a division within HHS, co-regulates a  shared list of select agents and toxins with the Animal Plant Health and  Inspection Service (APHIS) within the U.S. Department of Agriculture.  According to HHS, APHIS provides input during related retrospective  reviews of these regulations. This type of joint review can provide more  comprehensive and useful information about the collective impact of the  rules on performance and on the regulated community.
To better position agencies to conduct future retrospective analyses that  could contribute to performance reviews, it is important for agencies to  have considered in advance how they will evaluate their regulations. In  2007, we recommended that during the promulgation of certain new rules  (such as significant rules) agencies consider whether and how they will  measure the performance of the regulation, including how and when they  will collect, analyze, and report the data needed to conduct a  retrospective review. Consistent with that recommendation, OMB  guidance on implementing retrospective analysis states that future  regulations should be designed and written in ways that facilitate  evaluation of their consequences and thus promote retrospective  analyses. OMB asked agencies to identify in their retrospective review  plans what metrics the agencies will use to evaluate regulations after they  have been implemented. We asked OIRA the extent to which agencies  are implementing that guidance. One OIRA official said his office has  seen some evidence of agencies writing rules in ways that promote future  retrospective analyses. He cited a December 2013 final rule by  Commerce’s National Oceanic and Atmospheric Administration regarding  regulations to reduce the threat of ship collisions with North Atlantic right  whales. The final rule removed an expiration date (or “sunset clause”)  contained in the regulations but included provisions for the agency to  review the costs and benefits of the rule on a periodic basis, as required  by Executive Order 13563. The agency expects to conduct such a review  no later than 5 years from the publication of the final rule. The rule also  discussed data and metrics the agency could use as measures of  effectiveness. Ensuring that agencies build in such performance metrics  and a timeline for evaluating regulations after implementation would not  only help facilitate retrospective analyses, but also help to lay a  foundation to more closely tie retrospective analyses to reviews of  broader agency priority goals. Moreover, GPRAMA’s requirements for  agencies to identify and assess how their various programs and activities,  including regulations, contribute to agency performance goals and APGs  further underscore the need for agencies to take such action.
section_title: Conclusions
paragraphs: We have long advocated the potential usefulness to Congress, agencies,  and the public of conducting retrospective regulatory analyses while  recognizing the potential difficulties of doing so. From January 2011  through August 2013, executive agencies identified more than 650  initiatives (planned retrospective regulatory analyses). Of these initiatives,  246 resulted in agencies implementing some type of final action by  August 31, 2013. The majority of the completed reviews have led to  regulatory changes focused on outcomes such as reducing burden and  improving the effectiveness of regulations. While OMB has provided a  sample template for agencies to use for reporting on the progress and the  results of retrospective analyses, it could do more to enhance the  transparency and usefulness of the information provided to the public.  Although agencies posted their retrospective review plans online,  obtaining a comprehensive picture of agencies’ progress was difficult  when results were spread across multiple plans. In addition, consistently  providing links or citations to the supporting analyses and data, and  including more detail on the methodologies and key assumptions used to  estimate savings, would help Congress and the public to better  understand the basis for projected results.
Given how important regulations are to the missions of regulatory  agencies, the information generated from retrospective analyses can also  be leveraged to inform APGs, which are a subset of agencies’ broader  performance goals. The broader perspective and focus of performance  management and reviews may also enhance evaluation of the  effectiveness of regulations beyond what is possible through retrospective  analysis alone. This is especially true given the practical limits to the  number of analyses that could be done compared to the large and  growing inventory of regulations. We recognize that agencies conduct  retrospective analyses for many reasons. However, contributing to  evaluation of agency priority goals could also be among those reasons.  Therefore, agencies should at least consider whether opportunities exist  to inform performance reviews when selecting which regulations to  retrospectively review. Among the agencies that participated in our  roundtable, few identified examples where their retrospective review  analyses were linked to APGs. Steps that agencies can take to  strengthen the linkages between retrospective reviews and performance  reviews include identifying specific regulations that contribute to APGs  and including whether a regulation contributes to a key performance goal  expected to be reviewed by management as one of the criteria for  prioritizing retrospective reviews. Doing so may also provide greater  impetus for agencies to review the benefits and costs of existing  regulations as part of their retrospective analyses. Many of the agencies  we spoke with reported rarely doing so in the context of reviewing  individual regulations, and some agencies saw little value to the  investment. However, there would be more incentive to measure benefits  and costs if retrospective analyses were viewed in the broader context of  providing information on the actual, rather than projected, performance  and results of regulatory programs and agencies. This would also  reinforce existing guidance that agencies should identify how they will  measure the performance of new significant regulations when originally  published.
The executive branch has already incorporated many, but not all, of our  prior relevant recommendations into current guidance. Some of the  opportunities to improve reporting on outcomes of retrospective analyses  and strengthen linkages between these analyses and APGs could be  implemented through augmenting existing guidance. However, additional  opportunities for improvement depend in part on efforts to ensure that  agencies are consistently held accountable for implementing existing  guidance.
section_title: Recommendations for Executive Action
paragraphs: To improve agencies’ retrospective regulatory review processes and  reporting, and strengthen linkages between retrospective reviews and  agency performance management, we recommend the Director of the  Office of Management and Budget direct the Administrator of the Office of  Information and Regulatory Affairs to take the following three actions:  1.  Work with regulatory agencies to implement existing guidance, and  update guidance where needed, to improve the reporting of outcomes  in their retrospective regulatory review plans by taking actions such  as: publishing a link to updated plans, which list recent results and  anticipated outcomes, on the White House website;  submitting evidence that agencies listed updates of their plans on  their “Open Government” web pages; providing more comprehensive information on completed reviews  in agencies’ most recent plans and progress reports by (1)  ensuring the most recent published plan contains a complete  accounting of all completed reviews rather than expecting readers  to review multiple plans, and (2) including the supporting analysis  and data for results by listing a link or citation to the related  documentation.  2.  Ensure that the contributions made by regulations toward the  achievement of APGs are properly considered and improve how  retrospective regulatory reviews can be used to help inform  assessments of progress toward these APGs by directing in guidance  that agencies take such actions as:  identifying whether a regulation contributes to an APG expected to  be reviewed by management as one of the criteria for prioritizing  retrospective analyses and for the timing of these analyses; and once an agency prioritizes a retrospective analysis based, in part,  on its support of an APG, improving the usefulness of that  analysis by examining regulations that collectively contribute to  the goal in the scope of the review as appropriate.  3.  Ensure that OIRA, as part of its oversight role, monitor the extent to  which agencies have implemented the guidance on retrospective  regulatory review requirements outlined in the related executive  orders and confirm that agencies have identified how they will assess  the performance of regulations in the future.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Director of the Office of  Management and Budget, the Secretaries of Commerce, Energy, Health  and Human Services, Homeland Security, State, and Transportation, as  well as the Administrator of the Environmental Protection Agency and the  Commissioners of the Federal Communications Commission and Federal  Trade Commission.
In oral comments received on April 3, 2014, staff from OMB’s Office of  Information and Regulatory Affairs (OIRA) generally agreed with the  recommendations in this report. In response to our first recommendation,  OIRA staff generally agreed but said they were open to taking actions  other than updating the White House website in light of a decision to  discontinue posting agencies’ updates as individual agencies assume  primary responsibility for posting updates on their “Open Government”  web pages. OIRA staff were supportive of identifying ways to improve  agencies’ posting and reporting of retrospective analyses and progress  updates. In response to our second recommendation, OIRA staff agreed  and said the agency is currently working on strategies to help facilitate  agencies’ ability to consider and improve how retrospective reviews can  be used to help inform APGs, and otherwise be better integrated into  agency strategic decision making. In response to our third  recommendation, OIRA staff agreed, emphasizing that this remains a  priority and therefore they would continue to monitor the extent to which  agencies implement the guidance on retrospective regulatory review  requirements. Further, OIRA staff said that as part of its review of  agencies’ rules, unified agendas, and regulatory plans, OIRA will continue  to encourage agencies to identify beforehand how they will evaluate the  effectiveness of a regulation after it has been put in place. OIRA staff also  provided technical comments and clarifications, which we incorporated as  appropriate.
The Departments of Commerce, Homeland Security, Health and Human  Services, and State, the Environmental Protection Agency, and the  Federal Trade Commission provided technical comments, which we  incorporated as appropriate.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to the Director  of the Office of Management and Budget, the Secretaries of Commerce,  Energy, Health and Human Services, Homeland Security, State, and  Transportation, as well as the Administrator of the Environmental  Protection Agency and the Commissioners of the Federal  Communications Commission and Federal Trade Commission. We are  also sending copies of this report to relevant congressional committees.  In addition, the report is available at no charge on GAO’s website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-6806 or sagerm@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who have made contributions to this report  are listed in appendix II.
section_title: Appendix I: Key Retrospective Analysis Requirements Listed in Executive Orders and Memorandums Issued Since January 2011
paragraphs: Table 2 provides a summary of the various executive orders and related  memorandums that were issued since January 2011 related to agencies’  retrospective regulatory review plans and analyses. For presentation  purposes, we highlighted the elements of these directives that relate to  (1) planning retrospective analyses, (2) implementing these plans and  analyses, and (3) reporting the results of completed analyses.
section_title: Appendix II: Staff Contact and Acknowledgments
subsections: section_title: GAO Contact
paragraphs: Michelle Sager, (202) 512-6806 or sagerm@gao.gov.
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Tim Bober, Assistant Director,  Leah Q. Nash, Latesha Love, Lou V.B. Smith, and Wesley Sholtes made  major contributions to this report. Tom Beall, Tim Guinane, Andrea  Levine, Benjamin T. Licht, Donna Miller, Cindy Saunders, Stephanie  Shipman, and Stewart Small also made key contributions to this report.