Managing for Results: Implementation of GPRA Modernization Act Has Yielded Mixed Progress in Addressing Pressing Governance Challenges
Published: 2015-09-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Effective implementation of GPRAMA can help address significant and long-standing budget, management, and performance challenges the federal government faces. This report is the latest in a series of GAO work in response to a statutory provision to review GPRAMA implementation. It examines how implementation has affected progress in addressing (1) crosscutting issues; (2) the extent to which performance information is useful and used; (3) alignment of daily operations with results; and (4) communication of performance information.
To address these objectives, GAO reviewed GPRAMA and related guidance, recent and ongoing work related to these four areas, and the status of the 69 recommendations made to OMB and agencies as part of GAO's prior work on GPRAMA implementation. GAO also interviewed OMB and Performance Improvement Council (PIC) staff. GAO included work issued since June 2013, when GAO issued the previous summary report on GPRAMA's initial implementation.
GAO is not making any new recommendations in this report. OMB and agencies generally agreed with the 69 related recommendations GAO made between 2012, when GAO issued its first report in response to the statutory provision in GPRAMA, and now, but most recommendations (about 80 percent) have not yet been implemented. GAO shared a draft of this report with OMB. OMB staff generally agreed with the information presented in the report and provided technical clarifications, which GAO incorporated as appropriate.
section_title: What GAO Found
paragraphs: GAO's work over the past 2 years shows that implementation of the GPRA Modernization Act (GPRAMA) continues to be uneven, with varying effects on agencies' performance management. Some progress has been made in areas where GAO has made prior recommendations; however, GAO has continued to identify a range of long-standing challenges in the four areas discussed below.
The executive branch still needs to take additional actions to address crosscutting issues, but the Office of Management and Budget (OMB) has increased emphasis on governance of cross-agency priority (CAP) goals. For example, OMB has issued new guidance and governance for CAP goals, which cover areas where cross-agency collaboration is needed. However, more effective implementation of GPRAMA requirements, such as the requirement that agencies develop inventories of their programs, would help address crosscutting issues by providing decision makers with comprehensive program and funding information.
Ensuring performance information is useful and used by managers remains a challenge, but OMB and agencies are implementing processes that may lead to improvements. Agencies continue to have problems effectively using performance information. GAO's analysis indicates that agencies' reported use of performance information generally did not improve between 2007 and 2013. However, as OMB and agencies continue to implement data-driven and strategic review processes, the use of performance information should improve. For example, GAO found that nearly all of the 22 agencies that reported holding in-person data-driven reviews of agency priority goals (APG)—which represent agencies' highest priorities—said they use the reviews to assess progress on APGs and that they have had a positive effect on goal progress. Similarly, some agencies have increased their use of or enhanced their efforts to improve their capacity to use other evidence-based tools, such as program evaluations.
Agencies continue to face challenges linking individual and agency performance to results . GPRAMA provisions, such as the requirement that agencies identify a goal leader responsible for APG achievement, promote linkages between individual performance and agency results. GAO has recommended that agencies strengthen some mechanisms that can promote this connection, such as through Senior Executive Service performance plans. Agencies also need to take additional actions to address GAO recommendations on measuring performance in a number of areas, such as customer service.
OMB and agencies have not clearly communicated reliable and complete financial and performance information, but more effective implementation of GPRAMA requirements would improve transparency. GPRAMA requirements for reporting on the quality of performance information have the potential to improve the transparency of that information. While OMB has updated some of its guidance, improved reporting on the quality of information is not expected from the agencies until the fiscal year 2016 and 2017 reporting cycle.

section_title: Letter
paragraphs: The federal government is one of the world’s largest and most complex  entities, with about $3.5 trillion in outlays in fiscal year 2014 funding a  vast array of programs and operations. It faces a number of significant  budget, management, and performance challenges as it seeks to achieve  diverse and complex results. For example, since 2011, our series of  annual reports has identified more than 200 areas in which actions are  needed to address fragmentation, overlap, and duplication; achieve other  cost savings; or enhance revenue. In addition, weaknesses in  management capacity, both government-wide and in individual agencies,  impair efficient and effective government operations. Since 1990, our  high-risk list has identified 57 areas that are vulnerable to fraud, waste,  abuse, or in need of broad-based transformation, of which 32 remain on  the current list.
Addressing these challenges will require tough choices in setting priorities  and reforming programs and management practices. We previously  reported that the performance planning and reporting framework originally  put into place by the Government Performance Results Act of 1993  (GPRA), and significantly enhanced by the GPRA Modernization Act of  2010 (GPRAMA), provides important tools that can help decision makers  address challenges facing the federal government.  Full and effective  implementation of GPRAMA will be also be instrumental in addressing  these pressing governance issues in anticipation of the transition to the  next presidential administration in 2017.
Congress included a statutory provision for GAO to evaluate and report  on (1) how implementation of GPRAMA is affecting performance  management at the Chief Financial Officers (CFO) Act agencies,  including whether performance management is being used to improve the  efficiency and effectiveness of agency programs; and (2) crosscutting  goal implementation, not later than September 30, 2015. Based on our  issued and ongoing work since our previous assessment of the initial  implementation of GPRAMA required by the act in June 2013, for this  report we reviewed progress in four key areas: (1) addressing cross- cutting efforts; (2) ensuring performance information is useful and used;  (3) aligning daily operations with results; and (4) communicating  performance information. We also reviewed the status of  recommendations made to the Office of Management and Budget (OMB)  and agencies as part of our issued work on GPRAMA from 2012 through  September 2015. To perform this work, we reviewed GPRAMA, OMB  guidance, and our recent and ongoing work related to our four key areas,  including the status of our recommendations. We also interviewed OMB  and Performance Improvement Council (PIC) staff.
Our recent work under GPRAMA, both ongoing and issued since June  2013, covered the 24 CFO Act agencies and the Army Corps of  Engineers-Civil Works. Eight of our 12 reviews that are the basis of this  report used selected agencies as case illustrations. Half of the 12 reports  included government-wide reviews, some involving surveys of all or most  of the CFO Act agencies. Figure 1 shows the agencies covered in our  recent and ongoing work under GPRAMA.
This report also includes some results from our ongoing work on cross- agency priority (CAP) goals and on major management challenges, on  which we plan to issue reports at the end of 2015. For our ongoing work  on CAP goals, we selected 7 of the 15 CAP goals established in March  2014 for review, interviewed agency officials with responsibility for  implementing these goals, and reviewed relevant guidance and  documentation. For our ongoing work on the progress agencies are  making in using GPRAMA to address their major management  challenges, we evaluated agency reporting against GPRAMA  requirements and OMB guidance and interviewed OMB staff and relevant  agency officials. Appendix I provides additional information about our  objectives, scope, and methodology.
We conducted this performance audit from April 2015 to September 2015  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: GPRAMA is a significant enhancement of GPRA, which was the  centerpiece of a statutory framework that Congress put in place during  the 1990s to help resolve long-standing performance and management  problems in the federal government and provide greater accountability for  results. GPRAMA was likewise intended to address a number of federal  performance management challenges, including focusing attention on  crosscutting issues, enhancing the use and usefulness of performance  information, increasing transparency, and ensuring leadership  commitment and attention to improving performance. Our June 2013  report assessing the initial government-wide implementation of GPRAMA  described some of the important steps OMB and agencies had taken to  implement key provisions of GPRAMA. These included developing  agency-level and government-wide goals, designating officials to key  leadership roles, and using the Performance Improvement Council (PIC)  to facilitate the exchange of information to strengthen agency  performance management.
GPRAMA revises existing provisions and adds new requirements,  including the following:  GPRAMA includes requirements that OMB and agencies establish  different types of government-wide and agency-level performance goals.  These include:
Government-wide: cross-agency priority (CAP) goals. OMB is  required to coordinate with agencies to establish federal government  priority goals—otherwise referred to as CAP goals—that include  outcome-oriented goals covering a limited number of policy areas as  well as goals for management improvements needed across the  government. The act also requires that OMB—with agencies— develop annual federal government performance plans to, among  other things, define the level of performance to be achieved through  each of the CAP goals. OMB established the first set of CAP goals for  a 2-year interim period in February 2012. In March 2014, OMB  identified the next set of CAP goals, which it is to update every 4  years.
Agency-level: agency priority goals (APG). Every 2 years,  GPRAMA requires the heads of certain agencies, in consultation with  OMB, to identify a subset of agency goals to be identified as APGs.  These goals are to reflect the highest priorities of each of these  agencies, and to be informed by the CAP goals as well as  consultations with relevant congressional committees and other  interested parties. Twenty-three agencies identified a total of 91 APGs  covering fiscal years 2014 through 2015.
GPRAMA provided a statutory basis for selected senior leadership  positions that had been created by executive orders, presidential  memorandums, or OMB guidance. GPRAMA established these  positions in law, provided responsibilities for various aspects of  performance improvement, and elevated some of them as described  below.
Chief operating officer (COO). The deputy agency head, or  equivalent, is designated COO, with overall responsibility for  improving agency management and performance.
Performance improvement officer (PIO). Agencies are required to  designate a senior executive within the agency as PIO, who reports  directly to the COO and has responsibilities to assist the agency head  and COO with performance management activities.
Goal leaders. GPRAMA requires that goal leaders be designated for  CAP goals and APGs and OMB guidance requires goal leaders be  designated for strategic objectives. They are designated for CAP  goals, strategic objectives, and APGs. CAP goals have at least two  goal leaders—one from the Executive Office of the President and the  other from a key responsible agency. APGs have a goal leader, and  OMB guidance directs agencies to designate a deputy goal leader to  support the goal leader.
GPRAMA also established the PIC in law and included additional  responsibilities. Originally created by a 2007 executive order, the PIC is  charged with assisting OMB to improve the performance of the federal  government and achieve the CAP goals. Among its other  responsibilities, the PIC is to facilitate the exchange among agencies of  useful performance improvement practices and work to resolve  government-wide or crosscutting performance issues. The PIC is chaired  by the Deputy Director for Management at OMB and includes agency  PIOs from each of the 24 CFO Act agencies as well as other PIOs and  individuals designated by the chair.
GPRAMA and related OMB guidance require the regular review of  progress in achieving goals and objectives through performance reviews.
Strategic reviews. OMB’s 2012 guidance implementing GPRAMA  established a strategic review process in which agencies, beginning in  2014, were to conduct leadership-driven, annual reviews of their  progress toward achieving each strategic objective—the outcome or  impact the agency is intending to achieve through its various  programs and initiatives—established in their strategic plans (and  updated in their annual performance plans).
Data-driven reviews. Data-driven performance reviews are regularly  scheduled—at least quarterly—structured meetings used by  organizational leaders and managers to review and analyze data on  progress toward key performance goals and other management- improvement priorities. For each APG, GPRAMA requires agencies  to conduct reviews to assess progress toward the goal and risk of not  meeting it, and develop strategies to improve performance, as  needed. These reviews are to be led by the agency head and COO,  with the support of the PIO, and include relevant goal leaders.  Coordination with relevant parties both within and outside the agency  that contribute to goal accomplishment is also required. GPRAMA  also requires that the Director of OMB, with the support of the PIC,  review progress toward each CAP goal with the appropriate lead  government official at least quarterly. Specifically, these reviews  should examine the progress made over the most recent quarter,  overall trends, the likelihood of meeting the planned level of  performance and, if necessary, strategies to improve performance.
GPRAMA includes several provisions related to reporting of performance  information.
Performance.gov. OMB is required to develop a single, government- wide performance website to communicate government-wide and  agency performance information. The website—implemented by  OMB as Performance.gov—is required to make available information  on APGs and CAP goals, updated on a quarterly basis; agency  strategic plans, annual performance plans, and annual performance  reports; and an inventory of all federal programs.
Program inventory. OMB is required to make publicly available, on a  central government-wide website, a list of all federal programs  identified by agencies, along with related budget and performance  information.
Performance information quality. Agencies are required to describe  how they are ensuring the accuracy and reliability of the data used to  measure progress toward APGs and performance goals, including an  identification of the following five areas:  o  The means used to verify and validate ;  o the sources for the data;  o  the level of accuracy required for the intended use of the  data;  o  any limitations to the data at the required level of accuracy;  o  how the agency will compensate for such limitations (if  needed) to reach the required level of accuracy.
Agencies are required to provide information to OMB that addresses all  five requirements for each of their APGs for publication on  Performance.gov. Agencies also must address all five requirements for  performance goals in their performance plans and reports.
Major management challenges. Agencies are required to address  major management challenges in their performance plans. These  challenges may include programs or management functions that have  greater vulnerability to fraud, waste, abuse, and mismanagement,  such as those issues included in our high-risk list or identified by  inspectors general, where a failure to perform well could seriously  affect an agency’s ability to achieve its mission or goals.
The concepts described above and their relationships to each other are  represented in figure 2, which summarizes them and highlights areas in  which our recent work has focused.
section_title: The Executive Branch Needs to Take Additional Actions to Address Crosscutting Issues, but OMB Has Increased Emphasis on Governance of Cross-Agency Priority Goals
subsections: section_title: OMB and Agencies Continue to Miss Opportunities to Address Crosscutting Issues
paragraphs: Many of the meaningful results that the federal government seeks to  achieve, such as those related to protecting the environment, promoting  public health, and providing homeland security, require the coordinated  efforts of more than one federal agency, level of government, or sector.  Even with sustained leadership, crosscutting issues are difficult to  address because they may require agencies and Congress to reexamine  (within and across various mission areas) the fundamental structure,  operation, funding, and performance of a number of long-standing federal  programs or activities. Collaboration and improved working relationships  across agencies are critical tools for addressing the issues of  fragmentation, overlap, and duplication our recent work has highlighted.  Additionally, they are fundamental to addressing many of the issues that  we have designated as high risk due to their vulnerabilities to fraud,  waste, abuse, and mismanagement or most in need of broad-based  transformation. We have found that resolving many of these issues  requires better collaboration among agencies, levels of government, and  sectors.
For more than two decades, we have reported on agencies’ missed  opportunities for improved collaboration through the effective  implementation of GPRA and, more recently, GPRAMA. Now, more than  20 years since GPRA’s passage, our work continues to demonstrate that  the needed collaboration is not sufficiently widespread. The examples in  the textbox below show areas from our high-risk list–improving and  modernizing federal disability programs and improving federal oversight  of food safety–that demonstrate the need for greater collaboration on  crosscutting issues.
Examples of 2015 High-Risk Areas Demonstrating the Continued  Need to Address Crosscutting Issues
Federal Disability Programs Remain Fragmented and a High-Risk  Federal disability programs across government remain fragmented  and in need of modernization. Numerous federal programs provide a  patchwork of services and supports to people with disabilities and  work independently without a unified vision and strategy or set of  goals to guide their outcomes. Our 2015 update to our High-Risk  Series found that progress in improving and modernizing disability  programs has been mixed. OMB has made some progress toward  enhancing coordination across programs that support employment for  people with disabilities, but it has not established a larger vision for  disability programs that include appropriate government-wide goals  and strategies for achieving those goals. OMB needs a government- wide action plan that describes how federal agencies will improve  coordination and set measurable goals that support employment for  people with disabilities beyond the public sector. Such a plan should  identify additional opportunities to build capacity and leverage existing  government resources. Continued planning, management focus, and  coordination can improve and modernize federal disability programs.
Federal Food Safety Is a High-Risk Area and in Need of Improved  Foodborne illness is a common, costly, yet largely preventable public  health concern. According to the Centers for Disease Control and  Prevention, each year nearly 50 million people in the United States  get sick and roughly 3,000 die due to foodborne illness. For more than  a decade, we have reported on the fragmented federal food safety  system and we added federal oversight of food safety to our high-risk  areas because of risks to the economy and public health and safety.  In December 2014 we reported that the Department of Health and  Human Services (HHS) and the U.S. Department of Agriculture  (USDA) have taken steps to implement GPRAMA requirements, but  could more fully address crosscutting food safety efforts. HHS and  USDA vary in the amount of detail they provide on their crosscutting  food safety efforts and they do not include several relevant  crosscutting efforts in their strategic and performance planning  documents. HHS and USDA have mechanisms in place to facilitate  interagency coordination on food safety that focus on specific issues,  but they do not provide for broad-based, centralized collaboration. A  centralized collaborative mechanism on food safety is important to  foster effective interagency collaboration and enhance food safety  oversight. We recommended that HHS and USDA build upon their  efforts to implement GPRAMA requirements to fully address  crosscutting food safety efforts. We asked Congress to consider  directing OMB to develop a government-wide food safety performance  plan and formalize the Food Safety Working Group through statute to  help ensure sustained leadership across food safety agencies over  time. HHS and USDA agreed with our recommendation and in  February 2015, HHS updated its strategic plan to more fully describe  how it is working with other agencies to achieve its food-safety-related  goals and objectives. As of August 2015, our recommendation to  USDA and matters for congressional consideration remain  unimplemented.
Our annual reports on areas where opportunities exist for executive  branch agencies or Congress to reduce, eliminate, or better manage  fragmentation, overlap, or duplication; to achieve cost savings; or to  enhance revenue have also included areas in which greater collaboration  is needed to address crosscutting issues. See textbox below.
Examples of Crosscutting Issues Identified in GAO’s 2015 Annual  Report on Fragmentation, Overlap, and Duplication
Fragmentation, Overlap, and Potential for Duplication Exists  among Nonemergency Medical Transportation Programs  Access to transportation services is essential for millions of Americans  to fully participate in society and access human services, including  medical care. Our April 2015 report on opportunities to reduce  fragmentation, overlap, and duplication reported that 42 programs  across six federal departments provide funding for nonemergency  medical transportation (NEMT) to individuals who cannot provide their  own transportation due to age, disability, or income constraints.  Coordination of NEMT programs at the federal level is limited and  there is fragmentation, overlap, and potential for duplication across  these programs. An interagency coordinating council was developed  to enhance federal, state, and local coordination activities, and it has  taken some actions to address program coordination. However, the  council has provided limited leadership and has not convened since  2008. To improve efficiency, we recommended that the Department of  Transportation, which chairs the council, take steps to enhance  coordination among the programs that provide NEMT. The  department agreed that more work is needed, and said that the  Federal Transit Administration is asking its technical assistance  centers to assist in developing responses to NEMT challenges. In  addition, as of June 1, 2015, the Federal Transit Administration  reported working to develop a new 2-year strategy for addressing  NEMT coordination among federal agencies and plans to develop and  propose a cost-sharing model that can be applied to federal programs  that provide funding for NEMT.
Strengthened Coordination Could Increase Efficiency and  Effectiveness of Consumer Product Safety Oversight  The oversight of consumer product safety is a complex system  involving a number of federal agencies. However, as our April 2015  report on opportunities to reduce fragmentation, overlap, and  duplication highlighted, oversight of consumer product safety is  fragmented across agencies, overlaps jurisdictions, or is unclear for  certain products. In some cases agencies regulate different  components of or carry out different regulatory activities for the same  product. Agencies reported that they collaborate to address specific  consumer product safety topics. For example, officials from the Coast  Guard, which regulates safety standards for recreational boats, said  they work informally with the Consumer Product Safety Commission  when the need arises. However, we did not identify a formal  mechanism for addressing such issues more comprehensively, and  no single entity or mechanism exists to help the agencies that  collectively oversee consumer product safety. Without this, agencies  may miss opportunities to leverage resources and address  challenges, including those related to fragmentation and overlap. In  response to our recommendation that the Coast Guard and the  Consumer Product Safety Commission establish a formal coordination  mechanism, in May 2015 the two agencies signed a formal policy  document establishing such a mechanism. We also recommended  that Congress should establish a formal collaboration mechanism to  address comprehensive oversight and inefficiencies related to  fragmentation and overlap. As of August 2015, no formal collaboration  mechanism had been established.
The textbox below shows additional examples of areas in which we have  identified the need for additional work to address crosscutting issues.
Examples from GAO’s Work from 2013-2015 of Continued  Challenges in Addressing Crosscutting Issues
Additional Leadership Needed to Achieve Interagency Efforts for  Department of Agriculture and Department of Health and Human  Services (HHS) veterinarians perform critical work for public and  animal health and for emergency response to economically  devastating or highly contagious animal diseases. In May 2015, we  reported that the Office of Personnel Management (OPM) and other  federal agencies have taken steps toward achieving the goals outlined  in OPM’s government-wide strategic plan for the veterinarian  workforce, primarily through an interagency group OPM created.  However, in each of the three goals, the group did not follow through  on next steps and made limited progress. According to OPM officials,  the group did not consistently monitor progress toward goals in part  because it did not have sufficient leadership support from participating  agencies. OPM agreed with our recommendation that it obtain  leadership support for achieving its goals, and stated that it designed  and will aid in establishing a Veterinary Medical Officer Executive  Steering Committee that will, among other things, provide leadership  and ensure progress toward stated goals.
Federal Strategy Needed to Ensure Efficient and Effective  Delivery of Services for Older Adults  In May 2015, we reported that five federal agencies across four  departments had one or more programs that operate within a system  of home and community-based services (HCBS) and related supports  that older adults often require to live as independently as possible in  their homes and communities. The Older Americans Act of 1965  requires the Administration on Aging, within HHS, to facilitate  collaboration among federal agencies; however, the five agencies that  fund these services and supports do so, for the most part,  independently. To help ensure that agencies’ resources for HCBS and  supports are used efficiently and effectively, we recommended that  HHS facilitate development of a cross-agency federal strategy. HHS  agreed with our recommendation.
While much of our recent work has focused on the need for improved  collaboration to address crosscutting issues, we have also reported on  areas, including one high-risk area, in which agencies have made  progress or are generally effectively coordinating. The text box below  discusses two of these examples.
Examples from GAO’s Work from 2013-2015 of Areas in Which  Agencies Are Doing Well or Making Progress in Addressing  Crosscutting Issues
Coordination of DOD’s and NNSA’s Nuclear Weapons Stockpile  Responsibilities Is Generally Consistent with Key Practices  The Nuclear Weapons Council (Council) serves as the focal point of  Department of Defense (DOD) and National Nuclear Security  Administration (NNSA) interagency activities to maintain the U.S.  nuclear weapons stockpile. In May 2015, we reported that the  Council’s actions to coordinate DOD’s and NNSA’s nuclear weapons  stockpile responsibilities are generally consistent with most of the key  practices we have identified for collaborating across agency  boundaries. For example, according to Council documents, the  Council and its support committees meet on a regular basis to  monitor, evaluate, and report on nuclear weapons stockpile issues.  These meetings include periodic oversight briefings on nuclear  weapon refurbishment programs. We made recommendations to the  Secretaries of Defense and Energy to address two areas in which  actions could be enhanced: (1) having up-to-date, written agreements  and guidance that establish compatible policies, procedures, and  other means to operate across agency boundaries and defines roles  and responsibilities and (2) regularly including all relevant participants.  The departments generally agreed with our recommendations.
Progress Made on High-Risk Area of Sharing and Managing  The federal government has made significant progress in promoting  the sharing of information on terrorist threats, an area we designated  as high risk in 2005. In February 2015, we reported that significant  progress was made in this area by developing a more structured  approach to achieving the Information Sharing Environment  (Environment) and by defining the highest priority initiatives to  accomplish. In December 2012, the President signed the National  Strategy for Information Sharing and Safeguarding (Strategy), which  provides guidance on the implementation of policies, standards, and  technologies that promote secure and responsible national security  information sharing. In 2013, in response to the strategy, the Program  Manager for the Environment released the Strategic Implementation  Plan for the National Strategy for Information Sharing and  Safeguarding (Implementation Plan). The Implementation Plan  provides a roadmap for the implementation of the priority objectives in  the Strategy, assigns stewards to coordinate each priority objective,  and provides time frames and milestones for achieving the outcomes  in each objective. The steward is responsible for ensuring that  participating agencies communicate and collaborate to complete the  objective, while also raising to senior management any issues that  might hinder progress. Although progress has been made, more work  remains to be done to fully address the issues identified in this high- risk area.
section_title: More Effective Implementation of GPRAMA and the DATA Act Would Help Address Crosscutting Issues
paragraphs: If fully and effectively implemented, GPRAMA and the Digital  Accountability and Transparency Act of 2014 (DATA Act) hold promise for  helping to address crosscutting issues. For example, GPRAMA  establishes a framework aimed at taking a more crosscutting and  integrated approach to focusing on results and improving government  performance. Effective implementation of GPRAMA could help clarify  desired outcomes, address program performance spanning multiple  organizations, and facilitate future actions to reduce, eliminate, or better  manage fragmentation, overlap, and duplication. The DATA Act also  offers the potential to help address crosscutting issues, as it requires  agencies to publicly report information about any funding made available  to, or expended by, an agency. These actions would allow executive  branch agencies and Congress to accurately measure the costs and  magnitude of federal investments. As we have previously reported, the  DATA Act holds great promise for improving the efficiency and  effectiveness of the federal government and for addressing persistent  government management challenges. For example, as our annual  reports on fragmentation, overlap, and duplication have highlighted, a  complete picture of federal programs, along with related funding and  performance information, is critical for addressing these issues.
Data-driven reviews. Data-driven reviews have had a positive effect on  collaboration among officials within agencies, but agencies are still  missing opportunities to include stakeholders from other federal agencies  and thus promote collaboration across agencies. Specifically, in our July  2015 report on data-driven reviews, we found that 21 of the 22 agencies  we surveyed that reported holding in-person data-driven reviews said that  their data-driven reviews have had a positive effect on collaboration  among officials from different offices or programs within the agency.
Despite the positive effects of reviews on internal collaboration, most  agencies reported that relevant contributors from other federal agencies  did not participate in their reviews. This situation has not changed since  our July 2014 report on the role of the agency priority goal leader, in  which we found that some goal leaders reported that goal contributors  from other federal agencies, and even different components within the  same federal agency, were not included in their data-driven reviews.
As we previously reported in 2013, failing to include all goal contributors  may lead to missed opportunities to have all the relevant parties apply  their knowledge of the issues and participate in developing solutions to  performance problems. As a result, in that 2013 report, we  recommended that OMB work with the PIC and other relevant groups to  identify and share promising practices to help agencies extend their  performance reviews to include, as relevant, representatives from outside  organizations that contribute to achieving their agency performance  goals. As of June 2015, OMB had not taken action in response to this  recommendation. OMB staff said that while agencies have found that at  times it is useful to engage external stakeholders in improving program  delivery, officials view data-driven reviews as internal agency  management meetings and believe it would not always be appropriate to  regularly include external representatives. We continue to believe that  more active involvement from external contributors is needed, as  appropriate, and continue to urge OMB to implement our recommended  actions.
Strategic reviews. Effective implementation of strategic reviews could  help identify opportunities to reduce, eliminate, or better manage  instances of fragmentation, overlap, and duplication because agencies  are to identify the various organizations, program activities, regulations,  tax expenditures, policies, and other activities that contribute to each  objective, both within and outside the agency. Where progress in  achieving an objective is lagging, the reviews are intended to identify  strategies for improvement, such as strengthening collaboration to better  address crosscutting challenges or using evidence to identify and  implement more effective program designs. If successfully implemented  in a way that is open, inclusive, and transparent—to Congress, delivery  partners, and a full range of stakeholders—this approach could help  decision makers assess the relative contributions of various programs to  a given objective.
Successful strategic reviews could also help decision makers identify and  assess the interplay of public policy tools that are being used to ensure  that those tools are effective and mutually reinforcing and that results are  being efficiently achieved. To that end, in July 2015 we reported on seven  practices that can help ensure agencies conduct effective strategic  reviews. These practices include identifying the various strategies and  other factors that influence outcomes and determining which are most  important, identifying key stakeholders to participate in the review, and  assessing the effectiveness in achieving strategic objectives and  identifying actions to improve implementation and impact.
Program inventories. One of the GPRAMA provisions that has the  potential to help in addressing crosscutting issues is the requirement that  agencies develop inventories of their programs, though in October 2014  we reported that several issues limit the usefulness of the inventories.  As our prior work has highlighted, creating a comprehensive list, or  program inventory, of federal programs, along with related performance  and funding information, could provide decision makers with critical  information that could be used to better address crosscutting issues.  However, in developing the inventory OMB allowed agencies to define  their programs using different approaches, but within a broad definition of  what constituted a program consistent with several characteristics.  Moreover, OMB’s guidance presents five possible approaches agencies  could take to define their programs and notes that agencies could use  one or more of those approaches in doing so. As a result, we found that  the use of inconsistent approaches by agencies to define their programs  limits the comparability of programs within agencies as well as  government-wide.
To illustrate the shortcomings of the inventory, in our report on program  inventories we compared relevant agencies’ inventories for various  science, technology, engineering, and mathematics education and  nuclear nonproliferation programs to programs identified in our past  work. We were unable to identify in the inventories a large majority of  the programs previously identified in our work: 9 of the 179 programs  matched exactly and 51 others were identified based on program  descriptions. According to OMB staff, agencies used different approaches  for valid and legitimate reasons and a one-size-fits-all approach would not  work for all agency inventories. While this may be true, OMB could do  more to direct agencies to find common ground on similar programs. One  of OMB’s stated purposes for the inventories is to facilitate coordination  across programs that contribute to similar outcomes. However, as we  discovered through our interviews with agency officials involved with the  inventory efforts, none of the agencies sought input from other agencies  on how they defined and identified their programs. We concluded that if  agencies worked together to more consistently define their programs, it  could also help them identify where they have programs that contribute to  similar outcomes, and therefore have opportunities to collaborate.
We made several recommendations to OMB aimed at presenting a more  coherent picture of all federal programs in agency inventories, but OMB  has not yet taken action to address these items. OMB planned to  publish updated inventories in May 2014. However, OMB put the plans for  updating the inventories on indefinite hold and agencies have not  published updated inventories with program-level budget information, in  part due to the enactment of the DATA Act. OMB staff told us that they  are considering how implementation of DATA Act requirements can be  tied to the program inventories. Agency reporting for both sets of  requirements is web-based, which could more easily enable linkages  between the two or facilitate incorporating information from each other.  The House and Senate versions of the Taxpayers Right-to-Know Act  would require that program inventories also include, to the extent  practicable, financial information required to be reported under the DATA  Act for each program activity. If enacted, the Taxpayers Right-to-Know  Act could result in detailed financial and performance information for  federal programs, all in one place.
In our July 2015 testimony on the implementation of the DATA Act, we  recommended that OMB accelerate efforts to determine how best to  merge DATA Act purposes and requirements with the GPRAMA  requirement to produce a federal program inventory. OMB and Treasury  did not comment on this recommendation. However, the Acting Deputy  Director for Management and Controller at OMB stated at the July 2015  hearing that, because the staff that would be involved in working on the  program inventories were heavily involved in DATA Act implementation,  he would not expect an update of the program inventories to happen  before May 2017.
section_title: OMB Has Increased Emphasis on Crosscutting Issues through Cross- Agency Priority Goal Guidance and Governance
subsections: section_title: Cross-Agency Priority Goals Focus Attention on Crosscutting Issues
paragraphs: GPRAMA’s provisions for establishing and managing achievement toward  cross-agency priority (CAP) goals make up another area in which the act  offers the potential to address crosscutting issues. CAP goals, which  GPRAMA requires OMB to develop in coordination with agencies, are  intended to cover areas where increased cross-agency collaboration is  needed to improve progress toward shared, complex policy or  management objectives, such as improving our nation’s cybersecurity.  OMB established the first set of CAP goals for a 2-year interim period in  February 2012. In March 2014, OMB released the next set of CAP goals,  which it will update every 4 years, as required by GPRAMA. OMB is  required to coordinate with agencies to publish progress updates on  Performance.gov on a quarterly basis for each CAP goal. Of the 15  current goals, 7 are mission-oriented goals and 8 are management- focused goals (see figure 3).
When selecting the current set of CAP goals, OMB staff told us they  considered factors such as the administration’s priorities, GPRAMA  requirements, and our prior work. According to documents we reviewed  and agency officials we spoke with, OMB also consulted with  government-wide councils, agencies, and congressional committees  when developing potential CAP goals. For example, OMB staff told us  that they added the Insider Threat and Security Clearance Reform CAP  goal based on congressional input. We are conducting an ongoing  assessment of the current set of CAP goals, shown in figure 3 above, and  selected 7 goals for our review. The objectives of this review are to (1)  assess the extent to which lessons learned from implementing the interim  CAP goals were incorporated into the governance of the current CAP  goals; (2) assess the extent to which GPRAMA requirements for  assessing and reporting on CAP goal progress are included in the  selected CAP goal quarterly progress updates; and (3) assess the initial  progress in implementing the selected CAP goals. We plan to issue this  work at the end of 2015 and will provide updated information on selected  CAP goals’ progress at that time.
CAP goal leaders or their teams from each of the seven selected CAP  goals told us that the CAP goal designation led to increased emphasis  and leadership attention within their agencies and the Executive Office of  the President (EOP) on the CAP goal area.
section_title: OMB Is Taking Steps to Enhance the Governance and Implementation of Current CAP Goals
paragraphs: OMB has made several improvements to its CAP goal guidance, in part in  response to our prior work, which found that OMB should strengthen CAP  goal reviews. Our June 2014 report on CAP goal reviews found, among  other things, that all reviews did not meet leading practices for leadership  involvement, participation by key officials, and follow-up. In response to  our recommendations, OMB and the PIC took several actions, including  updating the guidance to CAP goal teams and outlining the role of OMB  leadership and the PIC. For example, the guidance specifies that OMB’s  Deputy Director for Management will chair implementation-focused  meetings for the 8 management CAP goals approximately three times a  year and OMB’s Deputy Director for Budget will chair meetings for the 7  mission-focused CAP goals, as necessary. OMB staff confirmed that as  of August 2015, such meetings had been held for 11 of the 15 CAP goals.  OMB staff also told us that regular senior-level meetings, consistent with  OMB’s guidance for CAP goal reviews, also took place for 3 additional  goals—Cybersecurity, Infrastructure Permitting Modernization, and  Insider Threat and Security Clearance. No such reviews have yet taken  place for the Climate Change CAP goal, according to OMB staff.
OMB also changed the CAP goal governance structure to build capacity  for goal implementation. For the interim CAP goal process, each CAP  goal was assigned a goal leader from the EOP. For the current CAP  goals, in addition to the EOP goal leader, OMB assigned a co-goal leader  from key agencies to jointly manage and oversee the goal. For example,  the Customer Service CAP goal leaders are OMB’s Associate Director for  Personnel and Performance and the Acting Commissioner of the Social  Security Administration (SSA). According to OMB, this new governance  structure reflects agency leadership and expertise in CAP goal subject  areas and more effectively leverages agency resources for crosscutting  efforts and to promote greater coordination across multiple agencies. For  example, the OMB goal leader for the Customer Service goal told us that  it is helpful to have SSA in a leadership role because the agency provides  its perspective on implementation efforts related to improving customer  service, including piloting new activities before they are implemented  government-wide. According to the second quarterly update of fiscal year  2015, SSA helped OMB to launch a pilot for a customer service regional  community of practice in Denver, Colorado, to help field staff work across  agencies and identify opportunities for joint trainings and joint recruiting  efforts.
We found that OMB and the PIC have also implemented strategies  intended to build agency capacity to work across agencies. These  strategies include:
Providing ongoing guidance and assistance to CAP goal teams.  The seven CAP goal teams we spoke with told us that OMB and the  PIC staff are available on a regular basis to provide them with ongoing  support, such as assisting with the regular collection of performance  data and updating Performance.gov. For example, the Science  Technology Engineering and Mathematics (STEM) Education CAP  goal leader from the National Science Foundation told us the PIC  facilitated meetings to assist the goal team in developing milestones  and performance indicators and to define actionable next steps.
Developing a template to enhance reporting and management.
OMB provided CAP goal teams with a simplified reporting template to  use for managing implementation of the goal and to meet quarterly  reporting requirements on Performance.gov. According to OMB staff  and the seven CAP goal teams we spoke with, the new reporting  template makes it easier for goal leaders to review the updates and  track progress from one quarter to the next.
Piloting a government-wide White House leadership  development program. In December 2014, the President announced  a White House leadership development program which is designed to  provide selected civil servants (i.e., GS-15 level or equivalent) with  rotational assignments across agencies to focus on managing CAP  goals. According to OMB staff, the program will begin in October 2015  and the participants will spend the next year helping the White House  and agencies work on implementing the CAP goals.
section_title: CAP Goal Teams Reported Some Initial Progress and Are Developing Performance Indicators
paragraphs: During the first year of implementation of the current set of CAP goals,  CAP goal teams reported initial progress to the extent performance data  were available. For example, in its progress update for the second  quarter of fiscal year 2015, published in June 2015, the Open Data CAP  goal team reported an increase in the use of open government data as  indicated by a 25 percent quarterly increase in the number of visits to  Data.gov, the government’s platform for publishing its data. The goal  team has identified this indicator as a way to measure progress toward  one of their goals to fuel economic growth and innovation.
However, a few of the CAP goal teams we spoke with told us that in some  cases performance data are not always available and developing  meaningful performance indicators to assess progress is a challenge. Our  June 2014 assessment of the interim CAP goal process found that 6 of  the 14 interim CAP goals did not report performance data, in some cases  because data needed to assess and report progress toward the goals  were unavailable. As a result, we recommended that OMB direct CAP  goal leaders to develop plans to identify, collect, and report data  necessary to demonstrate progress being made toward each goal. OMB  and the PIC updated guidance directing CAP goal teams to establish  performance targets and report on any performance indicators that are  under development.
According to the second quarterly update for fiscal year 2015, 5 of the 7  CAP goals we reviewed have indicators under development for some of  their goals. For example, the Customer Service CAP goal team reported  that they are developing a standardized performance indicator to  measure improvements in citizen satisfaction across government, but the  progress update does not provide any information on intermediate  deliverables, roles and responsibilities, or time frames for completion.  On the other hand, another progress update we examined did include  information on steps the CAP goal team is taking to develop a  government-wide performance indicator. The STEM Education CAP goal  team reported that one of its working groups is developing common  evaluation elements to be used across federal agencies, with an  expected completion date in early 2016. The goal team also provided  information on currently available data, near-term and long-term steps  they are taking, and additional research needs. Our June 2014  recommendation remains open because OMB’s updated guidance did not  direct CAP goal teams to report on the steps they are taking to develop  indicators and associated time frames.
We have previously reported that tracking and monitoring progress for  cross-agency activities is difficult for a number of reasons such as  competing mission priorities, incompatible processes and systems across  agencies, resources, and staffing. Given these challenges, when  developing performance indicators for government-wide activities, it is  important that CAP goal teams provide information on the steps they plan  to take to successfully develop meaningful indicators, which will enable  them to better track progress over time and hold contributors accountable  for implementation.
section_title: The Executive Branch Has Still Not Taken Action to Assess the Outcomes of Tax Expenditures
paragraphs: For over 20 years, we have recommended greater scrutiny of the  performance of tax expenditures—reductions in a taxpayer’s tax liability  that are the result of special exemptions and exclusions from taxation,  deductions, credits, deferrals of tax liability, or preferential tax rates. If  the Department of the Treasury’s (Treasury) estimates are summed,  approximately $1.2 trillion in revenue was forgone from the 169 tax  expenditures reported for fiscal year 2014, nearly the same as  discretionary spending that year. In June 1994 and again in September  2005, we recommended that OMB develop a framework for reviewing tax  expenditure performance. Periodic reviews could help determine how  well specific tax expenditures work to achieve their stated purposes and  how their benefits and costs compare to those of spending programs with  similar goals. Given the significant investment tax expenditures  represent, such reviews could help identify the most effective approaches  for achieving results—vital information for federal decision makers in an  era of scarce resources. Despite the strong case to evaluate the  performance of tax expenditures, OMB has not yet developed a  framework for doing so.
Fully implementing GPRAMA requirements could provide the foundation  of such a framework. GPRAMA requires OMB to identify tax expenditures  that contribute to the CAP goals. In addition, since 2012, OMB’s  guidance has directed agencies to identify tax expenditures that  contribute to their APGs. Our past work reviewing initial GPRAMA  implementation in 2012 and 2013 found that OMB and agencies rarely  identified tax expenditures as contributors to these goals. As a result,  we made several recommendations to improve efforts to identify and  assess the contributions of tax expenditures toward executive branch  goals.
To date, OMB and agencies have taken little action to address these  recommendations. OMB has directed agencies, beginning with its 2013  update to its guidance, to identify tax expenditures that contribute to each  of their strategic objectives, in response to a recommendation we made in  June 2013. However, our work reviewing GPRAMA implementation  continues to find that OMB and agencies have not adequately identified  the contributions of tax expenditures to CAP goals and APGs. For  example, we found in June 2014 that although the goal leader for the  Broadband interim CAP goal told us he was aware that tax deductions  available to businesses making capital investments contributed to the  goal by incentivizing investments in broadband, the deductions were not  identified as contributors on Performance.gov.
Given their government-wide purview and familiarity with administering  the tax code, OMB and Treasury, respectively, are well positioned to  assist agencies in identifying tax expenditures that relate to their goals.  To that end, OMB’s 2013 and 2014 Circular A-11 guidance noted that it  would work with Treasury and agencies to identify where tax expenditures  align with their goals and this information was to be published on  Performance.gov and included in relevant agency plans, beginning in  February 2014. However, as we found in October 2014, according to  OMB staff, they did not begin to engage Treasury on this effort until after  agency plans were published and the website was updated. OMB staff  told us in August 2015 that they had not yet made any progress on this  effort. Moreover, OMB removed the language about working with  Treasury and agencies to align tax expenditures with agency goals in the  June 2015 update to its guidance. Although OMB staff told us they intend  to focus on this effort, they did not provide us with any plans or time  frames for doing so.
We have previously identified additional steps OMB could take to help  agencies consider the contributions of tax expenditures to the  achievement of their goals. We recommended in our October 2014 report  on the federal program inventory required under GPRAMA that OMB  should include tax expenditures. The federal program inventory is the  primary tool for agencies to identify programs that contribute to their  goals, according to OMB’s guidance. By including tax expenditures in the  inventory, OMB could help ensure that agencies are properly identifying  the contributions of tax expenditures to the achievement of their goals.  OMB staff neither agreed nor disagreed with these tax expenditure  recommendations, and told us that until they had firmer plans on how  program inventory and DATA Act implementation would be merged, they  could not determine if implementing these recommendations would be  feasible. As previously described, program inventory implementation  remains on hold and OMB has not taken any actions to address these  recommendations, according to OMB staff. Without including tax  expenditures in the inventory, OMB forgoes an important opportunity to  increase the transparency of tax expenditures and the outcomes to which  they contribute.
section_title: Ensuring Performance Information Is Useful and Used by Managers Remains a Challenge, but OMB and Agencies Are Implementing Processes That May Lead to Improvements
subsections: section_title: Agencies Continue to Face Challenges in Using Performance Information for Decision Making
paragraphs: We have long reported that agencies are better equipped to address  management and performance challenges when managers effectively  use performance information for decision making. Unfortunately, agencies  continue to struggle to do so. Our work has found that federal agencies  can use performance information to identify performance improvement  opportunities, improve program implementation and organizational  processes, and make other important management and resource  allocation decisions.
However, our recent work shows that agencies continue to have  problems effectively using performance information. Our September 2014  report on trends in agencies’ use of performance information compared  agencies’ reported use of performance information from our 2007 and  2013 federal managers surveys. To analyze use, we developed a use of  performance information index, based on a set of survey questions in  both surveys that reflected the extent to which managers reported that  their agencies used performance information for various management  activities and decision making. The index runs from 1 to 5, where a 1  reflects that managers feel the agency engages “to no extent” and a 5  reflects that managers feel the agency engages “to a very great extent” in  the use of performance information activities. Most agencies showed no  statistically significant change in use during this period. As shown in  figure 4, only two agencies experienced a statistically significant  improvement in the use of performance information, while four agencies  experienced a statistically significant decline.
We have previously reported that in order for performance information to  be useful, it must have certain characteristics. Specifically, agencies  should ensure that performance information meets various users’ needs  for completeness, accuracy, consistency, timeliness, validity, and ease of  use. Without complete and reliable performance information, Congress,  other decision makers, and stakeholders at all levels of government are  hampered in their ability to set priorities, identify improvement  opportunities, and allocate resources.
Our work over the past 2 years has identified weaknesses in each of the  areas that affect the usefulness of performance information. For example,  our September 2015 report on affordable rental assistance programs  identified an incomplete picture of performance of these programs as a  problem. We found that federal, state, and local jurisdictions involved in  these efforts reported their performance to varying extents, but that there  was incomplete information on their collective performance. Accordingly,  we recommended that the Department of Housing and Urban  Development (HUD), in consultation with an interagency working group  on rental policy, should work with states and localities to develop an  approach for compiling and reporting on the collective performance of  federal, state, and local rental assistance programs. Treasury and IRS,  which are agency members of this working group, did not comment on  this recommendation. HUD was concerned that compiling and reporting  collective performance information would require significant funding and  resources. We continue to believe the overall recommendation is valid.  Specifically, we noted that (1) our recommendation is to develop an  approach for compiling and reporting such data as a first step, and (2)  that our recommendation is purposefully not prescriptive and allows HUD,  in consultation with the working group, to design an approach. Additional  examples of problems that affect the usefulness of performance  information are illustrated in table 1.
section_title: Agency Implementation of Performance Reviews Should Improve the Use of Performance Information for Decision Making
paragraphs: Performance reviews required under GPRAMA and other guidance by  their nature promote the use of performance information, as they focus on  assessing performance in order to determine progress toward meeting  goals and objectives.
Data-driven reviews. GPRAMA requires that reviews of progress of  agency priority goals (APG) be held at least quarterly; as this requirement  is more fully implemented, the use of performance information for  decision making should improve. OMB emphasized that frequent, data- driven performance reviews provide a mechanism for agency leaders to  use data to assess the organization’s performance, diagnose  performance problems and identify improvement opportunities, and  decide on next steps to improve performance. These practices are  designed to shift the emphasis away from the passive collection and  reporting of performance information to a model where performance  information is actively used by agency officials to inform decision-making,  which is more likely to lead to performance improvements.
In our July 2015 report on data-driven reviews, we found that PIOs had  reported that the reviews had positive effects on their agencies’ use of  performance information. Nearly all of the 22 agencies that reported  holding in-person reviews responded that they always or often use their  review meetings to assess progress on APGs and to identify goals at risk  and strategies for performance improvement. Additionally, as shown in  figure 5, nearly all of these agencies also reported that their data-driven  review meetings have had a positive effect on progress toward the  achievement of their agency’s goals and on their ability to identify and  mitigate risks to goal achievement.
In our discussions with officials from selected agencies, data-driven  review meetings were described as venues for agency leaders and  managers to assess progress toward key goals and milestones, the  status of ongoing initiatives and planned actions, potential solutions for  problems or challenges hindering progress, and additional support or  resources needed to improve performance. Agency officials emphasized  that discussions in their review meetings tend to focus on those goals or  issues most in need of attention, where the achievement of a goal or  milestone is at risk. In this way, reviews can serve as early warning  systems and facilitate focused discussions on external, technical, or  operational obstacles that may be hindering progress and the specific  actions that should be taken to overcome them (see sidebar on the  following page).
Increasing Online Registration through my  Social Security  The Social Security Administration (SSA) has  an APG to increase the number of  registrations for its “my Social Security” portal  by 15 percent per year in fiscal years 2014  and 2015. However, we reported in July 2015  that during SSA’s 2014 third quarter review  meeting, it became apparent to SSA  leadership that the agency was not on track to  achieve its target for this goal. SSA shifted  focus to what could be done by offices  throughout the agency to support efforts to  increase the number of registrations using  currently available or attainable resources and  technology. To achieve this, SSA leadership  had different offices within the agency specify  the contributions they would make to help  increase the number of registrations. Since  then, the agency’s quarterly review meetings  have been used to review and reinforce the  commitments each office made. While SSA  was unable to meet the registration goal for  fiscal year 2014, according to SSA officials,  these efforts recently undertaken as a result  of the review process have helped generate  an increase in registrations. Data from SSA’s  fiscal year 2015 first quarter review show that  there was a 46 percent increase in new  account registrations in October 2014  compared to the number of new registrations  in October 2013, and a 26 percent increase in  December 2014 relative to December 2013. performance reviews. Reexamining the benefits and costs achieved  after a regulation is implemented could provide useful data for these  reviews. Despite the potential to leverage retrospective review  information, agencies reported mixed experiences linking retrospective  analyses to APGs. Agencies typically selected rules to review based on  criteria such as the number of complaints or comments from regulated  parties and the public. Including whether a regulation contributed to an  APG as one of these criteria would help agencies prioritize retrospective  analyses that could contribute useful information to APG assessments.  We recommended that OMB’s Office of Information and Regulatory  Affairs direct in guidance that agencies take actions to ensure that  contributions made by regulations toward the achievement of APGs are  properly considered and improve how retrospective regulatory reviews  can be used to help inform assessments of progress toward these  APGs. OMB staff agreed with this recommendation and stated that the  agency was working on strategies to help facilitate agencies’ ability to use  retrospective reviews to inform APGs, but as of June 2015 they have not  provided additional details on their actions.
Strategic reviews. Like data-driven reviews of APGs, and retrospective  reviews of regulations, agencies’ annual strategic reviews also have  potential to increase the use of performance information. For example, we  reported in July 2015 that, to ensure effective strategic reviews,  participants should use relevant performance information and evidence to  assess whether strategies are being implemented as planned and  whether they are having the desired effect, and to identify areas where  action is needed to improve or enhance implementation and impact.  Where progress in achieving an objective is lagging, the reviews are  intended to identify strategies for improvement, such as strengthening  collaboration to better address crosscutting challenges, building skills and  capacity, or using evidence to identify and implement more effective  program designs. Strategic reviews can also be used to identify any  evidence gaps or areas where additional analyses of performance data  are needed to determine effectiveness or to help set priorities. For  example, we reported that for the Department of Homeland Security’s  (DHS) goal to safeguard and expedite lawful trade and travel, officials  determined that sufficient progress was being made but identified gaps in  monitoring efforts, such as a lack of performance measures related to  travel. As a result, DHS officials are taking steps to develop measures to  address the gaps.
section_title: Other Leading Practices and Evidence-Based Tools Also Have the Potential to Increase Use of Performance Information
paragraphs: We have in the past identified leading practices, such as demonstrating  management commitment, that can enhance and facilitate the use of  performance information. Our recent periodic survey of federal  managers found that specific practices were related to greater use of  performance information. As described previously, we developed a use  of performance information index, composed of questions from the 2007  and 2013 surveys, to analyze responses to our surveys of federal  managers. We used statistical testing to determine if the relationship  between additional survey questions, shown in figure 6, from the 2013  survey and an agency’s use of performance information index was  statistically significant. We found that an agency’s average use of  performance information index score increased when managers reported  their agencies engaged to a greater extent in these practices as reflected  in the survey questions. The questions that were statistically and  positively related to the use of performance information index are also  shown in figure 6. For example, we found that the strongest driver of the  use of performance information was whether federal managers had  confidence in that information’s validity. A greater focus on these  practices may help agencies improve their use of performance  information. Prompted by our work, several agencies—including the  Departments of the Treasury and Labor, the National Aeronautics and  Space Administration, and the Nuclear Regulatory Commission—asked  us to provide them with underlying data for their agencies from the 2007  and 2013 managers’ surveys, so that they could conduct additional  analyses of their agencies’ use of performance information.
Some of the practices reflected in these questions are ones that we have  identified elsewhere in our work as important. For example, demonstrated  leadership commitment is an area we have emphasized in our work on  government operations we identify as high risk. Our high-risk program  serves to identify and help resolve serious weaknesses in areas that  involve substantial resources and provide critical services to the public.  Our experience with the high-risk list over the past 25 years has shown  that one of the key elements needed to make progress in high-risk areas  is demonstrated strong commitment and top leadership support.  Additionally, providing, arranging, or paying for training may also be  related to employee engagement. In July 2015, we reported that career  development and training—as measured by the Federal Employee  Viewpoint Survey question “I am given a real opportunity to improve my  skills in my organization”—is one of the six practices that are key drivers  of employee engagement.
In addition, evidence-based tools—such as program evaluations and “pay  for success” funding mechanisms—can also facilitate the use of  performance information.
Program evaluations. Our recent work on program evaluations— systematic studies of program performance—found that agencies have  varying levels of evaluation capacity. OMB has encouraged agencies to  strengthen their program evaluations and expand their use in  management and policy making, but our 2014 examination of agencies’  ability to conduct and use program evaluations found it to be uneven. As  part of our work, we surveyed and received responses from the PIOs at  the 24 CFO Act agencies. About half (11) of the 24 agencies reported  committing resources to obtain evaluations by establishing a central office  responsible for evaluating agency programs, operations, or projects; on  the other hand, 7 reported having no recent evaluations for any of their  performance goals. Although agencies may not have many evaluations,  more than a third reported using them to a moderate to a very great  extent to support several aspects of program management and policy  making.
While agency program evaluation capacity is mixed, some agencies  reported increasing use of evaluations and capacity-building activities  after GPRAMA was enacted. About half of agencies reported increasing  their use of evaluations for various activities, as shown in figure 7, since  GPRAMA was enacted. Additionally, half of the PIOs we surveyed  reported that efforts to improve their capacity to conduct credible  evaluations had increased at least somewhat over this time.
Our work found that implementing certain GPRAMA requirements are  among the reported actions agencies can take to improve their capacity  to conduct evaluations and make use of evaluation information. About  two-thirds of agencies (15) reported hiring staff with research analysis  and expertise, and nearly half (11) reported that doing so was useful for  improving agency capacity to conduct credible evaluations. Additionally,  about half of PIOs reported that conducting data-driven reviews of APGs  and holding goal leaders accountable for progress on APGs, both of  which are required under GPRAMA, were moderately to very useful for  improving agency capacity to make use of evaluation information in  decision making. Engaging program staff was also rated very useful.  Furthermore, in our June 2013 report on strategies to facilitate agencies’  use of evaluation, we identified three strategies to facilitate the influence  of evaluations on program management and policy: demonstrating  leadership support of evaluation for accountability and program  improvement; building a strong body of evidence; and engaging  stakeholders throughout the evaluation process.
Pay for Success. Another evidence-based tool that promotes the use of  performance information is Pay for Success (PFS), also known as Social  Impact Bonds. PFS is a new contracting mechanism to fund prevention  programs, where investors provide capital to implement a social service,  for example, to reduce recidivism by former prisoners. If the service  provider achieves agreed upon outcomes, the government pays the  investor, usually with a rate of return, based on savings from decreased  use of more costly remedial services, such as incarceration. In  September 2015, we reported that stakeholders from the 10 PFS projects  we reviewed said that PFS offers potential benefits to all parties to the  project. For example, governments can implement prevention programs  that potentially lead to reduced spending on social services and transfer  the risk of failing to achieve outcomes to investors. Figure 8 shows the  roles of organizations involved in PFS projects.
PFS emphasizes the use of performance information because the  government contracts for specific performance outcomes and generally  includes a requirement that a program’s impact be independently  evaluated. While the structures of the PFS examples we reviewed in our  September 2015 report varied, stakeholders we interviewed reported that  PFS oversight bodies established in the projects’ contracts regularly  reviewed performance data during service delivery. Additionally,  stakeholders told us that intermediaries and investors can bring  performance management expertise to service providers and provide a  rigorous focus on performance management and accountability. For  example, an official we interviewed from one service provider noted that  her organization invested in data entry and data analyst positions and has  a team that collects, analyzes, and processes data that it submits to the  intermediary.
As federal agencies consider expanding their involvement in PFS, it  becomes increasingly important for officials at all levels of government to  collaborate to share knowledge and experiences. We found that while the  federal government could play a role in addressing challenges in  implementing PFS at the state and local levels of government, a formal  means to collaborate and share lessons learned does not exist. We  recommended in our September 2015 report that OMB establish a formal  means for federal agencies to collaborate on PFS. Having such a  mechanism as the field grows would allow agencies to leverage the  experience of early federal actors in the PFS field and would decrease  the potential for missteps in developing projects due to information gaps  and failure to learn from experience with this evolving tool of government.  OMB concurred with this recommendation and is working with agencies  to explore options for continued collaboration on PFS.
section_title: Agencies Continue to Face Challenges Linking Individual and Agency Performance to Results
subsections: section_title: Goal Leader Designations and Participation in Performance Reviews Have Positive Effects, but Agencies Are Missing Opportunities to Further Strengthen Performance and Accountability
paragraphs: Our previous work has highlighted the importance of creating a “line of  sight” showing how unit and individual performance can contribute to  overall organizational goals. At the individual level, an explicit alignment  of daily activities with broader results is one of the defining features of  effective performance management systems. This link reinforces  employee engagement and accountability for achieving goals. GPRAMA  and related guidance provide several mechanisms that can help  individuals and agencies see this connection and help them contribute to  agency and government-wide goals.
subsections: section_title: Goal Leader Designation, Performance Reviews, and Other Factors Have Positive Effects
paragraphs: GPRAMA requires agencies to identify an individual—the goal leader— responsible for the achievement of each APG, and related OMB guidance  more recently directed agencies to identify a deputy goal leader to  support each goal leader, as we had recommended in our July 2014  report on the role of the agency priority goal leader. Additionally, data- driven reviews required under the act offer the opportunity to hold  responsible officials, such as the goal leaders, personally accountable for  addressing problems and identifying strategies for improvement.
Agency priority goal leaders. Our July 2014 review of the agency  priority goal leader role found that most of the 46 goal leaders we  interviewed thought the goal leader designation had positive effects,  including providing accountability. Other benefits goal leaders identified  as resulting from the position include that it provided greater visibility for  the goal, facilitated coordination, heightened focus on the goal, and  improved access to resources. Goal leaders told us that several  mechanisms promote accountability for goal progress, including personal  reputation and accountability to agency and other leadership. For  example, the Assistant Secretary of Labor for Occupational Safety and  Health, who was the goal leader for two APGs for 2012 and 2013  (Reduce Worker Fatalities and Develop a Model Safety and Return-to- Work Program), told us that the interest of Congress and the Department  of Labor’s Inspector General, in their respective oversight roles, both  operated to hold him accountable.
Deputy goal leaders supported day-to-day goal management and  provided continuity during times of goal leader transition. However, we  found that nearly a quarter (11 of 46) of the goal leaders we interviewed  had not assigned an official to the deputy goal leader position, a  designation that provides clear responsibility and additional accountability  for goal achievement. Because of the importance of this position, we  recommended that OMB work with agencies to ensure that they  appointed a deputy goal leader to support each agency priority goal  leader. In response to our recommendation, in April 2015, the Director of  OMB issued a memorandum stating that, in addition to identifying a goal  leader for each APG, agencies must identify a senior career leader to  support implementation. OMB also updated its 2015 Circular A-11  guidance to reflect this requirement.
Data-driven and strategic reviews. Our work has found that regular, in- person data-driven reviews are an effective mechanism for holding goal  leaders and other goal contributors individually accountable for goal  progress. In our July 2015 examination of data-driven reviews, we  reported that 22 agencies we surveyed reported holding in-person data- driven reviews. Of these, 21 reported that their data-driven reviews have  had a positive effect on their agency’s ability to hold goal leaders and  other officials accountable for progress toward goals and milestones.  According to officials from selected agencies, the transparency of  performance information and a review process that ensures it receives  appropriate scrutiny produce an increased sense of personal  accountability for results. For example, officials from the Department of  Commerce told us that they are using their regular review meetings with  bureau heads and goal leaders to support a cultural change throughout  the agency and reinforce accountability for performance at multiple levels  of the organization.
NASA Officials Report that Strategic  Reviews Encourage Accountability  We reported in July 2015 that the National  Aeronautics and Space Administration’s  (NASA) experience reviewing strategic  objectives illustrates their potential for  promoting accountability. Agency officials told  us that their chief operating officer (COO)  determines final ratings for strategic  objectives during a briefing attended by the  agency’s performance improvement officer,  strategic objective leaders, and relevant  program staff. According to NASA officials,  the personal involvement of the COO  encouraged accountability for results and  performance improvements.
Similar to data-driven reviews, our work on agency strategic reviews  noted that they also have potential for promoting individual accountability  for organizational results. In our July 2015 report, in which we identified  and illustrated practices that facilitate effective strategic reviews, we  reported that accountability for results is one of the key features for  planning reviews. Specifically, we stated that the focus of accountability  should be on the responsible objective leader’s role in using evidence to  credibly assess progress in achieving strategic objectives. Agency  leaders should hold objective leaders and other responsible managers  accountable for knowing the progress being made in achieving outcomes  and, if progress is insufficient, understanding why and having a plan for  improvement. If evidence is insufficient for assessing progress, managers  should be held accountable for improving the availability and quality of the  evidence so that it can be used effectively for decision making. Managers  should also be held accountable for identifying and replicating effective  practices to improve performance (see sidebar).
Employee engagement. Research on both private- and public-sector  organizations has found that increased levels of engagement—generally  defined as the sense of purpose and commitment employees feel toward  their employer and its mission—can lead to better organizational  performance. Our July 2015 report on employee engagement identified  specific practices that drive employee engagement. Specifically, our  regression analysis of selected Federal Employee Viewpoint Survey  (FEVS) questions identified six practices as key drivers of employee  engagement, as measured by OPM’s Employee Engagement Index.  These practices are detailed in figure 9. Of these six, having constructive  performance conversations is the strongest driver of employee  engagement.
Performance appraisal systems, which include performance plans, are a  powerful mechanism for promoting alignment with and accountability for  organizational goals. There are several benefits to aligning performance  with results, including increased use of performance information. As  shown in the textbox, our work has found problems with oversight and  accountability in the Department of Veterans Affairs’ (VA) Health Care  System. In response to these and other problems, Congress has taken  action, such as passing the Veterans Access, Choice, and Accountability  Act of 2014, to hold senior VA leadership accountable for performance  and is considering other means of increasing accountability.
Inadequate Oversight and Accountability in VA’s Health Care  System  Despite substantial budget increases in recent years, for more than a  decade there have been numerous reports—by GAO, VA’s Office of the  Inspector General, and others—of VA facilities failing to provide timely  health care. In some cases, the delays in care or VA’s failure to provide  care at all have reportedly resulted in harm to veterans. These and other  serious and long-standing problems with the timeliness, cost- effectiveness, quality, and safety of veterans’ health care led to our  designation of VA’s health care system as a high-risk area in 2015.
To facilitate accountability for achieving its organizational goal of ensuring  that veterans have timely access to health care, VA included measures  related to wait times for primary and specialty care appointments (1) in  the performance contracts for senior leaders and (2) in the agency’s  annual budget submissions and performance and accountability reports.  However, we found that data used to monitor performance on these  measures were unreliable and that inconsistent implementation of VA’s  scheduling policies may have resulted in increased wait times or delays in  scheduling outpatient medical appointments at VA facilities. Scheduling  staff in some locations told us that they had changed desired dates for  medical appointment to show that wait times were within VA’s  performance goals. The VA Office of the Inspector General has published  reports with similar findings. VA has since announced that it has modified  its performance measures that relate to wait times and removed  measures related to wait times from senior leaders’ performance  contracts.
Goal leaders’ Senior Executive Service performance plans. Although  goal leaders told us that the designation provides accountability, we found  their Senior Executive Service (SES) performance plans generally did not  reflect their responsibility for goal achievement. As part of our work on the  role of the agency priority goal leader in July 2014, we reviewed the  performance plans of all of the goal leaders and deputy goal leaders for  the 47 APGs in our sample, where applicable. These performance plans  covered a range of responsibilities, but many did not reference the APGs  for which the goal leaders and deputies were responsible. Additionally,  the vast majority (all but one of the 32 goal leader plans and one of the 35  deputy goal leader plans) failed to link performance standards to goal  outcomes. Failing to fully reflect goal achievement in performance plans  is a missed opportunity to ensure that goal leaders and deputies are held  accountable for goal progress and to reinforce links. Because APGs by  definition reflect the highest priorities of each agency, accountability for  goal achievement is especially important. To ensure goal leader and  deputy goal leader accountability, we recommended that the Director of  OMB work with agencies to ensure that goal leader and deputy goal  leader performance plans demonstrate a clear connection with APGs. As  of June 2015, OMB had not yet taken action in response to this  recommendation.
Senior Executive Ratings. Our recent work has also raised questions  about agency processes for rating senior executive performance, which  can promote alignment with and accountability for organizational goals.  For our January 2015 report on SES ratings and performance awards, we  reviewed performance award data from the 24 CFO Act agencies and we  examined performance appraisal systems at five case study agencies.  Specifically, we looked at the performance appraisal system that the  Office of Personnel Management (OPM) and other agency  representatives developed in 2012. This system is intended to provide a  more consistent and uniform framework for SES evaluation. We found  that the five agencies we studied in detail had all linked SES performance  plans with agency goals, a key practice for effective performance  management systems, and a feature that promotes the line of sight  between individual performance and organizational goals.
Disparities in Agencies’ Executive Ratings  Distributions  We reported in January 2015 that there is  wide variation in SES ratings distributions  among agencies. For example, in fiscal year  2013 the Department of Defense rated 30.6  percent of its SES employees at the highest  rating level, while the Department of Justice  rated 73.6 percent of its SES employees at  this level. As we have previously reported,  one of the key practices in promoting a line of  sight between individual performance and  organizational goals is making meaningful  distinctions in performance.
However, although one of the primary purposes for establishing the new  appraisal system included increasing equity in ratings across agencies,  we found disparities in rating distributions and pay (see sidebar). This  disparity in ratings between agencies raises questions about whether  agencies are consistently applying performance definitions and whether  performance ratings are meaningful. We recommended that the Director  of OPM, which certifies—with OMB concurrence—SES performance  appraisal systems, should consider the need for refinements to the  performance certifications guidelines addressing distinctions in  performance and pay differentiation. OPM partially concurred with the  recommendation, though we maintain that additional action should be  considered to ensure equity in ratings and performance awards across  agencies. As of June 2015, OPM officials said that they had convened a  cross-agency working group that developed several recommendations  that are intended to make agencies’ justifications for high SES ratings  more transparent.
Senior executives’ use of performance information for decision  making. Aligning SES performance with results is a key feature of  effective performance management, and our recent work has found that it  also may promote use of performance information. As we found in our  September 2014 report on trends in the use of performance information,  managers’ responses to a question we asked them on aligning an  agency’s goals, objectives, and measures was significantly related to the  use of performance information, controlling for other factors.  Specifically, an increase in the extent to which managers aligned  performance measures with agency-wide goals and objectives was  associated with an increase on the five-point scale we used for our use  index. However, our analysis also found that there was a gap between  SES and non-SES managers in reported use of performance information.  SES managers government-wide and at nine agencies scored statistically  significantly higher than the non-SES managers at those agencies. As  shown in figure 10, SES and non-SES managers from the Departments of  Homeland Security and Veterans Affairs had the largest gap in use of  performance information between their SES and non-SES managers.
section_title: Agencies Have Long- standing Difficulties in Measuring Performance of Selected Program Types
paragraphs: A critical element in an organization’s efforts to manage for results is its  ability to set meaningful goals for performance and to measure progress  toward these goals. GPRAMA reinforces the need to set meaningful goals  by directing agencies to establish a balanced set of performance  measures, such as output, outcome, customer service, and efficiency,  across program areas.
Agencies have been responsible for measuring program outcomes since  GPRA was enacted in 1993, but still have difficulty developing and using  performance measures. As we reported nearly 20 years ago,  performance measures should demonstrate to each organizational level  how well it is achieving its goals. As shown in the illustrations in the  textbox, however, agencies continue to make insufficient progress in  establishing and using outcome-oriented performance measures.
Examples of Agency Difficulties in Developing and Using Outcome  Measures  Outcome-Oriented Metrics and Goals Are Needed to Gauge DOD’s  and VA’s Progress in Achieving Interoperability of Electronic Health  Records Systems  The Departments of Defense (DOD) and Veterans Affairs (VA) operate  two of the nation’s largest health care systems, serving approximately 16  million veterans and active duty service members and their beneficiaries,  at a cost of more than $100 billion a year. With guidance from the  Interagency Program Office (IPO) that is tasked with facilitating the  departments’ efforts to share health information, the two agencies have  taken actions to increase interoperability between their electronic health  record systems. Developing electronic health records is particularly  important for optimizing the health care provided to military personnel and  veterans, as they and their families tend to be highly mobile and may  have health records residing at multiple medical facilities. In August 2015,  we reported that the IPO had taken steps to develop process metrics  intended to monitor progress of these efforts, but had not yet specified  outcome-oriented metrics or established related goals that are important  to gauging the impact that interoperability capabilities have on improving  health care services for shared patients. Using outcome-based metrics  could provide DOD and VA a more accurate, ongoing picture of their  progress toward achieving interoperability and the value and benefits  generated. We recommended that DOD and VA, working with the IPO,  establish a time frame for identifying outcome-oriented metrics; define  related goals to provide a basis for assessing and reporting on the status  of interoperability; and update IPO guidance to reflect the metrics and  goals identified. DOD and VA concurred with our recommendations.
Measuring Progress in Addressing Incarceration Challenges  The federal inmate population has increased more than eight-fold since  1980, and the Department of Justice (DOJ) has identified prison crowding  as a critical issue since 2006. In June 2015, we reported that DOJ had  implemented three key initiatives to address the federal incarceration  challenges of overcrowding, rising costs, and offender recidivism. The  department had several early efforts underway to measure the success of  these initiatives, but we concluded that its current approach could be  enhanced. For example, the Clemency Initiative is intended to encourage  federal inmates who meet criteria that DOJ established to apply to have  their sentences commuted (reduced) by the President. DOJ tracked some  statistics related to this initiative, such as the number of petitions received  and the disposition of each, but it did not track how long, on average, it  took for petitions to clear each step in its review process. Such tracking  would help DOJ identify processes that might be contributing to any  delays. Without this tracking, DOJ cannot be sure about the extent to  which the additional resources it is dedicating to this effort are helping to  identify inmate petitions that meet DOJ’s criteria and expedite their  review. We recommended that the Attorney General direct the Office of  the Pardon Attorney to (1) track how long it takes, on average, for  commutation of sentence petitions to clear each step in the review  process under DOJ’s control, and (2) identify and address, to the extent  possible, any processes that may contribute to unnecessary delays. DOJ  concurred with the recommendation and stated that it would consider our  findings and recommendations during the course of its ongoing efficiency  reviews.
Measuring Effectiveness of Military Sexual Assault Prevention  Efforts  Our recent work has identified issues in establishing goals and metrics to  measure the effectiveness of efforts to reduce incidents of sexual assault  in the military, which according to the Department of Defense (DOD)  represent a significant and persistent problem within the department. For  example, in March 2015, we reported that DOD had not established goals  or metrics to gauge sexual assault-related issues for male service  members. DOD’s Sexual Assault Prevention and Response Office had  three different general officers in the director position since 2011. Given  this high level of turnover, we stated that establishing goals and metrics is  key to institutionalizing efforts to address sexual assault of male service  members. We recommended that DOD develop clear goals and  associated metrics to drive the changes needed to address sexual  assaults of males and articulate these goals. DOD agreed with this  recommendation.
Measuring the performance of different program types—such as grants,  regulations and tax expenditures—is a significant and long-standing  government-wide challenge and one we have addressed in our previous  work. In our June 2013 report on initial GPRAMA implementation, we also  reported that agencies have experienced common issues in measuring  various types of programs. We recommended that the Director of OMB  work with the PIC to develop a detailed approach to examine these  difficulties, including identifying and sharing any promising practices.  Additionally, our July 2014 report on the role of the agency priority goal  leader noted that several APGs we examined identified certain program  types, such as grants, as key contributors to their goals. However, goal  leaders and their deputies lacked the means to identify and share  information with other goal leaders who were facing similar challenges or  were interested in similar topics. We recommended that the Director of  OMB work with the PIC to further involve agency priority goal leaders and  their deputies in sharing information on common challenges and practices  related to APG management. OMB and PIC staff told us in June 2015  that they have taken some actions to facilitate information sharing on  common topics. For example, the PIC developed a law enforcement  working group, which aims to address challenges in measuring law  enforcement functions. Despite these steps, additional actions are  needed to fully implement these recommendations and address this long- standing issue. We will continue to monitor OMB’s and the PIC’s efforts.
Illustrative examples from our recent work that show how agencies need  to make better progress in measuring certain program types are provided  in table 2.
One program type—direct service—is one of the areas in which our  recent work has highlighted problems with agencies’ performance  measurement in multiple agencies. Our October 2014 report on customer  service standards examined how selected agencies are using customer  service standards and measuring performance against those  standards. We reviewed the customer services standards for six  federal programs and compared them to key elements of effective  customer services standards, which we identified based on our review of  GPRAMA and executive orders that focused on providing greater  accountability, oversight, and transparency. Two of the key elements of  customer services standards are that they (1) include targets or goals for  performance, and (2) include performance measures. We found that three  of the six programs did not have customer services standards that met  these two elements. For example, we reported that because the National  Park Service (NPS) did not have performance goals or measures directly  linked to those goals, the agency is unable to determine the extent to  which the standards are being met agency-wide or strategies to close  performance gaps. We made several recommendations related to  improving the NPS’s and other agencies’ customer service standards,  including that the Department of the Interior (of which NPS is a part) to  ensure NPS standards include (1) performance targets or goals, and (2)  performance measures. In July 2015, NPS officials reported that they had  made plans to implement these recommendations. Additionally, OMB  is focusing on improving the federal government’s customer service by  developing a related CAP goal. According to information on  Performance.gov, as part of its work on the Customer Service CAP goal,  the administration is working to streamline transactions, develop  standards for high impact services, and utilize technology to improve the  customer experience. We will be assessing OMB’s progress in  implementing this CAP goal as part of our ongoing review.
section_title: OMB and Agencies Have Not Clearly Communicated Key Performance Information, but More Effective Implementation of GPRAMA Requirements Would Improve Transparency
paragraphs: To operate as effectively and efficiently as possible and to make difficult  decisions to address the federal government’s fiscal and performance  challenges, Congress, the administration, and federal managers must  have ready access to reliable and complete financial and performance  information—both for individual federal entities and for the federal  government as a whole.
However, in our work since 2013 we have identified areas in which  agencies have not clearly reported information related to billions of dollars  in government spending (see textbox).
Examples of Agencies Not Clearly Reporting Information on  Government Spending  Agencies Fail to Properly Report Over $600 Billion in Assistance  Awards  The Federal Funding Accountability and Transparency Act (FFATA) was  enacted in 2006 to increase the accountability and transparency over the  more than $1 trillion spent by the federal government on contracts,  grants, loans, and other awards annually. The act required OMB to  establish a website that contains data on federal awards and guidance on  agency reporting requirements for the website, USASpending.gov. The  website is to promote transparency in government spending by providing  the public with the ability to track where and how federal funds are spent.  However, we reported in June 2014 that although agencies generally  reported required contract information, they did not properly report  information on assistance awards (e.g., grants or loans), totaling  approximately $619 billion in fiscal year 2012. In addition, we found that  few awards on the website contained information that was fully consistent  with agency records. We estimated with 95 percent confidence that  between 2 and 7 percent of the awards contained information that was  fully consistent with agencies’ records for all 21 data elements examined.  We concluded that without accurate data, the usefulness of  USASpending.gov will be hampered. To improve the reliability of  information on USASpending.gov, we recommended that OMB (1) clarify  guidance on reporting award information and maintaining supporting  records, and (2) develop and implement oversight processes to ensure  that award data are consistent with agency records. OMB generally  agreed with our recommendations but, as of August 2015, had not taken  actions to address them. Full implementation of the DATA Act, which  amended FFATA and which OMB is currently working on, may address  these recommendations.
USDA Performance Reporting Does Not Reflect Effects of  Approximately $3 Billion in Spending on Broadband  Access to affordable broadband Internet is seen as vital to economic  growth and improved quality of life, yet deployment in rural areas tends to  lag behind urban and suburban areas. The American Recovery and  Reinvestment Act of 2009 (Recovery Act) appropriated funding for the  Broadband Initiatives Program (BIP), a Department of Agriculture (USDA)  Rural Utilities Service (RUS) program to fund broadband projects  primarily to serve rural areas. However, we reported in June 2014 that  RUS has reported limited information on BIP’s impact since awarding  funds to projects, and that BIP results are not tracked in USDA’s annual  performance reporting. As a result, RUS has not shown how much the  program’s approximately $3 billion in project funding has affected  broadband availability. GPRAMA directs agencies to establish  performance goals in annual performance plans and to report on progress  made toward these goals in annual performance reports. However, USDA  did not update or include BIP results as compared to the related  performance goals in its annual performance reports. We concluded that  without an updated performance goal and regular information reported on  the results of BIP projects, it is difficult for USDA, RUS, and policymakers  to determine the impact of Recovery Act funds or BIP’s progress on  improving broadband availability. We recommended that the Secretary of  Agriculture include BIP performance information as part of USDA’s  annual performance plan and report by comparing actual results achieved  against the current subscribership goal. USDA agreed with our  recommendation, and stated that it planned to modify its next annual  performance plan and report to include the number of subscribers  receiving new and improved service as a result of the program.
Our work has also identified other problems with transparency. As  described in the textbox below, only one of the six federal services for  which we reviewed customer service standards had standards that were  easily publicly available.
Most Agencies Reviewed Did Not Make Customer Service Standards  Easily Publicly Available  Our recent work has also found issues with transparency related to  agencies’ customer service standards. In October 2014 we identified key  elements of customer service standards—which should inform customers  as to what they have a right to expect when they request services—that  would allow agencies to better serve the needs of their customers by  providing greater accountability, oversight, and transparency. One of the  elements that we identified is that customer service standards be easily  publicly available. Easily available standards help customers know what  to expect, when to expect it, and from whom. As part of our work, we  assessed the extent to which customer service standards at six services  within five federal agencies (including two services within one of those  agencies) included key elements, including easily publicly accessible  standards. We found that only one of these services had standards that  were easily available to the public. That service—Customs and Border  Protection (CBP) inspection of individuals—posts its standards on its  website as well as at points of service in entry ports, field offices, and  headquarters, according to CBP officials.
The other five services, we found, did not make their standards easily  accessible to the public. For example, we had reported in 2010 that the  Forest Service did not make its customer service standards available to  its customers because officials felt that the standards would not be helpful  to the visitors who evaluate such things as the cleanliness of restrooms  against their own standards and not those set forth by the Forest Service.  In 2014, Forest Service officials told us that there has been no change  since 2010. However, according to executive orders and guidance,  standards are specifically intended to inform the public, and should be  publicly available. We recommended that the Department of Agriculture  (of which the Forest Service is a part) ensure that the Forest Service’s  standards are easily publicly available, among other things. In addition,  we made recommendations to the other five services that had not made  their standards easily accessible to the public.
Although GPRAMA requirements have the potential to increase  transparency of performance information, we have found mixed progress  in implementing these requirements.
Program inventories. GPRAMA’s requirements for program inventories  have the potential to improve transparency of performance information,  but, as previously described, our October 2014 report identified several  issues that affect these inventories’ usefulness. For example, although  GPRAMA requires agencies to describe each program’s contribution to  the agency’s goals, we found instances where agencies omitted that  information. Ensuring agencies illustrate this alignment would better  explain how programs support the results agencies are achieving.
As stated earlier, OMB has put plans for updating the inventories on hold,  in part due to the enactment of the DATA Act, which is intended to  increase accountability and transparency in federal spending by requiring  agencies to publicly report information about any funding made available  to, or expended by, an agency. As noted in our July 2015 testimony on  DATA Act implementation, effective implementation of both the DATA Act  and GPRAMA’s program inventory provisions, especially the ability to  crosswalk spending data to individual programs, could provide vital  information to assist federal decision makers in addressing the significant  challenges the government faces. We identified a potential approach  OMB could take in merging program inventory efforts with DATA Act  implementation. That is, OMB could explore ways to improve the  comparability of program data by using tagging or similar approaches that  allow users to search by key words or terms and combine elements  based on the user’s interests and needs. This merging could help ensure  consistency in the reporting of related program-level spending  information. As mentioned previously, OMB does not expect an update of  program inventories to happen before May 2017.
Other planned changes to the program inventories could also improve the  transparency of their information. For example, OMB staff told us that  they also planned to present the 24 program inventories during the  planned May 2014 update in a more dynamic, web-based format. This  approach, too, has been put on hold. A web-based approach could make  it easier to tag and sort related or similar programs. For instance, OMB  plans to have agencies tag each of their programs by one or more  program type in a future iteration of the inventory to provide a sorting  capability for identifying the same type of program. By providing a sorting  mechanism by program type, OMB could help address one of our open  recommendations, described previously, that OMB work with the PIC to  develop a detailed approach to examine common, long-standing  difficulties agencies face in measuring the performance of various types  of federal programs and activities.
A sorting mechanism could help by identifying (1) all programs in a given  type, and (2) of those programs, any of which have developed strategies  to effectively overcome measurement challenges. Additionally, in its  guidance for the 2014 update before it was put on hold, OMB intended for  agencies to link each program to the existing web pages on  Performance.gov for strategic goals, strategic objectives, APGs, and CAP  goals. According to OMB staff, once they move forward with the next  inventory update and move to a web-based presentation, users will be  able to sort programs by the goals to which they contribute. This  approach also would allow users to identify programs that contribute to  broader themes on Performance.gov. The themes generally align with  budget functions from the President’s Budget and include administration  of justice; general science, space, and technology; national defense; and  transportation, among others. Currently, the themes can be used to sort  goals on Performance.gov that contribute to those broad themes.
Major management challenges. Another area in which our work has  identified problems with transparency and communication of performance  information is related to the GPRAMA requirement that agencies report in  their annual performance plans key performance information related to  their major management challenges, including performance goals,  milestones, indicators, and planned actions that they have developed to  address such challenges. Major management challenges include  programs  or management functions, within or across agencies, that have  greater vulnerability to fraud, waste, abuse, and mismanagement, such  as those issues identified by GAO as high risk, where a failure to perform  well could seriously affect an agency’s or the government’s ability to  achieve its mission or goals. We have ongoing work, which we plan to  issue in late 2015, which is examining how federal agencies are  addressing their major management challenges. As of August 2015, we  found that agencies generally did not report key performance information  about their major management challenges in their annual performance  plans and reports in a transparent manner. For example, while some  agencies told us that they had internal plans for addressing their major  management challenges, 12 of 24 agencies that issued agency  performance plans or similar documents for fiscal year 2015 did not  publicly report planned actions for addressing their major management  challenges. While the reasons for why agencies did not report complete  information varied, such as readability and redundancy with other similar  topics in the performance plan, agencies told us that OMB’s guidance  appeared to give them flexibility on what information they needed to  report. We will provide updated information on major management  challenges in our forthcoming report.
CAP goals. Another area in which transparency is important is in  communicating progress on performance goals, but our June 2014 report  on CAP goal reviews found that the quarterly updates for the 14 interim  CAP goals did not always provide a complete picture of progress. For  each of the CAP goals, GPRAMA requires OMB to coordinate with  agencies to establish annual and quarterly performance targets and  milestones and to report quarterly the results achieved compared to the  targets. The updates we reviewed were inconsistent, and some were  missing key performance information, such as performance targets,  milestone due dates, and key contributors to the goals, that was needed  to track progress toward the goals. In one case, we were told that the  data needed to track progress toward a goal were not available. Staff  from the Real Property interim CAP goal team told us that they did not  have data available for tracking progress toward the goal of holding the  federal real property footprint at its fiscal year 2012 baseline level. In  addition, we found that in some cases information on the organizations  and program types that contributed to an interim CAP goal, such as  relevant tax expenditures, was missing.
We concluded that the incomplete information in many of the updates  provided a limited basis for ensuring accountability for progress toward  targets and milestones for those interim CAP goals and recommended  that OMB take a number of actions to ensure that all key contributors  were identified and that quarterly and overall progress toward CAP goals  could be fully reported. These included identifying all key contributors to  the achievement of the goals; identifying annual planned levels of  performance and quarterly targets for each of the goals; developing plans  to identify, collect, and report data necessary to demonstrate progress  being made toward each of the goals or developing an alternative  approach for tracking and reporting on progress quarterly; and reporting  the time frames for the completion of milestones, the status of milestones,  and how milestones are aligned with strategies or initiatives that support  the achievement of each goal.
As described previously, OMB has increased its emphasis on CAP goal  governance for the current set of CAP goals, and OMB has taken actions  to address concerns our work raised about CAP goal reviews. One of the  actions OMB has taken, together with the PIC, was to develop revised  guidance, in the form of a template, for CAP goal teams to use to report  quarterly progress updates for these goals. This template responded to  three of our recommendations related to CAP goal progress reporting by  including a section for the CAP goal teams to identify programs that  contribute to their goals; directing the teams to list targets for the key  indicators they use to track progress; and directing the teams to establish  work plans with a list of specific milestones that should include milestone  due dates and information on milestone status.
The template also indicated that goal teams can organize milestones by  each identified sub-goal, aligning specific activities with the objectives to  which they contribute. In addition, the PIC provided guidance in January  2015 that further addressed two of our recommendations. The guidance  directs CAP goal teams to report all agencies, organizations, programs,  activities, regulations, tax expenditures, policies, and other activities that  contribute to each goal. It also specifically notes that GPRAMA requires  the teams to report on performance against targets and states that  quarterly progress updates should identify areas where progress has  exceeded expectations or been slower than expected or where targets for  performance measures have been missed. The actions that OMB and the  PIC have taken to address our recommendations have helped to improve  the transparency of the CAP goal progress updates. For example, nearly  all of the quarterly updates released in June 2014 for the second quarter  of fiscal year 2014 included milestone due dates and information on their  status. Many (8 of 15) of the lists of milestones aligned with specific  sub-goals.
Quality of performance information. GPRAMA requirements for  reporting on the quality of performance information also have the potential  to increase transparency, as they require agencies to publicly report on  how they are ensuring the accuracy and reliability of the performance  information they use to measure progress toward APGs and performance  goals. Specifically, for each APG, agencies must provide information  addressing five requirements to OMB for publication on  Performance.gov. Additionally, agencies must address all five  requirements for performance goals, which include APGs.
Our September 2015 report on the quality of publicly reported  performance information found limited information on Performance.gov on  the quality of performance information used to assess progress on six  selected agencies’ 23 APGs. In response to our review, OMB updated  its A-11 guidance in June 2015 to direct agencies to either provide this  information for publication on Performance.gov on how they are ensuring  the quality of performance information for their APGs, or provide a  hyperlink from Performance.gov to an appendix in their performance  report that discusses the quality of their performance information. OMB  staff stated that this information will likely not be available until agencies  start reporting on the next set of APGs (for fiscal years 2016 and  2017). This is because OMB will need to update a template that  agencies complete for their Performance.gov updates.
Further, the agencies we reviewed generally did not describe how they  addressed all five requirements for their individual APGs in their  performance plans and reports. While all six agencies described how they  ensured the quality of their performance information overall, we found that  only DHS’s performance plans and reports included discussions about  performance information quality addressing all five GPRAMA  requirements, as shown in table 3 and described in more detail in the  textbox below.
DHS Addressed GPRAMA Requirements in Explaining How  Performance Information Quality is Ensured for All Agency   Priority Goals  In September 2015, we reported that of the 23 APGs in our sample from  six agencies, we could only find discussions about performance  information quality that addressed all five of the GPRAMA requirements  for three APGs, which belonged to DHS. DHS presented information  about performance information quality for all three of its APGs in its  performance plans and reports. Specifically, DHS published an appendix  to its performance plans and reports with detailed discussions of  performance information quality for 10 performance measures used to  measure progress on these APGs. For each measure, DHS’s appendix  described the related program, the scope of the data, the source and  collection methodology for the data, and an assessment of data  reliability.
In our September 2015 report, we recommended that all six of the  agencies in our review work with OMB to describe on Performance.gov  how they are ensuring the quality of their APGs’ performance information  and that the agencies, except for DHS, also describe this in their annual  performance plans and reports. We also noted that to help improve the  reliability and quality of performance information, OMB and the PIC  established the Data Quality Cross-Agency Working Group in February  2015. The group could serve as a vehicle for disseminating good  practices in public reporting on data quality. As a result, we also  recommended that OMB, working with the PIC, focus on ways the PIC’s  data quality working group can improve public reporting for APGs. OMB  did not comment on the recommendations, but the six agencies generally  concurred or identified actions they planned to take to implement them.
section_title: OMB and Agencies Generally Agreed with GAO’s Prior Recommendations to Improve GPRAMA Implementation, but Most Have Not Yet Been Implemented
paragraphs: Our work examining aspects of GPRAMA implementation and its effects  on agency performance management has identified a number of areas in  which improvements are needed. Since GPRAMA was enacted in  January 2011, we have made a total of 69 recommendations to OMB and  agencies aimed at improving its implementation. OMB and the agencies  have generally agreed with the recommendations we have made thus far,  and have implemented some of them. However, of the 69  recommendations we have made, 55 (about 80 percent) have not yet  been implemented, while 14 recommendations (about 20 percent) have  been implemented. Additional details on these recommendations and  their status are included in appendixes II, III and IV.
We made 21 recommendations to OMB and agencies between May  2012, when we issued our first report on GPRAMA implementation, and  June 2013, when we issued our previous summary report. Fourteen  (about 67 percent) of these recommendations have not been  implemented. Between July 2013 and September 2015, we made 48  additional recommendations to OMB and the agencies. Forty-one (about  85 percent) of these recommendations have not been implemented.  Figure 11 shows the number of recommendations we have made, by  year, and the number that have been implemented.
OMB, which has been the focus of most of our recommendations, has  implemented just over one-third (14) of the 38 recommendations we have  made to it. Because of the agency’s central role in implementing  GPRAMA, we made more recommendations to OMB in our work under  the act than to all other agencies combined. Most of the actions OMB has  taken to implement our recommendations involve updating or issuing new  guidance. Agencies have yet to implement any of the 31  recommendations we have made, although we made most (23) of these  recommendations in reports that we have issued since July 2015.  Specifically, these 23 recommendations were included in our recent work  on data-driven reviews and the quality of performance information, and  they focus on ensuring that agency data-driven review processes and  reporting on the quality of performance information are consistent with  GPRAMA requirements, OMB guidance, and leading practices.
While OMB has implemented some of our recommendations, some of  those that have yet to be implemented focus on long-standing and  significant issues. For example, as described previously, we have made  several recommendations to identify and assess the contributions of tax  expenditures toward executive branch goals, but OMB and agencies have  taken little action to address these recommendations. Additionally, we  have reported that agencies have difficulty measuring the performance of  different program types –such as grants and regulations. We have  identified individual examples of these problems, but our work has also  shown that some areas—such as customer service—are common  problems across multiple agencies. Agencies have not yet implemented  recommendations we made in our October 2014 report on agency  customer service standards.
We have also made numerous recommendations aimed at improving the  effectiveness of various aspects of GPRAMA implementation. These  recommendations focus on a range of areas, including making federal  program inventories more useful, strengthening data-driven review  practices, and improving goal leader accountability mechanisms. As we  have stated, effective GPRAMA implementation has the potential to  improve performance management across government and can help  address crosscutting issues, promote the use of performance information,  increase alignment of performance with results, and improve  transparency. We will continue to monitor OMB’s and agencies’ actions to  implement our recommendations.
section_title: Agency Comments
paragraphs: We provided a copy of this draft report to the Director of the Office of  Management and Budget for its review and comment. On September 18,  2015, OMB staff provided us with oral comments on the report. OMB staff  generally agreed with the information presented in the report, and  provided us with technical clarifications, which we have incorporated as  appropriate.
We are sending copies of this report to interested congressional  committees, the Director of the Office of Management and Budget, and  other interested parties. This report will also be available at no charge on  the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-6806 or mihmj@gao.gov. Contract points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of our report. Key contributors to this report are listed in appendix  VI.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The GPRA Modernization Act of 2010 (GPRAMA) includes a provision for  us to review implementation of the act at several critical junctures and  provide recommendations for improvements to implementation.  Specifically, we are required to evaluate and report on how  implementation of the act is affecting performance management at the  agencies subject to the Chief Financial Officers Act of 1990, as amended,  and to evaluate the implementation of cross-agency priority (CAP) goals,  federal government performance plans, and related reporting by  September 2015. This report pulls together findings from our work  related to the act and on federal performance and coordination issues,  focusing on ongoing work and work issued since our last summary report  on GPRAMA was issued in June 2013, as well as some results from our  work on two ongoing engagements.
Our objectives for this report were to evaluate how GPRAMA  implementation has affected progress in addressing four areas: (1)  crosscutting issues; (2) the extent to which performance information is  useful and used; (3) aligning daily operations with results; and (4)  communication of performance information. To address these objectives,  we reviewed GPRAMA, Office of Management and Budget (OMB)  guidance, and our past and recent work related to managing for results  and the act. We also interviewed OMB and Performance Improvement  Council staff.
Our recent work under GPRAMA, both ongoing and issued since June  2013, covered the 24 CFO Act agencies and the Army Corps of  Engineers-Civil Works. Most (8) of the 12 reports that are the focus of  this report used selected agencies as case illustrations. Half of the 12  reports included government-wide reviews, and in some cases involved  surveys of all or most of the CFO Act agencies.
This report also includes some results from our ongoing work examining  the implementation of CAP goals, which we plan to issue at the end of  2015. We identified lessons learned from the interim CAP goal period,  and we assessed initial progress implementing the current set of CAP  goals. To do this, we selected 7 of the 15 CAP goals for examination,  interviewed officials with responsibility for implementing these goals, and  reviewed relevant guidance and documentation. In order to provide some  insight into both interim and new CAP goals, the team initially randomly  selected 2 of each, resulting in selecting Open Data and STEM  Education, which were also interim CAP goals, and Job-Creating  Investment and Lab-to-Market, which are new CAP goals. Because GAO  did recent work on three additional CAP goals—Customer Service,  People and Culture, and the Smarter IT Delivery—we also selected those  goals. We interviewed OMB and PIC staff responsible for management  and implementation of the current CAP goals and responsible agency  officials, including CAP goal leaders and members of the seven CAP goal  implementation teams. We reviewed OMB and PIC guidance, relevant  documentation, and quarterly progress updates published on  Performance.gov from the second quarter of fiscal year 2014 through the  second quarter of fiscal year 2015, published in June 2015.
This report also reflects some results from our ongoing work on major  management challenges, which we also plan to issue at the end of 2015.  We compared information reported in 24 agency performance plans and  reports against GPRAMA requirements and OMB Circular A-11 guidance  to identify agency activities and reporting related to major management  challenges. We interviewed OMB staff about their guidance related to  major management challenges. We also interviewed 24 CFO Act agency  performance officials, including performance improvement officers,  program offices officials, and, when appropriate, officials from agencies’  Chief Financial Officer and Chief Human Capital Offices to understand  how agencies defined and addressed their major management  challenges.
We conducted this performance audit from April 2015 to September 2015  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Recommendations to OMB from GAO’s Work under the GPRA Modernization Act That Have Been Implemented
paragraphs: Table 4 shows recommendations we have made as part of our work  under the GPRA Modernization Act (GPRAMA) that the Office of  Management and Budget (OMB) has implemented.
section_title: Appendix III: Recommendations to OMB from GAO Work under the GPRA Modernization Act That Have Not Been Implemented
paragraphs: Table 5 shows recommendations we have made to the Office of  Management and Budget (OMB) as part of our work under the GPRA  Modernization Act (GPRAMA) that have not been implemented.
section_title: Appendix IV: Recommendations to Agencies from GAO’s Work under the GPRA Modernization Act That Have Not Been Implemented
paragraphs: Table 6 shows recommendations we have made to agencies as part of  our work under the GPRA Modernization Act (GPRAMA) that have not  been implemented.
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact:
section_title: Staff Acknowledgments
paragraphs: In addition to the above contact, Sarah E. Veale (Assistant Director) and  Kathleen Padulchick supervised this review and the development of the  resulting report. Margaret M. Adams, Shea Bader, Lisette Baylor, Peter  Beck, Elizabeth Curda, Dewi Djunaidy, Deirdre Duffy, Karin Fangman,  Jennifer M. Felder, Farrah Graham, Emily Gruenwald, Jonathan Harmatz,  Jennifer Kamara, Barbara Lancaster, Dainia Lawes, Benjamin T. Licht,  Adam Miles, Michael O’Neill, Lisa Pearson, Steven Putansu, MaryLynn  Sergent, Stephanie Shipman, Matthew Sweeney, and Dan Webb also  made key contributions.
section_title: Related GAO Products
paragraphs: Managing for Results: Greater Transparency Needed in Public Reporting  on the Quality of Performance Information for Selected Agencies’ Priority  Goals. GAO-15-788. Washington, D.C.: September 10, 2015.
DATA Act: Progress Made in Initial Implementation but Challenges Must  be Addressed as Efforts Proceed. GAO-15-752T. Washington, D.C.:   July 29, 2015.
Managing for Results: Practices for Effective Agency Strategic Review.,  GAO-15-602. Washington, D.C.: July 29, 2015.
Managing for Results: Agencies Report Positive Effects of Data-Driven  Reviews on Performance but Some Should Strengthen Practices.   GAO-15-579. Washington, D.C.: July 7, 2015.  2015 Annual Report: Additional Opportunities Exist to Reduce  Fragmentation, Overlap, and Duplication and Achieve Other Financial  Benefits. GAO-15-404SP. Washington D.C.: April 14, 2015.
Fragmentation, Overlap, and Duplication: An Evaluation and  Management Guide. GAO-15-49SP. Washington, D.C.: April 14, 2015.
Government Efficiency and Effectiveness: Opportunities to Reduce  Fragmentation, Overlap, and Duplication and Achieve Other Financial  Benefits. GAO-15-522T. Washington, D.C. April 14, 2015.
High-Risk Series: An Update. GAO-15-290. Washington, D.C.:   February 11, 2015.
Federal Data Transparency: Effective Implementation of the DATA Act  Would Help Address Government-wide Management Challenges and  Improve Oversight. GAO-15-241T. Washington, D.C.: December 3, 2014.
Program Evaluation: Some Agencies Reported that Networking, Hiring,  and Involving Program Staff Help Build Capacity. GAO-15-25.  Washington, D.C.: November 13, 2014.
Government Efficiency and Effectiveness: Inconsistent Definitions and  Information Limit the Usefulness of Federal Program Inventories.   GAO-15-83. Washington, D.C.: October 31, 2014.
Managing for Results: Selected Agencies Need to Take Additional Efforts  to Improve Customer Service. GAO-15-84. Washington, D.C.: October  24, 2014.
Managing for Results: Agencies’ Trends in the Use of Performance  Information to Make Decisions. GAO-14-747. Washington, D.C.:  September 26, 2014.
Managing for Results: Enhanced Goal Leader Accountability and  Collaboration Could Further Improve Agency Performance. GAO-14-639.  Washington, D.C.: July 22, 2014.
Managing for Results: OMB Should Strengthen Reviews of Cross-Agency  Goals. GAO-14-526. Washington, D.C.: June 10, 2014.
Government Efficiency and Effectiveness: Views on the Progress and  Plans for Addressing Government-wide Management Challenges.   GAO-14-436T. Washington, D.C.: March 12, 2014.
Managing for Results: Implementation Approaches Used to Enhance  Collaboration in Interagency Groups. GAO-14-220. Washington, D.C.:  February 14, 2014.
Financial and Performance Management: More Reliable and Complete  Information Needed to Address Federal Management and Fiscal  Challenges. GAO-13-752T. Washington, D.C.: July 10, 2013.
Managing for Results: Executive Branch Should More Fully Implement  the GPRA Modernization Act to Address Pressing Governance  Challenges. GAO-13-518. Washington, D.C.: June 26, 2013.