Managing for Results: Implementation of GPRA Modernization Act Has Yielded Mixed Progress in Addressing Pressing Governance Challenges
Released: 2015-09-30T00:00:00

Letter

Background

GPRAMA is a significant enhancement of GPRA, which was the centerpiece of a statutory framework that Congress put in place during the 1990s to help resolve long-standing performance and management problems in the federal government and provide greater accountability for results. GPRAMA was likewise intended to address a number of federal performance management challenges, including focusing attention on crosscutting issues, enhancing the use and usefulness of performance information, increasing transparency, and ensuring leadership commitment and attention to improving performance. Our June 2013 report assessing the initial government-wide implementation of GPRAMA described some of the important steps OMB and agencies had taken to implement key provisions of GPRAMA. These included developing agency-level and government-wide goals, designating officials to key leadership roles, and using the Performance Improvement Council (PIC) to facilitate the exchange of information to strengthen agency performance management.

GPRAMA revises existing provisions and adds new requirements, including the following: GPRAMA includes requirements that OMB and agencies establish different types of government-wide and agency-level performance goals. These include:

Government-wide: cross-agency priority (CAP) goals. OMB is required to coordinate with agencies to establish federal government priority goals—otherwise referred to as CAP goals—that include outcome-oriented goals covering a limited number of policy areas as well as goals for management improvements needed across the government. The act also requires that OMB—with agencies— develop annual federal government performance plans to, among other things, define the level of performance to be achieved through each of the CAP goals. OMB established the first set of CAP goals for a 2-year interim period in February 2012. In March 2014, OMB identified the next set of CAP goals, which it is to update every 4 years.

Agency-level: agency priority goals (APG). Every 2 years, GPRAMA requires the heads of certain agencies, in consultation with OMB, to identify a subset of agency goals to be identified as APGs. These goals are to reflect the highest priorities of each of these agencies, and to be informed by the CAP goals as well as consultations with relevant congressional committees and other interested parties. Twenty-three agencies identified a total of 91 APGs covering fiscal years 2014 through 2015.

GPRAMA provided a statutory basis for selected senior leadership positions that had been created by executive orders, presidential memorandums, or OMB guidance. GPRAMA established these positions in law, provided responsibilities for various aspects of performance improvement, and elevated some of them as described below.

Chief operating officer (COO). The deputy agency head, or equivalent, is designated COO, with overall responsibility for improving agency management and performance.

Performance improvement officer (PIO). Agencies are required to designate a senior executive within the agency as PIO, who reports directly to the COO and has responsibilities to assist the agency head and COO with performance management activities.

Goal leaders. GPRAMA requires that goal leaders be designated for CAP goals and APGs and OMB guidance requires goal leaders be designated for strategic objectives. They are designated for CAP goals, strategic objectives, and APGs. CAP goals have at least two goal leaders—one from the Executive Office of the President and the other from a key responsible agency. APGs have a goal leader, and OMB guidance directs agencies to designate a deputy goal leader to support the goal leader.

GPRAMA also established the PIC in law and included additional responsibilities. Originally created by a 2007 executive order, the PIC is charged with assisting OMB to improve the performance of the federal government and achieve the CAP goals. Among its other responsibilities, the PIC is to facilitate the exchange among agencies of useful performance improvement practices and work to resolve government-wide or crosscutting performance issues. The PIC is chaired by the Deputy Director for Management at OMB and includes agency PIOs from each of the 24 CFO Act agencies as well as other PIOs and individuals designated by the chair.

GPRAMA and related OMB guidance require the regular review of progress in achieving goals and objectives through performance reviews.

Strategic reviews. OMB’s 2012 guidance implementing GPRAMA established a strategic review process in which agencies, beginning in 2014, were to conduct leadership-driven, annual reviews of their progress toward achieving each strategic objective—the outcome or impact the agency is intending to achieve through its various programs and initiatives—established in their strategic plans (and updated in their annual performance plans).

Data-driven reviews. Data-driven performance reviews are regularly scheduled—at least quarterly—structured meetings used by organizational leaders and managers to review and analyze data on progress toward key performance goals and other management- improvement priorities. For each APG, GPRAMA requires agencies to conduct reviews to assess progress toward the goal and risk of not meeting it, and develop strategies to improve performance, as needed. These reviews are to be led by the agency head and COO, with the support of the PIO, and include relevant goal leaders. Coordination with relevant parties both within and outside the agency that contribute to goal accomplishment is also required. GPRAMA also requires that the Director of OMB, with the support of the PIC, review progress toward each CAP goal with the appropriate lead government official at least quarterly. Specifically, these reviews should examine the progress made over the most recent quarter, overall trends, the likelihood of meeting the planned level of performance and, if necessary, strategies to improve performance.

GPRAMA includes several provisions related to reporting of performance information.

Performance.gov. OMB is required to develop a single, government- wide performance website to communicate government-wide and agency performance information. The website—implemented by OMB as Performance.gov—is required to make available information on APGs and CAP goals, updated on a quarterly basis; agency strategic plans, annual performance plans, and annual performance reports; and an inventory of all federal programs.

Program inventory. OMB is required to make publicly available, on a central government-wide website, a list of all federal programs identified by agencies, along with related budget and performance information.

Performance information quality. Agencies are required to describe how they are ensuring the accuracy and reliability of the data used to measure progress toward APGs and performance goals, including an identification of the following five areas: o The means used to verify and validate ; o the sources for the data; o the level of accuracy required for the intended use of the data; o any limitations to the data at the required level of accuracy; o how the agency will compensate for such limitations (if needed) to reach the required level of accuracy.

Agencies are required to provide information to OMB that addresses all five requirements for each of their APGs for publication on Performance.gov. Agencies also must address all five requirements for performance goals in their performance plans and reports.

Major management challenges. Agencies are required to address major management challenges in their performance plans. These challenges may include programs or management functions that have greater vulnerability to fraud, waste, abuse, and mismanagement, such as those issues included in our high-risk list or identified by inspectors general, where a failure to perform well could seriously affect an agency’s ability to achieve its mission or goals.

The concepts described above and their relationships to each other are represented in figure 2, which summarizes them and highlights areas in which our recent work has focused.

The Executive Branch Needs to Take Additional Actions to Address Crosscutting Issues, but OMB Has Increased Emphasis on Governance of Cross-Agency Priority Goals

OMB and Agencies Continue to Miss Opportunities to Address Crosscutting Issues

Many of the meaningful results that the federal government seeks to achieve, such as those related to protecting the environment, promoting public health, and providing homeland security, require the coordinated efforts of more than one federal agency, level of government, or sector. Even with sustained leadership, crosscutting issues are difficult to address because they may require agencies and Congress to reexamine (within and across various mission areas) the fundamental structure, operation, funding, and performance of a number of long-standing federal programs or activities. Collaboration and improved working relationships across agencies are critical tools for addressing the issues of fragmentation, overlap, and duplication our recent work has highlighted. Additionally, they are fundamental to addressing many of the issues that we have designated as high risk due to their vulnerabilities to fraud, waste, abuse, and mismanagement or most in need of broad-based transformation. We have found that resolving many of these issues requires better collaboration among agencies, levels of government, and sectors.

For more than two decades, we have reported on agencies’ missed opportunities for improved collaboration through the effective implementation of GPRA and, more recently, GPRAMA. Now, more than 20 years since GPRA’s passage, our work continues to demonstrate that the needed collaboration is not sufficiently widespread. The examples in the textbox below show areas from our high-risk list–improving and modernizing federal disability programs and improving federal oversight of food safety–that demonstrate the need for greater collaboration on crosscutting issues.

Examples of 2015 High-Risk Areas Demonstrating the Continued Need to Address Crosscutting Issues

Federal Disability Programs Remain Fragmented and a High-Risk Federal disability programs across government remain fragmented and in need of modernization. Numerous federal programs provide a patchwork of services and supports to people with disabilities and work independently without a unified vision and strategy or set of goals to guide their outcomes. Our 2015 update to our High-Risk Series found that progress in improving and modernizing disability programs has been mixed. OMB has made some progress toward enhancing coordination across programs that support employment for people with disabilities, but it has not established a larger vision for disability programs that include appropriate government-wide goals and strategies for achieving those goals. OMB needs a government- wide action plan that describes how federal agencies will improve coordination and set measurable goals that support employment for people with disabilities beyond the public sector. Such a plan should identify additional opportunities to build capacity and leverage existing government resources. Continued planning, management focus, and coordination can improve and modernize federal disability programs.

Federal Food Safety Is a High-Risk Area and in Need of Improved Foodborne illness is a common, costly, yet largely preventable public health concern. According to the Centers for Disease Control and Prevention, each year nearly 50 million people in the United States get sick and roughly 3,000 die due to foodborne illness. For more than a decade, we have reported on the fragmented federal food safety system and we added federal oversight of food safety to our high-risk areas because of risks to the economy and public health and safety. In December 2014 we reported that the Department of Health and Human Services (HHS) and the U.S. Department of Agriculture (USDA) have taken steps to implement GPRAMA requirements, but could more fully address crosscutting food safety efforts. HHS and USDA vary in the amount of detail they provide on their crosscutting food safety efforts and they do not include several relevant crosscutting efforts in their strategic and performance planning documents. HHS and USDA have mechanisms in place to facilitate interagency coordination on food safety that focus on specific issues, but they do not provide for broad-based, centralized collaboration. A centralized collaborative mechanism on food safety is important to foster effective interagency collaboration and enhance food safety oversight. We recommended that HHS and USDA build upon their efforts to implement GPRAMA requirements to fully address crosscutting food safety efforts. We asked Congress to consider directing OMB to develop a government-wide food safety performance plan and formalize the Food Safety Working Group through statute to help ensure sustained leadership across food safety agencies over time. HHS and USDA agreed with our recommendation and in February 2015, HHS updated its strategic plan to more fully describe how it is working with other agencies to achieve its food-safety-related goals and objectives. As of August 2015, our recommendation to USDA and matters for congressional consideration remain unimplemented.

Our annual reports on areas where opportunities exist for executive branch agencies or Congress to reduce, eliminate, or better manage fragmentation, overlap, or duplication; to achieve cost savings; or to enhance revenue have also included areas in which greater collaboration is needed to address crosscutting issues. See textbox below.

Examples of Crosscutting Issues Identified in GAO’s 2015 Annual Report on Fragmentation, Overlap, and Duplication

Fragmentation, Overlap, and Potential for Duplication Exists among Nonemergency Medical Transportation Programs Access to transportation services is essential for millions of Americans to fully participate in society and access human services, including medical care. Our April 2015 report on opportunities to reduce fragmentation, overlap, and duplication reported that 42 programs across six federal departments provide funding for nonemergency medical transportation (NEMT) to individuals who cannot provide their own transportation due to age, disability, or income constraints. Coordination of NEMT programs at the federal level is limited and there is fragmentation, overlap, and potential for duplication across these programs. An interagency coordinating council was developed to enhance federal, state, and local coordination activities, and it has taken some actions to address program coordination. However, the council has provided limited leadership and has not convened since 2008. To improve efficiency, we recommended that the Department of Transportation, which chairs the council, take steps to enhance coordination among the programs that provide NEMT. The department agreed that more work is needed, and said that the Federal Transit Administration is asking its technical assistance centers to assist in developing responses to NEMT challenges. In addition, as of June 1, 2015, the Federal Transit Administration reported working to develop a new 2-year strategy for addressing NEMT coordination among federal agencies and plans to develop and propose a cost-sharing model that can be applied to federal programs that provide funding for NEMT.

Strengthened Coordination Could Increase Efficiency and Effectiveness of Consumer Product Safety Oversight The oversight of consumer product safety is a complex system involving a number of federal agencies. However, as our April 2015 report on opportunities to reduce fragmentation, overlap, and duplication highlighted, oversight of consumer product safety is fragmented across agencies, overlaps jurisdictions, or is unclear for certain products. In some cases agencies regulate different components of or carry out different regulatory activities for the same product. Agencies reported that they collaborate to address specific consumer product safety topics. For example, officials from the Coast Guard, which regulates safety standards for recreational boats, said they work informally with the Consumer Product Safety Commission when the need arises. However, we did not identify a formal mechanism for addressing such issues more comprehensively, and no single entity or mechanism exists to help the agencies that collectively oversee consumer product safety. Without this, agencies may miss opportunities to leverage resources and address challenges, including those related to fragmentation and overlap. In response to our recommendation that the Coast Guard and the Consumer Product Safety Commission establish a formal coordination mechanism, in May 2015 the two agencies signed a formal policy document establishing such a mechanism. We also recommended that Congress should establish a formal collaboration mechanism to address comprehensive oversight and inefficiencies related to fragmentation and overlap. As of August 2015, no formal collaboration mechanism had been established.

The textbox below shows additional examples of areas in which we have identified the need for additional work to address crosscutting issues.

Examples from GAO’s Work from 2013-2015 of Continued Challenges in Addressing Crosscutting Issues

Additional Leadership Needed to Achieve Interagency Efforts for Department of Agriculture and Department of Health and Human Services (HHS) veterinarians perform critical work for public and animal health and for emergency response to economically devastating or highly contagious animal diseases. In May 2015, we reported that the Office of Personnel Management (OPM) and other federal agencies have taken steps toward achieving the goals outlined in OPM’s government-wide strategic plan for the veterinarian workforce, primarily through an interagency group OPM created. However, in each of the three goals, the group did not follow through on next steps and made limited progress. According to OPM officials, the group did not consistently monitor progress toward goals in part because it did not have sufficient leadership support from participating agencies. OPM agreed with our recommendation that it obtain leadership support for achieving its goals, and stated that it designed and will aid in establishing a Veterinary Medical Officer Executive Steering Committee that will, among other things, provide leadership and ensure progress toward stated goals.

Federal Strategy Needed to Ensure Efficient and Effective Delivery of Services for Older Adults In May 2015, we reported that five federal agencies across four departments had one or more programs that operate within a system of home and community-based services (HCBS) and related supports that older adults often require to live as independently as possible in their homes and communities. The Older Americans Act of 1965 requires the Administration on Aging, within HHS, to facilitate collaboration among federal agencies; however, the five agencies that fund these services and supports do so, for the most part, independently. To help ensure that agencies’ resources for HCBS and supports are used efficiently and effectively, we recommended that HHS facilitate development of a cross-agency federal strategy. HHS agreed with our recommendation.

While much of our recent work has focused on the need for improved collaboration to address crosscutting issues, we have also reported on areas, including one high-risk area, in which agencies have made progress or are generally effectively coordinating. The text box below discusses two of these examples.

Examples from GAO’s Work from 2013-2015 of Areas in Which Agencies Are Doing Well or Making Progress in Addressing Crosscutting Issues

Coordination of DOD’s and NNSA’s Nuclear Weapons Stockpile Responsibilities Is Generally Consistent with Key Practices The Nuclear Weapons Council (Council) serves as the focal point of Department of Defense (DOD) and National Nuclear Security Administration (NNSA) interagency activities to maintain the U.S. nuclear weapons stockpile. In May 2015, we reported that the Council’s actions to coordinate DOD’s and NNSA’s nuclear weapons stockpile responsibilities are generally consistent with most of the key practices we have identified for collaborating across agency boundaries. For example, according to Council documents, the Council and its support committees meet on a regular basis to monitor, evaluate, and report on nuclear weapons stockpile issues. These meetings include periodic oversight briefings on nuclear weapon refurbishment programs. We made recommendations to the Secretaries of Defense and Energy to address two areas in which actions could be enhanced: (1) having up-to-date, written agreements and guidance that establish compatible policies, procedures, and other means to operate across agency boundaries and defines roles and responsibilities and (2) regularly including all relevant participants. The departments generally agreed with our recommendations.

Progress Made on High-Risk Area of Sharing and Managing The federal government has made significant progress in promoting the sharing of information on terrorist threats, an area we designated as high risk in 2005. In February 2015, we reported that significant progress was made in this area by developing a more structured approach to achieving the Information Sharing Environment (Environment) and by defining the highest priority initiatives to accomplish. In December 2012, the President signed the National Strategy for Information Sharing and Safeguarding (Strategy), which provides guidance on the implementation of policies, standards, and technologies that promote secure and responsible national security information sharing. In 2013, in response to the strategy, the Program Manager for the Environment released the Strategic Implementation Plan for the National Strategy for Information Sharing and Safeguarding (Implementation Plan). The Implementation Plan provides a roadmap for the implementation of the priority objectives in the Strategy, assigns stewards to coordinate each priority objective, and provides time frames and milestones for achieving the outcomes in each objective. The steward is responsible for ensuring that participating agencies communicate and collaborate to complete the objective, while also raising to senior management any issues that might hinder progress. Although progress has been made, more work remains to be done to fully address the issues identified in this high- risk area.

More Effective Implementation of GPRAMA and the DATA Act Would Help Address Crosscutting Issues

If fully and effectively implemented, GPRAMA and the Digital Accountability and Transparency Act of 2014 (DATA Act) hold promise for helping to address crosscutting issues. For example, GPRAMA establishes a framework aimed at taking a more crosscutting and integrated approach to focusing on results and improving government performance. Effective implementation of GPRAMA could help clarify desired outcomes, address program performance spanning multiple organizations, and facilitate future actions to reduce, eliminate, or better manage fragmentation, overlap, and duplication. The DATA Act also offers the potential to help address crosscutting issues, as it requires agencies to publicly report information about any funding made available to, or expended by, an agency. These actions would allow executive branch agencies and Congress to accurately measure the costs and magnitude of federal investments. As we have previously reported, the DATA Act holds great promise for improving the efficiency and effectiveness of the federal government and for addressing persistent government management challenges. For example, as our annual reports on fragmentation, overlap, and duplication have highlighted, a complete picture of federal programs, along with related funding and performance information, is critical for addressing these issues.

Data-driven reviews. Data-driven reviews have had a positive effect on collaboration among officials within agencies, but agencies are still missing opportunities to include stakeholders from other federal agencies and thus promote collaboration across agencies. Specifically, in our July 2015 report on data-driven reviews, we found that 21 of the 22 agencies we surveyed that reported holding in-person data-driven reviews said that their data-driven reviews have had a positive effect on collaboration among officials from different offices or programs within the agency.

Despite the positive effects of reviews on internal collaboration, most agencies reported that relevant contributors from other federal agencies did not participate in their reviews. This situation has not changed since our July 2014 report on the role of the agency priority goal leader, in which we found that some goal leaders reported that goal contributors from other federal agencies, and even different components within the same federal agency, were not included in their data-driven reviews.

As we previously reported in 2013, failing to include all goal contributors may lead to missed opportunities to have all the relevant parties apply their knowledge of the issues and participate in developing solutions to performance problems. As a result, in that 2013 report, we recommended that OMB work with the PIC and other relevant groups to identify and share promising practices to help agencies extend their performance reviews to include, as relevant, representatives from outside organizations that contribute to achieving their agency performance goals. As of June 2015, OMB had not taken action in response to this recommendation. OMB staff said that while agencies have found that at times it is useful to engage external stakeholders in improving program delivery, officials view data-driven reviews as internal agency management meetings and believe it would not always be appropriate to regularly include external representatives. We continue to believe that more active involvement from external contributors is needed, as appropriate, and continue to urge OMB to implement our recommended actions.

Strategic reviews. Effective implementation of strategic reviews could help identify opportunities to reduce, eliminate, or better manage instances of fragmentation, overlap, and duplication because agencies are to identify the various organizations, program activities, regulations, tax expenditures, policies, and other activities that contribute to each objective, both within and outside the agency. Where progress in achieving an objective is lagging, the reviews are intended to identify strategies for improvement, such as strengthening collaboration to better address crosscutting challenges or using evidence to identify and implement more effective program designs. If successfully implemented in a way that is open, inclusive, and transparent—to Congress, delivery partners, and a full range of stakeholders—this approach could help decision makers assess the relative contributions of various programs to a given objective.

Successful strategic reviews could also help decision makers identify and assess the interplay of public policy tools that are being used to ensure that those tools are effective and mutually reinforcing and that results are being efficiently achieved. To that end, in July 2015 we reported on seven practices that can help ensure agencies conduct effective strategic reviews. These practices include identifying the various strategies and other factors that influence outcomes and determining which are most important, identifying key stakeholders to participate in the review, and assessing the effectiveness in achieving strategic objectives and identifying actions to improve implementation and impact.

Program inventories. One of the GPRAMA provisions that has the potential to help in addressing crosscutting issues is the requirement that agencies develop inventories of their programs, though in October 2014 we reported that several issues limit the usefulness of the inventories. As our prior work has highlighted, creating a comprehensive list, or program inventory, of federal programs, along with related performance and funding information, could provide decision makers with critical information that could be used to better address crosscutting issues. However, in developing the inventory OMB allowed agencies to define their programs using different approaches, but within a broad definition of what constituted a program consistent with several characteristics. Moreover, OMB’s guidance presents five possible approaches agencies could take to define their programs and notes that agencies could use one or more of those approaches in doing so. As a result, we found that the use of inconsistent approaches by agencies to define their programs limits the comparability of programs within agencies as well as government-wide.

To illustrate the shortcomings of the inventory, in our report on program inventories we compared relevant agencies’ inventories for various science, technology, engineering, and mathematics education and nuclear nonproliferation programs to programs identified in our past work. We were unable to identify in the inventories a large majority of the programs previously identified in our work: 9 of the 179 programs matched exactly and 51 others were identified based on program descriptions. According to OMB staff, agencies used different approaches for valid and legitimate reasons and a one-size-fits-all approach would not work for all agency inventories. While this may be true, OMB could do more to direct agencies to find common ground on similar programs. One of OMB’s stated purposes for the inventories is to facilitate coordination across programs that contribute to similar outcomes. However, as we discovered through our interviews with agency officials involved with the inventory efforts, none of the agencies sought input from other agencies on how they defined and identified their programs. We concluded that if agencies worked together to more consistently define their programs, it could also help them identify where they have programs that contribute to similar outcomes, and therefore have opportunities to collaborate.

We made several recommendations to OMB aimed at presenting a more coherent picture of all federal programs in agency inventories, but OMB has not yet taken action to address these items. OMB planned to publish updated inventories in May 2014. However, OMB put the plans for updating the inventories on indefinite hold and agencies have not published updated inventories with program-level budget information, in part due to the enactment of the DATA Act. OMB staff told us that they are considering how implementation of DATA Act requirements can be tied to the program inventories. Agency reporting for both sets of requirements is web-based, which could more easily enable linkages between the two or facilitate incorporating information from each other. The House and Senate versions of the Taxpayers Right-to-Know Act would require that program inventories also include, to the extent practicable, financial information required to be reported under the DATA Act for each program activity. If enacted, the Taxpayers Right-to-Know Act could result in detailed financial and performance information for federal programs, all in one place.

In our July 2015 testimony on the implementation of the DATA Act, we recommended that OMB accelerate efforts to determine how best to merge DATA Act purposes and requirements with the GPRAMA requirement to produce a federal program inventory. OMB and Treasury did not comment on this recommendation. However, the Acting Deputy Director for Management and Controller at OMB stated at the July 2015 hearing that, because the staff that would be involved in working on the program inventories were heavily involved in DATA Act implementation, he would not expect an update of the program inventories to happen before May 2017.

OMB Has Increased Emphasis on Crosscutting Issues through Cross- Agency Priority Goal Guidance and Governance

Cross-Agency Priority Goals Focus Attention on Crosscutting Issues

GPRAMA’s provisions for establishing and managing achievement toward cross-agency priority (CAP) goals make up another area in which the act offers the potential to address crosscutting issues. CAP goals, which GPRAMA requires OMB to develop in coordination with agencies, are intended to cover areas where increased cross-agency collaboration is needed to improve progress toward shared, complex policy or management objectives, such as improving our nation’s cybersecurity. OMB established the first set of CAP goals for a 2-year interim period in February 2012. In March 2014, OMB released the next set of CAP goals, which it will update every 4 years, as required by GPRAMA. OMB is required to coordinate with agencies to publish progress updates on Performance.gov on a quarterly basis for each CAP goal. Of the 15 current goals, 7 are mission-oriented goals and 8 are management- focused goals (see figure 3).

When selecting the current set of CAP goals, OMB staff told us they considered factors such as the administration’s priorities, GPRAMA requirements, and our prior work. According to documents we reviewed and agency officials we spoke with, OMB also consulted with government-wide councils, agencies, and congressional committees when developing potential CAP goals. For example, OMB staff told us that they added the Insider Threat and Security Clearance Reform CAP goal based on congressional input. We are conducting an ongoing assessment of the current set of CAP goals, shown in figure 3 above, and selected 7 goals for our review. The objectives of this review are to (1) assess the extent to which lessons learned from implementing the interim CAP goals were incorporated into the governance of the current CAP goals; (2) assess the extent to which GPRAMA requirements for assessing and reporting on CAP goal progress are included in the selected CAP goal quarterly progress updates; and (3) assess the initial progress in implementing the selected CAP goals. We plan to issue this work at the end of 2015 and will provide updated information on selected CAP goals’ progress at that time.

CAP goal leaders or their teams from each of the seven selected CAP goals told us that the CAP goal designation led to increased emphasis and leadership attention within their agencies and the Executive Office of the President (EOP) on the CAP goal area.

OMB Is Taking Steps to Enhance the Governance and Implementation of Current CAP Goals

OMB has made several improvements to its CAP goal guidance, in part in response to our prior work, which found that OMB should strengthen CAP goal reviews. Our June 2014 report on CAP goal reviews found, among other things, that all reviews did not meet leading practices for leadership involvement, participation by key officials, and follow-up. In response to our recommendations, OMB and the PIC took several actions, including updating the guidance to CAP goal teams and outlining the role of OMB leadership and the PIC. For example, the guidance specifies that OMB’s Deputy Director for Management will chair implementation-focused meetings for the 8 management CAP goals approximately three times a year and OMB’s Deputy Director for Budget will chair meetings for the 7 mission-focused CAP goals, as necessary. OMB staff confirmed that as of August 2015, such meetings had been held for 11 of the 15 CAP goals. OMB staff also told us that regular senior-level meetings, consistent with OMB’s guidance for CAP goal reviews, also took place for 3 additional goals—Cybersecurity, Infrastructure Permitting Modernization, and Insider Threat and Security Clearance. No such reviews have yet taken place for the Climate Change CAP goal, according to OMB staff.

OMB also changed the CAP goal governance structure to build capacity for goal implementation. For the interim CAP goal process, each CAP goal was assigned a goal leader from the EOP. For the current CAP goals, in addition to the EOP goal leader, OMB assigned a co-goal leader from key agencies to jointly manage and oversee the goal. For example, the Customer Service CAP goal leaders are OMB’s Associate Director for Personnel and Performance and the Acting Commissioner of the Social Security Administration (SSA). According to OMB, this new governance structure reflects agency leadership and expertise in CAP goal subject areas and more effectively leverages agency resources for crosscutting efforts and to promote greater coordination across multiple agencies. For example, the OMB goal leader for the Customer Service goal told us that it is helpful to have SSA in a leadership role because the agency provides its perspective on implementation efforts related to improving customer service, including piloting new activities before they are implemented government-wide. According to the second quarterly update of fiscal year 2015, SSA helped OMB to launch a pilot for a customer service regional community of practice in Denver, Colorado, to help field staff work across agencies and identify opportunities for joint trainings and joint recruiting efforts.

We found that OMB and the PIC have also implemented strategies intended to build agency capacity to work across agencies. These strategies include:

Providing ongoing guidance and assistance to CAP goal teams. The seven CAP goal teams we spoke with told us that OMB and the PIC staff are available on a regular basis to provide them with ongoing support, such as assisting with the regular collection of performance data and updating Performance.gov. For example, the Science Technology Engineering and Mathematics (STEM) Education CAP goal leader from the National Science Foundation told us the PIC facilitated meetings to assist the goal team in developing milestones and performance indicators and to define actionable next steps.

Developing a template to enhance reporting and management.

OMB provided CAP goal teams with a simplified reporting template to use for managing implementation of the goal and to meet quarterly reporting requirements on Performance.gov. According to OMB staff and the seven CAP goal teams we spoke with, the new reporting template makes it easier for goal leaders to review the updates and track progress from one quarter to the next.

Piloting a government-wide White House leadership development program. In December 2014, the President announced a White House leadership development program which is designed to provide selected civil servants (i.e., GS-15 level or equivalent) with rotational assignments across agencies to focus on managing CAP goals. According to OMB staff, the program will begin in October 2015 and the participants will spend the next year helping the White House and agencies work on implementing the CAP goals.

CAP Goal Teams Reported Some Initial Progress and Are Developing Performance Indicators

During the first year of implementation of the current set of CAP goals, CAP goal teams reported initial progress to the extent performance data were available. For example, in its progress update for the second quarter of fiscal year 2015, published in June 2015, the Open Data CAP goal team reported an increase in the use of open government data as indicated by a 25 percent quarterly increase in the number of visits to Data.gov, the government’s platform for publishing its data. The goal team has identified this indicator as a way to measure progress toward one of their goals to fuel economic growth and innovation.

However, a few of the CAP goal teams we spoke with told us that in some cases performance data are not always available and developing meaningful performance indicators to assess progress is a challenge. Our June 2014 assessment of the interim CAP goal process found that 6 of the 14 interim CAP goals did not report performance data, in some cases because data needed to assess and report progress toward the goals were unavailable. As a result, we recommended that OMB direct CAP goal leaders to develop plans to identify, collect, and report data necessary to demonstrate progress being made toward each goal. OMB and the PIC updated guidance directing CAP goal teams to establish performance targets and report on any performance indicators that are under development.

According to the second quarterly update for fiscal year 2015, 5 of the 7 CAP goals we reviewed have indicators under development for some of their goals. For example, the Customer Service CAP goal team reported that they are developing a standardized performance indicator to measure improvements in citizen satisfaction across government, but the progress update does not provide any information on intermediate deliverables, roles and responsibilities, or time frames for completion. On the other hand, another progress update we examined did include information on steps the CAP goal team is taking to develop a government-wide performance indicator. The STEM Education CAP goal team reported that one of its working groups is developing common evaluation elements to be used across federal agencies, with an expected completion date in early 2016. The goal team also provided information on currently available data, near-term and long-term steps they are taking, and additional research needs. Our June 2014 recommendation remains open because OMB’s updated guidance did not direct CAP goal teams to report on the steps they are taking to develop indicators and associated time frames.

We have previously reported that tracking and monitoring progress for cross-agency activities is difficult for a number of reasons such as competing mission priorities, incompatible processes and systems across agencies, resources, and staffing. Given these challenges, when developing performance indicators for government-wide activities, it is important that CAP goal teams provide information on the steps they plan to take to successfully develop meaningful indicators, which will enable them to better track progress over time and hold contributors accountable for implementation.

The Executive Branch Has Still Not Taken Action to Assess the Outcomes of Tax Expenditures

For over 20 years, we have recommended greater scrutiny of the performance of tax expenditures—reductions in a taxpayer’s tax liability that are the result of special exemptions and exclusions from taxation, deductions, credits, deferrals of tax liability, or preferential tax rates. If the Department of the Treasury’s (Treasury) estimates are summed, approximately $1.2 trillion in revenue was forgone from the 169 tax expenditures reported for fiscal year 2014, nearly the same as discretionary spending that year. In June 1994 and again in September 2005, we recommended that OMB develop a framework for reviewing tax expenditure performance. Periodic reviews could help determine how well specific tax expenditures work to achieve their stated purposes and how their benefits and costs compare to those of spending programs with similar goals. Given the significant investment tax expenditures represent, such reviews could help identify the most effective approaches for achieving results—vital information for federal decision makers in an era of scarce resources. Despite the strong case to evaluate the performance of tax expenditures, OMB has not yet developed a framework for doing so.

Fully implementing GPRAMA requirements could provide the foundation of such a framework. GPRAMA requires OMB to identify tax expenditures that contribute to the CAP goals. In addition, since 2012, OMB’s guidance has directed agencies to identify tax expenditures that contribute to their APGs. Our past work reviewing initial GPRAMA implementation in 2012 and 2013 found that OMB and agencies rarely identified tax expenditures as contributors to these goals. As a result, we made several recommendations to improve efforts to identify and assess the contributions of tax expenditures toward executive branch goals.

To date, OMB and agencies have taken little action to address these recommendations. OMB has directed agencies, beginning with its 2013 update to its guidance, to identify tax expenditures that contribute to each of their strategic objectives, in response to a recommendation we made in June 2013. However, our work reviewing GPRAMA implementation continues to find that OMB and agencies have not adequately identified the contributions of tax expenditures to CAP goals and APGs. For example, we found in June 2014 that although the goal leader for the Broadband interim CAP goal told us he was aware that tax deductions available to businesses making capital investments contributed to the goal by incentivizing investments in broadband, the deductions were not identified as contributors on Performance.gov.

Given their government-wide purview and familiarity with administering the tax code, OMB and Treasury, respectively, are well positioned to assist agencies in identifying tax expenditures that relate to their goals. To that end, OMB’s 2013 and 2014 Circular A-11 guidance noted that it would work with Treasury and agencies to identify where tax expenditures align with their goals and this information was to be published on Performance.gov and included in relevant agency plans, beginning in February 2014. However, as we found in October 2014, according to OMB staff, they did not begin to engage Treasury on this effort until after agency plans were published and the website was updated. OMB staff told us in August 2015 that they had not yet made any progress on this effort. Moreover, OMB removed the language about working with Treasury and agencies to align tax expenditures with agency goals in the June 2015 update to its guidance. Although OMB staff told us they intend to focus on this effort, they did not provide us with any plans or time frames for doing so.

We have previously identified additional steps OMB could take to help agencies consider the contributions of tax expenditures to the achievement of their goals. We recommended in our October 2014 report on the federal program inventory required under GPRAMA that OMB should include tax expenditures. The federal program inventory is the primary tool for agencies to identify programs that contribute to their goals, according to OMB’s guidance. By including tax expenditures in the inventory, OMB could help ensure that agencies are properly identifying the contributions of tax expenditures to the achievement of their goals. OMB staff neither agreed nor disagreed with these tax expenditure recommendations, and told us that until they had firmer plans on how program inventory and DATA Act implementation would be merged, they could not determine if implementing these recommendations would be feasible. As previously described, program inventory implementation remains on hold and OMB has not taken any actions to address these recommendations, according to OMB staff. Without including tax expenditures in the inventory, OMB forgoes an important opportunity to increase the transparency of tax expenditures and the outcomes to which they contribute.

Ensuring Performance Information Is Useful and Used by Managers Remains a Challenge, but OMB and Agencies Are Implementing Processes That May Lead to Improvements

Agencies Continue to Face Challenges in Using Performance Information for Decision Making

We have long reported that agencies are better equipped to address management and performance challenges when managers effectively use performance information for decision making. Unfortunately, agencies continue to struggle to do so. Our work has found that federal agencies can use performance information to identify performance improvement opportunities, improve program implementation and organizational processes, and make other important management and resource allocation decisions.

However, our recent work shows that agencies continue to have problems effectively using performance information. Our September 2014 report on trends in agencies’ use of performance information compared agencies’ reported use of performance information from our 2007 and 2013 federal managers surveys. To analyze use, we developed a use of performance information index, based on a set of survey questions in both surveys that reflected the extent to which managers reported that their agencies used performance information for various management activities and decision making. The index runs from 1 to 5, where a 1 reflects that managers feel the agency engages “to no extent” and a 5 reflects that managers feel the agency engages “to a very great extent” in the use of performance information activities. Most agencies showed no statistically significant change in use during this period. As shown in figure 4, only two agencies experienced a statistically significant improvement in the use of performance information, while four agencies experienced a statistically significant decline.

We have previously reported that in order for performance information to be useful, it must have certain characteristics. Specifically, agencies should ensure that performance information meets various users’ needs for completeness, accuracy, consistency, timeliness, validity, and ease of use. Without complete and reliable performance information, Congress, other decision makers, and stakeholders at all levels of government are hampered in their ability to set priorities, identify improvement opportunities, and allocate resources.

Our work over the past 2 years has identified weaknesses in each of the areas that affect the usefulness of performance information. For example, our September 2015 report on affordable rental assistance programs identified an incomplete picture of performance of these programs as a problem. We found that federal, state, and local jurisdictions involved in these efforts reported their performance to varying extents, but that there was incomplete information on their collective performance. Accordingly, we recommended that the Department of Housing and Urban Development (HUD), in consultation with an interagency working group on rental policy, should work with states and localities to develop an approach for compiling and reporting on the collective performance of federal, state, and local rental assistance programs. Treasury and IRS, which are agency members of this working group, did not comment on this recommendation. HUD was concerned that compiling and reporting collective performance information would require significant funding and resources. We continue to believe the overall recommendation is valid. Specifically, we noted that (1) our recommendation is to develop an approach for compiling and reporting such data as a first step, and (2) that our recommendation is purposefully not prescriptive and allows HUD, in consultation with the working group, to design an approach. Additional examples of problems that affect the usefulness of performance information are illustrated in table 1.

Agency Implementation of Performance Reviews Should Improve the Use of Performance Information for Decision Making

Performance reviews required under GPRAMA and other guidance by their nature promote the use of performance information, as they focus on assessing performance in order to determine progress toward meeting goals and objectives.

Data-driven reviews. GPRAMA requires that reviews of progress of agency priority goals (APG) be held at least quarterly; as this requirement is more fully implemented, the use of performance information for decision making should improve. OMB emphasized that frequent, data- driven performance reviews provide a mechanism for agency leaders to use data to assess the organization’s performance, diagnose performance problems and identify improvement opportunities, and decide on next steps to improve performance. These practices are designed to shift the emphasis away from the passive collection and reporting of performance information to a model where performance information is actively used by agency officials to inform decision-making, which is more likely to lead to performance improvements.

In our July 2015 report on data-driven reviews, we found that PIOs had reported that the reviews had positive effects on their agencies’ use of performance information. Nearly all of the 22 agencies that reported holding in-person reviews responded that they always or often use their review meetings to assess progress on APGs and to identify goals at risk and strategies for performance improvement. Additionally, as shown in figure 5, nearly all of these agencies also reported that their data-driven review meetings have had a positive effect on progress toward the achievement of their agency’s goals and on their ability to identify and mitigate risks to goal achievement.

In our discussions with officials from selected agencies, data-driven review meetings were described as venues for agency leaders and managers to assess progress toward key goals and milestones, the status of ongoing initiatives and planned actions, potential solutions for problems or challenges hindering progress, and additional support or resources needed to improve performance. Agency officials emphasized that discussions in their review meetings tend to focus on those goals or issues most in need of attention, where the achievement of a goal or milestone is at risk. In this way, reviews can serve as early warning systems and facilitate focused discussions on external, technical, or operational obstacles that may be hindering progress and the specific actions that should be taken to overcome them (see sidebar on the following page).

Increasing Online Registration through my Social Security The Social Security Administration (SSA) has an APG to increase the number of registrations for its “my Social Security” portal by 15 percent per year in fiscal years 2014 and 2015. However, we reported in July 2015 that during SSA’s 2014 third quarter review meeting, it became apparent to SSA leadership that the agency was not on track to achieve its target for this goal. SSA shifted focus to what could be done by offices throughout the agency to support efforts to increase the number of registrations using currently available or attainable resources and technology. To achieve this, SSA leadership had different offices within the agency specify the contributions they would make to help increase the number of registrations. Since then, the agency’s quarterly review meetings have been used to review and reinforce the commitments each office made. While SSA was unable to meet the registration goal for fiscal year 2014, according to SSA officials, these efforts recently undertaken as a result of the review process have helped generate an increase in registrations. Data from SSA’s fiscal year 2015 first quarter review show that there was a 46 percent increase in new account registrations in October 2014 compared to the number of new registrations in October 2013, and a 26 percent increase in December 2014 relative to December 2013. performance reviews. Reexamining the benefits and costs achieved after a regulation is implemented could provide useful data for these reviews. Despite the potential to leverage retrospective review information, agencies reported mixed experiences linking retrospective analyses to APGs. Agencies typically selected rules to review based on criteria such as the number of complaints or comments from regulated parties and the public. Including whether a regulation contributed to an APG as one of these criteria would help agencies prioritize retrospective analyses that could contribute useful information to APG assessments. We recommended that OMB’s Office of Information and Regulatory Affairs direct in guidance that agencies take actions to ensure that contributions made by regulations toward the achievement of APGs are properly considered and improve how retrospective regulatory reviews can be used to help inform assessments of progress toward these APGs. OMB staff agreed with this recommendation and stated that the agency was working on strategies to help facilitate agencies’ ability to use retrospective reviews to inform APGs, but as of June 2015 they have not provided additional details on their actions.

Strategic reviews. Like data-driven reviews of APGs, and retrospective reviews of regulations, agencies’ annual strategic reviews also have potential to increase the use of performance information. For example, we reported in July 2015 that, to ensure effective strategic reviews, participants should use relevant performance information and evidence to assess whether strategies are being implemented as planned and whether they are having the desired effect, and to identify areas where action is needed to improve or enhance implementation and impact. Where progress in achieving an objective is lagging, the reviews are intended to identify strategies for improvement, such as strengthening collaboration to better address crosscutting challenges, building skills and capacity, or using evidence to identify and implement more effective program designs. Strategic reviews can also be used to identify any evidence gaps or areas where additional analyses of performance data are needed to determine effectiveness or to help set priorities. For example, we reported that for the Department of Homeland Security’s (DHS) goal to safeguard and expedite lawful trade and travel, officials determined that sufficient progress was being made but identified gaps in monitoring efforts, such as a lack of performance measures related to travel. As a result, DHS officials are taking steps to develop measures to address the gaps.

Other Leading Practices and Evidence-Based Tools Also Have the Potential to Increase Use of Performance Information

We have in the past identified leading practices, such as demonstrating management commitment, that can enhance and facilitate the use of performance information. Our recent periodic survey of federal managers found that specific practices were related to greater use of performance information. As described previously, we developed a use of performance information index, composed of questions from the 2007 and 2013 surveys, to analyze responses to our surveys of federal managers. We used statistical testing to determine if the relationship between additional survey questions, shown in figure 6, from the 2013 survey and an agency’s use of performance information index was statistically significant. We found that an agency’s average use of performance information index score increased when managers reported their agencies engaged to a greater extent in these practices as reflected in the survey questions. The questions that were statistically and positively related to the use of performance information index are also shown in figure 6. For example, we found that the strongest driver of the use of performance information was whether federal managers had confidence in that information’s validity. A greater focus on these practices may help agencies improve their use of performance information. Prompted by our work, several agencies—including the Departments of the Treasury and Labor, the National Aeronautics and Space Administration, and the Nuclear Regulatory Commission—asked us to provide them with underlying data for their agencies from the 2007 and 2013 managers’ surveys, so that they could conduct additional analyses of their agencies’ use of performance information.

Some of the practices reflected in these questions are ones that we have identified elsewhere in our work as important. For example, demonstrated leadership commitment is an area we have emphasized in our work on government operations we identify as high risk. Our high-risk program serves to identify and help resolve serious weaknesses in areas that involve substantial resources and provide critical services to the public. Our experience with the high-risk list over the past 25 years has shown that one of the key elements needed to make progress in high-risk areas is demonstrated strong commitment and top leadership support. Additionally, providing, arranging, or paying for training may also be related to employee engagement. In July 2015, we reported that career development and training—as measured by the Federal Employee Viewpoint Survey question “I am given a real opportunity to improve my skills in my organization”—is one of the six practices that are key drivers of employee engagement.

In addition, evidence-based tools—such as program evaluations and “pay for success” funding mechanisms—can also facilitate the use of performance information.

Program evaluations. Our recent work on program evaluations— systematic studies of program performance—found that agencies have varying levels of evaluation capacity. OMB has encouraged agencies to strengthen their program evaluations and expand their use in management and policy making, but our 2014 examination of agencies’ ability to conduct and use program evaluations found it to be uneven. As part of our work, we surveyed and received responses from the PIOs at the 24 CFO Act agencies. About half (11) of the 24 agencies reported committing resources to obtain evaluations by establishing a central office responsible for evaluating agency programs, operations, or projects; on the other hand, 7 reported having no recent evaluations for any of their performance goals. Although agencies may not have many evaluations, more than a third reported using them to a moderate to a very great extent to support several aspects of program management and policy making.

While agency program evaluation capacity is mixed, some agencies reported increasing use of evaluations and capacity-building activities after GPRAMA was enacted. About half of agencies reported increasing their use of evaluations for various activities, as shown in figure 7, since GPRAMA was enacted. Additionally, half of the PIOs we surveyed reported that efforts to improve their capacity to conduct credible evaluations had increased at least somewhat over this time.

Our work found that implementing certain GPRAMA requirements are among the reported actions agencies can take to improve their capacity to conduct evaluations and make use of evaluation information. About two-thirds of agencies (15) reported hiring staff with research analysis and expertise, and nearly half (11) reported that doing so was useful for improving agency capacity to conduct credible evaluations. Additionally, about half of PIOs reported that conducting data-driven reviews of APGs and holding goal leaders accountable for progress on APGs, both of which are required under GPRAMA, were moderately to very useful for improving agency capacity to make use of evaluation information in decision making. Engaging program staff was also rated very useful. Furthermore, in our June 2013 report on strategies to facilitate agencies’ use of evaluation, we identified three strategies to facilitate the influence of evaluations on program management and policy: demonstrating leadership support of evaluation for accountability and program improvement; building a strong body of evidence; and engaging stakeholders throughout the evaluation process.

Pay for Success. Another evidence-based tool that promotes the use of performance information is Pay for Success (PFS), also known as Social Impact Bonds. PFS is a new contracting mechanism to fund prevention programs, where investors provide capital to implement a social service, for example, to reduce recidivism by former prisoners. If the service provider achieves agreed upon outcomes, the government pays the investor, usually with a rate of return, based on savings from decreased use of more costly remedial services, such as incarceration. In September 2015, we reported that stakeholders from the 10 PFS projects we reviewed said that PFS offers potential benefits to all parties to the project. For example, governments can implement prevention programs that potentially lead to reduced spending on social services and transfer the risk of failing to achieve outcomes to investors. Figure 8 shows the roles of organizations involved in PFS projects.

PFS emphasizes the use of performance information because the government contracts for specific performance outcomes and generally includes a requirement that a program’s impact be independently evaluated. While the structures of the PFS examples we reviewed in our September 2015 report varied, stakeholders we interviewed reported that PFS oversight bodies established in the projects’ contracts regularly reviewed performance data during service delivery. Additionally, stakeholders told us that intermediaries and investors can bring performance management expertise to service providers and provide a rigorous focus on performance management and accountability. For example, an official we interviewed from one service provider noted that her organization invested in data entry and data analyst positions and has a team that collects, analyzes, and processes data that it submits to the intermediary.

As federal agencies consider expanding their involvement in PFS, it becomes increasingly important for officials at all levels of government to collaborate to share knowledge and experiences. We found that while the federal government could play a role in addressing challenges in implementing PFS at the state and local levels of government, a formal means to collaborate and share lessons learned does not exist. We recommended in our September 2015 report that OMB establish a formal means for federal agencies to collaborate on PFS. Having such a mechanism as the field grows would allow agencies to leverage the experience of early federal actors in the PFS field and would decrease the potential for missteps in developing projects due to information gaps and failure to learn from experience with this evolving tool of government. OMB concurred with this recommendation and is working with agencies to explore options for continued collaboration on PFS.

Agencies Continue to Face Challenges Linking Individual and Agency Performance to Results

Goal Leader Designations and Participation in Performance Reviews Have Positive Effects, but Agencies Are Missing Opportunities to Further Strengthen Performance and Accountability

Our previous work has highlighted the importance of creating a “line of sight” showing how unit and individual performance can contribute to overall organizational goals. At the individual level, an explicit alignment of daily activities with broader results is one of the defining features of effective performance management systems. This link reinforces employee engagement and accountability for achieving goals. GPRAMA and related guidance provide several mechanisms that can help individuals and agencies see this connection and help them contribute to agency and government-wide goals.

Goal Leader Designation, Performance Reviews, and Other Factors Have Positive Effects

GPRAMA requires agencies to identify an individual—the goal leader— responsible for the achievement of each APG, and related OMB guidance more recently directed agencies to identify a deputy goal leader to support each goal leader, as we had recommended in our July 2014 report on the role of the agency priority goal leader. Additionally, data- driven reviews required under the act offer the opportunity to hold responsible officials, such as the goal leaders, personally accountable for addressing problems and identifying strategies for improvement.

Agency priority goal leaders. Our July 2014 review of the agency priority goal leader role found that most of the 46 goal leaders we interviewed thought the goal leader designation had positive effects, including providing accountability. Other benefits goal leaders identified as resulting from the position include that it provided greater visibility for the goal, facilitated coordination, heightened focus on the goal, and improved access to resources. Goal leaders told us that several mechanisms promote accountability for goal progress, including personal reputation and accountability to agency and other leadership. For example, the Assistant Secretary of Labor for Occupational Safety and Health, who was the goal leader for two APGs for 2012 and 2013 (Reduce Worker Fatalities and Develop a Model Safety and Return-to- Work Program), told us that the interest of Congress and the Department of Labor’s Inspector General, in their respective oversight roles, both operated to hold him accountable.

Deputy goal leaders supported day-to-day goal management and provided continuity during times of goal leader transition. However, we found that nearly a quarter (11 of 46) of the goal leaders we interviewed had not assigned an official to the deputy goal leader position, a designation that provides clear responsibility and additional accountability for goal achievement. Because of the importance of this position, we recommended that OMB work with agencies to ensure that they appointed a deputy goal leader to support each agency priority goal leader. In response to our recommendation, in April 2015, the Director of OMB issued a memorandum stating that, in addition to identifying a goal leader for each APG, agencies must identify a senior career leader to support implementation. OMB also updated its 2015 Circular A-11 guidance to reflect this requirement.

Data-driven and strategic reviews. Our work has found that regular, in- person data-driven reviews are an effective mechanism for holding goal leaders and other goal contributors individually accountable for goal progress. In our July 2015 examination of data-driven reviews, we reported that 22 agencies we surveyed reported holding in-person data- driven reviews. Of these, 21 reported that their data-driven reviews have had a positive effect on their agency’s ability to hold goal leaders and other officials accountable for progress toward goals and milestones. According to officials from selected agencies, the transparency of performance information and a review process that ensures it receives appropriate scrutiny produce an increased sense of personal accountability for results. For example, officials from the Department of Commerce told us that they are using their regular review meetings with bureau heads and goal leaders to support a cultural change throughout the agency and reinforce accountability for performance at multiple levels of the organization.

NASA Officials Report that Strategic Reviews Encourage Accountability We reported in July 2015 that the National Aeronautics and Space Administration’s (NASA) experience reviewing strategic objectives illustrates their potential for promoting accountability. Agency officials told us that their chief operating officer (COO) determines final ratings for strategic objectives during a briefing attended by the agency’s performance improvement officer, strategic objective leaders, and relevant program staff. According to NASA officials, the personal involvement of the COO encouraged accountability for results and performance improvements.

Similar to data-driven reviews, our work on agency strategic reviews noted that they also have potential for promoting individual accountability for organizational results. In our July 2015 report, in which we identified and illustrated practices that facilitate effective strategic reviews, we reported that accountability for results is one of the key features for planning reviews. Specifically, we stated that the focus of accountability should be on the responsible objective leader’s role in using evidence to credibly assess progress in achieving strategic objectives. Agency leaders should hold objective leaders and other responsible managers accountable for knowing the progress being made in achieving outcomes and, if progress is insufficient, understanding why and having a plan for improvement. If evidence is insufficient for assessing progress, managers should be held accountable for improving the availability and quality of the evidence so that it can be used effectively for decision making. Managers should also be held accountable for identifying and replicating effective practices to improve performance (see sidebar).

Employee engagement. Research on both private- and public-sector organizations has found that increased levels of engagement—generally defined as the sense of purpose and commitment employees feel toward their employer and its mission—can lead to better organizational performance. Our July 2015 report on employee engagement identified specific practices that drive employee engagement. Specifically, our regression analysis of selected Federal Employee Viewpoint Survey (FEVS) questions identified six practices as key drivers of employee engagement, as measured by OPM’s Employee Engagement Index. These practices are detailed in figure 9. Of these six, having constructive performance conversations is the strongest driver of employee engagement.

Performance appraisal systems, which include performance plans, are a powerful mechanism for promoting alignment with and accountability for organizational goals. There are several benefits to aligning performance with results, including increased use of performance information. As shown in the textbox, our work has found problems with oversight and accountability in the Department of Veterans Affairs’ (VA) Health Care System. In response to these and other problems, Congress has taken action, such as passing the Veterans Access, Choice, and Accountability Act of 2014, to hold senior VA leadership accountable for performance and is considering other means of increasing accountability.

Inadequate Oversight and Accountability in VA’s Health Care System Despite substantial budget increases in recent years, for more than a decade there have been numerous reports—by GAO, VA’s Office of the Inspector General, and others—of VA facilities failing to provide timely health care. In some cases, the delays in care or VA’s failure to provide care at all have reportedly resulted in harm to veterans. These and other serious and long-standing problems with the timeliness, cost- effectiveness, quality, and safety of veterans’ health care led to our designation of VA’s health care system as a high-risk area in 2015.

To facilitate accountability for achieving its organizational goal of ensuring that veterans have timely access to health care, VA included measures related to wait times for primary and specialty care appointments (1) in the performance contracts for senior leaders and (2) in the agency’s annual budget submissions and performance and accountability reports. However, we found that data used to monitor performance on these measures were unreliable and that inconsistent implementation of VA’s scheduling policies may have resulted in increased wait times or delays in scheduling outpatient medical appointments at VA facilities. Scheduling staff in some locations told us that they had changed desired dates for medical appointment to show that wait times were within VA’s performance goals. The VA Office of the Inspector General has published reports with similar findings. VA has since announced that it has modified its performance measures that relate to wait times and removed measures related to wait times from senior leaders’ performance contracts.

Goal leaders’ Senior Executive Service performance plans. Although goal leaders told us that the designation provides accountability, we found their Senior Executive Service (SES) performance plans generally did not reflect their responsibility for goal achievement. As part of our work on the role of the agency priority goal leader in July 2014, we reviewed the performance plans of all of the goal leaders and deputy goal leaders for the 47 APGs in our sample, where applicable. These performance plans covered a range of responsibilities, but many did not reference the APGs for which the goal leaders and deputies were responsible. Additionally, the vast majority (all but one of the 32 goal leader plans and one of the 35 deputy goal leader plans) failed to link performance standards to goal outcomes. Failing to fully reflect goal achievement in performance plans is a missed opportunity to ensure that goal leaders and deputies are held accountable for goal progress and to reinforce links. Because APGs by definition reflect the highest priorities of each agency, accountability for goal achievement is especially important. To ensure goal leader and deputy goal leader accountability, we recommended that the Director of OMB work with agencies to ensure that goal leader and deputy goal leader performance plans demonstrate a clear connection with APGs. As of June 2015, OMB had not yet taken action in response to this recommendation.

Senior Executive Ratings. Our recent work has also raised questions about agency processes for rating senior executive performance, which can promote alignment with and accountability for organizational goals. For our January 2015 report on SES ratings and performance awards, we reviewed performance award data from the 24 CFO Act agencies and we examined performance appraisal systems at five case study agencies. Specifically, we looked at the performance appraisal system that the Office of Personnel Management (OPM) and other agency representatives developed in 2012. This system is intended to provide a more consistent and uniform framework for SES evaluation. We found that the five agencies we studied in detail had all linked SES performance plans with agency goals, a key practice for effective performance management systems, and a feature that promotes the line of sight between individual performance and organizational goals.

Disparities in Agencies’ Executive Ratings Distributions We reported in January 2015 that there is wide variation in SES ratings distributions among agencies. For example, in fiscal year 2013 the Department of Defense rated 30.6 percent of its SES employees at the highest rating level, while the Department of Justice rated 73.6 percent of its SES employees at this level. As we have previously reported, one of the key practices in promoting a line of sight between individual performance and organizational goals is making meaningful distinctions in performance.

However, although one of the primary purposes for establishing the new appraisal system included increasing equity in ratings across agencies, we found disparities in rating distributions and pay (see sidebar). This disparity in ratings between agencies raises questions about whether agencies are consistently applying performance definitions and whether performance ratings are meaningful. We recommended that the Director of OPM, which certifies—with OMB concurrence—SES performance appraisal systems, should consider the need for refinements to the performance certifications guidelines addressing distinctions in performance and pay differentiation. OPM partially concurred with the recommendation, though we maintain that additional action should be considered to ensure equity in ratings and performance awards across agencies. As of June 2015, OPM officials said that they had convened a cross-agency working group that developed several recommendations that are intended to make agencies’ justifications for high SES ratings more transparent.

Senior executives’ use of performance information for decision making. Aligning SES performance with results is a key feature of effective performance management, and our recent work has found that it also may promote use of performance information. As we found in our September 2014 report on trends in the use of performance information, managers’ responses to a question we asked them on aligning an agency’s goals, objectives, and measures was significantly related to the use of performance information, controlling for other factors. Specifically, an increase in the extent to which managers aligned performance measures with agency-wide goals and objectives was associated with an increase on the five-point scale we used for our use index. However, our analysis also found that there was a gap between SES and non-SES managers in reported use of performance information. SES managers government-wide and at nine agencies scored statistically significantly higher than the non-SES managers at those agencies. As shown in figure 10, SES and non-SES managers from the Departments of Homeland Security and Veterans Affairs had the largest gap in use of performance information between their SES and non-SES managers.

Agencies Have Long- standing Difficulties in Measuring Performance of Selected Program Types

A critical element in an organization’s efforts to manage for results is its ability to set meaningful goals for performance and to measure progress toward these goals. GPRAMA reinforces the need to set meaningful goals by directing agencies to establish a balanced set of performance measures, such as output, outcome, customer service, and efficiency, across program areas.

Agencies have been responsible for measuring program outcomes since GPRA was enacted in 1993, but still have difficulty developing and using performance measures. As we reported nearly 20 years ago, performance measures should demonstrate to each organizational level how well it is achieving its goals. As shown in the illustrations in the textbox, however, agencies continue to make insufficient progress in establishing and using outcome-oriented performance measures.

Examples of Agency Difficulties in Developing and Using Outcome Measures Outcome-Oriented Metrics and Goals Are Needed to Gauge DOD’s and VA’s Progress in Achieving Interoperability of Electronic Health Records Systems The Departments of Defense (DOD) and Veterans Affairs (VA) operate two of the nation’s largest health care systems, serving approximately 16 million veterans and active duty service members and their beneficiaries, at a cost of more than $100 billion a year. With guidance from the Interagency Program Office (IPO) that is tasked with facilitating the departments’ efforts to share health information, the two agencies have taken actions to increase interoperability between their electronic health record systems. Developing electronic health records is particularly important for optimizing the health care provided to military personnel and veterans, as they and their families tend to be highly mobile and may have health records residing at multiple medical facilities. In August 2015, we reported that the IPO had taken steps to develop process metrics intended to monitor progress of these efforts, but had not yet specified outcome-oriented metrics or established related goals that are important to gauging the impact that interoperability capabilities have on improving health care services for shared patients. Using outcome-based metrics could provide DOD and VA a more accurate, ongoing picture of their progress toward achieving interoperability and the value and benefits generated. We recommended that DOD and VA, working with the IPO, establish a time frame for identifying outcome-oriented metrics; define related goals to provide a basis for assessing and reporting on the status of interoperability; and update IPO guidance to reflect the metrics and goals identified. DOD and VA concurred with our recommendations.

Measuring Progress in Addressing Incarceration Challenges The federal inmate population has increased more than eight-fold since 1980, and the Department of Justice (DOJ) has identified prison crowding as a critical issue since 2006. In June 2015, we reported that DOJ had implemented three key initiatives to address the federal incarceration challenges of overcrowding, rising costs, and offender recidivism. The department had several early efforts underway to measure the success of these initiatives, but we concluded that its current approach could be enhanced. For example, the Clemency Initiative is intended to encourage federal inmates who meet criteria that DOJ established to apply to have their sentences commuted (reduced) by the President. DOJ tracked some statistics related to this initiative, such as the number of petitions received and the disposition of each, but it did not track how long, on average, it took for petitions to clear each step in its review process. Such tracking would help DOJ identify processes that might be contributing to any delays. Without this tracking, DOJ cannot be sure about the extent to which the additional resources it is dedicating to this effort are helping to identify inmate petitions that meet DOJ’s criteria and expedite their review. We recommended that the Attorney General direct the Office of the Pardon Attorney to (1) track how long it takes, on average, for commutation of sentence petitions to clear each step in the review process under DOJ’s control, and (2) identify and address, to the extent possible, any processes that may contribute to unnecessary delays. DOJ concurred with the recommendation and stated that it would consider our findings and recommendations during the course of its ongoing efficiency reviews.

Measuring Effectiveness of Military Sexual Assault Prevention Efforts Our recent work has identified issues in establishing goals and metrics to measure the effectiveness of efforts to reduce incidents of sexual assault in the military, which according to the Department of Defense (DOD) represent a significant and persistent problem within the department. For example, in March 2015, we reported that DOD had not established goals or metrics to gauge sexual assault-related issues for male service members. DOD’s Sexual Assault Prevention and Response Office had three different general officers in the director position since 2011. Given this high level of turnover, we stated that establishing goals and metrics is key to institutionalizing efforts to address sexual assault of male service members. We recommended that DOD develop clear goals and associated metrics to drive the changes needed to address sexual assaults of males and articulate these goals. DOD agreed with this recommendation.

Measuring the performance of different program types—such as grants, regulations and tax expenditures—is a significant and long-standing government-wide challenge and one we have addressed in our previous work. In our June 2013 report on initial GPRAMA implementation, we also reported that agencies have experienced common issues in measuring various types of programs. We recommended that the Director of OMB work with the PIC to develop a detailed approach to examine these difficulties, including identifying and sharing any promising practices. Additionally, our July 2014 report on the role of the agency priority goal leader noted that several APGs we examined identified certain program types, such as grants, as key contributors to their goals. However, goal leaders and their deputies lacked the means to identify and share information with other goal leaders who were facing similar challenges or were interested in similar topics. We recommended that the Director of OMB work with the PIC to further involve agency priority goal leaders and their deputies in sharing information on common challenges and practices related to APG management. OMB and PIC staff told us in June 2015 that they have taken some actions to facilitate information sharing on common topics. For example, the PIC developed a law enforcement working group, which aims to address challenges in measuring law enforcement functions. Despite these steps, additional actions are needed to fully implement these recommendations and address this long- standing issue. We will continue to monitor OMB’s and the PIC’s efforts.

Illustrative examples from our recent work that show how agencies need to make better progress in measuring certain program types are provided in table 2.

One program type—direct service—is one of the areas in which our recent work has highlighted problems with agencies’ performance measurement in multiple agencies. Our October 2014 report on customer service standards examined how selected agencies are using customer service standards and measuring performance against those standards. We reviewed the customer services standards for six federal programs and compared them to key elements of effective customer services standards, which we identified based on our review of GPRAMA and executive orders that focused on providing greater accountability, oversight, and transparency. Two of the key elements of customer services standards are that they (1) include targets or goals for performance, and (2) include performance measures. We found that three of the six programs did not have customer services standards that met these two elements. For example, we reported that because the National Park Service (NPS) did not have performance goals or measures directly linked to those goals, the agency is unable to determine the extent to which the standards are being met agency-wide or strategies to close performance gaps. We made several recommendations related to improving the NPS’s and other agencies’ customer service standards, including that the Department of the Interior (of which NPS is a part) to ensure NPS standards include (1) performance targets or goals, and (2) performance measures. In July 2015, NPS officials reported that they had made plans to implement these recommendations. Additionally, OMB is focusing on improving the federal government’s customer service by developing a related CAP goal. According to information on Performance.gov, as part of its work on the Customer Service CAP goal, the administration is working to streamline transactions, develop standards for high impact services, and utilize technology to improve the customer experience. We will be assessing OMB’s progress in implementing this CAP goal as part of our ongoing review.

OMB and Agencies Have Not Clearly Communicated Key Performance Information, but More Effective Implementation of GPRAMA Requirements Would Improve Transparency

To operate as effectively and efficiently as possible and to make difficult decisions to address the federal government’s fiscal and performance challenges, Congress, the administration, and federal managers must have ready access to reliable and complete financial and performance information—both for individual federal entities and for the federal government as a whole.

However, in our work since 2013 we have identified areas in which agencies have not clearly reported information related to billions of dollars in government spending (see textbox).

Examples of Agencies Not Clearly Reporting Information on Government Spending Agencies Fail to Properly Report Over $600 Billion in Assistance Awards The Federal Funding Accountability and Transparency Act (FFATA) was enacted in 2006 to increase the accountability and transparency over the more than $1 trillion spent by the federal government on contracts, grants, loans, and other awards annually. The act required OMB to establish a website that contains data on federal awards and guidance on agency reporting requirements for the website, USASpending.gov. The website is to promote transparency in government spending by providing the public with the ability to track where and how federal funds are spent. However, we reported in June 2014 that although agencies generally reported required contract information, they did not properly report information on assistance awards (e.g., grants or loans), totaling approximately $619 billion in fiscal year 2012. In addition, we found that few awards on the website contained information that was fully consistent with agency records. We estimated with 95 percent confidence that between 2 and 7 percent of the awards contained information that was fully consistent with agencies’ records for all 21 data elements examined. We concluded that without accurate data, the usefulness of USASpending.gov will be hampered. To improve the reliability of information on USASpending.gov, we recommended that OMB (1) clarify guidance on reporting award information and maintaining supporting records, and (2) develop and implement oversight processes to ensure that award data are consistent with agency records. OMB generally agreed with our recommendations but, as of August 2015, had not taken actions to address them. Full implementation of the DATA Act, which amended FFATA and which OMB is currently working on, may address these recommendations.

USDA Performance Reporting Does Not Reflect Effects of Approximately $3 Billion in Spending on Broadband Access to affordable broadband Internet is seen as vital to economic growth and improved quality of life, yet deployment in rural areas tends to lag behind urban and suburban areas. The American Recovery and Reinvestment Act of 2009 (Recovery Act) appropriated funding for the Broadband Initiatives Program (BIP), a Department of Agriculture (USDA) Rural Utilities Service (RUS) program to fund broadband projects primarily to serve rural areas. However, we reported in June 2014 that RUS has reported limited information on BIP’s impact since awarding funds to projects, and that BIP results are not tracked in USDA’s annual performance reporting. As a result, RUS has not shown how much the program’s approximately $3 billion in project funding has affected broadband availability. GPRAMA directs agencies to establish performance goals in annual performance plans and to report on progress made toward these goals in annual performance reports. However, USDA did not update or include BIP results as compared to the related performance goals in its annual performance reports. We concluded that without an updated performance goal and regular information reported on the results of BIP projects, it is difficult for USDA, RUS, and policymakers to determine the impact of Recovery Act funds or BIP’s progress on improving broadband availability. We recommended that the Secretary of Agriculture include BIP performance information as part of USDA’s annual performance plan and report by comparing actual results achieved against the current subscribership goal. USDA agreed with our recommendation, and stated that it planned to modify its next annual performance plan and report to include the number of subscribers receiving new and improved service as a result of the program.

Our work has also identified other problems with transparency. As described in the textbox below, only one of the six federal services for which we reviewed customer service standards had standards that were easily publicly available.

Most Agencies Reviewed Did Not Make Customer Service Standards Easily Publicly Available Our recent work has also found issues with transparency related to agencies’ customer service standards. In October 2014 we identified key elements of customer service standards—which should inform customers as to what they have a right to expect when they request services—that would allow agencies to better serve the needs of their customers by providing greater accountability, oversight, and transparency. One of the elements that we identified is that customer service standards be easily publicly available. Easily available standards help customers know what to expect, when to expect it, and from whom. As part of our work, we assessed the extent to which customer service standards at six services within five federal agencies (including two services within one of those agencies) included key elements, including easily publicly accessible standards. We found that only one of these services had standards that were easily available to the public. That service—Customs and Border Protection (CBP) inspection of individuals—posts its standards on its website as well as at points of service in entry ports, field offices, and headquarters, according to CBP officials.

The other five services, we found, did not make their standards easily accessible to the public. For example, we had reported in 2010 that the Forest Service did not make its customer service standards available to its customers because officials felt that the standards would not be helpful to the visitors who evaluate such things as the cleanliness of restrooms against their own standards and not those set forth by the Forest Service. In 2014, Forest Service officials told us that there has been no change since 2010. However, according to executive orders and guidance, standards are specifically intended to inform the public, and should be publicly available. We recommended that the Department of Agriculture (of which the Forest Service is a part) ensure that the Forest Service’s standards are easily publicly available, among other things. In addition, we made recommendations to the other five services that had not made their standards easily accessible to the public.

Although GPRAMA requirements have the potential to increase transparency of performance information, we have found mixed progress in implementing these requirements.

Program inventories. GPRAMA’s requirements for program inventories have the potential to improve transparency of performance information, but, as previously described, our October 2014 report identified several issues that affect these inventories’ usefulness. For example, although GPRAMA requires agencies to describe each program’s contribution to the agency’s goals, we found instances where agencies omitted that information. Ensuring agencies illustrate this alignment would better explain how programs support the results agencies are achieving.

As stated earlier, OMB has put plans for updating the inventories on hold, in part due to the enactment of the DATA Act, which is intended to increase accountability and transparency in federal spending by requiring agencies to publicly report information about any funding made available to, or expended by, an agency. As noted in our July 2015 testimony on DATA Act implementation, effective implementation of both the DATA Act and GPRAMA’s program inventory provisions, especially the ability to crosswalk spending data to individual programs, could provide vital information to assist federal decision makers in addressing the significant challenges the government faces. We identified a potential approach OMB could take in merging program inventory efforts with DATA Act implementation. That is, OMB could explore ways to improve the comparability of program data by using tagging or similar approaches that allow users to search by key words or terms and combine elements based on the user’s interests and needs. This merging could help ensure consistency in the reporting of related program-level spending information. As mentioned previously, OMB does not expect an update of program inventories to happen before May 2017.

Other planned changes to the program inventories could also improve the transparency of their information. For example, OMB staff told us that they also planned to present the 24 program inventories during the planned May 2014 update in a more dynamic, web-based format. This approach, too, has been put on hold. A web-based approach could make it easier to tag and sort related or similar programs. For instance, OMB plans to have agencies tag each of their programs by one or more program type in a future iteration of the inventory to provide a sorting capability for identifying the same type of program. By providing a sorting mechanism by program type, OMB could help address one of our open recommendations, described previously, that OMB work with the PIC to develop a detailed approach to examine common, long-standing difficulties agencies face in measuring the performance of various types of federal programs and activities.

A sorting mechanism could help by identifying (1) all programs in a given type, and (2) of those programs, any of which have developed strategies to effectively overcome measurement challenges. Additionally, in its guidance for the 2014 update before it was put on hold, OMB intended for agencies to link each program to the existing web pages on Performance.gov for strategic goals, strategic objectives, APGs, and CAP goals. According to OMB staff, once they move forward with the next inventory update and move to a web-based presentation, users will be able to sort programs by the goals to which they contribute. This approach also would allow users to identify programs that contribute to broader themes on Performance.gov. The themes generally align with budget functions from the President’s Budget and include administration of justice; general science, space, and technology; national defense; and transportation, among others. Currently, the themes can be used to sort goals on Performance.gov that contribute to those broad themes.

Major management challenges. Another area in which our work has identified problems with transparency and communication of performance information is related to the GPRAMA requirement that agencies report in their annual performance plans key performance information related to their major management challenges, including performance goals, milestones, indicators, and planned actions that they have developed to address such challenges. Major management challenges include programs or management functions, within or across agencies, that have greater vulnerability to fraud, waste, abuse, and mismanagement, such as those issues identified by GAO as high risk, where a failure to perform well could seriously affect an agency’s or the government’s ability to achieve its mission or goals. We have ongoing work, which we plan to issue in late 2015, which is examining how federal agencies are addressing their major management challenges. As of August 2015, we found that agencies generally did not report key performance information about their major management challenges in their annual performance plans and reports in a transparent manner. For example, while some agencies told us that they had internal plans for addressing their major management challenges, 12 of 24 agencies that issued agency performance plans or similar documents for fiscal year 2015 did not publicly report planned actions for addressing their major management challenges. While the reasons for why agencies did not report complete information varied, such as readability and redundancy with other similar topics in the performance plan, agencies told us that OMB’s guidance appeared to give them flexibility on what information they needed to report. We will provide updated information on major management challenges in our forthcoming report.

CAP goals. Another area in which transparency is important is in communicating progress on performance goals, but our June 2014 report on CAP goal reviews found that the quarterly updates for the 14 interim CAP goals did not always provide a complete picture of progress. For each of the CAP goals, GPRAMA requires OMB to coordinate with agencies to establish annual and quarterly performance targets and milestones and to report quarterly the results achieved compared to the targets. The updates we reviewed were inconsistent, and some were missing key performance information, such as performance targets, milestone due dates, and key contributors to the goals, that was needed to track progress toward the goals. In one case, we were told that the data needed to track progress toward a goal were not available. Staff from the Real Property interim CAP goal team told us that they did not have data available for tracking progress toward the goal of holding the federal real property footprint at its fiscal year 2012 baseline level. In addition, we found that in some cases information on the organizations and program types that contributed to an interim CAP goal, such as relevant tax expenditures, was missing.

We concluded that the incomplete information in many of the updates provided a limited basis for ensuring accountability for progress toward targets and milestones for those interim CAP goals and recommended that OMB take a number of actions to ensure that all key contributors were identified and that quarterly and overall progress toward CAP goals could be fully reported. These included identifying all key contributors to the achievement of the goals; identifying annual planned levels of performance and quarterly targets for each of the goals; developing plans to identify, collect, and report data necessary to demonstrate progress being made toward each of the goals or developing an alternative approach for tracking and reporting on progress quarterly; and reporting the time frames for the completion of milestones, the status of milestones, and how milestones are aligned with strategies or initiatives that support the achievement of each goal.

As described previously, OMB has increased its emphasis on CAP goal governance for the current set of CAP goals, and OMB has taken actions to address concerns our work raised about CAP goal reviews. One of the actions OMB has taken, together with the PIC, was to develop revised guidance, in the form of a template, for CAP goal teams to use to report quarterly progress updates for these goals. This template responded to three of our recommendations related to CAP goal progress reporting by including a section for the CAP goal teams to identify programs that contribute to their goals; directing the teams to list targets for the key indicators they use to track progress; and directing the teams to establish work plans with a list of specific milestones that should include milestone due dates and information on milestone status.

The template also indicated that goal teams can organize milestones by each identified sub-goal, aligning specific activities with the objectives to which they contribute. In addition, the PIC provided guidance in January 2015 that further addressed two of our recommendations. The guidance directs CAP goal teams to report all agencies, organizations, programs, activities, regulations, tax expenditures, policies, and other activities that contribute to each goal. It also specifically notes that GPRAMA requires the teams to report on performance against targets and states that quarterly progress updates should identify areas where progress has exceeded expectations or been slower than expected or where targets for performance measures have been missed. The actions that OMB and the PIC have taken to address our recommendations have helped to improve the transparency of the CAP goal progress updates. For example, nearly all of the quarterly updates released in June 2014 for the second quarter of fiscal year 2014 included milestone due dates and information on their status. Many (8 of 15) of the lists of milestones aligned with specific sub-goals.

Quality of performance information. GPRAMA requirements for reporting on the quality of performance information also have the potential to increase transparency, as they require agencies to publicly report on how they are ensuring the accuracy and reliability of the performance information they use to measure progress toward APGs and performance goals. Specifically, for each APG, agencies must provide information addressing five requirements to OMB for publication on Performance.gov. Additionally, agencies must address all five requirements for performance goals, which include APGs.

Our September 2015 report on the quality of publicly reported performance information found limited information on Performance.gov on the quality of performance information used to assess progress on six selected agencies’ 23 APGs. In response to our review, OMB updated its A-11 guidance in June 2015 to direct agencies to either provide this information for publication on Performance.gov on how they are ensuring the quality of performance information for their APGs, or provide a hyperlink from Performance.gov to an appendix in their performance report that discusses the quality of their performance information. OMB staff stated that this information will likely not be available until agencies start reporting on the next set of APGs (for fiscal years 2016 and 2017). This is because OMB will need to update a template that agencies complete for their Performance.gov updates.

Further, the agencies we reviewed generally did not describe how they addressed all five requirements for their individual APGs in their performance plans and reports. While all six agencies described how they ensured the quality of their performance information overall, we found that only DHS’s performance plans and reports included discussions about performance information quality addressing all five GPRAMA requirements, as shown in table 3 and described in more detail in the textbox below.

DHS Addressed GPRAMA Requirements in Explaining How Performance Information Quality is Ensured for All Agency Priority Goals In September 2015, we reported that of the 23 APGs in our sample from six agencies, we could only find discussions about performance information quality that addressed all five of the GPRAMA requirements for three APGs, which belonged to DHS. DHS presented information about performance information quality for all three of its APGs in its performance plans and reports. Specifically, DHS published an appendix to its performance plans and reports with detailed discussions of performance information quality for 10 performance measures used to measure progress on these APGs. For each measure, DHS’s appendix described the related program, the scope of the data, the source and collection methodology for the data, and an assessment of data reliability.

In our September 2015 report, we recommended that all six of the agencies in our review work with OMB to describe on Performance.gov how they are ensuring the quality of their APGs’ performance information and that the agencies, except for DHS, also describe this in their annual performance plans and reports. We also noted that to help improve the reliability and quality of performance information, OMB and the PIC established the Data Quality Cross-Agency Working Group in February 2015. The group could serve as a vehicle for disseminating good practices in public reporting on data quality. As a result, we also recommended that OMB, working with the PIC, focus on ways the PIC’s data quality working group can improve public reporting for APGs. OMB did not comment on the recommendations, but the six agencies generally concurred or identified actions they planned to take to implement them.

OMB and Agencies Generally Agreed with GAO’s Prior Recommendations to Improve GPRAMA Implementation, but Most Have Not Yet Been Implemented

Our work examining aspects of GPRAMA implementation and its effects on agency performance management has identified a number of areas in which improvements are needed. Since GPRAMA was enacted in January 2011, we have made a total of 69 recommendations to OMB and agencies aimed at improving its implementation. OMB and the agencies have generally agreed with the recommendations we have made thus far, and have implemented some of them. However, of the 69 recommendations we have made, 55 (about 80 percent) have not yet been implemented, while 14 recommendations (about 20 percent) have been implemented. Additional details on these recommendations and their status are included in appendixes II, III and IV.

We made 21 recommendations to OMB and agencies between May 2012, when we issued our first report on GPRAMA implementation, and June 2013, when we issued our previous summary report. Fourteen (about 67 percent) of these recommendations have not been implemented. Between July 2013 and September 2015, we made 48 additional recommendations to OMB and the agencies. Forty-one (about 85 percent) of these recommendations have not been implemented. Figure 11 shows the number of recommendations we have made, by year, and the number that have been implemented.

OMB, which has been the focus of most of our recommendations, has implemented just over one-third (14) of the 38 recommendations we have made to it. Because of the agency’s central role in implementing GPRAMA, we made more recommendations to OMB in our work under the act than to all other agencies combined. Most of the actions OMB has taken to implement our recommendations involve updating or issuing new guidance. Agencies have yet to implement any of the 31 recommendations we have made, although we made most (23) of these recommendations in reports that we have issued since July 2015. Specifically, these 23 recommendations were included in our recent work on data-driven reviews and the quality of performance information, and they focus on ensuring that agency data-driven review processes and reporting on the quality of performance information are consistent with GPRAMA requirements, OMB guidance, and leading practices.

While OMB has implemented some of our recommendations, some of those that have yet to be implemented focus on long-standing and significant issues. For example, as described previously, we have made several recommendations to identify and assess the contributions of tax expenditures toward executive branch goals, but OMB and agencies have taken little action to address these recommendations. Additionally, we have reported that agencies have difficulty measuring the performance of different program types –such as grants and regulations. We have identified individual examples of these problems, but our work has also shown that some areas—such as customer service—are common problems across multiple agencies. Agencies have not yet implemented recommendations we made in our October 2014 report on agency customer service standards.

We have also made numerous recommendations aimed at improving the effectiveness of various aspects of GPRAMA implementation. These recommendations focus on a range of areas, including making federal program inventories more useful, strengthening data-driven review practices, and improving goal leader accountability mechanisms. As we have stated, effective GPRAMA implementation has the potential to improve performance management across government and can help address crosscutting issues, promote the use of performance information, increase alignment of performance with results, and improve transparency. We will continue to monitor OMB’s and agencies’ actions to implement our recommendations.

Agency Comments

We provided a copy of this draft report to the Director of the Office of Management and Budget for its review and comment. On September 18, 2015, OMB staff provided us with oral comments on the report. OMB staff generally agreed with the information presented in the report, and provided us with technical clarifications, which we have incorporated as appropriate.

We are sending copies of this report to interested congressional committees, the Director of the Office of Management and Budget, and other interested parties. This report will also be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-6806 or mihmj@gao.gov. Contract points for our Offices of Congressional Relations and Public Affairs may be found on the last page of our report. Key contributors to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

The GPRA Modernization Act of 2010 (GPRAMA) includes a provision for us to review implementation of the act at several critical junctures and provide recommendations for improvements to implementation. Specifically, we are required to evaluate and report on how implementation of the act is affecting performance management at the agencies subject to the Chief Financial Officers Act of 1990, as amended, and to evaluate the implementation of cross-agency priority (CAP) goals, federal government performance plans, and related reporting by September 2015. This report pulls together findings from our work related to the act and on federal performance and coordination issues, focusing on ongoing work and work issued since our last summary report on GPRAMA was issued in June 2013, as well as some results from our work on two ongoing engagements.

Our objectives for this report were to evaluate how GPRAMA implementation has affected progress in addressing four areas: (1) crosscutting issues; (2) the extent to which performance information is useful and used; (3) aligning daily operations with results; and (4) communication of performance information. To address these objectives, we reviewed GPRAMA, Office of Management and Budget (OMB) guidance, and our past and recent work related to managing for results and the act. We also interviewed OMB and Performance Improvement Council staff.

Our recent work under GPRAMA, both ongoing and issued since June 2013, covered the 24 CFO Act agencies and the Army Corps of Engineers-Civil Works. Most (8) of the 12 reports that are the focus of this report used selected agencies as case illustrations. Half of the 12 reports included government-wide reviews, and in some cases involved surveys of all or most of the CFO Act agencies.

This report also includes some results from our ongoing work examining the implementation of CAP goals, which we plan to issue at the end of 2015. We identified lessons learned from the interim CAP goal period, and we assessed initial progress implementing the current set of CAP goals. To do this, we selected 7 of the 15 CAP goals for examination, interviewed officials with responsibility for implementing these goals, and reviewed relevant guidance and documentation. In order to provide some insight into both interim and new CAP goals, the team initially randomly selected 2 of each, resulting in selecting Open Data and STEM Education, which were also interim CAP goals, and Job-Creating Investment and Lab-to-Market, which are new CAP goals. Because GAO did recent work on three additional CAP goals—Customer Service, People and Culture, and the Smarter IT Delivery—we also selected those goals. We interviewed OMB and PIC staff responsible for management and implementation of the current CAP goals and responsible agency officials, including CAP goal leaders and members of the seven CAP goal implementation teams. We reviewed OMB and PIC guidance, relevant documentation, and quarterly progress updates published on Performance.gov from the second quarter of fiscal year 2014 through the second quarter of fiscal year 2015, published in June 2015.

This report also reflects some results from our ongoing work on major management challenges, which we also plan to issue at the end of 2015. We compared information reported in 24 agency performance plans and reports against GPRAMA requirements and OMB Circular A-11 guidance to identify agency activities and reporting related to major management challenges. We interviewed OMB staff about their guidance related to major management challenges. We also interviewed 24 CFO Act agency performance officials, including performance improvement officers, program offices officials, and, when appropriate, officials from agencies’ Chief Financial Officer and Chief Human Capital Offices to understand how agencies defined and addressed their major management challenges.

We conducted this performance audit from April 2015 to September 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Recommendations to OMB from GAO’s Work under the GPRA Modernization Act That Have Been Implemented

Table 4 shows recommendations we have made as part of our work under the GPRA Modernization Act (GPRAMA) that the Office of Management and Budget (OMB) has implemented.

Appendix III: Recommendations to OMB from GAO Work under the GPRA Modernization Act That Have Not Been Implemented

Table 5 shows recommendations we have made to the Office of Management and Budget (OMB) as part of our work under the GPRA Modernization Act (GPRAMA) that have not been implemented.

Appendix IV: Recommendations to Agencies from GAO’s Work under the GPRA Modernization Act That Have Not Been Implemented

Table 6 shows recommendations we have made to agencies as part of our work under the GPRA Modernization Act (GPRAMA) that have not been implemented.

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact:

Staff Acknowledgments

In addition to the above contact, Sarah E. Veale (Assistant Director) and Kathleen Padulchick supervised this review and the development of the resulting report. Margaret M. Adams, Shea Bader, Lisette Baylor, Peter Beck, Elizabeth Curda, Dewi Djunaidy, Deirdre Duffy, Karin Fangman, Jennifer M. Felder, Farrah Graham, Emily Gruenwald, Jonathan Harmatz, Jennifer Kamara, Barbara Lancaster, Dainia Lawes, Benjamin T. Licht, Adam Miles, Michael O’Neill, Lisa Pearson, Steven Putansu, MaryLynn Sergent, Stephanie Shipman, Matthew Sweeney, and Dan Webb also made key contributions.

Related GAO Products

Managing for Results: Greater Transparency Needed in Public Reporting on the Quality of Performance Information for Selected Agencies’ Priority Goals. GAO-15-788. Washington, D.C.: September 10, 2015.

DATA Act: Progress Made in Initial Implementation but Challenges Must be Addressed as Efforts Proceed. GAO-15-752T. Washington, D.C.: July 29, 2015.

Managing for Results: Practices for Effective Agency Strategic Review., GAO-15-602. Washington, D.C.: July 29, 2015.

Managing for Results: Agencies Report Positive Effects of Data-Driven Reviews on Performance but Some Should Strengthen Practices. GAO-15-579. Washington, D.C.: July 7, 2015. 2015 Annual Report: Additional Opportunities Exist to Reduce Fragmentation, Overlap, and Duplication and Achieve Other Financial Benefits. GAO-15-404SP. Washington D.C.: April 14, 2015.

Fragmentation, Overlap, and Duplication: An Evaluation and Management Guide. GAO-15-49SP. Washington, D.C.: April 14, 2015.

Government Efficiency and Effectiveness: Opportunities to Reduce Fragmentation, Overlap, and Duplication and Achieve Other Financial Benefits. GAO-15-522T. Washington, D.C. April 14, 2015.

High-Risk Series: An Update. GAO-15-290. Washington, D.C.: February 11, 2015.

Federal Data Transparency: Effective Implementation of the DATA Act Would Help Address Government-wide Management Challenges and Improve Oversight. GAO-15-241T. Washington, D.C.: December 3, 2014.

Program Evaluation: Some Agencies Reported that Networking, Hiring, and Involving Program Staff Help Build Capacity. GAO-15-25. Washington, D.C.: November 13, 2014.

Government Efficiency and Effectiveness: Inconsistent Definitions and Information Limit the Usefulness of Federal Program Inventories. GAO-15-83. Washington, D.C.: October 31, 2014.

Managing for Results: Selected Agencies Need to Take Additional Efforts to Improve Customer Service. GAO-15-84. Washington, D.C.: October 24, 2014.

Managing for Results: Agencies’ Trends in the Use of Performance Information to Make Decisions. GAO-14-747. Washington, D.C.: September 26, 2014.

Managing for Results: Enhanced Goal Leader Accountability and Collaboration Could Further Improve Agency Performance. GAO-14-639. Washington, D.C.: July 22, 2014.

Managing for Results: OMB Should Strengthen Reviews of Cross-Agency Goals. GAO-14-526. Washington, D.C.: June 10, 2014.

Government Efficiency and Effectiveness: Views on the Progress and Plans for Addressing Government-wide Management Challenges. GAO-14-436T. Washington, D.C.: March 12, 2014.

Managing for Results: Implementation Approaches Used to Enhance Collaboration in Interagency Groups. GAO-14-220. Washington, D.C.: February 14, 2014.

Financial and Performance Management: More Reliable and Complete Information Needed to Address Federal Management and Fiscal Challenges. GAO-13-752T. Washington, D.C.: July 10, 2013.

Managing for Results: Executive Branch Should More Fully Implement the GPRA Modernization Act to Address Pressing Governance Challenges. GAO-13-518. Washington, D.C.: June 26, 2013.