Managing For Results: Executive Branch Should More Fully Implement the GPRA Modernization Act to Address Pressing Governance Challenges
Released: 2013-06-26T00:00:00

Letter

Background

GPRAMA is a significant enhancement of GPRA, which was the centerpiece of a statutory framework that Congress put in place during the 1990s to help resolve long-standing management problems in the federal government and provide greater accountability for results. GPRA sought to focus federal agencies on performance by requiring agencies to develop long-term and annual goals—contained in strategic and annual performance plans—and measure and report on progress towards those goals on an annual basis.

In our past reviews of its implementation, we found that GPRA provided a solid foundation to achieve greater results in the federal government, but several key governance challenges remained—particularly related to: addressing crosscutting issues; ensuring performance information was useful and used by agency leadership and managers and the Congress; strengthening the alignment between individual performance and agency results as well as holding individuals and organizations responsible for achieving those results; measuring performance for certain types of programs; and providing timely, useful information about the results achieved by agencies.

To help address these and other challenges, GPRAMA revises existing provisions and adds new requirements, including the following:

Cross-agency priority (CAP) goals: OMB is required to coordinate with agencies to establish federal government priority goals— otherwise referred to as CAP goals—that include outcome-oriented goals covering a limited number of policy areas as well as goals for management improvements needed across the government. The act also requires that OMB—with agencies—develop annual federal government performance plans to, among other things, define the level of performance to be achieved toward the CAP goals.

Agency priority goals (APGs): Certain agencies are required to develop a limited number of APGs every 2 years. Both the agencies required to develop these goals and the number of goals to be developed are determined by OMB. These goals are to reflect the highest priorities of each selected agency, as identified by the head of the agency, and be informed by the CAP goals as well as input from relevant congressional committees.

Leadership positions: Although most of these positions previously existed in government, they were created by executive orders, presidential memoranda, or OMB guidance. GPRAMA established these roles in law, provided responsibilities for various aspects of performance improvement, and elevated some of them.

Chief operating officer (COO): The deputy agency head, or equivalent, is designated COO, with overall responsibility for improving agency management and performance.

Performance improvement officer (PIO): Agencies are required to designate a senior executive within the agency as PIO, who reports directly to the COO and has responsibilities to assist the agency head and COO with performance management activities.

Goal leader: For each CAP goal, OMB must identify a lead government official—referred to by OMB as a goal leader— responsible for coordinating efforts to achieve each of the goals. For agency performance goals, including APGs, agencies must also designate a goal leader, who is responsible for achieving the goal.

Performance Improvement Council (PIC): Originally created by a 2007 executive order, GPRAMA establishes the PIC in law and included additional responsibilities. The PIC is charged with assisting OMB to improve the performance of the federal government and achieve the CAP goals. Among its other responsibilities, the PIC is to facilitate the exchange among agencies of useful performance improvement practices and work to resolve government-wide or crosscutting performance issues. The PIC is chaired by the Deputy Director for Management at OMB and includes agency PIOs from each of the 24 CFO Act agencies as well as other PIOs and individuals designated by the chair.

Quarterly performance reviews (QPR): For each APG, agencies are required to conduct QPRs to review progress towards the goals and develop strategies to improve performance, as needed. These reviews are to be led by the agency head and COO and include the PIO, relevant goal leaders, and other relevant parties both within and outside the agency.

Performance.gov: OMB is required to develop a single, government- wide performance website to communicate government-wide and agency performance information. The website—implemented by OMB as Performance.gov—is required to make available information on APGs and CAP goals, updated on a quarterly basis; agency strategic plans, annual performance plans, and annual performance reports; and an inventory of all federal programs.

Performance management capacity: The Office of Personnel Management (OPM) is charged with three responsibilities under the act. OPM is to (1) in consultation with the PIC, identify key skills and competencies needed by federal employees to carry out a variety of performance management activities; (2) incorporate these skills and competencies into relevant position classifications; and (3) work with agencies to incorporate these key skills into agency training.

The Executive Branch Has Taken Important Steps to Implement Key GPRAMA Provisions

Since GPRAMA’s enactment in January 2011, OMB and agencies have taken a number of important steps to implement key provisions related to the act’s planning and reporting requirements. In February 2012, OMB identified 14 interim CAP goals concurrent with the submission of the President’s Budget. Nine of the goals related to crosscutting policy areas and 5 covered management improvements. In addition, at the same time, 24 agencies selected by OMB developed 103 APGs for 2012 and 2013, and OMB published information about these goals as well as the CAP goals on Performance.gov, which OMB considers to comprise the federal government performance plan. In December 2012, OMB expanded the information available on the site by providing an update on fiscal year 2012 performance for both sets of goals, and in March 2013, quarterly updates of the site began. All 24 CFO Act agencies are conducting QPRs, according to our survey of PIOs at these agencies. Our 2013 survey indicates that approximately one-third (33 percent) of federal managers across the government are at least somewhat familiar with the QPRs. These and related efforts were based on OMB guidance on implementing the act issued in 2011 and 2012.

As another positive development, OMB and agencies have also put into place key aspects of the act’s performance management leadership roles. We recently reported that, at the agency level, all 24 CFO Act agencies have assigned senior-level officials to the COO, PIO, and goal leader roles. Furthermore, OMB guidance directed agencies with PIOs who are political appointees or other officials with limited-term appointments to appoint a career senior executive to serve as deputy PIO. Nearly all (22) of the CFO Act agencies have assigned officials to the deputy PIO role, according to our PIO survey. PIOs we surveyed reported that most performance management officials (COOs, PIOs, deputy PIOs and goal leaders) had large involvement in four primary tasks that summarize the performance management responsibilities required by GPRAMA: (1) strategic and performance planning and goal setting, (2) performance measurement and analysis, (3) communicating agency progress toward goals, and (4) agency quarterly performance reviews.

At the government-wide level, the PIC has taken steps to meet its requirement to facilitate the exchange of useful practices and tips and tools to strengthen agency performance management. For example, it established the Goal Setting Working Group to help agencies set their 2012 to 2013 APGs; the Internal Agency Reviews Working Group to share best practices for QPRs; and the Business Intelligence Working Group to share tools for data analytics. PIOs we surveyed reported that, in general, they found the PIC helpful and that there was strong agency participation in the PIC and its working groups. However, in April 2013 we reported that the PIC has not routinely assessed its performance and recommended that OMB work with the PIC to conduct formal feedback on the PIC’s performance from member agencies on an ongoing basis; and update the PIC’s strategic plan and review the PIC’s goals, measures, and strategies for achieving performance, and revise them if appropriate.

OMB staff agreed with these recommendations.

In addition, OPM has completed its work identifying key skills and competencies needed by performance management staff and incorporating those skills and competencies into relevant position classifications. OPM identified 15 competencies for performance management staff and published them in a January 2012 memorandum from the OPM Director. It also identified relevant position classifications that are related to the competencies for performance management staff and worked with a PIC working group to develop related guidance and tools for agencies. Furthermore, OPM has taken steps to work with agencies to incorporate the key competencies into agency training. However, we reported in April 2013 that these efforts have been broad- based and not informed by specific assessments of agency training needs. We recommended that, in coordination with the PIC and the Chief Learning Officers Council, OPM (1) identify competency areas needing improvement within agencies, (2) identify agency training that focuses on needed performance management competencies, and (3) share information about available agency training on competency areas needing improvement. OPM agreed with these recommendations and reported that it will take actions to implement them.

The Executive Branch Needs to More Fully Use the GPRAMA Framework to Address Pressing Federal Governance Challenges

OMB and Agencies Have Made Some Progress Addressing Crosscutting Issues, but Are Missing Additional Opportunities

Many of the meaningful results that the federal government seeks to achieve, such as those related to protecting food and agriculture and providing homeland security, require the coordinated efforts of more than one federal agency, level of government, or sector. However, agencies face a range of challenges and barriers when they attempt to work collaboratively. The need for improved collaboration has been highlighted throughout our work over many years, in particular in two bodies of work. First, our reports over the past 3 years identified more than 80 areas where opportunities exist for executive branch agencies or Congress to reduce fragmentation, overlap, and duplication. Figure 1 defines and illustrates these terms.

We found that resolving many of these issues requires better collaboration among agencies. Second, collaboration and improved working relationships across agencies are fundamental to many of the issues that we have designated as high risk due to their vulnerabilities to fraud, waste, abuse, and mismanagement, or most in need of transformation.

For almost 2 decades we have reported on agencies’ missed opportunities for improved collaboration through the effective implementation of GPRA. In our 1997 assessment of the status of the implementation of GPRA, we reported that agencies faced challenges addressing crosscutting issues, which led to fragmentation and overlap. Again, we reported in 2004—10 years after the enactment of GPRA—that there was still an inadequate focus on addressing issues that cut across federal agencies. On a government-wide level, we reported that OMB did not fully implement a government-wide performance plan, as was required by GPRA. Additionally, few agency strategic and performance plans addressed crosscutting efforts and coordination. At that time, almost half of federal managers in our 2003 survey reported that they coordinated program efforts to a great or very great extent with other internal or external organizations. Now, almost 20 years since GPRA’s passage, our work continues to demonstrate that the needed collaboration is not sufficiently widespread. Accordingly, in 2012 we developed a guide on key considerations for implementing collaborative mechanisms. The results of our 2013 survey of federal managers show that the percentage of managers reporting that they use information obtained from performance measurement when coordinating program efforts with other internal or external organizations to a great or very great extent has not increased since 1997. Based on this survey, an estimated 23 percent of the managers reported that they coordinated program efforts to a small extent or not at all.

The following three examples, among many, highlight the need for improved collaboration to help address crosscutting issues:

Food safety: One area that has been identified in both bodies of work is the fragmented nature of federal food safety oversight. The U.S. food safety system is characterized by inconsistent oversight, ineffective coordination, and inefficient use of resources; these characteristics have placed the system on our high-risk list since 2007 and in all three of our annual reports on fragmentation, overlap, and duplication. We have reported that the U.S. Department of Agriculture (USDA) and the Food and Drug Administration (FDA), the two primary agencies responsible for food safety, have taken some steps to increase collaboration. However, agencies have not developed a government-wide performance plan for food safety that includes results-oriented goals and performance measures, as we recommended when we put federal oversight of food safety on the high-risk list in January 2007. In the absence of this plan, we have reported cases of fragmentation, overlap, and duplication. The 2010 nationwide recall of more than 500 million eggs because of Salmonella contamination highlights a negative consequence of this fragmentation. Several agencies have different roles and responsibilities in the egg production system. Through the Food Safety Working Group, federal agencies have taken steps designed to increase collaboration in some areas that cross regulatory jurisdictions. For example, both USDA and FDA set goals to reduce illness from Salmonella within their own areas of egg safety jurisdiction by the end of 2011 and developed a memorandum of understanding on information sharing regarding egg safety. While such actions are encouraging, without a government-wide performance plan for food safety, fragmentation, overlap, and duplication is likely to continue.

Climate change: Climate change is a complex, crosscutting issue that poses risks to many environmental and economic systems— including agriculture, infrastructure, ecosystems, and human health— and presents a significant financial risk to the federal government. Among other impacts, climate change could threaten coastal areas with rising sea levels, alter agricultural productivity, and increase the intensity and frequency of severe weather events such as floods, drought, and hurricanes. Weather-related events have cost the nation tens of billions of dollars in damages over the past decade. For example, in 2012, the administration requested $60.4 billion for Superstorm Sandy recovery efforts. However, the federal government is not well positioned to address the fiscal exposure presented by climate change, partly because of the complex, crosscutting nature of the issue. Given these challenges and the nation’s precarious fiscal condition, we added “Limiting the Federal Government’s Fiscal Exposure to Climate Change” to our high-risk list in 2013. In adding climate change to this list, we reported that the federal government would be better positioned to respond to the risks posed by climate change if federal efforts were more coordinated and directed toward common goals. In October 2009, we recommended that the appropriate entities within the Executive Office of the President, in consultation with relevant federal agencies, state and local governments, and key congressional committees of jurisdiction, develop a strategic plan to guide the nation’s efforts to adapt to climate change, including the establishment of clear roles, responsibilities, and working relationships among federal, state, and local governments. In written comments, the Council on Environmental Quality generally agreed with the report’s recommendations, noting that leadership and coordination is necessary within the federal government to ensure an effective and appropriate adaptation response and that such coordination would help to catalyze regional, state, and local activities. Some actions have subsequently been taken to improve the coordination of federal adaptation efforts, including the development of an interagency climate change adaptation task force.

Federal disability programs: In June 2012, we identified 45 programs in nine agencies that helped people with disabilities obtain or retain employment, reflecting a fragmented system of services and supports. Many of these programs overlapped in whom they served and the types of services they provided. Such fragmentation and overlap may frustrate and confuse program beneficiaries and limit the overall effectiveness of the federal effort. Having extensive coordination and overarching goals can help address program fragmentation. Although we identified promising coordination efforts among some programs, most reported not coordinating with each other, and some officials told us they lacked funding and staff time to pursue coordination. Coordination efforts can be enhanced when programs work toward a common goal; however, the number and type of outcome measures used by the 45 programs varied greatly. To improve coordination, efficiency, and effectiveness, we suggested that OMB consider establishing government-wide goals for employment of people with disabilities. Consistent with this suggestion, OMB officials stated that the Domestic Policy Council began an internal review intended to improve the effectiveness of some disability programs through better coordination and alignment. However, as we noted in our 2013 high-risk update, OMB still needs to maintain and expand its role in improving coordination across programs—such as the 45 we identified—that support employment for those with disabilities, and ultimately work with all relevant agencies to develop measurable government-wide goals to spur further coordination and improved outcomes for those who are seeking to find and maintain employment.

On the other hand, we have recently highlighted progress that the executive branch and Congress have made in addressing areas that we previously identified as being at risk of fragmentation, overlap, and duplication. For example, the nation’s surface transportation system is critical to the economy and affects the daily life of most Americans. However, in our 2011 annual report on fragmentation, overlap, and duplication, we reported that over the years federal surface transportation programs grew increasingly fragmented. At the core of this fragmentation was the fact that federal goals and roles for the programs were unclear or conflicted with other federal priorities, programs lacked links to the performance of the transportation system or of the grantees, and programs did not use the best tools to target investments in transportation to the areas of greatest benefit. Accordingly, since 2004, we have made several recommendations and matters for congressional consideration to address the need for a more goal-oriented approach to surface transportation, introduce greater performance and accountability for results, and break down modal stovepipes. As we reported in February 2013, there was progress in clarifying federal goals and roles and linking federal programs to performance when the Moving Ahead for Progress in the 21st Century Act was enacted in July 2012. The act addressed fragmentation by eliminating or consolidating programs, and made progress in clarifying federal goals and roles and linking federal programs to performance to better ensure accountability for results.

The challenge of collaboration has also been highlighted in our reviews of related GPRAMA requirements, such as those for CAP goals, APGs, and QPRs. While agencies have implemented some of these provisions, these efforts have not included all of the relevant agency, program, and other contributors. When agencies do not include all relevant contributors, they may miss important opportunities to work with others who are instrumental to achieving intended outcomes.

Including all contributors is also a requirement of GPRAMA.

At the government-wide level, OMB is required to list all of the agencies, organizations, program activities, regulations, tax expenditures, policies, and other activities that contribute to each CAP goal. With relevant stakeholders, OMB is required to review the progress of all contributors towards each goal on a quarterly basis.

At the agency level, agencies are required to identify the various federal organizations, programs, and activities—both within and external to the agency—that contribute to each goal, and for APGs, review progress on a quarterly basis with relevant stakeholders.

However, as shown in table 1, we have found that agencies are not including all stakeholders as they implement GPRAMA.

While we continue to see challenges to collaboration across federal agencies, as a positive development, our survey of federal managers shows that reported collaboration increases when individuals contribute to the CAP goals, APGs, or QPRs. Our 2013 survey data indicate that 58 percent of federal managers reported they were somewhat or very familiar with CAP goals. Among these individuals, federal managers who viewed their programs as contributing to CAP goals to a great or very great extent were more likely to report collaborating outside their program to a great or very great extent to help achieve CAP goals, as figure 2 shows.

We saw a similar pattern in responses from managers who were familiar with the APGs and the extent to which their programs contributed to the APGs. Eighty-two percent of federal managers reported they were somewhat or very familiar with APGs. Among these individuals, those who viewed their programs as contributing to APGs to a great or very great extent were more likely to report collaborating outside their program to a great or very great extent to help achieve APGs, as figure 3 shows.

While the questions on our survey were designed to examine collaboration outside individual programs, they were not designed to distinguish between collaboration within or outside agency boundaries. As discussed in table 1, we found that collaboration was more common within agencies than between agencies. This may be appropriate in some cases; however, in other cases this might point to a need for broader inclusion of external stakeholders. We found that more managers reported collaborating with officials external to their agency to a great or very great extent when they also reported that their programs were involved in QPRs to a similar extent.

Tax expenditures represent a significant federal investment. If the Department of the Treasury (Treasury) estimates are summed, an estimated $1 trillion in revenue was forgone from the 169 tax expenditures reported for fiscal year 2012, nearly the same as discretionary spending that year. For some tax expenditures, forgone revenue can be of the same magnitude or larger than related federal spending for some mission areas. For example, in fiscal year 2010, tax expenditures represented about 78 percent ($132 billion) of federal support for housing. Since 1994, we have recommended greater scrutiny of tax expenditures, as periodic reviews could help determine how well specific tax expenditures work to achieve their goals and how their benefits and costs compare to those of spending programs with similar goals. In November 2012, we issued a guide that identifies criteria for assessing tax expenditures and provides questions for the Congress to ask about a tax expenditure’s effectiveness. However, OMB has not developed a framework for reviewing tax expenditure performance, as we recommended in June 1994 and again in September 2005. Because OMB has not yet established such a framework, little is known about how tax expenditures contribute to broad federal outcomes and how they are related to spending programs seeking the same or a similar outcome.

OMB guidance has shown some progress in addressing how agencies should incorporate tax expenditures in strategic plans and annual performance plans and reports, as we first recommended in September 2005. GPRAMA specifically requires OMB to identify tax expenditures among the various federal activities that contribute to each CAP goal, when applicable. Although the act does not explicitly require agencies to identify tax expenditures among the various federal programs and activities that contribute to their performance goals, OMB’s guidance directs agencies to do so for their APGs, which are a small subset of their performance goals. However, our review of the APGs developed for 2012 to 2013 found that only one agency, for one of its APGs, identified two relevant tax expenditures. We recently reported that OMB was missing an opportunity to more broadly identify how tax expenditures contribute to each agency’s overall performance.

Even among the CAP goals, OMB and agencies are missing opportunities to identify tax expenditures as contributors. In the original information on Performance.gov in February 2012, OMB included tax expenditures as potential contributors for 5 of the 14 CAP goals (veteran career readiness, entrepreneurship and small businesses, energy efficiency, job training, and improper payments). In the December 2012 and March 2013 updates to Performance.gov, only two goals (veteran career readiness and improper payments) discussed two tax expenditures, which represent $2.7 billion or 0.3 percent of the $1 trillion sum across the tax expenditures listed by Treasury. Tax expenditures were no longer mentioned as contributing to the entrepreneurship and small businesses, energy efficiency, and job training CAP goals. For example, under the energy efficiency CAP goal, OMB originally listed both spending programs and tax expenditures that contribute to the goal. However, in the December 2012 update to Performance.gov, OMB had deleted all of the tax expenditures even though many of these tax expenditures remained unchanged. In one case, OMB deleted the credit for energy efficiency improvements to existing homes (estimated at $780 million for fiscal year 2012), but highlighted the Department of Energy’s (DOE) weatherization assistance spending program (estimated at $68 million in obligations for fiscal year 2012), even though both fund residential energy efficiency. Overall, we identified eight tax expenditures, totaling $2.4 billion in forgone revenue, which share the purpose of achieving energy efficiency, but are no longer identified as potential contributors. When asked about these changes, OMB staff shared that for the entrepreneurship and small business CAP goal the goal leaders narrowed the focus of the goal, which resulted in an updated list of contributing programs and activities that no longer included tax expenditures. For the energy efficiency and job training CAP goals, OMB staff told us that the exclusion of tax expenditures from the December 2012 and March 2013 updates was an oversight. OMB staff told us they planned to add the appropriate tax expenditures as contributors to those goals in the next quarterly update to Performance.gov, which occurred in June 2013. However, none were added to the job training CAP goal update, and as of June 19, 2013, the energy efficiency CAP goal had not yet been updated.

However, these examples raise concerns as to whether OMB previously ensured all relevant tax expenditures were identified as contributors to the 14 CAP goals when they were published in February 2012, especially since only 5 CAP goals listed tax expenditures as contributors at that time. We have previously reported that, as with spending programs, tax expenditures represent a substantial federal commitment to a wide range of mission areas. Given the lack of scrutiny tax expenditures receive compared to spending programs—especially absent a comprehensive framework for reviewing them—it is possible that additional tax expenditures should have been identified and included as contributors to one or more of the other 9 CAP goals. Moreover, for the 2 CAP goals where tax expenditures were listed as contributors and mistakenly removed, it is unclear if OMB and the goal leaders assessed the contributions of those tax expenditures toward the CAP goal efforts, since they were not listed in the December 2012 and March 2013 updates. Without information about which tax expenditures support these goals and measures of their performance, Congress and other decision makers will not have the needed information to assess overall federal contributions towards desired results, and the costs and relative effectiveness associated with those contributions.

Ensuring Performance Information Is Useful and Used by Managers to Improve Results Remains a Weakness, but Key Performance Management Practices Hold Promise

GAO Continues to Find Widespread Progress Is Needed to Use Data to Drive Performance

We have previously reported that data-driven decision making leads to better results. Moreover, we have reported that if agencies do not use performance measures and performance information to track progress toward goals, they may be at risk of failing to achieve their goals. The textbox illustrates this problem in the high risk area of the Department of Defense’s (DOD) approach to business transformation.

DOD Is Not Regularly Reviewing Performance Information to Assess Progress towards Goals in Transforming Its Business Operations In 2005, we identified DOD’s approach to business transformation as high-risk because DOD had not established clear and specific management responsibility, accountability and control over its business transformation and it lacked a plan with specific goals, measures, and mechanisms to monitor progress. We subsequently reported that DOD made improvements to strengthen its management approach, but we also identified additional steps that are needed. For example, DOD has broadly outlined a performance management approach, and established governance structures, such as the Defense Business Council, to help monitor progress in its business transformation efforts. However, we found the Council had not regularly reviewed performance data and when reviews did occur, it did not have sufficient information to assess progress. To enhance DOD’s ability to set strategic direction for its business transformation efforts, better assess overall progress toward business transformation goals, and take any necessary corrective actions, we recommended in February 2013 that DOD take a number of steps to improve its approach to performance management. DOD agreed with this recommendation and said it would continue to improve and institutionalize the Council’s operations.

In the first 4 months of 2013 alone, we issued numerous testimonies and reports that illustrate how performance management weaknesses can hinder agencies’ abilities to achieve critical results. This work also illustrates that the scope of these problems is widespread, affecting agencies such as DOD, Treasury, the Departments of Transportation (DOT), Homeland Security (DHS), Health and Human Services, Housing and Urban Development (HUD), and State. The impact of these weaknesses is far reaching as well: These agencies are responsible for performing functions that affect every aspect of Americans’ lives, from education, healthcare, and housing to national security and illicit drug use, as described in the textbox.

Office of National Drug Control Policy Has Established a Performance Monitoring System to Address Illicit Drug Use, but Not Yet Reported on Results The public health, social, and economic consequences of illicit drug use, coupled with the nation’s constrained fiscal environment, highlight the need for federal programs to use resources efficiently and effectively to address this problem. However, we reported in March 2013 that the Office of National Drug Control Policy and federal agencies have not made progress toward achieving most of the goals in the 2010 National Drug Control Strategy, although they reported to be on track to implement most Strategy action items in support of these goals. In April 2012, the Office established the Performance Reporting System, a monitoring mechanism intended to provide specific, routine information on progress toward Strategy goals and help identify factors for performance gaps and options for improvement. We reported that this could help increase accountability for improving results and identify ways to bridge the gap that existed between the lack of progress toward the Strategy’s goals and the strong progress made on implementing the Strategy’s actions. While this was promising, the Office does not plan to report on results until later in 2013, and until then, operational information is not available to evaluate its effectiveness.

GAO, Office of National Drug Control Policy: Office Could Better Identify Opportunities to Increase Program Coordination, GAO-13-333, (Washington, D.C.: Mar. 26, 2013).

Surveys of Federal Managers Show that Managers’ Use of Performance Information for Decision Making Has Stagnated

Our prior work has shown that performance information can be used across a range of management functions to improve results, from setting program priorities and allocating resources to taking corrective action to solve program problems. Since our 2007 survey there was statistically significant improvement on two survey items related to use of performance information. More managers reported in 2013—after GPRAMA’s enactment and initial implementation—that they used performance information to a great or very great extent in developing program strategy and refining program performance measures. However, the 2013 improvement on the refining program performance measures item followed an earlier decline and does not represent an improvement in comparison to our 1997 survey results. While there was also a statistically significant change between 1997 and 2013 in the percentage of managers who reported to a great or very great extent that they used performance information in adopting new program approaches or changing work processes, the initial decline on this item occurred between our 1997 and 2000 surveys with no significant changes since then. Overall, our periodic surveys of federal managers since 1997 indicate that with the few exceptions described above, the use of performance information has not changed significantly at the government- wide level, as shown in figure 4.

In addition, we introduced an item in the 2013 survey on streamlining programs, a performance management activity that can help address the overlap and duplication challenges and opportunities described earlier in this report. Less than half of federal managers (44 percent) reported to a great or very great extent that they used performance information for “streamlining programs to reduce duplicative activities.”

Our prior work has identified practices that can promote the use of performance information for management decision making, such as leadership demonstrating commitment to using performance information, communicating performance information frequently and effectively, ensuring that performance information is useful, and building capacity to use performance information. Moreover, many of the requirements put in place by GPRAMA reinforce the importance of these practices. Our past government-wide surveys of federal managers indicated that these key practices were not always being employed across various agencies. Our 2013 survey suggests that effectively adopting these practices continues to be a substantial weakness across the government as described below.

Demonstrating leadership commitment: Our prior work has shown that the demonstrated commitment of leadership and management to achieving results and using performance information can encourage the federal workforce to apply the principles of performance management. GPRAMA requires top leadership involvement in performance management, such as requiring agency leadership to routinely review performance information and progress toward APGs during the QPRs. However, results from our 2013 survey show almost no statistically significant changes in managers’ perceptions of their leaders’ and supervisors’ attention and commitment to the use of performance information since our last survey in 2007. The only statistically significant change from 2007 to 2013 was a decline in the percentage of managers that agreed to a great or very great extent that their agencies’ top leadership demonstrates a strong commitment to achieving results, from 67 percent to 60 percent. Moreover, less than two-thirds of managers agreed to a great or very great extent with other survey items related to leadership commitment and attention to performance information, as shown in figure 5.

Communicating performance information: Our prior work showed that communicating performance information frequently and effectively throughout an agency can help managers to inform staff and other stakeholders of their commitment to achieve the agency’s goals and to keep these goals in mind as they pursue their day-to-day activities. Frequently reporting progress toward achieving performance targets also allows managers to review the information in time to make improvements. GPRAMA includes requirements for communicating performance information, such as sharing performance information at least quarterly and directing agencies to update performance indicators on their websites at least annually. However, there was no statistically significant change between 2007 and 2013 in the percentage of federal managers agreeing to a great or very great extent that agency managers at their level effectively communicate performance information on a routine basis (41 percent in 2013 and 43 percent in 2007). Our analysis suggests that easy access to performance information is related to the effective communication of performance information. Of the 49 percent of federal managers who agreed to a great or very great extent that performance information is easily accessible to managers at their level, 62 percent also agreed that agency managers at their level effectively communicate performance information on a routine basis to a great or very great extent. Conversely, of the 19 percent that agreed to only a small or no extent that performance information is easily accessible to managers at their level, only 9 percent also agreed that agency managers at their level effectively communicate performance information on a routine basis to a great or very great extent.

Ensuring performance information is useful: As we previously reported, to facilitate the use of performance information, agencies should ensure that information meets various users’ needs for completeness, accuracy, consistency, timeliness, validity, and ease of use. GPRAMA introduced several requirements that could help to address these various dimensions of usefulness. For example, agencies must disclose more information about the accuracy and validity of their performance data and actions to address limitations to the data. Without useful performance information, it is difficult to monitor agencies’ progress toward critical goals, such as improving veterans’ access to health care provided by the Department of Veterans Affairs (VA), as illustrated in the textbox.

Performance Information on Veterans’ Wait Times for Medical Appointments Was Unreliable The Veterans Health Administration (VHA), within the VA, provided nearly 80 million outpatient medical appointments to veterans in fiscal year 2011. Although access to timely medical appointments is important to ensuring veterans obtain needed care, long wait times and inadequate scheduling processes have been persistent problems. VHA is implementing a number of initiatives to improve veterans’ access to medical appointments such as use of technology to interact with patients and provide care. However, we testified in March 2013 that certain aspects of VHA’s policies and policy implementation contributed to unreliable performance information on veterans’ wait times. VA concurred with our recommendations and identified actions planned or under way to address them.

GAO, VA Health Care: Appointment Scheduling Oversight and Wait Time Measures Need Improvement, GAO-13-372T (Washington, D.C.: Mar. 14, 2013).

Responses to four survey items on hindrances related to the usefulness of performance information indicate some limited improvement. There was a statistically significant improvement between the 2007 and 2013 surveys on two of these four items (shown as declines because they concern hindrances), but no significant change otherwise, as illustrated in figure 6.

In addition, related survey items introduced after 1997 showed no significant change between 2007 and 2013, with about 40 percent of managers agreeing to a great or very great extent that “agency managers at my level take steps to ensure that performance information is useful and appropriate” and 36 percent agreeing to the same extent that “I have sufficient information on the validity of the performance data I use to make decisions.”

Despite these limited improvements, the overall picture from the 2013 results—with about one-fifth to nearly one-third of managers reporting hindrances, as indicated in figure 6, and less than half agreeing with most of the positive statements about the format, timeliness, and accessibility of their performance information in figure 7—remains a major concern.

Building capacity to use performance information: We have previously reported that building the capacity to use performance information is critical to using performance information in a meaningful fashion, and that inadequate staff expertise, among other factors, can hinder agencies from using performance information. GPRAMA lays out specific requirements for OPM to identify skills and competencies for performance management functions, among other actions, which reinforce the importance of staff capacity to use performance information. Managers’ survey responses and our recent work indicate areas of weakness in agencies’ analysis and evaluation tools and staff’s skills and competencies, both of which are critical components of performance management capacity. About a third (36 percent) of managers reported in 2013 that they agreed to a great or very great extent that their agencies have sufficient analytical tools for managers at their levels to collect, analyze, and use performance information. Furthermore, less than a third of managers reported that their agencies were investing resources to improve the use and quality of performance information. Thirty percent of managers reported that they agree to a great or very great extent that the programs they are involved with have sufficient staff with the knowledge and skills needed to analyze performance information.

Additionally, our recent work found gaps in performance management competencies among agency staff. Although PIOs we surveyed at 24 agencies in 2012 for our April 2013 report on performance management leadership roles reported that their staff generally possessed core competencies identified by OPM for performance management staff, certain competencies—performance measurement, information management, organization performance analysis, and planning and evaluating—were present to a lesser extent.

Training is one way agencies can address a lack of staff capacity to use performance information, as illustrated in the sidebar. Between 1997 and 2013, there was a statistically significant increase in the percentage of managers reporting that their agencies made training available in the past 3 years on most of the performance management tasks we asked about. However, between 2007 and 2013, there was either no significant change or a decline in the percentage of managers responding positively to the same items, as shown in figure 8.

Our prior work has indicated that effective data-driven reviews can serve as a leadership strategy, requiring leadership and other responsible parties to come together to review performance information and progress toward results and identify important opportunities to drive performance improvements. According to our 2012 survey of PIOs at 24 agencies, the majority (21 of 24) reported that actionable opportunities for performance improvement are identified through the reviews at least half the time. In addition, most officials we interviewed at DOE, Treasury, and the Small Business Administration (SBA) attributed improvements in performance and decision making to their QPRs. The textbox presents one such improvement described by officials at Treasury.

Treasury Credits QPRs with Decision to Stop Minting $1 Coins for Circulation and Saving U.S. Government Millions Treasury’s Deputy Secretary said that it was a performance review session with the U.S. Mint that first led him to question the direction they had been taking with the $1 coin. Performance data he reviewed for the meeting indicated that the Mint was producing 400 million new $1 coins annually, while the Federal Reserve already had 1.4 billion existing ones in storage. Digging deeper, he learned that the Federal Reserve had previously estimated that there were enough $1 coins to meet demand for more than a decade.

This estimate was based on the assumption that demand would remain at 2012 levels.

While our case studies and survey of PIOs indicated the benefits of QPRs, our 2013 government-wide federal managers’ survey indicated that the majority of federal managers are not familiar with the QPRs at their agencies, although a greater percentage of Senior Executive Service (SES) managers reported that they were familiar with the QPRs, as shown in figure 9.

Our analysis suggests that, while familiarity with QPRs may be somewhat limited government-wide, it is positively related to managers’ perceptions of their leadership’s demonstrated commitment to using performance information. Of the 12 percent of all federal managers who reported they were very familiar with QPRs, 76 percent agreed that their top leadership demonstrates a strong commitment to using performance information to guide decision making to a great or very great extent. In contrast, of the 66 percent who reported they were not familiar with QPRs, 36 percent agreed to a great or very great extent with the same statement.

Similarly, our analysis suggests that being the subject of a QPR is positively related to the extent to which managers view the QPRs as being used to accomplish certain purposes to a great or very great extent. For example, federal managers who reported that their programs have been the subject of a QPR to a great or very great extent were more likely to report that their agencies use QPRs to identify problems or opportunities than those who reported that their programs have been the subject of a QPR to a moderate or small or no extent. Figure 10 shows this trend, along with a similar one for federal managers’ ratings of agency leadership use of QPRs to help achieve performance goals.

Our analysis also suggests that being the subject of a QPR may be positively related to managers’ perceptions of their agencies employment of key practices that we have previously reported can promote successful data-driven performance reviews. For example, federal managers who reported that their programs have been the subject of a QPR to a great or very great extent were more likely to report that the reviews included key practices, such as leadership actively participating in reviews, than those who reported that their programs have been the subject of QPRs to a moderate or small or no extent. This trend and similar ones for other key practices are shown in figure 11.

Federal managers’ responses to items about other key practices—holding QPRs on a regular, routine basis and having a process for following up on QPRs—were similarly related to the extent to which managers’ programs were the subject of a QPR.

Agencies Have Taken Steps to Align Daily Operations with Agency Results, but Some Continue to Face Difficulties Measuring Performance

Agencies Have Taken Steps to Align Individual Performance with Results

It is important for individuals to see a connection between their daily operations and results to help understand how individual performance can contribute to organizational success. While our past work has shown that agencies have encountered challenges linking individual performance with broader organizational results, progress has been made over the last decade in establishing this linkage and holding individuals accountable for organizational results through performance management systems. For example, while agencies have been required to hold senior executives accountable for their individual and organizational performance by linking performance expectations with GPRA-required goals since 2000, OPM and OMB have continued to reinforce the importance of this alignment in improvements in SES performance management. Most recently, in January 2012, OPM and OMB released a government-wide performance appraisal system for senior executives that provides agencies with a standard framework for managing the performance of its executives. While striving to provide greater clarity and equity in the development of performance standards and link to compensation, among other things, the Directors of OPM and OMB stated that the new system is intended to provide agencies with the necessary flexibility and capability to customize the system in order to meet their needs. As part of this framework, agencies are to identify expectations for the senior executives that focus on measurable outcomes from the strategic plan or other measurable outputs and outcomes clearly aligned to organizational goals and objectives.

In addition, the Goals-Engagement-Accountability-Results (GEAR) model, established in 2011, focuses on aligning employee performance with organizational performance, creating a culture of engagement, and implementing accountability at all levels, among other things. The GEAR model outlines a series of recommended actions for agencies to adopt in order to help improve employee and organizational performance. We reported in September 2012 that DOE’s GEAR implementation plan includes aligning employee performance management with organizational performance management and developing training to support these goals, which along with initiating knowledge-sharing activities, will promote improvement of DOE’s organizational performance, according to DOE officials. We have ongoing work looking at GEAR implementation in the five pilot agencies and plan to issue the results of our work later in 2013.

To further institutionalize individual accountability for achieving results, GPRAMA established in law several mechanisms that help individuals and agencies see this connection and hold them accountable for their contributions to agency and government-wide goals. As we recently reported, agency leaders should hold goal leaders and other responsible managers accountable for knowing the progress being made in achieving goals and, if progress is insufficient, understanding why and having a plan for improvement including improvements in the quality of the data to help ensure they are sufficient for decision making. For example, PIOs are responsible for, among other things, assisting the agency head and COO in developing and using performance measures specifically for assessing individual performance in the agency. QPRs offer an opportunity for organizational performance to be assessed and responsible officials to be held accountable for addressing problems and identifying strategies for improvement.

As agencies implement the accountability provisions of GPRAMA, they will need to ensure managers have decision-making authority commensurate with the responsibility to identify and address performance problems as they arise. Since our 1997 government-wide survey of federal managers, SES managers have reported improvements in accountability for agency goals and results and the decision-making authority to help achieve agency goals. However, there has been a gap between SES managers’ perceptions of their accountability for program performance as opposed to their decision-making authority since our initial survey in 1997. In 2013, 80 percent of SES managers reported that they are held accountable for the results of the programs for which they are responsible to a great or very great extent, while 61 percent reported that they have the decision-making authority they need to help the agency achieve its strategic goals, a 19 percentage point difference. See figure 12.

Using performance information in employee performance management helps individuals track their performance and progress toward achieving organizational goals and can help emphasize the importance of individual contributions to organizational success. However, the percentage of federal managers reporting use of performance information in employee performance management to a great or very great extent has stagnated with no statistically significant change in reported use from 1997 to 2013. See figure 13.

A fundamental element in an organization’s efforts to manage for results is its ability to set meaningful goals for performance and to measure progress toward those goals. In our 1996 Executive Guide, we underscored the importance of taking a balanced approach to setting goals and measuring performance. If a balance across an organization’s various priorities does not exist, the measures in place can overemphasize some goals and create skewed incentives. This need for agencies to have a balanced set of performance measures was reinforced in GPRAMA, which calls for agencies to develop a variety of measures, such as output, outcome, customer service, and efficiency, across program areas.

As we have previously reported, based on our government-wide federal managers surveys, federal managers reported a statistically significant increase in the presence of different types of performance measures for their programs to a great or very great extent following initial implementation of GPRA. Despite this early progress in establishing a variety of performance measures, since our 2003 federal managers survey, there generally has been no statistically significant increase in the reported presence of these measures to a great or very great extent. More recently, as illustrated in figure 14, the only statistically significant increase between 2007 and 2013 is in the percentage of managers reporting the presence of quality measures.

We have further found over the years and through our more recent work that there has been uneven development of outcome-oriented performance measures across federal programs, even though agencies have been responsible for measuring program outcomes, among other things, since the passage of GPRA in 1993. As demonstrated in the textbox, outcome-oriented performance measures help agencies determine if the program is achieving its intended purpose. Additionally, these performance measures are essential for assessing the vast number of results of federal efforts that span multiple agencies and organizations.

GAO Has Reported on Agency Difficulties in Developing and Using Outcome Measures In May 2006, we recommended that USDA and DHS adopt meaningful performance measures for assessing the effectiveness of the Agriculture Quarantine Inspection (AQI) program at intercepting foreign pests and disease on agricultural materials entering the country by all pathways and posing a risk to U.S. agriculture.

We reported in March 2013 that the Federal Emergency Management Agency has not yet established clear, objective, and quantifiable capability requirements and performance measures to identify capability gaps in a national preparedness assessment, as recommended in our March 2011 report.

We reported in April 2013 that the Federal Communications Commission, DHS, DOD, and Department of Commerce had taken a variety of actions to support the security of the nation’s communications networks, including ones related to developing cyber policy and standards, securing Internet infrastructure, sharing information, supporting national security and emergency preparedness, and promoting sector protection efforts.

GAO, Homeland Security: Management and Coordination Problems Increase the Vulnerability of U.S. Agriculture to Foreign Pests and Disease, GAO-06-644 (Washington, D.C.: May 19, 2006). GAO, Homeland Security: Agriculture Inspection Program Has Made Some Improvements, but Management Challenges Persist, GAO-12-885 (Washington, D.C.: Sept. 27, 2012). GAO-11-318SP. GAO, Communications Networks: Outcome-Based Measures Would Assist DHS in Assessing Effectiveness of Cybersecurity Efforts, GAO-13-275 (Washington, D.C.: Apr. 3, 2013).

Our work over the last 20 years has identified difficulties agencies face in measuring performance across various program types, such as regulations and grants. Some commonly reported difficulties that cut across the various program types include: accounting for factors that are both outside of an agency’s control and impact the results of a program; developing appropriate performance measures, especially for programs without a clearly defined purpose or that require a long time period to achieve intended results; and obtaining complete, timely, and accurate performance information of the program.

Illustrative examples from our recent work that show how agencies have experienced difficulties in measuring program performance are provided in table 2. In our 2013 annual report on fragmentation, overlap, and duplication, we identified the need for improving the measurement of performance and results—including program evaluation—as a theme that cuts across our suggested actions to address fragmentation, overlap, and duplication in federal agencies.

While some agencies have faced difficulties in measuring program performance, some progress has been made in developing performance measures and using the resulting performance information to measure performance in the applicable program area. For example:

HUD has made progress in measuring grant program performance.

As we reported in November 2011, HUD measured progress toward some green building goals by collecting energy consumption data for participating properties receiving grants or loans under its Green Retrofit Program for Multifamily Housing before and after the properties are retrofitted and planned to use this data to calculate savings and evaluate effectiveness. In January 2011, we reported that the Federal Railroad Administration (FRA) has created a set of performance goals and measures that address important dimensions of program performance related to its regulatory safety activities. In its proposed fiscal year 2011 budget, FRA included specific safety goals to reduce the rate of train accidents caused by various factors, including human errors and track defects. These goals were quantitative, with a targeted accident rate per every million train miles. Collecting such accident data equips FRA with a clear way to measure whether or not those safety goals are met. FRA’s budget request has also linked FRA’s performance goals and measures with DOT’s strategic goals.

Moving forward, we will continue to examine the availability and use of performance measures across a variety of program types and update our work in this area. Given that we have found that agencies across the federal government have experienced similar difficulties in measuring the performance of different program types and have not made consistent progress in addressing them, a comprehensive examination of these difficulties is needed. The PIC could help facilitate this examination. As discussed earlier, GPRAMA requires the PIC, in part, to resolve crosscutting performance issues and facilitate the exchange of practices that have led to performance improvements within specific programs or agencies or across agencies. Although measuring the performance of different program types is a significant and long-standing challenge, the PIC has not yet addressed this issue in a systematic way, such as through a working group to identify common difficulties in developing and using performance measures to assess program performance and share best practices from instances in which agencies have overcome these difficulties. Without a comprehensive examination, it will be difficult for the PIC and agencies to fully understand these measurement issues and develop a crosscutting approach to help address them, which will likely result in agencies experiencing difficulties in measuring program performance in the future.

Communication of Performance Information Could Better Meet Users’ Needs

Federal Managers Reported that Performance Information Is Not Always Available or Easily Accessible to Federal Managers or the Public

According to our 2013 survey of federal managers, 34 percent reported that performance information is easily accessible to agency employees to a great or very great extent, while 17 percent reported that their agency’s performance information is easily accessible to the public to a great or very great extent. Survey data also indicate that agencies are not communicating to their employees about contributions to CAP goals or their progress toward achieving APGs. In fact, of the 58 percent of federal managers who indicated they were familiar with CAP goals, 22 percent reported that their agency has communicated to its employees on those goals to a great or very great extent. Of the 82 percent of federal managers who indicated familiarity with APGs, 40 percent reported that their agency has communicated on progress toward achieving them to great or very great extent.

We recently reported that Performance.gov, as the central repository for federal government performance information, can assist in oversight and lead to a greater focus within government on the activities and efforts necessary to improve performance. OMB’s stated goals for Performance.gov include, among others, providing both a public view into government performance to support transparency as well as providing executive branch management capabilities to enhance senior leadership decision making. According to OMB staff, OMB will maintain responsibility for the website, but going forward, the plans are that the effort will be driven more by the General Services Administration (GSA) and the PIC, with GSA continuing to provide technological support. For future development of Performance.gov, OMB, the PIC, and GSA are working with federal agencies to develop the Performance Management Line of Business that, according to OMB staff, will standardize the collection and reporting of performance information by agencies.

Performance.gov has the potential to increase the accessibility of performance information for users both inside and outside the federal government. An analysis of statements from OMB and GSA staff, agency officials, and feedback we obtained from potential users, however, indicates that there are varying expectations regarding the primary uses of Performance.gov. For example, OMB and GSA staff emphasized that they have viewed Performance.gov as a tool for agencies to support cross-agency coordination and efforts to achieve agency goals. Consistent with this, OMB staff said that Performance.gov has been used to facilitate conversations between OMB examiners and agency managers about progress on APGs. While most officials we interviewed said that OMB had collected feedback from the agencies in the development of Performance.gov, officials from most of these agencies also said that Performance.gov is not being used as a resource by agency leadership or other staff, as they have information sources tailored to meet their needs, and Performance.gov does not contain critical indicators or the ability to display some visualizations used for internal agency performance reviews. In addition, a performance management practitioner and other potential users of the website noted that the detailed, technical nature of Performance.gov seemed primarily oriented toward a government rather than a public audience.

According to OMB staff, the specific legal requirements of GPRAMA have been the primary framework used to guide efforts to develop Performance.gov thus far. They noted that they have been focused on working to comply with these requirements by providing information on CAP goals and APGs, and by establishing a phased development plan for the integration of additional information from agency strategic plans, performance plans, and performance reports. OMB and GSA staff members have said, however, that the leading practices for developing federal websites will be helpful in guiding the future development of Performance.gov. OMB and GSA staff have also noted that as the phased development of Performance.gov unfolds, they expect to use broader outreach to, and usability testing with, a wider audience, including members of the public, to make Performance.gov more “public- facing” and “citizen-centric.”

In accordance with this transition, we recommended in June 2013 that OMB work with GSA and the PIC to clarify the specific ways that intended audiences could use the information on Performance.gov. HowTo.gov, a leading source of best practices and guidance on the development of federal government websites, recommends identifying the purposes of a website, and the ways in which specific audiences could use a website to accomplish various tasks, and then structuring information and providing tools to help visitors quickly complete these tasks. With greater clarity about the intended uses of Performance.gov, OMB and GSA should have sufficient direction to design Performance.gov to make it a relevant and accessible source of information for a variety of potential users including those specified under GPRAMA—members and committees of Congress and the public.

In the same report, we also recommended that OMB should work with GSA and the PIC to systematically collect information on the needs of intended audiences and collect recommended performance metrics that help identify improvements to the website. For example, HowTo.gov practices recommend that a website use consistent navigation. Although users we interviewed had mixed opinions on the organization and navigation of Performance.gov, simplifying the website’s navigation, adding an effective internal search engine, and providing an appropriate level of detail and information for intended audiences could increase the overall usability of Performance.gov. Outreach and testing on the ease of navigation and searching would help OMB systematically collect information on the needs of various audiences and how these could be addressed through Performance.gov. With performance goals and measures for the website, it would also be possible for the developers of Performance.gov to identify the gap between current capabilities and what is needed to fulfill stated goals and to identify and set priorities for improvements. OMB staff agreed with these recommendations.

Agency Performance Information Is Not Always Useful for Congressional Decision Making

Congressional support has played a critical role in sustaining interest in management improvement initiatives over time. As we have previously reported, Congress has served as an institutional champion for many government-wide management reform initiatives over the years, such as the CFO Act and GPRA in the 1990s and more recently GPRAMA. Further, Congress has often played an important role in performance improvement and management reforms at individual agencies. Congress has also provided a consistent focus on oversight and has reinforced important policies.

As we have previously reported, having pertinent and reliable performance information available is necessary for Congress to adequately assess agencies’ progress in making performance and management improvements and ensure accountability for results. However, our work has found that the performance information that agencies provided to Congress was not always useful for congressional decision making because the information was not clear, directly relevant, or sufficiently detailed. As stated earlier, in order for performance information to be useful, it should meet the needs of different users— including Congress—in terms of completeness, accuracy, consistency, timeliness, validity, and ease of use.

GPRA required agencies to consult with Congress and obtain the views of interested stakeholders as a part of developing their strategic plans. However, according to the Senate committee report that accompanied the bill that ultimately became GPRAMA, agencies did not adequately consider the input of Congress in developing strategic plans, often because the agencies waited until strategic plans were substantially drafted and reviewed within the executive branch before consulting with Congress. In doing so, agencies limited the opportunities for Congress to provide input on their strategic plans and related goals, as well as the performance information that would be most useful for congressional oversight.

To help ensure agency performance information is useful for congressional decision making, GPRAMA strengthens the consultation requirement. The act requires agencies to consult at least once every two years with relevant appropriations, authorization and oversight committees, obtaining majority and minority views, when developing or updating strategic plans—which include APGs. Subsequently, agencies are to describe how congressional input was incorporated into those plans and goals. Similarly, OMB is required to consult with relevant committees with broad jurisdiction at least once every two years when developing or updating CAP goals, and describe how that input was incorporated into those goals. At the request of Congress, in June 2012, we developed a guide to assist Members of Congress and their staffs in ensuring the consultations required under GPRAMA are useful to the Congress. The guide outlines general approaches for successful consultations, including creating shared expectations and engaging the right people in the process at the right time. The guide also provides key questions that Members and congressional staff can ask as part of the consultation process to ensure that agency performance information reflects congressional priorities.

However, it is unclear if agencies incorporated congressional input on their updated strategic plans and APGs published in 2012, and therefore if this information will be useful for congressional decision making. In our recent review of APGs, we found that agencies reported engaging Congress during the development of their strategic plans and goals to varying degrees, and only 1 of the 24 agencies we reviewed explained how congressional input was incorporated into its APGs, as required by GPRAMA. We recommended in April 2013 that OMB ensure that agencies adhere to OMB’s guidance for website updates by providing a description of how input from congressional consultations was incorporated into each goal. OMB staff concurred with our recommendation.

In addition, our recent work indicated that the performance information provided on Performance.gov also may not be meeting congressional needs. We found that outreach from OMB to congressional staff was limited, as were opportunities for staff to provide input on the development of Performance.gov. According to OMB staff, they met several times with staff from the Senate Homeland Security and Governmental Affairs Committee, House Oversight and Government Reform Committee, and the Senate Budget Committee to discuss the development of Performance.gov, and used this outreach to identify several specific website modifications. Of the three congressional staff that we spoke to that said they had received briefings on the development of Performance.gov, however, only one told us she had been consulted on website input. In addition, since 2010, OMB staff has not held meetings on the development of Performance.gov with staff from other committees in the House or Senate that might use the website to inform their oversight of federal agencies. As previously mentioned, we also found that OMB has not articulated how various intended audiences, including Congress, can use the site to accomplish specific tasks, such as supporting coordination and decision making to advance shared goals.

At the request of the Congress, in December 2011 and June 2012, we highlighted several instances in which Congress has used agency performance information in various oversight and legislative activities, including (1) identifying issues that the federal government should address; (2) measuring the federal government’s progress toward addressing those issues; and (3) identifying better strategies to address the issues when necessary. For example, to help promote the use of e- filing of tax returns with the IRS, Congress used performance information to set clear expectations for agency performance, support oversight activities, and inform the development of additional legislation to help IRS achieve its goals. For further information, see the textbox.

Congressional Use of Performance Information to Promote E-filing of Tax Returns Congress sought to promote the use of e-filing, which allows taxpayers to receive refunds faster, is less prone to errors, and provides IRS significant cost savings. Congress took the following actions to increase the use of e-filing:  Setting Expectations: As part of the Internal Revenue Service Restructuring and Reform Act of 1998, Congress established a performance goal of having 80 percent of individual tax returns e-filed by 2007.  Oversight: Congress monitored IRS’s progress in meeting the established goal for e- filings; held 22 hearings related to IRS filing seasons and e-filings; and requested annual GAO reports to Congress on filing season performance, including e-filing.  Additional Legislation: Congress saw the need for further actions to help IRS achieve the goal, and subsequently passed legislation to require tax return preparers who file more than 10 returns per year to do so electronically. Although IRS did not meet the 80 percent e-filing target by 2007 (58 percent were e-filed that year), increased use of e-filing has substantially reduced IRS’s cost to process returns. IRS subsequently met this goal for individual tax returns as of the 2012 tax filing season, with 82 percent of individual returns e-filed.

IRS has yet to reach the 80 percent e-file goal for some types of returns other than individual income tax returns. See GAO, 2012 Tax Filing: IRS Faces Challenges Providing Service to Taxpayers and Could Collect Balances Due More Effectively, GAO-13-156 (Washington, D.C.: Dec. 18, 2012).

Conclusions

Moving forward, the federal government will need to make tough choices in setting priorities as well as reforming programs and management practices to address the pressing and complex economic, social, security, sustainability, and other issues the nation confronts. GPRAMA provides a number of tools that could help address these challenges. Since enactment in 2011, the executive branch has taken a number of important steps to implement key provisions of the act, by developing interim CAP goals and APGs, conducting quarterly reviews, assigning key performance management roles and responsibilities, and communicating results more frequently and transparently through Performance.gov.

However, the executive branch needs to do more to fully implement and leverage the act’s provisions to address these challenges. Our recent work reviewing federal performance issues and implementation of the act has pointed to several areas where improvements are needed and, accordingly, we recommended a number of actions. In addition, examples from our past work along with the most recent results from our survey of federal managers show that the executive branch has made little progress addressing long-standing governance challenges related to improving coordination and collaboration to address crosscutting issues, using performance information to drive decision making, measuring the performance of certain types of federal programs, and engaging Congress in a meaningful way in agency performance management efforts to ensure the resulting information is useful for congressional decision making.

Of particular concern, OMB has yet to develop a framework for reviewing the performance of tax expenditures, which represented approximately $1 trillion in forgone revenue in fiscal year 2012. In some areas, forgone revenue due to tax expenditures is nearly equal to or greater than spending for federal outlay programs. Since 1994 we have recommended OMB take this action, and the act puts into place explicit requirements for the CAP goals that OMB identify related tax expenditures and measure their contributions to broader federal outcomes. While early implementation of CAP goals showed some promise, with tax expenditures being identified as contributing to 5 of the 14 goals, many of those tax expenditures were subsequently removed. For example, our work shows that eight tax expenditures, representing about $2.4 billion in forgone revenue, should be listed as contributing to the energy efficiency CAP goal. The few tax expenditures that continue to be listed as contributors to a CAP goal only represent about $2.7 billion in forgone revenue—approximately 0.3 percent of the total estimate of forgone revenue from tax expenditures. While OMB staff told us the removal of these tax expenditures was an oversight and that they will be added as contributors in the near future, it raises concerns as to whether OMB previously ensured all relevant tax expenditures were identified as contributors to the 14 CAP goals when they were published in February 2012. Tax expenditures represent a substantial federal commitment to a wide range of mission areas, but do not receive the same scrutiny as spending programs. Therefore, it is possible that additional tax expenditures should have been identified and included as contributors to one or more of the other 9 CAP goals. Moreover, for the 2 CAP goals where tax expenditures were mistakenly removed, it is unclear if OMB and the goal leaders assessed the contributions of those tax expenditures toward the CAP goal efforts, since they were not listed in the December 2012 and March 2013 updates. Without information about which tax expenditures support these goals and measures of their performance, Congress and other decision makers will not have the needed information to assess overall federal contributions towards desired results and the costs and relative effectiveness associated with those contributions.

OMB took another promising action in 2012 by directing agencies to identify tax expenditures among the various federal programs and activities that contribute to their APGs—above and beyond what the act requires for all performance goals, which include APGs. However, the 103 APGs developed for 2012 to 2013 at 24 agencies represent only a small subset of all performance goals across the government. In addition, our review of the APGs for 2012 to 2013 found that only one agency, for one of its APGs, identified two relevant tax expenditures. OMB and agencies are missing important opportunities to more broadly identify how tax expenditures contribute to each agency’s overall performance.

In addition to measuring the contributions of tax expenditures to their goals, our work has found that agencies have experienced common issues in measuring the performance of various other types of programs and have not made consistent progress in addressing them in the last 20 years. As such, a comprehensive and concerted effort to address these long-standing difficulties needs to be taken. With responsibilities to resolve crosscutting performance issues and facilitate the exchange of proven practices, the PIC should lead such an assessment. The PIC has not yet addressed this issue in a systematic way, and without a comprehensive examination, it will be difficult for the PIC and agencies to fully understand these measurement issues and develop a crosscutting strategy to address them. That would likely result in agencies continuing to experience difficulties in measuring program performance in the future. The PIC’s upcoming strategic planning effort provides a venue for developing an approach for tackling this issue by putting in place the necessary plans and accountability. The PIC’s strategy should detail specific actors and actions to be made within set time frames to ensure that these persistent measurement challenges are adequately addressed.

Recommendations for Executive Action

To improve implementation of GPRAMA and help address pressing governance issues, we make the following four recommendations.

To help ensure that the contributions made by tax expenditures toward the achievement of agency goals and broader federal outcomes are properly recognized, we recommend that the Director of OMB take the following three actions:

Revise relevant OMB guidance to direct agencies to identify relevant tax expenditures among the list of federal contributors for each appropriate agency goal.

Review whether all relevant tax expenditures that contribute to a CAP goal have been identified, and as necessary, include any additional tax expenditures in the list of federal contributors for each goal.

Assess the contributions relevant tax expenditures are making toward the achievement of each CAP goal.

Given the common, long-standing difficulties agencies continue to face in measuring the performance of various types of federal programs and activities—contracts, direct services, grants, regulations, research and development, and tax expenditures—we also recommend the Director of OMB work with the PIC to develop a detailed approach to examine these difficulties across agencies, including identifying and sharing any promising practices from agencies that have overcome difficulties in measuring the performance of these program types. This approach should include goals, planned actions, and deliverables along with specific time frames for their completion, as well as the identification of the parties responsible for each action and deliverable.

Agency Comments

We provided a draft of this report for review and comment to the Director of OMB. Via e-mail, staff from OMB’s Office of Performance and Personnel Management agreed with the recommendations in this report. The staff also provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to the Director of OMB as well as interested congressional committees and other interested parties. This report will also be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-6806, or mihmj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of our report. Key contributors to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The GPRA Modernization Act of 2010 (GPRAMA) lays out a schedule for gradual implementation of its provisions during a period of interim implementation—from its enactment in January 2011 to February 2014 when a new planning and reporting cycle begins. GPRAMA also includes provisions requiring us to review implementation of the act at several critical junctures and provide recommendations for improvements to its implementation. This report is the final in a series responding to the mandate to assess initial implementation of the act by June 2013, and pulls together findings from our recent work related to the act, the results of our periodic survey of federal managers, and our related recent work on federal performance and coordination issues.

Our specific objectives for this report were to assess the executive branch’s (1) progress in implementing the act and (2) effectiveness in using tools provided by the act to address challenges the federal government faces. To address both objectives, we reviewed GPRAMA, related congressional documents and Office of Management and Budget (OMB) guidance, and our past and recent work related to managing for results and the act. We also interviewed OMB staff.

In addition, to further address the second objective, we administered a web-based questionnaire on organizational performance and management issues to a stratified random sample of 4,391 persons from a population of approximately 148,300 mid-level and upper-level civilian managers and supervisors working in the 24 executive branch agencies covered by the Chief Financial Officers (CFO) Act of 1990, as amended. The survey results provided information about the extent to which key performance management practices are in place to help address challenges. The sample was drawn from the Office of Personnel Management’s (OPM) Central Personnel Data File (CPDF) as of March 2012, using file designators indicating performance of managerial and supervisory functions. In reporting the questionnaire data, when we use the term “government-wide” and the phrases “across the government” or “overall” we are referring to these 24 CFO Act executive branch agencies, and when we use the terms “federal managers” and “managers” we are referring to both managers and supervisors.

The questionnaire was designed to obtain the observations and perceptions of respondents on various aspects of results-oriented management topics such as the presence and use of performance measures, hindrances to measuring performance and using performance information, agency climate, and program evaluation use. In addition, to address implementation of GPRAMA, the questionnaire included a section requesting respondents’ views on various provisions of GPRAMA, such as cross-agency priority goals, agency priority goals, and quarterly performance reviews. For the agency priority goal questions, we directed the federal managers from the Nuclear Regulatory Commission to not answer these questions since OMB did not require the agency to develop agency priority goals for 2012 to 2013.

This survey is comparable to surveys we have conducted four times previously at the 24 CFO Act agencies—1997, 2000, 2003, and 2007. The 1997 survey was conducted as part of the work we did in response to a GPRA requirement that we report on implementation of the act. The 2000, 2003, and 2007 surveys were designed to update the results from each of the previous surveys. The 2007 survey also included a section requesting the respondent’s view on OMB’s Program Assessment Rating Tool and the priority that should be placed on various potential improvements to it. The 2000 and 2007 surveys, unlike the other two surveys, were designed to support analysis of the data at the department and agency level as well as government-wide. For this report, we focus on comparing the 2013 survey results with those from the 1997 baseline survey; and with the results of the 2007 survey, which is the most recent survey conducted before GPRAMA was enacted in 2011. We noted the results from the other two surveys—2000 and 2003—when statistically significant trends compared to 2013 occurred.

Similar to the four previous surveys, the sample was stratified by agency and by whether the manager or supervisor was a member of the Senior Executive Service (SES) or non-SES. The management levels covered general schedule (GS) or equivalent schedules at levels comparable to GS-13 through GS-15 and career SES or equivalent. Similar to our 2000, 2003, and 2007 surveys, we also incorporated managers or supervisors in other pay plans at levels generally equivalent to the GS-13 through career SES levels into the population and the selected sample to ensure at least a 90 percent coverage of all mid- to upper-level managers and supervisors at the departments and agencies we surveyed.

Most of the items on the questionnaire were closed-ended, meaning that depending on the particular item, respondents could choose one or more response categories or rate the strength of their perception on a 5-point extent scale ranging from “to no extent” at the low end of the scale to “to a very great extent” at the high end. On most items, respondents also had an option of choosing the response category “no basis to judge/not applicable.” A few items had yes, no, or do not know options for respondents.

Many of the items on the questionnaire were asked in our earlier surveys; the sections of the questionnaire asking about GPRAMA, program evaluations, and availability of performance information are new. For these new questions, we conducted pretests with federal managers in several of the 24 CFO Act agencies. For the 2013 survey, based on feedback we obtained from our pretests with managers, we moved the placement of question 8 in the survey to accommodate the insertion of a new question. In previous surveys, only those respondents who answered yes to question 5—that they had performance measures available for their programs—were asked to answer question 8—a series of items about the extent to which they used information obtained from performance measurement when participating in certain activities. Respondents answering “no” or “do not know” to question 5 could skip past the question 8 items. For the 2013 survey, all respondents were asked to answer question 8 given the new question added. To maintain the consistency and comparability with how we have previously analyzed and reported question 8 results, we applied the skip pattern used in prior surveys to question 8 by removing those individuals who did not answer yes to question 5 (and in the past would have been directed to skip out of answering the question). However, in the e-supplement we report the results as the federal managers answered the questionnaire, regardless of how they had answered question 5.

To administer the survey, an e-mail was sent to managers in the sample that notified them of the survey’s availability on the GAO website and included instructions on how to access and complete the survey. With the exception of the managers at the Department of Justice (DOJ), which is discussed below, managers in the sample who did not respond to the initial notice were sent up to four subsequent e-mail reminders and follow- up phone calls asking them to participate in the survey.

In our prior surveys, we worked with OPM to obtain the names of the managers and supervisors in our sample as selected through the CPDF.

However, since our last survey in 2007, some agencies had requested from OPM that the names of individuals within selected subcomponents be withheld from the CPDF. We worked with officials at these agencies to attempt to gain access to these individuals to maintain continuity of the population of managers surveyed from previous years. Due to DOJ’s national security concerns about providing identifying information (e.g., names, e-mail addresses, phone numbers) of federal agents to us, we administered the current survey to all DOJ managers in our sample through a DOJ official. To identify the sample of managers whose names were withheld from the CPDF, we provided DOJ with the last four digits of Social Security numbers, the subcomponent, duty location, and pay grade information. To ensure that DOJ managers received the same survey administration process as the rest of the managers in our sample to the extent possible, we provided DOJ with copies of the notification, activation (including the web link to our survey), and follow-up e-mails that managers at other agencies received from us. DOJ administered the survey to its managers and conducted follow-up with the nonrespondents. We administered the survey to all 24 agencies from November 2012 through February 2013.

To help determine the reliability and accuracy of the CPDF data elements used to draw our sample of federal managers, we checked the data for reasonableness and the presence of any obvious or potential errors in accuracy and completeness. For example, we identified cases where the managers’ names were withheld and contacted OPM to determine the reason and extent of this issue. We also checked the names of the managers in our selected sample provided from OPM with the applicable agency contacts to verify these managers were still employed with the agency in the role. We noted discrepancies when they occurred and excluded them from our population of interest, as applicable. We also reviewed our past analyses of the reliability of the CPDF data. On the basis of these procedures, we believe the data we used from the CPDF are sufficiently reliable for the purpose of this report.

Of the 4,391 managers selected for this survey, we found that 266 of the sampled managers had retired, separated, died, or otherwise left the agency or had some other reason that excluded them from the population of interest. We received usable questionnaires from 2,762 sample respondents, or about 69 percent of the remaining eligible sample. In addition, there were 29 persons that we were unable to locate and therefore unable to request that they participate in the survey. The response rate across the 24 agencies ranged from 57 percent to 88 percent.

The overall survey results are generalizable to the population of managers as described above at each of the 24 agencies and government-wide. The responses of each eligible sample member who provided a usable questionnaire were weighted in the analyses to account statistically for all members of the population. All results are subject to some uncertainty or sampling error as well as nonsampling error. Because we followed a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. Since each sample could have provided different estimates, we express our confidence in the precision of our particular sample’s results as a 95 percent confidence interval. This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn.

The percentage estimates presented in this report based on our sample for the 2013 survey have 95 percent confidence intervals within plus or minus 5 percentage points of the estimate itself, unless otherwise noted. An online e-supplement shows the questions asked on the survey along with the percentage estimates and associated 95 percent confidence intervals for each item for each agency and government-wide.

Because a complex survey design was used in the current survey as well as the four previous surveys, and different types of statistical analyses are being done, the magnitude of sampling error will vary across the particular surveys, groups, or items being compared due to differences in the underlying sample sizes, usable sample respondents, and associated variances of estimates. For example, the 2000 and 2007 surveys were designed to produce agency-level estimates and had effective sample sizes of 2,510 and 2,943, respectively. However, the 1997 and 2003 surveys were designed to obtain government-wide estimates only, and their sample sizes were 905 and 503, respectively. Consequently, in some instances, a difference of a certain magnitude may be statistically significant. In other instances, depending on the nature of the comparison being made, a difference of equal or even greater magnitude may not achieve statistical significance. We note in this report when we are 95 percent confident that the difference is statistically significant. Also, as part of any interpretation of observed shifts in individual agency responses between the 2013 and the 2000 surveys, it should be kept in mind that components of some agencies and all of the Federal Emergency Management Agency became part of the Department of Homeland Security.

In addition to sampling errors, the practical difficulties of conducting any survey may also introduce other types of errors, commonly referred to as nonsampling errors. For example, difficulties in how a particular question is interpreted, in the sources of information available to respondents, or in how the data were entered into a database or were analyzed can introduce unwanted variability into the survey results. With this survey, we took a number of steps to minimize these nonsampling errors. For example, our staff with subject matter expertise designed the questionnaire in collaboration with our survey specialists. As noted earlier, the new questions added to the survey were pretested to ensure they were relevant and clearly stated. When the data were analyzed, a second independent GAO analyst independently verified the analysis programs to ensure the accuracy of the code and the appropriateness of the methods used for the computer-generated analysis. Since this was a web-based survey, respondents entered their answers directly into the electronic questionnaire, thereby eliminating the need to have the data keyed into a database, thus avoiding a source of data entry error.

We conducted this performance audit from August 2012 to June 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Status of Key Recommendations Related to GPRAMA

Status OMB staff agreed with our recommendations. clarify the ways that intended audiences could use the information on the Performance.gov website to accomplish specific tasks and specify the design changes that would be required to facilitate that use; seek to more systematically collect information on the needs of a broader audience, including through the use of customer satisfaction surveys and other approaches recommended by HowTo.gov; and seek to ensure that all performance, search, and customer satisfaction metrics, consistent with leading practices outlined in HowTo.gov, are tracked for the website, and, where appropriate, create goals for those metrics to help identify and prioritize potential improvements to Performance.gov.

OMB staff agreed with our recommendations. provide a definition of what constitutes “data of significant value;” direct agencies to develop and publish on Performance.gov interim quarterly performance targets for their agency priority goal performance measures when the above definition applies; direct agencies to provide and publish on Performance.gov completion dates, both in the near- term and longer-term for their milestones; and direct agencies to describe in their performance plans how the agency’s performance goals— including priority goals—contribute to any of the cross-agency priority goals.

When such revisions are made, the Director of OMB should work with the PIC to test and implement these provisions.

Title and GAO product number

Status OMB staff agreed with our recommendations. complete information about the organizations, program activities, regulations, policies, tax expenditures, and other activities—both within and external to the agency—that contribute to each goal; and a description of how input from congressional consultations was incorporated into each goal.

To improve performance management staff capacity to support performance management in federal agencies, the Director of OPM should, in coordination with the PIC and the Chief Learning Officer Council, work with agencies to: identify competency areas needing improvement within agencies; identify agency training that focuses on needed performance management competencies; and share information about available agency training on competency areas needing improvement.

OPM agreed with our recommendations, and explained that it will work with agencies, and in particular with PIOs, to assess the competencies of the performance management workforce. OPM also stated that it will support the use of the PIC’s performance learning website to facilitate the identification and sharing of training related to competencies in need of improvement.

OMB staff agreed with our recommendations. conduct formal feedback on the performance of the PIC from member agencies, on an ongoing basis; and update its strategic plan and review the PIC’s goals, measures, and strategies for achieving performance, and revise them if appropriate.

To better leverage agency quarterly performance reviews as a mechanism to manage performance toward agency priority and other agency-level performance goals, the Director of OMB should—working with the PIC and other relevant groups—identify and share promising practices to help agencies extend their quarterly performance reviews to include, as relevant, representatives from outside organizations that contribute to achieving their agency performance goals.

OMB staff agreed with our recommendation.

Summary of related recommendations The Director of OMB, in considering additional programs with the potential to contribute to the crosscutting goals, should review the additional departments, agencies, and programs that we have identified, and consider including them in the federal government performance plan, as appropriate.

Status OMB staff agreed with our recommendation. In December 2012 and March 2013, OMB updated information on Performance.gov on the CAP goals. OMB included some of the agencies and programs we identified for select goals, but in other instances eliminated key contributors that were previously listed.

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the above contact, Elizabeth Curda (Assistant Director) and Benjamin T. Licht supervised this review and the development of the resulting report. Tom Beall, Peter Beck, Mallory Barg Bulman, Virginia Chanley, Laura Miller Craig, Sara Daleski, Karin Fangman, Stuart Kaufman, Don Kiggins, Judith Kordahl, Jill Lacey, Janice Latimer, Adam Miles, Kathleen Padulchick, Mark Ramage, Daniel Ramsey, Marylynn Sergent, Megan Taylor, Sarah Veale, Kate Hudson Walker, and Dan Webb made significant contributions to this report. Pawnee Davis, Shannon Finnegan, Quindi Franco, Ellen Grady, Robert Gebhart, Tom James, Donna Miller, Michael O’Neill, Robert Robinson, and Stephanie Shipman also made key contributions.