Results-Oriented Government: GPRA Has Established a Solid Foundation for Achieving Greater Results
Released: 2004-03-10T00:00:00

Letter

Introduction

From defending the homeland against terrorists, to preventing the spread of infectious diseases, to providing a reliable stream of social security income to retirees and supporting the transition from welfare to work, the federal government provides funding and services to the American public that can affect their lives in critical ways every day. However, the federal government is in a period of profound transition and faces an array of challenges and opportunities to enhance performance, ensure accountability, and position the nation for the future. A number of overarching trends, such as diffuse security threats and homeland security needs, increasing global interdependency, the shift to knowledge-based economies, and the looming fiscal challenges facing our nation, drive the need to reconsider the proper role for the federal government in the 21st century, how the government should do business (including how it should be structured), and in some instances, who should do the government’s business.

Without effective short- and long-term planning, which takes into account the changing environment and needs of the American public and the challenges they face and establishes goals to be achieved, federal agencies risk delivering programs and services that may or may not meet society’s most critical needs. At a cost to taxpayers of over $2 trillion annually, the federal government should be able to demonstrate to the American public that it can anticipate emerging issues, develop sound strategies and plans to address them, and be accountable for the results that have been achieved.

Concerned that the federal government was more focused on program activities and processes than the results to be achieved, Congress passed the Government Performance and Results Act of 1993 (GPRA). The act required federal agencies to develop strategic plans with long-term strategic goals, annual goals linked to achieving the long-term goals, and annual reports on the results achieved. Now that GPRA has been in effect for 10 years, you asked us to assess the effectiveness of GPRA in creating a focus on results in the federal government. Specifically, this report discusses (1) the effect of GPRA over the last 10 years in creating a governmentwide focus on results and the government’s ability to deliver results to the American public, including an assessment of the changes in the overall quality of agencies’ strategic plans, annual performance plans, and annual performance reports; (2) the challenges agencies face in measuring performance and using performance information in management decisions; and (3) how the federal government can continue to shift toward a more results-oriented focus.

Impact of Emerging Trends and Fiscal Challenges

With the 21st century challenges we are facing, it is more vital than ever to maximize the performance of federal agencies in achieving their long-term goals. The federal government must address and adapt to major trends in our country and around the world. At the same time, our nation faces serious long-term fiscal challenges. Increased pressure also comes from world events: both from the recognition that we cannot consider ourselves “safe” between two oceans—which has increased demands for spending on homeland security—and from the United States (U.S.) role in combating terrorism in an increasingly interdependent world. To be able to assess federal agency performance and hold agency managers accountable for achieving their long-term goals, we need to know what the level of performance is. GPRA planning and reporting requirements can provide this essential information.

Our country’s transition into the 21st century is characterized by a number of key trends, including the national and global response to terrorism and other threats to our personal and national security; the increasing interdependence of enterprises, economies, markets, civil societies, and national governments, commonly referred to as globalization; the shift to market-oriented, knowledge-based economies; an aging and more diverse U.S. population; rapid advances in science and technology and the opportunities and challenges created by these changes; challenges and opportunities to maintain and improve the quality of life for the nation, communities, families, and individuals; and the changing and increasingly diverse nature of governance structures and tools.

As the nation and government policymakers grapple with the challenges presented by these evolving trends, they do so in the context of rapidly building fiscal pressures. GAO’s long-range budget simulations show that this nation faces a large and growing structural deficit due primarily to known demographic trends and rising health care costs. The fiscal pressures created by the retirement of the baby boom generation and rising health costs threaten to overwhelm the nation’s fiscal future. As figure 1 shows, by 2040, absent reform or other major tax or spending policy changes, projected federal revenues will likely be insufficient to pay more than interest on publicly held debt. Further, our recent shift from surpluses to deficits means the nation is moving into the future in a more constrained fiscal position.

The United States has had a long-range budget deficit problem for a number of years, even during recent years in which we had significant annual budget surpluses. Unfortunately, the days of surpluses are gone, and our current and projected budget situation has worsened significantly. The bottom line is that our projected budget deficits are not manageable without significant changes in “status quo” programs, policies, processes, and operations.

GPRA Background

GPRA is the centerpiece of a statutory framework that Congress put in place during the 1990s to help resolve the long-standing management problems that have undermined the federal government’s efficiency and effectiveness and to provide greater accountability for results. In addition to GPRA, the framework comprises the Chief Financial Officers Act of 1990, as amended by the Government Management Reform Act of 1994, and information technology reform legislation, including the Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of 1996. Together, these laws provide a powerful framework for developing and integrating information about agencies’ missions and strategic priorities, the results- oriented performance goals that flow from those priorities, performance data to show the level of achievement of those goals, and the relationship of reliable and audited financial information and information technology investments to the achievement of those goals.

GPRA was intended to address several broad purposes, including strengthening the confidence of the American people in their government; improving federal program effectiveness, accountability, and service delivery; and enhancing congressional decision making by providing more objective information on program performance.

2. The agency must develop annual performance plans covering each program activity set forth in the agencies’ budgets* Building on the decisions made as part of the strategic planning process, GPRA requires executive agencies to develop annual performance plans covering each program activity set forth in the agencies’ budgets. Annual performance plans, covering the upcoming fiscal year, are to be submitted to Congress after the President’s Budget is submitted, which generally occurs in February. Each plan is to contain an agency’s annual performance goals and associated measures, which the agency is to use in order to gauge its progress toward accomplishing its strategic goals. OMB is to use the agencies’ performance plans to develop an overall federal government performance plan that is to be submitted with the President’s Budget. The performance plan for the federal government is to present to Congress a single cohesive picture of the federal government’s annual performance goals for a given fiscal year. 3. The agency must prepare annual reports on program performance for the previous fiscal year, to be issued by March 31 each year GPRA requires executive agencies to prepare annual reports on program performance for the previous fiscal year, to be issued by March 31 each year. In each report, an agency is to compare its performance against its goals, summarize the findings of program evaluations completed during the year, and describe the actions needed to address any unmet goals. Recent OMB guidance states that executive agencies must combine their program performance report with their accountability report and transmit the combined report for fiscal year 2003 by January 30, 2004, and the combined report for fiscal year 2004 by November 15, 2004. *Program activity refers to the lists of projects and activities in the appendix portion of the Budget of the United States Government. Program activity structures are intended to provide a meaningful representation of the operations financed by a specific budget account.

The Office of Management and Budget (OMB) plays an important role in the management of the federal government’s performance, and specifically GPRA implementation. Part of OMB’s overall mission is to ensure that agency plans and reports are consistent with the President’s Budget and administration policies. OMB is responsible for receiving and reviewing agencies’ strategic plans, annual performance plans, and annual performance reports. To improve the quality and consistency of these documents, OMB issues annual guidance to agencies for their preparation, including guidelines on format, required elements, and submission deadlines. GPRA requires OMB to prepare the overall governmentwide performance plan, based on agencies’ annual performance plan submissions. OMB also played an important role in the pilot phase of GPRA implementation by designating agencies for pilot projects in performance measurement, managerial accountability and flexibility, and performance budgeting, and assessing the results of the pilots. Finally, GPRA provides OMB with authority to grant agencies waivers to certain administrative procedures and controls.

Recent OMB guidance requires agencies to submit “performance budgets” in lieu of annual performance plans for their budget submission to OMB and Congress. Performance budgets are to meet all the statutory requirements of GPRA for annual performance plans. In addition, agencies are to include all performance goals used in the assessment of program performance done under OMB’s Program Assessment Rating Tool (PART) process. Moreover, the guidance states that until all programs have been assessed by PART, the performance budget will also for a time include performance goals for agency programs that have not yet been assessed using PART. The expectation is that agencies are to substitute new or revised performance goals resulting from OMB’s review for goals it deemed unacceptable.

In crafting GPRA, Congress recognized that managerial accountability for results is linked to managers having sufficient flexibility, discretion, and authority to accomplish desired results. GPRA authorizes agencies to apply for managerial flexibility waivers in their annual performance plans beginning with fiscal year 1999. The authority of agencies to request waivers of administrative procedural requirements and controls is intended to provide federal managers with more flexibility to structure agency systems to better support program goals. The nonstatutory requirements that OMB can waive under GPRA generally involve the allocation and use of resources, such as restrictions on shifting funds among items within a budget account. Agencies must report in their annual performance reports on the use and effectiveness of any GPRA managerial flexibility waivers that they receive.

OMB was to designate at least five agencies from the first set of pilot projects to test managerial accountability and flexibility during fiscal years 1995 and 1996. We previously reported on the results of the pilot project to implement managerial flexibility waivers and found that the pilot did not work as intended. OMB did not designate any of the seven departments and one independent agency that submitted a total of 61 waiver proposals as GPRA managerial accountability and flexibility pilots. For about three- quarters of the waiver proposals, OMB or other central management agencies determined that the waivers were not allowable for statutory or other reasons or that the requirement for which the waivers were proposed no longer existed. For the remaining proposals, OMB or other central management agencies approved waivers or developed compromises by using authorities that were already available independent of GPRA.

Under GPRA, another set of pilot projects, which were scheduled for fiscal years 1998 and 1999, were to test performance budgeting—i.e., the presentation of the varying levels of performance that would result from different budget levels. We previously reported that OMB initially deferred these pilots—originally to be designated in fiscal years 1998 and 1999—to give federal agencies time to develop the capability of calculating the effects of marginal changes in cost or funding on performance. When the pilots began in August 1999, OMB designed them as case studies prepared by OMB staff to demonstrate how performance information could be used to compare alternatives and to develop funding recommendations for incorporation into the President’s fiscal year 2001 budget submission.

On January 18, 2001, OMB reported the results of five performance budgeting pilots that explored agencies’ capabilities of more formally assessing the effects of different funding levels on performance goals. OMB selected the pilots to reflect a cross section of federal functions and capabilities so that a representative range of measurement and reporting issues could be explored. In its report, OMB concluded that legislative changes were not needed. OMB reported that the pilots demonstrated that assuring further performance measurement improvements and steadily expanding the scope and quality of performance measures is paramount, and that the existing statute provides sufficient latitude for such improvement.

Overall, OMB concluded that the pilots raised several key challenges about performance budgeting at the federal level including, for example, the following: In many instances, measuring the effects of marginal, annual budget changes on performance is not precise or meaningful.

While continuing to change from an almost total reliance on output measures to outcome measures, it will be much more difficult to associate specific resource levels with those outcomes, particularly over short periods of time.

Establishing clear linkages between funding and outcomes will vary by the nature of the program and the number of external factors.

Delays in the availability of performance data, sometimes caused by agencies’ reliance on nonfederal program partners for data collection, will continue to present synchronization problems during budget formulation.

Scope and Methodology

To meet the three objectives stated earlier, we reviewed our extensive prior work on GPRA best practices and implementation and collected governmentwide data to assess the government’s overall focus on results. We conducted a random, stratified, governmentwide survey of federal managers comparable to surveys we conducted in 1997 and 2000. We also held eight in-depth focus groups—seven comprised of federal managers from 23 federal agencies and one with GPRA experts. We also interviewed top appointed officials from the current and previous administrations. Finally, we judgmentally selected a sample of six agencies to review for changes in the quality of their strategic plans, performance plans, and performance reports since their initial efforts. The agencies we selected were the Departments of Education (Education), Energy (DOE), Housing and Urban Development (HUD), and Transportation (DOT) and the Small Business (SBA) and Social Security Administrations (SSA). In making this selection, we chose agencies that collectively represented the full range of characteristics in the following four areas: (1) agency size (small, medium, large); (2) primary program types (direct service, research, regulatory, transfer payments, and contracts or grants); (3) quality of fiscal year 2000 performance plan based on our previous review (low, medium, high); and (4) type of agency (cabinet department and independent agency). Appendix I contains a more detailed discussion of our scope and methodology.

We performed our work in Washington, D.C., from January through November 2003 in accordance with generally accepted government auditing standards. Major contributors to this report are listed in appendix XII.

GPRA Established the Foundation for a More Results-Oriented Federal Government

Among the purposes of GPRA cited by Congress was to improve federal program effectiveness and service delivery by promoting a new focus on results, service quality, and customer satisfaction by setting program goals, measuring performance against goals, and reporting publicly on progress. Furthermore, GPRA was to improve congressional decision making by providing better information on achieving objectives, and on the relative effectiveness and efficiency of federal programs and spending. Ten years after enactment, GPRA’s requirements have laid a foundation of results- oriented agency planning, measurement, and reporting that have begun to address these purposes. Focus group participants, high-level political appointees, and OMB officials we interviewed cited positive effects of GPRA that they generally attributed to GPRA’s statutory requirements for planning and reporting. Our survey results indicate that since GPRA went into effect governmentwide in 1997, federal managers reported having significantly more of the types of performance measures called for by GPRA—particularly outcome-oriented performance measures. GPRA has also begun to facilitate the linking of resources to results, although much remains to be done in this area.

GPRA Statutory Requirements Laid a Foundation for Agencywide Results- Oriented Management

Prior to enactment of GPRA, our 1992 review of the collection and use of performance data by federal agencies revealed that, although many agencies collected performance information at the program level, few agencies had results-oriented performance information to manage or make strategic policy decisions for the agency as a whole. Federal agencies surveyed indicated that many had a single, long-term plan that contained goals, standards, or objectives for the entire agency or program. Many of these agencies also reported they collected a wide variety of performance measures. However, in validating the survey responses with a sample of agencies, we found that measures were typically generated and used by program-level units within an agency and focused on measuring work activity levels and outputs or compliance with statutes. Little of this performance information was transparent to Congress, OMB, or the public and few of the agencies we visited used performance measures to manage toward long-term objectives. Few of the agencies surveyed had the infrastructure in place, such as a unified strategic plan with measurable goals, an office that collected performance measures, and regular consolidated reports, to tie plans and measures.

GPRA addressed these shortcomings by creating a comprehensive and consistent statutory foundation of required agencywide strategic plans, annual performance plans, and annual performance reports. In contrast to prior federal government efforts to measure performance, GPRA explicitly emphasized that, in addition to performance indicators that agencies may need to manage programs on a day-to-day basis, such as quantity, quality, timeliness, and cost, agencies also needed outcome-oriented goals and measures that assess the actual results, effects, or impact of a program or activity compared to its intended purpose.

Expert and agency focus group participants cited the creation of this statutory foundation as one of the key accomplishments of GPRA. Participants agreed that GPRA created a framework in statute for federal agencies to plan their activities in order to become more results oriented and provided a managerial tool for program accountability. Using this framework, agencies could develop and focus on strategies to carry out the programs they administer; set goals and identify performance indicators that will inform them whether or not they achieved the performance they expected; and determine what impact, if any, their programs have had on the American public. According to the experts in one of our focus groups, comparing federal agencies’ current mission statements contained in their strategic plans to what they were in the past demonstrates that the agencies have done some “soul searching” to get a better sense of what their role is (or should be) and how they can achieve it. Given that GPRA is in statute, participants indicated that the use of this planning framework is likely to be sustained within agencies.

One of the premises of GPRA is that both congressional and executive branch oversight of federal agency performance were seriously hampered by a lack of adequate results-oriented goals and performance information. As noted above, prior to the enactment of GPRA few agencies reported their performance information externally. OMB officials we interviewed as part of our current review suggested that OMB has been a key consumer of the performance information produced under GPRA and that it has provided a foundation for their efforts to oversee agency performance.

For example, during the development of the fiscal year 2004 budget, OMB used PART to review and rate 234 federal programs. We recently reported that one of PART’s major impacts was its ability to highlight OMB’s recommended changes in program management and design. PART reviews look at four elements—program purpose and design, strategic planning, program management, and program results/accountability—and rate the program on how well each of these elements is executed. However, without the foundation of missions, goals, strategies, performance measures, and performance information generated under GPRA, such oversight would be difficult to carry out.

Participants in most of our focus groups also agreed that GPRA has been a driving force behind many cultural changes that have occurred within federal agencies. Highlighting the focus on results, participants stated that GPRA had stimulated a problem-solving approach within federal agencies and encouraged agency managers to think creatively when developing performance indicators for their programs. GPRA has also changed the dialogue within federal agencies; front-line managers and staff at lower levels of the organization now discuss budget issues in connection with performance. Similarly, experts noted that information about performance management and resource investments are more frequently communicated between agency officials and Congress than in the past. Within agencies, GPRA documents can provide a context of missions, goals, and strategies that political appointees can use to articulate agencies’ priorities.

Views on GPRA’s Effect on the Federal Government’s Ability to Deliver Results to the American Public Were Mixed

A key purpose of GPRA was “to improve the confidence of the American people in the capability of the Federal Government, by systematically holding Federal agencies accountable for achieving program results.” When asked about the direct effects of GPRA on the public in our 2003 survey, an estimated 23 percent of federal managers agreed to a moderate or greater extent that GPRA improved their agency’s ability to deliver results to the American public; a larger percentage—38 percent—chose a “no basis to judge/not applicable” category.

When a similar question was posed in our focus groups with experts and federal managers, participants’ views were generally mixed. Some federal managers in our focus groups agreed that GPRA has had a positive effect on raising awareness on many performance issues, and that in and of itself is a way of delivering results. The information gathered and reported for GPRA allows agencies to make better-informed decisions, which improves their ability to achieve results. Other participants stated that while certain aspects of GPRA-related work have been positive, agencies’ ability to deliver results and public awareness of their activities cannot always be exclusively attributed to GPRA. For example, some participants stated that many agencies rely on grant recipients to carry out their work, and delivering results to the American public depends, to a large extent, on the diligence of these organizations to implement their programs; such results would not change dramatically if GPRA were no longer a requirement.

A number of the political appointees we interviewed cited examples of outcomes they believe would not have occurred without the structure of GPRA. For example, a former deputy secretary of the Department of Veterans Affairs (VA) stated that “the Results Act brought about a fundamental rethinking of how we managed our programs and processes. . . . We developed a strategic plan that was veteran- focused. . . . We made every effort to define program successes from the veteran’s perspective.” A former Chief Financial Officer (CFO) cited Customs Service goals to reduce the quantity of illegal drugs flowing into the United States and the Food and Drug Administration’s focus on speeding up the approval of new drugs as examples of outcomes that can make a big difference in people’s lives.

Another major accomplishment of GPRA cited by our focus group participants is that GPRA improved the transparency of government results to the American public. As noted above, prior to GPRA, few agencies reported performance results outside of their agencies. Focus group participants indicated a key accomplishment of GPRA was its value as a communication tool by increasing the transparency to the public of what their agencies did in terms the public could understand. For example, information on agencies’ strategic plans, performance goals, measures, and results are easily obtainable from agency Web sites. One focus group participant commented that GPRA helps bureaucrats explain to nonbureaucrats what the federal government does in terms they can better understand. Other comments indicated that because of GPRA agencies could now tell Congress and the American public what they are getting for their money.

More Managers Reported Having Performance Measures

A fundamental element in an organization’s efforts to manage for results is its ability to set meaningful goals for performance and to measure performance against those goals. From our 2003 survey we estimate that 89 percent of federal managers overall said there were performance measures for the programs they were involved with. This is a statistically significantly higher percentage than the 76 percent of managers who answered yes to this item on our 1997 survey. (See fig. 2.)

Moreover, when we asked managers who said they had performance measures which of five types of measures they had to a great or very great extent, they reported increases in all five types of measures between 1997 and 2003, all of which were statistically significant. (See fig. 3.) Notably, managers indicated the existence of outcome measures, defined as “performance measures that demonstrate to someone outside the organization whether or not intended results are being achieved,” grew from a low of 32 percent in 1997 to the current estimate of 55 percent, a level that is on par with output measures for the first time since we began our survey.

Similarly, focus group participants commented on certain cultural changes that had taken place within their agencies since the passage of GPRA in which the “vocabulary” of performance planning and measurement—e.g., a greater focus on performance management; orientation toward outcomes over inputs and outputs; and an increased focus on program evaluation— had become more pervasive. This perception is partly born out by our survey results. Since 1997 those reporting a moderate to extensive knowledge of GPRA and its requirements shifted significantly from 26 percent to 41 percent in 2003, while those reporting no knowledge of GPRA declined significantly from 27 percent to 20 percent. (See fig. 4.)

Consistent with our survey results indicating increases in results-oriented performance measures and increasing GPRA knowledge, we also observed a significant decline in the percentage of federal managers who agreed that certain factors hindered measuring performance or using the performance information. For example, as shown in figure 5, of those who expressed an opinion, the percentage of managers who noted that determining meaningful measures was a hindrance to a great or very great extent was down significantly from 47 percent in 1997 to 36 percent in 2003. Likewise, the percentage that agreed to a great or very great extent that different parties are using different definitions to measure performance was a hindrance also declined significantly from 49 percent in 1997 to 36 percent in 2003.

Finally, our survey data suggested that more federal managers, especially at the Senior Executive Service (SES) level, believed that OMB was paying attention to their agencies’ efforts under GPRA. Moreover, there was no corresponding increase in their concern that OMB would micromanage the programs in their agencies. In our survey, we asked respondents to assess the extent to which OMB pays attention to their agencies’ efforts under GPRA. As seen in figure 6, in 2003, the percentage of respondents who responded “Great” or “Very Great” to this question (31 percent) was significantly higher than in 2000 (22 percent). Of those, SES respondents showed an even more dramatic increase, from 33 to 51 percent. We also asked respondents the extent to which their concern that OMB would micromanage programs in their agencies was a hindrance to measuring performance or using performance information. The percentage among those expressing an opinion that it was a hindrance to a great or very great extent was low—around 24 percent in 2003—with no significant difference between 2000 and 2003.

GPRA Has Begun to Establish a Link between Resources and Results

Among its major purposes, GPRA aims for a closer and clearer linkage between requested resources and expected results. The general concept of linking performance information with budget requests is commonly known as performance budgeting. Budgeting is and will remain an exercise in political choice, in which performance can be one, but not necessarily the only, factor underlying decisions. However, efforts to infuse performance information into resource allocation decisions can more explicitly inform budget discussions and focus them—both in Congress and in agencies—on expected results, rather than on inputs.

GPRA established a basic foundation for performance budgeting by requiring that an agency’s annual performance plan cover each program activity in the President’s budget request for that agency. GPRA does not specify any level of detail or required components needed to achieve this coverage. Further, GPRA recognizes that agencies’ program activity structures are often inconsistent across budget accounts and thus gives agencies the flexibility to consolidate, aggregate, or disaggregate program activities, so long as no major function or operation of the agency is omitted or minimized. In addition, OMB guidance has traditionally required agencies to display, by budget program activity, the funding level being applied to achieve performance goals. OMB’s guidance on developing fiscal year 2005 performance budgets also encourages a greater link between performance and funding levels, however, it places greater emphasis on linking agencies’ long-term and annual performance goals to individual programs. At a minimum, agencies are to align resources at the program level, but they are encouraged to align resources at the performance goal level. Resources requested for each program are to be the amounts needed to achieve program performance goal targets.

Our 1998 assessment of fiscal year 1999 performance plans found that agencies generally covered the program activities in their budgets, but most plans did not identify how the funding for those program activities would be allocated to performance goals. However, our subsequent reviews of performance plans indicate that agencies have made progress in demonstrating how their performance goals and objectives relate to program activities in the budget.

Over the first 4 years of agency efforts to implement GPRA, we observed that agencies continued to tighten the required link between their performance plans and budget requests. Of the agencies we reviewed over this period, all but three met the basic requirement of GPRA to define a link between their performance plans and the program activities in their budget requests, and most of the agencies in our review had moved beyond this basic requirement to indicate some level of funding associated with expected performance described in the plan. Most importantly, more of the agencies we reviewed each year—almost 75 percent in fiscal year 2002 compared to 40 percent in fiscal year 1999—were able to show a direct link between expected performance and requested program activity funding levels—the first step in defining the performance consequences of budgetary decisions. However, we have also observed that the nature of these linkages varied considerably. Most of the agencies in our review of fiscal year 2002 performance plans associated funding requests with higher, more general levels of expected performance, rather than the more detailed “performance goals or sets of performance goals” suggested in OMB guidance.

Although not cited by our group of experts, participants at six of our seven focus groups with federal managers cited progress in this area as a key accomplishment of GPRA. However, the participants also commented that much remains to be done in this area. The comments ranged from the general—GPRA provides a framework for planning and budgeting, to the more specific—GPRA created a definition of programs and how they will help the agency achieve its goals/objectives and the amount of money that will be required to achieve said goals/objectives. One of the comments implied that GPRA has helped to prioritize agency efforts by helping agencies align their efforts with programs or activities that make a difference. A political appointee we interviewed echoed this comment, stating that GPRA was pushing the department to think about what it gets out of the budget, not just what it puts into it—12 to 15 years ago the “so what” was missing from the budget process. Another political appointee we interviewed stated that the department was in the process of tying its goals to its budget formulation and execution processes and linking program costs to departmental goals. A former political appointee discussed how his department used program performance information to inform a major information systems investment decision.

Furthermore, GAO case studies on the integration of performance information in budget decision making found that performance information has been used to inform the allocation of resources and for other management purposes at selected agencies. For example, the Veterans Health Administration provides its health care networks with performance information on patterns of patient care and patient health outcomes, which can be used to analyze resource allocation and costs and reallocate resources as appropriate. Officials at the Administration for Children and Families said that training and technical assistance and salaries and expense funds are often allocated based on program and performance needs. The Nuclear Regulatory Commission monitors performance against targets and makes resource adjustments, if needed, to achieve those targets.

Although there has been progress in formally establishing the linkages between budgets and plans, our survey results are somewhat conflicting and have not reflected any notable changes either in managers’ perceptions governmentwide as to their personal use of plans or performance information when allocating resources, or in their perceptions about the use of performance information when funding decisions are made about their programs. Our 2003 survey data show that a large majority of federal managers reported that they consider their agency’s strategic goals when they are allocating resources. As shown in figure 7, on our 2003 survey, an estimated 70 percent of all federal managers agreed to a great or very great extent that they considered their agency’s strategic goals when allocating resources. However, using our 1997 survey responses as a baseline, it was not a statistically significant increase over 64 percent of the managers who responded comparably then. As shown in figure 8, a similar, but somewhat smaller, majority (60 percent) of managers who expressed an opinion on our 2003 survey agreed to a great or very great extent that they used information from performance measurement when they were involved in allocating resources. In 1997, the comparable response was about the same at 62 percent. When we asked managers on another item, however, about the extent to which they perceived funding decisions for their programs being based on results or outcome-oriented performance information, only 25 percent of federal managers in 2003 endorsed this view to a great or very great extent. In 1997, 20 percent of managers expressed a comparable view, again not a significant increase. (See fig. 9.)

Agencies Have Addressed Many Critical Performance Planning and Reporting Challenges, but Weaknesses Persist

Beginning with federal agencies’ initial efforts to develop effective strategic plans in 1997 and annual performance plans and reports for fiscal year 1999, Congress, GAO, and others have commented on the quality of those efforts and provided constructive feedback on how agency plans and reports could be improved. On the basis of our current review of the strategic plans, annual performance plans, and annual performance reports of six selected agencies—Education, DOE, HUD, DOT, SBA, and SSA—we found that these documents reflect much of the feedback that was provided. For example, goals were more quantifiable and results oriented, and agencies were providing more information about goals and strategies to address performance and accountability challenges and the limitations to their performance data. However, certain weaknesses, such as lack of detail on how annual performance goals relate to strategic goals and how agencies are coordinating with other entities to achieve common objectives, persist. A detailed discussion of our scope and methodology and the results of our reviews of the six agencies’ most recent strategic plans, annual performance plans, and annual performance reports compared to initial efforts are contained in appendixes III, IV, and V, respectively.

Quality of Selected Strategic Plans Reflects Improvements over Initial Drafts

Under GPRA, strategic plans are the starting point and basic underpinning for results-oriented management. GPRA requires that an agency’s strategic plan contain six key elements: (1) a comprehensive agency mission statement; (2) agencywide long-term goals and objectives for all major functions and operations; (3) approaches (or strategies) and the various resources needed to achieve the goals and objectives; (4) a description of the relationship between the long-term goals and objectives and the annual performance goals; (5) an identification of key factors, external to the agency and beyond its control, that could significantly affect the achievement of the strategic goals; and (6) a description of how program evaluations were used to establish or revise strategic goals and a schedule for future program evaluations.

Our 1997 review of agencies’ draft strategic plans found that a significant amount of work remained to be done by executive branch agencies if their strategic plans were to fulfill the requirements of GPRA, serve as a basis for guiding agencies, and help congressional and other policymakers make decisions about activities and programs. Our assessment of 27 agencies’ initial draft strategic plans revealed several critical strategic planning issues that needed to be addressed. These planning issues were as follows: Most of the draft plans did not adequately link required elements in the plans, such as strategic goals to annual performance goals.

Long-term strategic goals often tended to have weaknesses.

Many agencies did not fully develop strategies explaining how their long-term strategic goals would be achieved.

Most agencies did not reflect in their draft plans the identification and planned coordination of activities and programs that cut across multiple agencies.

The draft strategic plans did not adequately address program evaluations.

We noted that Congress anticipated that it may take several planning cycles to perfect the process and that strategic plans would be continually refined as various planning cycles occur. We also recognized that developing a strategic plan is a dynamic process and that agencies, with input from OMB and Congress, were continuing to improve their plans.

Agencies have now had 6 years to refine their strategic planning processes. Although the six strategic plans we looked at for this review reflected many new and continuing strengths as well as improvements over the 1997 initial drafts, we continued to find certain persistent weaknesses. As depicted in table 1, of the six elements required by GPRA, the plans generally discussed all but one—program evaluation, an area in which we have found capacity is often lacking in federal agencies. Although the strategic plans generally listed the program evaluations agencies planned to complete over the planning period, they generally did not address how the agencies planned to use their evaluations to establish new or revise existing strategic goals, as envisioned by GPRA. Finally, although not required by GPRA, the strategic plans would have benefited from more complete discussions of how agencies planned to coordinate with other entities to address common challenges or achieve common or complementary goals. Appendix III provides a more detailed discussion of (1) the required and other useful elements we reviewed to assess strategic plan strengths and weaknesses and (2) changes in the quality of the six agencies’ strategic plans we reviewed.

Strategic Planning Strengths and Improvements from Initial Draft Plans

Consistent with our review of agencies’ 1997 strategic plans, the recent strategic plans we reviewed generally contained mission statements that were results oriented, distinct from other agencies, and covered the agencies’ major activities. DOT’s mission statement had improved by reflecting additional language from its enabling legislation that we recommended adding during our 1997 review. Still improvement could be made in this area as is shown by DOE’s mission statement. DOE’s mission was results oriented but did not address the department’s activities related to energy supply and conservation.

Our review of the current strategic plans also revealed improvements in the development of agencies’ long-term, strategic goals—essential for results- oriented management. Although GPRA does not require that all of an agency’s long-term, strategic goals be results oriented, the intent of GPRA is to have agencies focus their strategic goals on results to the extent feasible. In addition, as required by GPRA, the goals should be expressed in a manner that could be used to gauge success in the future and should cover an agency’s major functions or activities. All of the strategic plans we reviewed contained long-term, strategic goals that demonstrated improvements in the quality of their 1997 goals. Agencies’ long-term strategic goals generally covered their missions, were results oriented, and were expressed in a manner that could be used to gauge future success. For example, SBA improved the quality of its long-term goals by focusing more on key outcomes to be achieved and less on process improvements, as was the case in its 1997 plan. In some cases, we observed strategic goals that addressed the agency’s organizational capacity to achieve results, such as SSA’s long-term goal to strategically manage and align staff to support its mission.

We also found improvements in how agencies’ current plans addressed performance and accountability challenges we had identified, a key weakness we identified in our earlier review. Each of the agency plans we reviewed discussed the long-term goals and strategies to address the challenges that we had identified. For example, Education’s strategic plan contained a long-term strategic goal to modernize the Federal Student Assistance programs and address identified problems in this area, which we have designated as high risk since 1990. SSA noted that it considered GAO-identified performance and accountability challenges when it determined its strategic goals and objectives, however not all of the challenges are clearly addressed in the plan.

A third area of improvement we observed was in the description of the strategies agencies planned to use to achieve their long-term strategic goals. In our review of agencies’ 1997 draft strategic plans, we found that many agencies did not fully develop strategies explaining how their long- term strategic goals would be achieved. In contrast, all six of the current strategic plans we reviewed contained strategies that appeared logically linked to achieving the agencies’ long-term goals.

Other strengths and improvements we observed in meeting GPRA’s basic requirements involved the reporting of external factors that could affect the achievement of the long-term goals and the identification of crosscutting activities, although as indicated below these discussions could be improved. The six agencies reviewed for this report each reported on external factors in current strategic plans. For example, for each of the strategic objectives in DOT’s strategic plan, DOT lists factors external to its control and how those factors could affect the achievement of its objectives. Although not a requirement, some of the better plans we reviewed discussed strategies to ameliorate the effect of external factors. For example, for an external factor on teacher certification under a goal on reading, Education’s plan states that the agency “will work with the states and national accreditation bodies to encourage the incorporation of research-based reading instruction into teacher certification requirements.”

We have frequently reported that a focus on results, as envisioned by GPRA, implies that federal programs contributing to the same or similar results should be closely coordinated to ensure that goals are consistent and, as appropriate, program efforts are mutually reinforcing. This means that federal agencies are to look beyond their organizational boundaries and coordinate with other agencies to ensure that their efforts are aligned. During our 1997 review, we found that most agencies did not reflect in their draft plans the identification and planned coordination of activities and programs that cut across multiple agencies. In contrast, each of the six current agency strategic plans that we reviewed identified at least some activities and programs that cut across multiple agencies. For example, SBA’s 1997 plan contained no evidence of how the agency coordinated with other agencies, but the current plan contained a separate section describing crosscutting issues in the areas of innovation and research assistance, international trade assistance, business development assistance, veterans affairs, and disaster assistance.

Critical Strategic Planning Issues Needing Further Improvement

First, consistent with our 1997 review, the strategic plans we reviewed did not adequately link required elements in the plans. Although all of the agencies we reviewed provided some information on the relationship between their long-term and annual goals, the extent of information provided on how annual goals would be used to measure progress in achieving the long-term goals varied greatly. In the case of DOE, the plan provides a very brief description of the overall relationship between its long-term and annual goals with examples, but does not demonstrate how it will assess progress for each of its long-term goals and objectives. Another plan, DOT’s, refers the reader to the annual performance plan for information about annual goals. We have reported that this linkage is critical for determining whether an agency has a clear sense of how it will assess progress toward achieving its intended results.

Second, although the agencies’ descriptions of their strategies had improved since our initial reviews, with few exceptions, their strategies generally did not include information on how the agencies plan to align their activities, core processes, human capital, and other resources to support their mission-critical outcomes and whether they have the right mix of activities, skills, and resources to achieve their goals. Such information is critical to understanding the viability of the strategies. Furthermore, none of the agencies discussed alternative strategies they had considered in developing their plans. Without such discussions, it is unclear whether agency planning processes were truly strategic or simply a recasting of existing activities, processes, etc.

HUD was the only agency that provided any details of how it intended to coordinate with other agencies to achieve common or complementary goals for its crosscutting programs or activities. For example, to support its goal of “Equal Opportunity in Housing,” HUD’s plan states that HUD and the Department of Justice continue to coordinate their fair housing enforcement activities, especially with respect to responding quickly and effectively to Fair Housing Act complaints that involve criminal activity (e.g., hate crimes), a pattern and practice of housing discrimination, or the legality of state and local zoning or other land use laws or ordinances. We have reported that mission fragmentation and program overlap are widespread throughout the federal government. As such, interagency coordination is important for ensuring that crosscutting programs are mutually reinforcing and efficiently implemented.

Finally, the draft strategic plans did not adequately address program evaluations. In combination with an agency’s performance measurement system, program evaluations can provide feedback to the agency on how well its activities and programs contributed to achieving strategic goals. For example, evaluations can be a potentially critical source of information for Congress and others in assessing (1) the appropriateness and reasonableness of goals; (2) the effectiveness of strategies by supplementing performance measurement data with impact evaluation studies; and (3) the implementation of programs, such as identifying the need for corrective action. Evaluations are important because they potentially can be critical sources of information for ensuring that goals are reasonable, strategies for achieving goals are effective, and that corrective actions are taken in program implementation. Five out of the six current plans that we reviewed included a discussion of program evaluations, however for most of these plans the discussions lacked critical information required by GPRA, such as a discussion of how evaluations were used to establish strategic goals or a schedule of future evaluations. For example, DOE’s plan stated that internal, GAO, and Inspector General (IG) evaluations were used as resources to develop its draft strategic plan, but specific program evaluations were not identified.

Fiscal Year 2004 Annual Performance Plans Addressed Some Weaknesses of Earlier Plans, but Still Have Room for Significant Improvement

According to our review of agencies’ first annual performance plans, which presented agencies’ annual performance goals for fiscal year 1999, we found that substantial further development was needed for these plans to be useful in a significant way to congressional and other decision makers. Most of the fiscal year 1999 plans that we reviewed contained major weaknesses that undermined their usefulness in that they (1) did not consistently provide clear pictures of agencies’ intended performance, (2) generally did not relate strategies and resources to performance, and (3) provided limited confidence that agencies’ performance data will be sufficiently credible. Although all of the fiscal year 1999 plans contained valuable information for decision makers, their weaknesses caused their usefulness to vary considerably within and among plans.

As shown in table 2, our current review of agencies’ fiscal year 2004 performance plans found that five agencies—Education, HUD, SBA, SSA, and DOT—improved their efforts to provide a clear picture of intended performance, with SSA and DOT being the clearest. Furthermore, the same five agencies improved the specificity of the strategies and resources they intended to use to achieve their performance goals, with DOT being the most specific. Finally, the same five agencies—Education, HUD, SBA, SSA, and DOT—made improvements in the area of greatest weakness— reporting on how they will ensure performance data will be credible. However, only DOT’s plan provided a full level of confidence that the performance data the agency intended to collect would be credible. Appendix IV provides a more detailed discussion of (1) the required and other useful elements we reviewed to assess the clarity of the picture of intended performance, the specificity of the strategies and resources, and the level of confidence in the performance data and (2) changes in the quality of the six agencies’ annual performance plans we reviewed.

Plans Generally Provided a Clearer Picture of Intended Performance, Except for Crosscutting Areas

At the most basic level, an annual performance plan is to provide a clear picture of intended performance across the agency. Such information is important to Congress, agency managers, and others for understanding what the agency is trying to achieve, identifying subsequent opportunities for improvement, and assigning accountability. Our current review of agencies’ fiscal year 2004 performance plans found that five of the six agencies provided a clearer picture of intended performance than their fiscal year 1999 plans did, although only two of the 2004 plans—DOT’s and SSA’s—received the highest rating possible. As shown in table 2, except for DOT, the six agencies we reviewed for this report initially provided a limited picture of intended performance. Most of the fiscal year 1999 performance plans we previously reviewed had at least some objective, quantifiable, and measurable goals, but few plans consistently included a comprehensive set of goals that focused on the results that programs were intended to achieve. Moreover, agencies did not consistently follow OMB’s guidance that goals for performance and accountability challenges be included in the plans. Agencies’ plans generally showed how their missions and strategic goals were related to their annual performance goals and covered all of the program activities in the agencies’ budget requests. In addition, many agencies took the needed first step of identifying their crosscutting efforts, with some including helpful lists of other agencies with which they shared a responsibility for addressing similar national issues. However, the plans generally did not go further to describe how agencies expected to coordinate their efforts with other agencies.

The fiscal year 2004 plans improved the picture of performance by making annual goals and performance measures more results oriented, objective, and quantifiable. For example, Education’s plan included a measure for the number of states meeting their eighth grade mathematics achievement targets under the long-term goal to improve mathematics and science achievement for all students. We previously criticized Education’s 1999 plan for lacking such outcome-oriented measures. Another overall improvement we observed was that all of the plans described intended efforts to address performance and accountability challenges we and others had previously identified. For instance, to address the governmentwide high-risk area of strategic human capital management, HUD states that to develop its staff capacity, it will complete a comprehensive workforce analysis in 2004 to serve as the basis to fill mission critical skill gaps through succession planning, hiring, and training initiatives in a 5-year human capital management strategy. The clarity of DOE’s plan remained limited because its annual goals were not clearly linked to its mission, the long-term goals in its strategic plan, or the program activities in its budget request.

Although five of the six agencies improved the clarity of the picture of intended performance, improvement is still needed in reporting on crosscutting efforts. In both the 1999 and 2004 plans, many agencies identified their crosscutting efforts, with some including helpful lists of other agencies with which they shared a responsibility for addressing similar national issues. Our review of fiscal year 2004 plans shows that the six agencies we reviewed still did not discuss how they expected to coordinate with other agencies to address common challenges or to achieve common or complementary performance goals. As we have reported previously, improved reporting on crosscutting efforts can help Congress use the annual performance plan to evaluate whether the annual goals will put the agency on a path toward achieving its mission and long- term strategic goals. In addition, the plans can aid in determining efforts to reduce significant program overlap and fragmentation that can waste scarce resources, confuse and frustrate program customers, and limit overall program effectiveness.

None of the six agencies’ plans indicated an intention to request waivers of specific administrative procedural requirements and controls that may be impeding an agencies’ ability to achieve results. This provision of GPRA allows agencies greater managerial flexibility in exchange for accountability for results. We previously reported on the results of the pilot project to implement this provision of GPRA and found that the pilot did not work as intended. OMB did not designate any of the seven departments and one independent agency that submitted a total of 61 waiver proposals as GPRA managerial accountability and flexibility pilots. For about three-quarters of the waiver proposals, OMB or other central management agencies determined that the waivers were not allowable for statutory or other reasons or that the requirement for which the waivers were proposed no longer existed. For the remaining proposals, OMB or other central management agencies approved waivers or developed compromises by using authorities that were already available independent of GPRA.

Plans More Specifically Related Strategies and Resources to Performance Goals

To judge the reasonableness of an agency’s proposed strategies and resources, congressional and other decision makers need complete information on how the proposed strategies and resources will contribute to the achievement of agency goals. Agencies generally improved their plans by better relating strategies and resources to performance. Education’s, HUD’s, SBA’s, and SSA’s 1999 plans had a limited discussion, while DOE’s and DOT’s 1999 plans had a general discussion. In 2004, five of the six plans—Education’s, DOE’s, HUD’s, SBA’s, and SSA’s—provided general discussions of how their strategies and resources would contribute to achieving their performance goals. DOT’s 2004 plan improved to include a specific discussion.

Our review of the 1999 plans found that most agencies’ performance plans did not provide clear strategies that described how performance goals would be achieved. In contrast, the 2004 performance plans we reviewed generally provided lists of the agencies’ current array of programs and initiatives. Several plans provided a perspective on how these programs and initiatives were necessary or helpful for achieving results. For example, DOE and HUD included in their plans a “means and strategies” section for each of their goals that described how the goal would be achieved. One strategy DOE identified to meet its goal of contributing unique, vital facilities to the biological environmental sciences was to conduct peer reviews of the facilities to assess the scientific output, user satisfaction, the overall cost-effectiveness of each facility’s operations, and their ability to deliver the most advanced scientific capability.

In addition, each of the agencies’ plans identified the external factors that could influence the degree to which goals are achieved. Some of the better plans, such as DOT’s and SBA’s, provided strategies to mitigate the negative factors or take advantage of positive factors, as appropriate. For example, for its transportation accessibility goals, DOT’s plan states that as the population ages, more people will require accessible public transit, for which states and local agencies decide how best to allocate federally provided resources. One of the strategies DOT intends to employ to address this external factor is the “Special Needs of Elderly Individuals and Individuals with Disabilities” grant program. The plan states the grant program will help meet transportation needs of the elderly and persons with disabilities when regular transportation services are unavailable, insufficient, or inappropriate to meet their needs.

Agencies’ 2004 plans did not consistently describe all the resources needed and how they would be used to achieve agency goals. Our review of agencies’ fiscal year 1999 plans found that most did not adequately describe—or reference other appropriate documents that describe—the capital, human, information, and financial resources needed to achieve their agencies’ performance goals. The 2004 plans we reviewed generally described the funding levels needed to achieve their performance goals overall and in some cases broke out funding needs by high-level performance goal. For example, SSA’s plan provides a general perspective on the financial resources needed to achieve its performance goals because it provides budget information by account and program activity. However, the plan is neither structured by budget program activity or account, nor does it provide a crosswalk between the strategic goals and budget program accounts. In contrast, HUD’s plan presented its requested funding and staffing levels at the strategic goal level, but did not present budget information at the level of its annual goals. In addition, although the plans make brief mention of nonfinancial resources, such as human capital, information technology, or other capital investments, little information is provided on how such resources would be used to achieve performance goals.

Plans Continue to Provide Less Than Full Confidence That Performance Data Will Be Credible

Credible performance information is essential for accurately assessing agencies’ progress towards the achievement of their goals and, in cases where goals are not met, identifying opportunities for improvement or whether goals need to be adjusted. Under GPRA, agencies’ annual performance plans are to describe the means that will be used to verify and validate performance data. To help improve the quality of agencies’ performance data, Congress amended GPRA through the Reports Consolidation Act of 2000 to require that agencies assess the completeness and reliability of the performance data in their performance reports. Agencies were also required to discuss in their report any material inadequacies in the completeness and reliability of their performance data and discuss actions to address these inadequacies. Meeting these new requirements suggests the need for careful planning to ensure that agencies can comment accurately on the quality of the performance data they report to the public.

As shown in table 2, although five of the six agencies we reviewed improved in reporting how they plan to ensure that performance data will be credible, only one agency—DOT—improved enough over its 1999 plan to provide a full level of confidence in the credibility of its performance data. Four agencies—Education, HUD, SBA, and SSA—improved enough to provide a general level of confidence. However, DOE provided the same limited level of confidence in the credibility of the performance data as in its 1999 plan. Regarding all 24 of the fiscal year 1999 performance plans we reviewed, we found most provided only superficial descriptions of procedures that agencies intended to use to verify and validate performance data. Moreover, in general, agencies’ performance plans did not include discussions of documented limitations in financial and other information systems that may undermine efforts to produce high-quality data. As we have previously noted, without such information, and strategies to address those limitations, Congress and other decision makers cannot assess the validity and reliability of performance information.

We found that each of the 2004 plans we reviewed contained some discussion of the procedures the agencies would use to verify and validate performance information, although in some cases the discussion was inconsistent or limited. For example, the discussions of SBA’s verification and validation processes for its indicators in the 2004 plan were generally one- or two-sentence statements. SBA also noted that it does not independently verify some of the external data it gathers or that it does not have access to the data for this purpose. In contrast, the DOT plan referred to a separate compendium available on-line that provides source and accuracy statements, which give more detail on the methods used to collect performance data, sources of variation and bias in the data, and methods used to verify and validate the data.

In addition, all of the agencies except DOE discussed known limitations to performance data in their plans. These agencies’ plans generally provided information about the quality of each performance measure, including any limitations. According to DOE officials, DOE’s plan generally does not discuss data limitations because the department selected goals for which data are expected to be available and therefore did not anticipate finding any limitations. However, in our 2003 Performance and Accountability Series report on DOE, we identified several performance and accountability challenges where data were a concern, such as the need for additional information on the results of contractors’ performance to keep projects on schedule and within budget. DOE’s contract management continues to be a significant challenge for the department and remains at high risk.

Finally, the remaining five agencies also discussed plans to address limitations to the performance data. For example, DOT’s plan provided a general discussion of the limitations to the internal and external sources of data used to measure performance. Detailed discussions were contained in an appendix to the plan and separate source and accuracy statements. This information had been lacking in its 1999 plan. Education, HUD, SBA, and SSA also provided information on limitations to their performance data and plans for improvement.

Strengths and Weaknesses of Selected Agencies’ Fiscal Year 2002 Annual Performance Reports

Key to improving accountability for results as Congress intended under GPRA, annual performance reports are to document the results agencies have achieved compared to the goals they established. To be useful for oversight and accountability purposes, the reports should clearly communicate performance results, provide explanations for any unmet goals as well as actions needed to address them, and discuss known data limitations as well as how the limitations are to be addressed in the future. Compared to our reviews of the six agencies’ fiscal year 1999 performance reports, we identified a number of strengths and improvements as well as areas that continued to need improvement. Because the scope of our review of the fiscal year 2002 reports was broader than that for the fiscal year 1999 reports we previously reviewed, we were unable to make specific comparisons for the three characteristics we used to assess the fiscal year 2002 reports. However, we discuss comparative information on aspects of the reports where available. Table 3 shows the results of our assessment of the six agencies’ annual performance reports for fiscal year 2002. Appendix V provides a more detailed discussion of (1) the required and other useful elements we reviewed to assess the clarity of the picture of performance, the clarity of the linkage between costs and performance, and the level of confidence in the performance data and (2) changes in the quality of the six agencies’ annual performance plans we reviewed.

Progress in Providing a Clear Picture of Performance

The six agency reports that we reviewed contained a number of strengths, some of which we can describe as improvements over the reports on fiscal year 1999 performance. A key strength of four of the 2002 reports (Education, HUD, DOT, SSA) was a discussion of the relationship between the strategic plan, performance plan, and performance report. For example, SSA’s report identified relevant results that were linked to its strategic objective to deliver “citizen-centered, world-class service,” such as maintaining the accuracy, timeliness, and efficiency of service to people applying for its benefit programs. The clarity of the DOE and SBA reports was limited by not clearly relating agency performance results to strategic and annual performance goals. For example, the structure of SBA’s report reflected the objectives in its draft 2003 to 2008 strategic plan rather than those in its 2002 performance plan, making it difficult to assess progress against the original 2002 objectives. Furthermore, although there is no “right” number of performance measures to be used to assess progress, a number of the plans allowed for an easier review of results by limiting the overall number of measures presented or by highlighting key performance measures of greatest significance to their programs. For example, SBA discussed a total of 19 performance goals and DOT discussed a total of 40. Although SSA discussed a total of 69 performance goals, the report highlighted its progress in achieving 14 key goals. In contrast, Education, HUD, and DOE presented a total of 120, 184, and 260 measures, respectively. Furthermore, while Education and SSA each provided a table showing progress across all its measures, the other agencies did not provide such summary information.

As we found in our earlier reviews, the six agencies’ fiscal year 2002 reports generally allowed for an assessment of progress made in achieving agency goals. Some of the reports made this assessment easier than others by providing easy-to-read summary information. For example, SSA provided a table at the beginning of the report that summarized the results for each of its 69 indicators with the following dispositions: met, not met, almost met, and data not yet available. Other reports, such as HUD’s, required an extensive review to make this assessment. In addition, to place current performance in context, each of the agencies’ reports contained trend information, as required by GPRA, which allowed for comparisons between current year and prior year performance.

In addition, the majority of agencies maintained, or demonstrated improvements over, the quality of their 1999 reports in discussing the progress achieved in addressing performance and accountability challenges identified by agency IGs and GAO. For example, SBA’s report contained two broad overviews and an appendix describing the status of GAO audits and recommendations, as well as a description of the most serious management challenges SBA faces as identified by the agency’s IG.

Unfortunately, many of the weaknesses we identified in the agencies’ fiscal year 2002 reports were similar to those we found in their fiscal year 1999 reports related to the significant number of performance goals (1) which were not achieved and lacked explanations or plans for achieving the goal in the future and (2) for which performance data were unavailable. Three of the six agencies we reviewed—HUD, SSA, and Transportation—did not consistently report the reasons for not meeting their goals. For example, Transportation provided explanations for only 5 of the 14 goals it did not meet. In addition, similar to our 1999 report findings, three of the six agencies we reviewed—HUD, SBA, and DOT—did not discuss their plans or strategies to achieve unmet goals in the future. For example, HUD reported “substantially meeting” only 47 percent of the performance targets in fiscal year 2002. However, although HUD provides various reasons for not meeting all its targets, it offers no information on plans or time frames to achieve the goals in the future. Finally, we continued to observe a significant number of goals for which performance data were unavailable. For example, performance data for 10 of SBA’s 19 performance goals were unavailable.

In addition, the majority of the reports we reviewed did not include other GPRA requirements. The reports generally did not evaluate the performance plan for the current year relative to the performance achieved toward the performance goals in the fiscal year covered by the report. The reports also did not discuss the use or effectiveness of any waivers in achieving performance goals. In addition, for two of the agencies—DOE and SBA—program evaluation findings completed during the fiscal year were not summarized. As we have previously noted, such evaluations could help agencies understand the relationship between their activities and the results they hope to achieve.

Progress in Linking Resources to Results

Although linking costs to performance goals is not a requirement of GPRA, both GPRA and the CFO Act emphasized the importance of linking program performance information with financial information as a key feature of sound management and an important element in presenting to the public a useful and informative perspective on federal spending. The committee report for GPRA suggested that developing the capacity to relate the level of program activity with program costs, such as cost per unit of result, cost per unit of service, or cost per unit of output, should be a high priority. In our survey of federal managers, this year we asked for the first time the extent to which federal managers had measures of cost- effectiveness for the programs they were involved with. Only 31 percent of federal managers we surveyed reported having such measures to a great or very great extent, lower than any of the other types of measures associated with GPRA we asked about by at least 12 percent (see fig. 3 in ch. 2). Under the PMA, the current administration has set an ambitious agenda for performance budgeting, calling for agencies to better align budgets with performance goals and focus on capturing full budgetary costs and matching those costs with output and outcome goals. All this suggests that agencies will need to develop integrated financial and performance management systems that will enable the reporting of the actual costs associated with performance goals and objectives along with presentations designed to meet other budgetary or financial purposes, such as the accounts and program activities found in the President’s Budget and responsibility segments found in financial statements.

Of the six agencies we reviewed, only Education’s report clearly linked its budgetary information to the achievement of its performance goals or objectives. Education’s report laid out, using both graphics and text, the estimated appropriations associated with achieving each of its 24 objectives. In addition the report provided the staffing in full-time equivalent employment (FTEs) and an estimate of the funds from salaries and expenses contributing to the support of each of these objectives. SBA’s report contained crosswalks that showed the relationship between SBA’s strategic goals, outcome goals, performance goals, and programs. Because SBA shows the resources for each program, a reader can infer a relationship between SBA’s resources and performance goals. However, the linkage between resources and results would be clearer if results and resources were presented by performance goal as well. SSA provided a limited view of the costs of achieving its performance goals by providing the costs associated with four out of five of its strategic goals. However, as reported by the IG, SSA needs to further develop its cost accounting system, which would help link costs to performance. DOE also provided a limited view of the costs of achieving its performance goals by organizing its performance information by budget program activity and associated net costs. According to DOE officials, the department plans to link its individual performance measures to the costs of program activities in future reports. Neither HUD nor DOT provided information on the cost of achieving individual performance goals or objectives.

Progress in Providing Confidence in the Credibility of Performance Data

To assess the degree to which an agency’s report provided full confidence that the agency’s performance information was credible, we examined the extent to which the reports discussed the quality of the data presented. As shown in table 3, only DOT’s report provided a full level of confidence in the quality of the data. The other agencies provided general or limited confidence in their data.

All six agencies in our current review complied with the Reports Consolidation Act of 2000 by including assessments of the completeness and reliability of their performance data in their transmittal letters. In contrast, we found that only 5 of the 24 CFO Act agencies included this information in their fiscal year 2000 performance reports. Of the six agencies in our current review, only DOE provided this assessment in its fiscal year 2000 report. For example, the Secretary of DOT stated in the transmittal letter that the 2002 report “contains performance and financial data that are substantially complete and reliable.” However, only two of the six agencies also disclosed material inadequacies in the completeness and reliability of their performance data and discussed actions to address the inadequacies in their transmittal letters. For example, SBA stated in its transmittal letter that it is “working to improve the completeness and reliability of the performance data for the advice provided to small business through SBA’s resource partners.” SBA explained that data for this aspect of its performance are collected through surveys, which are inconsistent and not comparable, and for which client responses are difficult to obtain. SBA stated that it is working to improve the survey instruments it uses to obtain performance data.

In addition to the requirements of the Reports Consolidation Act, we have previously reported on other practices that enhance the credibility of performance data that are not specifically required by GPRA. For instance, discussions of standards and methods used by agencies to assess the quality of their performance data in their performance reports provide decision makers greater insight into the quality and value of the performance data. None of the reports explicitly referred to a specific standard they used, however, DOE described its method for assuring data quality. The report states that the heads of DOE’s organizational elements certified the accuracy of their performance data. DOE subsequently reviewed the data for quality and completeness.

Other useful practices that help foster transparency to the public and assist decision makers in understanding the quality of an agency’s data include: (1) discussion of data quality, including known data limitations and actions to address the limitations, and (2) discussion of data verification and validation procedures, including proposals to review data collection and verification and validation procedures.

All six agencies’ reports described data limitations, although discussions were mostly brief and very high level. One exception was DOT, which directed readers to the DOT Web site to obtain an assessment of the completeness and reliability of its performance data and detailed information on the source, scope, and limitations of the performance data. HUD and SBA also discussed plans for addressing the limitations. For example, HUD stated that to address problems with its indicator on the number of homeowners who have been assisted with the Home Investment Partnership Program (HOME), HUD has established a team of managers, technical staff, and contractors to make a series of improvements to the Integrated Disbursement and Information System beginning in fiscal year 2003 that should reduce the need to clean up the data.

Each of the six agencies’ reports also discussed the procedures they used to verify and validate their performance data. However, these discussions ranged from the very general description of the DOE method (noted previously), to the very detailed discussions provided by DOT. DOT provides an on-line compendium that discusses the source and accuracy of its data. Furthermore, DOT’s 2002 report also describes strategies being undertaken to address the quality of its data. The report states that a DOT intermodal working group addressed data quality issues by developing departmental statistical standards and by updating source and accuracy statements for all of DOT’s data programs. The working group also worked to improve quality assurance procedures, evaluate sampling and nonsampling errors, and develop common definitions for data across modes.

Challenges to GPRA Implementation Persist

While a great deal of progress has been made in making federal agencies more results oriented, numerous challenges still exist to effective implementation of GPRA. The success of GPRA depends on the commitment of top leadership within agencies, OMB, and Congress. However, according to federal managers surveyed, top leadership commitment to achieving results has not grown significantly since our 1997 survey. Furthermore, although OMB has recently shown an increased commitment to management issues, it significantly reduced its guidance to agencies on GPRA implementation compared to prior years, and it is not clear how the program goals developed through its PART initiative will complement and integrate with the long-term, strategic focus of GPRA. Obtaining leadership commitment to implement a strategic plan depends in part on the usefulness and relevance of agency goals and strategies to agency leaders, Congress, and OMB. However, GPRA’s requirement to update agency strategic plans every 3 years is out of sync with presidential and congressional terms and can result in updated plans that do not have the support of top administration leadership and key congressional stakeholders.

As noted in chapter 2, more federal managers surveyed reported having results-oriented performance measures for their programs and we would expect to have seen similar increases in the use of this information for program management. However, we did not observe any growth in their reported use of this information for key management activities, such as adopting new program approaches or changing work processes. Additionally, managers noted human capital-related challenges that impede results-oriented management, including a lack of authority and training to carry out GPRA requirements, as well as a lack of recognition for the results achieved.

Consistent with our previous work, federal managers in our focus groups reported that significant challenges persist in setting outcome-oriented goals, measuring performance, and collecting useful data. However, our survey data suggested that federal managers do not perceive issues, such as “difficulty distinguishing between the results produced by the program and results caused by other factors” and “difficulty obtaining data in time to be useful,” to be substantial hindrances to measuring performance or using performance information.

Additionally, mission fragmentation and overlap contribute to difficulties in addressing crosscutting issues, particularly when those issues require a national focus, such as homeland security, drug control, and the environment. GAO has previously reported on a variety of barriers to interagency cooperation, such as conflicting agency missions, jurisdiction issues, and incompatible procedures, data, and processes. We have also reported that OMB could use the provision of GPRA that calls for OMB to develop a governmentwide performance plan to integrate expected agency- level performance. Unfortunately, this provision has not been fully implemented and the federal government lacks a tool, such as a strategic plan, that could provide a framework for a governmentwide reexamination of existing programs, as well as proposals for new programs. Finally, federal managers in our focus groups and political appointees we interviewed believed that Congress does not use performance information to the fullest extent to conduct oversight and to inform appropriations decisions. While there is concern regarding Congress’ use of performance information, it is important to make sure that this information is initially useful. As a key user of performance information, Congress needs to be considered a partner in shaping agency goals at the outset. GPRA provides Congress opportunities to influence agency performance goals through the consultation requirement for strategic plans and through Congress’ traditional oversight role.

Top Leadership Does Not Consistently Show Commitment to Achieving Results

We have previously testified that perhaps the single most important element of successful management improvement initiatives is the demonstrated commitment of top leaders to change. This commitment is most prominently shown through the personal involvement of top leaders in developing and directing reform efforts. Organizations that successfully address their long-standing management weaknesses do not “staff out” responsibility for leading change. Top leadership involvement and clear lines of accountability for making management improvements are critical to overcoming organizations’ natural resistance to change, marshalling the resources needed in many cases to improve management, and building and maintaining the organizationwide commitment to new ways of doing business.

Results from our surveys show that while the majority of managers continue to indicate top leadership demonstrates a strong commitment to achieving results, we have not seen a noteworthy improvement in the percentage of managers expressing this view. From our 1997 survey, we estimated about 57 percent of managers overall reported such commitment to a great or very great extent. On our 2003 survey, 62 percent of managers expressed a comparable view—a higher but not statistically significant increase. (See fig. 10.)

As shown in figure 11, however, we continued to see a significant difference between the perceptions of SES and non-SES managers on this issue. That is, the percentage of SES managers reporting that top leadership demonstrated strong commitment to a great or very great extent in 2003 was 22 percent higher than for non-SES managers.

We observed in our 1997 and 2000 reports on governmentwide implementation of GPRA that we would expect to see managers’ positive perceptions on items, such as the extent to which top leadership is committed to achieving results, become more prevalent and the gap between SES and non-SES managers begin to narrow as GPRA and related reforms are implemented; however, these changes do not appear to be happening as expected.

Demonstrating the willingness and ability to make decisions and manage programs based on results and the ability to inspire others to embrace such a model are important indicators of leadership commitment to results- oriented management. However, in both our 1997 and 2000 surveys, only about 16 percent of managers reported that changes by management above their levels to the programs for which they were responsible were based on results or outcome-oriented performance information to a great or very great extent. In our 2003 survey, this indicator increased to 23 percent, a statistically significant increase from prior surveys. Twenty-eight percent of federal managers surveyed who expressed an opinion reported that the lack of ongoing top executive commitment or support for using performance information to make program/funding decisions hindered measuring performance or using performance information to a great or very great extent.

Our interviews with 10 top political appointees from the Clinton and current Bush administrations indicated a high level of support and enthusiasm for effectively implementing the principles embodied in GPRA. For example, one appointee noted that GPRA focused senior management on a set of goals and objectives to allow the organization to understand what is important and how to deal with accomplishment at a macro-level, as well as provided a structure for problem solving. Another political appointee noted that GPRA has made it important to look at what you get out of the budget, not just what you put into it, while another concluded that GPRA brought about a fundamental rethinking of how they managed their programs and processes. Such indications of support for GPRA are promising. However, to support the transition to more results-oriented agency cultures, top agency management will need to make a more concerted effort to translate their enthusiasm for GPRA into actions that communicate to employees that top management cares about performance results and uses the information in its decision making.

The need for strong, committed leadership extends to OMB as well. OMB has shown a commitment to improving the management of federal programs, both through its leadership in reviewing agency program performance using the PART tool as well as through the PMA, which calls for improved financial performance, strategic human capital management, competitive sourcing, expanded electronic government, and performance budget integration. Using the foundation of information generated by agencies in their strategic plans, annual performance plans, and program performance reports, OMB has used the PART tool to exercise oversight of selected federal programs by assessing program purpose and design, the quality of strategic planning, the quality of program management, and the extent to which programs can demonstrate results. PART provides OMB a lens through which to view performance information for use in the budget formulation process. PART, and OMB’s use of performance data in the budget formulation process, potentially can complement GPRA’s focus on increasing the supply of credible performance information by promoting the demand for this information in the budget formulation process. As we reported in chapter 2, more federal managers noted that OMB was paying attention to their agencies’ efforts under GPRA. (See fig. 6.) Additionally, OMB convened a performance measurement workshop in April 2003 to identify practical strategies for addressing common performance measurement challenges. As a result of this workshop, it produced a paper in June 2003 that included basic performance measurement definitions and concepts and common performance measurement problems that were discussed at the workshop. This was part of OMB’s continuing efforts to improve PART as an evaluation tool.

However, there are areas where OMB could further enhance its leadership. OMB has stated that the PART exercise presents an opportunity to inform and improve on agency GPRA plans and reports and establish a meaningful, systematic link between GPRA and the budget process. OMB has instructed agencies that, in lieu of a performance plan, they are to submit a performance budget that includes information from the PART assessments, including all performance goals used in the assessment of program performance done under the PART process. The result is that program-specific performance measures developed through the PART review are to substitute for other measures developed by the agency through its strategic planning process. GPRA is a broad legislative framework that was designed to be consultative with Congress and other stakeholders and address the needs of many users of performance information—Congress to provide oversight and inform funding decisions, agency managers to manage programs and make internal resource decisions, and the public to provide greater accountability. Changing agency plans and reports for use in the budget formulation process may not satisfy the needs of these other users. Users other than OMB are not likely to find the information useful unless it is credible and valid for their purposes. PART’s program-specific focus may fit with OMB’s agency-by- agency budget reviews, but it is not well suited to achieving one of the key purposes of strategic plans—to convey agencywide, long-term goals and objectives for all major functions and operations. PART’s focus on program-specific measures does not substitute for the strategic, long-term focus of GPRA on thematic goals and department- and governmentwide crosscutting comparisons.

To reach the full potential of performance management, agency planning and reporting documents need to reflect the full array of uses of performance information, which may extend beyond those needed for formulating the President’s Budget. However, it is not yet clear whether the results of those reviews, such as changes to agencies’ program performance measures, will complement and be integrated with the long- term, strategic goals and objectives agencies have established in consultation with Congress and other stakeholders under GPRA. OMB has not yet clearly articulated how PART is to complement GPRA. Focus group participants suggested that the administration and OMB needed to reinforce GPRA’s usefulness as a management tool for agencies. They also emphasized the need for OMB to help agencies understand how to integrate GPRA with other management initiatives, such as PART.

As we noted in chapter 3, agencies’ plans and reports still suffer from persistent weaknesses and could improve in a number of areas, such as attention to issues that cut across agency lines, and better information about the quality of the data that underlie agency performance goals. However, OMB’s July 2003 guidance for the preparation and submission of strategic plans, annual performance plans, and annual performance reports is significantly shorter and less detailed than its 2002 guidance. For example, OMB no longer provides detailed guidance to agencies for the development of performance plan components. OMB’s 2002 guidance on the preparation and submission of annual performance plans is approximately 39 pages long; in its 2003 guidance, that discussion spans only 2 pages. The 2003 guidance in this area does not include entire sections found in the 2002 guidance, such as principles for choosing performance goals and indicators for inclusion in the annual plan, types of performance goals, crosscutting programs, and requirements for verifying and validating data.

OMB needs to maintain and strengthen its leadership role in working with agencies to help them produce the highest quality GPRA documents through its formal guidance and reviews of strategic plan and report submissions. Focus group participants discussed the need for consistent guidance on how to implement GPRA. Furthermore, there is no evidence that agencies have institutional knowledge of GPRA requirements that would obviate the need for OMB’s guidance. New managers will need a consistent resource that provides practical guidance on what agencies need to include in their planning and reporting documents to comply with GPRA and reflect best practices. Consistent, explicit OMB guidance on preparing GPRA documents can help ensure that gains in the quality of GPRA documents are maintained and provide a resource for agencies to make further improvements in those documents. For example, guidance on how to discuss coordination of crosscutting programs or improvements to the credibility of performance data in agency performance plans goes hand-in-hand with OMB’s enhanced oversight of agency performance through the PART exercise.

The success of GPRA depends on the commitment of top leadership within agencies, OMB, and Congress. Obtaining such leadership commitment depends in part on the usefulness and relevance of agency goals and strategies to these parties. GPRA requires an agency to develop a strategic plan at least every 3 years to cover the following 5-year period. Thus, there have been two required updates of strategic plans since the initial strategic plans were submitted for fiscal year 1997—fiscal year 2000 and fiscal year 2003. The fiscal year 2000 update occurred the year before a new presidential term began. According to our focus group participants—both the experts and federal managers—it makes little sense to require an update of a strategic plan shortly before a new administration is scheduled to take office. For example, changes in political leadership generally result in a new agenda with new objectives. Such changes force agencies to revise their plans, management initiatives, and strategies, which translates into additional GPRA-related work. A strategic plan that does not reflect the participation and buy-in of top administration leadership and key congressional stakeholders is unlikely to be successfully implemented. Therefore, GPRA’s requirement to update agency strategic plans according to a schedule that is out of sync with presidential and congressional terms means that effort may be wasted on plans that lack the support of top leadership.

Managers Report Mixed Results in Use of Performance Information

GPRA’s usefulness to agency leaders and managers as a tool for management and accountability was cited as a key accomplishment numerous times by focus group participants. However, a number of alternative views indicated use of performance information for key management decisions has been mixed. For example, one participant said they did not believe GPRA has been used as a tool yet, while another participant commented that only better managers take advantage of GPRA as a management tool. According to focus group participants, although many federal managers understand and use results-oriented management concepts in their day-to-day activities, such as strategic planning and performance measurement, they do not always connect these concepts to the requirements of GPRA.

This view was strongly supported by our survey results. Prior to mentioning GPRA in our survey, we asked federal managers the extent to which they consider their agency’s strategic goals when engaging in key management tasks such as setting program activities, allocating resources, or considering changes in their programs. A relatively high percentage of managers—ranging from 66 to 79 percent—responded to a great or very great extent. However, when we asked similar questions about the extent to which they considered their agency’s annual performance goals as set forth in the agency’s GPRA annual performance plan for the same activities, the comparable responses were considerably lower, ranging from 22 to 27 percent.

Because the benefit of collecting performance information is only fully realized when this information is actually used by managers, we asked them about the extent to which they used the information obtained from measuring performance for various program management activities. As shown in figure 12, for seven of the nine activities we asked about, the majority of managers who expressed an opinion reported using performance information to a great or very great extent in 2003. Across all nine activities, the percentage of managers saying they used performance information to a great or very great extent ranged from 41 percent for developing and managing contracts to 60 percent for allocating resources, setting individual job expectations, and rewarding staff. While we had observed a decline in the reported use of performance information to this extent for many of these activities between 1997 and 2000, our 2003 results increased to levels not significantly different from 1997 for all but one category—adopting new program approaches or changing work processes. This category of use continued to be significantly lower at 56 percent in 2003 than it was in 1997 at 66 percent. Although another category, coordinating program efforts with other internal or external organizations, shows a similar pattern of limited recovery, the difference between the 1997 and 2003 results is not statistically significant.

We have reported that involving program managers in the development of performance goals and measures is critical to increasing the relevance and usefulness of this information to their day-to-day activities. Yet, our survey data indicate that participation in activities related to the development and use of performance information has also been mixed. In 2003, only 14 percent of managers believed to a great or very great extent that their agencies considered their contributions to or comments on their agency’s GPRA plans or reports. However, significantly more SES managers (43 percent) than non-SES managers (12 percent) expressed this view. Also, when compared to our 2000 survey when we first asked this question, the percentage of SES managers expressing this view in 2003 was significantly higher than in 2000 (32 percent). The percentage of non-SES managers was essentially unchanged from 2000 (10 percent).

Furthermore, as shown in figure 13, overall around half or fewer of managers responded “yes” on our 2003 survey to questions about being involved in developing ways to measure whether program performance goals are being achieved (46 percent), gathering and analyzing data to measure whether programs were meeting their specific performance goals (51 percent), or using measures for program performance goals to determine if the agency’s strategic goals were being achieved (43 percent). None of these overall results were significantly different from our 1997 results. We did find, however, that significantly more SES managers responded “yes” on the 2003 survey (72 percent) than the 1997 survey (55 percent) with regard to being involved in using performance measurement information to determine if the agency’s strategic goals were being achieved when compared to our 1997 results.

Managers Continue to Confront a Range of Human Capital Management Challenges

Managing people strategically and maintaining a highly skilled and energized workforce that is empowered to focus on results are critically important. Such human capital management practices are essential to the success of the federal government in the 21st century and to maximizing the value of its greatest asset—its people. Our survey results showed continuing challenges related to the adequacy of managerial decision making authority, training, and incentives.

Federal Managers Report That They Are Held Accountable for Program Results but Do Not Have the Decision-Making Authority They Need to Accomplish Agency Goals

High-performing organizations seek to shift the focus of management and accountability from activities and processes to contributions and achieving results. In each of our three surveys, we asked managers about the amount of decision-making authority they had and the degree to which they were held accountable for results.

As shown in figure 14, for 2003, an estimated 40 percent of federal managers overall reported that they had the decision-making authority they needed to help the agency accomplish its strategic goals to a great or very great extent. This was a statistically significant increase over our 1997 estimate of 31 percent. While there were more SES and non-SES managers expressing this view on our 2003 survey than the 1997 survey, it was the non-SES managers that showed the significant increase. Despite this promising trend, however, there continued to be substantial differences in 2003, as well as on the two previous surveys, between the responses of SES and lower-level managers on this question. Compared to the 57 percent of SES managers who reported having such authority to a great or very great extent in 2003, only 38 percent of non-SES managers reported having such authority to a great or very great extent.

However, when asked the extent to which managers or supervisors at their levels were held accountable for the accomplishment of agency strategic goals, 57 percent responded to a great or very great extent in 2003. Unlike in other areas, where SES managers had significantly different views from non-SES managers, there was little difference in the area of accountability. (See fig. 15.)

This 57 percent is significantly higher than the 40 percent of managers overall who indicated that they had comparable decision-making authority. However, in contrast to the question on authority, as shown in figure 14, where more SES managers than non-SES managers expressed the view that they had the authority, there was little difference, as shown in figure 15, between the two groups in their views about being held accountable for achieving agency strategic goals to a great or very great extent. As figures 14 and 15 further illustrate, roughly the same percentage of SES managers perceived to a great or very great extent that managers at their level had decision-making authority and accountability for achieving agency strategic goals. This result suggests that their authority was perceived to be on par with their accountability. In contrast, only 38 percent of non-SES managers perceived that managers at their levels had the decision-making authority they needed to a great or very great extent, while 57 percent perceived that they were held accountable to a comparable extent.

Managers are hard-pressed to achieve results when they do not have sufficient authority to act. In our report containing the results of our 1997 survey, we noted that agencies needed to concentrate their efforts on areas where managers were not perceiving or experiencing progress, such as that concerning devolving decision-making authority to managers throughout their organizations. While authority for achieving results appears to be in a modestly upward trend, the balance between authority and accountability that fosters decision making to achieve results could be further improved, particularly among non-SES managers.

Fewer Than Half of Managers Reported Training on Key Tasks

We previously reported on the need for agencies to expend resources on effective training and professional development to equip federal employees to work effectively. Among the resources focus group participants cited as lacking included federal managers and staff with competencies and skills needed to plan strategically, develop robust measures of performance, and analyze what the performance data mean. Our 2003 Guide calls for training and development efforts to be strategically focused on improving performance toward the agency’s goals and put forward with the agency’s organizational culture firmly in mind. Throughout this process it is important that top leaders in the agencies communicate that investments in training and development are expected to produce clearly identified results. By incorporating valid measures of effectiveness into the training and development programs they offer, agencies can better ensure that they will adequately address training objectives and thereby increase the likelihood that desired changes will occur in the target population’s skills, knowledge, abilities, attitudes, or behaviors. Furthermore, if managers understand and support the objectives of training and development efforts, they can provide opportunities to successfully use the new skills and competencies on the job and model the behavior they expect to see in their employees.

In response to our 2003 survey, fewer than half of managers answered “yes” when we asked them whether, during the past 3 years, their agencies had provided, arranged, or paid for training that would help them accomplish any of seven critical results-oriented management-related tasks. However, progress is indicated in our survey results. As shown in figure 16, more managers answered “yes” in 2003 on all seven training areas than in previous surveys. These increases were statistically significant for five of the tasks—setting program performance goals, using program performance information to make management decisions, linking program performance to the achievement of agency strategic goals, and implementing the requirements of GPRA.

As with our 2000 survey results, the 2003 survey results continued to demonstrate that there is a positive relationship between agencies providing training and development on setting program performance goals and the use of performance information when setting or revising performance goals. For those managers who responded “yes” to training on setting performance goals, 60 percent also reported that they used information obtained from performance measurement when setting new or revising existing performance goals to a great or very great extent. In contrast, for those managers who responded “no” to training on setting performance goals, only 38 percent reported that they used information obtained from performance measurement for setting new or revising existing performance goals to a great or very great extent. The difference between these percentages is statistically significant. Effective training and development programs are an integral part of a learning environment that can enhance the federal government’s ability to attract and retain employees with the skills and competencies needed to achieve results. Training and developing new and current staff to fill new roles and work in different ways will be a crucial part of the federal government’s endeavors to meet its transformation challenges. Ways that employees learn and achieve results will also continue to transform how agencies do business and engage employees in further innovation and improvements.

Managers Perceive a Lack of Positive Recognition for Helping Agencies Achieve Results

Another fundamental aspect of the human capital management challenge agencies face is providing the incentives to their employees to encourage results-oriented management. Monetary and nonmonetary incentives can be used as a method for federal agencies to reward employees and to motivate them to focus on results.

Overall, an increasing but still small percentage of managers reported in 1997, 2000, and 2003 that employees in their agencies received positive recognition to a great or very great extent for helping agencies accomplish their strategic goals. In 1997, 26 percent of federal managers reported such an extent of positive recognition as compared to 37 percent in 2003, a statistically significant increase. Interestingly, this improvement is seen in the responses of non-SES managers. As shown in figure 17, the percentage of SES managers expressing this view stayed at about the same level over the three surveys, while the percentage of non-SES managers holding this view was significantly higher in 2003 than in 1997. Even with this improvement on the part of the responses from non-SES managers, significantly more SES managers (47 percent) than non-SES managers (36 percent) expressed this perception to a comparable extent in 2003.

Unfortunately, most existing federal performance appraisal systems are not designed to support a meaningful performance-based pay system in that they fail to link institutional, program, unit, and individual performance measurement and reward systems. In our view, one key need is to modernize performance management systems in executive agencies so that they link to the agency’s strategic plan, related goals, and desired outcomes and are therefore capable of adequately supporting more performance- based pay and other personnel decisions.

We have reported federal agencies can develop effective performance management systems by implementing a selected, generally consistent set of key practices. These key practices helped public sector organizations both in the United States and abroad create a clear linkage—“line of sight”—between individual performance and organizational success and, thus, transform their cultures to be more results oriented, customer- focused, and collaborative in nature. Examples of such practices include aligning individual performance expectations with organizational goals, connecting performance expectations to crosscutting goals, linking pay to individual and organizational performance, and making meaningful distinctions in performance.

Beyond implementing these key practices, high-performing organizations understand that their employees are assets whose value to the organization must be recognized, understood, and enhanced. They view an effective performance management system as an investment to maximize the effectiveness of people by developing individual potential to contribute to organizational goals. To maximize this investment, an organization’s performance management system is designed, implemented, and continuously assessed by the standard of how well it helps the employees help the organization achieve results and pursue its mission.

Persistent Challenges in Setting Outcome- Oriented Goals, Measuring Performance, and Collecting Useful Data

In prior reports, we have described difficulties faced by federal managers in developing useful, outcome-oriented measures of performance and collecting data indicating progress achieved. One of the most persistent challenges has been the development of outcome-oriented performance measures. Additionally, it is difficult to distinguish the impact of a particular federal program from the impact of other programs and factors, thus making it difficult to attribute specific program performance to results. The lack of timely and useful performance information can also hinder GPRA implementation.

Meaningful, Outcome- Oriented Performance Measures Are Sometimes Hard to Develop

In the past, we have noted that federal managers found meaningful performance measures difficult to develop. Focus group participants and survey respondents noted that outcome-oriented performance measures were especially difficult to establish when the program or line of effort was not easily quantifiable. The challenge of the “complexity of establishing outcome-oriented goals and measuring performance” was cited by six of the eight focus groups as one of the key challenges that managers face in implementing GPRA. Focus group participants agreed that they often felt as if they were trying to measure the immeasurable, not having a clear understanding of which performance indicators could accurately inform the agency how it is carrying out a specific activity. Managers from agencies engaged in basic science research and development and grant- making functions noted that this effort was particularly difficult for them because federal programs, especially those that are research-based, often take years to achieve the full scope of their goals. On our most recent survey, we estimated that 36 percent of federal managers who had an opinion indicated that the determination of meaningful measures hindered the use of performance information or performance measurement to a great or very great extent. While this number was significantly lower than the percentage of managers expressing the comparable view on the 1997 or 2000 survey and may reflect some lessening of this as a hindrance to some managers, it nonetheless continues to be among those items having the largest percentage of managers citing it as a substantial hindrance.

Impact of Federal Programs Difficult to Discern

In our June 1997 report on GPRA, we noted that “the often limited or indirect influence that the federal government has in determining whether a desired result is achieved complicates the effort to identify and measure the discrete contribution of the federal initiative to a specific program result.” This occurs primarily because many federal programs’ objectives are the result of complex systems or phenomena outside the program’s control. In such cases, it is particularly challenging for agencies to confidently attribute changes in outcomes to their program—the central task of program impact evaluation. This is particularly challenging for regulatory programs, scientific research programs, and programs that deliver services to taxpayers through third parties, such as state and local governments.

We have reported that determining the specific outcomes resulting from federal research and development has been a challenge that will not be easily resolved. Due to the difficulties in identifying outcomes, research and development agencies typically have chosen to measure a variety of proxies for outcomes, such as the number of patents resulting from federally funded research, expert review and judgments of the quality and importance of research findings, the number of project-related publications or citations, and contributions to expanding the number of research scientists.

We have also reported that implementing GPRA in a regulatory environment is particularly challenging. Although federal agencies are generally required to assess the potential benefits and costs of proposed major regulatory actions, they generally do not monitor the benefits and costs of how these and other federal programs have actually performed. For example, in the case of the Environmental Protection Agency (EPA), to determine if existing environmental regulations need to be retained or improved, we previously recommended that EPA study the actual costs and benefits of such regulations.

In the past, regulatory agencies have cited numerous barriers to their efforts to establish results-oriented goals and measures. These barriers included problems in obtaining data to demonstrate results, accounting for factors outside of the agency’s control that affect results, and dealing with the long time periods often needed to see results. Our prior work discussed best practices for addressing challenges to measuring the results of regulatory programs. In particular, to address the challenge of discerning the impact of a federal program, when other factors also affect results, we suggested agencies “establish a rationale of how the program delivers results.” Establishing such a rationale involves three related practices: (1) taking a holistic or “systems” approach to the problem being addressed, (2) building a program logic model that described how activities translated to outcomes, and (3) expanding program assessments and evaluations to validate the model linkages and rationale.

We have also reported on the difficulties encountered in meeting GPRA reporting requirements for intergovernmental grant programs. Programs that do not deliver a readily measurable product or service are likely to have difficulty meeting GPRA performance measurement and reporting requirements. Intergovernmental grant programs, particularly those with the flexibility inherent in classic block grant design, may be more likely to have difficulty producing performance measures at the national level and raise delicate issues of accountability. Although most flexible grant programs we reviewed reported simple activity or client counts, relatively few of them collected uniform data on the outcomes of state or local service activities. Collecting such data requires conditions (such as uniformity of activities, objectives, and measures) that do not exist under many flexible program designs, and even where overall performance of a state or local program can be measured, the amount attributable to federal funding often cannot be separated out.

Focus group participants also suggested that they faced challenges in obtaining timely performance data from relevant partner organizations and in identifying what the federal government’s contribution has been to a specific outcome. Furthermore, survey respondents provided some corroboration for these views. Across all three of our surveys, we estimate that roughly a quarter of all federal managers reported this difficulty— distinguishing between the results produced by the program they were involved with and results caused by other factors—as a substantial hindrance. In response to a survey question about what the federal government could do to improve its overall focus on managing for results, one respondent noted: “Defining meaningful measures for the work we do is extremely difficult; and even if they could be defined, performance and accomplishment is (sic) dependent on so many factors outside our control that it is difficult, if not impossible, to make valid conclusions.”

Timely, Useful Performance Information Not Always Available

In February 2000, we reported that intergovernmental programs pose potential difficulties in collecting timely and consistent national data. We also noted that agencies had limited program evaluation capabilities and weaknesses in agencies’ financial management capabilities make it difficult for decision makers to effectively assess and improve many agencies’ financial performance. On the basis of our current findings, these issues still exist. Federal managers who participated in our focus groups cited difficulties in gathering data from state or local entities, as well as statutory limitations regarding the nature and breadth of data that they were permitted to collect. However, in our 2003 survey, only 27 percent of federal managers indicated that obtaining data in time to be useful was a substantial hindrance; 31 percent expressed a comparable view with regard to obtaining valid or reliable data.

Focus group participants also noted that OMB’s accelerated time frames for reporting performance information will contribute to the challenge of producing complete, timely information in their agencies’ performance reports. Over the past 2 fiscal years, OMB has moved the deadline for submission of agencies’ performance reports (now performance and accountability reports) back from the statutory requirement of March 31; for fiscal year 2003 data, the deadline is January 30, 2004. In fiscal year 2004, these reports will be due on November 15, 2004. According to the managers, individual agencies may work on different time frames based partially on the population they serve or the stakeholders they must work with, such as state or local agencies. This “one size fits all” approach does not take such differences into account.

Additionally, OMB requires agencies to report on their performance data quarterly; managers noted that this was particularly difficult for outcomes that may be achieved over extended periods of time, such as outcomes associated with basic science. As we have previously reported, measuring the performance of science-related projects can be difficult because a wide range of factors determine if and how a particular research and development project will result in a commercial application or have other benefits. Efforts to cure diseases or pursue space exploration are difficult to quantify and break down into meaningful quarterly performance measures.

Crosscutting Issues Hinder Successful GPRA Implementation

Crosscutting issues continue to be a challenge to GPRA implementation. Mission fragmentation and program overlap are widespread across the federal government. Moreover, addressing this challenge is essential to the success of national strategies in areas such as homeland security, drug control, and the environment.

We have reported that agencies could use the annual performance planning cycle and subsequent annual performance reports to highlight crosscutting program efforts and to provide evidence of the coordination of those efforts. Our review of six agencies’ strategic and annual performance plans showed some improvement in addressing their crosscutting program efforts, but a great deal of improvement is still necessary. Few of the plans we reviewed attempted the more challenging task of discussing planned strategies for coordination and establishing complementary performance goals and complementary or common performance measures. For example, SSA’s 2004 performance plan makes some mention of the agency’s efforts to coordinate with other agencies to preserve the integrity of the Social Security number as a personal identifier, but there are very few details about this important component of its mission.

Previous GAO reports and agency managers identified several barriers to interagency coordination. First, missions may not be mutually reinforcing or may even conflict, making reaching a consensus on strategies and priorities difficult. In 1998 and 1999, we found that mission fragmentation and program overlap existed in 12 federal mission areas, ranging from agriculture to natural resources and the environment. Implementation of federal crosscutting programs is often characterized by numerous individual agency efforts that are implemented with little apparent regard for the presence of related activities. Second, we reported on agencies’ interest in protecting jurisdiction over missions and control over resources. Focus group participants echoed this concern, noting that there can be “turf battles” between agencies, where jurisdictional boundaries, as well as control over resources, are hotly contested. Finally, incompatible procedures, processes, data, and computer systems pose difficulties for agencies to work across agency boundaries. For example, we reported how the lack of consistent data on federal wetlands programs implemented by different agencies prevented the government from measuring progress toward achieving the governmentwide goal of no net loss of the nation’s wetlands.

We have previously reported and testified that GPRA could provide OMB, agencies, and Congress with a structured framework for addressing crosscutting program efforts. OMB, for example, could use the provision of GPRA that calls for OMB to develop a governmentwide performance plan to integrate expected agency-level performance. Unfortunately, this provision has not been fully implemented. OMB issued the first and only such plan in February 1998 for fiscal year 1999. In our review of the plan, we found that it included a broad range of governmentwide management objectives and a mission-based presentation of key performance goals based on agency performance plans and the plan’s framework should ultimately allow for a cohesive presentation of governmentwide performance. However, the specific contents of this initial plan did not always deliver an integrated, consistent, and results-oriented picture of fiscal year 1999 federal government performance goals.

OMB officials we interviewed at the time stressed that developing the governmentwide plan was viewed as an essential and integral component of the President’s budget and planning process. From OMB’s perspective, both the plan and the budget submission were intended to serve as communication tools for a range of possible users. In their opinion, the plan added value by reflecting a governmentwide perspective on policy choices made throughout the budget formulation process. OMB acknowledged that the plan itself did not serve to change the process through which decisions on government priorities were made, but enhanced it by placing a greater emphasis on results. As one official described it, the governmentwide performance plan was a derivative document, reflecting the budget and management decisions made throughout the process of formulating the President’s budget submission. However, we found that focusing broadly on governmentwide outcomes should be a central and distinguishing feature of the federal government performance plan. To be most effective and supportive of the purposes of GPRA, the governmentwide plan must be more than a compilation of agency-level plans; integration, rather than repetition, must be its guiding principle.

OMB has not issued a distinct governmentwide performance plan since fiscal year 1999. Most recently, the President’s fiscal year 2004 budget focused on describing agencies’ progress in addressing the PMA and the results of PART reviews of agency programs. Although such information is important and useful, it does not provide a broader and more integrated perspective of planned performance on governmentwide outcomes. Additionally, the fiscal year 2004 budget identified budget requests and performance objectives by agency, such as the U.S. Department of Defense, as opposed to crosscutting governmentwide themes. From this presentation, one could assume that the only activities the U.S. government planned to carry out in support of national defense were those listed under the chapter “Department of Defense.” However, the chapter of the fiscal year 2004 budget discussing “the Department of State and International Assistance Programs,” contains a heading titled, “Countering the Threat from Weapons of Mass Destruction.” And while OMB may have a technical reason for not classifying this task as being related to national defense or homeland security, it is unclear that a lay reader could make that distinction. The fiscal year 2005 budget also identified budget requests by agency, not by crosscutting theme. Without such a governmentwide focus, OMB is missing an opportunity to assess and communicate the relationship between individual agency goals and outcomes that cut across federal agencies and more clearly relate and address the contributions of alternative federal strategies. The governmentwide performance plan also could help Congress and the executive branch address critical federal performance and management issues, including redundancy and other inefficiencies in how we do business. It could also provide a framework for any restructuring efforts.

A strategic plan for the federal government, supported by key national indicators to assess the government’s performance, position, and progress, could provide an additional tool for governmentwide reexamination of existing programs, as well as proposals for new programs. If fully developed, a governmentwide strategic plan could potentially provide a cohesive perspective on the long-term goals of the federal government and provide a much needed basis for fully integrating, rather than merely coordinating, a wide array of federal activities. Successful strategic planning requires the involvement of key stakeholders. Thus, it could serve as a mechanism for building consensus. Further, it could provide a vehicle for the President to articulate long-term goals and a road map for achieving them. In addition, a strategic plan could provide a more comprehensive framework for considering organizational changes and making resource decisions.

Developing a strategic plan for the federal government would be an important first step in articulating the role, goals, and objectives of the federal government. It could help provide critical horizontal and vertical linkages. Horizontally, it could integrate and foster synergies among components of the federal government as well as help to clarify the role of the federal government vis-à-vis other sectors of our society. Vertically, it could provide a framework of federal missions and goals within which individual federal agencies could align their own missions and goals that would cascade down to individual employees. The development of a set of key national indicators could be used as a basis to inform the development of the governmentwide strategic and annual performance plans. The indicators could also link to and provide information to support outcome- oriented goals and objectives in agency-level strategic and annual performance plans.

Managers View Congress’ Use of Performance Information as Limited

Focus group members believed that one of the main challenges to GPRA implementation was the reluctance of Congress to use that information when making decisions, especially appropriations decisions. This concern was cited as a significant challenge in each of the focus groups, and was one of the top three “challenges” in five of the eight focus groups. In some cases, managers in our focus groups noted that this lack of usage was a significant disincentive to doing a good job in preparing GPRA plans and reports. Agency managers made the following criticisms regarding the perceived lack of congressional use of performance information: appropriators have not bought into GPRA, so there is no incentive to do failure of congressional leadership in developing and using performance appropriators do not use performance data or tools to make decisions, GPRA does not drive public policy decisions.

Results from our survey provide some further information in support of this view. On our 2003 survey, when we asked federal managers about the extent to which they thought congressional committees paid attention to agency efforts under GPRA, only 22 percent of federal managers responded in the great to very great categories. This result was not significantly different from the results we observed on our 2000 survey when we asked this question about three specific types of congressional committees— authorization, appropriation, and oversight. On the 2000 survey, only 18 percent of federal managers held a similar view concerning authorizing committees, 19 percent for appropriations committees, and 20 percent for oversight committees. As we noted earlier, when this item was asked in relation to OMB, there was a significant increase in the percentage of managers responding to a great or very great extent from 2000 to 2003. The 31 percent of managers who viewed OMB as paying attention to a great or very great extent in 2003 was significantly higher than the 22 percent holding a comparable view of congressional committees.

Although managers expressed these concerns about the use of this information, a recent review by the CRS suggested that Congress uses performance information to some extent, as evidenced by citations in legislation and committee reports. For example, in the 106th Congress (1999-2000), 42 public laws contained statutory language relating to GPRA and performance measures, and 118 legislative reports contained GPRA- associated passages. As shown in figure 18, across all three of our surveys, only a minority of federal managers governmentwide viewed the lack of ongoing congressional commitment for using performance information as a hindrance to a great or very great extent.

While there is concern regarding Congress’ use of performance information, it is important to make sure that this information is initially useful. One of GPRA’s purposes is to respond to a need for accurate, reliable information for congressional decision making. In 2000, we reported that congressional staffs stated that they were looking for recurring information on spending priorities within programs; the quality, quantity, and efficiency of program operations; the populations served or regulated; as well as programs’ progress in meeting their objectives. For example, learning who benefits from a program can help in addressing questions about how well services are targeted to those most in need. Some of these recurring needs were met through formal agency documents, such as annual performance plans. However, some information the agencies provided did not fully meet the congressional staffs’ needs because the presentation was not clear, directly relevant, or sufficiently detailed. For example, congressional staffs wanted to see more direct linkages among the agencies’ resources, strategies, and goals. In other cases, the information was not readily available to the congressional staffs, either because it had not been requested or reported, or because staff were not informed that it was available.

As a key user of performance information, Congress also needs to be considered a partner in shaping agency goals at the outset. For example, through the strategic planning requirement, GPRA requires federal agencies to consult with Congress and key stakeholders to reassess their missions and long-term goals as well as the strategies and resources they will need to achieve their goals. GPRA also provides a vehicle for Congress to explicitly state its performance expectations in outcome-oriented terms when establishing new programs or in exercising oversight of existing programs that are not achieving desired results. Congress could use authorizing and appropriations hearings to determine if agency programs have clear performance goals, measures, and data with which to track progress and whether the programs are achieving their goals. If goals and objectives are unclear or not results oriented, Congress could use legislation to articulate the program outcomes it expects agencies to achieve. This would provide important guidance to agencies that could then be incorporated in agency strategic and annual performance plans.

Conclusions and Recommendations

Agenda for Achieving a Sustainable, Governmentwide Focus on Results

As we have shown in this report, in the 10 years since the enactment of GPRA, significant progress has been made in instilling a focus on results in the federal government. First, GPRA statutory requirements laid a foundation for results-oriented management in federal agencies. Expert and agency focus group participants cited the creation of this statutory foundation as one of the key accomplishments of GPRA. Since GPRA began to be implemented governmentwide in fiscal year 1997, we have observed significant increases in the percentage of federal managers who reported having results-oriented performance measures for their programs. Focus group participants’ views on whether GPRA has had a positive effect on the federal government’s ability to deliver results to the American public were mixed. For example, the information gathered and reported for GPRA allows agencies to make better-informed decisions, which improves their ability to achieve results. In addition, GPRA has made the results of federal programs more transparent to the public. Other participants stated that while certain aspects of GPRA-related work have been positive, agencies’ ability to deliver results and public awareness of their activities cannot be exclusively attributed to GPRA.

Second, GPRA has increased the connection between resources and results by creating more formal linkages between agency performance goals and objectives and the program activities in the budget. Over the first 4 years of agency efforts to implement GPRA, we observed that agencies continued to tighten the required linkage between their performance plans and budget requests. However, much remains to be done in this area. For example, we have not observed notable increases in federal managers’ perceptions about their personal use of plans or performance information when allocating resources, or about the use of performance information when funding decisions are made about their programs. However, it should be noted that we estimate a majority have positive perceptions about the use of performance information to allocate resources.

Third, GPRA has provided a foundation for examining agency missions, performance goals and objectives, and the results achieved. We have seen improvements in the quality of agency strategic plans, annual performance plans, and performance reports since initial efforts. However, few of the six agencies we reviewed in this report produced GPRA planning and reporting documents that met all of our criteria for the highest level of quality. Most of these agencies continued to miss opportunities to present clear pictures of their intended and actual performance results in their GPRA plans and reports and to show how resources are aligned with actual performance results. Furthermore, most of the agencies we reviewed did not provide a full level of confidence in the credibility of their performance data.

Performance-based management, as envisioned by GPRA, requires transforming organizational cultures to improve decision making, maximize performance, and assure accountability. This transformation is not an easy one and requires investments of time and resources as well as sustained leadership commitment and attention. Challenges to successful implementation of GPRA include inconsistent top leadership commitment to creating a focus on results; an approach to setting goals and developing strategies for achieving critical outcomes that creates individual agency stovepipes rather than an integrated, holistic governmentwide approach; getting federal managers to make greater use of performance information to manage their programs and providing them authority to act that is commensurate with their accountability for results; difficulty in establishing meaningful measures of outcomes and assessing results of federal programs that are carried out by nonfederal entities; and untimely performance data.

The challenges identified in this report are not new—most have not changed significantly since we first reported on governmentwide implementation of GPRA. However, we have frequently reported on approaches that agencies, OMB, and Congress could use to address the challenges. These approaches include strengthening the commitment of top leadership to creating and sustaining a focus on results; taking a governmentwide approach to achieving outcomes that are crosscutting in nature; improving the usefulness of performance information to managers, Congress, and the public; and improving the quality of performance measures and data. Collectively, these approaches form the agenda that federal agencies, OMB, and Congress will need to follow to bring about a more sustainable, governmentwide focus on results.

Strengthening Top Leadership Commitment to Creating and Sustaining Results-Oriented Cultures

Successfully addressing the challenges that federal agencies face requires leaders who are committed to achieving results, who recognize the importance of using results-oriented goals and quantifiable measures, and who integrate performance-based management into the culture and day-to- day activities of their organizations. Top leadership must play a critical role in creating and sustaining high-performing organizations. Without the clear and demonstrated commitment of agency top leadership—both political and career—organizational cultures will not be transformed, and new visions and ways of doing business will not take root.

To be positioned to address the array of challenges faced by our national government, federal agencies will need to transform their organizational cultures so that they are more results oriented, customer-focused, and collaborative. Leading public organizations here in the United States and abroad have found that strategic human capital management must be the centerpiece of any serious change management initiative and efforts to transform the cultures of government agencies. Performance management systems are integral to strategic human capital management. Such systems can be key tools to maximizing performance by aligning institutional performance measures with individual performance and creating a “line of sight” between individual and organizational goals. Leading organizations use their performance management systems as a key tool for aligning institutional, unit, and employee performance; achieving results; accelerating change; managing the organization day to day; and facilitating communication throughout the year so that discussions about individual and organizational performance are integrated and ongoing.

Furthermore, achieving this cultural transformation requires people to have the knowledge and skills to develop and use performance information to improve program performance. Our survey data indicated a significant relationship between those managers who reported they received training on setting performance goals and those who used performance information when setting or revising performance goals. However, federal agencies have not consistently showed a commitment to investing in needed training and development opportunities to help ensure that managers and employees have the requisite skills and competencies to achieve agency goals.

The commitment to focusing on and using performance information needs to extend to OMB and Congress as well. Through the administration’s PMA and PART initiatives, OMB has clearly placed greater emphasis on management issues over the past several years. However, the focus of such oversight needs to extend beyond the emphasis on formulating the President’s Budget to include an examination of the many challenges agencies face that may be contributing to poor performance. In spite of the persistent weaknesses we found in agencies’ strategic plans and annual performance plans and reports, OMB significantly reduced the scope of its guidance to agencies on how to prepare these documents. By emphasizing a focus on resource allocation through its PART exercise and providing less information on how to comply with GPRA, OMB may be sending a message to agencies that compliance with GPRA is not important. Without strong leadership from OMB, the foundation of performance information that has been built could deteriorate.

OMB leadership is critical to addressing the continuing challenges presented in GPRA implementation and the transformation of the federal government to an increasingly results-oriented culture. OMB, as the primary focal point for overall management in the federal government, can provide the needed impetus by providing guidance, fostering communication among agencies, and forming intragovernmental councils and work groups tasked with identifying potential approaches and solutions to overcoming the persistent challenges to results-oriented management.

Congress can also play a decisive role in fostering results-oriented cultures in the federal government by using information on agency goals and results at confirmation, oversight, authorization, and appropriation hearings. Consistent congressional interest in the status of an agency’s GPRA efforts, performance measures, and uses of performance information to make decisions, will send an unmistakable message to agencies that Congress expects GPRA to be thoroughly implemented.

We also found that timing issues may affect the development of agency strategic plans that are meaningful and useful to top leadership. The commitment of top leadership within agencies, OMB, and Congress is critical to the success of strategic planning efforts. A strategic plan should reflect the policy priorities of an organization’s leaders and the input of key stakeholders if it is to be an effective management tool. However, GPRA specifies time frames for updating strategic plans that do not correspond to presidential or congressional terms. As a result, an agency may be required to update its strategic plan a year before a presidential election and without input from a new Congress. If a new president is elected, the updated plan is essentially moot and agencies must spend additional time and effort revising it to reflect new priorities. Our focus group participants, including GPRA experts, strongly agreed that this timing issue should be addressed by adjusting time frames to correspond better with presidential and congressional terms.

Addressing Governmentwide Needs

Mission fragmentation and program overlap are widespread throughout the federal government. We have noted that interagency coordination is important for ensuring that crosscutting program efforts are mutually reinforcing and efficiently implemented. Our review of six agencies’ strategic and annual performance plans along with our previous work on crosscutting issues has demonstrated that agencies’ still present their goals and strategies in a mostly stovepiped manner. They have generally not used their plans to communicate the nature of their coordination with other agencies, in terms of the development of common or complementary goals and objectives or strategies jointly undertaken to achieve those goals.

We have also reported that GPRA could provide a tool to reexamine federal government roles and structures governmentwide. GPRA requires the President to include in his annual budget submission a federal government performance plan. Congress intended that this plan provide a “single cohesive picture of the annual performance goals for the fiscal year.” The governmentwide performance plan could help Congress and the executive branch address critical federal performance and management issues, including redundancy and other inefficiencies in how we do business. It could also provide a framework for any restructuring efforts. Unfortunately, this provision has not been fully implemented. Instead, OMB has used the President’s Budget to present high-level information about agencies and certain program performance issues. The agency-by- agency focus of the budget does not provide the integrated perspective of government performance envisioned by GPRA.

If the governmentwide performance plan were fully implemented, it could provide a framework for such congressional oversight. For example, in recent years, OMB has begun to develop common measures for similar programs, such as job training. By focusing on broad goals and objectives, oversight could more effectively cut across organization, program, and other traditional boundaries. Such oversight might also cut across existing committee boundaries, which suggests that Congress may benefit from using specialized mechanisms to perform oversight (i.e., joint hearings and special committees).

A strategic plan for the federal government, supported by key national indicators to assess the government’s performance, position, and progress, could provide an additional tool for governmentwide reexamination of existing programs, as well as proposals for new programs. If fully developed, a governmentwide strategic plan can potentially provide a cohesive perspective on the long-term goals of the federal government and provide a much needed basis for fully integrating, rather than merely coordinating, a wide array of federal activities. Successful strategic planning requires the involvement of key stakeholders. Thus, it could serve as a mechanism for building consensus. Further, it could provide a vehicle for the President to articulate long-term goals and a road map for achieving them. In addition, a strategic plan can provide a more comprehensive framework for considering organizational changes and making resource decisions.

Developing a strategic plan for the federal government would be an important first step in articulating the role, goals, and objectives of the federal government. It could help provide critical horizontal and vertical linkages. Horizontally, it could integrate and foster synergies among components of the federal government as well as help to clarify the role of the federal government vis-à-vis other sectors of our society. Vertically, it could provide a framework of federal missions and goals within which individual federal agencies could align their own missions and goals that would cascade down to individual employees. The development of a set of key national indicators could be used as a basis to inform the development of governmentwide strategic and annual performance plans. The indicators could also link to and provide information to support outcome- oriented goals and objectives in agency-level strategic and annual performance plans.

Improving Usefulness of Performance Information

We have found that leading organizations that progressed the farthest to results-oriented management did not stop after strategic planning and performance measurement. They applied their acquired knowledge and data to identify gaps in their performance, report on that performance, and finally use that information to improve their performance to better support their missions.

Performance data can have real value only if they are used to identify the gap between an organization’s actual performance level and the performance level it has identified as its goal. Once the performance gaps are identified for different program areas, managers can determine where to target their resources to improve overall mission accomplishment. When managers are forced to reduce their resources, the same analysis can help them target reductions to keep to a minimum the impact on their organization’s overall mission.

Under GPRA, agencies produce a single strategic plan, annual performance plan, and annual performance report. However, there are many potential consumers of agencies’ performance information—Congress, the public, and the agency itself. One size need not fit all. Clearly, an agency will need more detailed information on its programs for operational purposes than would be suitable for external audiences. Of the six agencies’ performance reports we reviewed, some of them provided useful summary tables or information that provided overall snapshots of performance or highlighted progress in achieving key goals. Other reports that lacked such a summary made it difficult to assess the progress achieved.

To improve the prospect that agency performance information will be useful to and used by these different users, agencies need to consider the different information needs and how to best tailor their performance information to meet those needs. For example, we have reported that, although many information needs were met, congressional staff also identified gaps in meeting their information needs. Key to addressing these information gaps was improving communication between congressional staff and agency officials to help ensure that congressional information needs are understood, and that, where feasible, arrangements are made to meet them. Improved two-way communication might also make clear what information is and is not available, as well as what is needed and what is not needed. This might entail the preparation of simplified and streamlined plans and reports for Congress and other external users.

Another challenge that limits the usefulness of agency performance reports is the lack of timely data on performance. For the six performance reports we reviewed we continued to observe a significant number of goals for which performance data were unavailable. Policy decisions made when designing federal programs, particularly intergovernmental programs, may make it difficult to collect timely and consistent national data. In administering programs that are the joint responsibility of state and local governments, Congress and the executive branch continually balance the competing objectives of collecting uniform program information to assess performance with giving states and localities the flexibility needed to effectively implement intergovernmental programs.

Improving Performance Measures and Data Quality

Another key challenge to achieving a governmentwide focus on results is that of developing meaningful, outcome-oriented performance goals and collecting performance data that can be used to assess results. Performance measurement under GPRA is the ongoing monitoring and reporting of program accomplishments, particularly progress toward preestablished goals. It tends to focus on regularly collected data on the level and type of program activities, the direct products and services delivered by the programs, and the results of those activities. For programs that have readily observable results or outcomes, performance measurement may provide sufficient information to demonstrate program results. In some programs, however, outcomes are not quickly achieved or readily observed, or their relationship to the program is uncertain. In such cases, more in-depth program evaluations may be needed, in addition to performance measurement, to examine the extent to which a program is achieving its objectives.

Given the difficult measurement challenges we have identified, it is all the more important that agency strategic planning efforts include the identification of the most critical evaluations that need to take place to address those challenges. However, our previous work has raised concerns about the capacity of federal agencies to produce evaluations of program effectiveness. Few of the agencies we reviewed deployed the rigorous research methods required to attribute changes underlying outcomes to program activities. Yet we have also seen how some agencies have profitably drawn on systematic program evaluations to improve their measurement of program performance or understanding of performance and how it might be improved. Our review of six agencies’ strategic plans and performance reports in this report revealed weaknesses in their discussions of program evaluation. Most of the strategic plans lacked critical information required by GPRA, such as a discussion of how evaluations were used to establish strategic goals or a schedule of future evaluations. Furthermore, two of the six performance reports did not summarize the results of program evaluations completed that year, as required.

Our work has also identified substantial, long-standing limitations in agencies’ abilities to produce credible data and identify performance improvement opportunities that will not be quickly or easily resolved. According to our review, five of six agencies’ annual performance plans showed meaningful improvements in how they discussed the quality of performance data. However, only DOT’s performance plan and report contained information that provided a full level of confidence in the credibility of its performance data. In particular, the plans and reports did not always provide detailed information on how the agencies verified and validated their performance data.

Recommendations for Executive Action

To provide a broader perspective and more cohesive picture of the federal government’s goals and strategies to address issues that cut across executive branch agencies, we recommend that the Director of OMB fully implement GPRA’s requirement to develop a governmentwide performance plan.

To achieve the greatest benefit from both GPRA and PART, we recommend that the Director of OMB articulate and implement an integrated and complementary relationship between the two. GPRA is a broad legislative framework that was designed to be consultative with Congress and other stakeholders, and allows for varying uses of performance information. PART looks through a particular lens for a particular use—the executive budget formulation process.

To improve the quality of agencies’ strategic plans, annual performance plans, and performance reports and help agencies meet the requirements of GPRA, we recommend that the Director of OMB provide clearer and more consistent guidance to executive branch agencies on how to implement GPRA. Such guidance should include standards for communicating key performance information in concise as well as longer formats to better meet the needs of external users who lack the time or expertise to analyze lengthy, detailed documents.

To help address agencies’ performance measurement challenges, we recommend that the Director of OMB engage in a continuing dialogue with agencies about their performance measurement practices with a particular focus on grant-making, research and development, and regulatory functions to identify and replicate successful approaches agencies are using to measure and report on their outcomes, including the use of program evaluation tools. Additionally, we recommend that the Director of OMB work with executive branch agencies to identify the barriers to obtaining timely data to show progress against performance goals and the best ways to report information where there are unavoidable lags in data availability. Interagency councils, such as the President’s Management Council and the Chief Financial Officers’ Council, may be effective vehicles for working on these issues.

To facilitate the transformation of agencies’ management cultures to be more results-oriented, we recommend that the Director of OMB work with agencies to ensure they are making adequate investments in training on performance planning and measurement, with a particular emphasis on how to use performance information to improve program performance.

Matters for Congressional Consideration

To ensure that agency strategic plans more closely align with changes in the federal government leadership, Congress should consider amending GPRA to require that updates to agency strategic plans be submitted at least once every 4 years, 12-18 months after a new administration begins its term. Additionally, consultations with congressional stakeholders should be held at least once every new Congress and interim updates made to strategic and performance plans as warranted. Congress should consider using these consultations along with its traditional oversight role and legislation as opportunities to clarify its performance expectations for agencies. This process may provide an opportunity for Congress to develop a more structured oversight agenda.

To provide a framework to identify long-term goals and strategies to address issues that cut across federal agencies, Congress also should consider amending GPRA to require the President to develop a governmentwide strategic plan.

Agency Comments

We provided a copy of the draft report to OMB for comment. OMB’s written comments are reprinted in appendix VIII. In general, OMB agreed with our findings and conclusions. OMB agreed to implement most of our recommendations, noting that these recommendations will enhance its efforts to make the government more results oriented. OMB agreed to (1) work with agencies to ensure they are provided adequate training in performance management, (2) revise its guidance to clarify the integrated and complementary relationship between GPRA and PART, and (3) continue to use PART to improve agency performance measurement practices and share those practices across government.

In response to our recommendation that OMB fully implement GPRA’s requirement to develop a governmentwide performance plan, OMB stated that the President’s Budget represents the executive branch’s governmentwide performance plan. However, the agency-by-agency focus of the budget over the past few years does not provide an integrated perspective of government performance, and thus does not meet GPRA’s requirement to provide a “single cohesive picture of the annual performance goals for the fiscal year.” In response to our matter for congressional consideration that Congress should consider amending GPRA to require the President to develop a governmentwide strategic plan, OMB noted that the budget serves as the governmentwide strategic plan. However, in our opinion, the President’s Budget focuses on establishing agency budgets for the upcoming fiscal year. Unlike a strategic plan, it provides neither a long-term nor an integrated perspective on the federal government’s activities. A governmentwide strategic plan should provide a cohesive perspective on the long-term goals of the federal government and provide a basis for fully integrating, rather than primarily coordinating, a wide array of federal activities.

We provided relevant sections of the draft report to Education, DOE, HUD, SBA, SSA, and DOT. Education and SBA did not provide any comments, while DOT provided minor technical comments. Written comments from DOE, HUD, and SSA are reprinted in appendixes IX, X, and XI, respectively, along with our responses.

DOE disagreed with portions of our analyses of its 2004 Annual Performance Plan and its 2002 Performance and Accountability Report. Our analysis of DOE’s documents was based on specific criteria (see appendixes IV and V for details) and was characterized in relation to our reviews of the other five agencies’ documents. We modified or clarified certain characterizations in response to DOE comments, but for the most part found that our characterizations were appropriate.

SSA generally agreed with our observations and agreed to incorporate them in its future planning efforts. SSA made several points of clarification and disagreed with our observation that its performance and accountability report does not clearly state how program evaluations were used to answer questions about program performance and results and how they can be improved. SSA noted that its evaluations rely on surveys, and these surveys form the basis for its efforts to deliver high-quality service. SSA also noted that it lists other evaluations that are of great importance to its ongoing operations. We do not discount the usefulness of SSA’s surveys in assessing its day-to-day management of programs. Rather, we believe that it would be helpful for SSA to clearly identify the range of evaluations conducted and how each of them contributed to improved program performance.

HUD noted that all of the areas we suggested for further improvement are under consideration for improvement. However, they disagreed with us on two observations related to the strategic plan: (1) that the link between its long-term and intermediate goals is difficult to discern and (2) that it did not explain how it used the results of program evaluations to update the current plan and did not include a schedule for future evaluations. On the basis of OMB guidance for preparing strategic plans and the criteria we used to evaluate all six agencies’ strategic plans (see app. III for more detail), we maintain that these two observations are valid and require further attention. HUD also disagreed with how we presented the performance information in its summary report cards (see fig. 22). HUD noted that many of the results were explained in the individual indicator write-ups that followed the summary information. Our analysis of HUD’s information included qualitative aspects of how the information was presented, such as its usefulness to inform the average reader with little or no exposure to the subject matter, and the extent to which HUD presented a complete summary of performance information in a user-friendly format.

Technical comments from DOE, HUD, and SSA were incorporated, as appropriate.