Managing For Results: Critical Issues for Improving Federal Agencies' Strategic Plans
Released: 1997-10-16T00:00:00

Letter

Scope and Methodology

This summary report is based on our analysis of the information contained in our reviews of 27 agencies’ draft strategic plans. To do those 27 reviews and the related reports, we used the Results Act supplemented by OMB’s guidance on developing the plans (Circular A-11, part 2) as criteria to determine whether draft plans complied with the requirement for the six specific elements that are to be in the strategic plans. To make judgments about the overall quality of the plans, we used our May 1997 guidance for congressional review of the plans. We recognized in each instance that the plans were drafts and that our assessment thus represented a snapshot at a given point in time. To make judgments about the planning issues needing attention, we also relied on other related work, including our recent report on governmentwide implementation of the Results Act and our guidance for congressional review of Results Act implementation, as tools.

Background

The Results Act is the centerpiece of a statutory framework Congress put in place during the 1990s to address long-standing weaknesses in federal operations, improve federal management practices, and provide greater accountability for achieving results. Under the Results Act, strategic plans are the starting point and basic underpinning for results-oriented management. The Act requires that an agency’s strategic plan contain six key elements: (1) a comprehensive agency mission statement; (2) agencywide long-term goals and objectives for all major functions and operations; (3) approaches (or strategies) and the various resources needed to achieve the goals and objectives; (4) a description of the relationship between the long-term goals and objectives and the annual performance goals; (5) an identification of key factors, external to the agency and beyond its control, that could significantly affect the achievement of the strategic goals; and (6) a description of how program evaluations were used to establish or revise strategic goals and a schedule for future program evaluations.

In addition to the Results Act, the statutory framework includes the Chief Financial Officers (CFO) Act, as expanded and amended by the Government Management Reform Act of 1994; and information technology reform legislation, in particular the Clinger-Cohen Act of 1996 and the Paperwork Reduction Act of 1995. Congress enacted the CFO Act to remedy decades of serious neglect in federal financial management by establishing chief financial officers across the federal government and requiring the preparation and audit of annual financial statements. The information technology reform legislation is based on the best practices used by leading public and private sector organizations to manage information technology more effectively. Under the information technology reform legislation, agencies are to better link their planned and actual use of technology to their programs’ missions and goals to improve performance.

Congress has demonstrated its commitment to the Results Act and reinforced to executive agencies the importance it places on the full and complete implementation of the Act. One such prominent demonstration occurred on February 25, 1997, when the Speaker of the House, the Majority Leader of the Senate, and other senior members of the House and Senate sent a letter to the Director of OMB. The letter underscored the importance that the congressional Majority places on the implementation of the Results Act, noted a willingness on the part of Congress to work cooperatively with the administration, and established expectations for congressional consultations with agencies on their draft strategic plans.

Under the Results Act, those consultations are to be an integral part of strategic planning. For example, consultations can help to create a basic understanding among the stakeholders of the competing demands that confront agencies and how those demands and available resources require careful and continuous balancing. In the House, the consultation effort was led by teams consisting of staff from various committees that focused on specific agencies. In an August 1997 letter, the House Majority Leader provided the Director of OMB with an overview of recent congressional consultations and highlighted some recurring themes, such as the need for interagency coordination. Although the consultation process in the Senate has been less structured than the one in the House, a number of consultations have been held there as well.

In addition to consulting with agencies, several House and Senate authorizing committees also held hearings on draft strategic plans in July 1997, which further underscored congressional interest in agencies creating good strategic planning processes that support performance-based management. For example, these hearings included those held by the Subcommittee on Water and Power, House Committee on Resources; the House Committee on Science; the Subcommittee on Human Resources, House Committee on Government Reform and Oversight; the House Committee on Banking and Financial Services; the Subcommittee on Forests and Forest Health, House Committee on Resources; and the Senate Committee on Indian Affairs.

The Senate and House appropriations committees have been expanding their focus on the Results Act as well. For example, the Senate Appropriations and Governmental Affairs committees held a joint hearing on the status of Results Act implementation, with particular emphasis on agencies’ strategic planning efforts. The Senate Appropriations Committee has included comments on the Results Act in its reports on the fiscal year 1998 appropriations bills. The report language has discussed the Committee’s views on the status and quality of individual agencies’ efforts to implement the Results Act and expressed the need for continuing consultations, among other issues. In the House, the Appropriations Committee has included a standard statement in its appropriations reports that strongly endorses the Results Act. This standard statement notes that each appropriations subcommittee “takes (the annual performance plan) requirement of the Results Act very seriously and plans to carefully examine agency performance goals and measures during the appropriations process.”

Most Plans Lacked Some Required Elements

A significant amount of work remains to be done by executive branch agencies before their strategic plans can fulfill the requirements of the Results Act, serve as a basis for guiding agencies, and help congressional and other policymakers make decisions about activities and programs. Although all 27 of the draft plans included a mission statement, 21 plans lacked 1 or more of the other required elements. Specifically, of the 27 draft strategic plans:

2 did not include agencywide strategic goals and objectives,

6 did not describe approaches or strategies for achieving those goals and

19 did not describe the relationship between long-term goals and objectives and annual performance goals,

6 did not identify key factors that are external to the agency and beyond its control that could affect the achievement of the goals and objectives, and

16 did not discuss program evaluations that the agency used to establish or revise goals and objectives or provide a schedule of future program evaluations.

Moreover, while all but six of the plans were missing at least one required element, one-third were missing two required elements. Just over one-fourth of the plans failed to cover at least three of the required elements.

Because most of the draft plans did not contain all six required elements, Congress did not have access to critical pieces of information for its consultations with the agencies on their draft strategic plans; and, if these elements are not included in the final plans, federal managers will not have a clear strategic direction upon which to base their daily activities. For example, agencies whose plans lacked strategic goals and strategies for achieving those goals will not have a solid foundation upon which to build the performance measurement and reporting efforts that are required by the Results Act. The incomplete or inadequate coverage of the six required elements in the plans is an indication of the amount of additional work necessary to fulfill the Act’s minimum requirements that agencies had to undertake prior to the submission of strategic plans to Congress on September 30, 1997.

Critical Strategic Planning Issues Most in Need of Sustained Attention

Many agencies showed progress in developing comprehensive mission statements upon which they can build strategic goals and strategies for achieving those goals. A mission statement is important because it focuses an agency on its intended purpose. It explains why the agency exists and tells what it does and is the basic starting point of successful planning efforts. However, our reviews of draft strategic plans for 27 agencies found several critical strategic planning issues that are in need of sustained attention to ensure that those plans better meet the needs of agencies, Congress, and other stakeholders and that agencies shift their focus from activities to results. These issues were: the lack of linkages among required elements in the draft plans, the weaknesses in long-term strategic goals, the lack of fully developed strategies to achieve the goals, the lack of evidence that agencies’ plans reflect coordination with other federal agencies having similar or complementary programs, the limited capacity of agencies to gather performance information, and the lack of attention to program evaluations.

Most Plans Lacked Critical Linkages

The majority of the draft strategic plans lacked critical linkages among required elements in the plans. We have noted in our Executive Guide and other recent reports that for strategic plans to drive an agency’s operations, a straightforward linkage is needed among its long-term strategic goals, strategies for achieving goals, annual performance goals, and day-to-day activities. First, as prior work has shown, a direct alignment between strategic goals and strategies for achieving those goals is important for assessing an agency’s ability to achieve those goals. Second, we have noted that the linkage between long-term strategic goals and annual performance goals is important because without this linkage, agency managers and Congress may not be able to judge whether an agency is making annual progress toward achieving its long-term goals.

Linkage Between Goals and Strategies

In several draft strategic plans, the agencies’ presentation of information on strategic goals, objectives, and strategies made it difficult to determine which strategy was supposed to achieve which goal or objective and what unit or component within the agency was supposed to carry out the strategy. For example, in the Small Business Administration’s (SBA) plan, objectives were listed as a group under goals, followed by strategies, which were also listed as a group. This presentation does not convey how specific strategies would lead to achieving specific goals. In another example, the Federal Emergency Management Agency (FEMA) listed several areas of focus and operational objectives under each of its five strategies, but if did not establish linkages among them or between the strategies and the agency’s strategic objectives. Accordingly, although an affiliation between specific strategies and objectives may exist, it was not readily apparent from these agencies’ draft strategic plans.

In contrast, the Department of Education’s plan linked each strategic goal to a set of objectives that were, in turn, linked to a set of strategies. For example, the strategic goal to “build a solid foundation for learning” had as one of its objectives, “every eighth grader masters challenging mathematics, including the foundations of algebra and geometry.” Two of the strategies listed under this objective were to develop and use a national, voluntary test in mathematics as a means to encourage schools, school districts, states, businesses, and communities to move toward improving math curricula and instruction, among other things; and to increase public understanding and support of mastering mathematics by the end of eighth grade through partnerships with key education, mathematics, and professional organizations.

Linkage Between Strategic Goals and Performance Goals

As noted in our section on required elements, 19 of 27 draft plans did not describe the linkages between long-term strategic goals and annual performance goals. As we have reported, without this linkage, it may not be possible to determine whether an agency has a clear sense of how it will assess the progress made toward achieving its intended results. However, some agencies made good attempts at providing this linkage in their draft plans. For example, the Department of Education, the General Services Administration (GSA), and the Postal Service used a matrix to illustrate the linkages among their strategic goals, objectives, and the measures that are to be reflected in their annual performance goals.

Linkage Between Component Goals and Agencywide Goals

Our work on the draft plans found that clearly aligning required strategic planning elements is especially important in those cases where agencies, as allowed under OMB guidance, chose to submit a strategic plan for each of their major components and a strategic overview that under the guidance is to show the linkages among these plans, instead of a single agencywide plan. A few agencies, including the Departments of Agriculture (USDA), Labor, and the Interior, used this approach. USDA, Labor, and Interior are large agencies with disparate functions that are implemented by a number of subagencies. For example, USDA has 18 subagencies working in 7 different mission areas, such as farm and foreign agricultural services and food safety and inspection service. None of the three agencies adequately linked component-level goals to the agencywide strategic goals. For example, their plans did not consistently demonstrate how the components’ goals and objectives would contribute to the achievement of agencywide goals. Furthermore, Labor’s overview plan did not contain agencywide goals, even though the Secretary set forth agencywide goals in recent congressional testimony.

Strategic Goals Often Tended to Have Weaknesses

Leading organizations we have studied set long-term strategic goals that were an outgrowth of a clearly stated mission. Setting long-term strategic goals is essential for results-oriented management, because such goals explain in greater specificity the results organizations are intending to achieve. The goals form a basis for an organization to identify potential strategies for fulfilling its mission and for improving its operations to support achievement of that mission. Congress recognized both the importance and difficulty of setting results-oriented strategic goals. Under the Results Act, all of an agency’s strategic goals do not need to be explicitly results oriented, although the intent of the Act is to have agencies focus on results to the extent feasible.

Although most agencies attempted to articulate agencywide strategic goals and objectives in their plans, many of those goals and objectives tended to be weak. We often found that the draft plans contained goals and objectives that were not as results oriented as they could have been. For example, one of the Department of Veterans Affair’s (VA) goals, to “improve benefit programs,” could be more results oriented if VA identified the purpose of the benefit programs (e.g., to ease veterans’ transition to civilian life). In contrast, GSA’s goals and objectives reflect a positive attempt to define the results that it expects from its major functions. For example, one of the goals in the draft strategic plan states that GSA will become the space/supplies/telecommunications provider of choice for all federal agencies by delivering quality products and services at the best value.

In several plans, agencies expressed goals and objectives in a manner that would make them difficult to measure or difficult to assess in the future. Although strategic goals need not be expressed in a measurable form, OMB guidance says goals must be expressed in a manner that allows for future assessment of whether they are being achieved. One example of an objective that was not measurable as written is the Social Security Administration’s (SSA) goal “to promote valued, strong, and responsive social security programs through effective policy development and research.” This goal recognized that program leadership cannot be achieved without a strong policy and research capability—the lack of which we have criticized SSA for in the past. Yet, the goal itself and the supporting discussion in the draft strategic plan were difficult to understand and the results SSA expects were unclear. In addition, the goal was not stated in a manner that allows for a future assessment of its achievement.

Three plans were missing goals for major functions and operations that are reflected in statute or are otherwise important to their missions. The Department of Health and Human Services (HHS) stated in its draft plan that the plan’s goals relate to those activities that have HHS priority over the next 6 years and that the goals did not cover every HHS activity. However, we found that the plan made no mention of a major function—that is, HHS’ responsibilities for certifying medical facilities, such as clinical laboratories and mammography providers. The section on goals in the Agency for International Development’s (AID) draft plan also did not fully encompass the agency’s major functions, because the section did not specifically address some programs, such as assistance to Eastern Europe and the former Soviet Union and Economic Support Funds, which represent about 60 percent of AID’s budget. In addition, the Postal Service’s draft plan did not contain goals and objectives for two major functions: providing mail delivery service to all communities and providing ready access to postal retail services.

Strategies Often Not Fully Developed

In our reports on draft strategic plans, we noted that strategies should be specific enough to enable an assessment of whether they would help achieve the goals in the plan. In addition, the strategies should elaborate on specific actions the agency is taking or plans to take to carry out its mission, outline planned accomplishments, and schedule their implementation. However, many of the strategies in the plans we reviewed lacked descriptions of approaches or actions to be taken or failed to address management challenges that threatened agencies’ ability to meet long-term strategic goals.

Plans Frequently Lacked Descriptions of Approaches

Incomplete and underdeveloped strategies were a frequent problem with the draft plans we reviewed. For example, the draft plan for the Department of State did not specifically identify the actions needed to meet the plan’s goals but rather often focused on describing the Department’s role in various areas. For example, the Department’s first strategy, “maintaining effective working relationships with leading regional states through vigorous diplomacy, backed by strong U.S. and allied military capability to react to regional contingencies,” did not describe how the Department planned to maintain effective working relationships or coordinate with the other lead agency, the Department of Defense (DOD), identified in the strategy.

In some cases, such as in the plans of Justice and Energy, strategies frequently read more like goals or objectives, rather than approaches for achieving goals. Justice’s strategy to promote compliance with the country’s civil rights laws and Energy’s strategy to maintain an effective capability to deter and/or respond to energy supply disruptions did not describe what actions the agencies planned to take to implement their related goals. Instead, their labelled strategies sounded like additional goals and objectives in that they discussed what the agencies expected to achieve. In other cases, such as in the plans of HHS and Commerce, strategies read like program justifications. Under strategies for addressing alcohol abuse, HHS’ draft plan states that “he National Institutes of Health conducts research and develops and disseminates information on prevention and treatment effectiveness.” Under strategies for providing technical leadership for the nation’s measurement and standards infrastructure, the Commerce plan stated that the “laboratories of the National Institute of Standards and Technology provide companies, industries, and the science and technology community with the common language needed in every stage of technical activity.” Without fully developed strategies, it will be difficult for managers, Congress, and other stakeholders to assess whether the planned approach will be successful in achieving intended results.

Key Management Challenges Often Not Addressed

One purpose of the Results Act is to improve the management of federal agencies. Therefore, it is particularly important that agencies develop strategies that address management challenges that threaten their ability to meet long-term strategic goals as well as this purpose of the Act. However, we found that most of the plans did not adequately address the major management challenges and high-risk areas that we and others have identified. For example, in our recent high-risk report series, we noted that DOD has long-standing management problems in six high-risk areas, including financial management, information technology, infrastructure, and inventory management. However, DOD’s draft plan generally paid little—and in one case, no—attention to high-risk management issues. We also placed Medicare, one of the largest federal entitlement programs, on our high-risk list, because of Medicare’s losses each year due to fraudulent and abusive claims. For example, the recent audit of financial statements performed by the Inspector General of HHS disclosed improper payments of $23.2 billion nationwide, or about 14 percent of total Medicare fee for service benefit payments. However, HHS’ draft plan did not address the long-standing problem the agency has with Medicare claims processing.

Another management-related issue that presents a challenge to agencies is ongoing and proposed restructuring of federal activities, which will likely require adjustments to agencies’ management practices, processes, and systems. For example, the administration has ongoing efforts to integrate (1) the Department of State, the U.S. Information Agency, and the Arms Control and Disarmament Agency into one agency with the intent to better serve the U.S. national interests and foreign policy goals in the 21st century; and (2) certain shared administrative functions of State and AID. However, State’s draft plan did not discuss how State planned to integrate these agencies into its organizational structure or address substantive support requirements for the reorganization.

Information Technology

For many years, we have reported on federal agencies’ chronic problems in developing and modernizing their information systems. Given the government’s ever-increasing dependency on computers and telecommunications to carry out its work, agencies must make dramatic improvements in how they manage their information resources in order to achieve mission goals, reduce costs, and improve service to the public. Moreover, without reliable information systems, agencies will not be able to gather and analyze the information they need to measure their performance, as required by the Results Act. Yet most of the 27 plans did not cover strategies for improving the information management needed to achieve their strategic goals or provided little detail on specific actions that agencies planned to take in this critical area.

In its draft plan, for example, DOD—which receives 15 percent of the federal budget—did not explicitly discuss how it plans to correct information technology investment problems. These problems led us to place its Corporate Information Management initiative on our high-risk list, because DOD continues to spend billions of dollars on automated information systems with little sound analytic justification. Without such discussions, Congress will not be able to assess the agencies’ planned approaches for upgrading information technology to improve the agencies’ performance.

Furthermore, we have identified as high risk two technology-related areas that represent significant challenges for the federal government: resolving the need for computer systems to be changed to accommodate dates beyond the year 1999, which is referred to as the “year 2000 problem”; and providing information security for computer systems. Yet most of the plans did not contain discussions of how agencies intend to address the year 2000 problem, and none of the plans addressed strategies for information security. For example, the draft plan of the Office of Personnel Management (OPM) did not discuss the year 2000 problem even though many of its critical information systems are date dependent and exchange information with virtually every federal agency. In another example, DOD’s draft plan did not specifically address information security even though DOD recognizes that information warfare capability is one of a number of areas of particular concern, especially as it involves vulnerabilities that could be exploited by potential opponents of the United States.

OMB’s guidance stated that agencies’ strategies for achieving goals should include a description of the process for communicating goals throughout an agency and for holding managers and staff accountable for achieving the goals. However, a few of the plans that we evaluated, such as those for Education and SSA, indicated that agencies had developed, or are planning to develop, approaches for communicating goals to employees or for holding managers and staff accountable for achieving results. We noted that assigning clear expectations and accountability to employees so that they see how their jobs relate to the agency’s mission and goals can be useful in implementing a strategic plan. It is especially important that managers and staff understand how their daily activities contribute to the achievement of their agencies’ goals and that they are held accountable for achieving results.

Noteworthy Progress

In contrast to the lack of strategies in most plans for addressing management weaknesses, we found that a few plans had operational strategies that indicated agencies are beginning to consider management, financial, and information technology weaknesses that need to be corrected to ensure that management practices, processes, and systems support the achievement of agency goals. For example, Education took an important step toward implementing results-oriented management by outlining in its draft strategic plan changes needed in activities, processes, and operations to better support its mission. To illustrate, Education’s plan contained core strategies for the goal that schools are safe, disciplined, and drug-free. These strategies included proposals for new legislation, public outreach, improved data systems, and interagency coordination. Energy and Education were among those agencies that included agencywide strategies to address needed process and operational realignments that would better enable them to achieve their missions. For example, Energy’s plan discussed strategies that emphasize changing contracting approaches to focus on results, contractor accountability, and customer satisfaction.

Little Evidence Regarding Interagency Coordination

As we recently reported, a focus on results, as envisioned by the Results Act, implies that federal programs contributing to the same or similar results should be closely coordinated to ensure that goals are consistent and, as appropriate, program efforts are mutually reinforcing. This means that federal agencies are to look beyond their organizational boundaries and coordinate with other agencies to ensure that their efforts are aligned.

Our work has underscored the need for such coordination efforts. Uncoordinated program efforts can waste scarce funds, confuse and frustrate program customers, and limit the overall effectiveness of the federal effort. Our recent report to you provided further information on mission fragmentation and program overlap in the federal government.We have often noted that the Results Act presents to Congress and the administration a new opportunity to address mission fragmentation and program overlap.

OMB and Congress recognize that the Results Act provides an approach for addressing overlap and fragmentation of federal programs. OMB’s guidance stated that agencies’ final submission of strategic plans should contain a summary of agencies’ consultation efforts with Congress and other stakeholders, including discussions with other agencies on crosscutting activities. During its Summer Review of 1996, OMB provided feedback to agencies where it found little sign of significant interagency coordination to ensure consistent goals among crosscutting programs and activities. This feedback also underscored the need for such coordination. In an August 1997 letter to heads of selected independent agencies and members of the President’s Management Council, OMB reiterated the importance of interagency coordination and stated that during the 1997 Fall Budget Review, it intended to place a particular emphasis on reviewing whether goals and objectives for crosscutting functions or interagency programs were consistent among strategic plans.

Congress has also shown active interest in using the Results Act to better ensure that crosscutting programs are properly coordinated. The February 25, 1997, letter from congressional Majority leaders to the Director of OMB outlined the leaderships’ interest in agencies’ strategic plans addressing how the agencies were coordinating their activities (especially for crosscutting programs) with other federal agencies working on similar activities. In addition, the staff teams in the House of Representatives, which were to coordinate and facilitate committee consultations with executive branch agencies, often have asked agencies about crosscutting activities and programs.

Despite this interest, we found that 20 of the 27 draft plans lacked evidence of interagency coordination as part of the agency and stakeholder consultations and that some of the plans—including those from some agencies that are involved in crosscutting program areas where interagency coordination is clearly implied—lacked any discussion of coordination. For example:

According to Energy, it does not have any crosscutting programs because its functions are unique. However, our review of draft strategic plans indicated areas of potential overlap concerning Energy’s programs. For example, Energy’s science mission was to maintain leadership in basic research and to advance scientific knowledge. The National Science Foundation’s (NSF) mission included promoting the progress of science and enabling the United States to uphold a position of world leadership in all aspects of science, mathematics, and engineering. NSF’s plan also did not discuss the possible overlap between the two missions. Another area of potential overlap for Energy included environmental and energy resources issues addressed by Energy as well as the Environmental Protection Agency (EPA) and other agencies. Similarly, nuclear weapons production issues involve Energy and DOD.

The draft plan for HHS did not address coordination of alcohol and drug abuse prevention and treatment programs, even though these programs are located in several of its subagencies and in 15 other federal agencies. These other agencies include VA, Education, Housing and Urban Development, and Justice. In the June 27, 1997, consultation with congressional staff on OPM’s draft plan, OPM officials said that they had not yet involved stakeholders, including other federal agencies, in developing their strategic plan. Among the organizations with which OPM must work to achieve its desired results are the Interagency Advisory Group of federal personnel directors, the Personnel Automation Council, the National Partnership Council, the Security Policy Board and Security Policy Forum, the Federal Bureau of Investigation, the Equal Employment Opportunity Commission, the Federal Labor Relations Authority, and the Merit Systems Protection Board.

Even if an agency’s draft plan recognized the need to coordinate with others, it generally contained little information about what strategies the agency pursued to identify and address mission fragmentation and program overlap. For example:

State’s draft plan recognizes several crosscutting issues but does not clearly address how the agency will coordinate those issues with other agencies. State and over 30 agencies and offices in the federal government are involved in trade policy and export promotion, about 35 are involved in global programs, and over 20 are involved in international security functions.

Treasury’s draft plan listed as a strategy that it will “continue participating in productive Federal, State, and local anti-drug task forces” but did not provide any detail about which bureaus or other federal agencies would participate in those task forces or what their respective responsibilities would be.

Even though it recognized the roles of other organizations, Labor’s draft plan did not discuss how the agency’s programs could fit in with a broader national job training strategy and the coordination required to develop and implement such a strategy. In 1995, we identified 163 employment training programs spread across 15 federal agencies, including Labor.

Commerce’s draft plan did not indicate how its emphasis on restructuring export controls to promote economic growth complements or contrasts with the strong emphasis of State’s Office of Defense Trade Controls and the U.S. Nuclear Regulatory Commission, which are both responsible for licensing exports overseas on safeguarding against proliferation of dual-use technology.

Many Agencies’ Capacity to Gather Performance Information Is Questionable

To efficiently and effectively operate, manage, and oversee activities, we have reported that agencies need reliable information on the performance of agency programs, the financial condition of programs and their operations, and the costs of programs and operations. For example, agencies need reliable data during their planning efforts to set realistic goals and later, as programs are being implemented, to gauge their progress toward achievement of those goals. However, our prior work indicated that agencies often lacked information and that even when this information existed, its reliability was frequently questionable.

On the basis of our recent report on implementing the Results Act, we found that some agencies lacked results-oriented performance information to use as a baseline for setting appropriate improvement targets. Our survey of federal managers done for that report suggested that those agencies were not isolated examples of the lack of performance information in the federal government. In this survey, we found that fewer than one-third of managers in the agencies reported that results-oriented performance measures existed for their programs to a great or very great extent. The existence of other types of performance measures also was reported as low. For example, of the managers reporting the existence of such measures to a great or very great extent, 38 percent reported the existence of measures of output, 32 percent reported the existence of customer satisfaction measures, 31 percent reported the existence of measures of product or service quality, and 26 percent reported the existence of measures of efficiency.

Our prior work also suggests that even when information existed, its reliability was frequently questionable. In our report on the Department of Transportation’s (DOT) draft plan, we stated that we had identified information resources and database management as one of the top management issues facing DOT. For example, the Federal Aviation Administration, which is a component of DOT, may rely on source data that are incomplete, inconsistent, and inaccurate for an aviation safety database that is under development. In our report on the draft HHS strategic plan, we stated that the agency had only limited data on the Medicaid program, some of which were of questionable accuracy. Some of these data problems stemmed from data originating in the 50 states and the District of Columbia, which did not all use identical definitions for data categories.

In addition to HHS, other agencies will likely have difficulties collecting reliable data from parties outside the federal government. Some agencies, such as Education, HHS, and EPA, planned to use or to strengthen partnerships with outside parties; thus, those agencies will also need to rely on those parties to provide performance data. During our recent review of analytic challenges that agencies faced in measuring their performance, agency officials with experience in performance measurement cited ascertaining the accuracy and quality of performance data as 1 of the top 10 challenges to performance measurement. The fact that data were largely collected by others was the most frequent explanation for why ascertaining the accuracy and quality of performance data was a challenge. In our report on implementing the Results Act, we also reported on the difficulties agencies were experiencing with their reliance on outside parties for data.

These experiences suggest that agencies face many challenges in gathering reliable information and that it is important that agencies follow through with the implementation of the CFO Act, the Clinger-Cohen Act, and the Paperwork Reduction Act. These experiences also suggest that coherent strategies for using or strengthening partnerships with outside parties would also include a strategy for data collection and verification plans. To Education’s credit, its draft plan recognized that improvements were needed in these areas. For example, Education’s plan identified core strategies for improving the efficiency and effectiveness of operations through the use of information technology, such as development of an agencywide information collection and dissemination system. As another example, EPA’s draft plan discusses the agency’s initiative to draft “core performance measures” with the environmental commissioners of state governments.

Program Evaluations Not Adequately Addressed in Most Plans

As we noted in our guide on assessing strategic plans, program evaluations are a key component of results-oriented management. In combination with an agency’s performance measurement system, evaluations can provide feedback to the agency on how well an agency’s activities and programs contributed to achieving strategic goals. For example, evaluations can be a potentially critical source of information for Congress and others in assessing (1) the appropriateness and reasonableness of goals; (2) the effectiveness of strategies by supplementing performance measurement data with impact evaluation studies; and (3) the implementation of programs, such as identifying the need for corrective action.

In our recent report on the analytic challenges facing agencies in measuring performance, we stated that supplementing performance data with impact evaluations may help provide agencies with a more complete picture of program effectiveness. A recurring source of the programs’ difficulty in both selecting appropriate outcome measures and in analyzing their results stemmed from two features common to many federal programs: the interplay of federal, state, and local government activities and objectives and the aim to influence complex systems or phenomena whose outcomes are largely outside government control. Evaluations can play a critical role in helping to address the measurement and analysis difficulties agencies face. Furthermore, systematic evaluation of how a program was implemented can provide important information about why a program did or did not succeed and suggest ways to improve it.

In that report, we also said that evaluation offices can provide analytical support for developing a performance measurement system. When asked where they needed assistance in performance measurement, agency officials were most likely to report that they could have used more evaluation help with creating quantifiable, measurable performance indicators and developing or implementing data collection and verification plans. Under the Results Act, program managers may wish to turn to their evaluation offices for formal program evaluations and for assistance in developing and using a performance measurement system. However, we have also reported that a 1994 survey found a continuing decline in evaluation capacity in the federal government.

Although the Results Act requires agencies to discuss program evaluations in their strategic plans, 16 of the draft plans we reviewed did not contain such a discussion. Of the 11 plans that did contain a section on evaluations, most of those sections lacked critical information specified in OMB guidance, such as a discussion of how evaluations were used to establish strategic goals or a schedule of future evaluations. Given the importance of evaluation for results-based management and the continuing decline in evaluation capacity, it is important that agencies’ strategic plans systematically address this issue.

Conclusion

It is clear that much work remains to be done if strategic plans are to be as useful for congressional and agency decisionmaking as they could be. We found that agencies’ draft strategic plans were very much works in progress. This situation suggests that agencies are struggling with the first step of performance-based management—that is, adopting a disciplined approach to setting results-oriented goals and formulating strategies to achieve the goals.

As agencies continue their strategic planning efforts and prepare for the next step of performance-based management—measuring performance against annual performance goals—it is important that the agencies, working with Congress and other stakeholders, address those strategic planning issues that appear to need particularly sustained attention. Our past work has shown that leading organizations focus on strategic planning as a dynamic and continuous process and not simply on the production of a strategic plan. They also understand that stakeholders, particularly Congress in the case of federal agencies, are central to the success of their planning efforts. Therefore, it is important that agencies recognize that strategic planning does not end with the submission of a plan in September 1997 and that a constant dialogue with Congress is part of a purposeful and well-defined strategic planning process.

Authorization, appropriation, budget, and oversight committees each have key interests in ensuring that the Results Act is successful, because once fully implemented, it should provide valuable data to help inform the decisions that each committee must make. In that regard, Congress can continue to express its interest in the effective implementation of the Results Act through iterative consultations with agencies on their missions and goals. Congress can also show its interest by continuing to ask about the status of agencies’ implementation of the Act during congressional hearings and by using performance information that agencies provide to help make management in the federal government more performance based.

Agency Comments and Our Evaluation

On September 3, 1997, we provided a draft of this report to the Director of OMB for comment. We did not provide a draft to individual agencies discussed in this report, because the drafts of the reports we prepared on individual agency plans in response to your request were provided to the relevant agency for comment. Those comments were reflected, as appropriate, in the final versions of those reports.

On September 10, 1997, a senior OMB official provided us with comments on this report. He generally agreed with our observations and said that the report was a useful summary of the 27 reports we issued on agencies’ draft strategic plans. The official also said that by identifying areas of widespread compliance or noncompliance with requirements of the Results Act, the report can be used to focus on those parts of plans that may require further work.

The senior OMB official did, however, raise an issue regarding program evaluations and the Results Act. He said that many strategic goals and objectives included in strategic plans will not require a program evaluation to help determine whether the goal was achieved. Thus, the absence of a schedule for future program evaluations should not be the basis for a categorical conclusion that a plan is deficient for this requirement. He also said that process evaluations can be useful in defining why a program is not working; they may be less instructive on why a program is succeeding.

In his view, process evaluations are more aligned with the strategies section of a strategic plan than with determinations of whether strategic goals and objectives are being achieved. In addition, the OMB official said that an evaluation of program impact is beyond the scope of the Results Act and that agencies are not required or expected to define their goals or objectives in terms of impact.

We note that the Results Act establishes two approaches for assessing an agency’s performance: annual measurement of program performance against performance goals outlined in a performance plan and program evaluations to be conducted by the agency as needed. Although the Act gives agencies wide discretion in determining the need for program evaluations, the Act also requires that agencies report to Congress and other stakeholders in their strategic plans on their planned use of evaluations to assess achievement of goals. Therefore, although program evaluations may not be necessary for determining whether every strategic goal in the strategic plan is achieved, a fuller discussion of how evaluations will, or will not, be used to measure performance is critical. Without this discussion, Congress and other stakeholders will not have assurances that agencies, as intended by the Act, systematically considered the use of program evaluations, where appropriate, to validate program accomplishments and identify strategies for program improvement. Thus, in cases where an agency concludes that program evaluations are not needed, we continue to believe that the agency’s plan would be more helpful to Congress if it contained such a statement and the reasons for the agency’s conclusion.

Moreover, the Senate report that accompanied the Results Act described program evaluations in broad terms, specifically “including evaluations of . . . operating policies and practices when the primary concern is about these issues rather than program outcome.” In this context, program evaluations are to be used to assess both the extent to which a program achieves its results-oriented goals (outcome evaluations) and the extent to which a program is operating as it was intended (process evaluation.) Understanding how a program’s operations produced, or did not produce, desired outcomes is critical information for agencies’ senior managers and Congress to consider as decisions are being made about programs and strategic goals.

Although the Act does not explicitly mention impact evaluations, it does require programs to measure progress toward achieving goals and explain why a performance goal was not met. Impact evaluations can be employed when external factors are known to influence the program’s objectives in order to isolate the program’s contribution to achievement of its objectives. Given the complexity of crosscutting federal programs as well as state and local programs, we continue to believe that in some circumstances, impact evaluations could be useful in helping to provide a more accurate picture of program effectiveness than might be portrayed by annual performance data alone or by other types of evaluations.

As arranged with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after its issue date. At that time, we will send copies of this report to the Minority Leader of the House; Ranking Minority Members of your Committees; other appropriate congressional committees; and the Director, Office of Management and Budget. We also will make copies available to others on request.

If you or your staffs have any questions concerning this report, please contact me on (202) 512-2700. The major contributors to this letter are listed in appendix II.

Overview of the Government Performance and Results Act

The Government Performance and Results Act (GPRA) is the primary legislative framework through which agencies will be required to set strategic goals, measure performance, and report on the degree to which goals were met. It requires each federal agency to develop, no later than by the end of fiscal year 1997, strategic plans that cover a period of at least 5 years and include the agency’s mission statement; identify the agency’s long-term strategic goals; and describe how the agency intends to achieve those goals through its activities and through its human, capital, information, and other resources. Under GPRA, agency strategic plans are the starting point for agencies to set annual goals for programs and to measure the performance of the programs in achieving those goals.

Also, GPRA requires each agency to submit to the Office of Management and Budget (OMB), beginning for fiscal year 1999, an annual performance plan. The first annual performance plans are to be submitted in the fall of 1997. The annual performance plan is to provide the direct linkage between the strategic goals outlined in the agency’s strategic plan and what managers and employees do day-to-day. In essence, this plan is to contain the annual performance goals the agency will use to gauge its progress toward accomplishing its strategic goals and identify the performance measures the agency will use to assess its progress. Also, OMB will use individual agencies’ performance plans to develop an overall federal government performance plan that OMB is to submit annually to Congress with the president’s budget, beginning for fiscal year 1999.

GPRA requires that each agency submit to the president and to the appropriate authorization and appropriations committees of Congress an annual report on program performance for the previous fiscal year (copies are to be provided to other congressional committees and to the public upon request). The first of these reports, on program performance for fiscal year 1999, is due by March 31, 2000; and subsequent reports are due by March 31 for the years that follow. However, for fiscal years 2000 and 2001, agencies’ reports are to include performance data beginning with fiscal year 1999. For each subsequent year, agencies are to include performance data for the year covered by the report and 3 prior years.

In each report, an agency is to review and discuss its performance compared with the performance goals it established in its annual performance plan. When a goal is not met, the agency’s report is to explain the reasons the goal was not met; plans and schedules for meeting the goal; and, if the goal was impractical or not feasible, the reasons for that and the actions recommended. Actions needed to accomplish a goal could include legislative, regulatory, or other actions or, when the agency found a goal to be impractical or infeasible, a discussion of whether the goal ought to be modified.

In addition to evaluating the progress made toward achieving annual goals established in the performance plan for the fiscal year covered by the report, an agency’s program performance report is to evaluate the agency’s performance plan for the fiscal year in which the performance report was submitted. (For example, in their fiscal year 1999 performance reports, due by March 31, 2000, agencies are required to evaluate their performance plans for fiscal year 2000 on the basis of their reported performance in fiscal year 1999.) This evaluation will help to show how an agency’s actual performance is influencing its plans. Finally, the report is to include the summary findings of program evaluations completed during the fiscal year covered by the report.

Congress recognized that in some cases not all of the performance data will be available in time for the March 31 reporting date. In such cases, agencies are to provide whatever data are available, with a notation as to their incomplete status. Subsequent annual reports are to include the complete data as part of the trend information.

In crafting GPRA, Congress also recognized that managerial accountability for results is linked to managers having sufficient flexibility, discretion, and authority to accomplish desired results. GPRA authorizes agencies to apply for managerial flexibility waivers in their annual performance plans beginning with fiscal year 1999. The authority of agencies to request waivers of administrative procedural requirements and controls is intended to provide federal managers with more flexibility to structure agency systems to better support program goals. The nonstatutory requirements that OMB can waive under GPRA generally involve the allocation and use of resources, such as restrictions on shifting funds among items within a budget account. Agencies must report in their annual performance reports on the use and effectiveness of any GPRA managerial flexibility waivers that they receive.

GPRA called for phased implementation so that selected pilot projects in the agencies could develop experience from implementing GPRA requirements in fiscal years 1994 through 1996 before implementation is required for all agencies. When this part of the pilot phase concluded at the end of fiscal year 1996, a total of 68 pilot projects representing 28 agencies were project participants. OMB also was required to select at least five agencies from among the initial pilot agencies to pilot managerial accountability and flexibility for fiscal years 1995 and 1996; however, we found that the pilot did not work as intended. OMB did not designate as pilot projects any of the 7 departments and 1 independent agency that submitted a total of 61 waiver proposals because, among other reasons, changes in federal management practices and laws that occurred after the Act was enacted affected agencies’ need for the managerial flexibility waivers.

Finally, GPRA required OMB to select at least five agencies, at least three of which have had experience developing performance plans during the initial GPRA pilot phase, to test performance budgeting for fiscal years 1998 and 1999. Performance budgets to be prepared by pilot projects for performance budgeting are intended to provide Congress with information on the direct relationship between proposed program spending and expected program results and the anticipated effects of varying spending levels on results. However, we found that the performance budgeting pilots are likely to be delayed. According to OMB, few agencies currently have either sufficient baseline performance or financial information or the ability to use sophisticated analytic techniques to calculate the effects that marginal changes in funding can have on performance.

Major Contributors to This Report

General Government Division, Washington, D.C.

Related GAO Products

Agencies’ Draft Strategic Plans

The Results Act: Observations on the Draft Strategic Plan of the Department of Agriculture (GAO/RCED-97-169R, July 10, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Department of Commerce (GAO/GGD-97-152R, July 14, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Department of Defense (GAO/NSIAD-97-219R, Aug. 5, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Department of Education (GAO/HEHS-97-176R, July 18, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Department of Energy (GAO/RCED-97-199R, July 11, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Department of Health and Human Services (GAO/HEHS-97-173R, July 11, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Department of Housing and Urban Development (GAO/RCED-97-224R, Aug. 8, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Department of the Interior (GAO/RCED-97-207R, July 21, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Department of Justice (GAO/GGD-97-153R, July 11, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Department of Labor (GAO/HEHS-97-172R, July 11, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Department of State (GAO/NSIAD-97-198R, July 18, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Department of Transportation (GAO/RCED-97-208R, July 30, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Treasury (GAO/GGD-97-162R, July 31, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Department of Veterans Affairs (GAO/HEHS-97-174R, July 11, 1997).

The Results Act: Observations on the Draft Strategic Plan of the U.S. Agency for International Development (GAO/NSIAD-97-197R, July 11, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Environmental Protection Agency (GAO/RCED-97-209R, July 30, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Federal Emergency Management Agency (GAO/RCED-97-204R, July 22, 1997).

The Results Act: Observations on the Draft Strategic Plan of the General Services Administration (GAO/GGD-97-147R, July 7, 1997).

The Results Act: Observations on the Draft Strategic Plan of the National Aeronautics and Space Administration (GAO/NSIAD-97-205R, July 22, 1997).

The Results Act: Observations on the Draft Strategic Plan of the National Science Foundation (GAO/RCED-97-203R, July 11, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Nuclear Regulatory Commission (GAO/RCED-97-206R, July 31, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Office of Management and Budget (GAO/GGD-97-169R, Aug. 1997).

The Results Act: Observations on the Draft Strategic Plan of the Office of Personnel Management (GAO/GGD-97-150R, July 11, 1997).

The Results Act: Observations on the Draft Strategic Plan of the U.S. Postal Service (GAO/GGD-97-163R, July 31, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Small Business Administration (GAO/RCED-97-205R, July 11, 1997).

The Results Act: Observations on the Draft Strategic Plan of the Social Security Administration (GAO/HEHS-97-179R, July 22, 1997).

The Results Act: Observations on the Draft Strategic Plan of the U.S. Trade Representative (GAO/NSIAD-97-199R, July 18, 1997).

The Results Act: Observations on Federal Science Agencies (GAO/T-RCED-97-220, July 30, 1997).

Financial Management: Indian Trust Fund Strategic Plan (GAO/T-AIMD-97-138, July 30, 1997).

The Results Act: Observations on Draft Strategic Plans of Five Financial Regulatory Agencies (GAO/T-GGD-97-164, July 29, 1997).

National Labor Relations Board: Observations on the NLRB’s July 8, 1997, Draft Strategic Plan (GAO/T-HEHS-97-183, July 24, 1997).

The Results Act: Observations on the Forest Service’s May 1997 Draft Strategic Plan (GAO/T-RCED-97-223, July 23, 1997).

Results Act: Observations on the Department of Energy’s August 15, 1997, Draft Strategic Plan (GAO/RCED-97-248R, Sept. 2, 1997).

Other Related GAO Products

Managing for Results: Using the Results Act to Address Mission Fragmentation and Program Overlap (GAO/AIMD-97-146, Aug. 29, 1997).

Managing for Results: The Statutory Framework for Improving Federal Management and Effectiveness (GAO/T-GGD/AIMD-97-144, June 24, 1997).

The Results Act: Comments on Selected Aspects of the Draft Strategic Plans of the Departments of Energy and the Interior (GAO/T-RCED-97-213, July 17, 1997).

Managing for Results: Prospects for Effective Implementation of the Government Performance and Results Act (GAO/T-GGD-97-113, June 3, 1997).

The Government Performance and Results Act: 1997 Governmentwide Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).

Managing for Results: Analytic Challenges in Measuring Performance (GAO/HEHS/GGD-97-138, May 30, 1997).

Agencies’ Strategic Plans Under GPRA: Key Questions to Facilitate Congressional Review (GAO/GGD-10.1.16, May 1997).

Performance Budgeting: Past Initiatives Offer Insights for GPRA Implementation (GAO/AIMD-97-46, Mar. 27, 1997).

Measuring Performance: Strengths and Limitations of Research Indicators (GAO/RCED-97-91, Mar. 21, 1997).

Managing for Results: Enhancing the Usefulness of GPRA Consultations Between the Executive Branch and Congress (GAO/T-GGD-97-56, Mar. 10, 1997).

Managing for Results: Using GPRA to Assist Congressional and Executive Branch Decisionmaking (GAO/T-GGD-97-43, Feb. 12, 1997).

Executive Guide: Effectively Implementing the Government Performance and Results Act (GAO/GGD-96-118, June 1996).

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.