Managing for Results: An Agenda To Improve the Usefulness of Agencies' Annual Performance Plans
Released: 1998-09-08T00:00:00

Letter

Background

In crafting the Results Act, Congress drew on the experiences of foreign governments and state and local governments in the United States and recognized that the results-oriented goal setting and performance measurement requirements of the Results Act would constitute a new way of doing business for many agencies. Congress also realized that the effective implementation of the Results Act may take several years. To advance this effort, the Results Act provided for a series of pilot projects so that agencies could gain experience and share lessons learned in implementing the key provisions of the Results Act before its governmentwide implementation. One set of these pilot projects covered the Act’s annual performance planning and reporting provisions. Over 70 federal organizations participated in this pilot phase, which covered fiscal year 1994 through fiscal year 1996. To further help agencies, several Members of Congress asked us to develop—on the basis of the experiences of leading foreign, state, and federal organizations—a guide for agency managers to use to effectively implement the Act.

We observed in our June 1997 report on the implementation of the Results Act and related performance-based management initiatives that despite the rich body of experience the pilots provided, initial governmentwide implementation of the Results Act would be highly uneven. We identified a series of daunting implementation challenges and predicted that the initial set of agency strategic and annual performance plans would not be of consistently high quality or as useful for congressional and executive branch decisionmakers as they could be.

At the request of several members of the congressional leadership, in May 1997 we issued a guide for congressional staff to use in assessing agencies’ strategic plans. We subsequently reviewed draft and September 30, 1997, strategic plans that agencies submitted to Congress. In our January 1998 summary report on our reviews of the September plans, we highlighted three difficult planning challenges that especially needed continued progress: setting a strategic direction, including establishing clear, results-oriented goals and performance measures; coordinating crosscutting programs; and ensuring the capacity to gather and use performance information. We suggested that agencies’ annual performance plans could help address these challenges.

As a next step in our efforts to assist Congress and agencies in effectively implementing the Results Act, we issued two related guides—one for congressional decisionmakers and one for evaluators and others interested in more detailed assessments—on assessing annual performance plans. These guides, developed with the assistance of congressional staff, senior officials in agencies, members of the CFO Council, and others, integrated criteria from the Results Act, its legislative history, OMB’s guidance for developing the plans (OMB Circular A-11, part 2), and our work on implementation of the Results Act. The guides organize the Results Act’s criteria under three core questions that are aimed at ensuring that performance plans are useful for decisionmaking. The three core questions are: (1) To what extent does the agency’s performance plan provide a clear picture of intended performance across the agency? (2) How well does the agency’s performance plan discuss the strategies and resources the agency will use to achieve its performance goals? (3) To what extent does the agency’s performance plan provide confidence that its performance information will be credible?

We noted that as agencies and Congress gain experience in developing and using annual performance plans, additional issues and questions will emerge. We therefore have committed to issuing a combined, updated version of our congressional and evaluators’ performance plan guides reflecting those experiences and providing examples drawn from the agencies’ plans illustrating useful presentations. An exposure draft of that guide will be issued this fall.

Agency Performance Plans Did Not Consistently Provide Clear Pictures of Intended Performance

At the most basic level, an annual performance plan is to provide a clear picture of intended performance across the agency. Such information is important to Congress, agency managers, and others for understanding what the agency is trying to achieve, identifying subsequent opportunities for improvement, and assigning accountability. We found that the plans did not consistently provide the succinct and concrete statements of intended performance that are needed to help guide decisions and subsequently assess actual performance. The plans generally were successful in showing how an agency’s mission and strategic goals were related to its performance goals. This is a very positive development because it provides a basis for using the performance plans to track progress toward the achievement of agencies’ long-term strategic goals. The plans were much less successful in providing assurance that crosscutting program efforts were sufficiently coordinated. Agencies appear to be taking the first step of identifying crosscutting efforts, with some including helpful listings of other agencies with which they share common goals. However, few plans provided any descriptions of how the agency will coordinate with other agencies regarding national issues for which they share responsibility or reflected other substantive coordination.

Defining Expected Performance

Almost all of the annual performance plans that we reviewed contained at least some objective, quantifiable, and measurable annual performance goals—a key expectation of Congress in enacting the Results Act. Overall, however, the annual performance goals and accompanying measures in the plans would need significant development to improve the usefulness of the plans to congressional and other decisionmakers. Specifically, we found that the goals in the annual performance plans often were not as results-oriented as they could be; the relationship between goals and performance measures at times was either neglected or obscured; and the plans did not consistently set goals to address major management problems, as suggested in OMB guidance. On the other hand, some of the plans provided very helpful baseline and trend data for performance goals. Such information allows users of plans to judge whether performance targets are appropriate and reasonable based on past performance.

Focusing on Results

Goals in the performance plans that we reviewed typically focused on program outputs, such as the number of products and services delivered by the agency. The Results Act allows agencies to include output goals in their plans, and such goals can provide important information for agency managers to use in managing programs. However, the Act envisions that agencies’ plans would contain goals that focus on the results that programs are intended to achieve, which is particularly important for policymakers. We found that the annual performance plans did not consistently contain such results-oriented goals.

For example, the Social Security Administration (SSA), which was responsible for expenditures of about $400 billion in 1997—constituting one-fourth of the federal budget—did not consistently have results-oriented performance goals. For its high-risk Supplemental Security Income program, SSA’s plan included output goals on the number of claims processed and the number of nondisability redeterminations, but it did not include results-oriented goals. Likewise, for SSA’s Old Age and Survivors Insurance (OASI) and Disability Insurance (DI) programs, the performance plan had output goals related to the number of beneficiaries served, but it did not contain results-oriented goals related to services these beneficiaries receive. However, the governmentwide performance plan’s chapter on Social Security included a results-oriented discussion of the effect of Social Security on reducing poverty among the elderly in addition to addressing the number of beneficiaries served by the OASI and DI programs. Having output-oriented goals will provide decisionmakers with important information but will not directly provide a perspective on the degree to which the program is accomplishing the results it is intended to achieve.

In crafting the Results Act, Congress recognized that for some types of federal programs it may not be feasible for an agency to express its performance goals in an objective, quantifiable, and measurable form. The Results Act therefore allows an agency to propose, and OMB to authorize, that a goal be expressed in an alternative form, such as by describing a minimally effective program and a successful program. Although few agencies used the alternative form of measurement for fiscal year 1999, the experiences of the National Science Foundation (NSF) suggested how alternative forms of measurement could be employed. The agency’s performance plan used such alternative descriptions to establish annual performance goals for its scientific research and educational activities. For example, NSF’s plan described annual success in addressing the agency’s strategic goal of promoting scientific discovery as occurring when the agency’s awards lead to important discoveries and new knowledge within and across traditional disciplinary boundaries. NSF’s plan described corresponding minimal effectiveness as occurring when there is a steady stream of outputs of good scientific quality. By establishing definitions for successful and minimally effective levels of performance, NSF’s descriptions allowed the agency’s performance to be assessed, both by congressional and executive branch decisionmakers and by expert reviewers that NSF plans to use. NSF could build on its approach to measurement by better explaining what it means by such phrases as “important discoveries” and “steady stream of outputs of good scientific quality.” One way to do this would be to provide examples of past discoveries that illustrate each of the descriptive statements.

In contrast to many agencies, the Department of Health and Human Services’ (HHS) Centers for Disease Control and Prevention (CDC) had numerous concrete measurable results-oriented performance goals. For example, CDC had a results-oriented goal and measure to reduce the incidence of congenital syphilis in the general population from the 1995 rate of 39 per 100,000 live births to less than 30 in fiscal year 1999. CDC’s program outputs related to its goal, such as targeted prenatal screenings for congenital syphilis, were presented as the strategies CDC will use to achieve its end result rather than as the programmatic end in itself. Such a presentation suggests a clear understanding of the relationships and differences between the activities an agency undertakes and the results it hopes to achieve. In addition, when CDC used output-oriented performance measures in some cases, it explained why it used such measures rather than more results-oriented measures. The section on chronic disease prevention, for example, noted that health outcome measures for these programs have been difficult to define for a number of reasons, including the long latency period of chronic diseases like cancer and heart disease.

Relationships Between Goals and Measures

One of the major challenges that agencies face in moving from a focus on the activities they undertake to results they are trying to achieve is to develop performance measures that clearly and sufficiently relate to the performance they are meant to assess. At CDC, for example, the performance goal to reduce the incidence of congenital syphilis was clearly and sufficiently represented by the performance measure of reducing the occurrence of congenital syphilis from 39 per 100,000 live births to less than 30 per 100,000 live births.

Far more typical were situations where the relationships between what is being measured and desired results are not sufficient. For example, three of the Department of Labor’s (Labor) performance goals used the number of complaints received as measures of compliance with worker protection and civil rights laws. In one case, a decrease in the number of discrimination complaints filed by federal grant recipients and persons with disabilities in state and local governments was used to indicate progress towards the goal of ensuring that workplaces are fair for these groups. Used alone, such a measure is a questionable indicator of fairness in the workplace. A decrease in the number of complaints could also be a function of lack of information, fear, a complainant’s lack of confidence in Labor’s enforcement, or a tendency of agency managers to discourage the filing of otherwise meritorious complaints. An expanded or improved enforcement program could produce an increase in complaints as workers gain confidence in the enforcement agency. Without other independent measures that also demonstrate the existence of a fair workplace, measuring the decrease in complaints may be insufficient.

Also, programs often must achieve multiple goals or goals with several dimensions that reflect such competing demands or priorities as quality, timeliness, program cost, and outcomes. Annual performance plans that do not contain balanced sets of measures may not sufficiently assess all aspects of a goal or multiple goals for the agencies’ programs. One of the key priorities missed in many plans was cost. For example, the Office of Personnel Management’s (OPM) plan did not appear to have cost-based performance measures to show how efficiently it performs certain businesslike operations, such as the administration of health and retirement programs. On the other hand, the Department of Veterans Affairs’ (VA) plan provided both financial and nonfinancial measures for some of its program areas. For example, VA’s performance plan contained measures that addressed various program priorities, such as the accuracy and timeliness of claims processing, unit costs of providing benefits and services, and customer satisfaction with VA services.

In addition, agencies’ performance measures did not always have a clearly apparent or commonly accepted relationship to the performance goals. At the Department of the Interior’s (Interior) National Park Service (NPS), some performance measures did not provide clear definitions of the criteria that would be used to accurately assess the performance. For example, one of NPS’ performance goals was to ensure that 50 percent of the cultural landscapes on its Cultural Landscapes Inventory were in good condition. However, the plan did not define “good condition” or make reference to where such a definition could be found. Without such a definition, neither the precise relationship between the measure and the desired result nor whether performance is being measured consistently from year to year can be determined.

Addressing Major Management Problems

According to OMB guidance, an agency’s annual performance plan should also set performance goals to address major management problems that are mission-critical or impede the agency’s ability to meet its programmatic goals. We found, however, that agencies did not consistently set goals to address major management problems. Most significantly, the governmentwide performance plan’s first priority objective is to ensure that agencies’ business processes and supporting systems function successfully in the year 2000 and beyond. Even though there were agency plans that otherwise acknowledged this issue, most of the agencies’ plans neglected to include steps to address it. For example, in the Small Business Administration’s (SBA) case, although its performance plan discussed actions SBA planned to take to help small businesses deal with the Year 2000 problem, the plan did not discuss or provide information on SBA’s efforts to resolve the agency’s own Year 2000 problems. In addition, the Interior Departmental Overview section of its performance plan listed ensuring that the Department’s critical information systems and processes are Year 2000-compliant by March 31, 1999, as a strategic goal for the Department. However, six of the eight subagency plans did not address the problem.

On the other hand, some agencies’ plans included performance goals and measures to show how the Year 2000 issue would be addressed. OPM had a fiscal year 1999 annual performance goal to ensure that OPM’s information technology systems would operate properly on and after January 1, 2000.

One of OPM’s measures for this goal was that the agency’s systems would meet or improve on the OMB-established governmentwide target dates for Year 2000 compliance in that all systems would be renovated by September 1998 and would be implemented in a Year 2000-compliant environment by December 1998.

Baseline and Trend Performance Data

Agencies that go beyond the requirements of the Results Act and include baseline or trend data for their performance goals in their annual performance plans provide a more informative basis for assessment of expected performance. Reliable baseline and trend data are helpful to providing congressional and other decisionmakers with a context for assessing whether performance goals are reasonable and appropriate and suggesting areas for questions about how planned performance improvements will be achieved. For example, the Department of Commerce’s (Commerce) annual performance plan generally provided performance data from fiscal year 1997 when available, the fiscal year 1998 goal, and the fiscal year 1999 goal. As an illustration, the performance plan showed that although the fiscal year 1998 goals for the lead times and the accuracy of flash flood warnings were the same as the actual levels achieved in fiscal year 1997, performance improvements were planned in both areas for fiscal year 1999.

Connecting Mission, Goals, and Activities

The annual performance plans that we examined were generally successful in showing how the agencies’ missions and strategic goals were related to their performance goals. Indicating such relationships is important for showing how an agency will chart annual progress toward the achievement of its long-term strategic goals. The Environmental Protection Agency’s (EPA) performance plan included one of the most effective presentations in this regard. This plan included the mission statement and goals from the strategic plan and a section on the relationship between the two plans, including changes in the strategic goals and objectives since the strategic plan was issued. In addition, the plan was primarily organized by the strategic goals and objectives, with performance goals, resources, strategies, and performance measures grouped by strategic goal and objective.

As required by the Results Act, agencies generally developed performance plans that covered all program activities in their budget requests—with many agencies establishing for the first time a direct connection among plans, budgets, performance information, and the related congressional resource allocation and oversight processes. Agencies used various approaches to make this connection. Some provided descriptions or tables that associated performance goals with their existing program activities, and others took advantage of the flexibility provided by the Results Act to aggregate, disaggregate, and consolidate program activities to indicate coverage. For example, EPA’s and the Department of Transportation’s performance plans showed a relationship between performance goals and program activities by linking them to strategic goals and/or objectives. Whether the agencies used existing program activities or aggregated, disaggregated, or consolidated program activities, the most useful linkages indicated how funding from the agency’s program activities would be allocated to a discrete set of performance goals.

In contrast, the performance plans that associated one or more performance goals with one or more program activities were less informative in this regard. This is because such associations made it difficult to determine whether all activities were substantively covered or understand how specific program activities were intended to contribute to the agency’s results. For example, the SBA plan did not convey which performance goals covered which program activities or whether all of SBA’s program activities were covered by performance goals. The Interior plan contained some goals for NPS that were not associated with any program activities, even though the goals apparently required some funding. For example, NPS’ goal of improving the quality of its employee housing through removing, replacing, or upgrading units was not related to any program activity. Although the Department of Housing and Urban Development’s (HUD) plan related some of the agency’s program activities to its goals, it did not cover all of HUD’s program activities or explain whether those activities were aggregated, disaggregated, or consolidated. For example, HUD did not explain which performance goals covered the $310 million drug elimination grants for its low-income housing program.

Recognizing Crosscutting Efforts

Over the last several years, we have produced a body of work pointing to mission fragmentation and overlap in a wide variety of federal program areas. Our work has shown that uncoordinated program efforts can waste scarce funds, confuse and frustrate program customers, and limit the overall effectiveness of the federal effort. We have suggested that agencies’ efforts under the Results Act provide a potentially effective vehicle for ensuring that crosscutting program goals are consistent; strategies are mutually reinforcing; and, as appropriate, progress is assessed through the use of common performance measures.

Last fall, when we reviewed agencies’ strategic plans, we stressed that coordinating crosscutting programs can be a difficult and time-consuming process. To underscore our concern, we highlighted the issue as one of the most difficult planning challenges requiring continued progress. In our review of agencies’ September 1997 strategic plans, we found that those plans provided better descriptions of crosscutting programs and coordination efforts than the agencies’ draft strategic plans. The most useful of the strategic plans contained presentations that listed other agencies involved in crosscutting program areas and outlined approaches to coordinating such areas with those agencies. We noted that such presentations illustrated the magnitude of, and provided a foundation for, the much more difficult work that lies ahead—undertaking the substantive coordination that is needed to ensure that crosscutting programs are effectively managed.

Since then, agencies appear to have made uneven progress. Our review of agencies’ annual performance plans suggested that the needed first step is now being taken more consistently—the plans often identified crosscutting efforts, and some included helpful listings of other agencies with which responsibility for addressing similar national issues is shared. However, few plans attempted the more challenging description of how the agencies expected to coordinate their efforts with those of other agencies or reflected the existence of substantive coordination.

As an illustration, Commerce identified a number of other federal agencies’ programs that are related to Commerce’s three strategic themes and its bureaus’ activities. However, the agency’s performance plan did not indicate how Commerce would work with these other agencies in addressing shared activities. For example, the plan associated 12 other federal agencies with Commerce’s International Trade Administration through the Trade Promotion Coordinating Committee. However, neither the plan nor the supporting congressional budget justification documents explained how Commerce can use its key role in chairing the Committee to accomplish Commerce’s strategic goal of implementing the national export strategy.

Similarly, the National Aeronautical and Space Administration’s (NASA) performance plan also took the first step of identifying other agency or international partners involved in specific efforts related to NASA’s efforts. However, it did not discuss the extent to which NASA had coordinated with other agencies in establishing the goals, objectives, and associated performance targets. For example, in describing the objective of developing next-generation computational design tools, the plan indicated that NASA’s efforts were part of the Federal High Performance Computing and Communications initiative. However, there was no discussion about whether NASA coordinated its performance target of a 200-fold improvement with other federal partners; nor was there an explanation of how NASA’s effort will contribute to the overall federal initiative, separately from the contributions that other agencies will make.

A few performance plans were more useful in that they discussed how agencies expected to coordinate efforts with other agencies that have similar responsibilities. Similar to the most useful strategic plans, such discussions underscored the magnitude of the coordination work that lies ahead. For example, Education’s plan contained not only an extensive list of other agencies with which the Department shares a common result but also a general discussion of its coordination efforts and plans. For its strategic objective that every state have a school-to-work system that increases student achievement, improves technical skills, and broadens career opportunities for all, Education’s plan indicated that the agency plans a coordination effort with Labor to jointly administer the National School-to-Work Office Program and improve the management of that program by aligning grant-making, audit, technical assistance, and performance reporting functions. Education can build on its foundation by identifying (1) performance goals that reflect its crosscutting programs, (2) how Education and other agencies will work to ensure that program strategies are mutually reinforcing, and (3) whether any common performance measures are to be used.

Agency Performance Plans Generally Did Not Relate Strategies and Resources to Performance

In general, the annual performance plans did not provide sufficiently complete discussions of the strategies and resources that agencies will use to achieve their performance goals. Discussions in the plans of agency strategies, which can include program initiatives, partnerships, and operational processes, frequently did not yield clear understandings of how the strategies were to lead to improved agency performance and the achievement of annual performance goals. Moreover, the plans often lacked complete discussions of the capital, human, financial, and other resources that the agencies needed to achieve their goals. The absence of fully developed discussions relating strategies and resources to goals undermined the usefulness of the plans. As a result, congressional and other decisionmakers do not have complete information on which to judge the reasonableness of an agency’s proposed strategies and resources needed to achieve its goals.

Connecting Strategies to Results

Most agencies’ annual performance plans did not clearly describe how the performance goals would be achieved. The performance plans often provided listings of the agencies’ current array of programs and initiatives but provided only limited perspective on how these programs and initiatives were necessary to or helpful for achieving results. For example, the General Services Administration (GSA) performance plan provided descriptive information on GSA’s activities as opposed to specific strategies for achieving performance goals. One of GSA’s performance goals was to increase market share for its vehicle fleet program. Although the plan contained measures and target levels for fiscal years 1998 and 1999, the accompanying narrative gave little indication of how GSA intended to achieve the target levels. Instead, the plan provided general statements about leveraging GSA’s competitive pricing with broad market penetration and government downsizing. The plan offered no information on a specific approach or strategy for how GSA would achieve broad market penetration or take advantage of downsizing to meet the market share target levels for its vehicle fleet program.

Far less typical, but far more useful in our view, was the approach taken by the Federal Emergency Management Agency (FEMA), which generally presented strategies that were clear and appeared logically related to annual performance goals. For example, to achieve its strategic goal of protecting lives and preventing the loss of property, FEMA’s performance plan contained a performance goal to increase Project Impact communities in each state for its Project Impact program, which is designed to promote predisaster mitigation. Strategies for achieving this goal included working with states and federal agencies to identify candidate communities, providing grants as seed funding, providing technical information, and monitoring progress.

We also observed that most agencies did not build on the work they had done last fall in developing their strategic plans where they were to identify factors external to the agency that would affect the degree to which they achieve their strategic goals. Such factors could include economic, demographic, social, technological, or environmental factors. Assessments of external factors help agencies and Congress judge the likelihood of an agency achieving its strategic goals and the actions needed to better meet those goals. Similar to the situation with strategic goals, discussions of the influence external factors can have on annual performance goals, although not required by the Results Act, can provide important context for understanding both the factors other than agency performance that can affect whether goals are achieved and the adequacy of the agencies’ plans for mitigating negative factors and taking advantage of positive ones.

The value of including such an analysis in annual performance plans is shown by SBA’s plan. SBA provided an informative discussion of external factors, such as the economy and continued support from stakeholder and program partners, that might affect the agency’s ability to achieve performance goals related to its strategic goal to become a 21st century leading-edge financial institution. Within this context, the plan also included a discussion of actions SBA can take to mitigate these factors.

Connecting Resources to Strategies

Most agencies’ annual performance plans did not adequately describe capital, human, information, and financial resources and relate them to achieving performance goals. Even in cases where the achievement of performance goals seems to depend on increased staffing levels or capital expenditures, such increases were not always described in the plans. For example, VA included a discussion concerning the expansion and construction of its cemetery system, but it does not identify the additional dollars, people, or equipment necessary to achieve this goal.

Information Technology

Addressing information technology issues in annual performance plans is important because of technology’s critical role in achieving results, the sizable investment the federal government makes in information technology (about $145 billion between 1992 and 1997), and the long-standing weaknesses in virtually every agency in successfully employing technology to further mission accomplishment. The vital role that information technology can play in helping agencies achieve their goals was not clearly described in agency plans. In the absence of such discussions, the plans also generally did not reference other appropriate documents that might contain information on the agencies’ technology plans. The failure to recognize the central role of technology in achieving results is a cause of significant concern because, under the Paperwork Reduction and Clinger-Cohen Acts, Congress put in place clear statutory requirements for agencies to better link their technology plans and information technology use to their missions and programmatic goals. Without at least some discussion of information technology, agency plans are not complete, and their usefulness to congressional and other decisionmakers is accordingly undermined.

The Department of State’s (State) and Interior’s performance plans were fairly typical of all agencies’ plans in terms of the lack of attention to technology issues. Although the State plan discusses upgrading the information technology infrastructure, along with other resources, it does not address how such an upgrade would be used to improve performance or help achieve specific performance goals. At Interior, with the exception of the United States Geological Survey’s plan, the subagency plans generally did not discuss how information technology will be used to help achieve annual performance goals or improve performance for long-term objectives.

Linking Funding Requests to Performance

As discussed earlier in the report, most agencies developed performance goals that cover the program activities in their budget requests. In addition, OMB Circular A-11 states that agencies should display, by program activity, the funding level being applied to achieve performance goals. However, most agencies did not clearly convey in their annual performance plans the amount of funding needed to achieve a discrete set of performance goals. For example, as illustrated in figure 1, VA aggregated program activities under specific budget accounts and consolidated these activities across those budget accounts to align those accounts (and the underlying program activities) with groupings of performance goals for two of its business lines—(1) Education and (2) Vocational Rehabilitation and Counseling. Although this association established a relationship between numerous program activities and numerous performance goals, it did not identify the funding levels that are needed to achieve discrete sets of goals. As a result, the plan did not convey how requested funds under the related program activities will be allocated to achieve performance.

In contrast, by identifying how much funding is needed to support discrete sets of performance goals and showing where that funding was included in the agency’s budget request, an agency’s annual performance plan can give Congress and other decisionmakers information needed to relate decisions about desired performance with decisions about funding levels. A few agencies, such as EPA and some components of the Department of Treasury (Treasury), proposed changes to their program activity structures to better facilitate such an allocation and establish a clearer connection between budgetary decisionmaking and performance. For example, EPA proposed a uniform program activity structure across all of its budget accounts in which each program activity represents one of the agency’s strategic goals. Using this proposed program activity structure in its performance plan, EPA showed by account the funding it is requesting to achieve each strategic objective and the supporting annual performance goals. Figure 2 illustrates this relationship.

Similarly, Treasury’s Bureau of Alcohol, Tobacco and Firearms (ATF) has changed its program activities to align them with its strategic goals. Because ATF’s revised program activities reflect its strategic goals, its performance plan clearly showed how ATF would allocate its resources and the bureau’s performance goals can be readily and logically related to the program activity structure. Figure 3 shows the relationship described between performance goals and program activities in ATF’s plan.

Performance Plans Provided Limited Confidence That Performance Data Would Be Credible

Credible performance information is essential for accurately assessing agencies’ progress towards the achievement of their goals and, in cases where goals are not met, identifying opportunities for improvement or whether goals need to be adjusted. Under the Results Act, agencies’ annual performance plans are to describe the means that will be used to verify and validate performance data. However, the majority of the plans we reviewed provided only limited confidence that performance information would be credible. Specifically, although most of the plans describe procedures for verifying and validating performance information, these plans lack specific details on the actual procedures the agencies will use. In addition, few plans include a discussion of the known limitations in the agencies’ existing data systems. Of those plans that did include such a discussion, almost none discussed the agency’s strategies to address the known limitations.

Verifying and Validating Performance

In our report on agencies’ September 30 strategic plans, we noted that many agencies had long-standing and serious shortcomings in their ability to generate reliable and timely performance data. We suggested that the annual performance plans provided an opportunity to articulate how these shortcomings will be addressed. Performance plans can help do this by including a discussion intended to provide confidence that the means agencies will use to verify and validate performance information, such as audits, program evaluations, independent external reviews, and internal controls, will yield performance data of sufficient quality to support decisionmaking.

Some performance plans, such as NASA’s and SSA’s, did not appear to describe any verification and validation procedures that these agencies used or expected to use to ensure that performance data are sufficiently complete, accurate, and consistent. Most of the plans provided descriptions of procedures for verifying and validating performance data, but these descriptions were often superficial. For example, although SBA’s plan included a general discussion of verification and validation procedures, it did not specify how the agency would ensure that its performance data are credible. Specific verification and validation systems, related measures, and the milestones for verification and validation generally are not cited in the plans. The Nuclear Regulatory Commission’s (NRC) plan discussed validating the list of primary systems and measuring levels of satisfaction with the accuracy and availability of information in the systems, but it did not discuss how NRC intends to actually validate these data to ensure that they are accurate and complete.

The performance plans of some other agencies, such as Labor, mentioned only that their Inspectors General (IGs) would be responsible for auditing the agencies’ data systems. Agencies that expected to use the IGs generally suggested that the review of performance data would be done as part of the annual financial audit of the agency. The IGs and external auditors can make important contributions toward ensuring that performance data are valid, but these contributions cannot substitute for management actions to ensure that the data are sound. Moreover, these plans generally did not discuss whether agencies had coordinated with their IGs to perform this work. Agencies and IGs need to jointly determine how the IGs’ resources could best support verification and validation efforts, given the IGs’ continuing audit responsibilities. In making this determination, agencies need to carefully consider the most appropriate means for verifying and validating performance information.

Education and HHS’ Indian Health Service (IHS) proved to be exceptions regarding the verification and validation of performance data. Their performance plans included a variety of specific and credible procedures to ensure that Congress, agency managers, and other decisionmakers will have performance information of sufficient credibility to support decisions. For example, Education’s plan included such procedures as a mix of audits, independent external reviews, and program evaluations, as well as the scope and timing of what its IG would be undertaking. IHS’ plan included such procedures as performing editing checks, monitoring the reasonableness of data, and developing software to allow for the transmission of data to a centralized database.

Separate from the issues associated with the need to ensure that performance data are verified and valid, we continue to be concerned about the lack of a capacity in many federal agencies to undertake the program evaluations that will be vital to the success of the Results Act. In reviewing agencies’ strategic plans, we found that many agencies had not given sufficient attention to how program evaluation will be used in implementing the Results Act and improving performance. More recently, we reported that agencies’ program evaluation capabilities would be challenged to meet the new demands for information on program results. We found that the resources allocated to conducting program evaluations were small and unevenly distributed across the 13 departments and 10 independent agencies we surveyed for that report.

The findings of that report are a major concern because a federal environment that focuses on results—where federal efforts are often but one factor among many that determine whether and if goals are achieved—depends on program evaluations to provide vital information about the contribution of federal efforts. For example, the success of the United States Agency for International Development (USAID) in achieving its intended results is affected by many factors and programs beyond USAID’s control. Development programs of the international donor community and the governments and institutions within the developing countries themselves all can have greater or lesser influences on advancing social and economic development. USAID, as well as other agencies, can use program evaluations to help isolate the degree to which its efforts are contributing to results and what actions it specifically can take to better meet its goals.

Recognizing Data Limitations

In general, agencies’ annual performance plans did not include discussions of known data limitations and strategies to address them. Such limitations can be a significant challenge to performance measurement. Over the years, we and others have identified problems with the financial and information systems at several agencies. The recent governmentwide financial statement audit further raised concerns about the reliability of data. The amount of progress still needed to obtain high-quality financial data suggests the types of challenges that agencies will face in obtaining high-quality performance data.

Agencies face particular challenges when they must rely on other organizations to provide important performance information. For example, State’s performance plan acknowledges that the agency is to rely on data from external sources to measure performance. However, the plan did not describe how limitations in the quality of that data would affect efforts to assess and improve performance. These data limitations included inconsistencies in data collection from location to location; from year to year; or from one data source to another, especially when data from more than one source must be combined to measure performance. The plan would be more useful if it recognized and identified significant data limitations and their implications for assessing performance.

Education’s performance plan contained a good example of an agency’s recognition of data limitations. In the quality of performance data section of its plan, Education stated that ensuring the accurate and efficient collection of its student loan data is vital to achieving one of its strategic goals. However, we and Education’s IG have previously reported about the inadequacy of Education’s student loan data. Because of these inadequacies, Education had been unable to report on the Department’s financial position in a complete, accurate, and reliable manner. In its performance plan, Education acknowledged that its student aid delivery system has suffered from significant data quality problems. The plan outlined several steps the Department plans to take to address these problems, including improving data accuracy by establishing industrywide standards for data exchanges, receiving individual student loan data directly from lenders, expanding efforts to verify data reported to the National Student Loan Data System, and preparing a systems architecture for the delivery of federal student aid.

Opportunities to Improve Future Plans

The fiscal year 1999 annual performance plans represent the first attempt across all executive agencies to carry out the annual performance planning called for under the Results Act. Although these plans collectively suggested that annual performance planning, as established under the Act, can be a powerful device for better informing congressional and executive branch decisionmaking, substantial further development is needed before the plans will consistently be able to support that goal. In crafting the Results Act, Congress understood—and similar foreign experiences confirmed—that effectively implementing management changes of the magnitude envisioned under the Act would take several cycles, although each cycle should see marked improvements over the preceding ones.

Both OMB’s fiscal year 1999 annual performance plan and the fiscal year 1999 governmentwide performance plan contain commitments to implement the Results Act. The OMB annual performance plan includes a goal to improve the performance of government programs by meeting the statutory requirements of the Act. The governmentwide performance plan includes Results Act implementation as 1 of 22 priority management objectives that the administration will focus on in 1999. According to a senior OMB official, OMB incorporates lessons learned from performance plans in the guidance it provides agencies on developing future plans. OMB has also committed to reviewing agencies’ subsequent plans to ensure that improvements and appropriate changes are made. On the basis of our review of the agency plans, it appears that OMB can build on its commitment and design and implement a broad, aggressive performance planning improvement effort. Specifically, our work suggests that giving priority attention to the following key opportunities for improvement will lead to the greatest increases in usefulness:

Better articulating a results-orientation. Agency performance plans could be more useful if they more consistently incorporated results-oriented goals and showed more direct relationships among goals and measures. More results-oriented agency goals will also facilitate understanding the relationships among agencies’ efforts and planned contributions and goals included in the governmentwide performance plan. The value of agencies’ annual performance plans also could increase if the plans consistently included useful and informative baseline and trend data, as some agencies did in their 1999 performance plans. Such data provide decisionmakers with a context for assessing whether performance targets are appropriate and reasonable.

The value of the performance plans could also be augmented if they more fully included goals that addressed mission-critical management issues (for example, historic problems in maximizing the use of information technology). Precise and measurable goals for resolving mission-critical management problems are important to ensuring that the agencies have the institutional capacity to achieve their more results-oriented programmatic goals. Consistently including goals in individual agency plans to address mission-critical management issues also will facilitate the integration of governmentwide and agency performance planning processes. Section IV of the governmentwide annual performance plan is devoted to improving performance through better management and lists the administration’s priority management objectives. We found in our review of the governmentwide plan that the clarity and effectiveness of OMB’s discussion of the objectives in that plan could be improved by a more integrated and focused discussion of the strategies associated with the objectives. Augmented agency performance plans can be helpful in this regard by showing that agencies, where appropriate, are positioned to address governmentwide priority management objectives, such as the Year 2000 computing crisis. To facilitate this effort, we recommend in our review of the governmentwide performance plan that OMB ensure that the governmentwide management priorities and performance goals contained in the governmentwide performance plan be reflected in relevant agency performance plans.

Coordinating crosscutting programs. Our work has suggested that program overlap and mission fragmentation are important issues that need to be addressed. We also have noted, consistent with OMB’s guidance, that a focus on results implies that crosscutting programs will be coordinated. At the time of our reviews, many agencies’ annual performance plans identified crosscutting efforts, with some listing other agencies with which they shared the same or similar result, but the substantive work of coordination was not yet apparent. Specifically, few of the plans showed evidence of the work necessary to ensure that crosscutting programs have mutually reinforcing goals; complementary strategies; and, as appropriate, common performance measures. Not surprisingly, given the amount of coordination that still needs to take place, in our review of the fiscal year 1999 governmentwide performance plan we found that substantial opportunities exist for enhancing the discussion of crosscutting efforts in that plan as well.

By building on the initial progress that some agencies have made, the usefulness of performance plans could be enhanced if all agencies more consistently identified the results-oriented annual performance goals that involve other federal agencies and set intermediate goals that clarify the agency’s specific contribution to the common result. Moreover, because of the still early state of coordination of crosscutting programs, the more useful plans will continue to describe relevant interagency coordination efforts.

Clearly showing how strategies will be used to achieve goals. Although not explicitly required by the Results Act, the more useful annual performance plans discussed how the strategies and approaches would lead to results. The listings of current programs and initiatives that often were included in agencies’ plans are useful in providing an understanding of what agencies do. Presentations that more directly explain how programs and initiatives achieve goals will be most helpful to Congress as it assesses the degree to which strategies are appropriate and reasonable. Discussions of external factors and how different governing tools (for example, intergovernmental partnerships, performance-based contracts, financial credits) will be, or can be, used in achieving goals could further enhance the plans. Such discussions could also assist in the development of a base of governmentwide information on the strengths and weaknesses of various tools in addressing differing public policy issues.

Showing performance consequences of budget decisions. The Results Act was intended to help Congress develop a clearer understanding of what is being achieved in relation to the money being spent. In the fiscal year 1999 performance plans, agencies generally covered all of the program activities in their budget requests. However, most plans did not clearly convey the requested funding level associated with achieving a discrete set of performance goals and clearly identify where that funding was included in the structure of agencies’ budget requests. Agencies, OMB, and Congress can take advantage of three initiatives—budget and program activity changes, the implementation of cost accounting, and the initiation of performance budgeting pilots—to help ensure that performance plans better convey the performance consequences of budget decisions.

Congress and OMB have clearly expressed a willingness to consider changes in agencies’ budget account and/or program activity structures in future years to more clearly and readily relate expected performance to funding requests. For example, in its fiscal year 1998 appropriations reports, the House Appropriations Committee stated that it would consider any requests for program activity changes that ensure that budget submissions display amounts requested against program activity structures for which annual performance goals and measures have been established. Similarly, OMB’s Circular A-11 encouraged agencies to consider proposing changes to the budget account structure to facilitate an understanding of performance.

In addition to more closely linking expected performance to agency budget requests, Congress, in crafting the Results Act expected that agencies, whenever possible, would develop performance measures that correlated the level of program activity with program costs, such as costs per unit of result, costs per unit of service, or costs per unit of output. Agencies were expected to assign a high priority to developing these types of unit cost measures. The successful implementation of the managerial cost accounting standards recommended by the Federal Accounting Standards Advisory Board and issued by OMB and GAO are vital to providing agencies the program cost information needed to develop such performance measures.

The Results Act also demonstrates Congress’ interest in determining the extent to which performance can be related to changes in funding levels. The Act requires the Director of OMB to designate at least five federal agencies to participate in a 2-year pilot in performance budgeting in which the budgets of those agencies will display varying levels of performance that would result from different budgeted amounts for one or more of an agency’s major functions or operations. Agencies’ progress in establishing reliable cost accounting systems and allocating resources to performance goals, as well as progress in defining goals and measuring performance, may affect how OMB designs and determines participation in the performance budgeting pilots. More broadly, as agencies continue to define relationships between performance planning and budget structures, Congress, OMB, and agencies can explore whether changes in budget presentations can provide agencies with needed flexibility and accountability while ensuring appropriate congressional oversight and control.

Building capacity within agencies to gather and use performance information. Our work suggests that few agencies have adequate procedures in place to ensure that the performance data generated will be of sufficient quality to confidently make decisions. The financial audits under the CFO Act—where only 10 of the 24 CFO Act agencies have been able to obtain an unqualified opinion from independent auditors—have shown how far most agencies have to go to be able to generate reliable year-end financial information. It is important that agencies continue to make progress on developing financial systems that can produce timely financial information throughout the year and work with their IGs to explore how the IGs can contribute to improving the credibility of performance data. Moreover, our recent work continues to show that many agencies are not well-positioned to undertake the program evaluations that will be critical to identifying why goals are not met and determining the best improvement strategies. The relatively limited level of agencies’ evaluation capabilities suggests that evaluation resources will need to be carefully targeted and coordinated with nonfederal evaluation efforts to ensure that key questions about program results are adequately addressed.

In our Executive Guide, we noted that leading results-oriented organizations consistently strive to ensure that their day-to-day activities support their organizational missions and move them closer to accomplishing their strategic goals. We reported that in practice, these organizations see the production of a strategic plan—that is, a particular document issued on a particular day—as one of the least important parts of the planning process. Annual performance plans should be viewed the same way. The performance improvements expected under the Results Act will not occur because an agency has issued strategic and annual performance plans. Rather, performance improvements occur when agency managers and external decisionmakers use those documents and the planning and management processes that underpin them.

Because fiscal year 1999 is to mark the first year of governmentwide implementation of the Results Act’s annual performance planning requirements, Congress and the agencies lack a common base of experience for how the performance-based approach to management envisioned under the Results Act can best be used to support congressional and executive branch decisionmaking. Building this base of experience will require ensuring that performance-based management is integrated into the way programs are managed and decisions are made. The importance of building this base of experience also suggests that any successful effort to improve the usefulness of agency performance plans will require the active partnership of Congress, OMB, and the agencies because of the potentially broad use of such plans both within Congress and the executive branch.

Conclusions

We found that on the whole, future annual performance plans would be more useful if they provide clearer pictures of intended performance across an agency, more fully articulate what strategies and resources will be used to achieve goals and how those strategies and resources will lead to improved performance, and provide much greater confidence that performance information will be credible and useful for decisionmaking. OMB’s efforts to identify lessons learned from the agencies’ fiscal year 1999 performance plans and its commitment in the fiscal year 1999 governmentwide performance plan to review agencies’ subsequent plans to ensure improvements are made are a first step. However, the need for progress across all agencies and the range of annual planning issues that need to be addressed underscore the scope of the effort that lies ahead and suggest that a more concerted, active, and specific improvement agenda needs to be developed and put in place.

Beyond improving the quality of agencies’ written plans, experience is needed in using those plans to inform congressional and executive branch decisionmaking. In this regard, it is vital that all agencies begin implementing their fiscal year 1999 annual performance plans in October 1998 and seek to prepare improved plans for fiscal year 2000. However, the limited experience with the use of the Results Act at the federal level thus far suggests that targeting key program areas for special congressional and executive branch attention can help agencies develop a common base of experience in using Results Act principles and processes to drive performance and management improvements. A coordinated OMB, congressional, and agency effort could be helpful in three ways:

First, the effort could develop a body of specific examples that demonstrate where congressional and executive branch use of the performance-based approach to management and accountability contained in the Results Act helped to inform decisionmaking. Initially, the evidence for this use will be seen in such areas as improved and more focused program management within agencies; more informed executive branch and congressional budget decisions, including the better alignment of performance planning and budgeting processes; and better information available and used as part of congressional authorization and oversight efforts. Most importantly, over time, the use of performance plans should lead to substantial improvements in program performance.

Second, a common OMB, congressional, and agency focus on selected program areas also will aid in the development of a set of agreed-upon “best practices” in performance planning and the integration of results-oriented performance information into decisionmaking and management. Including programs that represent a cross-section of service delivery mechanisms will provide insights into how the various tools of government (for example, regulation, direct service, intergovernmental funding, tax expenditures, loans, or loan guarantees) can be used individually and together to address public policy issues. This will aid in building an understanding of how implementation of the Results Act may differ—such as in the nature of federal goals—depending on the specific characteristics of different tools.

Third, a common focus on using the Results Act to make decisions for selected program areas can also assist in the identification of experienced-based similarities and differences in the congressional and executive branch needs for the content of annual performance plans and any changes to OMB guidance and Results Act statutory requirements that may be necessary to better meet those needs.

Recommendation to the Director of OMB

To fully implement OMB’s commitment to evaluate its and agencies’ experience in developing the fiscal year 1999 performance plans and to improve agencies’ performance plans for the future, we recommend that the Director, OMB, implement a concrete agenda aimed at substantially enhancing the usefulness of agencies’ performance plans for congressional and executive branch decisionmaking. The five key opportunities for improvement that we identified—better articulating a results orientation; coordinating crosscutting programs; clearly showing how strategies will be used to achieve goals; showing performance consequences of budget decisions; and building capacity within agencies to gather and use performance information, including program evaluation—can serve as core elements of the improvement effort. For example, OMB could work with agencies to ensure that annual performance plans include presentations that more directly explain how agency programs and initiatives will be achieved. Similarly, discussions of external factors and how different governing tools (e.g., intergovernmental partnerships, performance-based contracts, financial credits) will be, or can be, used in achieving goals would help enhance the usefulness of agencies’ plans.

To go beyond the formal requirements of the Results Act to issue annual performance plans and performance reports, and to build a base of experience for how the performance-based approach to management envisioned under the Results Act can be used to improve program results and support congressional and executive branch decisionmaking, we also recommend that the Director of OMB work with Congress and the agencies to identify specific program areas that can be used as best practices. This would help demonstrate the use and benefits of results-oriented management and where concrete information about program results contributes directly to executive branch and congressional decisionmaking. This effort could also assist Congress in identifying and considering opportunities to merge and expand results-oriented performance information into existing authorization, oversight, and appropriations processes. For the effort to be most effective, several criteria should be used to identify specific program areas, such as those program areas where agreement exists between Congress and the administration that the areas likely will be on legislative and oversight agendas; those programs that have the most direct influence on meeting the central Results Act purpose of improving citizens’ confidence in government; and programs needing priority management attention, such as those listed in the fiscal year 1999 governmentwide performance plan.

Agency Comments and Our Evaluation

On July 17, 1998, we provided a draft of this report to the Director of OMB for comment. We did not provide a draft to individual agencies discussed in this report because the reports we prepared on individual agency plans in response to your request were provided to the relevant agencies for comment. Those comments were reflected, as appropriate, in the final versions of those reports.

On August 19, 1998, we received OMB’s written comments; see appendix IV for a copy of the letter from the Acting Deputy Director for Management. On August 20, 1998, we received additional technical comments from a senior OMB official; we have incorporated these comments where appropriate. OMB’s August 19, 1998, letter includes comments on both this report and our companion report on the fiscal year 1999 governmentwide performance plans. Our evaluation of OMB comments on this report are provided below; OMB’s comments on our assessment of the governmentwide performance plan are discussed in our companion report.

OMB generally agreed with our observations and said that the report was an expansive portrait of the fiscal year 1999 plans that contained many useful suggestions. OMB did, however, raise two related issues about the report. OMB commented that the report predominantly focuses on what was included or lacking in the annual performance plans rather than on how the plans would be used to provide better services and products to the American public and to improve the quality and nature of programming, funding, and management decisions made within the executive branch and by Congress. OMB also commented that the report does not clearly distinguish between major and secondary elements of an annual plan, such as between factors that are and are not required by statute.

We agree with OMB that the use of annual performance plans by congressional and executive branch decisionmakers is the essential indicator of the effective implementation of the Results Act. As we said in our report, performance improvements will not occur because an agency issues a plan, but rather when plans—and the planning and management processes that underpin them—are used by agency managers and other decisionmakers. However, as our report shows, significant improvements are needed in agencies’ plans before they will be useful in a significant way to congressional and executive branch decisionmakers. Thus, we reported on the presence or absence of characteristics that affect the usefulness of annual performance plans and identified the key opportunities for improving those plans. Similarly, because we focused our review on the elements that are most important to developing useful plans, we disagree with OMB’s comment that we did not distinguish between major and secondary elements of performance plans. Moreover, as we note in the report, the major elements of our evaluation—goals and performance measures, program strategies, and the existence of valid performance data—are based on specific criteria set forth in the Results Act, the Act’s legislative history, or OMB’s guidance to agencies on preparing performance plans.

We are sending copies of this report to the Minority Leader of the House; the Ranking Minority Members of your Committees; Committee Chairmen who requested our review of the fiscal year 1999 governmentwide annual plan and the Ranking Minority Members of their respective Committees; other appropriate congressional committees; and the Director, Office of Management and Budget. We will also make copies available to others on request.

The major contributors to this report are listed in appendix V. Please contact L. Nye Stevens on (202) 512-8676 or Paul L. Posner on (202) 512-9573 if you or your staff have any questions.

Agencies Covered by the Chief Financial Officers Act

Scope and Methodology

To summarize our observations on agencies’ fiscal year 1999 annual performance plans and to help us identify opportunities for agencies to improve future performance plans, we analyzed the information contained in our reviews of the annual performance plans of the 24 CFO Act agencies. Our review of each of the agencies’ performance plans and our summary analysis of all 24 plans were based on our guides for congressional and evaluator review of annual performance plans. For purposes of assessing the plans, we collapsed the Results Act’s requirements for annual performance plans into the three core questions that structure those guides. The three questions are:

To what extent does the agency’s performance plan provide a clear picture of intended performance across the agency?

How well does the performance plan discuss the strategies and resources the agency will use to achieve its performance goals?

To what extent does the agency’s performance plan provide confidence that its performance information will be credible?

We used the questions and associated issues contained in the guides to help us identify strengths and weaknesses in the performance plans, with a particular focus on assessing the overall usefulness of the plans for congressional and other decisionmakers. In doing our summary analysis, we examined and classified our reviews of the individual agency plans as related to the questions, issues, and criteria in the guides to discern any themes or trends and then to develop an overall, descriptive characterization of our observations and judgments about the agencies’ plans. We also reviewed parts of selected agencies’ annual performance plans, as needed, to supplement our analysis of our individual agency reviews and to elaborate further on particular issues. To further help us identify opportunities for agencies to improve future performance plans, we also drew on other related work, including our recent reports on Results Act implementation.

We reviewed agency performance plans from February through June 1998 and did our work according to generally accepted government auditing standards. We requested comments from the Director of OMB on a draft of this report. OMB’s comments are discussed in the “Agency Comments and Our Evaluation” section in this report. In addition, we provided drafts of our individual reviews on agencies’ plans to the relevant agencies for comment. The agencies’ comments are reflected, as appropriate, in our products on their respective plans.

GAO Products on Agencies’ Fiscal Year 1999 Annual Performance Plans

Departments

Results Act: Observations on the U.S. Department of Agriculture’s Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98R, June 11, 1998).

Results Act: Observations on the Department of Commerce’s Annual Performance Plan for Fiscal Year 1999 (GAO/GGD-98-135R, June 24, 1998).

Results Act: DOD’s Annual Performance Plan for Fiscal Year 1999 (GAO/NSIAD-98-188R, June 5, 1998).

The Results Act: Observations on the Department of Education’s Fiscal Year 1999 Annual Performance Plan (GAO/HEHS-98-172R, June 8, 1998).

Results Act: Observations on DOE’s Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98-194R, May 28, 1998).

The Results Act: Observations on the Department of Health and Human Services’ Fiscal Year 1999 Annual Performance Plan (GAO/HEHS-98-180R, June 17, 1998).

Results Act: Observations on the Department of Housing and Urban Development’s Fiscal Year 1999 Annual Performance Plan (GAO/RCED-98-159R, June 5, 1998).

Results Act: Department of the Interior’s Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98-206R, May 28, 1998).

Observations on the Department of Justice’s Fiscal Year 1999 Performance Plan (GAO/GGD-98-134R, May 29, 1998).

Results Act: Observations on Labor’s Fiscal Year 1999 Performance Plan (GAO/HEHS-98-175R, June 4, 1998).

The Results Act: Observations on the Department of State’s Fiscal Year 1999 Annual Performance Plan (GAO/NSIAD-98-210R, June 17, 1998).

Results Act: Observations on the Department of Transportation’s Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98-180R, May 12, 1998).

Results Act: Observations on Treasury’s Fiscal Year 1999 Annual Performance Plan (GAO/GGD-98-149, June 30, 1998).

Results Act: Observations on VA’s Fiscal Year 1999 Performance Plan (GAO/HEHS-98-181R, June 10, 1998).

Independent Agencies

Results Act: EPA’s Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98-166R, Apr. 28, 1998).

Results Act: Observations on the Federal Emergency Management Agency’s Fiscal Year 1999 Annual Performance Plan (GAO/RCED-98-207R, June 1, 1998).

Results Act: Observations on the General Services Administration’s Annual Performance Plan (GAO/GGD-98-110, May 11, 1998).

Managing for Results: Observations on NASA’s Fiscal Year 1999 Performance Plan (GAO/NSIAD-98-181, June 5, 1998).

Results Act: NSF’s Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98-192R, May 19, 1998).

Results Act: NRC’s Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98-195R, May 27, 1998).

Results Act: Observations on the Office of Personnel Management’s Annual Performance Plan (GAO/GGD-98-130, July 28, 1998).

Results Act: Observations on the Small Business Administration’s Fiscal Year 1999 Annual Performance Plan (GAO/RCED-98-200R, May 28, 1998).

The Results Act: Observations on the Social Security Administration’s Fiscal Year 1999 Performance Plan (GAO/HEHS-98-178R, June 9, 1998).

The Results Act: Observations on USAID’s Fiscal Year 1999 Annual Performance Plan (GAO/NSIAD-98-194R, June 25, 1998).

Comments From the Office of Management and Budget

Major Contributors

General Government Division

Accounting and Information Management Division

Acknowledgements

The examples used in this report are drawn from the assessments of the individual agency annual performance plans that were done by staff across GAO. Thus, in addition to the individuals noted above, the staff who worked on the individual agency plan assessments also made important contributions to this report. These individuals are identified in the separate reports on agency plans.

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.