This report is based on our reviews of 24 major agencies’ strategic plans that were formally submitted to Congress and OMB by September 30, 1997. To do these reviews, we used the Results Act supplemented by OMB’s guidance on developing the plans (Circular A-11, part 2) as criteria to determine whether the plans contained the six elements required by the Act. As agreed, we focused our reviews on the progress of agencies’ strategic planning efforts, specifically their efforts to improve their strategic plans, with particular attention to the key planning challenges that are most in need of sustained attention. Agencies included in our analysis are listed in appendix I, and our observations on individual agencies are summarized in appendixes II through XXV. To gather information on how annual performance planning and measurement could be used to address the critical planning challenges we observed in our reviews of the September plans, we relied on our recent report on critical challenges needing sustained attention, our report on governmentwide implementation of the Results Act, our guidance for congressional review of Results Act implementation, and our guidance on effectively implementing the Act. We reviewed individual agency plans from September 30, 1997, through November 1997. Our work was conducted in accordance with generally accepted government auditing standards. We provided a draft of this report for comment to the Director of OMB on January 5, 1998; a discussion of OMB’s comments appears at the end of this letter. In addition, we provided drafts of the appendices we prepared on individual agency plans to the relevant agencies for comment. The comments from those agencies are summarized in the relevant appendixes. The Results Act is the centerpiece of a statutory framework that Congress put in place during the 1990s to help resolve the long-standing management problems that have undermined the federal government’s effectiveness and efficiency and to provide greater accountability for results. In addition to the Results Act, the framework comprises the CFO Act and information technology reform legislation, including the Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of 1996. Congress enacted the CFO Act to remedy decades of serious neglect in federal financial management by establishing chief financial officers across the federal government and requiring the preparation and audit of annual financial statements. The information technology reform legislation is based on the best practices used by leading public and private sector organizations to manage information technology more effectively. Under the Results Act, strategic plans are the starting point and basic underpinning for performance-based management. In our report on agencies’ draft strategic plans, we noted that complete strategic plans were crucial if they were to serve as a basis for guiding agencies’ operations and be used to help congressional and other policymakers make decisions about activities and programs. The Act requires that an agency’s strategic plan contain six key elements. These elements are (1) a comprehensive agency mission statement; (2) agencywide long-term goals and objectives for all major functions and operations; (3) approaches (or strategies) and the various resources needed to achieve the goals and objectives; (4) a description of the relationship between the long-term goals and objectives and the annual performance goals; (5) an identification of key factors, external to the agency and beyond its control, that could significantly affect the achievement of the strategic goals; and (6) a description of how program evaluations were used to establish or revise strategic goals and a schedule for future program evaluations. Building on the decisions made as part of the strategic planning process, the Results Act requires executive agencies to develop annual performance plans covering each program activity set forth in the agencies’ budgets. The first annual performance plans, covering fiscal year 1999, are to be submitted to Congress after the President’s budget is submitted, which is approximately February 1998. Each plan is to contain an agency’s annual performance goals and associated measures, which the agency is to use in order to gauge its progress toward accomplishing its strategic goals. OMB is to use the agencies’ performance plans to develop an overall federal government performance plan that is to be submitted with the President’s budget. The performance plan for the federal government is to present to Congress a single cohesive picture of the federal government’s annual performance goals for a given fiscal year. Agencies’ September plans appear to provide a workable foundation for the continuing implementation of the Results Act. These plans represent a significant improvement over the draft plans we reviewed last summer.In those reviews, we found that all but six of the draft strategic plans were missing at least one required element, and about a third were missing two of the six required elements. In addition, just over a fourth of those plans failed to cover at least three of the required elements. Moreover, we found that many of the elements included in the plans contained weaknesses—some that were more significant than others. The agencies, on the whole, made a concerted effort during August and September to improve their plans. For example, all of the September plans we reviewed contained at least some discussion of each element required by the Act. And, in many cases, those elements that contained weaknesses were substantially improved by September. For example: The Department of Transportation explained more clearly how its mission statement is linked to its authorizing legislation. The Small Business Administration (SBA) improved its ability to assess progress toward its strategic goals by stating when specific performance objectives would be met. The Nuclear Regulatory Commission (NRC) better explained the scope of its crosscutting functions by identifying major crosscutting functions and interagency programs and its coordination with those agencies. The Department of Education improved its discussion of external factors that could affect its achievement of strategic goals by describing agency actions to mitigate against those factors. Appendixes II through XXV contain our observations on the progress and remaining challenges of individual agencies’ strategic planning efforts. Although the September plans appear to provide a workable foundation for the continuing implementation of the Results Act, we found that critical planning challenges remain. Among the remaining critical challenges are (1) clearly establishing a strategic direction for agencies by improving goal-setting and measurement; (2) improving the management of crosscutting program efforts by ensuring that those programs are appropriately coordinated to avoid duplication, fragmentation, and overlap; and (3) ensuring that agencies have the data systems and analytic capacity in place to better assess program performance and costs, improve management and performance, and establish accountability. The forthcoming annual performance planning and measurement processes offer agencies an opportunity to make progress in addressing these challenges. By improving on their draft strategic plans, agencies’ September plans undertook the first steps toward setting a strategic direction for their programs and activities. However, we found that the September plans often lacked clear articulation of the agency’s strategic direction: (1) strategic goals and objectives were not as measurable and results oriented as possible, (2) linkages among planning elements were not clear, and (3) strategies for achieving those goals and objectives were incomplete or underdeveloped. However, the performance planning and measurement phase of the Results Act offers agencies an opportunity to continue to refine their strategic directions. In our reviews of agencies’ September plans, we found that some agencies have begun to address the challenge of setting a strategic direction. For example: The most notable improvement in the plan for the Department of Health and Human Services (HHS) is the inclusion of an outline of strategic objectives for accomplishing the Department’s six strategic goals. Those objectives are largely focused on outcomes and are defined in measurable terms. This plan also identifies for each strategic objective the key measures of progress. For example, one measure of progress for the outcome-oriented objective of “reducing the use of illicit drugs” is “death rate of persons aged 15 to 65 attributed to drug use.” The September plans of the Departments of Agriculture, Education, and the Treasury now include helpful matrixes to link various planning elements, such as goals, objectives, measures, and programs or responsible organizational components. These matrixes are also useful in assessing a plan’s underlying logic, determining programmatic accountability, and identifying crosscutting programs and potential duplication and overlap among program efforts. For example, Treasury’s September plan contained an appendix that identified which bureau or office is responsible for achieving its Department-wide goals and objectives. The September plan for the Department of Energy (DOE) better explains how it plans to accomplish many of its goals. The plan provides greater specificity on the money, staff, workforce skills, and facilities that the agency plans to employ to meet its goals. For example, to support its national security goal, DOE’s plan says it will need to change the skills of its workforce and how it constructs new experimental test facilities. Although improvements were not isolated to these agencies, we also found that agencies need to further clarify their strategic directions if the Results Act is to be effective in guiding the agencies and informing congressional and other decisionmakers. The goals and objectives of many agencies could be more results oriented and expressed in a manner that will better allow for a subsequent assessment of whether the goals and objectives have been achieved. For example, the plan for the Department of Veterans Affairs (VA) contains the following objectives supporting the goal for its compensation and pension area: “(1) evaluate compensation and pension programs and (2) modify these programs, as appropriate.” Also, although the first goal in the Social Security Administration’s (SSA) September plan “o promote valued, strong, and responsive social security programs and conduct effective policy development, research, and program evaluation” sets a strategic direction for the agency, it could be stated in more measurable terms to better enable the agency to make a future assessment of whether it is being achieved. Another challenging area for agencies in setting strategic direction in the September plans was to establish linkages among planning elements, such as goals, objectives, and strategies. For example, Treasury’s plan says that the Internal Revenue Service (IRS) has a role in three law enforcement objectives—to reduce counterfeiting, money laundering, and drug smuggling. However, the IRS plan contained no specific strategy to help achieve any of those objectives. In another example, the September plan for the Federal Emergency Management Agency (FEMA) included lists of objectives and strategies under each goal with no explanation of how the strategies would contribute to achievement of the objectives. Another weakness of agencies’ September plans was incomplete and underdeveloped strategies for achieving long-term strategic goals and objectives. More specifically, we found that agencies did not always provide an adequate discussion of the resources needed to achieve goals. For example, SBA’s September plan did not contain any discussion on the resources, such as human resources and information technology, needed to achieve its goals. Although other plans we reviewed discussed resources, the discussions were incomplete. For example, few plans discussed the physical capital resources, such as facilities and equipment, needed to achieve their goals. Although many agencies may not rely heavily on physical capital resources, even the plans of some of those that do, such as the General Services Administration and the National Park Service, a component of the Department of the Interior, did not provide a focused discussion of their capital needs and usage. The role that information technology played, or can play, in achieving agencies’ long-term strategic goals and objectives was generally neglected in the September plans. The government’s track record in employing information technology is poor, and the strategic plans we reviewed often contained only limited discussions of technology issues. For example, most of the Department of Defense’s (DOD) strategic goals are fundamentally linked to information technology. However, we have placed DOD’s management of critical information management processes on our high-risk list. We believe DOD’s strategic plan would be significantly enhanced if it more explicitly linked its strategic goals to a strategy for improving management and oversight of information technology resources. Additionally, DOD should recognize the dramatic impact the Year 2000 problem will likely have on its computer operations, including the mission-critical applications identified in its strategic plan. The Department of State’s September plan also does not specifically address the serious deficiencies in State’s information and financial accounting systems. Rather, the plan notes, in more general terms, that it will take State several years to develop performance measures and related databases in order to provide sufficient information on achievement of its long-term goals. The lack of such a discussion in many of the plans is of particular concern because, without it, agencies cannot be certain that they are (1) addressing the federal government’s information technology problems and (2) better ensuring that technology acquisition and use are targeted squarely on program results. Strategic planning—setting a strategic direction for agency operations—did not end with the submission of a strategic plan to Congress last September. Performance-based management, as envisioned by the Results Act, is not a linear, sequential process but, rather, an iterative one in which strategic and performance planning cycles will result in subsequent revisions to both strategic and annual performance plans. Each cycle of strategic planning and performance planning, particularly in the first few years of governmentwide implementation of the Results Act, will likely result in agencies making significant changes and improvements in those documents. Consequently, agencies can continue to address the critical planning challenges associated with setting a strategic direction as they develop their first annual performance plans. For example, the process of defining targeted levels of performance within set time frames and providing baselines against which to compare actual performance will likely produce opportunities for agencies to revisit and improve upon their strategic goals and objectives so that those goals are as results oriented and measurable as they can be. If successfully developed, those annual performance goals can function as a bridge between long-term strategic planning and day-to-day operations, thereby assisting agencies in establishing better linkages among planning elements. For example, agencies can use performance goals to show clear and direct relationships in two directions—to the goals in the strategic plans and to operations and activities within the agency. By establishing those relationships, agencies can (1) provide straightforward roadmaps that show managers and staff how their daily activities can contribute to attaining agencywide strategic goals, (2) hold managers and staff accountable for contributing to the achievement of those goals, and (3) provide decisionmakers with information on their annual progress in meeting the goals. As agencies gain experience in developing these annual performance goals, they likely will become better at identifying and correcting misalignment among strategic goals, objectives, and strategies within their plans. The importance of clearly showing how strategies are linked to goals is underscored by the Results Act requirement that annual goals are to be based on budgetary program activities. Unlike previous federal reform initiatives, the Results Act requires agencies to plan and measure performance using the same program activity structures that form the basis for their budget requests. However, we have found that the relationships among the budget structures, performance plans, and strategic plans will require coordinated and recurring attention by Congress, OMB, and agencies as they move to implement the annual performance planning and measurement phase of the Act. This attention is important because the wide variability of the budget structures indicates that the suitability of those structures for the Results Act’s performance planning and measurement will also vary. For example, we reported in 1997 that agency officials we spoke with confirmed the varying suitability of their program activity structures for the Results Act’s purposes. One agency successfully worked through its recent performance-planning process using its existing program activities. A second agency had a program activity structure that reflected its organizational units—a structure that is useful for traditional accountability purposes, such as monitoring outputs and staff levels—but less useful for results-oriented planning. Still other agencies separated performance planning from program activity structures, believing it necessary to first establish appropriate program goals, objectives, and measures before considering the link to the budget. These agencies planned to rely on the Results Act’s provision to aggregate, disaggregate, or consolidate program activities in constructing their annual performance plans. In addition, annual performance planning can be used to better define strategies for achieving strategic and annual performance goals. For example, annual performance plans provide agencies with another opportunity to further discuss strategies for information technology investments and the operational improvements expected from those investments. The annual performance plans should also provide annual performance measures that Congress and other decisionmakers can use to determine if those investments are achieving the expected improvements. Thus, annual performance planning and measurement can provide decisionmakers with an early warning of information investment strategies that need to be revisited. A focus on results, as envisioned by the Results Act, implies that federal programs that contribute to the same or similar results should be closely coordinated to ensure that goals are consistent and, as appropriate, program efforts are mutually reinforcing. We have found that uncoordinated program efforts can waste scarce funds, confuse and frustrate program customers, and limit the overall effectiveness of the federal effort. This suggests that federal agencies are to look beyond their organizational boundaries and coordinate with other agencies to ensure that their efforts are aligned and complementary. Agencies’ September plans show progress in this area, but coordination of crosscutting programs continues to be a strategic planning challenge. During the summer of 1996, in reviewing early strategic planning efforts, OMB alerted agencies that augmented interagency coordination was needed at that time to ensure consistency among goals in crosscutting programs areas. However, the draft strategic plans we reviewed during the summer of 1997 often lacked evidence that agencies in crosscutting program areas had worked with other agencies to ensure that goals were consistent; strategies were coordinated; and, as appropriate, performance measures were similar. Agencies’ September plans better described crosscutting programs and coordination efforts. Some plans, for example, contained references to other agencies that shared responsibilities in a crosscutting program area or discussed the need to coordinate their programs with other agencies. For example, as noted earlier, NRC better explained its crosscutting functions in its September plan. In addition, the Environmental Protection Agency’s (EPA) plan contains an appendix that lists the federal agencies with which EPA coordinated. This appendix describes the major steps in the coordination process and lists by strategic goal the agencies with which EPA believes greater integration and review of efforts will be needed. Similarly, the plan for the Department of Transportation contains a table that shows the contributions of other federal agencies to each of its major mission areas. NRC’s, EPA’s and Transportation’s plans illustrate the kind of presentation that could be especially helpful to Congress and the administration in identifying program areas to monitor for overlap and duplication. These presentations, and similar ones in other agencies’ September plans that identify agencies with crosscutting programs, also provide a foundation for the much more difficult work that lies ahead—undertaking the substantive coordination that is needed to ensure that those programs are effectively managed. For example, in an improvement over its draft plan, the Department of Labor’s September plan refers to a few other agencies with responsibilities in the area of job training programs and notes that Labor plans to work with them. However, the plan contains no discussion of what specific coordination mechanism Labor will use to realize efficiencies and possible strategies to consolidate job training programs to achieve a more effective job training system. Our work has shown that the next phases of the Results Act’s implementation will offer a structured framework to address crosscutting issues. For example, the Act’s emphasis on results-based performance measures as part of the annual performance planning process should lead to more explicit discussions concerning the contributions and accomplishments of crosscutting programs and encourage related programs to develop common performance measures. As agencies work with OMB to develop their annual performance plans, they can consider the extent to which agency goals are complementary and the need for common performance measures to allow for cross-agency evaluations. Also, the Results Act’s requirement that OMB prepare a governmentwide performance plan that is based on the agencies’ annual performance plans can be used to facilitate the identification of program overlap, duplication, and fragmentation. Our work also indicates that if agencies and OMB use the annual planning process to highlight crosscutting program efforts and provide evidence of joint planning and coordination of those efforts, the individual agency performance plans and the governmentwide performance plan should help provide Congress with the information needed to identify agencies and programs addressing similar missions. Once these programs are identified, Congress can consider the associated policy, management, and performance implications of crosscutting program efforts and whether individual programs make a sufficiently distinguishable contribution to a crosscutting national issue. This information should also help identify the performance and cost consequences of program fragmentation and the implications of alternative policy and service delivery options. These options, in turn, can lead to decisions concerning department and agency missions and the allocation of resources among those missions. Our previous work has shown that agencies need to have reliable data during their planning efforts to set realistic goals and later, as programs are being implemented, to gauge their progress toward achieving those goals. In addition, in combination with an agency’s performance measurement system, a strong program evaluation capacity is needed to provide feedback on how well an agency’s activities and programs contributed to achieving its goals and to identify ways to improve performance. However, our work has also found serious shortcomings in agencies’ ability to generate reliable and timely data to measure their progress in achieving goals and to provide the analytic capacity to use that data. The Results Act’s requirement that annual performance plans discuss the verification and validation of data provides agencies with an opportunity to be forthcoming about data limitations and to show how those limitations will be addressed. Verified and validated performance information, in conjunction with augmented program evaluation efforts, will help ensure that agencies are able to report progress in meeting goals and identify specific strategies to improve performance. The absence of both sound program performance and cost data and the capacity to use those data to improve performance is a critical challenge that agencies must confront if they are to effectively implement the Results Act. Efforts under the CFO Act have shown that most agencies are still years away from generating reliable, useful, relevant, and timely financial information, which is urgently needed to make our government fiscally responsible. The widespread lack of available program performance information is equally troubling. For example, in our June report on a survey of managers in the largest federal agencies, we found that fewer than one-third of those managers said that results-oriented performance measures existed to a great or very great extent for their programs. Our work also suggests that even when performance information exists, its reliability is frequently questionable. For example, our work has shown that the reliability of performance data currently available to a number of agencies is suspect, because the agencies must rely on data collected by parties outside the federal government. In a recent report, we noted that the fact that data were largely collected by others was the most frequent explanation offered by agency officials for why determining the accuracy and quality of performance data was a challenge. In our June 1997 report on the implementation of the Results Act, we also reported on the difficulties that agencies were experiencing as a result of their reliance on outside parties for performance information. Agencies are required under the Results Act to describe in their annual performance plans how they will verify and validate the performance information that will be collected. This section of the performance plan can provide important contextual information for Congress and agencies to address the weaknesses in this area. For example, this section can provide an agency with the opportunity to alert Congress to the problems the agency has had or anticipates having in collecting needed results-oriented performance information. Agencies can also use this section to alert Congress to the cost and data quality trade-offs associated with various collection strategies, such as relying on sources outside the agency to provide performance data and the degree to which those data are expected to be reliable. The discussion in this section can also provide Congress with a mechanism for examining whether the agency currently has the data to confidently set performance improvement targets and will later have the ability to report on its performance. More broadly, continuing efforts to implement the CFO Act also are central for ensuring that agencies resolve their long-standing problems in generating vital information for decisionmakers. In that regard, the Federal Accounting Standards Advisory Board (FASAB) has developed a new set of accounting concepts and standards that underpin OMB’s guidance to agencies on the form and content of their agencywide financial statements. As part of that effort, FASAB developed managerial cost accounting standards that were to be effective for fiscal year 1997. These standards are to provide decisionmakers with information on the costs of all resources used and the costs of services provided by others to support activities or programs. Such information would allow for comparisons of costs across various levels of program performance. However, because of serious agency shortfalls in cost accounting systems, the Chief Financial Officers Council—an interagency council of the CFOs of the major agencies—requested an additional 2 years before the standard would be effective. FASAB recommended extending the date by 1 year, to fiscal year 1998, with a clear expectation that there would be no further delays. Under the Results Act, another aspect of performance planning is a requirement for agencies to discuss the use and planned use of program evaluations that can provide feedback on how well an agency’s activities and programs contributed to the achievement of its goals and to assess the reasonableness and appropriateness of those goals. However, our recent report on agencies’ draft plans stated that 16 of the 27 draft plans did not discuss program evaluations. Although all the September plans included discussions of program evaluations, we continued to find weaknesses in those discussions. However, this is not surprising because agencies that had not undertaken program evaluations prior to the preparation of the first cycle of strategic plans would not likely be able to discuss in their September plans how they used program evaluations to help develop the plans. Of greater concern, many agencies, including the Departments of Health and Human Services, Justice, and Labor, also did not discuss how they planned to use evaluations in the future to assess progress or did not offer a schedule for future evaluation as required by the Results Act. In contrast, the National Science Foundation’s September plan contains a noteworthy exception to this trend. The plan discusses how the agency used evaluations to develop key investment strategies, action plans, and its annual performance plan. It also discusses plans for future evaluations and provides a general schedule for their implementation. Over the longer term, the program performance information that agencies are to generate under the Results Act should be a valuable new resource for Congress to use in its program authorization, oversight, budget, and appropriation responsibilities. As we have noted before, to be most useful in these various contexts, that information needs to be consolidated with budget data and critical financial and program cost data, which agencies are to produce and have audited under the CFO Act. This consolidated program performance, cost, and budget information, in conjunction with the annual performance plans, should provide congressional and other decisionmakers with a more complete picture of the results, operational effectiveness, and costs of agencies’ operations. Agencies, on the whole, made significant progress in improving their plans during August and September 1997. The strategic plans they formally submitted to Congress and OMB in September 1997 appear to provide a workable foundation for the continuing implementation of the Results Act. Nonetheless, the critical planning challenges that we found demonstrate that the effective implementation of performance-based management and accountability, as envisioned by the Results Act, is still, as to be expected, very much a work in progress. Since performance-based management is not a linear, sequential process but, rather, an iterative one, each subsequent strategic and performance planning cycle can, and likely will, result in revisions to preceding planning documents. Therefore, Congress, OMB, and agencies’ senior managers can use the next stage of performance-based management —performance planning and measurement—to ensure that agencies continue to address the critical planning challenges as well as maintain momentum on the implementation of the Results Act. On January 5, 1998, we provided a draft of this report to the Director of OMB for comment. We provided drafts of the appendixes we prepared on individual agency plans to the relevant agencies for comment, and the comments from those agencies are summarized in the relevant appendixes. On January 13, 1998, a senior OMB official provided us with OMB’s comments on this report. He generally agreed with our observations and said that the report was a useful compilation of our work on agencies’ September strategic plans. The official also said that this report underscores that the implementation of the Results Act will be an ongoing, iterative process in which agencies will learn from their initial experiences in developing strategic plans and can then apply those lessons learned as they continue to develop strategic planning processes. In addition, the official provided technical comments that were incorporated in this report. As arranged with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after its issue date. At that time, we will send copies of this report to the Minority Leader of the House; the Ranking Minority Members of your Committees; other appropriate congressional committees; and the Director, Office of Management and Budget. We will also make copies available to others on request. If you have any questions concerning this report, please contact me on (202) 512-8676. On July 10, 1997, we issued a report on the U.S. Department of Agriculture’s (USDA) May draft strategic plan (Results Act: Observations on USDA’s Draft Strategic Plan, GAO/RCED-97-196R). USDA’s publicly issued strategic plan was submitted to the President and Congress on September 30, 1997. As requested, we have reviewed USDA’s September strategic plan and compared the results of our assessment with our observations on the draft plan, as reported in July. On October 17, 1997, we briefed your staffs on our assessment of the September strategic plan. The key points from that briefing are summarized herein. USDA’s May 1997 draft strategic plan included a Department-wide strategic overview as well as 30 plans for the mission areas, agencies, and staff offices that make up the Department. We reviewed the overview and the 16 agency plans that are directly related to accomplishing USDA’s mission and implementing its programs. We also reviewed the plans for the offices of the Chief Financial Officer and the Chief Information Officer. We observed that the May draft strategic plan did not fulfill the requirements of the Results Act. USDA’s overall mission and goals were contained in the Department-wide strategic overview; the overview then referred the reader to the agencies’ plans for information on the six required elements. However, only one of the agencies’ plans we reviewed contained all six required elements. The draft strategic plan also fell short in several other areas necessary for achieving the purposes of the Results Act. Among other things, the draft strategic plan lacked an emphasis on externally focused goals and objectives, adequate quantifiable performance measures, and good linkages between the agencies’ goals and the Department’s goals. We also reported that we could not determine the extent to which coordination with other federal agencies, both within and outside the Department, occurred in the formulation of the draft strategic plan. It was also unclear whether agencies’ goals and objectives had been assessed for duplication and complementary functions. USDA’s Department-wide strategic overview acknowledged the role of USDA agencies that carry out similar and/or complementary functions but did not recognize the role of other federal agencies. Many of the agencies’ plans generically recognized the roles of other federal agencies in accomplishing their missions. However, there was little evidence in either the Department-wide strategic overview or the agencies’ plans to suggest that the agencies coordinated with other agencies—internally or externally—when developing their goals and objectives. USDA’s draft plan addressed some, but not all, of the high-risk issues and management problems we had previously identified. Generally, information on how USDA planned to address these high-risk issues and management problems, such as the need to reduce losses in the farm loan program, was included as goals and objectives in the agencies’ plans. However, USDA’s draft plan did not address some management issues, such as the need to reform milk marketing orders, improve the management of agricultural trade programs, and strengthen financial controls under credit reform. In addition, we have identified significant, long-standing Department-wide problems in information technology, accounting, and financial management. However, USDA’s draft strategic plan did not adequately recognize and address these problems. For example, the plan for the Office of the Chief Information Officer lacked time frames and milestones and the resources needed to accomplish the stated goals. We also noted that it lacked an explanation of how the goals were specifically linked to the agencies’ plans. USDA made significant improvements in its September strategic plan. This plan incorporates many changes that make it more responsive to the requirements of the Results Act. The strategic plan complies with the six elements required by the Results Act and includes many of the key attributes necessary for a quality plan. It also includes information on some management challenges that we identified in the past. While all 16 agencies’ plans contain the six required elements, the clarity of information presented varies across the plans. For example: Most of the agencies’ plans have comprehensive and concise mission statements. However, the mission statements for two agencies’ plans—concerning the Agricultural Marketing Service and Rural Development—are stated so broadly that it is difficult to determine what the basic purpose of the agency is or how it differs from that of other agencies. For example, it is unclear how the mission of the Agricultural Marketing Service differs from the missions of the Grain Inspection, Packers and Stockyards Administration and the Foreign Agricultural Service. Most of the agencies’ plans have results-oriented goals and objectives. However, some plans—those of the Farm Service Agency, Food and Consumer Service, Animal and Plant Health Inspection Service, and the Forest Service—have too many goals and objectives structured around existing programs and activities rather than the ultimate results that these agencies should achieve. For example, the Farm Service Agency’s plan has four goals that we believe could be combined under two that would fulfill the agency’s mission—(1) improving the economic viability of the agriculture sector and (2) protecting the environment. All of the agencies’ plans provide more detailed strategies and improved information on the resources needed for achieving goals and objectives, compared with the information provided in the May draft plan. All of the agencies’ plans provide a detailed discussion of the external factors beyond the control of the agency that could affect the achievement of the goals. However, the linkages between external factors and their impact on specific goals could be improved in some plans, such as the plans for the four research agencies. Unlike the May draft, in which only 1 of the 16 agencies’ plans included information on the relationship between annual performance goals and strategic goals, all of the agencies’ September plans include this information. However, the quality of the descriptions provided in this section of the agencies’ plans varies by agency. For example, some plans, such as those for the Food and Consumer Service, Farm Service Agency, Food Safety and Inspection Service, and Center for Nutrition Policy and Programs, easily allow the reader to envision how the annual performance goals relate to the strategic goals; other agencies’ plans, such as those for the Economic Research Service and the Risk Management Agency, are less clear. Most of the agencies’ plans provide greater detail than they did in the May draft on how program evaluations were used to develop the strategic plan and how they will be used in the future. However, two plans—those for the Food Safety and Inspection Service and the Agricultural Research Service—state that program evaluations were not used to develop the strategic plan, although information on program evaluations planned for the future is included; and the plan for the Agricultural Marketing Service states that program evaluations were not used to develop the plan and are not planned for the future. While these agencies state that they did not use formal program evaluations when developing their plans, the information provided in the plans indicates that the results of relevant studies and assessments were actually used to help develop the plans—which in our opinion meets the requirements of the Results Act. Consequently, we believe that these agencies may be using too narrow a definition for the term “program evaluation.” According to an August 7, 1997, letter, sent by the House Majority Leader to the Director, OMB, program evaluations should include all significant evaluations relevant to the development and future assessment of an agency’s plan. The letter suggested that this definition include reviews by the Inspector General, GAO, and others that deal with program implementation and operating policies and practices. Moreover, we found that many of the key attributes necessary for a quality plan, which were missing in the May draft plan, have been included in the September strategic plan. These include clear linkages between the agencies’ goals and their statutory authorities as well as the Department-wide goals; a better focus on external goals rather than internal processes (the result of a separation of strategic goals from management initiatives); and a more complete discussion of relevant performance measures, although some agencies are still developing baseline information and targets. For targets included in the plans, it is sometimes unclear whether they are annual or 5-year targets. Some of the management challenges facing USDA that we raised in the past have been included in the September plan. For example, reform of the milk marketing orders is included in the Agricultural Marketing Service’s plan as an objective. Similarly, USDA revised its strategic plan to address certain accounting and financial management issues that the draft plan did not adequately address. For example, the strategic plan reflects USDA’s efforts to strengthen controls for establishing and reestimating loan subsidy costs, as required under credit reform. Also, the strategic plan recognizes that additional staff and resources may be needed to ensure that USDA can accomplish the goals set out in the plan for the Office of the Chief Financial Officer. In addition to the suggestions that we have made herein to improve the clarity of some agencies’ plans, some more significant aspects of the strategic plan could be further improved. These improvements include (1) explaining interagency coordination for crosscutting issues and (2) addressing previously identified management problems. USDA’s September strategic plan provides more detailed information about other agencies—both internal and external to the Department—that share responsibilities for achieving the stated goals and objectives. The Department-wide strategic overview now includes links to agencies outside of the Department that are important partners to USDA agencies. In addition, the agencies’ plans not only identify the agencies that they coordinate and consult with, in some cases they also identify the specific roles of these other agencies. However, we still could not determine from the information provided in most of the agencies’ plans whether consultations actually took place with these agencies to resolve crosscutting issues. Moreover, we could not determine whether an assessment of duplicative or complementary programs and activities was performed when the agencies were developing their goals and objectives. In addition, we found that while many agencies’ plans explain that stakeholders were consulted during the plan’s development, they usually do not clearly identify the stakeholders. Although this information is not required to be included in the strategic plan by the Results Act, we believe that including information in the agencies’ plans that clearly identifies all stakeholders would be helpful. In addition, the September plan still does not include two management issues that we identified in the past. In particular, the Foreign Agricultural Service’s plan still does not address the numerous problems we have identified in agricultural trade programs. Furthermore, there is little evidence to suggest that substantial progress has been made in addressing our concerns about information technology. Although USDA has added time frames for completing the 14 objectives appearing in its Office of Chief Information Officer’s plan, each time frame has a completion date “through FY 2002.” We are concerned about the absence of earlier time frames, or at least interim ones, for resolving major Department-wide information technology problems, such as the Year 2000 issue. By establishing such time frames, it is not clear what priority USDA is really placing on solving its information technology problems or whether the Department has adequate strategies for doing so. In addition, although the Office of Chief Information Officer’s plan includes a number of goals and objectives to better manage its $1 billion in annual investments for information technology, we remain concerned about the lack of information in the plan on the resources needed to accomplish these goals and objectives and how they link to the agencies’ plans. We provided a draft of our observations on USDA’s strategic plan for the Department’s review and comment. We met with USDA’s Acting Chief Financial Officer and the Director, Planning and Accountability Division, Office of Chief Financial Officer, who told us that they were pleased that we had recognized the significant improvements made to the strategic plan and that the additional comments made by us would help them as they continue to refine and enhance the plan. In addition, USDA made the following observations: USDA disagreed with our statement that program evaluations were not used to develop the Animal and Plant Health Inspection Service’s plan. While we agree that this plan recognizes the importance of using program evaluations to set performance goals, it does not clearly identify how the results of program evaluations were used to develop the strategic plan. Consequently, we have deleted this statement from our report to reflect the agency’s comment, but we would suggest that the Animal and Plant Health Inspection Service add language to clarify how program evaluations were used to develop its plan. USDA noted that while there is no duplication of services between the Agricultural Marketing Service and the Grain Inspection, Packers and Stockyards Administration and the Foreign Agricultural Service, the mission statement of the Agricultural Marketing Service would be clarified, in future versions of the plan, to distinguish it from the mission statements of the other two agencies. In connection with our observation about the Food and Consumer Service’s plan having too many goals that were structured around current programs rather than results, USDA told us that the Food and Consumer Service had considered structuring its plan around a smaller number of generic goals. However, the agency chose to establish six goals corresponding to its existing programs because it believed that a plan structured in this manner would be more meaningful to all interested parties, including external partners and program participants. While we agree that setting up goals around familiar programs and activities may make the plan easier to understand, this approach may ultimately defeat the purpose of the Results Act—which is to require agencies to focus on outcomes by reevaluating what they do and why they do it. Therefore, we would suggest that the Food and Consumer Service consider restructuring the goals in its plan around broader outcomes rather than current programs. USDA disagreed with our statement that the Foreign Agricultural Service plan still does not address the numerous problems that we have identified in the past relating to agricultural trade programs. For example, USDA believes that the Foreign Agricultural Service has addressed the concerns outlined in our report entitled U.S. Department of Agriculture: Foreign Agricultural Service Could Benefit From Better Strategic Planning (GAO/GGD-95-225, Sept. 28, 1995) by including information on agency resource allocation, overseas priorities, and trade opportunities under the management initiatives section of the plan. According to USDA, other issues raised by us, such as streamlining the agency’s foreign service, will be addressed in the annual performance plan. Although we agree that some issues that we have raised in the past can be appropriately addressed by including them in the annual performance plan, others cannot. Over the past decade, we have issued a series of reports that raise serious concerns about the fundamental operations of the Foreign Agricultural Service’s export programs, such as the Foreign Market Development Program, Market Access Program, P.L. 480 Program, and Export Credit Guarantee Program. We believe that solutions to these problems will require long-term planning that has not been adequately addressed in the strategic plan. Finally, USDA stated that it did not believe that the Office of Chief Information Officer’s strategic plan had to be the medium to address specific solutions to the individual agencies’ issues identified in previous audits. Our observations on the Office of Chief Information Officer’s plan, however, did not discuss the need for specific solutions; rather, we noted that the plan lacked sufficient information on time frames, resources, and how the goals and objectives were linked to other USDA agencies’ plans. We believe that such information is essential to clearly identify what priority USDA is placing on solving its information technology problems and determining whether the Department has adequate strategies for addressing these issues. This is especially important given the Secretary of Agriculture’s May 1997 direction to subcabinet officials that fixing USDA’s long-standing, pervasive information technology management problems must be a top priority. USDA also disagreed with our statement that there is a perceived lack of attention on the Year 2000 issue. While we recognize that the plan discusses the Year 2000 issue, we are concerned about the stated time frames for completion for this objective. By stating a “through FY 2002” completion time frame for the Year 2000 problem, we believe that the plan does not present an adequate strategy for resolving one of USDA’s most pressing information technology management problems and one that must be solved within the next 2 years. Robert A. Robinson, Director, Food and Agriculture Issues; Resources, Community, and Economic Development Division, (202) 512-5138. On July 14, 1997, we issued a report on the Department of Commerce’s draft strategic plan (The Results Act: Observations on Commerce’s June 1997 Draft Strategic Plan, GAO/GGD-97-152R). Commerce’s formally issued strategic plan was submitted to OMB and Congress on September 30, 1997. As requested, we reviewed Commerce’s revised strategic plan, compared it with the earlier draft version that we reported on in July, and identified significant changes or improvements that Commerce made in the areas covered by our July report and areas or required plan elements where additional improvements still could be made as the plan evolves. We briefed your staffs on our findings on October 17, 1997. Our findings are summarized herein. Commerce’s draft strategic plan was inadequate and incomplete in several respects. Of the six elements required by the Results Act, four were included in the draft plan—a mission statement, goals and objectives, strategies for achieving goals and objectives, and a discussion of key external factors—but each of these had weaknesses, some more significant than others. For example, the mission statement included the core functions of the Department and mentioned the role of businesses and universities but not the important role also played by other government entities. While there were useful linkages among themes, goals, objectives, and responsible components, the goals and objectives were not as results oriented as they could be. The draft plan identified the Commerce bureau responsible for each goal and objective but did not adequately discuss strategies for achieving those goals and objectives or include required information describing the operational processes, staff skills, and technologies, as well as the human, capital, information, and other resources needed to achieve them. Many but not all key external factors were discussed, but the factors that were identified appeared to be used to justify programs rather than to show how those factors could affect the achievement of goals. Commerce’s draft strategic plan did not explicitly discuss the other two elements required by the Results Act—the relationship between long-term goals and objectives and annual performance goals and the description of program evaluations used to establish general goals and objectives and a schedule for future program evaluations. The draft plan said that relating long-term goals and objectives to annual performance goals will more appropriately be done in the Department’s future annual budget requests. The draft plan made limited references in various sections to a few past studies of Commerce programs, but those references did not describe how the studies were used to establish general goals and objectives, and the draft plan did not provide a schedule for future program evaluations. Concerning other plan components, the draft plan provided much useful information on Commerce’s statutory authorities. However, the draft plan could have been more useful to Commerce, Congress, and other stakeholders if it had provided a more explicit discussion of crosscutting activities, the major management challenges the Department faces, and the Department’s ability to provide reliable financial and other management and program information to measure achievement of its goals. Commerce’s publicly issued strategic plan incorporated improvements in several areas and now addresses, to some extent, all of the elements required by the Results Act. The improvements that the Department made are steps in the right direction and address some but not all of the weaknesses discussed in our July 1997 report on an earlier draft of the plan. The plan’s discussions of strategic goals have been expanded to briefly indicate Commerce’s strategy for achieving each goal. For example, under the theme of keeping America competitive with cutting-edge science and technology, the National Oceanic and Atmospheric Administration (NOAA) has a goal to “predict and assess decadal to centennial change.” The plan now describes how NOAA will approach this goal by addressing questions dealing with air quality, ozone depletion, greenhouse warming, and climate change. Also, the plan now more explicitly acknowledges the need to link strategic goals and objectives to annual performance goals and includes an illustrative performance measure for each of the objectives under the three strategic themes. For example, the illustrative performance measure for two of the Patent and Trademark Office’s (PTO) objectives is “reduced pendency time.” This illustrative performance measure is one of several that addresses PTO’s goal of granting exclusive rights for limited times to inventors for their discoveries. Similarly, the plan’s three strategic theme chapters now more strongly emphasize the importance of external factors that could affect achievement of Commerce’s strategic goals and identify more key external factors. Under the economic infrastructure strategic theme, for example, the plan now includes a reference to the International Trade Administration’s (ITA) strategy to identify obstacles to U.S. exports and plans for removing such obstacles and marshaling U.S. government resources to eliminate barriers. Commerce’s revised strategic plan includes new sections on program evaluations, interagency linkages, and major management challenges. The new section on the role of program evaluations discusses current evaluations as well as future evaluation plans, provides examples, highlights the difficulties in specifying the level and focus of future evaluations because of year-to-year competition for funds, and states that future evaluations for many Commerce bureaus will be included in annual performance plans and budgets. The new section on interagency linkages acknowledges the importance of close interagency ties and emphasizes the Department’s commitment to strengthen those ties by reaching out to other federal agencies with complementary responsibilities. In addition, the partnership sections of the three strategic theme chapters now more fully identify and discuss Commerce’s shared mission responsibilities with other federal agencies. Under the economic infrastructure theme, for example, the plan now emphasizes those aspects of Commerce’s mission that are complementary. It points out that Commerce chairs the Trade Promotion Coordinating Committee (TPCC), a 20-member interagency task force charged by the President and Congress with developing and implementing the National Export Strategy. The new section on management challenges recognizes and discusses three of the key management challenges facing the Department that were highlighted in our July report—weather service modernization, Census 2000, and financial management systems. Finally, the usefulness of the plan has been improved by the addition of an index or matrix, which shows which Commerce bureaus are responsible for which strategic themes and goals; and an appendix, which provides clearer and more comprehensive information on, and consolidates in one place in the plan, the statutory and other authorities for the Department and its bureaus, themes, and goals and objectives. While the overall quality of Commerce’s strategic plan has been improved since we reported in July 1997 on an earlier draft of the plan, further improvements still could be made in each of the elements required by the Results Act. As we indicated in our July report, the mission statement could be made more complete by explicitly recognizing that several other federal agencies as well as state and local governments also play major roles in the areas covered by Commerce’s three strategic themes. In the export controls area under the economic infrastructure theme, for example, the plan acknowledges that Commerce shares mission responsibilities with the Departments of Defense, Energy, and State and the Arms Control and Disarmament Agency, but the mission statement does not recognize this or other shared responsibilities. Similarly, the treatment of crosscutting functions could clarify Commerce’s role in the three strategic theme areas, specify how the Department’s efforts intersect with or complement the efforts of the other participants, and identify which other government entities Commerce coordinated its plan with and the results of that coordination. The Department’s September 30, 1997, letter transmitting the revised plan to Congress said that Commerce consulted with stakeholders, provided them and congressional committees with copies of its draft plans, and responded to stakeholder and congressional comments. According to Commerce’s transmittal letter, there were no unresolved contrary views concerning its plan. The strategies for achieving each strategic goal could be further expanded to specify how Commerce will hold its bureaus and managers accountable for meeting strategic goals and the resources that will be required to meet them. The linkages between long-term strategic goals and objectives and annual performance goals could be improved by (1) making the illustrative performance measures more outcome oriented, such as by using the “number of counseling sessions” as a measure of ITA’s economic infrastructure objective to “increase trade assistance targeted to small and medium-sized businesses,” or (2) showing how the performance measures that were added cause results. The discussion of external factors could identify and discuss more key factors beyond Commerce’s direct control that could affect achievement of its goals, such as congressional concerns about the Census Bureau’s plans for conducting Census 2000, and specify how the external factors that are identified will be addressed or mitigated. The discussion of program evaluations could indicate more specifically how evaluations were used to establish goals/objectives and performance measures. Finally, the discussions of Commerce’s major management challenges and its capacity to provide reliable data on performance could acknowledge and discuss more of the major management challenges and data capacity problems that we emphasized in our July 1997 report, such as managing modern information technology and the “year 2000 computer problem.” Also, the plan could relate identified management challenges, including performance measurement limitations, to Commerce’s strategic goals and objectives, discuss their implications for achievement of its strategic goals and objectives, and indicate more specifically how and when the Department expects to overcome these challenges. The plan could be made more useful to stakeholders and would better meet the intent of the Act if it identified and discussed these types of problems as well as other material weaknesses or high-risk areas, such as NOAA’s fleet for acquiring marine data, that are disclosed in Commerce’s Federal Managers’ Financial Integrity Act reports or financial statements. Given the diversity of its programs and activities and its bureaus’ independence, Commerce faced an especially formidable challenge in developing its strategic plan. The Department developed a “thematic” strategic plan that covers its major functions and activities; is consistent with relevant statutory and other authorities; and addresses, to some extent, the various elements required by the Results Act. The plan’s readability, usefulness, and overall effectiveness as a planning and oversight tool could be enhanced by streamlining its organization and content to eliminate many of the details that do not relate directly to the Act’s requirements, thus reducing its 178-page length. We provided a copy of a draft of this briefing document to the Department of Commerce for review and comment. On October 17, 1997, the Director for Budget, Management and Information and Deputy Chief Information Officer provided us with written comments. He characterized our review as balanced and fair and said that the Department clearly agrees that it needs to do more planning with other agencies and crosscutting programs and that this is a very high departmental priority. In this regard, he said that the Department has stepped forward as the lead agency to link with the National Academy of Public Administration (NAPA) in forming the Performance Consortium and that a dozen other federal departments and agencies have joined Commerce in this effort to develop common planning activities and elements. The Director also said that the Department disagrees with our suggestion that its plan could be improved by providing additional information in certain areas and eliminating many of the details that do not relate directly to the Act’s requirements. He said that the Department made a specific decision to have a single, integrated strategic plan that covers all its bureaus. The Department believes that its plan demonstrates clearly how the Commerce bureaus fit together and provide critical service to the nation and that it addresses some of the administration’s key priorities and secures the buy-in of its bureaus. As Commerce’s strategic plan evolves, we continue to believe that its readability and specificity could be improved by streamlining its organization, content, and presentation. L. Nye Stevens, Director, Federal Management and Workforce Issues; General Government Division, (202) 512-8676. On August 5, 1997, we issued a report on DOD’s draft strategic plan (The Results Act: Observations on DOD’s Draft Strategic Plan, GAO/NSIAD-97-219R). The Department of Defense’s formally issued Results Act strategic plan was submitted to OMB and Congress on September 30, 1997. As requested, we reviewed the strategic plan and compared its changes to DOD’s draft plan. On October 17, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from that briefing are summarized herein. Our prior evaluation revealed that DOD’s draft plan included discussions of each of the six critical components required in strategic plans but that some were of higher quality than others. We noted, for example, that DOD’s draft plan contained a succinct mission statement and general goals and objectives that cover its major functions and operations and reflect its broad statutory defense responsibilities, but it did not include schedules for initiating and completing significant actions to achieve its goals. We also noted that, although DOD included some discussion of other elements, such as formidable management problems, these discussions could be more complete. And, we suggested several improvements to the draft plan, including that DOD (1) more completely state strategies for achieving its goals and include schedules of significant actions; (2) link and discuss how external factors could affect its ability to achieve its goals; (3) discuss how program evaluations were used in developing its goals and identify key issues for future evaluations; (4) discuss planned or ongoing actions to resolve persistent management problems, including time frames and required resources; and (5) identify and discuss coordination efforts for programs that crosscut with other agencies’ programs. Finally, we suggested that DOD develop one clear and succinct document to serve as its strategic plan. DOD revised its general goals and objectives to provide a clearer presentation. In line with the revision, DOD also rearranged its description of how the performance goals it is developing will be related to the general goals in an effort to improve the description’s clarity. It also defined some terms and included some additional information in the rearranged description. DOD’s general goals and objectives as reworded still cover its major functions and operations and reflect its broad statutory defense responsibilities. DOD also included a table listing the major management problems that we have identified as high-risk areas and the documents, such as the DOD Logistics Strategic Plan, that address each of the high-risk areas. However, it did not include an explanation of what actions will be taken to address the high-risk areas and when the problems in these areas are expected to be corrected. Additionally, DOD did not adopt our other suggested improvements, nor did it consolidate the strategic plan into one succinct document. We believe that DOD’s strategic plan could be further improved by adopting the suggestions we made in our August 5, 1997, report (summarized herein). We believe that addressing these areas would provide decisionmakers and stakeholders the information necessary to ensure that DOD has well-thought-out strategies for resolving ongoing problems, achieving its goals and objectives, coordinating crosscutting activities, and becoming more results oriented, as expected by the Results Act. DOD officials reiterated that the QDR, alone, has been the Department’s finalized strategic plan since it was issued in May 1997. They also stated that they included a table listing the underlying plans that address the high-risk management problems noted in our August 5 report but did not attach or include significant detail from the underlying plans because that would have made their submission too voluminous. They said that they did not include details in summary fashion because that would not have provided enough information. They noted that DOD is working to address its management problems and said that those interested in seeing how the problems are being addressed should read the underlying plans. Additionally, the officials noted that although coordination of programs and activities that crosscut other agencies’ programs are not discussed in DOD’s strategic plan, DOD coordinates and cooperates extensively with other federal agencies as part of its ongoing strategic planning process. Finally, DOD officials commented that in congressional consultations, the only change suggested was that DOD reword a couple of its general goals and objectives. David R. Warren, Director, Defense Management Issues; National Security and International Affairs Division, (202) 512-8412. On July 18, 1997, we issued a report on the Department of Education’s draft strategic plan (The Results Act: Observations on the Department of Education’s June 1997 Draft Strategic Plan, GAO/HEHS-97-176R). Education’s formally issued strategic plan was submitted to OMB and Congress on October 3, 1997. As requested, we have reviewed the publicly issued strategic plan and compared it with the observations in our July 18 report. On October 16, 1997, we briefed your staffs on further observations on the strategic plan. The key points from that briefing are summarized herein. The Department’s June 17, 1997, draft plan generally complied with the Results Act. Overall, it is a useful document and included all but one of the six elements required by the Act—it did not discuss how the agency’s long-term goals and objectives will be related to its annual performance goals. The plan’s long-term goals and objectives were succinct and logically linked to its mission statement, and the quality of the goals and objectives reflected the Department’s thoughtful deliberation in its efforts to comply with the Results Act. In addition, the plan addressed in some form all of the Department’s major statutory responsibilities. Although the plan presented a logical and fairly complete description of how the Department intends to achieve its mission, we identified a few areas in the draft plan that could be improved. We observed that the plan could benefit from more information, clarity, and context in some of its components. The plan should have included an explanation of the relationship between its long-term goals and objectives and its annual performance goals as well as a complete description and schedule of program evaluations. It could also have better addressed the Department’s major statutory responsibilities. The Department has the primary responsibility for implementing federal education policy and programs, but several other federal agencies also provide education-related programs and services. In our past work, we have identified opportunities for consolidating programs in certain areas, such as job training and early childhood education, to eliminate inappropriate duplication. The draft strategic plan did a good job of identifying crosscutting program activities in elementary and secondary programs, but it did not identify or discuss activities for postsecondary programs that require coordination. By discussing the agencies and activities involved with the Department’s higher education programs, the strategic plan could provide Congress with a more complete picture of the scope of the Department’s coordination activities. In its discussion of core strategies for achieving its strategic goals and objectives, the Department identified several management challenges it will face in the coming years, but it provides little detail about these challenges and how it will meet them. This type of information could help the Department and its stakeholders identify major management problems that could impede the Department’s efforts to achieve its goals and objectives. Further, stakeholders could benefit from knowing what the Department has done, is doing, or plans to do to address such problems. The Department’s strategic plan issued October 3, 1997, included several significant improvements that make it more responsive to the requirements of the Results Act than its draft plan. The Department’s plan now addresses all six elements required by the Results Act. The plan addressed the relationship between the agency’s long-term goals and objectives and its annual performance plan—the only element missing from its draft plan—by including a matrix linking long-term goals and objectives in the strategic plan with fiscal year 1997 appropriation information and agency programs. The matrix indicated where programs have a significant number of activities or products supporting an objective. Though the strategic plan does not specifically describe how the Department intends to measure the performance of its programs each year, the matrix and the supplemental information on the Department’s performance indicators (shown in appendix A of the plan) provided a better understanding of the relationship between the Department’s strategic and annual performance plans. The Department states that the strategic plan was based, in part, on objectives and indicators in draft program performance plans prepared for key programs in the winter of 1997. According to the plan, the annual performance plan (which includes budget and performance plans for each of the Department’s programs) will further clarify this linkage. The Department’s October 1997 plan also provided a description of the program evaluations and assessments that were used to develop each of its four strategic goals as well as evaluations that will help to “inform the implementation” of the plan and provide data for the performance indicators supporting the goals. For example, at the end of the narrative describing Goal 2 (build a solid foundation for learning for all children), the plan stated that early evaluations of the Even Start program and crosscutting evaluations of Goals 2000 and the reauthorized elementary and secondary education programs were used to develop this goal. In addition to the evaluations highlighted in the introduction of each goal, appendix B of the plan described 57 key program evaluations and other studies, including information on when the evaluation data were or will be collected and, in many instances, how often the data will be collected in the future. The Department’s strategic plan also identified agency efforts that will help to avoid duplication among its evaluation efforts and reduce respondent burden. In addition, the narrative supporting the Department’s mission statement now encompasses the Department’s major statutory responsibilities. Our review of the Department’s draft plan indicated that it had failed to address the agency’s statutory requirements for basic education for adults, vocational rehabilitation, education of individuals with disabilities, and school-to-work opportunities. In its October 1997 plan, the Department addressed this weakness by including as one of its key agency functions “providing grants for literacy, employment, and self-sufficiency.” The plan more clearly addressed the Department’s civil rights function within the goals and objectives sections. During our review of the Department’s draft plan, we observed that the agency’s civil rights function, although reflected in its mission statement, was not addressed in the plan’s long-term goals or objectives. In support of two objectives related to goals 1 and 4, the Department added strategies for addressing its civil rights function. Objective 1.5 is to get families and communities fully involved with schools and school improvement efforts. In support of this objective, the plan states that the Department will create collaborative partnerships among parents, community groups, and other stakeholders that ensure equal educational opportunity, and provide civil rights training and technical assistance to build these linkages. Objective 4.2 is to provide Education’s partners the support and flexibility they need without diminishing accountability. In support of this objective, the plan adds the following strategy: to build civil rights partnerships to achieve shared civil rights objectives and secure timely improvements for students. As required by the Results Act, the Department described in its draft plan several factors outside the agency’s program scope and responsibilities that could negatively affect its ability to achieve its strategic goals. The Department strengthened this discussion in the plan by describing agency actions intended to mitigate against seven key external factors that could affect the achievement of its long-term goals. For example, the plan states that school systems will need to undertake long-term investments in professional development and other capacity-building activities if education reforms are to succeed. Yet, pressures outside of the Department’s control may encourage school systems to focus instead on demonstrating short-term gains. To counter these pressures, the plan states that the Department will (1) work with program and technical assistance providers to highlight the importance of sustained professional development aligned with the standards and (2) emphasize the importance of professional development in its performance indicators. Consistent with our suggestions in our July report, the Department’s strategic plan also addressed several other issues. The plan specifically identified coordination activities related to the Department’s postsecondary education programs and activities. It listed interagency coordination and data matches with, for example, the Social Security Administration, the Immigration and Naturalization Service, and the Selective Service as a strategy for ensuring that postsecondary student aid delivery and program management are efficient (objective 3.3). Core strategies to achieve this objective also included working with the Internal Revenue Service on tax refund offsets and address matches and the Department of the Treasury on administrative offsets to increase defaulted student loan collections. In addition, the Department has taken the important step to revise the date for its Year 2000 conversion performance indicator. The plan established 1998, rather than 1999, as the year all of its relevant computer systems will be Year 2000 compliant, thus allowing more time for system testing and validation. While the Department had previously included the Year 2000 conversion effort in its draft plan, it established in its current plan December 31, 1999, as the deadline for repairing seven mission critical systems. As we pointed out, the Year 2000 problem is not technically challenging; however, it is massive and complex. With about 800 days before the Year 2000 deadline, the current plan’s performance indicator of assuring that all systems have been evaluated and, where necessary, converted to make them Year 2000 compliant by December 31, 1998, is a major improvement. In recognition of the critical challenge facing federal agencies in dealing with this issue, GAO has added the Year 2000 problem as one of its high-risk areas. The Department has made significant strides in its October 1997 plan in recognizing major management challenges facing the Department. In our review of its draft strategic plan, we discussed the Department’s particularly difficult challenge in improving its information systems for the student aid program. We discussed the problems that the lack of an integrated student financial aid system creates. We also discussed the Department’s reengineering effort, known as “Easy Access for Students and Institutions (EASI),” which was being developed to redesign the entire student assistance program delivery system. In its draft strategic plan, the Department identified EASI as an important part of its core strategy for integrating its aid systems. However, as we pointed out, the project had a history of false starts. We subsequently recommended in another report that the Department should first develop a systems architecture to address system integration deficiencies before proceeding with new major systems development. The Department’s plan eliminated EASI from its core strategies and adopted the broader core strategy of (1) developing an “integrated, accurate, and efficient student aid delivery system” and (2) ensuring that systems are mission-driven and consistent with the Department’s information technology architecture. In our July 18, 1997, report we also highlighted problems with the Department’s management, systems, and processes that affect its ability to ensure financial accountability, particularly among its student financial aid programs. The Department recognized these problems in its strategic plan and listed the following as its most important challenges: (1) student aid systems that are incompletely integrated, (2) financial data from aid programs that are only partially consolidated at the student level, and (3) too many contractors who use different operating systems. The plan stated that correcting this situation will require the redesign and modernization of the federal student financial aid system using the latest information engineering and computer system technology. To address these and other issues, the Department included under Goal 3 a new, separate objective for the management of its postsecondary student financial aid programs: “Postsecondary student aid delivery and program management is efficient, financially sound, and customer-responsive” (objective 3.3). The Department identified with specificity numerous strategies and performance indicators that will help to address and track agency efforts to achieve this objective. Postsecondary program management. To improve efforts in this area, the strategic plan states that the Department will develop and utilize a risk management system to target compliance and enforcement activities on poorly performing institutions while reducing burdens on high performing ones. Responding to a recommendation from its fiscal year 1996 Department-wide financial audit, the Department is currently developing this new risk analysis system to better utilize its limited monitoring resources towards the highest risk institutions. However, this system will not be fully implemented until fiscal year 1998. Another Department strategy to improve the management of its student financial aid programs involves expanding the use of the case management approach to maximize the effectiveness of institutional oversight. According to the plan, this approach encompasses review of recertification applications, compliance audits, financial statements, risk management system inputs, and program reviews. Financial integrity. The Department stated in its draft strategic plan that poor data from the Federal Family Education Loan Program (FFELP) have prevented it from obtaining an unqualified audit opinion on its annual financial statements for the past 4 years. The Department’s plan included several core strategies for addressing this data integrity problem, such as integrating the multiple student aid databases based on student-level records and improving contract performance for major information systems by increased use of performance-based contracting. The Department also added to its plan an indicator to track the accuracy and integrity of data supplied by applicants, institutions, lenders, and guaranty agencies. Data from these sources have been problematic in the past. In addition, the Department’s October 1997 strategic plan included a new performance indicator related to the financial integrity of the Department’s postsecondary financial aid programs. It states that: “There will be no material internal weaknesses identified in the student aid programs’ portions of the Department-wide financial statement audit and no student aid program issues that prevent the Department from receiving an unqualified opinion on the financial statements.” This indicator is linked to and supports indicator 26, which now definitively states that auditors will issue a clean opinion on the Department-wide financial statements every year. Although, in general terms, the plan better specifies how the Department will address this critical financial management weakness, it still has not completely clarified how it will resolve the data integrity issues for FFELP or accurately estimate the government liability that has prevented the Department from obtaining an unqualified opinion. On October 21, 1997, the Department provided written comments on a draft summary of our observations of its October 3, 1997, strategic plan. The Department generally had no objections to our observations but wanted to clarify several issues we raised in the draft. To measure the performance of Department programs, the agency will include program performance plans in its detailed annual plan currently being prepared by Department staff in conjunction with OMB. The individual program plans will be linked directly to “budget activity lines” and will accompany the Department’s fiscal year 1999 budget justification to Congress in February 1998. The Department submitted 17 draft program plans to Congress in March 1997 that, among other things, identified each program’s goals and objectives, key performance indicators, program evaluations and other data sources, the year the performance indicator data will first be available, and key strategies for achieving the objectives. These performance plans were developed by the program offices and have been reviewed extensively internally—some have been shared with stakeholders. Program performance plans covering the Department’s approximately 100 program activities will include essentially the same information as the 17 draft plans and will be reviewed and updated this fall for inclusion in the agency’s annual plan. The Department’s Chief Information Officer has contracted with Lockheed to work with the agency to ensure that it meets its Year 2000 performance indicator target of December 31, 1998. This activity will be monitored at the highest levels within the agency, and progress will be reported at least quarterly through the strategic planning tracking process. The Department is engaged in several activities that should help to resolve the data integrity issues for FFELP and accurately estimate the government’s liability for this program. Specifically, the Department (1) has developed a workplan, approved by the independent accounting firm of Price Waterhouse, to address concerns about the government’s liability estimate in time for the Department’s fiscal year 1997 audit; (2) is comparing data from the National Student Loan Data System (NSLDS) with audited data submitted by selected guaranty agencies; and (3) is working with E-Systems, Inc., and direct loan origination and servicing contractors to ensure the accuracy and timeliness of direct loan data submitted to NSLDS. Carlotta C. Joyner, Director, Education and Employment Issues; Health, Education, and Human Services Division, (202) 512-7014. On July 11, 1997, we issued a report on the Department of Energy’s (DOE) draft strategic plan dated June 16, 1997 (Results Act: Observations on the Department of Energy’s Draft Strategic Plan, GAO/RCED-97-199R). DOE formally submitted its strategic plan to OMB and Congress on September 30, 1997. As requested, we have reviewed this strategic plan and compared it with the observations in our July report. On October 14, 1997, we briefed your staffs on our further observations on DOE’s strategic plan. The key points from that briefing are summarized herein. As we reported in July 1997, the draft plan did not meet all the requirements of the Results Act. It fully addressed two of the six required elements of the Results Act—the mission statement and goals and objectives—partially addressed a third, and acknowledged that three others needed to be completed for the September plan. Furthermore, the draft plan did not expressly link its missions, goals, objectives, and strategies with DOE’s relevant major statutory responsibilities, although we noted that the missions and activities defined in DOE’s draft plan were generally supported by legislation and that the draft plan accurately reflected all of DOE’s major legislative requirements. However, we observed that DOE’s missions have evolved from those that Congress envisioned when it created the Department in 1977 and that the Results Act provides a forum through which Congress can review the appropriateness of these missions. Our July 1997 report also noted that the draft plan did not identify programs and activities that are crosscutting or similar to those of other federal agencies. In addition, some of the draft plan’s measures addressing management challenges appeared limited in scope or were unclear. Finally, we noted several weaknesses in the information system that DOE uses to track performance measures. In addition to our July report, the Secretary of Energy requested our continued involvement in refining DOE’s plan. On September 2, 1997, we provided the Department with our comments on its revised draft strategic plan—dated August 15, 1997 (Results Act: Observations on the Department of Energy’s August 15, 1997, Draft Strategic Plan, GAO/RCED-97-248R). In that report, we noted that the revised draft plan was much improved over the earlier draft. Specifically, the revised plan included all six elements required by the Results Act. However, we reported that some of the strategies and many of the measures still did not appear to be results oriented. DOE’s September 30, 1997, strategic plan incorporated several improvements that make it more responsive to the requirements of the Results Act than was the June draft plan. In July, we observed that DOE’s draft plan fully addressed two of the six required elements of the Results Act—the mission statement and goals and objectives—partially addressed a third, and acknowledged that three others needed to be completed for the September plan. The September plan complies with the Results Act requirements by including the remaining three sections and fully developing the third by adding a discussion of resource requirements. In describing its resource requirements, the September plan states that the Department assumed budget appropriations consistent with the administration’s and Congress’ agreed-upon 5-year budget deficit reduction targets through fiscal year 2002. Our July report also observed that the draft plan did not expressly link its missions, goals, objectives, and strategies with DOE’s relevant major statutory responsibilities. The September plan now shows the linkage between the Department’s business line objectives and its relevant major statutory responsibilities. Furthermore, DOE’s strategic plan now acknowledges—in its discussion of key external factors—that the Department participates in some crosscutting government functions and initiatives that are beyond the mission of any one agency. While the plan does not describe how DOE will work in concert with other agencies, it does acknowledge DOE’s commitment to work closely with other federal agencies, OMB, and Congress to ensure that its programs provide critical and unique contributions to these crosscutting efforts. DOE did not adopt all of the suggested improvements noted in our July and September reports. These suggestions were based on several of our past reports and represent areas in which we have had disagreements with DOE in the past. However, we still believe that if the Department made these suggested changes—as outlined in our July and September reports—the plan would better address the goals of the Results Act. One such example that we identified in our September report relates to an evaluation that we made concerning the vulnerability of U.S. oil supplies to disruptions. On the basis of that report, we believe that DOE’s measures for its objective to “reduce the vulnerability of the U.S. economy to disruptions in energy supplies” are not very useful indicators of how the Department’s programs will affect the economy’s vulnerability. DOE’s measures are based on six strategies: to (1) support activities capable of ending the decline in domestic oil production, (2) maintain an effective Strategic Petroleum Reserve, (3) diversify the international supply of oil and gas, (4) develop alternative transportation fuels and more efficient vehicles, (5) maximize the productivity of federal oil fields, and (6) take measures to avoid and respond to domestic energy disruptions. However, our report on the vulnerability of oil supplies observed that, in today’s world oil market, replacing oil imports with domestically produced oil would only marginally lower the potential costs of disruptions because oil prices are set in the global marketplace and the price for all oil rises during disruptions. While we agree that one of DOE’s strategies—diversify the international supplies—can lead to measures that contribute to reducing the vulnerability of the U.S. economy to disruptions in the energy supply, our vulnerability report offers five other factors that we believe would better focus DOE’s efforts in developing strategies and measures for its objective of reducing the vulnerability to energy supply disruptions: (1) excess world oil production capacity, (2) the oil intensity of the U.S. economy, (3) the oil dependency of the U.S. transportation sector, (4) world oil stocks, and (5) the dependence of the U.S. economy on oil imports. Finally, our July report noted several weaknesses in the information system that DOE uses to track performance measures. However, DOE’s September 1997 strategic plan makes no reference to these problems. We still believe that DOE will need to modify the information system it anticipates using to track the strategic plan’s performance measures and identify management problems. In addition, we also noted that the information used to update the tracking system depends on various other information systems that we and DOE’s Inspector General have found contain incomplete or inaccurate information. While DOE’s strategic plan is organized along four business lines—energy resources, national security, environmental quality, and science and technology—the agency is organized by program, and it is not clear from the plan which program offices are accountable for implementing the different sections of the plan. For example, several of the Department’s program offices have science missions, including the Office of Nuclear Energy, Science and Technology and the Office of Energy Research; the Office of Nonproliferation and National Security and the Assistant Secretary for Defense Programs have defense missions. However, the plan does not describe how the Department’s current organizational alignment is suited to the plan’s four business lines, nor does it provide a matrix showing which program offices will be held accountable for implementing each section of the plan. On October 10, 1997, we met with DOE officials, including the Acting Director, Office of Strategic Planning, Budget and Program Evaluation, to obtain the Department’s comments on our observations about its strategic plan. DOE officials made three points. First, they stated that development of performance measures is difficult—especially in the science area—and that they recognize the need to continually work to improve these measures. Second, in reference to the disagreements that they had with some of the policy positions of our past reports, they noted that these differences will continue; however, they do not believe that they are strategic planning differences. We disagree because such differences have an impact on the substance of the plan. For example, if DOE uses incorrect measures, it will not know if it has achieved its goals and objectives. Finally, the officials acknowledged that DOE’s strategic plan does not show program accountability but stated that the Department has developed a draft matrix document that provides a crosswalk between its performance measures and the programs. They also pointed out that after the Department-level matrix is completed, each program will need to cascade performance measure accountability to it subunits. Victor S. Rezendes, Director, Energy, Resources, and Sciences Issues; Resources, Community, and Economic Development Division, (202) 512-3841. On July 11, 1997, we issued a report on the Department of Health and Human Services’ (HHS) draft strategic plan (The Results Act: Observations on the Department of Health and Human Services’ April 1997 Draft Strategic Plan, GAO/HEHS-97-173R). HHS submitted its revised strategic plan to the Office of Management and Budget and Congress on September 30, 1997. As requested, we reviewed the revised plan and briefed your staffs on our observations. The key points from that briefing are summarized herein, together with a brief overview of our comments on the initial HHS plan. We found HHS’ draft strategic plan to be missing most of the key elements required by the Results Act and to be more a summary of current programs than a document projecting actions HHS might take in the next several years to achieve the goals of the Act. Although HHS had developed a mission statement that successfully captured the broad array of its activities, the draft plan did not define measurable goals and objectives, describe approaches to achieving these goals and objectives, describe the relationship between long-term goals and objectives and annual performance, identify key external factors beyond HHS’ control, or describe how program evaluations were used to establish or revise strategic goals. Furthermore, although the draft strategic plan recognized that many different HHS operating divisions and programs are responsible for meeting each of HHS’ goals, it did not discuss strategies for coordinating such efforts, nor did it discuss HHS’ need to coordinate its work with other federal agencies. Finally, we observed that HHS faces many major management challenges in carrying out both its program responsibilities and the type of strategic planning and performance measurement the Results Act requires. Two challenges that we highlighted were HHS’ reliance on state, local, and private agencies to carry out many programs for which it is responsible and HHS’ maintenance of financial management and program integrity. Although we believed HHS was aware of these challenges, its plan did not address them. By acknowledging these challenges in its plan, however, we pointed out that HHS could foster a more useful dialogue with Congress about its goals and the strategies for achieving them. HHS’ revised strategic plan incorporated many of the elements that were missing from its earlier draft, making it a more useful document and one that is more responsive to the requirements of the Results Act. The current strategic plan includes all six critical elements as required by the Act. The most notable improvement is in the plan’s outline of objectives for accomplishing HHS’ six strategic goals. The objectives are largely focused on outcomes, such as reducing the use of illicit drugs, and they are defined in measurable terms, such as increasing the percentage of the nation’s children and adults who have health insurance coverage. The plan also identifies for each strategic objective the key measures of progress. For example, the two measures to determine the reduction of tobacco use are the rate of tobacco use among the young and rate of smoking among adults. HHS also added descriptions of its efforts to coordinate both internally among its operating divisions and externally with other departments and agencies. It describes, for example, a range of approaches to improve internal coordination among the various operating divisions, such as special initiatives managed by two or more operating divisions and coordinating councils that integrate planning and policy development across HHS. The discussions of several strategic objectives include a recognition of the need to cooperate with other departments and agencies. For example, the plan indicates that HHS’ substance abuse treatment and prevention programs will work with the Health Resources and Services Administration as well as the Departments of Education and Justice to support an initiative to provide information to communities on the incidence of street and gang violence, domestic violence, and substance abuse and violence. The plan is also improved by HHS’ discussion of three types of challenges that could significantly affect its ability to achieve its strategic goals—external factors, management issues, and data administration. The plan discusses these issues as general obstacles to achieving the Department’s overall goals. Moreover, it describes the Department’s current status in improving performance with respect to these specific issues. As its strategic planning process evolves, HHS’ plan should continue to reflect its progress toward results-oriented management. In the meantime, however, we observed several opportunities for further improvements in the plan. The greatest opportunities for improvement, in our view, are in HHS’ discussion of its strategies for accomplishing its objectives. First, its strategies are not clearly linked to the attendant measures of success, making it difficult to determine how the strategies would contribute to the desired outcomes. For example, to increase the economic independence of families on welfare, the plan specifies three strategies—providing technical assistance, promoting employment, and improving access to child care. The four measures of success for economic independence, however, are all related to employment, with no apparent relationship to the strategies for child care or technical assistance. Second, the plan does not discuss the effectiveness of the outlined strategies, making no mention of either existing evaluations to indicate what is known about the effectiveness of these strategies or plans for future evaluation to determine their effectiveness. For example, some of the strategies were built around a common HHS approach to support state-administered programs: technical assistance, training, and identifying and disseminating best practices. Yet, we have found in our work on these programs that there have been problems in implementing such strategies: in some cases, HHS’ technical assistance was inadequate, the capacity of regional offices to provide assistance and training was limited, and the dissemination of research and best practices was lacking. In addition to drawing on past evaluations, HHS’ plans should identify future evaluations to determine how well its strategies are working. Third, the plan does not discuss the resources required to implement the strategies. For example, strategies to enhance the fiscal integrity of the Health Care Financing Administration (HCFA) programs include consolidation of Medicare payment systems to improve HHS’ ability to identify aberrant billing and improve payment accuracy. However, there is no mention of the resources necessary to implement such a strategy. Fourth, although the plan identifies key external factors that affect achievement of the strategic objectives, there is little discussion of how HHS intends to ameliorate these factors. For example, a key external factor to achieving a number of objectives is the state of the economy, yet the plan does not indicate how the strategies will adjust to changes in the economy. While the plan reflects a recognition of management and information challenges to achieving HHS’ goals, including those mentioned in our July correspondence, it provides little discussion of potential solutions. For example, the plan acknowledges HHS’ reliance on state, local, and tribal government organizations, contractors, and private entities and mentions the need to coordinate with them but is less specific on how it would do so. Similarly, HHS’ plan recognizes the importance of improving its financial management information. In July, we reported that HHS had not addressed its problems in complying with the Government Management Reform Act of 1994 (GMRA), which would furnish decisionmakers with reliable, consistent financial data. While the revised plan acknowledges that obtaining an unqualified or clean opinion on its financial statements is a fundamental and critical objective and challenge for HHS, it does not specify the corrective actions and timetables to address these concerns. With respect to information technology, we noted in our July correspondence that the Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of 1996 set forth requirements that promote more efficient and effective use of information technology to support agency missions and improve program performance. While the plan identified several information technology initiatives that may help HHS achieve some program objectives, the plan does not discuss how HHS intends to identify and coordinate information technology investments in support of overall Department-wide goals and missions. We provided HHS officials with a draft of this appendix. While they were pleased that we recognized the improvements made to the plan, they agreed that the plan can be further improved. In their view, strategic planning is a continuous process; ongoing assessments and updates will be needed to strengthen the plan and ensure that it continues to provide relevant direction for their program activities. Bernice Steinhardt, Director, Health Services Quality and Public Health Issues; Resources, Community, and Economic Development Division, (202) 512-7119. On August 8, 1997, we issued a report on the Department of Housing and Urban Development’s (HUD) draft strategic plan (Results Act: Observations on the Department of Housing and Urban Development’s Draft Strategic Plan, GAO/RCED-97-224R). HUD submitted its strategic plan to OMB and Congress on September 30, 1997. As requested, we have reviewed the strategic plan and compared it with the observations in our August report. On October 14, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from that briefing are summarized herein. HUD’s draft strategic plan included five of the six components required by the Results Act. The plan was missing a description of how program evaluations were used in establishing the strategic objectives, including a schedule of future evaluations. Also, HUD’s treatment of the other five required components did not yet fully comply with the Results Act or OMB’s guidance. The draft included two separate mission statements, which did not define the agency’s basic purpose or focus on its core programs. One of the statements, which focused on restoring the public’s trust, was not clearly supported by HUD’s strategic objectives. While the strategic objectives covered HUD’s major program activities, they did not clearly describe how HUD would assess whether it was making progress toward achieving those objectives. Also, the discussion of HUD’s strategies to achieve its objectives and the relationship of annual performance goals to the strategic objectives was missing a discussion of the resources needed and the type of information needed for its performance goals. The draft strategic plan only partially met the requirements of the Results Act to describe key factors that are external to an agency and beyond its control that could significantly affect the achievement of its objectives. The plan also did not cover the time frames specified by the Results Act. The draft strategic plan generally reflected consideration of HUD’s key authorizing statutes. The draft also discussed HUD’s consultation process and its many community partnerships but did not reflect whether the Department coordinated with other federal agencies and did not identify programs or activities that were crosscutting or similar to those of other agencies. HUD’s draft strategic plan acknowledged that it faced significant management challenges and broadly described how these problems would be addressed. However, we observed that HUD could improve the plan by more fully integrating its management reform plan with the strategic plan and providing specific information about how the plan addressed the Department’s financial and management information weaknesses. HUD’s capacity to provide reliable information on the achievement of its strategic objectives was uncertain because the draft strategic plan had not yet been developed sufficiently to identify the types and sources of the data needed to evaluate progress. The plan identified some annual performance goals for which obtaining reliable data could be difficult because of the weaknesses associated with HUD’s current financial and management information systems. HUD’s September 30, 1997, strategic plan covers all six components required by the Results Act and incorporates many improvements that make it more responsive to the requirements of the Act. Specifically, the plan discusses past evaluations and refines HUD’s mission statement. The new mission statement clearly identifies HUD’s role in achieving the nation’s housing mission. However, the language remains very broad in terms of how HUD can empower communities and individuals to succeed. A mission statement related to the management reforms was reworked and is now included in the plan as the Secretary’s personal mission to emphasize the importance the Secretary places on these reforms. The strategic plan also links the strategic objectives and the annual performance goals, expands the discussion of external factors, and covers the appropriate time frame. Additionally, the revised plan addresses HUD’s consultation process and interagency coordination efforts. The strategic plan discusses HUD’s ongoing and planned coordination with the Departments of Health and Human Services and Labor. This coordination will give HUD the opportunity to identify in future plan updates any programs that complement or duplicate those administered by other federal agencies. HUD has also improved the discussion of the management problems it faces and the corrective actions it plans to take. The strategic plan now includes (1) an explanation of the agency’s current efforts to integrate its program and financial management systems and clean up the data in those systems, (2) a discussion of HUD’s plans to address the issues that led to a qualified opinion on the agency’s financial statements for fiscal year 1996, (3) a discussion of the reform efforts that will affect each objective, and (4) an appendix that lists the management reform goals to be completed in fiscal year 1998. The plan also includes a brief discussion of HUD’s efforts to ensure the quality of performance measurement data by requiring program offices to develop quality assurance plans that will be reviewed and approved by the Chief Financial Officer. However, the agency’s ability to accurately measure progress in achieving its strategic objectives is uncertain because doing so depends on completing its goal of integrating program and financial management systems, cleaning up the data in most of HUD’s existing systems, and receiving accurate reporting from local and federal entities. Despite the improvements in the discussion of HUD’s management problems, the plan lacks details on how the agency will address the internal control weaknesses reported by the Office of Inspector General in the agency’s financial statement audit report. Some elements of HUD’s strategic plan could be further improved to better meet the purposes of the Results Act. HUD will have an opportunity to address these issues as the strategic plan evolves further over time. While the plan includes a listing of the program evaluations under each objective, it does not describe how the evaluations were used to develop the strategic objectives and does not include a schedule of future evaluations. Although wording was added to the plan stating that evaluation schedules are determined on an annual basis, the plan does not include a schedule, which is required by the Results Act. HUD’s discussion of its strategies does not discuss the staff, capital, and technology resources needed to achieve the Department’s strategic objectives, as called for by the Results Act. This issue is a critical one for HUD because of its downsizing efforts and planned organizational changes. While the discussion of external factors was expanded, the plan does not discuss the impact on the strategic plan or on HUD’s programs if the legislative proposals discussed in the plan are not enacted. Additionally, some of the discussions indicate that the external factors may have such a great impact on the strategic objectives that HUD may not be able to achieve its objectives. For example, under the strategic objective to provide self-sufficiency opportunities for low-income individuals, the plan states that HUD has no direct control over the extent to which funds will be used to address this objective. Furthermore, the plan states that “realistically, relatively few people who have reached their 30s with little education, with families, and little work history, will achieve great success in this economy.” While HUD included additional information to aid in the assessment of the strategic objectives, it is not yet clear whether the achievement of a number of the objectives will be assessable. The evaluation component is not yet complete, the discussions of strategies omit significant information about resources, and the discussions of external factors indicate that HUD sees significant impediments to achieving its objectives. As HUD is developing future strategic plan updates and annual performance plans, additional consideration should be given to what each objective is intended to achieve and how that can best be assessed. We provided HUD with a draft of this section of the report for review and comment. We met with HUD officials from the Office of the Chief Financial Officer and the Office of Policy Development and Research, who generally agreed with our observations. They said that information in the annual performance reports and the next update of the plan, which should be available around the end of fiscal year 1998, together should address our observations. Additionally, they said HUD prefers to keep the strategic objectives broad so that the program offices maintain a long-term focus and continue to think of ways to achieve the objectives. Judy A. England-Joseph, Director, Housing and Community Development Issues; Resources, Community, and Economic Development Division, (202) 512-7631. On July 18, 1997, we issued a report on the Department of the Interior’s draft strategic plan (The Results Act: Observations on the Department of Interior’s Draft Strategic Plan, GAO/RCED-97-207R). Interior formally submitted its strategic plan to OMB and Congress on September 29, 1997. As requested, we have reviewed the publicly issued strategic plan and compared it with the observations in our July 18 report. On October 14, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from that briefing are summarized herein. Interior’s draft strategic plan did not meet the requirements of the Results Act. The Department-wide strategic overview contained the Department’s overall mission and goals and referred to the plans of its eight components or subagencies for information on the six elements required by the Act. However, half of the eight subagency plans lacked at least two of the six required elements. Furthermore, the overall quality of the plan was not yet sufficient to achieve the purposes of the Act. Among other things, it did not provide clear linkages between the subagencies’ goals and objectives and the contributions of these goals and objectives to the Department’s major goals, and some of the goals and objectives in the subagencies’ plans were not stated in a manner to allow for a future assessment of whether the goals have been achieved. We pointed out that Interior has a number of crosscutting areas in which a more coordinated strategic planning process would help to provide Department-wide information on programs’ results. These include environmental protection and remediation, stewardship assets, Indian programs, land and natural resources management, and recreation programs. Although Interior identified information management resource goals in its strategic plan, how it plans to achieve and measure the success of those goals was not clearly delineated. Traditionally, Interior has allowed its subagencies to independently acquire and manage information technology. This culture has resulted in inefficiencies in technology investments and information sharing. We also noted that Interior needs to continue to address certain accounting and financial management internal control weaknesses, including, among other things, weaknesses in accounting for investments in fixed assets and project cost accounting controls. The September 1997 strategic plan—the Department’s strategic overview plan as well as each of the eight subagencies’ plans—incorporates several improvements that make it more responsive to the requirements of the Results Act than was the draft plan. As a whole, the plan provides a clearer presentation of how it covers the six required elements of the Act by providing explicit linkages between the requirements of the Act and the relevant parts of the plan. Furthermore, each of the four subagencies that had lacked a number of required elements in the draft plans has added or further developed many of these elements in the issued plan. In particular, each of these four subagencies—the National Park Service (NPS), Fish and Wildlife Service (FWS), Bureau of Indian Affairs (BIA), and Minerals Management Service (MMS)—added material to address the relationship between long-term goals and performance goals. In addition, both BIA and FWS have included discussions of the approaches or strategies they will use to achieve their respective goals and objectives—information that was not present in the draft plans for these subagencies. Also, additional information was added to the overview section of the plan to more fully explain the Department’s approach to program evaluations. In addition to more fully addressing several required elements, the September 1997 overview and subagencies’ plans now contain explicit linkages between the subagencies’ goals and objectives and the contributions of these goals and objectives to the Department’s goals and commitments. Also, many of the goals included in the issued plan have been restated in a quantitative manner. These are positive changes and will facilitate a future assessment of whether the goals have been or are being achieved. Consistent with the suggestions in our July report, Interior included a section in the departmental overview discussing its current efforts to address crosscutting issues throughout the Department and its strategy for further coordination. Interior’s strategic plan also includes a more aggressive goal for addressing internal control weaknesses. Additionally, a section has been added that specifically discusses accountability for personal, real, and museum property (fixed assets). The plan also discusses integrating the personal and real property systems with financial and procurement systems that would appear to represent progress toward attaining project cost accounting. There are a number of aspects of Interior’s plan that still can be improved to better meet the purposes of the Results Act. In particular, some of the subagencies—BIA, the Bureau of Reclamation (BOR), FWS, and NPS—need to more fully develop the program evaluation component of their plans. While each of these subagencies, as well as the departmental overview, has made revisions to its draft plan in this area, the revisions still do not provide a complete understanding of specifically how program evaluations were used in developing the plan or what future evaluations will be done and when for each of the subagencies. Including this kind of information is important because without it, it is difficult for both the subagencies and other users of the plan to have confidence that the goals are the correct ones and that the strategies will be effective. Furthermore, while the subagencies have made progress in restating a number of their goals and objectives in a more measurable way as we suggested in our July report, this area of the plan still can to be improved. Many of the goals and objectives are still process oriented, not results oriented, and/or expressed in a manner that will make meaningful performance measurement difficult. For example, one of the strategic goals in FWS’ plan states that: “By 2002, the current maintenance backlog will be reduced annually.” As stated, it is not clear what level of performance is expected or will be considered acceptable in achieving this goal. We observed similar difficulties in several of the subagencies’ plans. While the September 1997 plan now includes a discussion of ongoing efforts to coordinate a number of crosscutting issues facing the Department and identifies its future approach in this area, the plan still does not explicitly address the crosscutting issues identified in our July report. These included the Department’s environmental protection and remediation, stewardship assets, Indian programs, land and natural resource management, and recreation programs. In our July report, we noted that the plan needed to more fully address information management issues. This need still exists in the September plan. In the September plan, Interior has identified goals and actions needed to implement the provisions of the Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of 1996 but does not clearly describe how it plans to achieve, or to measure its success in achieving, its goals. Also, Interior needs to explain how it plans to address the Year 2000 problem as well as significant information security weaknesses—two issues that we have identified as high risk across the federal government. Furthermore, the September plan now states that Interior’s critical information systems will be Year 2000 compliant by September 30, 2000—9 months after the January 1, 2000, deadline. On October 10, 1997, we met with Interior officials, including the Deputy Assistant Secretary for Budget and Finance, to obtain the Department’s comments on our observations about its strategic plan. Interior believes that the September 1997 plan meets the requirements of the Results Act. However, the Department acknowledges that improvements can be made in several areas. Interior noted that the development of its strategic plan is an iterative process and that future versions of the plan will address areas in which we and others show a need for improvement. Furthermore, in connection with crosscutting issues, Interior commented that it believes that its current efforts and initiatives in this area are sufficient. However, in our view, focusing on results implies that federal programs’ contribution to the same or similar results should be closely coordinated to ensure that goals are consistent and that, as appropriate, program efforts are mutually reinforcing. In connection with information management issues, Interior commented that it has detailed plans to address Year 2000 issues and does not believe the level of detail that we suggested is necessary for inclusion in a strategic plan. We continue to believe that clear discussions of Year 2000 and information security issues would strengthen the strategic plan and provide linkages for its operational plans. This disclosure would help Congress, departmental customers, and the general public to better understand the Department’s goals, strategies, and measures. Victor S. Rezendes, Director, Energy, Resources, and Science Issues; Resources, Community, and Economic Development Division, (202) 512-3841. On July 11, 1997, we issued a report on the Department of Justice’s February draft strategic plan (The Results Act: Observations on the Department of Justice’s February 1997 Draft Strategic Plan, GAO/GGD-97-153R). On August 15, 1997, Justice revised its plan and we testified on September 30 on the plan’s compliance with the Act’s requirements (Results Act: Comments on Justice’s August Draft Strategic Plan, GAO/T-GGD-97-184). Justice’s formally issued strategic plan was submitted to OMB and Congress on September 30, 1997. As requested, we have reviewed the issued strategic plan and compared it with the observations in our July 11 report and September 30 testimony. On October 7, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from that briefing are summarized herein. In response to comments on its February strategic plan, Justice revised its plan in August. The revised plan addressed many of the issues we raised in our July report. In our July report, we pointed out that of the six elements required by the Act, three—the relationship between long-term goals and the annual performance plans, the key external factors that could affect Justice’s ability to meet its goals, and a program evaluation component—were not specifically identified in the draft plan. The remaining three elements—the mission statement, goals and objectives, and strategies to achieve those goals and objectives—were discussed, but each had weaknesses. The most important of these were that the mission statement did not cover a major statutory responsibility, goals and objectives were not consistently as results oriented or measurable as they could have been, and strategies were not fully developed. In addition, we observed that the February draft plan could be more useful to Justice, Congress, and other stakeholders if it provided a more explicit discussion of (1) crosscutting activities, (2) major management challenges, and (3) Justice’s capacity to provide reliable information to manage its programs or determine if it is achieving its strategic goals. Recognizing crosscutting issues and the coordination required to address them is particularly important for Justice because, as the federal government’s attorney, it helps the various federal law enforcement agencies enforce the law in federal courts. Explicit consideration of major management challenges, including the capacity to produce reliable information for management decisionmaking, is important because these challenges could affect Justice’s ability to develop and meet its goals. In our September testimony, we pointed out that Justice’s August draft plan discussed, to some degree, five of the six required elements—a mission statement, goals and objectives, key external factors, a program evaluation component, and strategies to achieve the goals and objectives. The August draft plan did not include a required discussion on the relationship between Justice’s long-term goals/objectives and its annual performance plans. In addition, we noted that the August draft plan could have better addressed how Justice plans to (1) coordinate with other federal, state, and local agencies that perform similar law enforcement functions, such as the Defense and State Departments with regard to counter-terrorism; (2) address the many management challenges it faces in carrying out its mission, such as internal control and accounting problems; and (3) increase its capacity to provide performance information for assessing its progress in meeting the goals and objectives over the next 5 years. Justice’s issued strategic plan incorporated several improvements that make it more responsive to the requirements of the Results Act than were the February and August draft plans. Its September plan discusses each of the Act’s required elements. In particular, Justice added in its August plan a discussion of eight key external factors that could significantly affect achievement of its long-term goals, information that is helpful to Congress in its consideration of Justice’s plan. However, information about alternatives that could reduce the potential impact of these external factors was not provided. In addition, Justice’s August strategic plan included a discussion of the role program evaluation is to play in Justice’s strategic planning efforts. Justice recognized that it has done few formal evaluations of Justice programs in the past, but the plan acknowledged that sound program evaluation is an essential aspect of achieving the purposes of the Act and stated that Justice plans to examine its evaluation approach to better align evaluations with strategic planning efforts. Further, Justice pointed out that it will continue to improve its efforts to benefit from our evaluations. This element of the plan could be more helpful to decisionmakers if it identified future planned evaluations and their general scope and time frames, as encouraged by OMB strategic plan guidance. address (1) its process for managing its information technology investments, steps taken to provide security over its information systems, and strategy to ensure that computer systems accommodate dates beyond the year 2000; and (2) aspects of its internal control processes that identify management weaknesses and vulnerabilities. Justice also added a discussion on “accountability,” debt collection, and asset forfeiture. However, the plan would be more helpful if it included a discussion of corrective actions Justice has planned for significant internally and externally identified management weaknesses, as well as how it plans to monitor the implementation of such actions. In addition, the plan does not address how Justice will correct significant problems identified during the Inspector General’s fiscal year 1996 financial statement audits, such as inadequate safeguarding and accounting for physical assets and weaknesses in the internal controls over data processing operations. Several elements of Justice’s issued strategic plan could be further improved to better meet the purposes of the Results Act. In particular, some of the plan’s goals and objectives still were not stated in as results oriented or measurable a form as they could be, and some of the strategies to achieve the goals and objectives did not clearly explain how and to what extent Justice programs would contribute to achieving the goals, how its resources are to be utilized to achieve the goals, or how Justice plans to assess progress in meeting those goals. For example, Justice has a goal to maximize deterrents to unlawful immigration by reducing the incentives of unauthorized employment and entitlements. It is likewise unclear how Justice will be able to determine the effect of its efforts to deter unlawful immigration, compared to the effect of changes in the economic and political conditions in countries from which illegal aliens originated. In addition, Justice’s mission statement, which we observed in July as seeming to be incomplete because it omitted one of its largest budget items—detention and incarceration function—was not changed. reflects high level and crosscutting annual goals and indicators and (2) more detailed component and appropriation-specific performance information. Justice added that goals and indicators will be supportive of, and derived from, those set forth in the strategic plan. Recognizing that the linkage between the strategic plan and the annual performance plan is a critical element of the Act, Justice said that it has revised its internal processes to ensure that the strategic plan serves as the foundation for the development of annual budgets and performance plans. In our opinion, Justice’s September strategic plan could better meet the purposes of the Act by discussing, as contained in OMB guidance, (1) the type, nature, and scope of the performance goals to be included in its performance plan; (2) the relation between the performance goals and the general goals and objectives; and (3) the relevance and use of performance goals in helping determine the achievement of general goals and objectives. This information is important because the linkage between the goals and objectives and annual performance plan provides a basis for judging whether an agency is making progress toward achieving its long-term goals, not just its annual goals, which would be reflected in the annual performance plan. We observed that the February and August draft plans did not include a discussion of how Justice’s activities would be coordinated with other related crosscutting law enforcement activities. The issued strategic plan includes a goal to coordinate and integrate law enforcement activities wherever possible and to cooperate fully with other federal agencies. However, the plan could better serve the purposes of the Results Act by discussing how Justice plans to implement that goal and to measure and assess inputs, outputs, and outcomes to achieve crosscutting law enforcement goals. On October 14, 1997, we obtained oral comments from Justice officials, including the Director, Management and Planning Staff, on a draft of our analysis and observations of Justice’s issued strategic plan. They said that our analysis and observations fairly represent Justice’s strategic plan. Norman J. Rabkin, Director, Administration of Justice Issues; General Government Division, (202) 512-8777. On July 11, 1997, we issued a report on the Department of Labor’s draft strategic plan (The Results Act: Observations on Department of Labor’s June 1997 Draft Strategic Plan, GAO/HEHS-97-172R). Labor formally submitted its plan to OMB and Congress on September 30, 1997. As requested, we have reviewed this strategic plan and compared it with the observations in our earlier report. On October 16, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from that briefing are summarized herein. To meet the Results Act requirements for a strategic plan, Labor submitted individual plans for 15 of its 24 component offices or subunits, which it supplemented with a “strategic plan overview.” In one case, one of Labor’s offices—the Employment Standards Administration (ESA)—did not submit a plan itself, but instead submitted plans for the four subunits under its responsibility. While OMB Circular No. A-11 provides agencies discretion to submit strategic plans that cover only major functions or operations, Labor provided no indication as to why its other offices or subunits did not provide plans. We reported that neither the overview nor the component plans fully met the Act’s requirements or OMB guidance. For example, the overview’s mission statement was not sufficiently descriptive of Labor’s basic purpose, and the overview did not include elements identified by the Act, such as strategies to achieve goals or evaluations used to establish goals. Further, a majority of the component plans did not include all of the elements required by the Act, such as the strategies to achieve the goals or key factors affecting goal attainment. We noted that the overview would be more useful if it included all of the elements identified by the Act and, regarding the mission statement specifically, if it communicated more about Labor’s purpose, referring to such basic responsibilities as job skills development, job placement, and worker protection. three broad programmatic categories that were not developed into goals. We observed that Department-wide goals enunciated by the Secretary in recent congressional testimony could serve as the basis from which to develop Department-wide goals that are results oriented and set out the long-term programmatic policy and management goals of the agency. We found that the goals in the overview and in the component plans were generally consistent with Labor’s statutory responsibilities, and the plans generally covered all of Labor’s major functions and operations. Regarding crosscutting issues, we reported that the strategic overview recognized the roles of other organizations in carrying out particular functions and the importance of establishing partnerships with these organizations to carry out such functions. However, we indicated that the overview could be improved if it recognized the importance and number of other participants—namely, the other 14 federal agencies—involved in one major area of responsibility—job training. In so doing, Labor could discuss how its programs fit in with a broader national job training strategy. We also found that the Labor officials responsible for preparing the plan and monitoring its progress had not consulted with congressional staff regarding the overview or the component plans. Finally, we reported that the strategic overview highlighted the need for information and data systems to ensure timely and sound evaluations to assess agency progress in meeting its goals. However, the overview did not describe Labor’s strategy for ensuring that this kind of information was collected and used to assess progress and performance. The overview also did not discuss how Labor planned to use information technology to achieve its mission, goals, and objectives, or to improve performance and reduce costs. We reported that the plan could be improved by including a discussion of Labor’s investment technology process, including how Labor planned to address the Year 2000 problem, or how Labor planned to comply with the Clinger-Cohen Act of 1996, which calls for agencies to implement modern technology management to improve performance and meet strategic goals. prepared draft plans previously (including 1 that has overall management responsibility for implementing the Results Act) and the 4 plans originally submitted by ESA’s subunits were consolidated into 1 ESA-level plan for a total of 15 component-level plans. In its overview, Labor provided a rationale for the components included, noting that “. . . strategic plans have only been required of the 15 program and management agencies of the Department. Several staff offices whose functions are in direct support of the Secretary’s office are not included.” Labor’s strategic overview and all but 1 of the 15 component unit plansinclude all 6 elements. Further, the overview’s mission statement now provides a more complete description of Labor’s basic purpose of “foster and promot the welfare of job seekers, wage earners, and retirees of the United States by improving their working conditions, advancing their opportunities for profitable employment, and protecting their retirement investments.” Moreover, discussions of strategies to achieve goals and external factors that could affect the achievement of goals are discussed alongside individual goals, which facilitates the understanding of how particular strategies and external factors are linked to each goal. The overview also appears to address Labor’s traditionally decentralized management approach, which has posed numerous management challenges for Labor in the past. For example, the overview now contains five clearly articulated Department-wide goals that are generally results oriented and that are consistent with those recently enunciated by the Secretary. The overview also includes a sixth Department-wide goal of maintaining a departmental strategic management process, which may be an indication of a renewed emphasis by Labor to develop a more strategic approach to departmental management. Other indications of this renewed approach to Department-wide leadership are evident in the similar organizational style of each of the component plans and the clear linkage between the strategic overview and the plans. For example, in the overview, the strategic goals of each of the units/offices are highlighted under the appropriate Department-wide goal; and in each of the plans for the offices/units, the office/unit strategic goals are categorized according to the Department-wide goal to which they correspond. Further, the overview now includes a discussion of the relationship between the goals in the annual performance plan and in the strategic plan. The overview and component plans we reviewed continue to describe all of Labor’s major functions, and the goals are consistent with relevant statutes. Although Labor has made significant improvements to its strategic plan overview, some sections in the overview may benefit from further elaboration. For example, the overview does not detail how information from evaluations was used to develop the plan, nor does it specify how future evaluations will help assess Labor’s success in achieving its stated goals. Instead, the overview discusses the fact that evaluations in the regulatory agencies have lagged behind those in the employment and training area. In that respect, it is even more important that the overview provide schedules or time lines for future evaluations, identify what evaluations will be done, and highlight how future program evaluations will be used to improve performance. Along those lines, we had earlier reported that the experiences of Labor’s Occupational Safety and Health Administration (OSHA) as a pilot could provide insight on how evaluations can be managed. OSHA has been involved in a number of activities geared toward making the management improvements intended by the Results Act. Although it is not a requirement of the strategic planning process, we continue to believe that a discussion in Labor’s overview related to the experiences gained from the OSHA pilot project—including lessons learned and whether best practices or other lessons could be applied Department-wide or in units with similar functions—may prove helpful. Labor could also improve the overview by continuing to enhance the discussion of crosscutting issues, such as coordination with others who have similar roles for particular functions. While the overview does make reference to a few other organizations with responsibilities in this area and notes that Labor will work with them, there is no discussion of what specific strategies Labor will use to realize efficiencies through coordination and possible consolidation of job training programs in order to achieve a more efficient employment training system. from additional discussion on how these agencies are working together to share information on efficient enforcement and public education strategies or measurement tools. The overview could also benefit from a more elaborate discussion of the strategies Labor will use to ensure that its information technology allows it to achieve its goals. While the overview continues to cite the vision of expanded use of technology across Labor and its component units, the plan does not adequately discuss the inclusion of a framework—sometimes called a systems architecture—that will serve as a blueprint for developing and maintaining integrated information systems. Such a framework would help ensure that the data being collected and maintained within Labor are structured and stored in a manner that makes them consistent, accessible, understandable, and useful. The overview also still does not include a clear, integrated, measurable Year 2000 strategy, which may be needed to adequately consider the multitude of system and information interfaces inside and outside of Labor that must be addressed prior to the millennium change. In an October 14, 1997, letter from Labor’s Acting Assistant Secretary for Administration and Management, Labor thanked us for acknowledging the substantial progress in Labor’s plan. The letter noted that there is more work to be done and that Labor will address the concerns we raised about the use of evaluations in developing plans and evaluating results, crosscutting issues, and internal coordination among safety and health agencies during the next revision. Labor also noted that it will expand its presentation in the strategic overview to provide additional information on its information technology. However, Labor noted that a more detailed discussion of its systems architecture and its Year 2000 compliant strategy are included in Labor’s separate Information Technology Strategic Plan and other documents. Additionally, it said its approach for addressing information technology in the overview was to describe the linkage and the importance of information technology in support of program agencies and the achievement of goals. While this approach is reasonable, and our preliminary review of the Information Technology strategic plan indicates that it tries to address many of the issues we outlined previously, the strategic overview could still benefit from clearer cross-referencing and linkage between the two plans. Additionally, the Information Technology Strategic Plan may benefit from clearer linkage between the components’ activities and Labor’s activities as a whole to enhance information technology. Carlotta C. Joyner, Director, Education and Employment Issues; Health, Education, and Human Services Division, (202) 512-7014. On July 18, 1997, we issued a report on the Department of State’s draft strategic plan (The Results Act: Observations on the Department of State’s May 1997 Draft Strategic Plan, GAO/NSIAD-97-198R). State issued its formal strategic plan and submitted it to OMB and Congress on September 27, 1997. As requested, we have reviewed the publicly issued strategic plan and compared it with the observations in our July 18 report. On October 20, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from that briefing are summarized in the following sections. State’s draft strategic plan was useful in setting and clarifying U.S. foreign policy goals, but it did not contain sufficient information to fully achieve the purposes of the Results Act and was incomplete in several important respects. In particular, the draft plan omitted two elements required by the Act: (1) components identifying the relationship between long-term goals/objectives and annual performance goals and (2) a description of how program evaluations were used to establish or revise strategic goals and a schedule for future program evaluations. To fully achieve the purposes of the Act, State’s draft plan needed to be more descriptive and consistent with OMB guidance. For example, the plan contained several sections labeled strategy for specific goals, but it did not specifically identify the actions and resources needed to meet the plan’s goals or include a schedule for taking significant actions. State’s strategies often focused on describing the Department’s role in various areas instead of describing how State’s programs and operations would help achieve the goals. We observed that State’s draft plan did not specifically discuss the likelihood that other agencies might have functions similar to or possibly duplicative of State’s role that could affect the formulation and implementation of strategies. administrative systems. These issues were discussed separately in a diplomatic readiness section of the strategic plan. We said that the draft plan would be strengthened if it better described how meeting these management challenges could affect achievement of the plan’s strategic goals. Furthermore, we noted that the draft plan would have been enhanced if it had included a discussion of how the proposed consolidation of State, U. S. Information Agency, and Arms Control and Disarmament Agency might affect goals, strategies, and resource requirements. In addition, we suggested that State’s plan would be easier to use if it contained a clearly labeled agency mission statement and included a discussion of the Department’s key legal authorities. We also observed that State’s capacity to provide reliable information about its operations and program performance was questionable because of long-standing deficiencies in the Department’s information and financial accounting systems. Successfully resolving a number of these material deficiencies in the Department’s financial and information management systems will be critical to implementing the plan. State’s September 1997 strategic plan incorporated some improvements that help make it more responsive to the requirements of the Results Act. With respect to the first element required by the Results Act that was missing in the draft plan, State introduced a separate section describing the relationship between the plan’s strategic goals and the goals and objectives in the Department’s performance plan. It used only one example to describe this relationship, discussing the linkages between operational and performance goals for achieving the strategic goal of eliminating the threat from weapons of mass destruction or destabilizing conventional arms. This example is helpful, but it would be more useful if it clearly described how the resource and performance measurement components will be handled in the Department’s annual program planning cycle. With respect to the second missing element, the plan now includes a section dealing with program evaluations. However, instead of including a description of the program evaluations used in developing goals and objectives, as required by the Results Act, the new section is largely a discussion of State’s rationale for not fully meeting this requirement. explaining how resources from various sources, and managed by different agencies, are established in the international affairs function of the President’s budget—the “150” account. State’s strategic plan focused on the Department’s mission and role in carrying out 16 strategic foreign policy goals. A few modifications were made in the strategic goals since our July review (for example, a goal of promoting broad-based economic growth in developing and transitional economies was added, and a goal of improving the well-being of the world’s poor was dropped), but the plan still did not consistently explain what results are expected from the Department’s major functions or when to expect the results. Some changes in the Department’s strategies were also made, but it remained unclear how some of the goals are to be achieved or what level of resources is required. State’s plan specifically acknowledged that more needs to be done to identify agencies’ capabilities and the resources needed to achieve the goals. The plan’s section on program evaluations is essentially an explanation of why the plan does not fully meet the requirements of the Results Act. The plan pointed out that no process existed for systematic evaluation of the foreign affairs goals. As a result, the plan did not identify any evaluations used for establishing or revising the strategic goals or include a schedule for future evaluations. It is State’s position that it should not be held strictly accountable for this and other requirements of the Act because of the complexities of foreign policy, the scope of the Department’s responsibilities that cover most other agencies, and the complexities of managing overseas missions. We recognize that program evaluations in the foreign affairs area are difficult, but we believe that an effective evaluation process will be critical to determining the extent to which State is successfully achieving/helping to achieve goals and what actions may be necessary to help improve performance. global programs, and international security functions. The plan also noted that the incorporation of management goals for the integration of other foreign affairs agencies awaits decisions concerning how the reorganization will proceed. The plan’s section on management issues emphasized the importance of the strategies for achieving diplomatic readiness but noted that this represents a first effort to set strategic goals for the Department’s major management responsibilities. It still did not address the serious management problems related to cost control, overseas embassy management, and financial management identified in our prior work and discussed in our July report. The plan’s discussion of data capacity did not specifically address the serious deficiencies in State’s financial accounting and information systems, but it noted in more general terms that it will take several years to develop performance measures and related databases in order to provide sufficient information on achievement of the goals. The Chief Financial Officers Act requires agencies to have accounting and financial accounting systems that provide for the development of cost information and systematic measurement of performance. Currently, State does not have a true cost accounting system, and, as a result, reliable cost information by function cannot be provided. In addition to developing the data capacity and information systems essential for measuring progress, State’s strategic plan also acknowledged that much more remains to be done to adequately develop the Department’s long-term strategic planning process. State’s plan identified several long-term actions as critical to the process, including the development of an agency performance plan and mission performance plans for each overseas embassy linking annual goals with long-term strategic goals, performance measures, and a process for conducting performance evaluations. State’s strategic plan cautioned that it will take substantial effort to develop a fully refined set of performance measures and a performance evaluation process. In discussing its efforts to develop an integrated planning process, State noted that a process linking overseas mission performance plans to the Department’s strategic and diplomatic readiness goals will first be in place for the Department’s fiscal year 2000 budget submission. As State’s strategic plan evolves over time, other matters will clearly require attention. For example, State’s performance plan will need to be definitive to compensate for the continued lack of specificity in State’s strategies concerning how the Department will achieve individual long-term goals and the level of resources needed for State’s activities. The overseas mission performance plans will also need to be high quality to be successfully integrated into State’s master plan, in view of (1) the billions of dollars in resources associated with State’s overseas operations and (2) the historical weaknesses we have identified in overseas embassy management. These weaknesses have included insufficient staff training, poor inventory controls, and questionable procurement practices. State’s lack of attention to the use of evaluations in setting and refining its long-term goals and the lack of a specific schedule for future evaluations are other areas that deserve attention. We believe that several areas may require evaluation to ensure that the Department’s strategic planning process is sound, including the adequacy of State’s performance and program planning processes, the extent to which State and other agencies’ functions may or may not be duplicative, and the adequacy of State’s overseas staffing decisions based on design and implementation of its new staffing model. We obtained oral comments from State officials responsible for the Department’s strategic planning efforts. They generally agreed with our description of the progress State made in its strategic plan and the issues that require further attention. However, they noted that in judging the quality of the plan, it is important to keep in mind the complexities of strategic planning in the foreign affairs area. These complexities include the scope of the strategic goals and the national interests involved, the numerous government agencies sharing responsibilities, and the lack of proven performance measures in key functional areas. Department officials also recognized that the strategic planning process has not yet paid sufficient attention to management and cost issues. However, they expect that the diplomatic readiness section of the plan will be revised in the spring of 1998, along with other parts of the plan, based on a series of seminars and stakeholder discussions scheduled to begin in November 1997. Benjamin F. Nelson, Director, International Relations and Trade Issues; National Security and International Affairs Division, (202) 512-4128. On July 30, 1997, we issued a report on the Department of Transportation’s (DOT) draft strategic plan (Results Act: Observations on the Department of Transportation’s Draft Strategic Plan, GAO/RCED-97-208R). DOT’s formally issued strategic plan was submitted to OMB and Congress on September 30, 1997. As requested, we have reviewed the September plan and compared it with the observations in our July report. On October 16, 1997, we briefed your staffs on our further observations on the strategic plan. Our key points are summarized herein. DOT’s draft strategic plan did not fulfill all of the requirements of the Results Act. The draft plan met the Results Act’s requirements for mission statement, long-term goals, and a description of program evaluations; however, each component had weaknesses that could be improved. The draft plan did not meet the Act’s requirements to describe strategies for achieving goals, a linkage between long-term goals and annual performance goals, and those key external factors that could significantly affect DOT’s achieving its goals. Overall, the draft was so general that it did not clearly identify the Department’s priorities. We reported that the quality of the draft plan could have been improved throughout by adhering more closely to OMB’s guidance for preparing strategic plans and including more detailed information. In addition, the draft plan did not (1) show evidence of coordination with other agencies that have programs and activities that are crosscutting or similar to DOT’s or (2) adequately address major management challenges and high-risk areas that we and others previously identified. We also observed that DOT’s ability to produce reliable performance information was uncertain because the draft plan was unclear about what information would be needed to measure performance. Finally, the draft plan reflected the Department’s key statutory authorities. contribute significantly to each goal. Finally, the discussion of program evaluations has been revised to include a table that lists future evaluations and, for each, describes the scope, methodology, key issues to be addressed, schedule, and relationship to the plan’s long-term goals. Moreover, the September plan meets two of three additional requirements of the Results Act that the draft plan did not meet. First, the revised plan now meets the Act’s requirements by discussing how the annual performance goals, which are being developed for the fiscal year 1999 budget submission, will link to DOT’s mission and long-term goals. The September plan also includes a table for each long-term goal showing examples of possible indicators that may be used to measure annual performance and the availability of data. Second, the discussion of those key external factors that could significantly affect DOT’s ability to achieve its goals has been rewritten and expanded to meet the Act’s requirements. The September plan identifies new factors—such as legislation to address long-term financing for the Federal Aviation Administration (FAA) and Amtrak—that will affect DOT’s ability to achieve its long-term goals. The plan also summarizes how economic, social, political, environmental quality, national defense and security, and technology trends affect each long-term goal; provides a few examples of activities needed to mitigate the effect of these trends; and explains each factor in greater detail in both a separate section and an appendix. security weaknesses; and (6) change computer systems to accommodate dates beyond the year 1999, that is, address the Year 2000 problem. Finally, the September plan continues to reflect DOT’s key statutory authorities in an appendix and includes minor clarifications. DOT’s September plan can be further improved in two areas. First, the plan’s discussion of strategies for achieving its long-term goals has improved in some areas but still does not meet all requirements of the Results Act. The revised plan describes corporate management strategies for implementing the plan that cut across the Department. These strategies provide useful information, for example, in explaining how long-term goals will be communicated to employees and how personnel will be assigned accountability for achieving the goals. However, the revised plan still does not describe the operational processes, skills, technology, and resources required to meet the long-term goals, as required by the Results Act. The general discussion of corporate management strategies does not meet these requirements, which should be addressed for each goal. Furthermore, the plan could be improved by following OMB’s guidance on strategic plans and providing additional detail when achieving a goal is predicated on a significant change in resource or technology levels. For example, we have reported that successful implementation of certain aviation security measures mentioned in the plan is contingent upon deciding who will finance the security improvements and developing the needed technology. In addition, the plan could be improved by following OMB’s guidance on including time frames for initiating or completing significant actions. The September plan contains time frames for some significant actions, such as addressing the Year 2000 problem and obtaining reliable financial statements by fiscal year 2000; but it does not include time frames for other significant actions, such as completing air traffic control modernization and improvements to Amtrak’s Northeast Corridor. these problems, it does not demonstrate a firm commitment to resolve them through specific strategies. For example, the plan mentions our concerns about the need to improve the oversight of highway and transit projects, which are continuing to incur cost increases, experience delays, and have difficulties acquiring needed funding commitments. The plan states that these concerns are addressed under corporate management strategies. The strategies, however, provide insufficient details to address the problems with these projects. As another example, the revised plan mentions Amtrak in an appendix but provides too little information to adequately address our concerns about the corporation’s very precarious financial condition, which threatens its survival. The plan could be improved by addressing Amtrak’s role in a national transportation framework and providing objectives concerning the future of Amtrak and strategies for meeting these objectives. The plan acknowledges the significance of financial management to the achievement of its long-term goals and is generally responsive to specific comments that we made about the draft plan. The revised plan includes a new section that discusses (1) general financial management and (2) the need for and plans to improve financial management of and accountability for the Department’s financial resources. However, while the revised plan acknowledges that unreliable accounting (including cost accounting) information exists at the program level, it does not provide specific strategies or timetables for resolving key problems. DOT has added specificity to the plan that greatly improves its overall quality. However, the plan still takes an “umbrella” approach—it is expansive enough to encompass all of DOT’s programs, but it does not describe the contributions from specific modes to implement the plan. The plan refers to the development of a “National Transportation Strategy” with subordinate strategies for air, surface, and maritime elements. This strategy might provide the missing link between the Department-wide goals and the programs throughout DOT. in two ways. First, in discussing each strategic goal, the plan includes a section entitled “How We Will Achieve the Strategic Goal” that describes the processes that DOT will employ to achieve the goal. Second, the Department stated that the plan meets this requirement in a section that describes six overarching management strategies—the “ONE DOT management philosophy,” human resources, customer service, resource and technology, information technology, and resource and business process management. We disagree that the sections of the plan mentioned by DOT fulfill the Act’s requirements. For the most part, the sections that discuss how DOT will achieve the goals are too general to do so. For example, the plan states that to achieve the mobility goal of ensuring an accessible, efficient transportation system, DOT will improve technical assistance. The plan does not explain the type of technical assistance, who will receive the assistance, or how the assistance will improve mobility. As another example, the plan states that to achieve its economic growth and trade goal, DOT will assess the performance of the transportation system as a whole. The plan does not explain how such an assessment will help the Department achieve this goal. Furthermore, as we mentioned, the plan’s corporate management strategies are also too general to meet the Act’s requirement, although they do provide useful information in certain areas, such as explaining how the goals will be communicated to employees. These strategies provide a philosophy for the Department to operate under, but not specific steps to achieve the goals. For example, the human resources management strategy states that the Department will “achieve its strategic goals with a workforce that is knowledgeable, flexible, efficient, and resilient.” The actions to accomplish this include redesigning human resources programs to “allow DOT to recruit, develop, and deploy a diverse workforce with those 21st Century competencies needed to achieve the DOT’s strategic goals.” The strategy does not explain what competencies are needed. Finally, DOT commented that achieving its strategic goals is not predicated on a significant change in resource or technological levels. We disagree. As we stated in our observations, successful implementation of certain aviation security measures mentioned under the national security goal is contingent upon deciding who will finance the security improvements and developing the needed technology. Phyllis F. Scheinberg, Associate Director, Transportation Issues; Resources, Community, and Economic Development Division, (202) 512-2834. On July 31, 1997, we issued a report on the Department of the Treasury’s draft strategic plan (The Results Act: Observations on the Department of the Treasury’s July 1997 Draft Strategic Plan, GAO/GGD-97-162R). Treasury’s formally issued strategic plan was submitted to OMB and Congress on September 30, 1997. As requested, we have reviewed the September 30 strategic plan and compared it with the observations in our July 31 report. On October 15, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from our July report and October briefing are summarized herein. Treasury’s July draft strategic plan was incomplete and did not meet all of the requirements of the Results Act. Of the six elements required by the Act, the Treasury draft plan included four. Of these four elements, two—the mission statement and key factors external to the agency that could significantly affect achievement of the strategic goals and objectives—generally met the Act’s requirements, but, as we stated, these could have been strengthened. The information contained in the plan on the two other elements—goals and objectives and the strategies to achieve them—was often too general and vague to be used effectively by Treasury management, Congress, and other stakeholders. We said these two elements could be improved if they were more specific, results oriented, and linked to the plans of Treasury’s bureaus and major program offices. Two elements—the relationship between long-term goals and objectives and annual performance goals, and a description of how program evaluations were used to establish or revise strategic plans—were missing from the draft plan we reviewed. The draft strategic plan did not adequately address crosscutting issues and made no mention of whether Treasury had coordinated with other federal departments and agencies that shared related functions. In addition, although a major part of the mission statement was focused on management, the draft plan did not adequately address some of the critical management problems facing Treasury that could affect its ability to achieve its strategic goals and objectives. Finally, we said that Treasury’s capacity to provide reliable information on the achievement of strategic and program performance was questionable. Treasury revised the four elements that were in its draft plan so that they better meet the requirements of the Results Act. In addition, the plan better addresses Treasury’s critical management problems and includes more information on how the Department plans to coordinate with other agencies on crosscutting issues. Treasury’s plan is now presented as an overview with more detailed information provided in the plans of its 17 bureaus and major program offices. Taken together, the 18 plans comprise Treasury’s strategic plan.Treasury’s goals and objectives are now linked with those of its bureaus and major program offices. Consequently, the plan provides a clearer discussion of which bureaus and program offices have responsibility for carrying out the goals and objectives. Treasury’s plan also states that details on resources needed to implement strategies are to be included in bureau and program office strategic plans as well as the Department’s budget submission. Because the plan also links the goals and objectives in the overview plan to the annual performance goals and measures in the strategic plans of the bureaus and program offices, it provides information on one of the elements required by the Results Act that was missing from the draft plan. Treasury has also made several other improvements to its plan. A section was added describing how program evaluations were used to develop the plan. This section also cites examples of planned evaluations that Treasury is to use as input for future plans. The plan includes more information aimed at addressing the critical management problems the Department faces. For example, an objective has been added to the plan to address the Year 2000 computer problems. Also, throughout its plan, Treasury addresses crosscutting issues by pointing out where coordination is required with other agencies. its bureaus and program offices. Treasury’s plan could also be improved if performance goals were provided for each objective and if some of these goals were more results oriented. Treasury could also improve its plan by more explicitly addressing its critical management problems. Finally, Treasury’s plan could better address issues relating to its capacity to provide the types of reliable data needed to measure performance and assess progress in meeting its goals and objectives. Treasury’s plan could be improved if the linkage between Treasury’s goals and objectives and those of its bureaus and program offices were more complete. Specifically, we found several gaps in the linkage between Treasury’s plan and the plans of its bureaus and components. For example, Treasury has an objective to “ensure strong financial management of Treasury accounts.” However, only six bureaus or program offices have corresponding objectives. For this objective, Treasury’s plan does not include a related objective for the Financial Management Service, which is responsible for managing the government’s finances, and IRS, the government’s primary revenue collector. Treasury’s plan could also be improved if more complete and detailed information on strategies for achieving goals and objectives were included in the plans of its bureaus and program offices. Treasury’s plan contains general information on some strategies that are needed to achieve its goals and objectives. It also states that more detailed information regarding resource needs is to be included in its budget submission and the plans of its bureaus and program offices. However, we found several instances where a Treasury objective was linked to a bureau objective, but the bureau plan contained no corresponding strategy. For example, Treasury has three law enforcement objectives—to reduce counterfeiting, money laundering, and drug smuggling—where IRS has a role. However, IRS’ plan contains no specific strategy related to these three objectives. Also, where strategies were found in the bureau plans, they could be improved if more detailed information, such as technological and resource needs, were included. For example, IRS lists several strategies, including expanding nationwide access to taxpayer information on-line and updating taxpayer information daily, to achieve its objective to “improve customer service.” However, IRS’ plan does not discuss the resources needed to carry out these strategies. Treasury’s plan has an objective to “improve capacity to recruit, develop, and retain high-caliber employees.” The plan lists six bureaus and program offices that have related objectives, but only one, the U.S. Mint, has a related performance goal. Likewise, Treasury could enhance its plan by making its performance measures more results oriented. For example, Customs’ strategic plan includes a strategy to prevent drug smuggling whose performance measures (the number of arrests, seizures, and convictions, for example) are output oriented. The plan could be improved if more results-oriented measures, focusing on lowered drug smuggling rates, were developed in support of Customs’ strategy to prevent drug smuggling. Treasury officials stated that they will attempt to develop results-oriented measures whenever possible, but that performance data may be difficult to collect in some cases, and output measures may be the best data available, at least for the near term. Furthermore, they felt that a balance of output-oriented and results-oriented measures may be desirable since the purpose of performance measures is to determine an agency’s effect on results. Nonetheless, Treasury’s plan could be further improved if results-oriented measures were developed to complement output measures wherever possible. As we observed in our review of Treasury’s draft strategic plan, the current plan could also be improved if it explicitly addressed all critical management problems. Although the plan states that it addresses all our high-risk areas and other critical management issues, its discussion of Treasury’s critical management problems is not always explicit. For example, IRS’ accounts receivable—a high-risk area—is not addressed specifically in Treasury’s or IRS’ plan. Both plans contain goals related to increasing compliance with the tax laws and improving customer service, which indirectly could address IRS’ accounts receivable. However, as we previously reported, Treasury’s strategic plan could be more useful to Congress and other stakeholders if it more clearly presented how Treasury will address its critical management problems and how this will facilitate the Department’s achievement of its strategic goals and objectives. performance measures are contained within the plans of the bureaus and program offices. However, the Treasury plan does not address the difficulties of developing measures and collecting reliable data for some important areas of performance. For example, IRS and the Bureau of Alcohol, Tobacco and Firearms both use taxpayer burden as a performance indicator, but neither agency has adequate measures or data for tracking taxpayer burden. We recognize that developing measures of some areas of Treasury’s performance, such as taxpayer burden, will be very challenging, but the Treasury plan does not discuss how the Department plans to deal with these challenges. On October 30, 1997, we obtained oral comments from Treasury officials, including the Director of the Office of Strategic Planning, on a draft of our analysis of Treasury’s strategic plan. The officials generally agreed with our observations but suggested several changes to clarify areas where Treasury has improved its plan. They said that it was important to emphasize that their July 1997 draft strategic plan was a “working document” issued as required for consultation purposes with Congress and other stakeholders. As a result of the consultation process, they said that the plan was revised to address the concerns of Congress and other stakeholders, including GAO and OMB. Also, while the officials agreed that the current plan could be further improved in several areas, they said that the plan meets the Results Act’s requirements in that it contains all six required elements. The officials also reiterated that the Department should be recognized for its Results Act implementation efforts. In particular, the officials told us that Treasury has reformatted its budget to serve as the performance plan required by the Results Act for the past 2 fiscal years. Also, last year, the Department issued its performance report for fiscal year 1996 as part of its budget submission—ahead of the Act’s requirements. They said that Treasury intends to better align its performance plan with the goals and objectives in its strategic plan and to submit the plan as part of its fiscal year 1999 budget request, scheduled to be released in February 1998. attempt to develop performance measures that are results oriented and for which reliable data exist. Jim White, Associate Director, Tax Policy and Administration Issues; General Government Division, (202) 512-9110. On July 11, 1997, we issued a report with our observations on the Department of Veterans Affairs’ (VA) draft strategic plan, dated June 9, 1997 (The Results Act: Observations on VA’s June 1997 Draft Strategic Plan, GAO/HEHS-97-174R). Following this review, VA issued two additional drafts, dated August 1 and August 15, 1997. The August 15 draft was sent by VA to OMB for review and interagency coordination. On September 18, 1997, we testified before the Subcommittee on Oversight and Investigations, House Committee on Veterans’ Affairs, on the improvements in VA’s August 15 draft and the challenges remaining for VA in implementing the Results Act.VA submitted its formally issued plan to Congress and OMB on September 25, 1997. On October 14, 1997, we briefed your staffs on the observations we made in our September 18 testimony and further observations based on our review of the formally issued strategic plan. We found that VA’s June 1997 draft strategic plan represented an inconsistent and incomplete application of the six key components of a strategic plan as required under the Results Act. Also, the draft plan was somewhat confusing and difficult to follow, mainly because it had several different levels of goals, objectives, and strategies. In addition, the draft plan had not clearly identified needs for VA to coordinate and share information with other federal agencies. In terms of the key strategic planning elements, VA’s draft plan (1) focused more on the process of providing benefits and services than on results of VA programs for veterans and their families; (2) lacked objectives and strategies for achieving some of VA’s major strategic goals—in particular, for veterans’ benefits programs; (3) provided only limited discussions of external factors beyond the control of VA that could affect achievement of strategic goals; and (4) was not based on formal program evaluations. VA officials acknowledged that these elements still need to be developed. The June 1997 draft included plans to establish a schedule of evaluations for VA’s major programs. These evaluations, in turn, would lead to development of results-oriented strategic goals. Also, the draft included plans to identify coordination efforts with other federal agencies and to develop communication mechanisms with them. VA made significant progress in making the strategic plan clearer, more complete, and more results oriented. Instead of presenting four overall goals, three of which were process oriented, VA has reorganized its draft strategic plan into two sections. The first section, entitled “Honor, Care, and Compensate Veterans in Recognition of Their Sacrifices for America,” is intended to incorporate VA’s results-oriented strategic goals. The second section, entitled “Management Strategies,” incorporates the three other general goals, related to customer service, workforce development, and taxpayer return on investment. VA believes that the process-oriented portions of the plan are important as a guide to VA’s management. We agree, as long as they are integrated with the plan’s primary focus on results. In addition, VA filled significant gaps in the discussions of strategic goals. The formally issued plan includes strategic goals covering all of its major programs and includes objectives, strategies, and performance goals supporting the strategic goals. VA’s strategic plan still needs improvement in four major areas: (1) development of results-oriented goals, (2) descriptions of how the goals are to be achieved, (3) discussion of external factors, and (4) discussion of coordination efforts with other agencies. Until VA makes improvements in these areas, its strategic plan will be incomplete and will not fully comply with the strategic planning requirements of the Results Act. Perhaps the most significant challenge for VA is to develop results-oriented goals for its major programs, particularly for benefit programs. For some major VA programs, the strategic plan’s goals are placeholders for results-oriented goals that have not yet been developed. For example, the general goals for four of five major benefit program areas—compensation and pensions, education, vocational rehabilitation, and housing credit assistance—are stated in terms of ensuring that VA is meeting the needs of veterans and their families. The objectives supporting VA’s general goal for its compensation and pension area are to (1) evaluate compensation and pension programs to determine their effectiveness in meeting the needs of veterans and their beneficiaries and (2) modify these programs, as appropriate. defining program results is difficult for programs where congressional statements of the program purposes and expected results are vague or nonexistent. This is an area where VA and Congress can make progress in further clarifying program purposes and expected results. Once VA has developed strategic goals focused on results, it can develop objectives and strategies for achieving the goals. Another remaining challenge for VA is to better integrate discussions of external factors that could affect its strategic planning. While VA added discussions of the implications of demographic changes among veterans, they are not linked to specific goals in the plan. For example, VA noted the impact of increased veteran death rates on demands for burials in VA and state veterans’ cemeteries. However, this is not linked to VA’s performance goals to complete specific numbers of cemetery construction and land acquisition projects by fiscal year 2002. Discussions of external factors were often limited to whether Congress would appropriate sufficient funds or make substantive legislative changes. Assessments of factors outside VA’s control, such as economic, social, and demographic changes, are also important in setting VA’s goals and in assessing VA’s progress in meeting them. The other remaining challenge for VA is to identify areas where it needs to coordinate and share information with other federal agencies, as well as develop coordination plans. VA’s strategic plan identifies this need and includes a goal to (1) identify overlaps and links with other agencies, (2) enhance communication links with other agencies, and (3) keep state directors of veterans’ affairs and other state officials apprised of VA benefits and opportunities for collaboration and coordination. VA had substantial consultations with Congress, and we participated in these consultations at the request of the House and Senate Committees on Veterans’ Affairs. In addition, VA held consultation sessions with representatives of veterans service organizations. VA has attributed improvements in its formally issued strategic plan to these consultations. VA officials have stressed that they consider strategic planning a continuing, long-term process. Based on comments by VA officials and the changes VA has already made to its strategic plan, we expect further improvements over the next few years. In transmitting the formally issued strategic plan to Congress and OMB, VA also provided detailed responses to comments on its draft plans from the House Committee on Veterans’ Affairs, GAO, OMB, veterans service organizations, and VA employee organizations. These comments addressed the observations in both our July 11 letter and September 18 testimony. In general, VA agreed with our observations and indicated areas where it has revised its plan since the June 1997 draft. Also, we sent a draft of this appendix to VA officials, who had no additional substantive comments. Cynthia M. Fagnoni, Associate Director, Veterans’ Affairs and Military Health Care Issues; Health, Education, and Human Services Division, (202) 512-7202. On July 30, 1997, we issued a report on the Environmental Protection Agency’s (EPA) draft strategic plan (Results Act: Observations on EPA’s Draft Strategic Plan, GAO/RCED-97-209R). EPA made revisions to the draft plan and formally submitted it to OMB and Congress on September 30, 1997. As requested, we have reviewed the September 1997 plan and compared the changes with the observations we made in our July 30 report. On October 15, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from that briefing are summarized herein. EPA’s draft strategic plan contained four of the six elements required by the Results Act: (1) a mission statement, (2) general goals and objectives, (3) approaches or strategies to achieve the goals and objectives, and (4) an identification of key external factors. For these four elements, we noted that the draft plan did not contain all of the details suggested by OMB Circular A-11 and/or that other improvements could be made to increase the plan’s usefulness. The two elements not included in the draft plan were (1) the relationship between the general goals and objectives and the annual performance goals and (2) the program evaluations used in developing the plan and a schedule for future evaluations. Although the draft plan contained a section on program evaluation, the discussion focused on the role of evaluation in assessing future results and provided general criteria for deciding which evaluations to perform in the future. The draft strategic plan did not discuss interagency coordination for crosscutting programs, activities, or functions that are similar to those of other federal agencies. It is important that the plan do so because EPA and other agencies carry out a number of mission-related activities that are crosscutting or similar. Our July 30, 1997, report noted that EPA had begun taking steps to coordinate its plan with other agencies, such as the Department of Energy and the National Aeronautics and Space Administration, to address crosscutting programs and activities. The draft plan included actions to address major management challenges that we had previously identified. However, it provided limited details on how these long-standing problems are to be resolved. addition, the agency strengthened the plan’s treatment of management problems by setting out several additional actions to resolve them. In the September 30, 1997, version of its strategic plan, EPA added the two elements required by the Results Act that were missing from the draft plan: (1) the relationship of the general goals in the strategic plan to the performance goals to be included in the annual performance plan and (2) the program evaluations used in developing its general goals and objectives. The issued plan also incorporates improvements in other elements required by the Results Act. For example, the section identifying key external factors was expanded to include other factors, such as producer and consumer behavior, that could directly affect the achievement of the plan’s goals and objectives. The mission statement was also revised to more closely coincide with the language of the agency’s statutes. EPA improved the clarity of its strategic plan in several ways. It added information that explains how the agency’s responsibilities for human health and the environment intersect with or support the work of other federal departments or agencies, such as the Departments of the Interior and Health and Human Services. It also added information that better describes the important role of the states as having primary responsibility for implementing many day-to-day environmental program activities, such as issuing permits and monitoring environmental conditions. In addition, EPA added statements to clarify the relationship among certain components of its plan, that is, the goals and objectives, guiding principles, and planned cross-agency program activities. Furthermore, an addendum listing the agency’s potential authorities was revised to identify the actual authorities by goal and objective. The information that EPA added on interagency coordination of the plan included the major steps it took to coordinate with other agencies. The plan also identifies a total of 25 federal agencies whose activities relate to EPA’s efforts under one or more of its goals. According to the plan, the actions taken to coordinate with other agencies on the plan will help to establish long-term efforts to address any inconsistencies, conflicts, or redundancies among federal programs, as identified in any future strategic and annual performance plans. Environmental Performance Partnership System was developed by EPA and the states in 1995 as a more collaborative approach to implementing environmental programs. The plan now sets out the objectives of the partnership system and identifies how they will be accomplished. In addition, the plan now makes conducting peer reviews and providing guidance on the science underlying the agency’s decisions an objective under the “sound science” goal. As noted in our July 1997 report, the use of peer review is an important means of ensuring the credibility of the scientific and technical documents that the agency uses in its work. Furthermore, EPA added a performance measure to the “effective management” goal dealing with the need to achieve success in implementing the Chief Financial Officers Act and the Government Management Reform Act. This performance measure will help ensure that EPA addresses financial management issues that resulted in the agency’s receiving a qualified opinion on its fiscal year 1996 financial statements. Several revisions that we suggested in our previous report have not been made. Some of these relate to improvements in aspects of the six elements required by the Results Act, while others deal with further improvement in the treatment of management and data problems and the effectiveness of the plan in conveying the agency’s priorities. Although the plan provides a general methodology for selecting future program evaluations and describes how they are to be used, it does not identify the general scope and time frames of the evaluations, as encouraged by OMB’s guidance. In addition, as in the draft plan, (1) some of the goals and objectives, such as those for effective management, are not stated in quantifiable or measurable terms; (2) staffing skills and resources are generally not discussed in describing how the plan’s goals and objectives are to be achieved; and (3) because strategies are generally organized by goal rather than objective, it is not always clear how specific strategies relate to specific objectives. Moreover, future revisions or updates of the plan could further benefit from a more detailed discussion of how other federal agencies and the states are to contribute to individual goals and objectives. demonstrating that the agency recognizes the significance of these problems and is committed to resolving them. As the strategic plan evolves over time, EPA could improve its effectiveness in conveying the agency’s priorities. The large number of goals and objectives, coupled with the guiding principles and planned cross-agency program actions, continues to make it difficult to discern EPA’s priorities. To better convey its priorities, EPA could directly relate the cross-agency programs to specific goals and objectives or further consolidate its goals or objectives. We provided a draft of our observations on EPA’s strategic plan for its review and comment. EPA officials, including the Director of the Office of Planning, Analysis, and Accountability, told us that the strategic plan was a product of a broader reform of the agency’s planning, budgeting, analysis, and accountability functions and that the consolidation and harmonization of these functions will, over time, bring about many of the improvements that we have suggested. EPA noted that as EPA refines its approaches to analysis and accountability, the agency will be better able to outline the prospective uses of program evaluation and consequent refinements to its goals, objectives, and performance measures. In addition, EPA said that the agency has taken the “unprecedented” step of submitting its first annual performance plan under the Results Act and its fiscal year 1999 budget request to OMB as a single document. According to EPA, this action has had the effect of transforming budgetary decisions into the structure of strategic goals and objectives, which makes the kind of direct budget and performance linkages that we have suggested. While we recognize that EPA’s annual performance plan will provide detailed information on resources, strategies, goals, and objectives, we believe that the strategic plan would be more complete and useful to congressional and other stakeholders if it provides an overview of resource needs and links the agency’s major strategies to individual goals and objectives. Peter F. Guerrero, Director, Environmental Protection Issues; Resources, Community, and Economic Development Division, (202) 512-6111. On July 22, 1997, we issued a report on the Federal Emergency Management Agency’s (FEMA) draft strategic plan (Results Act: Observations on the Federal Emergency Management Agency’s Draft Strategic Plan, GAO/RCED-97-204R). FEMA’s strategic plan was submitted to OMB and Congress on September 30, 1997. As requested, we have reviewed the September 30 strategic plan and compared it with the observations in our July 22 report. On October 16, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from that briefing are summarized herein. As we reported in July, FEMA’s draft plan indicated that the agency had made good progress toward fulfilling the requirements of the Results Act but needed improvements to fully meet those requirements. For instance, we observed that the draft plan lacked two of the six elements required by the Results Act: (1) the relationship between strategic goals and annual performance goals and (2) the role of program evaluations. The required elements contained in the plan could have better conformed to the Act’s requirements and OMB’s guidance. For example, the plan did not explicitly address the major legislation or executive orders that serve as a basis for FEMA’s mission statement, goals, and strategies or sufficiently deal with financial and information management issues that we and others have previously identified. We also noted that clarifying the linkage between FEMA’s strategic objectives and the strategies intended to achieve them would make the plan more useful to FEMA and to Congress. Furthermore, the draft plan did not address the roles of external stakeholders and how FEMA coordinated with them in developing the plan. FEMA’s September 30 plan incorporates many improvements that make it more responsive to the requirements of the Results Act. First, it contains explicit language describing the role of program evaluations, a key element missing from the earlier version. For instance, the plan includes for each of FEMA’s three strategic goals a discussion of the type(s) of program evaluation that have been completed and are planned to help assess the accomplishment of those goals. and the agency in ensuring that FEMA’s stated goals respond to the entire spectrum of its key statutory authorities. Following suggestions in our July report, FEMA’s September 30 plan elaborates on certain aspects of two issues that we felt were not fully discussed in the draft plan: (1) management issues that we and others have previously identified and (2) the agency’s capacity to provide reliable information assessing the achievement of its goals and objectives. For example, the plan now reflects consideration of containing disaster program costs and remedying financial management problems. It also now more fully addresses how FEMA intends to address the Year 2000 problem, which is an issue that we have identified as high risk across the government. Unlike the earlier version, the September 30 plan discusses external stakeholders involved in the development of the plan. Identifying external stakeholders is important given the many and varied stakeholders that have critical roles in determining the extent to which FEMA’s goals are met. For example, the U.S. Army Corps of Engineers provides assistance for constructing flood control facilities and clearing debris from disaster-ravaged areas. FEMA could further enhance its strategic plan by clearly identifying federal agencies, or programs within those agencies, with related missions or potentially crosscutting program activities, and how coordination with them shaped FEMA’s plan. Some of the elements of FEMA’s strategic plan could be further improved to more fully meet the purposes of the Results Act. For example, while the revised plan incorporates language on the relationship between annual goals and the strategic goals and objectives, the plan would benefit from elaboration on this issue. OMB’s guidance suggests that strategic plans include a discussion of the type, the nature, and the scope of the performance goals to be included in the annual performance plans. While FEMA’s September 30 plan states that the agency’s annual performance plans will illustrate how annual performance goals will support the strategic goals and objectives, it lacks an explicit discussion of the type, the nature, and the scope of the performance goals to be included in the plans and their linkages to the strategic goals and objectives. also describe how achieving the goals could be influenced by the factors. While the plan contains a section on external factors, it does not link the factors to specific goals or objectives or articulate strategies for mitigating the factors’ effects. Also, in our July report, we suggested that FEMA’s plan could be strengthened if the strategies were more integrally linked to FEMA’s strategic objectives. The September 30 plan does more clearly link strategies with overall goals, although not with specific objectives. Because of this structure, the plan is not as useful as it could be in assigning accountability for achieving specific objectives. The Results Act requires that strategic plans contain goals and objectives that are expressed in a manner allowing a future assessment of whether they are being achieved. While the goals in FEMA’s September 30 plan are not substantially different from those in the earlier version, the proposed assessment approaches are. The revised approaches raise questions as to their feasibility. For example, the first goal—“protect lives and prevent the loss of property from all hazards”—includes an approach that relies on incomplete modelling and data collection efforts and an implied but unquantified relationship between an increase in readiness and a decrease in risk. Because of the potential assessment difficulties, it is less clear that FEMA’s goals and objectives are expressed in a manner that allows a future assessment of whether they are being achieved. The plan’s usefulness could be enhanced if it were easier to read and follow. More explanatory language and/or a visual “road map” might help show how the major elements of the plan relate to one another. For example, a few sentences explaining that the operational objectives link to the strategic goals rather than the strategic objectives would be helpful. Finally, providing clarifying and simplified language would enhance the plan’s usefulness to audiences external to FEMA. achievement of goals and objectives. We incorporated their suggested changes where appropriate. Judy A. England-Joseph, Director, Housing and Community Development Issues; Resources, Community, and Economic Development Division, (202) 512-7631. On July 7, 1997, we issued a report on the General Services Administration’s draft strategic plan (The Results Act: Observations on GSA’s April 1997 Draft Strategic Plan, GAO/GGD-97-147R). GSA has since revised its strategic plan and formally submitted it to OMB and Congress on September 30, 1997. As requested, we have reviewed the September 30 strategic plan and compared it with the observations in our July 7 report. On October 16, 1997, we briefed your offices on our further observations on the strategic plan. The key points from that briefing are summarized herein. We reported in July that the April 28 draft plan included the six components required by the Results Act and that the general goals and objectives in the plan reflected GSA’s major statutory responsibilities. However, our analysis showed that the plan could have better met the purposes of the Act and related OMB guidance. Two of the required components—how goals and objectives were to be achieved and program evaluations—needed more descriptive information on how goals and objectives were to be achieved, how program evaluations were used in setting goals, and what the schedule would be for future evaluations to better achieve the purposes of the Act. The four other required components—mission statement, general goals and objectives, key external factors, and relating performance goals to general goals and objectives—were more responsive to the Act but needed greater clarity and context. We also noted that the general goals and objectives and the mission statement in the draft plan did not emphasize economy and efficiency, as a reflection of taxpayers’ interests. Also, the general goals and objectives seem to have been expressed in terms that may be challenging to translate into quantitative or measurable analysis, and there could have been better linkages between the various components of the plan. We also reported that the draft plan could have been made more useful to GSA, Congress, and other stakeholders by providing a fuller description of statutory authorities and an explicit discussion of crosscutting functions, major management problems, and the adequacy of data and systems. Although the plan reflected the major pieces of legislation that establish GSA’s mission and explained how GSA’s mission is linked to key statutes, we reported that GSA could provide other useful information, such as listing laws that broaden its responsibilities as a central management agency and that are reflected in the goals and objectives. Relatedly, the draft plan did not discuss the potential for crosscutting issues to arise or how these issues might affect successful accomplishment of goals and objectives. It also made no mention of whether GSA coordinated the plan with its stakeholders. The plan was also silent on the formidable management problems we have identified over the years—issues that are important because they could have a serious impact on whether GSA can achieve its strategic goals. Finally, the plan made no mention of how data limitations would affect GSA’s ability to measure performance and ultimately manage its programs. We reported that consideration of these areas would give GSA a better framework for developing and achieving its goals and help stakeholders better understand GSA’s operating constraints and environment. The September 30 plan reflects a number of the improvements that we suggested in our July 1997 report. The clarity of the September 30 plan is improved, and it provides more context, descriptive information, and linkages within and among the six components that are required by the Act. Compared to the April 28 draft, the September 30 plan generally should provide stakeholders with a better understanding of GSA’s overall mission and strategic outlook. Our analysis of the September 30 plan also showed that, in line with our suggestion, GSA placed more emphasis on economy and efficiency in the comprehensive mission statement and general goals and objectives components. The September 30 plan also generally described the operational processes, staff skills, and technology required, as well as the human, information, and other resources needed, to meet the goals and objectives. The plan now contains a listing of program evaluations that GSA used to prepare the plan and a more comprehensive discussion of the major pieces of legislation that serve as a basis for its mission, reflecting additional suggestions we made in our July 1997 report. improvements, which are described in the following section, would strengthen the strategic plan as it evolves over time. As we discussed in our July 7, 1997, report on the draft plan, the September 30 plan continues to have general goals and objectives that seem to be expressed in terms that may be challenging to translate into quantitative or measurable analysis. This could make it difficult to determine whether they are actually being achieved. For example, the goal to “compete effectively for the federal market” has such objectives as “provide quality products and services at competitive prices and achieve significant savings” and “open GSA to marketplace competition where appropriate to reduce costs to the government and improve customer service.” However, this goal, its related objectives, and the related narrative do not state specifically how progress will be measured, such as the amount of savings GSA intends to achieve or the timetable for opening the GSA marketplace for competition. OMB Circular A-11 specifies that general goals and objectives should be stated in a manner that allows a future assessment to be made of whether the goals are being met. The OMB guidance states that general goals that are quantitative facilitate this determination, but it also recognizes that the goals need not be quantitative and that related performance goals can be used as a basis for future assessments. However, we observed that many of the performance goals that GSA included in the plan also were not expressed in terms that could easily enable quantitative analysis, which could make gauging progress difficult in future assessments. The strategies component—how the goals and objectives will be achieved—described the operational processes, human resources and skills, and information and technology needed to meet the general goals and objectives. This component is an improvement over the prior version we reviewed, and applicable performance goals are listed with each of these factors. Although GSA chose to discuss generally the factors that will affect its ability to achieve its performance goals, we believe that a more detailed discussion of how each goal will actually be accomplished would be more useful to decisionmakers. To illustrate with a specific example, the plan could discuss the approaches that GSA will use to meet the performance goals related to its general goal of promoting responsible asset management using operational processes, human resources and skills, information and technology, and capital/other resources. The September 30 plan does discuss, in the general goals and objectives component, an operational/human resource change involving the appointment of a new Chief Measurement Officer in the Public Buildings Service. More discussion of this type of change in the strategies component would help stakeholders better understand GSA’s specific strategies to ensure that it is achieving its goals and objectives. We also noted that the strategies component does not discuss priorities among the goals and objectives. Such a discussion would be helpful to decisionmakers in determining where to focus priorities in the event of a sudden change in funding or staffing. Finally, GSA deferred to the President’s budget its discussion about capital and other resources. We believe it seems reasonable to include in this component at least some general discussion of how capital and other resources will be used to meet each general goal. Although the external factors component in the September 30 plan is much clearer and provides more context than the draft plan we reviewed, the factors are not clearly linked to the general goals and objectives. OMB Circular A-11 states that the plan should include this link, as well as describe how achieving the goals could be affected by the factors. This improvement would allow decisionmakers to better understand how the factors potentially will affect achievement of each general goal and objective. The program evaluations component in the September 30 plan provides a listing of the various program evaluations that GSA indicates were used in developing the plan. However, it still does not include a schedule of future evaluations. Instead, the plan states that the schedule for future program evaluations is under development and that GSA intends to use the remainder of the consultation process to obtain input from Congress and stakeholders concerning the issues that should be studied on a priority basis. However, OMB Circular A-11 indicates that the schedule should have been completed and included in the September 30 plan, together with an outline of the general methodology to be used and a discussion of the particular issues to be addressed. decisionmakers understand the crosscutting issues that affect GSA as a central management agency. However, explicit discussion of these issues is limited, and the September 30 plan makes no reference to the extent to which GSA coordinated with stakeholders. The September 30 plan references major management problems in the program evaluations component, but it does not explicitly discuss these problems or identify which problems could have an adverse impact on meeting the general goals and objectives. Our work has shown over the years that these types of problems have significantly hampered GSA’s and its stakeholder agencies’ abilities to accomplish their missions. For example, the plan could address how GSA will attempt to ensure that its information systems meet computer security requirements or how GSA plans to address the Year 2000 problem in its computer hardware and software systems. The plan does reference data reliability in the general goals and objectives component. However, the discussion of data reliability, which is so critical for measuring progress and results, is limited and not as useful as it could be in attempting to assess the impact that data problems could have on meeting the general goals and objectives. We continue to believe that greater emphasis on how GSA plans to resolve management problems and on the importance of data reliability could improve the plan. On October 9, 1997, we obtained oral comments from GSA’s Director of Performance Management on a draft of our analysis of GSA’s September 30 plan. She said that GSA generally agreed with our observations about the September 30 plan and said that many of the observations will be addressed in future versions of the plan and in the various performance plans that GSA has drafted. However, she added that GSA is concerned that many of our observations could lengthen the plan, thereby making it less usable or readable to GSA’s broad constituency, including Congress, OMB, GSA employees, and GSA’s various business partners. concerns in a succinct fashion that is both usable and reader-friendly while, at the same time, better achieving the purposes of the Act and related OMB guidance. Bernard L. Ungar, Director, Government Business Operations Issues; General Government Division, (202) 512-4232. On July 11, 1997, we issued a report on the Office of Personnel Management’s (OPM) draft strategic plan (The Results Act: Observations on OPM’s May 1997 Draft Strategic Plan, GAO/GGD-97-150R). OPM’s formally issued strategic plan was submitted to OMB and Congress on September 29, 1997. As requested, we have reviewed the publicly issued strategic plan and compared it with the observations in our July 11 report. On October 17, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from that briefing are summarized herein. Of the six components required by the Results Act, two—how the goals and objectives will be achieved and relating performance goals to general goals/objectives—were not specifically identified in the draft plan. The remaining four components—mission statement, goals and objectives, external factors, and program evaluations—were discussed in the draft plan. However, each of these components had weaknesses, some of more significance than others. Specifically, the mission statement was too broad, lacking explicit reference to certain key responsibilities; the goals and objectives suggested some results to be achieved but provided little basis for judging how OPM would know whether those goals were being achieved or what OPM’s contribution toward achieving those results might be; some external factors were identified, but others were not included in the plan, which also did not meet the Act’s requirement to link each factor to a particular goal or to identify how it might affect OPM’s success in meeting its goals; and, finally, the program evaluation component discussed customer satisfaction with OPM services but did not indicate how evaluations were used in developing strategic goals or provide a schedule for future evaluations as the Act requires and OMB guidance reiterated. In addition, while the draft plan did identify a number of OPM’s crosscutting program activities, it did not discuss coordination or indicate that OPM, in developing the plan, coordinated with the entities involved in these crosscutting activities. Including a fuller discussion of OPM’s interrelationship with other agencies in the plan would be consistent with the purposes of the Act. Such a discussion likely would also provide more information for Congress and other stakeholders to use in judging whether OPM’s crosscutting responsibilities should be modified in any way. competitive in obtaining future workers; (2) determining whether federal employee compensation (e.g., pay and benefits) is appropriate; and (3) ensuring that decisions for information technology projects are based on assessments of mission benefits, risks, and costs. Discussion of these challenges as well as of management problem areas where OPM has taken successful corrective actions would be informative and useful to both OPM and Congress. OPM’s publicly issued strategic plan incorporated several improvements that make it more responsive to the requirements of the Results Act than was the draft plan. In particular, OPM revised its mission statement to recognize its key responsibilities. OPM’s revised mission statement is more results oriented and outlines OPM’s functions and activities as the government’s central personnel agency. OPM has added specific sections to its plan describing OPM and what it does, OPM’s history, and its statutory responsibilities. These sections augment the mission statement by linking it to relevant statutory authorities. Two of the components required by the Results Act that were missing when we reviewed the draft plan in July—how the goals and objectives will be achieved and relating annual performance goals to general goals/objectives—have been added to the issued strategic plan. OPM has a section under each goal entitled “Strategies for Achieving Objectives” that lists general action items OPM has identified for achieving its goals and objectives. Also, under each strategic goal, OPM has proposed measures to assess progress toward its overall goals. In several cases, OPM has established measurable targets that can be used to gauge the agency’s progress. OPM’s issued plan includes a section on the relationship between its strategic goals and objectives and its forthcoming annual performance plans. This section generally states that strategic goals will be linked to specific performance goals and performance improvements in the annual performance plan and that the plan’s program evaluation section further links the strategic plan to the annual performance plans. OPM’s evaluation agenda and schedule and also describes the evaluations used to provide baseline data for some of OPM’ s performance measures. We observed in July that the draft plan did not assess the potential for overlap and duplication or, conversely, cooperation and coordination with agencies and others on crosscutting issues. The issued plan includes a strategy to identify and solve common problems and avoid duplication of effort by working cooperately with consortia, agencies, and interagency groups such as the Interagency Advisory Group of Federal Human Resources Directors. OPM’s issued plan includes specific sections on its information technology and financial management systems strategies. For example, the financial management strategies section addresses goals in improving OPM’s financial information. This is a positive step since data reliability is extremely important for obtaining reliable performance measures to evaluate management performance and measure progress and results. Although the plan does not have specific sections that address other challenges, such as attracting and retaining well-qualified employees and determining appropriate compensation, as we had suggested, we note that OPM has included under its strategic goals certain objectives and strategies regarding staffing and examining and federal compensation. Although OPM made several improvements that we suggested in our previous report, some elements of its issued plan could be further improved. OPM’s five strategic goals, which we previously characterized as process oriented as opposed to results oriented, have not been revised from OPM’s May 1997 draft. However, for each of the five goals, OPM has provided a number of corresponding results-oriented objectives. Several of these objectives are time specific and allow future assessments to be made of whether they were or are being achieved. We acknowledge that not all objectives may lend themselves to quantification; however, this element of the plan could be more helpful to decisionmakers if more of the objectives had specified time frames, quantifiable targets, or identified base points against which progress could be measured. For example, neither OPM’s objective for fostering movement by senior executives nor its associated measures of success provide a sense of how much more movement may be desirable or how OPM or others will know when movements of executives have reached a more appropriate level. As previously mentioned, OPM has added specific strategies for achieving its objectives. However, these strategies generally do not include a description of the processes and the human, capital, and information resources required to achieve the goals and objectives as called for by the Results Act. OPM officials point to the fifth goal and its accompanying strategies as providing this information. However, although one strategy states in part that OPM will “acquire the necessary resources,” thus implying that additional resources will be needed, none of the strategies under this goal specify the necessary resources, costs, or information technology OPM will need to achieve its goals. One of the elements required by the Results Act that was missing when we reviewed the draft plan in July—the relationship between long-term goals and objectives and annual performance plans—is addressed in the issued strategic plan as previously described. By explicitly recognizing that future annual performance goals will be needed to assess progress toward the targets set in the strategic plan and by adding additional measurable targets to its issued plan, OPM has provided a greater assurance that combined information in the strategic and annual plans will be useful to OPM and stakeholders in tracking OPM’s progress. However, particularly because a significant number of OPM’s goals and objectives are not expressed in a manner readily susceptible to progress assessments, additional discussion of how OPM will assess progress over the 5-year period covered by the plan would have been useful. OPM’s discussion of external factors also could be further improved. Specifically, OPM could provide more information on how the external factors may affect goals and also identify additional actions by OPM to reduce potential impact. For example, the issued plan notes that the accelerated loss of experienced managers and personnelists throughout the federal government may affect strategic goals II, III, and IV. However, the plan does not indicate how OPM believes this external factor will affect each goal, nor does it indicate the actions that OPM plans to take to reduce the potential impact of the factor on OPM’s efforts to achieve its goals. On October 17, 1997, the Acting Director of OPM provided written comments on a draft analysis of the issued strategic plan. OPM expressed appreciation for our recognition that improvements had been made in its plan and said that it would give consideration to our comments as it further revises and updates the strategic plan and also as it develops the annual performance plan. Michael Brostek, Associate Director, Federal Management and Workforce Issues; General Government Division, (202) 512-8676. On July 22, 1997, we issued a report on the National Aeronautics and Space Administration’s (NASA) draft strategic plan (Results Act: Observations on NASA’s May 1997 Draft Strategic Plan, GAO/NSIAD-97-205R). NASA formally transmitted its strategic plan to OMB and Congress on September 30, 1997. As requested, we reviewed this plan and compared it with the observations in our July report. On October 17, 1997, we briefed your staffs on our further observations on the plan. The key points from that briefing are summarized herein. NASA’s draft strategic plan included four of the six elements required by the Results Act. Those four elements were a mission statement, goals and objectives, strategies for achieving the goals and objectives, and a discussion of external factors. Two of the four elements had weaknesses. The other elements—relating annual performance goals to general goals and objectives, and providing a description of program evaluations used to establish general goals and objectives and a schedule of future program evaluations—were not explained in enough detail in the draft plan. Although many of NASA’s objectives are shared with or involve other agencies, the draft plan did not discuss whether interagency coordination occurred to address duplication or overlap of activities. Also, the draft plan did not address the importance of working with other agencies to achieve its objectives. Although NASA officials said that activities are coordinated at the program level, such efforts were not discussed in the plan. Major management problems that could affect NASA’s ability to achieve its mission were not explicitly discussed in the draft plan. For example, NASA did not discuss its long-standing problems with managing contracts, managing information technology, and developing a fully integrated accounting system, even though the agency has recognized them as problems and has initiated some steps designed to address them. This information could be beneficial to NASA and its stakeholders because major management problems could impede the agency’s efforts to achieve its goals and objectives. performance goals to general goals and objectives and (2) describing how program evaluations were used to establish general goals and objectives and a schedule for future program evaluations. In addition, the relationship between questions and missions has been clarified. NASA’s plan now includes a more detailed discussion of the element relating performance goals to general goals and objectives. A new chart (characterized by NASA as the “Strategic Management System Roadmap”) illustrates the relationship between agency-level goals and the goals and objectives of the four Enterprises. Also, in the crosscutting processes section of the plan, NASA provides examples of how agency goals relate to performance goals. In our July report, we observed that the draft plan did not clearly indicate whether near-term, mid-term, or long-term goals would be used for performance measurement, or whether performance would be measured against the strategic outcomes, agencywide goals, or Enterprise goals. The plan now includes provisions for reviewing performance goals against the near-term objectives of the Enterprises and the four crosscutting processes that support all agency activities. Responding to our concern that the draft plan did not provide a description of program evaluations used to establish general goals and objectives and a schedule of future program evaluations, NASA has included a description of its strategic plan provisions for semiannual reviews by NASA’s Senior Management Council. These reviews are to take place in March and September of each year. The plan also provides a more detailed explanation of NASA’s planning process. According to the plan, NASA’s Strategic Management System will provide the information and results to fulfill the planning and management requirements of the Results Act. Furthermore, a series of documents, such as the Headquarters Functional/Staff Office Implementation Plans, will explain how NASA plans to implement activities to accomplish its goals. The draft plan included “fundamental questions” posed by the NASA Administrator. In our July report, we noted that these questions were not discussed in the context of the stated missions of the agency. The plan addresses this concern by including the questions in the Strategic Management System Roadmap chart and linking them to the mission, Enterprises, and crosscutting processes. implementation of an integrated financial management system, contract management reform, and information technology management in the context of their having been long-standing management challenges. On October 15, 1997, NASA’s Senior Advisor for Strategic Planning and Management provided us with the agency’s comments on our observations about its strategic plan. The Senior Advisor made three points. First, he said that NASA strongly believes that the agency has addressed all six required elements of the Results Act. Second, he stated that NASA is pleased that we recognize that many improvements have been made. He added that NASA has gone to great lengths and effort to ensure that concerns about the draft strategic plan expressed by Congress, OMB, and GAO were addressed. In particular, he said that numerous examples have been added to illustrate the fact that NASA is committed to leveraging other agency programs and resources. Third, NASA agrees with our view that developing a strategic plan is a dynamic process and that NASA will consider our suggested improvements when the agency moves forward in future updates to the plan. Allen Li, Associate Director, Defense Acquisitions Issues; National Security and International Affairs Division, (202) 512-4841. On July 11, 1997, we issued a report on the National Science Foundation’s (NSF) draft strategic plan (Results Act: Observations on the National Science Foundation’s Draft Strategic Plan, GAO/RCED-97-203R). NSF’s formally issued strategic plan was submitted to OMB and Congress on September 30, 1997. As requested, we have reviewed the publicly issued strategic plan and compared it with the observations in our July 11 report. On October 15, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from that briefing are summarized herein. Of the six elements required by the Act, one—external factors that could affect the achievement of the plans’ goals—was not specifically identified in the draft plan. Of the remaining five elements, three—goals and objectives, strategies for achieving goals, and how program evaluation was used—were discussed but were not complete. Specifically, some of the goals were not expressed in a measurable form, the strategies to achieve NSF’s goals lacked precision, and the description of program evaluations was not fully developed. In addition, we observed that the draft plan could be more useful to NSF, Congress, and other stakeholders if it provided a more explicit discussion of crosscutting activities, statutory authorities, and NSF’s capacity to provide reliable information to manage its programs or determine if it is achieving its strategic goals. Recognizing crosscutting issues and the coordination required to address them is particularly important for NSF because in the science and technology area, for which the federal government spent $60 billion in fiscal year 1996, the potential for unnecessary overlap is particularly high. While NSF’s draft plan reflected its key statutory authority, other statutes broaden the scope of its responsibilities and are embedded in NSF’s goals and objectives. Explicit consideration of NSF’s capacity to produce reliable information for management decisionmaking is important because it could affect NSF’s ability to develop and meet its goals. future program evaluation efforts, (4) interagency crosscutting activities, and (5) how NSF plans to use information technology. External factors are now addressed in appendix 1 of NSF’s plan. In it, NSF describes the challenges that science and engineering faculty and students face in the current research environment and identifies how the achievement of four of its five goals could be affected by external factors. Consistent with OMB Circular No. A-11, Part 2, NSF briefly describes external factors, their link with a particular goal, and how the achievement of the goal could be affected by the factor. For example, for goal 1—“discoveries at and across the frontier of science and engineering,” NSF raises concern about the quality of research facilities and their influence on the pace of discovery. In particular, NSF relies on the academic research facilities available at colleges and universities to provide a base from which grantees can build their research programs. To the extent that moves toward cost efficiency in academic institutions affect this base, allowing it to deteriorate or failing to maintain it at the state of the art, NSF’s costs for the support of research will increase, which could slow the pace of discovery or change the types of discoveries open to researchers. NSF states that it would need to balance the number of researchers whose work could be supported with the added cost of conducting the research. NSF’s outcome goals are addressed more fully in appendix 2 and in the body of the report. In our earlier report on the draft plan, however, we had several reservations about NSF’s goals, some of which remain. In particular, we noted in our earlier report that while NSF’s draft plan provided some general dates for achieving its goals, it did not provide underlying assumptions, projections, or a schedule for initiating or completing significant actions. It also lacked a process for communicating goals and objectives throughout the agency and for assigning accountability to managers and staff for the achievement of goals. While the goals are still not expressed in a measurable fashion, the plan now describes examples of performance goals for NSF management and programs and includes both investment strategies and action plans for achieving each goal. According to OMB Circular A-11, when the goals are defined in a way that precludes a direct, future determination of achievement, the performance goals and indicators in the performance plan should be used to provide the basis for assessment. According to NSF, the action plans are provided to operationalize each strategy; to provide guidance to program officers; and through quantitative indicators, to link the goals to the development of annual budgets and performance plans. Current and future program evaluation efforts are now addressed in appendix 3. NSF discusses how the agency used specific formal and informal evaluations to develop key investment strategies, action plans for the strategic plan, and aspects of performance plans. Also included are details on future evaluations, a rough schedule for their implementation, and how the findings could be useful (1) in assessing NSF’s progress toward outcome goals and (2) for strategic planning discussions. For example, a Results Act pilot project on the physical sciences facilities gave NSF experience with setting performance goals and performance baselines for NSF’s oversight of the construction and operation of large facilities. This effort facilitated NSF’s development of appropriate performance goals for facilities management that are applicable across NSF for its performance plans. In addition, in connection with future evaluations and beginning in fiscal year 1998, NSF is planning to develop a formal process of assessment that includes periodic external assessment of progress toward outcome goals. programs are the U.S. Global Change Research Program and the High Performance Computing and Communications Program, both of which NSF participates in. Information technology in support of NSF’s mission is now discussed in appendix 5, and strategies for addressing information technology needs are identified in the section on “Critical Factors For Success.” In connection with its attention to Year 2000 issues, NSF states that it sent a notification to all grantees describing this potential problem and making clear that grantees bear the responsibility of addressing any difficulties it might create for the conduct of the research and education awards they hold. NSF refers to the fiscal year 1996 Annual Financial Report in its strategic plan; in that report, the Chief Financial Officer noted that NSF continues to meet or exceed virtually every federal goal for financial management performance. In addition, NSF has noted its commitment to manage its systems in support of the Results Act and the Chief Financial Officers Act as a key strategy in its plan. We observed in July that the draft plan could be enhanced by further discussion of statutory authorities and additional detail on NSF’s use of information. While our earlier report indicated that NSF’s statutory responsibilities were generally reflected in NSF’s draft strategic plan, we also stated that NSF is subject to other statutes related to its core functions. We suggested that providing a description of its responsibilities under its various statutory authorities could be useful, as a supplement to its plan, since the plan includes goals and objectives based on them. We also noted that it might be helpful to link the stated outcome goal to the relevant statutory objective. In this regard, NSF’s mission statement briefly touches on additional charges to the agency beyond the initial authorizing legislation, but the plan does not attempt to link NSF’s goals and strategies to the relevant statutory objective. As previously stated, information technology in support of NSF’s mission is now discussed in appendix 5, and strategies for addressing information technology needs are identified in the section on Critical Factors For Success. However, with respect to the high-risk issue of information security, the plan is still silent. Also, the revised plan does not discuss how NSF intends to improve its accounting for property, plant, and equipment in the possession of contractors and grantees in order to attain an unqualified audit opinion—which would seem to be a key goal for the financial management area. NSF’s performance goals for the results of its investments will appear as descriptive standards developed under the Results Act option to set performance goals in alternative formats. Since the timing of outcomes from NSF’s activities is unpredictable and annual change in the outputs does not provide an accurate indicator of progress toward outcome goals, performance goals for results are not specific to a fiscal year. NSF plans to use data and information on the products of NSF’s investments combined with the expert judgment of external panels to assess NSF’s performance over time and to provide a management tool for initiating changes in direction, when needed. As we stated in our earlier report, quantitative and qualitative indicators are widely used as proxies to assess research and development results because of the difficulties in identifying the impacts of research. Yet, while implying a degree of precision, these indicators were not originally intended to measure long-term research and development results. It remains to be seen whether NSF’s use of descriptive standards to evaluate results will become valuable sources of information for tracking progress and measuring outcomes. On October 10, 1997, we spoke with NSF’s Assistant to the Director for Science Policy and Planning to obtain the agency’s comments on our observations about its strategic plan. NSF generally supported our observations and agreed that some stakeholders may find useful the addition of an appendix explicitly identifying the links between the goals and strategies to the relevant statutory objective. In addition, NSF pointed out that attention to information security is addressed in another strategic plan in accord with the Information Technology Management Reform Act. Finally, with respect to NSF’s accounting for property, plant, and equipment, the agency indicated that NSF is taking necessary preliminary steps while awaiting guidance from the Federal Accounting Standards Advisory Board and OMB and expects to address this topic in a forthcoming performance plan. Victor S. Rezendes, Director, Energy, Resources, and Science Issues; Resources, Community, and Economic Development Division, (202) 512-3841. On July 31, 1997, we issued a report on the Nuclear Regulatory Commission’s (NRC) draft strategic plan (The Results Act: Observations on the Nuclear Regulatory Commission’s Draft Strategic Plan, GAO/RCED-97-206R). NRC’s formally issued strategic plan was submitted to OMB and Congress on September 30, 1997. As requested, we have reviewed the publicly issued strategic plan and compared it with the observations in our July 31 report. On October 15, 1997, we briefed your staffs on our further observations on the September 1997 plan. The key points from the briefing are summarized herein. The draft strategic plan contained two of the six required components of the Results Act—the mission and the goals and objectives. While NRC’s draft strategic plan met some of the requirements for three other components, it did not describe (1) the resources, such as staff skills and experiences, capital, and information, that will be needed to execute the plan’s strategies; (2) how key external factors could affect the achievement of its goals; and (3) its schedule for future program evaluations. Finally, NRC had not included in its draft plan the relationship between its long-term goals and objectives and its annual performance goals. Although NRC shared its draft and consulted with other agencies, the draft strategic plan did not fully discuss some programs and activities that were crosscutting or similar to those of other federal agencies. For example, NRC and the Department of Energy (DOE) share the responsibility for the federal government’s high-level waste disposal program; DOE builds such facilities, which NRC must license. Consequently, NRC is affected by changes in DOE’s strategies and program funding. The draft plan would have benefited by a more thorough discussion of these issues. realistic, which could mean future problems for those licensees not having sufficient funds to properly close their facilities. NRC’s September 1997 plan incorporated several improvements that make it more responsive to the requirements of the Results Act than was the draft strategic plan. In response to our concern that resource needs to execute strategies were not discussed, NRC added a statement to the September 1997 plan explaining that it did not anticipate any major, unique resource requirements and that its budget will identify the specific resources needed to implement the plan. NRC noted in its September 1997 plan that performance indicators have been established for human, capital, information, and funding resources in its performance plan. NRC explained that in the event legislation is enacted to have NRC oversee DOE’s facilities, changes to NRC’s strategies and resource needs could be required. NRC also added key external factors, which it called “major factors or assumptions,” affecting the achievement of its goals for the two strategic arenas that had none—“Protecting the Environment” and “Excellence.” NRC also expanded its goals section to provide a clearer link between the long-term (general) goals in its September 1997 strategic plan and those to be included in the annual performance plan. NRC added intermediate performance goals from the annual performance plan to the general goals to show the relationship between the final September 1997 plan and the annual performance plan; it also provided additional measures of results. In July, we observed that NRC did not fully address crosscutting program activities. The September 1997 plan was extensively revised to include a section in the appendix, entitled “Cross-Cutting Functions,” that identifies major crosscutting functions and interagency programs and discusses NRC’s coordination with other agencies, such as DOE and the Environmental Protection Agency (EPA). plan to discuss its actions to provide reliable performance information . Most of the data that NRC plans to use to measure performance will come from existing reports to Congress; and, in fiscal year 1998, it plans to identify any primary data systems that require improvement to provide any other information needed. NRC also addressed our concern that it had not discussed legislative needs that it may have had. NRC added a statement to its September 1997 plan to indicate its conclusion that it had not identified a need for any significant legislative changes to achieve its goals and strategies. NRC noted, for certain substrategies related to reactor and nonreactor decommissioning, that it is seeking legislation that would eliminate the overlap in the standard-setting authority of NRC and EPA in connection with Atomic Energy Act sites and materials by recognizing NRC’s and Agreement States’ standards in these areas. While NRC described in its September 1997 plan its program evaluation process, NRC still needs to include schedules for future program evaluations as required by the Results Act. Moreover, the September 1997 plan does not describe the general methodology to be used and the scope and issues to be addressed in such evaluations. The NRC plan indicates that no unique resources are anticipated, but it does not explicitly describe the resources and processes required to achieve its goals—in particular, its goal for nuclear waste safety. The Act states that the strategic plan is to contain a description of how the goals and objectives are to be achieved, including a description of the operational processes, skills and technology, and other resources required to meet the goals and objectives. To the extent that the achievement of a goal (i.e., the nuclear waste safety goal) relies on the resources or activities of others, NRC should describe those resources and activities in describing how its goals are to be achieved. Discussions of major management challenges and how NRC will meet them should appear in NRC’s plan, either under its “Excellence” goal or as strategies for achieving programmatic goals. While NRC’s 1997 strategic plan provides a set of strategies that are linked to specific goals, these strategies could be more complete. planned or directly under way and will not provide the information needed to assess the achievement of the strategic goals. Also, the precise meaning of some of its goals—in particular, its “Common Defense and Security and International Involvement” goal relating to international involvement—could be further clarified. We observed in July that NRC’s draft strategic plan did not discuss how NRC intended to plan for and use information technology to support the agency’s missions and improve its program performance. NRC modified its draft strategic plan to explain that annual performance plans that will delineate objective, quantifiable, and measurable goals to be achieved in a given fiscal year will be developed to further the general goals in the strategic plan. NRC’s September 1997 plan does not indicate how it intends to address such key information technology issues as the Year 2000 problem and the information security problem, or how it intends to plan for and use information technology to support the agency’s mission. Instead, the strategic plan says that these key issues are included in NRC’s fiscal year 1999 performance plan. NRC recognizes that assuming the regulation of the nuclear activities of DOE may be required in the future, but it has not yet begun projecting plans for that purpose. NRC has, however, agreed to pursue a pilot program of simulated regulation of DOE, in which regulatory concepts may be tested. NRC and DOE believe that information from the pilot program should be available before legislation to transfer regulatory responsibility is enacted. We had suggested that NRC link all of its goals and strategies to its major statutory authorities to facilitate a better understanding of the diversity and complexity of its overall mission, goals, and strategies. NRC responded to this suggestion by listing the statutory authorities for its general goals under five of its seven strategic arenas, but it did not include specific statutory references for its “Public Confidence” and “Excellence” arenas. On October 8, 1997, we met with NRC officials, including NRC’s Chief Financial Officer, to obtain NRC’s comments on our observations about the September 1997 plan. NRC said that the Commission is committed to implementing the Results Act and will continue to make improvements to its first strategic plan, including addressing our observations for future improvements, and take the other actions necessary to make managing for results a reality at NRC. Victor S. Rezendes, Director, Energy, Resources, and Science Issues; Resources, Community, and Economic Development Division, (202) 512-3841. On July 11, 1997, we issued a report on the Small Business Administration’s (SBA) draft strategic plan (Results Act: Observations on the Small Business Administration’s Draft Strategic Plan, GAO/RCED-97-205R). SBA’s formally issued strategic plan was submitted to OMB and Congress on September 30, 1997. As requested, we have reviewed the publicly issued strategic plan and compared it with the observations in our July 11 report. On October 16, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from that briefing are summarized herein. SBA’s draft strategic plan, as discussed in our July report, lacked two required elements: (1) a discussion of the relationship between the long-term goals and objectives and the annual performance goals and (2) a description of how program evaluations were used to establish or revise strategic goals and a schedule for future program evaluations. The four required elements contained in the plan could have better conformed to the Results Act’s requirements and OMB’s guidance. For example, (1) the mission statement did not encompass SBA’s significant disaster loan program for individuals, (2) many of the goals and objectives appeared less outcome oriented than process oriented, (3) the strategies consisted entirely of one-line statements and were not detailed enough to enable an assessment of how they would help achieve the plan’s goals and objectives, and (4) the plan did not discuss how identified external factors would be taken into account when assessing progress toward goals. Also, because of the way in which the information was presented, the linkages among specific performance measures, strategies, and objectives were not clear. SBA’s draft strategic plan also did not explicitly address the relationship of SBA’s activities to similar activities in other agencies and provided no evidence that SBA coordinated with other agencies in developing its plan. In addition, the plan could have benefited from an explicit acknowledgment of the extent to which SBA must rely on other federal agencies in carrying out its federal procurement-related responsibilities. processes but did not describe the specific strategies to achieve the objectives. SBA’s September 30, 1997, strategic plan includes several improvements that make it more responsive to the requirements of the Results Act than the earlier version. At the same time, SBA’s September plan differs significantly from the earlier draft in that it includes, as appendixes, separate strategic plans for SBA’s Office of Inspector General and Office of Advocacy. As discussed further on, SBA has not made clear the relationship between the goals and objectives in the plans included in the appendixes and those in the main text of the plan. With a discussion of (1) the relationship between the long-term goals and objectives and the annual performance goals and (2) how program evaluations were used to establish or revise strategic goals, SBA’s September plan addresses all six required elements. The plan’s five new strategic goals, as a group, are more clearly linked to SBA’s statutory mission than were the previous plan’s seven goals. In addition, the inclusion of date-specific performance objectives that incorporate performance measures make the strategic goals more amenable to a future assessment of SBA’s progress. For example, under the goal to “increase opportunities for small business success,” one of SBA’s performance objectives is as follows: “By the year 2000, SBA will help increase the share of federal procurement dollars awarded to small firms to at least 23 percent.” Also, SBA significantly improved its plan by more clearly and explicitly linking the strategies to the specific objectives that they are intended to accomplish. Other improvements include a mission statement that includes the disaster loan program for individuals and more accurately reflects SBA’s statutory authorities, a better recognition that SBA’s success in achieving certain goals and objectives in the plan is dependent on the actions of other agencies, and the addition of a section that discusses how SBA’s programs and activities interact with other federal agencies’ programs and activities. While the latter section states that SBA will coordinate with other agencies in the future, it does not provide evidence that SBA coordinated with the other agencies in the plan’s development. Also, the section that discusses SBA’s goal to improve internal controls implicitly addresses management problems that we and others have identified. However, specific strategies to address the identified management problems are not described. While SBA’s goals are more clearly linked to SBA’s statutory mission, the relationship of one goal—leading small business participation in the welfare-to-work effort—to SBA’s mission is unclear. While the plan’s performance objective places an emphasis on helping small businesses meet their workforce needs, the subsequent discussion implies a focus on helping welfare recipients find employment; for example, the plan states that “SBA’s goal is to help 200,000 work-ready individuals make the transition from welfare to work . . . .” It is not clear why SBA is focusing on welfare recipients only and not on other categories of potential employees to help meet small businesses’ workforce needs. SBA’s plan mentions certain program evaluations that SBA plans for future fiscal years, as well as the continuation of its goal of monitoring field and headquarters offices. However, the plan does not contain schedules of future comprehensive program evaluations for SBA’s major programs, including its 7(a) loan program and 8(a) business development program. (The Inspector General’s plan references future audits and evaluations that the Inspector General plans to conduct to improve SBA management.) In addition, the plan acknowledges that SBA needs a more systematic approach for using program evaluations for measuring progress toward achieving its goals and objectives, but it does not outline how SBA will develop and implement such an approach. Also, the strategy sections in the plan do not describe the human, capital, and information resources that are needed to achieve the goals and objectives. The September plan identifies various external factors, such as the economy and congressional support, that could affect the achievement of the plan’s goals. However, with the exception of “interagency coordination,” the plan does not link these factors to particular goals or consistently describe how the factor(s) could affect achievement of the goals and objectives. Furthermore, the plan also does not articulate strategies for mitigating the factors’ effects. Also, while recognizing the need for reliable information to measure progress toward the plan’s goals and objectives, the plan notes that SBA currently does not collect or report many of the measures that it will require to assess performance. The plan would benefit from brief descriptions of how SBA plans to collect the data to measure progress toward its goals and objectives. General’s plan as an appendix, without cross-reference to any specific SBA goal or objective. Also, the September plan includes an appendix containing a plan for the Office of Advocacy; this material did not appear in SBA’s earlier plan. Generally, the goals and objectives in the Inspector General and Advocacy plans appear consistent with, and may contribute to the achievement of, the goals and objectives in SBA’s plan, but the relationship is not explicit. SBA’s plan makes little mention of the Inspector General and Advocacy plans and does not indicate at all how, or if, the Inspector General and Advocacy activities are intended to help SBA achieve the agency’s goals and objectives. Similarly, the Inspector General and Advocacy plans do not make reference to the goals and objectives in the SBA plan. These plans could be more useful to decisionmakers if their relationships were clearer. We provided copies of a draft of these observations to SBA for review and comment. We received comments from the SBA Administrator. SBA commented that our analysis of the plan provided useful suggestions that will be used in its next draft of the plan. SBA also provided additional information concerning two of our observations. First, SBA stated that the emphasis of the fourth goal—to lead small business participation in the welfare-to-work initiative—is focused on helping small businesses rather than former welfare recipients. Second, SBA commented that the agency works with the Inspector General and Advocacy offices to carry out SBA’s mission, and the Inspector General and Advocacy plans were included as appendixes to its strategic plan to highlight the offices’ statutory independence. Judy A. England-Joseph, Director, Housing and Community Development Issues; Resources, Community, and Economic Development Division, (202) 512-7632. On July 22, 1997, we issued a report on the Social Security Administration’s (SSA) draft strategic plan (The Results Act: Observations on the Social Security Administration’s June 1997 Draft Strategic Plan, GAO/HEHS-97-179R). SSA’s formally issued strategic plan was submitted to OMB and Congress on September 30, 1997. As requested, we reviewed the September 30 plan and compared it with the observations in our July 22 report. On October 14, 1997, we briefed your staffs on our further observations on the strategic plan. The key points from that briefing are summarized herein. SSA’s draft strategic plan contained all six of the elements required by the Results Act and reflected its status as an independent agency. To the agency’s credit, the draft plan was forward-looking and provided a solid foundation for SSA’s consultation with Congress and other stakeholders. Also, the goals in the draft plan were more balanced than those of prior SSA plans because they emphasized sound program management in addition to customer service. However, some of the required elements in the plan could have been strengthened in important ways. For example, for some goals, it was not clear what SSA hoped to achieve and how it planned to measure its achievement, and some goals seemed to overlap. In addition, the plan cited many initiatives that SSA intends to begin or continue without additional agencywide resources and without setting priorities or delineating time frames and schedules. As a result, it was difficult to see how SSA could accomplish all of its planned initiatives. SSA went beyond minimum requirements by providing numerous performance measures. However, we noted that it was sometimes difficult to link the performance measures with specific objectives. The draft plan included a description of the external factors that SSA considered in developing the plan, but this discussion could have been improved had SSA more explicitly linked the effects of certain external factors, such as changes in available technology, with goal attainment and had it more clearly explained how it has used and plans to use program evaluations. disclose the challenges SSA has faced in redesigning its disability process and did not fully integrate a return-to-work strategy for its disabled beneficiaries throughout the agency’s operations. SSA’s draft plan accurately conveyed the agency’s strong reliance on improved information technology to provide world-class service and to better manage its programs with its existing resources. However, we observed that the plan would be strengthened by adding information on how SSA will use information technology to achieve the agency’s goals and objectives. Finally, the draft could have discussed in more detail SSA’s plans to cope with two technology-related high-risk areas—the Year 2000 computer problem and the need to adequately protect the sensitive data in its computer systems. SSA incorporated several of the changes we suggested in its formally issued plan, but the extent of the revisions and the attendant improvements vary from element to element. Throughout the plan, SSA added pieces of information on processes and technologies it will use to achieve its goals. However, this information, along with the needed staff skills and timetables, is not discussed uniformly for each goal. In response to the need to better link performance measures with specific objectives, SSA added a matrix that presents the goals, objectives, and related performance indicators. In most cases, the goals, objectives, and measures are clearly stated in the body of the plan, and the matrix provides a useful summary of how SSA will assess its performance. In other cases, however, it is difficult to relate the discussion of performance measures in the body of the plan with the indicators in the matrix. SSA appropriately included its program evaluation activities in its first goal; it also added more information about the types of program evaluations used or planned for the future and the timetables for some of the planned evaluations. discussion of goals and strategies. For example, SSA describes its need to cooperate with law enforcement agencies in its discussion on ways to combat fraud. SSA also acknowledged that some of its management challenges were not adequately addressed in its draft plan. SSA recognized SSI as a high-risk area and noted that the agency intends to develop a separate plan to improve the program. For its disability process redesign, SSA added a short explanation of the complexity of the redesign process and recent attempts to narrow its focus. SSA also expanded its discussion of its return-to-work efforts and included information on the studies it plans to undertake. Finally, SSA has improved its plan by including discussions of the Year 2000 problem, the importance of resolving it, and the need to mitigate any future problems with other agencies with whom SSA shares information. future if SSA linked the discussion of these external factors with goal attainment and consistently included any mitigation strategies. As previously stated, the success of several goals is dependent on technological improvements or changes in agency operational processes, but we found that SSA has encountered difficulty implementing some of these changes. SSA has not acknowledged these difficulties, such as the challenges it faces in developing new software to complement the redesigned disability determination process, and the plan could be improved by providing additional information on how information technology strategies will be used to achieve the agency’s goals and objectives. Relative to changes in technology, SSA has not incorporated any plans to begin the difficult task of assessing its current service delivery structure and how it should change in the future. We provided SSA with a draft of our observations on its strategic plan. In its written reply, SSA said that it appreciated that we recognized the improvements made to the draft plan. SSA also stated that it believes that the strategic plan contains as much detail as is possible and appropriate at this point in its planning cycle. It is refining and refocusing its current key initiatives, as necessary, and developing plans for new initiatives to ensure that the agency reaches its objectives. Jane L. Ross, Director, Income Security Issues; Health, Education, and Human Services Division, (202) 512-7215. On July 11, 1997, we issued a report on the U.S. Agency for International Development’s (USAID) draft strategic plan (The Results Act: Observations on USAID’s November 1996 Draft Strategic Plan, GAO/NSIAD-97-197R). USAID submitted its formally issued strategic plan to OMB and Congress on September 30, 1997. As requested, we have reviewed the publicly issued strategic plan and compared it with the observations in our July 11 report. On October 24, 1997, we briefed your staffs on our further observations on USAID’s strategic plan. We summarize the key points from that briefing herein. USAID’s November 1996 draft strategic plan included the six elements required by the Results Act. However, two components of the plan—sections on (1) relating performance goals to general goals and objectives and (2) program evaluations—did not contain sufficient information to fully achieve the purposes of the Results Act and related OMB guidance. More specifically, these sections did not include a discussion of performance goals, relevant evaluation findings USAID used to develop its plan, or USAID’s plan for conducting future evaluations. Many agencies are involved in activities directly related to USAID’s mission, goals, and objectives, and there is potential for crosscutting issues. Nevertheless, the draft strategic plan did not address areas of possible duplication and USAID’s efforts to minimize them or the extent to which USAID relies on other agencies to meet its goals and objectives. We also observed that the draft plan did not address key management challenges that the agency faces. The plan provided a general description of recent management initiatives but did not discuss how effective these initiatives have been in resolving critical management problems USAID has acknowledged in nearly all areas of its operations. In particular, the plan did not describe difficulties USAID has encountered in developing a performance measurement system, in reforming its personnel systems, in implementing the Chief Financial Officers Act of 1990 (P.L. 101-576), and in deploying a new information management system that is intended to correct several material weaknesses in its financial management processes. USAID’s publicly issued strategic plan incorporated some improvements that make it more responsive to the requirements of the Results Act. In particular, USAID has developed performance goals related to the agency’s overall goals and objectives. These goals generally appear to be objective, quantifiable, and measurable. The rationale and data sources for the indicators are described in detail in an appendix to the plan. Although the performance goals presented are generally long-term ones, it appears that USAID will be able to derive required annual performance goals from many of them in the future. We did not evaluate the appropriateness of these indicators or the reliability of the data sources cited. USAID’s plan is clearer and more explicit about its long-term goals and objectives. The seven goals are clearly identified in narrative form, and both the goals and related objectives are presented graphically in an appendix. USAID has also improved this element of its plan by omitting other implicit goals, included in the November 1996 draft plan, that made it unclear what USAID intended to achieve. However, USAID’s goals and objectives are targeted at results over which USAID does not have a reasonable degree of influence. As we previously reported, USAID officials have acknowledged that in only a few cases have USAID’s programs been directly linked to the types of country-level development results described in the plan. With regard to strategies to achieve these goals, USAID’s plan now includes the goal of improving its management efficiency and effectiveness, including the steps that it is taking in that regard, and indicators for measuring progress. Consistent with suggestions in our July report, the plan now also includes an explicit discussion of the program, support, and workforce resources USAID believes are necessary to achieve its performance goals. The plan presents resource needs at an aggregate level and does not specify the level of resources needed to achieve each of USAID’s strategic objectives. contribution that USAID’s development partners make toward achievement of the agency’s goals and objectives. In particular, the plan identifies the commitment of other donor countries and multilateral agencies as the major external factor affecting USAID’s performance. USAID’s plan now includes a discussion of crosscutting functions across the U.S. government. It recognizes that other agencies provide technical assistance to developing and transitional countries and that achievement of USAID’s goals is affected by the actions of these agencies. The plan states that mechanisms are in place to reduce or minimize duplication at the field level, and for each goal it identifies those agencies with which it coordinates on related activities. However, it does not indicate what these coordination mechanisms are and lacks the information to demonstrate that they are adequate. The plan implies that only limited coordination with these other agencies on strategic planning has taken place and indicates that USAID anticipates expanded and ongoing interagency dialogue. The plan more fully addresses key principles of the Foreign Assistance Act of 1961 (P.L. 87-195), as we suggested in our July report. For example, it more extensively discusses the principles of coordination of foreign assistance with other donors and supporting development goals chosen by the recipient country. However, it does not specifically address the principle of encouraging regional cooperation by developing countries. We suggested in our previous report that several elements of the USAID plan could be further improved to better meet the purposes of the Results Act. The plan still does not contain sufficient information on program evaluations. It does not show how program evaluations by USAID or external organizations were used to establish strategic goals and does not outline the scope, methodology, key issues, or schedule for future evaluations. Although the plan refers to other documents and means by which USAID communicates evaluation schedules and findings, a summary of that information would be appropriate in this section to demonstrate the role that program evaluation plays in USAID’s strategic planning and results assessment. difficulties USAID has encountered in its efforts to improve. For example, the plan indicates that USAID hopes to improve the availability of financial and program results information. However, it does not convey the significant problems USAID has had to date generating complete, timely, and reliable financial and performance data—problems that hamper USAID’s ability to identify costs and measure performance. Nor does the plan establish a time frame for achieving substantial and verifiable improvement in this area. Frank acknowledgement of specific management challenges in the area of information technology is also absent from the strategic plan. The plan describes progress USAID has made in implementing a new management system but is silent on the major setbacks it is having with this implementation, even though this system will be critical to the success of financial and program management reforms. Similarly, the plan does not address information security and the Year 2000 problem, which we have identified as high-risk areas governmentwide. Instead of dealing with these issues directly, the plan refers to a Strategic Information Resource Management Plan that is said to set the direction for USAID to meet its information needs through 2002. A summary of the plan would be helpful, inasmuch as it acknowledges the hurdles USAID must overcome in achieving its goals. While USAID recognizes its dependence on other donors and its susceptibility to factors beyond its control, as we had suggested, we believe that USAID has not adequately emphasized the importance of these issues. The plan could articulate the relative magnitude of USAID’s assistance within the donor community to more clearly convey the extent of USAID’s dependence on the contributions of other donors to meet the performance goals it has established. In addition, the plan could articulate the extent of USAID’s ability to offset country and international conditions that hamper development to more realistically convey the magnitude of the risk and uncertainty that USAID faces in trying to achieve its goals. Further, USAID’s strategic plan does not specifically discuss its Economic Support Fund programs and its programs in the East European and Baltic states and newly independent states of the former Soviet Union. We noted in our July report and continue to believe that the plan could benefit from greater discussion of these activities, which directly serve U.S. foreign policy interests and represent about 60 percent of USAID’s budget. whether the goals apply to each recipient country individually or to all collectively. In some, but not all, cases this is clarified within the text of the appendix containing the rationale for the indicators used. USAID substantially reorganized the strategic plan from the November 1996 version. Many key elements of the plan have been consolidated into one section with no indication of where one element ends and another begins. Separate sections or increased use of subheadings would significantly improve the presentation and the ease of using this plan. On October 10, 1997, we briefed USAID officials on our observations about the issued strategic plan. On November 3, 1997, USAID officials provided us with comments on a draft of this appendix. They generally believe that we have fairly recorded the progress made to date, but they provided additional comments and clarification of several points, which we have incorporated as appropriate. They acknowledged that in some cases, for the sake of brevity, the plan did not reflect the level of specificity that is called for by the OMB and our guidance, particularly with regard to program evaluations, crosscutting functions, and information resource management issues. They noted that such detail is readily available from other USAID sources and believe that including it in the plan would add little value and would unduly increase the plan’s size. We continue to believe that the clarity and credibility of USAID’s strategic plan could be improved with the inclusion of the type of detail we have outlined. USAID officials also contended that the plan acknowledges USAID’s management challenges by outlining management improvement strategies that would resolve the types of problems we raised. However, we believe that an explicit description of management challenges would provide the reader a better sense of the nature and gravity of the problems USAID must overcome and the implications for USAID’s performance if it is not successful in overcoming these problems. country-level results. They stated that USAID has been able to influence the use of the resources of other donors, which affects the development goals USAID seeks to achieve. Benjamin F. Nelson, Director, International Relations and Trade Issues; National Security and International Affairs Division, (202) 512-4128. The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Summary: Pursuant to a congressional request, GAO reviewed federal agencies' strategic plans submitted in response to the Government Performance and Results Act of 1993, focusing on: (1) summarizing its observations on agencies' September plans; and (2) providing additional information on how the next phase of the Results Act's implementation--performance planning measurement--can be used to address the critical planning issue GAO observed in reviewing the September strategic plans. GAO noted that: (1) on the whole, agencies' September plans appear to provide a workable foundation for Congress to use in helping to fulfill its appropriations, budget, authorization, and oversight responsibilities and for agencies to use in setting a general direction for their efforts; (2) agencies' strategic planning efforts are still very much a work in progress; (3) GAO's reviews of September plans indicate that continued progress is needed in how agencies address three difficult planning challenges--setting a strategic direction, coordinating crosscutting programs, and ensuring the capacity to gather and use performance and cost data; (4) GAO found that agencies can build upon their initial efforts to set a strategic direction for their programs and activities; (5) the next stage in the Results Act's implementation--performance planning and measurement--can assist agencies in addressing the challenge of setting a strategic direction; (6) as an agency develops its performance plan, it likely will identify opportunities to revise and clarify those strategic goals in order to provide a better grounding for the direction of the agency; (7) also, as agencies develop the objective, measurable annual performance goals as envisioned by the Act, those goals can serve as a bridge that links long-term strategic goals to agencies' daily operations; (8) the Results Act's requirements for annual performance plans and performance measurement can also provide a structured framework for Congress, Office of Management and Budget, and agencies to address agencies' crosscutting programs--the second critical planning challenge; (9) GAO found that although agencies have begun to recognize the importance of coordinating crosscutting programs, they must undertake the substantive coordination that is needed for the effective management of those programs; (10) the third critical planning challenge is the need for agencies to have the capacity to gather and use sound program performance and cost data to successfully measure progress toward their intended results; (11) under the Results Act, agencies are also to discuss in their annual performance plans how they will verify and validate the performance information that they plan to use to show whether goals are being met; and (12) verified and validated performance information, in conjunction with augmented program evaluation efforts, will help ensure that agencies are able to report progress in meeting goals and identify specific strategies for improving performance.