a

GAO

United States General Accounting Office

Report to Congressional Requesters

March 2004 RESULTS-ORIENTED 

GOVERNMENT

GPRA Has Established 

a Solid Foundation for 

Achieving Greater 

Results

GAO-04-38
www.gao.gov/cgi-bin/getrpt?GAO-04-38.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Patricia A. 

Dalton at (202) 512-6806 or 

daltonp@gao.gov. 

Highlights of GAO-04-38, a report to 

congressional requesters 

March 2004 

RESULTS-ORIENTED GOVERNMENT

GPRA Has Established a Solid 

Foundation for Achieving Greater Results 

GPRA’s requirements have established a solid foundation of results-oriented 

performance planning, measurement, and reporting in the federal 

government. Federal managers surveyed by GAO reported having 

significantly more of the types of performance measures called for by GPRA 

(see figure below). GPRA has also begun to facilitate the linking of 

resources to results, although much remains to be done in this area to 

increase the use of performance information to make decisions about 

resources. We also found agency strategic and annual performance plans 

and reports we reviewed have improved over initial efforts. 

Although a foundation has been established, numerous significant challenges 

to GPRA implementation still exist. Inconsistent top leadership commitment 

to achieving results within agencies and OMB can hinder the development of 

results-oriented cultures in agencies. Furthermore, in certain areas, federal 

managers continue to have difficulty setting outcome-oriented goals, 

collecting useful data on results, and linking institutional, program, unit, and 

individual performance measurement and reward systems. Finally, there is 

an inadequate focus on addressing issues that cut across federal agencies. 

OMB, as the focal point for management in the federal government, is 

responsible for overall leadership and direction in addressing these 

challenges. OMB has clearly placed greater emphasis on management issues 

during the past several years. However, it has showed less commitment to 

GPRA implementation in its guidance to agencies and in using the 

governmentwide performance plan requirement of GPRA to develop an 

integrated approach to crosscutting issues. In our view, governmentwide 

strategic planning could better facilitate the integration of federal activities 

to achieve national goals. 

Percentage of Federal Managers Who Reported Having Specific Types of Performance 

Measures Called for by GPRA 

Now that the Government 

Performance and Results Act 

(GPRA) has been in effect for 10 

years, GAO was asked to address 

(1) the effect of GPRA in creating a 

governmentwide focus on results 

and the government’s ability to 

deliver results to the American 

public, (2) the challenges agencies 

face in measuring performance and 

using performance information in 

management decisions, and 

(3) how the federal government 

can continue to shift toward a more 

results-oriented focus. 

GAO recommends that the Office 

of Management and Budget (OMB) 

improve its guidance and oversight 

of GPRA implementation, as well 

as develop a governmentwide 

performance plan. GAO also 

believes Congress should consider 

amending GPRA to require that 

(1) agencies update their strategic 

plans at least once every four 

years, consult with congressional 

stakeholders at least once every 

new Congress, and make interim 

updates to strategic and 

performance plans as appropriate; 

and (2) the President develop a 

governmentwide strategic plan. 

OMB generally agreed with our 

recommendations, but stated that 

the President’s Budget can serve as 

both a governmentwide strategic 

and annual plan. However, we 

believe the budget provides neither 

a long-term nor an integrated 

perspective on the federal 

government’s performance. 
Page i GAO-04-38 Results-Oriented Government

Contents

Transmittal Letter 1

Executive Summary 3

Purpose 3

Background 4

Results in Brief 6

Principal Findings 10

Recommendations for Executive Action 19

Matters for Congressional Consideration 20

Agency Comments 20

Chapter 1

Introduction

22

Impact of Emerging Trends and Fiscal Challenges 23

GPRA Background 25

Scope and Methodology 30

Chapter 2

GPRA Established the 

Foundation for a More 

Results-Oriented 

Federal Government

31

GPRA Statutory Requirements Laid a Foundation for Agencywide 

Results-Oriented Management 31

Chapter 3

Agencies Have 

Addressed Many 

Critical Performance 

Planning and Reporting 

Challenges, but 

Weaknesses Persist

46

Quality of Selected Strategic Plans Reflects Improvements over 

Initial Drafts 46

Fiscal Year 2004 Annual Performance Plans Addressed Some 

Weaknesses of Earlier Plans, but Still Have Room for Significant 

Improvement 53

Strengths and Weaknesses of Selected Agencies’ Fiscal Year 2002 

Annual Performance Reports 60

Chapter 4

Challenges to GPRA 

Implementation Persist

68

Top Leadership Does Not Consistently Show Commitment to 

Achieving Results 69
Contents

Page ii GAO-04-38 Results-Oriented Government

Managers Report Mixed Results in Use of Performance 

Information 75

Managers Continue to Confront a Range of Human Capital 

Management Challenges 79

Persistent Challenges in Setting Outcome-Oriented Goals, 

Measuring Performance, and Collecting Useful Data 88

Crosscutting Issues Hinder Successful GPRA Implementation 92

Managers View Congress’ Use of Performance Information as 

Limited 96

Chapter 5

Conclusions and 

Recommendations

100

Agenda for Achieving a Sustainable, Governmentwide Focus on 

Results 100

Recommendations for Executive Action 109

Matters for Congressional Consideration 110

Agency Comments 110

Appendixes

Appendix I: Objectives, Scope, and Methodology 113

Methodology for Governmentwide Survey 113

Methodology for Focus Groups 116

Methodology for Interviews with Political Appointees 118

Methodology for Selecting Agencies to Review for Changes in the 

Quality of Their Strategic Plans, Annual Performance Plans, and 

Annual Performance Reports 119

Appendix II: Focus Group Participants Agreed GPRA Provides a 

Framework for Federal Agencies to Become More Results 

Oriented 121

GPRA Accomplishments 122

Views on Delivering Results to the American Public Were

Mixed 124

Alternate Views on GPRA’s Effect 125

Challenges in Implementing and Overseeing GPRA Activities 126

Suggestions to Address GPRA Challenges 127

Appendix III: Observations on Agencies’ Strategic Plans 130

Required Elements of Agency Strategic Plans 130

Observations on Changes in the Quality of Education’s Strategic 

Plan 134

Observations on Changes in the Quality of DOE’s Strategic Plan 139
Contents

Page iii GAO-04-38 Results-Oriented Government

Observations on Changes in the Quality of HUD’s Strategic 

Plan 142

Observations on Changes in the Quality of SBA’s Strategic Plan 146

Observations on Changes in the Quality of SSA’s Strategic Plan 150

Observations on Changes in the Quality of DOT’s Strategic Plan 154

Appendix IV: Observations on Agencies’ Annual Performance Plans 160

Key Elements of Information for Annual Performance Plans 160

Observations on Changes in the Quality of Education’s Annual 

Performance Plan 164

Observations on Changes in the Quality of DOE’s Annual 

Performance Plan 169

Observations on Changes in the Quality of HUD’s Annual 

Performance Plan 172

Observations on Changes in the Quality of SBA’s Annual 

Performance Plan 175

Observations on Changes in the Quality of SSA’s Annual 

Performance Plan 179

Observations on Changes in the Quality of DOT’s Annual 

Performance Plan 183

Appendix V: Observations on Agencies’ Annual Performance and 

Accountability Reports 188

Observations on the Quality of Education’s Fiscal Year 2002 

Performance and Accountability Report 192

Observations on the Quality of DOE’s Fiscal Year 2002 Annual 

Performance and Accountability Report 200

Observations on the Quality of HUD’s Fiscal Year 2002 Annual 

Performance and Accountability Report 204

Observations on the Quality of SBA’s Fiscal Year 2002 Annual 

Performance and Accountability Report 208

Observations on the Quality of SSA’s Fiscal Year 2002 Annual 

Performance and Accountability Report 213

Observations on the Quality of DOT’s Fiscal Year 2002 Annual 

Performance and Accountability Report 217

Appendix VI: GAO Federal Managers’ Survey Data 223

Appendix VII: Agencies Subject to the Chief Financial Officers Act 236

Appendix VIII: Comments from the Office of Management and Budget 238

Appendix IX: Comments from the Department of Energy 240

GAO Comments 242
Contents

Page iv GAO-04-38 Results-Oriented Government

Appendix X: Comments from the Department of Housing and Urban 

Development 245

GAO Comments 251

Appendix XI: Comments from the Social Security Administration 255

GAO Comments 260

Appendix XII: GAO Contact and Staff Acknowledgments 263

GAO Contact 263

Acknowledgments 263

Related GAO Products 264

GPRA/Managing for Results 264

Strategic Human Capital Management 266

Linking Resources to Results 266

Measuring Performance 267

Data Credibility 268

Using Performance Information 268

Tables Table 1: Agencies’ Progress in Addressing Required Elements of 

Strategic Planning under GPRA 48

Table 2: Characterizations of Agencies’ Fiscal Year 1999 and 2004 

Annual Performance Plans 54

Table 3: Characterizations of Agencies’ 2002 Annual Performance 

Reports 61

Table 4: Summary of Characteristics of Agencies Selected for 

Review of Strategic Plans, Annual Performance Plans, and 

Annual Performance Reports 120

Table 5: Agencies’ Progress in Addressing Required Elements of 

Strategic Planning under GPRA 133

Table 6: Education’s Progress in Addressing Required Elements of 

Strategic Planning under GPRA 134

Table 7: DOE’s Progress in Addressing Required Elements of 

Strategic Planning under GPRA 139

Table 8: HUD’s Progress in Addressing Required Elements of 

Strategic Planning under GPRA 143

Table 9: SBA’s Progress in Addressing Required Elements of 

Strategic Planning under GPRA 146

Table 10: SSA’s Progress in Addressing Required Elements of 

Strategic Planning under GPRA 150

Table 11: DOT’s Progress in Addressing Required Elements of 

Strategic Planning under GPRA 154
Contents

Page v GAO-04-38 Results-Oriented Government

Table 12: Characterizations of Agencies’ Annual Performance

Plans 163

Table 13: Characterizations of Agencies’ Fiscal Year 2002 Annual 

Performance and Accountability Reports 192

Figures Figure 1: Composition of Spending as a Share of GDP Assuming 

Discretionary Spending Grows with GDP after 2003 and 

All Expiring Tax Provisions Are Extended 24

Figure 2: Percentage of Federal Managers Who Reported That 

There Were Performance Measures for the Programs with 

Which They Were Involved 35

Figure 3: Percentage of Federal Managers Who Reported Having 

Specific Types of Performance Measures to a Great or 

Very Great Extent 36

Figure 4: Percentage of Federal Managers Who Reported Their 

Awareness of GPRA 37

Figure 5: Percentage of Federal Managers Who Reported 

Hindrances to Measuring Performance or Using the 

Performance Information to a Great or Very Great 

Extent 38

Figure 6: Percentage of Federal Managers and SES Managers Who 

Reported That OMB Paid Attention to Their Agency’s 

Efforts under GPRA to a Great or Very Great Extent 39

Figure 7: Percentage of Federal Managers Who Reported They 

Considered Strategic Goals to a Great or Very Great 

Extent When Allocating Resources 43

Figure 8: Percentage of Federal Managers Who Reported They 

Considered Performance Information to a Great or Very 

Great Extent When Allocating Resources 44

Figure 9: Percentage of Federal Managers Who Reported That 

Funding Decisions Were Based on Results or 

Outcome-Oriented Performance Information to a Great 

or Very Great Extent 45

Figure 10: Percentage of Federal Managers Who Reported to a Great 

or Very Great Extent Their Top Leadership Has a Strong 

Commitment to Achieving Results 70

Figure 11: Percentage of SES and Non-SES Managers Who Reported 

to a Great or Very Great Extent Their Agency Top 

Leadership Demonstrated Strong Commitment to 

Achieving Results 71
Contents

Page vi GAO-04-38 Results-Oriented Government

Figure 12: Percentage of Federal Managers Who Reported Using 

Information Obtained from Performance Measurement to 

a Great or Very Great Extent for Various Management 

Activities 77

Figure 13: Percentage of Federal Managers Responding “Yes” about 

Being Involved in the Following Activities 79

Figure 14: Percentage of Federal Managers Reporting to a Great or 

Very Great Extent That Managers/Supervisors at Their 

Levels Had the Decision-Making Authority They Needed 

to Help the Agency Accomplish Its Strategic Goals 81

Figure 15: Percentage of Federal Managers, SES, and Non-SES in 

2003 Reporting to a Great or Very Great Extent That They 

Were Held Accountable for the Accomplishment of 

Agency Strategic Goals 82

Figure 16: Percentage of Federal Managers in Each Survey Year Who 

Reported That during the Past 3 Years Their Agencies 

Provided, Arranged, or Paid for Training That Would Help 

Them Accomplish Specific Tasks 84

Figure 17: Percentage of Federal Managers Who Reported to a Great 

or Very Great Extent That Employees in Their Agencies 

Received Positive Recognition for Helping Their 

Agencies Accomplish Their Strategic Goals 86

Figure 18: Percentage of Federal Managers Reporting to a Great or 

Very Great Extent That a Lack of Ongoing Congressional 

Commitment or Support for Using Performance 

Information in Making Program/Funding Decisions Is a 

Hindrance 98

Figure 19: Summary of Education’s Performance Indicators for 

Fiscal Year 2002 194

Figure 20: Inputs: Allocating Funds for Education’s Objective to 

Ensure That All Students Read on Grade Level by the 

Third Grade 198

Figure 21: Summary of DOE’s Performance Indicators for Fiscal 

Year 2002 201

Figure 22: Summary of HUD’s Performance Indicators for Fiscal 

Year 2002 206

Figure 23: Summary of SBA’s Performance Goals for Fiscal Year 

2002 209

Figure 24: Summary of SSA’s Performance Goals for Fiscal Year 

2002 214

Figure 25: Summary of DOT’s Performance Indicators for Fiscal 

Year 2002 218
Contents

Page vii GAO-04-38 Results-Oriented Government

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately.

Abbreviations

AP advanced placement

CDBG Community Development Block Grants

CFO Chief Financial Officer

CRS Congressional Research Service

DOE Department of Energy

DOT Department of Transportation

EPA Environmental Protection Agency

FAA Federal Aviation Administration

FSA Federal Student Assistance

FTE full-time employee

GM general management

GPRA Government Performance and Results Act of 1993

GS general schedule

HHS Department of Health and Human Services

HOME Home Investment Partnership Program

HUD Department of Housing and Urban Development

ICH Interagency Council on the Homeless

IG Inspector General

IT information technology

IRS Internal Revenue Service

JARC Job Access and Reverse Commute

NAEP National Assessment for Educational Progress

NASA National Aeronautics and Space Administration

OASI Old Age and Survivors Insurance 

OMB Office of Management and Budget

OPM Office of Personnel Management

PART Program Assessment Rating Tool

PMA President’s Management Agenda

SBA Small Business Administration

SBDC Small Business Development Centers

SES Senior Executive Service

SSA Social Security Administration

SSI Supplemental Security Income

SSN Social Security number

VA Department of Veterans Affairs
Page 1 GAO-04-38 Results-Oriented Government

United States General Accounting Office

Washington, D.C. 20548

Comptroller General A of the United States

March 10, 2004 TransmtLailetr

Congressional Requesters

As you requested, we have assessed the effectiveness of the Government 

Performance and Results Act (GPRA), in light of its 10-year anniversary in 

2003. Our review focused on GPRA’s accomplishments, challenges to its 

continued implementation, and an agenda for achieving a sustainable, 

governmentwide focus on results.

Upon issuance, we will send copies to the Director of the Office of 

Management and Budget and executive branch agencies (see appendix VII 

for a list). We will also make copies available to others upon request. In 

addition, this report will be available at no charge on the GAO web site at 

http://www.gao.gov.

If you have any questions concerning this report, please contact Patricia A. 

Dalton at (202) 512-6806 or daltonp@gao.gov. The major contributors to 

this report are listed in appendix XII.

David M. Walker

Comptroller General

of the United States
Page 2 GAO-04-38 Results-Oriented Government

List of Requesters

The Honorable Susan M. Collins

Chairman

The Honorable Joe Lieberman

Ranking Minority Member

Committee on Governmental Affairs

United States Senate

The Honorable George V. Voinovich

Chairman

The Honorable Richard Durbin

Ranking Minority Member

Subcommittee on Oversight of Government Management, the Federal 

Workforce, and the District of Columbia

Committee on Governmental Affairs

United States Senate

The Honorable Peter G. Fitzgerald

Chairman

The Honorable Daniel K. Akaka

Ranking Minority Member

Subcommittee on Financial Management, the Budget, and International 

Security

Committee on Governmental Affairs

United States Senate

The Honorable Tom Davis

Chairman

The Honorable Henry A. Waxman

Ranking Minority Member

Committee on Government Reform

House of Representatives

The Honorable Todd Russel Platts

Chairman

The Honorable Edolphus Towns

Ranking Minority Member

Subcommittee on Government Efficiency and Financial Management

Committee on Government Reform

House of Representatives
Page 3 GAO-04-38 Results-Oriented Government

Executive Summary

Purpose From defending the homeland against terrorists, to preventing the spread 

of infectious diseases, to providing a reliable stream of social security 

income to retirees and supporting the transition from welfare to work, the 

federal government provides funding and services to the American public 

that can affect their lives in critical ways every day. However, the federal 

government is in a period of profound transition and faces an array of 

challenges and opportunities to enhance performance, ensure 

accountability, and position the nation for the future. A number of 

overarching trends, such as diffuse security threats and homeland security 

needs, increasing global interdependency, the shift to knowledge-based 

economies, and the looming fiscal challenges facing our nation drive the 

need to reconsider the proper role for the federal government in the 21st 

century, how the government should do business (including how it should 

be structured), and in some instances, who should do the government’s 

business.

Without effective short- and long-term planning, which takes into account 

the changing environment and needs of the American public and the 

challenges they face and establishes goals to be achieved, federal agencies 

risk delivering programs and services that may or may not meet society’s 

most critical needs. At a cost to taxpayers of over $2 trillion annually, the 

federal government should be able to demonstrate to the American public 

that it can anticipate emerging issues, develop sound strategies and plans 

to address them, and be accountable for the results that have been 

achieved.

Concerned that the federal government was more focused on program 

activities and processes than the results to be achieved, Congress passed 

the Government Performance and Results Act of 1993 (GPRA).1

 The act 

required federal agencies to develop strategic plans with long-term, 

outcome-oriented goals and objectives, annual goals linked to achieving 

the long-term goals, and annual reports on the results achieved. Now that 

GPRA has been in effect for 10 years, you asked us to assess the 

effectiveness of GPRA in creating a focus on results in the federal 

government. Specifically, this report discusses (1) the effect of GPRA over 

the last 10 years in creating a governmentwide focus on results and the 

government’s ability to deliver results to the American public, including an 

assessment of the changes in the overall quality of agencies’ strategic plans, 

1

Pub. L. No. 103-62.
Executive Summary

Page 4 GAO-04-38 Results-Oriented Government

annual performance plans, and annual performance reports; (2) the 

challenges agencies face in measuring performance and using performance 

information in management decisions; and (3) how the federal government 

can continue to shift toward a more results-oriented focus.

To meet our objectives, we reviewed our extensive prior work on GPRA 

best practices and implementation and collected governmentwide data to 

assess the government’s overall focus on results. We conducted a random, 

stratified, governmentwide survey of federal managers comparable to 

surveys we conducted in 1997 and 2000. We also held eight in-depth focus 

groups—seven comprised of federal managers from 23 federal agencies 

and one with GPRA experts. We also interviewed top appointed officials 

from the current and previous administrations. Finally, we judgmentally 

selected a sample of six agencies to review for changes in the quality of 

their strategic plans, performance plans, and performance reports since 

their initial efforts. The agencies we selected included the Departments of 

Education (Education), Energy (DOE), Housing and Urban Development 

(HUD), and Transportation (DOT) and the Small Business (SBA) and Social 

Security Administrations (SSA). In making this selection, we chose 

agencies that collectively represented the full range of characteristics in 

the following four areas: (1) agency size (small, medium, large); (2) primary 

program type (direct service, research, regulatory, transfer payments, and 

contracts or grants); (3) quality of fiscal year 2000 performance plans based 

on our previous review;2

 and (4) type of agency (cabinet department and 

independent agency). Appendix I contains a more detailed discussion of 

our scope and methodology. We performed our work in Washington, D.C., 

from January through November 2003 in accordance with generally 

accepted government auditing standards.

Background GPRA is the centerpiece of a statutory framework that Congress put in 

place during the 1990s to help resolve the long-standing management 

problems that have undermined the federal government’s efficiency and 

effectiveness and to provide greater accountability for results. GPRA was 

intended to address several broad purposes, including strengthening the 

confidence of the American people in their government; improving federal 

program effectiveness, accountability, and service delivery; and enhancing 

2

U.S. General Accounting Office, Managing for Results: Opportunities for Continued 

Improvements in Agencies’ Performance Plans, GAO/GGD/AIMD-99-215 (Washington, 

D.C.: July 20, 1999). 
Executive Summary

Page 5 GAO-04-38 Results-Oriented Government

congressional decision making by providing more objective information on 

program performance.

GPRA requires executive agencies to complete strategic plans in which 

they define their missions, establish results-oriented goals, and identify the 

strategies that will be needed to achieve those goals. GPRA requires 

agencies to consult with Congress and solicit the input of others as they 

develop these plans. Through this strategic planning requirement, GPRA 

has required federal agencies to reassess their missions and long-term 

goals as well as the strategies and resources they will need to achieve their 

goals. Agencies developed their first strategic plans in fiscal year 1997, and 

are required to update the plans every 3 years since then.

GPRA also requires executive agencies to prepare annual performance 

plans that articulate goals for the upcoming fiscal year that are aligned with 

their long-term strategic goals. These performance plans are to include 

results-oriented annual goals linked to the program activities displayed in 

budget presentations as well as the indicators the agency will use to 

measure performance against the results-oriented goals. Agencies 

developed their first annual performance plans in fiscal year 1999 and are 

required to issue plans annually thereafter to correspond with budget 

submissions to Congress.

Finally, GPRA requires agencies to measure performance toward the 

achievement of the goals in the annual performance plan and report 

annually on their progress in program performance reports. If a goal was 

not met, the report is to provide an explanation and present the actions 

needed to meet any unmet goals in the future. These reports are intended 

to provide important information to agency managers, policymakers, and 

the public on what each agency accomplished with the resources it was 

given. Agencies issued their first annual performance reports on their 

fiscal year 1999 performance in fiscal year 2000 and are required to issue a 

report on each subsequent performance plan.

The Office of Management and Budget (OMB) plays an important role in 

the management of federal government performance and specifically 

GPRA implementation. Part of OMB’s overall mission is to ensure that 

agency plans and reports are consistent with the President’s budget and 

administration policies. OMB is responsible for receiving and reviewing 

agencies’ strategic plans, annual performance plans, and annual 

performance reports. To improve the quality and consistency of these 

documents, OMB issues annual guidance to agencies for their preparation, 
Executive Summary

Page 6 GAO-04-38 Results-Oriented Government

including guidelines on format, required elements, and submission 

deadlines. GPRA requires OMB to prepare a governmentwide performance 

plan, based on agencies’ annual performance plan submissions. OMB also 

played an important role in the pilot phase of GPRA implementation by 

designating agencies for pilot projects in performance measurement, 

managerial accountability and flexibility, and performance budgeting, and 

assessing the results of the pilots. Finally, GPRA provides OMB with 

authority to grant agencies waivers to certain administrative procedures 

and controls.

Recent OMB guidance—OMB Circular A-11, July 2003—requires agencies 

to submit “performance budgets” in lieu of annual performance plans for 

their fiscal year 2005 budget submission to OMB and Congress. According 

to OMB, performance budgets should satisfy all the statutory requirements 

of GPRA for annual performance plans. In addition, agencies are to include 

all performance goals used in the assessment of program performance 

done under OMB’s Program Assessment Rating Tool (PART) process.3

 

Moreover, the guidance states that until all programs have been assessed 

by PART, the performance budget will also for a time include performance 

goals for agency programs that have not yet been assessed using PART. 

The expectation is that agencies are to substitute new or revised 

performance goals resulting from OMB’s review for goals it deemed 

unacceptable.

Results in Brief Among the purposes of GPRA cited by Congress was to improve federal 

program effectiveness and service delivery by promoting a new focus on 

results, service quality, and customer satisfaction by setting program goals 

measuring performance against goals, and reporting publicly on progress. 

Furthermore, GPRA was to improve congressional decision making by 

providing more objective information on achieving objectives, and on the 

relative effectiveness and efficiency of federal programs and spending. Ten 

years after enactment, GPRA’s requirements have laid a solid foundation of 

results-oriented agency planning, measurement, and reporting that have 

3

PART is a diagnostic tool developed by OMB that it has been using to rate the effectiveness 

of federal programs with a particular focus on program results. OMB’s goal is to review all 

federal programs over a 5-year period using the PART tool. OMB used the tool to review 

approximately 400 programs between the fiscal year 2004 budget cycle and the fiscal year 

2005 budget cycle—234 programs were assessed last year and 173 were assessed this year. 

Some reassessed programs were combined for review for the 2005 budget, which is why the 

number of programs assessed over the 2 years does not add up to exactly 400 programs.
Executive Summary

Page 7 GAO-04-38 Results-Oriented Government

begun to address these purposes. Focus group participants and high-level 

political appointees, as well as OMB officials we interviewed, cited positive 

effects of GPRA that they generally attributed to GPRA’s statutory 

requirements for planning and reporting. Performance planning and 

measurement have slowly yet increasingly become a part of agencies’ 

cultures. The results of our stratified, random sample survey of federal 

managers indicate that since GPRA went into effect governmentwide in 

1997, federal managers reported having significantly more of the types of 

performance measures called for by GPRA—particularly outcome-oriented 

performance measures. Survey data also suggested that more federal 

managers, especially at the Senior Executive Service (SES) level, believed 

that OMB was paying attention to their agencies’ efforts under GPRA. 

GPRA has also begun to facilitate the linking of resources to results, 

although much remains to be done in this area.

Beginning with agencies’ initial efforts to develop effective strategic plans 

in 1997 and annual performance plans and reports for fiscal year 1999, 

Congress, GAO, and others have commented on the quality of those efforts 

and provided constructive feedback on how agency plans and reports 

could be improved. According to our current review of the strategic plans, 

annual performance plans, and annual performance reports of six selected 

agencies, these documents reflect much of the feedback that was provided. 

For example, goals are more quantifiable and results oriented, and agencies 

are providing more information about goals and strategies to address 

performance and accountability challenges and the limitations to their 

performance data. However, certain serious weaknesses persist, such as 

lack of detail on how annual performance goals relate to strategic goals and 

how agencies are coordinating with other entities to address common 

challenges and achieve common objectives.

While a great deal of progress has been made in making federal agencies 

more results oriented, numerous challenges still exist. As we have noted 

before, top leadership commitment and sustained attention to achieving 

results, both within the agencies and at OMB, is essential to GPRA 

implementation. While one might expect an increase in agency leadership 

commitment since GPRA was implemented governmentwide beginning in 

fiscal year 1997, federal managers reported that such commitment has not 

significantly increased. Furthermore, although OMB has recently 

demonstrated leadership in its review of performance information from a 

budgetary perspective using the PART tool, it is unclear whether the results 

of those reviews, such as changes in program performance measures, will 

complement and be integrated with the long-term, strategic focus of GPRA. 
Executive Summary

Page 8 GAO-04-38 Results-Oriented Government

OMB provided significantly less guidance on GPRA implementation for the 

fiscal year 2005 budget, compared to the very detailed guidance provided in 

prior years. Without consistent guidance from OMB on meeting GPRA 

requirements and following best practices, it may be difficult to maintain 

the improvements in agency performance plans and reports or bring about 

improvements in areas where weaknesses remain. The commitment of top 

leadership within agencies, OMB, and Congress is critical to the success of 

strategic planning efforts. However, GPRA specifies time frames for 

updating strategic plans that do not correspond to presidential or 

congressional terms. As a result, an agency may be required to update its 

strategic plan a year before a presidential election and without input from a 

new Congress. A strategic plan should reflect the policy priorities of an 

organization’s leaders and the input of key stakeholders if it is to be an 

effective management tool.

Managers reported they had more performance measures, but indications 

that managers are making greater use of this information to improve 

performance are mixed. Additionally, managers reported several human 

capital-related challenges that impede results-oriented management, 

including a lack of authority and training to carry out GPRA requirements, 

as well as a lack of recognition for completing these tasks. Unfortunately, 

most existing federal performance appraisal systems are not designed to 

support a meaningful performance-based pay system in that they fail to link 

institutional, program, unit, and individual performance measurement and 

reward systems. Fewer than half of federal managers reported receiving 

relevant training in critical results-oriented management-related tasks. 

Managers also reported significant challenges persist in setting outcomeoriented goals, measuring performance, and collecting useful data. In 

some agencies, particularly those that have a research and development 

component, managers reported difficulties in establishing meaningful 

outcome measures. Managers also identified difficulties in distinguishing 

between the results produced by the federal program and results caused by 

external factors or nonfederal actors, such as with grant programs. Timely 

and useful performance information is not always available to federal 

agencies, making it more difficult to assess and report on progress 

achieved. Finally, agency officials believe that Congress could make 

greater use of performance information to conduct oversight and to inform 

appropriations decisions. GPRA provides a vehicle for Congress to 

explicitly state its performance expectations in outcome-oriented terms 

when establishing new programs or in exercising oversight of existing 

programs that are not achieving desired results.
Executive Summary

Page 9 GAO-04-38 Results-Oriented Government

Mission fragmentation and overlap contribute to difficulties in addressing 

crosscutting issues, particularly when those issues require a national focus, 

such as homeland security, drug control, and the environment. GPRA 

requires a governmentwide performance plan, where these issues could be 

addressed in a centralized fashion, but OMB has not issued a distinct plan 

since 1999. Most recently, the President’s fiscal year 2004 budget focused 

on describing agencies’ progress in addressing the President’s Management 

Agenda (PMA) and the results of PART reviews of agency programs. Such 

information is important and useful, but is not adequate alone to provide a 

broader and more integrated perspective of planned performance on 

governmentwide outcomes. GAO has previously reported on a variety of 

barriers to interagency cooperation, such as conflicting agency missions, 

jurisdiction issues, and incompatible procedures, data, and processes. A 

strategic plan for the federal government, supported by a set of key 

national indicators to assess the government’s performance, position, and 

progress, could provide an additional tool for governmentwide 

reexamination of existing programs, as well as proposals for new 

programs. Such a plan could be of particular value in linking agencies’ 

long-term performance goals and objectives horizontally across the 

government. In addition, it could provide a basis for integrating, rather 

than merely coordinating, a wide array of federal activities.

To address these challenges, continued and sustained commitment and 

leadership are needed. OMB, as the primary focal point for overall 

management in the federal government, can provide this leadership and 

direction working with the various management councils and work groups 

of the government. Also, governmentwide planning could better facilitate 

the integration of federal activities to achieve national goals.

GAO recommends that the Director of OMB (1) fully implement GPRA’s 

requirement to develop a governmentwide performance plan; (2) articulate 

and implement an integrated, complementary relationship between GPRA 

and PART; (3) provide clearer and consistent guidance to executive branch 

agencies on how to implement GPRA; (4) continue to maintain a dialogue 

with agencies about their performance measurement practices with a 

particular focus on grant-making, research and development, and 

regulatory functions to identify and replicate successful approaches 

agencies are using to measure and report on their outcomes, including the 

use of program evaluation tools; and, work with executive branch agencies 

to identify the barriers to obtaining timely data to show progress against 

performance goals and the best ways to report information when there are 

unavoidable lags in data availability; and (5) work with agencies to ensure 
Executive Summary

Page 10 GAO-04-38 Results-Oriented Government

they are making adequate investments in training on performance planning 

and measurement, with a particular emphasis on how to use performance 

information to improve program performance.

We also suggest that Congress consider amending GPRA to require that 

updates to agency strategic plans be submitted at least once every 4 years, 

12-18 months after a new administration begins its term. Additionally, 

consultations with congressional stakeholders on existing strategic plans 

should be held at least once every new Congress and revisions should be 

made as needed. Further, we suggest Congress use these consultations and 

its oversight role to clarify its performance expectations for agencies. 

Congress should also consider amending GPRA to require the President to 

develop a governmentwide strategic plan.

In commenting on a draft of this report, OMB generally agreed with our 

findings and conclusions. OMB agreed to implement most of our 

recommendations, but stated that the President’s Budget represents the 

executive branch’s governmentwide performance plan and could also serve 

as a governmentwide strategic plan. However, because of the budget’s 

focus on agency-level expenditures for the upcoming fiscal year, we believe 

that the President’s Budget provides neither a long-term nor an integrated 

perspective on the federal government’s performance. OMB’s comments 

appear in appendix VIII. Our response appears in chapter 5. We also 

provided relevant sections of the draft to the six agencies whose plans and 

reports we reviewed. DOE, HUD, and SSA disagreed with some of our 

observations, and we changed or clarified relevant sections of the report, 

as appropriate. Written comments from DOE, HUD, and SSA are reprinted 

in appendixes IX, X, and XI, respectively, along with our responses.

Principal Findings

GPRA Laid the Foundation 

for a More Results-Oriented 

Federal Government

Prior to enactment of GPRA, our 1992 review of the collection and use of 

performance data by federal agencies revealed that, although many 

agencies collected performance information at the program level, few 

agencies had results-oriented performance information to manage or make
Executive Summary

Page 11 GAO-04-38 Results-Oriented Government

strategic policy decisions for the agency as a whole.4 GPRA addressed 

agencies’ shortcomings by creating a comprehensive and consistent 

statutory foundation of required agencywide strategic plans, annual 

performance plans, and annual performance reports. Participants in eight 

focus groups comprised of experts on GPRA and federal managers from 23 

agencies cited the creation of this statutory foundation as one of the key 

accomplishments of GPRA. One of the premises of GPRA is that both 

congressional and executive branch oversight of federal agency 

performance were seriously hampered by a lack of adequate resultsoriented goals and performance information. As noted above, prior to the 

enactment of GPRA few agencies reported their performance information 

externally. OMB officials we interviewed as part of our current review 

suggested that OMB has been a key consumer of the performance 

information produced under GPRA and that it has provided a foundation 

for their efforts to oversee agency performance.

Federal managers’ views of GPRA’s effect on the federal government’s 

ability to deliver results to the American public were mixed. When asked 

about the direct effects of GPRA on the public, 23 percent of the federal 

managers surveyed agreed to a moderate or greater extent that GPRA 

improved their agency’s ability to deliver results to the American public. 

High-level political appointees we interviewed cited a number of examples 

of how the structure of GPRA created a greater focus on results in their 

agencies. Participants in our focus groups had mixed perceptions of 

GPRA’s effect on their agency’s ability to deliver results to the American 

public. Participants indicated GPRA has had a positive effect by shifting 

the focus of federal management from program activities and processes to 

achieving the intended results of those programs. Another major 

accomplishment of GPRA cited by focus group participants is that GPRA 

improved the transparency of government results to the American public. 

Other focus group participants had difficulty attributing the results their 

agencies achieved directly to GPRA’s requirements.

4

U.S. General Accounting Office, Program Performance Measures: Federal Agency 

Collection and Use of Performance Data, GAO/GGD-92-65 (Washington, D.C.: May 4, 1992).
Executive Summary

Page 12 GAO-04-38 Results-Oriented Government

Focus group and survey results suggest that performance planning and 

measurement have slowly, but increasingly, become a part of agencies’ 

cultures. Compared to the results of our 1997 governmentwide survey of 

federal managers, in our 2003 governmentwide survey more managers 

reported having performance measures for their programs. When we 

asked managers who said they had performance measures which of the five 

types of measures they had to a great or very great extent, they reported 

increases in all five types of measures between 1997 and 2003,5 all of which 

were statistically significant.

Similarly, focus group participants commented on certain cultural changes 

that had taken place within their agencies since the passage of GPRA in 

which the “vocabulary” of performance planning and measurement—e.g., a 

greater focus on performance measurement, orientation toward outcomes 

over inputs and outputs, and an increased focus on program evaluation—

had become more pervasive. This perception is partly born out by our 

survey results. Consistent with our survey results indicating increases in 

results-oriented performance measures, we also observed a significant 

decline in the percentage of federal managers who agreed that certain 

factors hindered measuring performance or using the performance 

information. Finally, our survey data suggested that more federal 

managers, especially at the SES level, believed that OMB was paying 

attention to their agencies’ efforts under GPRA, but with no corresponding 

increase in their concern that OMB would micromanage the programs in 

their agencies. 

5

Types of measures were defined in the questionnaire as follows: performance measures 

that tell us how many things we produce or services we provide (output measures); 

performance measures that tell us if we are operating efficiently (efficiency measures); 

performance measures that tell us whether or not we are satisfying our customers 

(customer service measures); performance measures that tell us about the quality of the 

products or services we provide (quality measures); and performance measures that would 

demonstrate to someone outside of our agency whether or not we are achieving our 

intended results (outcome measures).
Executive Summary

Page 13 GAO-04-38 Results-Oriented Government

Agencies have begun to establish a link between results and resources. 

Our 1998 assessment of fiscal year 1999 performance plans found that 

agencies generally covered the program activities in their budgets, but 

most plans did not identify how the funding for those program activities 

would be allocated to performance goals.6 However, our subsequent 

reviews of performance plans indicate that agencies have made progress in 

demonstrating how their performance goals and objectives relate to 

program activities in the budget.

We reviewed a sample of six agencies’ strategic plans (Education, DOE, 

HUD, DOT, SBA, and SSA) and found the quality of the selected plans 

reflected improvements over these agencies’ initial strategic plans. Our 

1997 review of agencies’ draft strategic plans found that a significant 

amount of work remained to be done by executive branch agencies if their 

strategic plans were to fulfill the requirements of GPRA, serve as a basis for 

guiding agencies, and help congressional and other policymakers make 

decisions about agency activities and programs.7

 The six strategic plans we 

looked at for this 2003 review reflected many new and continuing strengths 

as well as improvements over the 1997 initial draft plans, but we continued 

to find certain persistent weaknesses. Of the six elements required by 

GPRA, the plans generally discussed all but one—program evaluation, an 

area in which we have found agencies often lack capacity. Although the 

strategic plans listed the program evaluations agencies intended to 

complete over the planning period, they generally did not address how the 

agencies planned to use their evaluations to establish new or revise 

existing strategic goals, as envisioned by GPRA. Finally, although not 

required by GPRA, the strategic plans would have benefited from more 

complete discussions of how agencies planned to coordinate and 

collaborate with other entities to address common challenges and achieve 

common or complementary goals and objectives.

6

U.S. General Accounting Office, Managing for Results: An Agenda to Improve the 

Usefulness of Agencies’ Annual Performance Plans, GAO/GGD/AIMD-98-228 (Washington, 

D.C.: Sept. 8, 1998).

7

U.S. General Accounting Office, Managing for Results: Critical Issues for Improving 

Agencies’ Strategic Plans, GAO/GGD-97-180 (Washington, D.C.: Sept. 16, 1997).
Executive Summary

Page 14 GAO-04-38 Results-Oriented Government

The six selected agencies’ fiscal year 2004 annual performance plans 

addressed some weaknesses of earlier plans, but there is still significant 

room for improvement. During our review of agencies’ first annual 

performance plans, which presented agencies’ annual performance goals 

for fiscal year 1999,8 we found that substantial further development was 

needed for these plans to be useful in a significant way to congressional 

and other decision makers. Most of the 2004 plans that we reviewed 

showed meaningful improvements over the fiscal year 1999 plans by 

showing a clearer picture of intended performance, providing strategies 

and resources that were more specifically related to achieving agency 

goals, and providing a greater level of confidence that performance data 

would be credible. But these plans also contained a number of serious 

weaknesses, such as inadequate discussion of coordination and 

collaboration and inconsistent or limited discussions of procedures used to 

verify and validate performance data, which limited their quality and 

undermined their usefulness.

Our review of the six agencies’ fiscal year 2002 performance reports 

showed a number of strengths and improvements over their fiscal year 

1999 performance reports, as well as areas that needed improvement. As 

we found in our earlier reviews, the six agencies’ fiscal year 2002 reports 

generally allowed for an assessment of progress made in achieving agency 

goals. In addition, the majority of agencies discussed the progress 

achieved in addressing performance and accountability challenges 

identified by agency inspectors general and GAO. However, as with the 

fiscal year 1999 reports, many of the weaknesses we identified in the 

agencies’ fiscal year 2002 reports were related to the significant number of 

performance goals not achieved or for which performance data were 

unavailable. In addition, the majority of the reports we reviewed did not 

include other GPRA requirements, such as a summary of the findings from 

program evaluations. Finally, only one of the six agencies clearly linked its 

costs to the achievement of performance goals or objectives.

Challenges to GPRA 

Implementation Exist

While a great deal of progress has been made in making federal agencies 

more results oriented, numerous challenges still exist to effective 

implementation of GPRA. We observed in our 1997 report that we would 

expect to see managers’ positive perceptions on items, such as the extent 

8

GAO/GGD/AIMD-98-228.
Executive Summary

Page 15 GAO-04-38 Results-Oriented Government

to which top leadership is committed to achieving results, become more 

prevalent and the gap between SES and non-SES managers begin to narrow 

as GPRA and related reforms are implemented. However, these changes do 

not appear to be happening to the extent anticipated. The need for strong, 

committed, and sustained leadership extends to OMB as well. OMB has 

shown a commitment to improving the management of federal programs, 

both through its leadership in reviewing agency program performance 

using the PART tool as well as through the PMA. As part of the President’s 

budget preparation, PART clearly must serve the President’s interests. 

However, it is not well suited to addressing crosscutting (or horizontal) 

issues or to looking at broad program areas in which several programs 

address a common goal. GPRA was designed to address the needs of many 

users of performance information, including (1) Congress to provide 

oversight and inform funding decisions, (2) agency managers to manage 

programs and make internal resource decisions, and (3) the public to 

provide greater accountability. It is not yet clear the extent to which PART 

performance goals and measures will compete with agencies’ long-term, 

strategic GPRA goals and objectives that were established in consultation 

with Congress and other stakeholders.

We also found that, while the quality of agency plans and reports have 

improved overall since their initial efforts, they continue to suffer from 

certain persistent weaknesses as noted above. However, OMB’s July 2003 

guidance for preparation and submission of annual performance plans is 

significantly shorter and less detailed than its 2002 guidance. Consistent, 

more explicit OMB guidance on preparing GPRA documents can help 

ensure that gains in the quality of GPRA documents are maintained and 

provide a resource for agency managers to make further improvements in 

those documents.

We also found that timing issues may affect the development of agency 

strategic plans that are meaningful and useful to top leadership. The 

commitment and sustained attention of top leadership within agencies, 

OMB, and Congress is critical to the success of strategic planning efforts. A 

strategic plan should reflect the policy priorities of an organization’s 

leaders and the input of key stakeholders if it is to be an effective 

management tool. However, GPRA specifies time frames for updating 

strategic plans that do not correspond to presidential or congressional 

terms. As a result, an agency may be required to update its strategic plan a 

year before a presidential election and without input from a new Congress. 

If a new president is elected, the updated plan is essentially moot and 

agencies must spend additional time and effort revising it to reflect new 
Executive Summary

Page 16 GAO-04-38 Results-Oriented Government

priorities. Our focus group participants, including GPRA experts, strongly 

agreed that this timing issue should be addressed by adjusting time frames 

to correspond better with presidential and congressional terms.

The benefit of collecting performance information is only fully realized 

when this information is actually used by managers to bring about desired 

results. However, federal managers reported mixed results in the use of 

performance information. Focus group participants and survey 

respondents noted that although many federal managers understand and 

use results-oriented management concepts in their day-to-day activities, 

such as strategic planning and performance measurement, they do not 

always connect these concepts to the requirements of GPRA. According to 

our 2003 survey results, the reported use of performance information to a 

great or very great extent for nine management activities, such as setting 

program priorities or setting individual job expectations for staff, ranging 

from 41 to 66 percent, has not changed significantly since our first survey in 

1997. One exception was the reported use to a great or very great extent of 

performance information to adopt new program approaches or change 

work processes, which was significantly lower than the 1997 results. 

GPRA’s usefulness to agency leaders and managers as a tool for 

management and accountability was cited as a key accomplishment 

numerous times by focus group participants. However, a number of 

alternative views indicated that the usefulness of GPRA as a management 

tool has been limited. Our survey data also indicate that managers’ 

perceive their participation in activities related to the development and use 

of performance information has been limited.

Federal managers continue to confront a range of important human capital 

management challenges. These managers report that they are held 

accountable for program results, but may not have the decision-making 

authority they need to accomplish agency goals. Moreover, fewer than half 

of managers reported receiving relevant training. Managers also perceive a 

lack of positive recognition for helping agencies achieve results. 

Unfortunately, most existing federal performance appraisal systems are not 

designed to support a meaningful performance-based pay system in that 

they fail to link institutional, program, unit, and individual performance 

measurement and reward systems. In our view, one key need is to 

modernize performance management systems in executive agencies so that 

they link to the agency’s strategic plan, related goals, and desired outcomes 

and are therefore capable of adequately supporting more performancebased pay and other personnel decisions.
Executive Summary

Page 17 GAO-04-38 Results-Oriented Government

Managers reported persistent challenges in setting outcome-oriented goals, 

measuring performance, and collecting useful data. Focus group 

participants and survey respondents noted that outcome-oriented 

performance measures were especially difficult to establish when the 

program or line of effort was not easily quantifiable. In some agencies, 

particularly those that have a research and development component, 

managers reported difficulties in establishing meaningful outcome 

measures. Managers also identified difficulties in distinguishing between 

the results produced by the federal program and results caused by external 

factors or nonfederal actors, such as with grant programs. Finally, 

managers reported that timely and useful performance information is not 

always available.

Crosscutting issues continue to be a challenge to GPRA implementation. 

Our review of six agencies’ strategic and annual performance plans showed 

some improvement in addressing their crosscutting program efforts, but a 

great deal of improvement is still necessary. We have previously reported 

and testified that GPRA could provide OMB, agencies, and Congress with a 

structured framework for addressing crosscutting policy initiatives and 

program efforts. OMB could use the provision of GPRA that calls for OMB 

to develop a governmentwide performance plan to integrate expected 

agency-level performance. It could also be used to more clearly relate and 

address the contributions of alternative federal strategies. Unfortunately, 

this provision has not been fully implemented. Instead, OMB has used the 

President’s Budget to present high-level information about agencies and 

certain program performance issues. The current agency-by-agency focus 

of the budget does not provide the integrated perspective of government 

performance envisioned by GPRA. For example, the fiscal year 2004 

budget identified budget requests and performance objectives by agency, 

such as the U.S. Department of Defense, as opposed to crosscutting 

governmentwide themes. From this presentation, one could assume that 

the only activities the U.S. government planned to carry out in support of 

national defense were those listed under the chapter “Department of 

Defense.” However, the chapter on the fiscal year 2004 budget discussing 

“the Department of State and International Assistance Programs,” contains 

a heading titled, “Countering the Threat from Weapons of Mass 

Destruction.” And while OMB may have a technical reason for not 

classifying this task as being related to national defense or homeland 

security, it is unclear that a lay reader could make that distinction. The 

fiscal year 2005 budget also identified budget requests and performance 

objectives by agency, not by crosscutting theme.
Executive Summary

Page 18 GAO-04-38 Results-Oriented Government

A strategic plan for the federal government could provide an additional tool 

for governmentwide reexamination of existing programs, as well as 

proposals for new programs. If fully developed, a governmentwide 

strategic plan could potentially provide a cohesive perspective on the longterm goals of the federal government and provide a much needed basis for 

fully integrating, rather than merely coordinating, a wide array of federal 

activities. Successful strategic planning requires the involvement of key 

stakeholders. Thus, it could serve as a mechanism for building consensus. 

Further, it could provide a vehicle for the President to articulate long-term 

goals and a road map for achieving them. In addition, a strategic plan could 

provide a more comprehensive framework for considering organizational 

changes and making resource decisions. The development of a set of key 

national indicators could be used as a basis to inform the development of 

governmentwide strategic and annual performance plans. The indicators 

could also link to and provide information to support outcome-oriented 

goals and objectives in agency-level strategic and annual performance 

plans.

Focus group members believed that one of the main challenges to GPRA 

implementation was the reluctance of Congress to use performance 

information when making decisions, especially appropriations decisions. 

However, less than one quarter of federal managers in the 2003 survey 

shared that concern. Further, a recent Congressional Research Service 

review suggests that Congress uses performance information to some 

extent, as evidenced by citations in legislation and committee reports. 

While there is concern regarding Congress’ use of performance 

information, it is important to make sure that this information is useful. In 

other words, the information presented and its presentation must meet the 

needs of the user. Regular consultation with Congress about both the 

content and format of performance plans and reports is critical.

As a key user of performance information, Congress also needs to be 

considered a partner in shaping agency goals at the outset. GPRA provides 

a vehicle for Congress to explicitly state its performance expectations in 

outcome-oriented terms when consulting with agencies on their strategic 

plans or when establishing new programs or exercising oversight of 

existing programs that are not achieving desired results. This would 

provide important guidance to agencies that could then be incorporated in 

agency strategic and annual performance plans.
Executive Summary

Page 19 GAO-04-38 Results-Oriented Government

Recommendations for 

Executive Action

GAO recommends that the Director of OMB implement five suggestions to 

improve its guidance and oversight of GPRA implementation.

To provide a broader perspective and more cohesive picture of the federal 

government’s goals and strategies to address issues that cut across 

executive branch agencies, we recommend that the Director of OMB fully 

implement GPRA’s requirement to develop a governmentwide performance 

plan.

To achieve the greatest benefit from both GPRA and PART, we recommend 

that the Director of OMB articulate and implement an integrated and 

complementary relationship between the two. GPRA is a broad legislative 

framework that was designed to be consultative with Congress and other 

stakeholders, and allows for varying uses of performance information. 

PART looks through a particular lens for a particular use—the executive 

budget formulation process.

To improve the quality of agencies’ strategic plans, annual performance 

plans, and performance reports and help agencies meet the requirements of 

GPRA, we recommend that the Director of OMB provide clearer and 

consistent guidance to executive branch agencies on how to implement 

GPRA. Such guidance should include standards for communicating key 

performance information in concise as well as longer formats to better 

meet the needs of external users who lack the time or expertise to analyze 

lengthy, detailed documents.

To help address agencies’ performance measurement challenges, we 

recommend the Director of OMB engage in a continuing dialogue with 

agencies about their performance measurement practices with a particular 

focus on grant-making, research and development, and regulatory 

functions to identify and replicate successful approaches agencies are 

using to measure and report on their outcomes, including the use of 

program evaluation tools. Additionally, we recommend that the Director of 

OMB work with executive branch agencies to identify the barriers to 

obtaining timely data to show progress against performance goals and the 

best ways to report information where there are unavoidable lags in data 

availability. Governmentwide councils, such as the President’s 

Management Council and the Chief Financial Officers Council, may be 

effective vehicles for working on these issues.
Executive Summary

Page 20 GAO-04-38 Results-Oriented Government

To facilitate the transformation of agencies’ management cultures to be 

more results oriented, we recommend that the Director of OMB work with 

agencies to ensure they are making adequate investments in training on 

performance planning and measurement, with a particular emphasis on 

how to use performance information to improve program performance.

Matters for 

Congressional 

Consideration

GAO also identified two matters for congressional consideration to 

improve the governmentwide focus on results.

To ensure that agency strategic plans more closely align with changes in 

the federal government leadership, Congress should consider amending 

GPRA to require that updates to agency strategic plans be submitted at 

least once every 4 years, 12-18 months after a new administration begins its 

term. Additionally, consultations with congressional stakeholders should 

be held at least once every new Congress and interim updates made to 

strategic and performance plans as warranted. Congress should consider 

using these consultations along with its traditional oversight role and 

legislation as opportunities to clarify its performance expectations for 

agencies. This process may provide an opportunity for Congress to 

develop a more structured oversight agenda.

To provide a framework to identify long-term goals and strategies to 

address issues that cut across federal agencies, Congress should consider 

amending GPRA to require the President to develop a governmentwide 

strategic plan.

Agency Comments We provided a copy of the draft report to OMB for comment. OMB’s 

written comments are reprinted in appendix VIII. In general, OMB agreed 

with our findings and conclusions. OMB agreed to implement most of our 

recommendations, noting that these recommendations will enhance their 

efforts to make the government more results oriented. OMB agreed to 

(1) work with agencies to ensure they are provided adequate training in 

performance management, (2) revise its guidance to clarify the integrated 

and complementary relationship between GPRA and PART, and 

(3) continue to use PART to improve agency performance measurement 

practices and share those practices across government. 

In response to our recommendation that OMB fully implement GPRA’s 

requirement to develop a governmentwide performance plan, OMB stated 
Executive Summary

Page 21 GAO-04-38 Results-Oriented Government

that the President’s Budget represents the executive branch’s 

governmentwide performance plan. However, according to GAO’s review, 

the agency-by-agency focus of the budget over the past few years does not 

provide an integrated perspective of government performance, and thus 

does not meet GPRA’s requirement to provide a “single cohesive picture of 

the annual performance goals for the fiscal year.” To clarify this point, we 

added an example that illustrates the lack of integration between 

crosscutting issues in the budget. 

In response to our matter for congressional consideration that Congress 

should consider amending GPRA to require the President to develop a 

governmentwide strategic plan, OMB noted that the budget serves as the 

governmentwide strategic plan. However, the President’s Budget focuses 

on establishing agency budgets for the upcoming fiscal year. Unlike a 

strategic plan, it provides neither a long-term nor an integrated perspective 

on the federal government’s activities. A governmentwide strategic plan 

should provide a cohesive perspective on the long-term goals of the federal 

government and provide a basis for fully integrating, rather than primarily 

coordinating, a wide array of existing and relatively short-term federal 

activities.

We provided relevant sections of the draft report to Education, DOE, HUD, 

SBA, SSA, and DOT. Education and SBA did not provide any comments, 

while DOT provided minor technical comments. DOE, HUD, and SSA 

disagreed with some of our observations on their strategic plans, 

performance plans, and performance reports; we changed or clarified 

relevant sections of the report, as appropriate. Written comments from 

DOE, HUD, and SSA are reprinted in appendixes IX, X, and XI, respectively, 

along with our responses.
Page 22 GAO-04-38 Results-Oriented Government

Chapter 1

Introduction Chapte1r

From defending the homeland against terrorists, to preventing the spread 

of infectious diseases, to providing a reliable stream of social security 

income to retirees and supporting the transition from welfare to work, the 

federal government provides funding and services to the American public 

that can affect their lives in critical ways every day. However, the federal 

government is in a period of profound transition and faces an array of 

challenges and opportunities to enhance performance, ensure 

accountability, and position the nation for the future. A number of 

overarching trends, such as diffuse security threats and homeland security 

needs, increasing global interdependency, the shift to knowledge-based 

economies, and the looming fiscal challenges facing our nation, drive the 

need to reconsider the proper role for the federal government in the 21st 

century, how the government should do business (including how it should 

be structured), and in some instances, who should do the government’s 

business.

Without effective short- and long-term planning, which takes into account 

the changing environment and needs of the American public and the 

challenges they face and establishes goals to be achieved, federal agencies 

risk delivering programs and services that may or may not meet society’s 

most critical needs. At a cost to taxpayers of over $2 trillion annually, the 

federal government should be able to demonstrate to the American public 

that it can anticipate emerging issues, develop sound strategies and plans 

to address them, and be accountable for the results that have been 

achieved.

Concerned that the federal government was more focused on program 

activities and processes than the results to be achieved, Congress passed 

the Government Performance and Results Act of 1993 (GPRA).1

 The act 

required federal agencies to develop strategic plans with long-term 

strategic goals, annual goals linked to achieving the long-term goals, and 

annual reports on the results achieved. Now that GPRA has been in effect 

for 10 years, you asked us to assess the effectiveness of GPRA in creating a 

focus on results in the federal government. Specifically, this report 

discusses (1) the effect of GPRA over the last 10 years in creating a 

governmentwide focus on results and the government’s ability to deliver 

results to the American public, including an assessment of the changes in 

the overall quality of agencies’ strategic plans, annual performance plans, 

and annual performance reports; (2) the challenges agencies face in 

1

Pub. L. No. 103-62.
Chapter 1

Introduction

Page 23 GAO-04-38 Results-Oriented Government

measuring performance and using performance information in 

management decisions; and (3) how the federal government can continue 

to shift toward a more results-oriented focus.

Impact of Emerging 

Trends and Fiscal 

Challenges

With the 21st century challenges we are facing, it is more vital than ever to 

maximize the performance of federal agencies in achieving their long-term 

goals. The federal government must address and adapt to major trends in 

our country and around the world. At the same time, our nation faces 

serious long-term fiscal challenges. Increased pressure also comes from 

world events: both from the recognition that we cannot consider ourselves 

“safe” between two oceans—which has increased demands for spending on 

homeland security—and from the United States (U.S.) role in combating 

terrorism in an increasingly interdependent world. To be able to assess 

federal agency performance and hold agency managers accountable for 

achieving their long-term goals, we need to know what the level of 

performance is. GPRA planning and reporting requirements can provide 

this essential information.

Our country’s transition into the 21st century is characterized by a number 

of key trends, including

• the national and global response to terrorism and other threats to our 

personal and national security;

• the increasing interdependence of enterprises, economies, markets, civil 

societies, and national governments, commonly referred to as 

globalization;

• the shift to market-oriented, knowledge-based economies;

• an aging and more diverse U.S. population;

• rapid advances in science and technology and the opportunities and 

challenges created by these changes;

• challenges and opportunities to maintain and improve the quality of life 

for the nation, communities, families, and individuals; and

• the changing and increasingly diverse nature of governance structures 

and tools.
Chapter 1

Introduction

Page 24 GAO-04-38 Results-Oriented Government

As the nation and government policymakers grapple with the challenges 

presented by these evolving trends, they do so in the context of rapidly 

building fiscal pressures. GAO’s long-range budget simulations show that 

this nation faces a large and growing structural deficit due primarily to 

known demographic trends and rising health care costs. The fiscal 

pressures created by the retirement of the baby boom generation and rising 

health costs threaten to overwhelm the nation’s fiscal future. As figure 1 

shows, by 2040, absent reform or other major tax or spending policy 

changes, projected federal revenues will likely be insufficient to pay more 

than interest on publicly held debt. Further, our recent shift from surpluses 

to deficits means the nation is moving into the future in a more constrained 

fiscal position.

Figure 1: Composition of Spending as a Share of GDP Assuming Discretionary 

Spending Grows with GDP after 2003 and All Expiring Tax Provisions Are Extended

Notes: Although all expiring tax cuts are extended, revenue as a share of gross domestic product 

(GDP) increases through 2013 due to (1) real bracket creep, (2) more taxpayers becoming subject to 

the Alternative Minimum Tax, and (3) increased revenue from tax-deferred retirement accounts. After 

0

10

20

30

40

50

2003 2015 2030 2040

Percent of GDP

Fiscal year

Revenue

Social Security

Net interest

Medicare and Medicaid

All other spending

Source: GAO’s January 2004 analysis.
Chapter 1

Introduction

Page 25 GAO-04-38 Results-Oriented Government

2013, revenue as a share of GDP is held constant. This simulation assumes that currently scheduled 

Social Security benefits are paid in full throughout the simulation period.

The United States has had a long-range budget deficit problem for a 

number of years, even during recent years in which we had significant 

annual budget surpluses. Unfortunately, the days of surpluses are gone, 

and our current and projected budget situation has worsened significantly. 

The bottom line is that our projected budget deficits are not manageable 

without significant changes in “status quo” programs, policies, processes, 

and operations.

GPRA Background GPRA is the centerpiece of a statutory framework that Congress put in 

place during the 1990s to help resolve the long-standing management 

problems that have undermined the federal government’s efficiency and 

effectiveness and to provide greater accountability for results. In addition 

to GPRA, the framework comprises the Chief Financial Officers Act of 

1990, as amended by the Government Management Reform Act of 1994, and 

information technology reform legislation, including the Paperwork 

Reduction Act of 1995 and the Clinger-Cohen Act of 1996. Together, these 

laws provide a powerful framework for developing and integrating 

information about agencies’ missions and strategic priorities, the resultsoriented performance goals that flow from those priorities, performance 

data to show the level of achievement of those goals, and the relationship 

of reliable and audited financial information and information technology 

investments to the achievement of those goals.

GPRA was intended to address several broad purposes, including 

strengthening the confidence of the American people in their government; 

improving federal program effectiveness, accountability, and service 

delivery; and enhancing congressional decision making by providing more 

objective information on program performance.
Chapter 1

Introduction

Page 26 GAO-04-38 Results-Oriented Government

The basic requirements of GPRA for the preparation of strategic plans, 

annual performance plans, and annual program performance reports by 

executive branch agencies are the following:

2. The agency must develop annual performance plans covering each program 

 activity set forth in the agencies’ budgets* 

Building on the decisions made as part of the strategic planning process, GPRA requires 

executive agencies to develop annual performance plans covering each program activity set 

forth in the agencies’ budgets. Annual performance plans, covering the upcoming fiscal year, 

are to be submitted to Congress after the President’s Budget is submitted, which generally 

occurs in February. Each plan is to contain an agency’s annual performance goals and 

associated measures, which the agency is to use in order to gauge its progress toward 

accomplishing its strategic goals. OMB is to use the agencies’ performance plans to develop 

an overall federal government performance plan that is to be submitted with the President’s 

Budget. The performance plan for the federal government is to present to Congress a single 

cohesive picture of the federal government’s annual performance goals for a given fiscal 

year.

1. The agency’s strategic plan must 

 contain these six key elements

3. The agency must prepare annual reports on program performance for the previous 

 fiscal year, to be issued by March 31 each year

GPRA requires executive agencies to prepare annual reports on program performance for 

the previous fiscal year, to be issued by March 31 each year. In each report, an agency is to 

compare its performance against its goals, summarize the findings of program evaluations 

completed during the year, and describe the actions needed to address any unmet goals. 

Recent OMB guidance states that executive agencies must combine their program 

performance report with their accountability report and transmit the combined report for 

fiscal year 2003 by January 30, 2004, and the combined report for fiscal year 2004 by 

November 15, 2004.

● a comprehensive agency mission 

 statement; 

● agencywide long-term goals and 

 objectives for all major functions 

 and operations; 

● approaches (or strategies) and the 

 various resources needed to 

 achieve the goals and objectives; 

● a description of the relationship 

 between the long-term goals and 

 objectives and the annual 

 performance goals; 

● an identification of key factors, 

 external to the agency and beyond 

 its control, that could significantly 

 affect the achievement of the 

 strategic goals; and 

● a description of how program 

 evaluations were used to establish 

 or revise strategic goals and a 

 schedule for future program 

 evaluations.

*Program activity refers to the lists of projects and activities in the appendix portion of the Budget of the United States Government. Program 

 activity structures are intended to provide a meaningful representation of the operations financed by a specific budget account.
Chapter 1

Introduction

Page 27 GAO-04-38 Results-Oriented Government

The Office of Management and Budget (OMB) plays an important role in 

the management of the federal government’s performance, and specifically 

GPRA implementation. Part of OMB’s overall mission is to ensure that 

agency plans and reports are consistent with the President’s Budget and 

administration policies. OMB is responsible for receiving and reviewing 

agencies’ strategic plans, annual performance plans, and annual 

performance reports. To improve the quality and consistency of these 

documents, OMB issues annual guidance to agencies for their preparation, 

including guidelines on format, required elements, and submission 

deadlines.2

 GPRA requires OMB to prepare the overall governmentwide 

performance plan, based on agencies’ annual performance plan 

submissions. OMB also played an important role in the pilot phase of 

GPRA implementation by designating agencies for pilot projects in 

performance measurement, managerial accountability and flexibility, and 

performance budgeting, and assessing the results of the pilots. Finally, 

GPRA provides OMB with authority to grant agencies waivers to certain 

administrative procedures and controls.

Recent OMB guidance3 requires agencies to submit “performance budgets” 

in lieu of annual performance plans for their budget submission to OMB 

and Congress. Performance budgets are to meet all the statutory 

requirements of GPRA for annual performance plans. In addition, agencies 

are to include all performance goals used in the assessment of program 

performance done under OMB’s Program Assessment Rating Tool (PART) 

process.4

 Moreover, the guidance states that until all programs have been 

assessed by PART, the performance budget will also for a time include 

performance goals for agency programs that have not yet been assessed 

using PART. The expectation is that agencies are to substitute new or 

revised performance goals resulting from OMB’s review for goals it deemed 

unacceptable.

2

The guidance on the preparation of strategic plans, annual performance plans, and program 

performance reports is contained in OMB Circular A-11, Part 6.

3

OMB Circular A-11, July 2003.

4

PART is a diagnostic tool developed by OMB that it has been using to rate the effectiveness 

of federal programs with a particular focus on program results. OMB’s goal is to review all 

federal programs over a 5-year period using the PART tool. OMB used the tool to review 

approximately 400 programs between the fiscal year 2004 budget cycle and the fiscal year 

2005 budget cycle—234 programs were assessed last year and 173 were assessed this year. 

Some reassessed programs were combined for review for the 2005 budget, which is why the 

number of programs assessed over the 2 years does not add up to exactly 400 programs.
Chapter 1

Introduction

Page 28 GAO-04-38 Results-Oriented Government

In crafting GPRA, Congress recognized that managerial accountability for 

results is linked to managers having sufficient flexibility, discretion, and 

authority to accomplish desired results. GPRA authorizes agencies to 

apply for managerial flexibility waivers in their annual performance plans 

beginning with fiscal year 1999. The authority of agencies to request 

waivers of administrative procedural requirements and controls is intended 

to provide federal managers with more flexibility to structure agency 

systems to better support program goals. The nonstatutory requirements 

that OMB can waive under GPRA generally involve the allocation and use 

of resources, such as restrictions on shifting funds among items within a 

budget account. Agencies must report in their annual performance reports 

on the use and effectiveness of any GPRA managerial flexibility waivers 

that they receive.

OMB was to designate at least five agencies from the first set of pilot 

projects to test managerial accountability and flexibility during fiscal years 

1995 and 1996. We previously reported on the results of the pilot project to 

implement managerial flexibility waivers and found that the pilot did not 

work as intended.5

 OMB did not designate any of the seven departments 

and one independent agency that submitted a total of 61 waiver proposals 

as GPRA managerial accountability and flexibility pilots. For about threequarters of the waiver proposals, OMB or other central management 

agencies determined that the waivers were not allowable for statutory or 

other reasons or that the requirement for which the waivers were proposed 

no longer existed. For the remaining proposals, OMB or other central 

management agencies approved waivers or developed compromises by 

using authorities that were already available independent of GPRA.

Under GPRA, another set of pilot projects, which were scheduled for fiscal 

years 1998 and 1999, were to test performance budgeting—i.e., the 

presentation of the varying levels of performance that would result from 

different budget levels. We previously reported that OMB initially deferred 

these pilots—originally to be designated in fiscal years 1998 and 1999—to 

give federal agencies time to develop the capability of calculating the 

effects of marginal changes in cost or funding on performance.6

 When the 

5

U.S. General Accounting Office, GPRA: Managerial Accountability and Flexibility Pilot 

Did Not Work as Intended, GAO/GGD-97-36 (Washington, D.C.: Apr. 10, 1997).

6

U.S. General Accounting Office, Managing for Results: Agency Progress in Linking 

Performance Plans With Budgets and Financial Statements, GAO-02-236 (Washington, 

D.C.: Jan. 4, 2002).
Chapter 1

Introduction

Page 29 GAO-04-38 Results-Oriented Government

pilots began in August 1999, OMB designed them as case studies prepared 

by OMB staff to demonstrate how performance information could be used 

to compare alternatives and to develop funding recommendations for 

incorporation into the President’s fiscal year 2001 budget submission.

On January 18, 2001, OMB reported the results of five performance 

budgeting pilots that explored agencies’ capabilities of more formally 

assessing the effects of different funding levels on performance goals. 

OMB selected the pilots to reflect a cross section of federal functions and 

capabilities so that a representative range of measurement and reporting 

issues could be explored. In its report, OMB concluded that legislative 

changes were not needed. OMB reported that the pilots demonstrated that 

assuring further performance measurement improvements and steadily 

expanding the scope and quality of performance measures is paramount, 

and that the existing statute provides sufficient latitude for such 

improvement.

Overall, OMB concluded that the pilots raised several key challenges about 

performance budgeting at the federal level including, for example, the 

following: 

• In many instances, measuring the effects of marginal, annual budget 

changes on performance is not precise or meaningful.

• While continuing to change from an almost total reliance on output 

measures to outcome measures, it will be much more difficult to 

associate specific resource levels with those outcomes, particularly over 

short periods of time.

• Establishing clear linkages between funding and outcomes will vary by 

the nature of the program and the number of external factors.

• Delays in the availability of performance data, sometimes caused by 

agencies’ reliance on nonfederal program partners for data collection, 

will continue to present synchronization problems during budget 

formulation.
Chapter 1

Introduction

Page 30 GAO-04-38 Results-Oriented Government

Scope and 

Methodology

To meet the three objectives stated earlier, we reviewed our extensive prior 

work on GPRA best practices and implementation and collected 

governmentwide data to assess the government’s overall focus on results. 

We conducted a random, stratified, governmentwide survey of federal 

managers comparable to surveys we conducted in 1997 and 2000. We also 

held eight in-depth focus groups—seven comprised of federal managers 

from 23 federal agencies and one with GPRA experts. We also interviewed 

top appointed officials from the current and previous administrations. 

Finally, we judgmentally selected a sample of six agencies to review for 

changes in the quality of their strategic plans, performance plans, and 

performance reports since their initial efforts. The agencies we selected 

were the Departments of Education (Education), Energy (DOE), Housing 

and Urban Development (HUD), and Transportation (DOT) and the Small 

Business (SBA) and Social Security Administrations (SSA). In making this 

selection, we chose agencies that collectively represented the full range of 

characteristics in the following four areas: (1) agency size (small, medium, 

large); (2) primary program types (direct service, research, regulatory, 

transfer payments, and contracts or grants); (3) quality of fiscal year 2000 

performance plan based on our previous review (low, medium, high);7

 and 

(4) type of agency (cabinet department and independent agency). 

Appendix I contains a more detailed discussion of our scope and 

methodology.

We performed our work in Washington, D.C., from January through 

November 2003 in accordance with generally accepted government 

auditing standards. Major contributors to this report are listed in appendix 

XII.

7

GAO/GGD/AIMD-99-215. Based on how we had rated agencies’ annual performance plans 

on their picture of performance, specificity of strategies and resources, and the degree of 

confidence that performance information will be credible, we assigned numeric values to 

each agencies’ rating (e.g., clear=3, general=2, limited=1, unclear=0) and added them up to 

determine overall quality of high, medium, or low. An agency’s plan was considered high 

quality if its score was between 7-9, a score of 5-6 was considered medium quality, and a 

score of 3-4 was low. No agencies received a score lower than 3.
Page 31 GAO-04-38 Results-Oriented Government

Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government Chapte2r

Among the purposes of GPRA cited by Congress was to improve federal 

program effectiveness and service delivery by promoting a new focus on 

results, service quality, and customer satisfaction by setting program goals, 

measuring performance against goals, and reporting publicly on progress. 

Furthermore, GPRA was to improve congressional decision making by 

providing better information on achieving objectives, and on the relative 

effectiveness and efficiency of federal programs and spending. Ten years 

after enactment, GPRA’s requirements have laid a foundation of resultsoriented agency planning, measurement, and reporting that have begun to 

address these purposes. Focus group participants, high-level political 

appointees, and OMB officials we interviewed cited positive effects of 

GPRA that they generally attributed to GPRA’s statutory requirements for 

planning and reporting. Our survey results indicate that since GPRA went 

into effect governmentwide in 1997, federal managers reported having 

significantly more of the types of performance measures called for by 

GPRA—particularly outcome-oriented performance measures. GPRA has 

also begun to facilitate the linking of resources to results, although much 

remains to be done in this area.

GPRA Statutory 

Requirements Laid a 

Foundation for 

Agencywide ResultsOriented Management

Prior to enactment of GPRA, our 1992 review of the collection and use of 

performance data by federal agencies revealed that, although many 

agencies collected performance information at the program level, few 

agencies had results-oriented performance information to manage or make 

strategic policy decisions for the agency as a whole.1

 Federal agencies 

surveyed indicated that many had a single, long-term plan that contained 

goals, standards, or objectives for the entire agency or program. Many of 

these agencies also reported they collected a wide variety of performance 

measures. However, in validating the survey responses with a sample of 

agencies, we found that measures were typically generated and used by 

program-level units within an agency and focused on measuring work 

activity levels and outputs or compliance with statutes. Little of this 

performance information was transparent to Congress, OMB, or the public 

and few of the agencies we visited used performance measures to manage 

toward long-term objectives. Few of the agencies surveyed had the 

infrastructure in place, such as a unified strategic plan with measurable 

goals, an office that collected performance measures, and regular 

consolidated reports, to tie plans and measures.

1

GAO/GGD-92-65. 
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 32 GAO-04-38 Results-Oriented Government

GPRA addressed these shortcomings by creating a comprehensive and 

consistent statutory foundation of required agencywide strategic plans, 

annual performance plans, and annual performance reports. In contrast to 

prior federal government efforts to measure performance, GPRA explicitly 

emphasized that, in addition to performance indicators that agencies may 

need to manage programs on a day-to-day basis, such as quantity, quality, 

timeliness, and cost, agencies also needed outcome-oriented goals and 

measures that assess the actual results, effects, or impact of a program or 

activity compared to its intended purpose.

Expert and agency focus group participants cited the creation of this 

statutory foundation as one of the key accomplishments of GPRA. 

Participants agreed that GPRA created a framework in statute for federal 

agencies to plan their activities in order to become more results oriented 

and provided a managerial tool for program accountability. Using this 

framework, agencies could develop and focus on strategies to carry out the 

programs they administer; set goals and identify performance indicators 

that will inform them whether or not they achieved the performance they 

expected; and determine what impact, if any, their programs have had on 

the American public. According to the experts in one of our focus groups, 

comparing federal agencies’ current mission statements contained in their 

strategic plans to what they were in the past demonstrates that the 

agencies have done some “soul searching” to get a better sense of what 

their role is (or should be) and how they can achieve it. Given that GPRA is 

in statute, participants indicated that the use of this planning framework is 

likely to be sustained within agencies.

One of the premises of GPRA is that both congressional and executive 

branch oversight of federal agency performance were seriously hampered 

by a lack of adequate results-oriented goals and performance information. 

As noted above, prior to the enactment of GPRA few agencies reported 

their performance information externally. OMB officials we interviewed as 

part of our current review suggested that OMB has been a key consumer of 

the performance information produced under GPRA and that it has 

provided a foundation for their efforts to oversee agency performance.
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 33 GAO-04-38 Results-Oriented Government

For example, during the development of the fiscal year 2004 budget, OMB 

used PART to review and rate 234 federal programs. We recently reported 

that one of PART’s major impacts was its ability to highlight OMB’s 

recommended changes in program management and design.2

 PART 

reviews look at four elements—program purpose and design, strategic 

planning, program management, and program results/accountability—and 

rate the program on how well each of these elements is executed. 

However, without the foundation of missions, goals, strategies, 

performance measures, and performance information generated under 

GPRA, such oversight would be difficult to carry out.

Participants in most of our focus groups also agreed that GPRA has been a 

driving force behind many cultural changes that have occurred within 

federal agencies. Highlighting the focus on results, participants stated that 

GPRA had stimulated a problem-solving approach within federal agencies 

and encouraged agency managers to think creatively when developing 

performance indicators for their programs. GPRA has also changed the 

dialogue within federal agencies; front-line managers and staff at lower 

levels of the organization now discuss budget issues in connection with 

performance. Similarly, experts noted that information about performance 

management and resource investments are more frequently communicated 

between agency officials and Congress than in the past. Within agencies, 

GPRA documents can provide a context of missions, goals, and strategies 

that political appointees can use to articulate agencies’ priorities.

Views on GPRA’s Effect on 

the Federal Government’s 

Ability to Deliver Results to 

the American Public Were 

Mixed

A key purpose of GPRA was “to improve the confidence of the American 

people in the capability of the Federal Government, by systematically 

holding Federal agencies accountable for achieving program results.” 

When asked about the direct effects of GPRA on the public in our 2003 

survey, an estimated 23 percent of federal managers agreed to a moderate 

or greater extent that GPRA improved their agency’s ability to deliver 

results to the American public; a larger percentage—38 percent—chose a 

“no basis to judge/not applicable” category.

When a similar question was posed in our focus groups with experts and 

federal managers, participants’ views were generally mixed. Some federal 

2

U.S. General Accounting Office, Performance Budgeting: Observations on the Use of 

OMB’s Program Assessment Rating Tool for the Fiscal Year 2004 Budget, GAO-04-174

(Washington, D.C.: Jan. 30, 2004).
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 34 GAO-04-38 Results-Oriented Government

managers in our focus groups agreed that GPRA has had a positive effect 

on raising awareness on many performance issues, and that in and of itself 

is a way of delivering results. The information gathered and reported for 

GPRA allows agencies to make better-informed decisions, which improves 

their ability to achieve results. Other participants stated that while certain 

aspects of GPRA-related work have been positive, agencies’ ability to 

deliver results and public awareness of their activities cannot always be 

exclusively attributed to GPRA. For example, some participants stated that 

many agencies rely on grant recipients to carry out their work, and 

delivering results to the American public depends, to a large extent, on the 

diligence of these organizations to implement their programs; such results 

would not change dramatically if GPRA were no longer a requirement.

A number of the political appointees we interviewed cited examples of 

outcomes they believe would not have occurred without the structure of 

GPRA. For example, a former deputy secretary of the Department of 

Veterans Affairs (VA) stated that “the Results Act brought about a 

fundamental rethinking of how we managed our programs and

processes. . . . We developed a strategic plan that was veteranfocused. . . . We made every effort to define program successes from the 

veteran’s perspective.” A former Chief Financial Officer (CFO) cited 

Customs Service goals to reduce the quantity of illegal drugs flowing into 

the United States and the Food and Drug Administration’s focus on 

speeding up the approval of new drugs as examples of outcomes that can 

make a big difference in people’s lives.

Another major accomplishment of GPRA cited by our focus group 

participants is that GPRA improved the transparency of government results 

to the American public. As noted above, prior to GPRA, few agencies 

reported performance results outside of their agencies. Focus group 

participants indicated a key accomplishment of GPRA was its value as a 

communication tool by increasing the transparency to the public of what 

their agencies did in terms the public could understand. For example, 

information on agencies’ strategic plans, performance goals, measures, and 

results are easily obtainable from agency Web sites. One focus group 

participant commented that GPRA helps bureaucrats explain to 

nonbureaucrats what the federal government does in terms they can better 

understand. Other comments indicated that because of GPRA agencies 

could now tell Congress and the American public what they are getting for 

their money.
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 35 GAO-04-38 Results-Oriented Government

More Managers Reported 

Having Performance 

Measures

A fundamental element in an organization’s efforts to manage for results is 

its ability to set meaningful goals for performance and to measure 

performance against those goals. From our 2003 survey we estimate that 

89 percent of federal managers overall said there were performance 

measures for the programs they were involved with. This is a statistically 

significantly higher percentage than the 76 percent of managers who 

answered yes to this item on our 1997 survey. (See fig. 2.)

Figure 2: Percentage of Federal Managers Who Reported That There Were 

Performance Measures for the Programs with Which They Were Involved

a

There was a statistically significant difference between 1997 and 2003 surveys.

76

84

89

0

10

20

30

40

50

60

70

80

90

100

Percent

Year

1997a 2000 2003a

Source: GAO.
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 36 GAO-04-38 Results-Oriented Government

Moreover, when we asked managers who said they had performance 

measures which of five types of measures they had to a great or very great 

extent, they reported increases in all five types of measures between 1997 

and 2003,3

 all of which were statistically significant. (See fig. 3.) Notably, 

managers indicated the existence of outcome measures, defined as 

“performance measures that demonstrate to someone outside the 

organization whether or not intended results are being achieved,” grew 

from a low of 32 percent in 1997 to the current estimate of 55 percent, a 

level that is on par with output measures for the first time since we began 

our survey.

Figure 3: Percentage of Federal Managers Who Reported Having Specific Types of 

Performance Measures to a Great or Very Great Extent

a

There was a statistically significant difference between 1997 and 2003 surveys.

3

Types of measures were defined in the questionnaire as follows: performance measures 

that tell us how many things we produce or services we provide (output measures); 

performance measures that tell us if we are operating efficiently (efficiency measures); 

performance measures that tell us whether or not we are satisfying our customers 

(customer service measures); performance measures that tell us about the quality of the 

products or services we provide (quality measures); and performance measures that would 

demonstrate to someone outside of our agency whether or not we are achieving our 

intended results (outcome measures).

Source: GAO.

0

10

20

30

40

50

38

50

54

26

35

43

32

39

47

31

39

46

32

44

55 60

Percent

Performance measures

Outputa Efficiencya Customer servicea Qualitya Outcomea

1997

2000

2003
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 37 GAO-04-38 Results-Oriented Government

Similarly, focus group participants commented on certain cultural changes 

that had taken place within their agencies since the passage of GPRA in 

which the “vocabulary” of performance planning and measurement—e.g., a 

greater focus on performance management; orientation toward outcomes 

over inputs and outputs; and an increased focus on program evaluation—

had become more pervasive. This perception is partly born out by our 

survey results. Since 1997 those reporting a moderate to extensive 

knowledge of GPRA and its requirements shifted significantly from 26 

percent to 41 percent in 2003, while those reporting no knowledge of GPRA 

declined significantly from 27 percent to 20 percent. (See fig. 4.)

Figure 4: Percentage of Federal Managers Who Reported Their Awareness of GPRA

a

There was a statistically significant difference between 1997 and 2003 surveys.

Consistent with our survey results indicating increases in results-oriented 

performance measures and increasing GPRA knowledge, we also observed 

a significant decline in the percentage of federal managers who agreed that 

certain factors hindered measuring performance or using the performance 

information. For example, as shown in figure 5, of those who expressed an 

38

26

41

0

10

20

30

40

50

60

70

80

90

100

Percent

Year

1997a

Moderate to extensive knowledge

2000 2003a

Source: GAO.

27 26

20

No knowledge
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 38 GAO-04-38 Results-Oriented Government

opinion, the percentage of managers who noted that determining 

meaningful measures was a hindrance to a great or very great extent was 

down significantly from 47 percent in 1997 to 36 percent in 2003. Likewise, 

the percentage that agreed to a great or very great extent that different 

parties are using different definitions to measure performance was a 

hindrance also declined significantly from 49 percent in 1997 to 36 percent 

in 2003.

Figure 5: Percentage of Federal Managers Who Reported Hindrances to Measuring 

Performance or Using the Performance Information to a Great or Very Great Extent

Note: Percentages are based on those respondents answering on the extent scale. 

a

There was a statistically significant difference between 1997 and 2003.

Finally, our survey data suggested that more federal managers, especially at 

the Senior Executive Service (SES) level, believed that OMB was paying 

attention to their agencies’ efforts under GPRA. Moreover, there was no 

corresponding increase in their concern that OMB would micromanage the 

programs in their agencies. In our survey, we asked respondents to assess 

Source: GAO.

0

10

20

30

40

50 47 44

36

49

40

36

60

Percent

Difficulty

determining

meaningful 

measuresa

Different parties are 

using different 

definitions to measure 

performancea

1997

2000

2003
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 39 GAO-04-38 Results-Oriented Government

the extent to which OMB pays attention to their agencies’ efforts under 

GPRA. As seen in figure 6, in 2003, the percentage of respondents who 

responded “Great” or “Very Great” to this question (31 percent) was 

significantly higher than in 2000 (22 percent). Of those, SES respondents 

showed an even more dramatic increase, from 33 to 51 percent. We also 

asked respondents the extent to which their concern that OMB would 

micromanage programs in their agencies was a hindrance to measuring 

performance or using performance information. The percentage among 

those expressing an opinion that it was a hindrance to a great or very great 

extent was low—around 24 percent in 2003—with no significant difference 

between 2000 and 2003.

Figure 6: Percentage of Federal Managers and SES Managers Who Reported That 

OMB Paid Attention to Their Agency’s Efforts under GPRA to a Great or Very Great 

Extent

a

There was a statistically significant difference between 2000 and 2003 surveys.

33

51

0

10

20

30

40

50

60

70

80

90

100

Percent

Year

2000a

Federal managers

2003a

Source: GAO.

22

22

31

29

SES

Non-SES
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 40 GAO-04-38 Results-Oriented Government

GPRA Has Begun to 

Establish a Link between 

Resources and Results

Among its major purposes, GPRA aims for a closer and clearer linkage 

between requested resources and expected results. The general concept of 

linking performance information with budget requests is commonly known 

as performance budgeting. Budgeting is and will remain an exercise in 

political choice, in which performance can be one, but not necessarily the 

only, factor underlying decisions. However, efforts to infuse performance 

information into resource allocation decisions can more explicitly inform 

budget discussions and focus them—both in Congress and in agencies—on 

expected results, rather than on inputs.

GPRA established a basic foundation for performance budgeting by 

requiring that an agency’s annual performance plan cover each program 

activity in the President’s budget request for that agency. GPRA does not 

specify any level of detail or required components needed to achieve this 

coverage. Further, GPRA recognizes that agencies’ program activity 

structures are often inconsistent across budget accounts and thus gives 

agencies the flexibility to consolidate, aggregate, or disaggregate program 

activities, so long as no major function or operation of the agency is 

omitted or minimized. In addition, OMB guidance has traditionally 

required agencies to display, by budget program activity, the funding level 

being applied to achieve performance goals. OMB’s guidance on 

developing fiscal year 2005 performance budgets also encourages a greater 

link between performance and funding levels, however, it places greater 

emphasis on linking agencies’ long-term and annual performance goals to 

individual programs. At a minimum, agencies are to align resources at the 

program level, but they are encouraged to align resources at the 

performance goal level. Resources requested for each program are to be 

the amounts needed to achieve program performance goal targets.

Our 1998 assessment of fiscal year 1999 performance plans found that 

agencies generally covered the program activities in their budgets, but 

most plans did not identify how the funding for those program activities 

would be allocated to performance goals.4

 However, our subsequent 

reviews of performance plans indicate that agencies have made progress in 

demonstrating how their performance goals and objectives relate to 

program activities in the budget.

4

GAO/GGD/AIMD-98-228. 
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 41 GAO-04-38 Results-Oriented Government

Over the first 4 years of agency efforts to implement GPRA, we observed 

that agencies continued to tighten the required link between their 

performance plans and budget requests.5

 Of the agencies we reviewed over 

this period, all but three met the basic requirement of GPRA to define a link 

between their performance plans and the program activities in their budget 

requests, and most of the agencies in our review had moved beyond this 

basic requirement to indicate some level of funding associated with 

expected performance described in the plan. Most importantly, more of the 

agencies we reviewed each year—almost 75 percent in fiscal year 2002 

compared to 40 percent in fiscal year 1999—were able to show a direct link 

between expected performance and requested program activity funding 

levels—the first step in defining the performance consequences of 

budgetary decisions. However, we have also observed that the nature of 

these linkages varied considerably. Most of the agencies in our review of 

fiscal year 2002 performance plans associated funding requests with higher, 

more general levels of expected performance, rather than the more 

detailed “performance goals or sets of performance goals” suggested in 

OMB guidance.

Although not cited by our group of experts, participants at six of our seven 

focus groups with federal managers cited progress in this area as a key 

accomplishment of GPRA. However, the participants also commented that 

much remains to be done in this area. The comments ranged from the 

general—GPRA provides a framework for planning and budgeting, to the 

more specific—GPRA created a definition of programs and how they will 

help the agency achieve its goals/objectives and the amount of money that 

will be required to achieve said goals/objectives. One of the comments 

implied that GPRA has helped to prioritize agency efforts by helping 

agencies align their efforts with programs or activities that make a 

difference. A political appointee we interviewed echoed this comment, 

stating that GPRA was pushing the department to think about what it gets 

out of the budget, not just what it puts into it—12 to 15 years ago the “so 

what” was missing from the budget process. Another political appointee 

we interviewed stated that the department was in the process of tying its 

goals to its budget formulation and execution processes and linking 

program costs to departmental goals. A former political appointee 

5

GAO-02-236; U.S. General Accounting Office, Performance Budgeting: Initial Experiences 

Under the Results Act in Linking Plans With Budgets, GAO/AIMD/GGD-99-67 (Washington, 

D.C.: Apr. 12, 1999); and Performance Budgeting: Fiscal Year 2000 Progress in Linking 

Plans With Budgets, GAO/AIMD-99-239R (Washington, D.C.: July 30, 1999).
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 42 GAO-04-38 Results-Oriented Government

discussed how his department used program performance information to 

inform a major information systems investment decision.

Furthermore, GAO case studies on the integration of performance 

information in budget decision making found that performance information 

has been used to inform the allocation of resources and for other 

management purposes at selected agencies. For example, the Veterans 

Health Administration provides its health care networks with performance 

information on patterns of patient care and patient health outcomes, which 

can be used to analyze resource allocation and costs and reallocate 

resources as appropriate.6

 Officials at the Administration for Children and 

Families said that training and technical assistance and salaries and 

expense funds are often allocated based on program and performance 

needs.7

 The Nuclear Regulatory Commission monitors performance 

against targets and makes resource adjustments, if needed, to achieve 

those targets.8

Although there has been progress in formally establishing the linkages 

between budgets and plans, our survey results are somewhat conflicting 

and have not reflected any notable changes either in managers’ perceptions 

governmentwide as to their personal use of plans or performance 

information when allocating resources, or in their perceptions about the 

use of performance information when funding decisions are made about 

their programs. Our 2003 survey data show that a large majority of federal 

managers reported that they consider their agency’s strategic goals when 

they are allocating resources. As shown in figure 7, on our 2003 survey, an 

estimated 70 percent of all federal managers agreed to a great or very great 

extent that they considered their agency’s strategic goals when allocating 

resources. However, using our 1997 survey responses as a baseline, it was 

not a statistically significant increase over 64 percent of the managers who 

responded comparably then. As shown in figure 8, a similar, but somewhat 

6

U.S. General Accounting Office, Managing for Results: Efforts to Strengthen the Link 

Between Resources and Results at the Veterans Health Administration, GAO-03-10

(Washington, D.C.: Dec. 10, 2002).

7

U.S. General Accounting Office, Managing for Results: Efforts to Strengthen the Link 

Between Resources and Results at the Administration for Children and Families, GAO-03-

09 (Washington, D.C.: Dec. 10, 2002).

8

U.S. General Accounting Office, Managing for Results: Efforts to Strengthen the Link 

Between Resources and Results at the Nuclear Regulatory Commission, GAO-03-258

(Washington, D.C.: Dec. 10, 2002).
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 43 GAO-04-38 Results-Oriented Government

smaller, majority (60 percent) of managers who expressed an opinion on 

our 2003 survey agreed to a great or very great extent that they used 

information from performance measurement when they were involved in 

allocating resources. In 1997, the comparable response was about the 

same at 62 percent. When we asked managers on another item, however, 

about the extent to which they perceived funding decisions for their 

programs being based on results or outcome-oriented performance 

information, only 25 percent of federal managers in 2003 endorsed this 

view to a great or very great extent. In 1997, 20 percent of managers 

expressed a comparable view, again not a significant increase. (See fig. 9.)

Figure 7: Percentage of Federal Managers Who Reported They Considered Strategic 

Goals to a Great or Very Great Extent When Allocating Resources

66 64

70

0

10

20

30

40

50

60

70

80

90

100

Percent

Year

1997 2000 2003

Source: GAO.
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 44 GAO-04-38 Results-Oriented Government

Figure 8: Percentage of Federal Managers Who Reported They Considered 

Performance Information to a Great or Very Great Extent When Allocating Resources

Note: Percentages are based on those respondents answering on the extent scale.

0

10

20

30

40

50

60

70

80

90

100

Percent

Year

1997 2000 2003

Source: GAO.

62

53

60
Chapter 2

GPRA Established the Foundation for a More 

Results-Oriented Federal Government

Page 45 GAO-04-38 Results-Oriented Government

Figure 9: Percentage of Federal Managers Who Reported That Funding Decisions 

Were Based on Results or Outcome-Oriented Performance Information to a Great or 

Very Great Extent

20 20

25

0

10

20

30

40

50

60

70

80

90

100

Percent

Year

1997 2000 2003

Source: GAO.
Page 46 GAO-04-38 Results-Oriented Government

Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist Chapte3r

Beginning with federal agencies’ initial efforts to develop effective strategic 

plans in 1997 and annual performance plans and reports for fiscal year 

1999, Congress, GAO, and others have commented on the quality of those 

efforts and provided constructive feedback on how agency plans and 

reports could be improved. On the basis of our current review of the 

strategic plans, annual performance plans, and annual performance reports 

of six selected agencies—Education, DOE, HUD, DOT, SBA, and SSA—we 

found that these documents reflect much of the feedback that was 

provided. For example, goals were more quantifiable and results oriented, 

and agencies were providing more information about goals and strategies 

to address performance and accountability challenges and the limitations 

to their performance data. However, certain weaknesses, such as lack of 

detail on how annual performance goals relate to strategic goals and how 

agencies are coordinating with other entities to achieve common 

objectives, persist. A detailed discussion of our scope and methodology 

and the results of our reviews of the six agencies’ most recent strategic 

plans, annual performance plans, and annual performance reports 

compared to initial efforts are contained in appendixes III, IV, and V, 

respectively.

Quality of Selected 

Strategic Plans 

Reflects Improvements 

over Initial Drafts

Under GPRA, strategic plans are the starting point and basic underpinning 

for results-oriented management. GPRA requires that an agency’s strategic 

plan contain six key elements: (1) a comprehensive agency mission 

statement; (2) agencywide long-term goals and objectives for all major 

functions and operations; (3) approaches (or strategies) and the various 

resources needed to achieve the goals and objectives; (4) a description of 

the relationship between the long-term goals and objectives and the annual 

performance goals; (5) an identification of key factors, external to the 

agency and beyond its control, that could significantly affect the 

achievement of the strategic goals; and (6) a description of how program 

evaluations were used to establish or revise strategic goals and a schedule 

for future program evaluations.
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 47 GAO-04-38 Results-Oriented Government

Our 1997 review of agencies’ draft strategic plans found that a significant 

amount of work remained to be done by executive branch agencies if their 

strategic plans were to fulfill the requirements of GPRA, serve as a basis for 

guiding agencies, and help congressional and other policymakers make 

decisions about activities and programs.1 Our assessment of 27 agencies’ 

initial draft strategic plans revealed several critical strategic planning 

issues that needed to be addressed. These planning issues were as follows:

• Most of the draft plans did not adequately link required elements in the 

plans, such as strategic goals to annual performance goals.

• Long-term strategic goals often tended to have weaknesses.

• Many agencies did not fully develop strategies explaining how their 

long-term strategic goals would be achieved.

• Most agencies did not reflect in their draft plans the identification and 

planned coordination of activities and programs that cut across multiple 

agencies.

• The draft strategic plans did not adequately address program 

evaluations.

We noted that Congress anticipated that it may take several planning cycles 

to perfect the process and that strategic plans would be continually refined 

as various planning cycles occur. We also recognized that developing a 

strategic plan is a dynamic process and that agencies, with input from OMB 

and Congress, were continuing to improve their plans.

Agencies have now had 6 years to refine their strategic planning processes. 

Although the six strategic plans we looked at for this review reflected many 

new and continuing strengths as well as improvements over the 1997 initial 

drafts, we continued to find certain persistent weaknesses. As depicted in 

table 1, of the six elements required by GPRA, the plans generally discussed 

all but one—program evaluation, an area in which we have found capacity 

is often lacking in federal agencies.2

 Although the strategic plans generally 

1

GAO/GGD-97-180.

2

U.S. General Accounting Office, Managing for Results: Challenges Agencies Face in 

Producing Credible Performance Information, GAO/GGD-00-52 (Washington, D.C.: Feb. 4, 

2000).
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 48 GAO-04-38 Results-Oriented Government

listed the program evaluations agencies planned to complete over the 

planning period, they generally did not address how the agencies planned 

to use their evaluations to establish new or revise existing strategic goals, 

as envisioned by GPRA. Finally, although not required by GPRA, the 

strategic plans would have benefited from more complete discussions of 

how agencies planned to coordinate with other entities to address common 

challenges or achieve common or complementary goals. Appendix III 

provides a more detailed discussion of (1) the required and other useful 

elements we reviewed to assess strategic plan strengths and weaknesses 

and (2) changes in the quality of the six agencies’ strategic plans we 

reviewed.

Table 1: Agencies’ Progress in Addressing Required Elements of Strategic Planning under GPRA

Sources: GAO analysis of agencies’ strategic plans in effect at the time of our review. See also, U.S. General Accounting Office, The Results Act: Observations on the Department of Education’s June 1997 

Draft Strategic Plan, GAO/HEHS-97-176R (Washington, D.C.: July 18, 1997); Results Act: Observations on DOE’s Draft Strategic Plan, GAO/RCED-97-199R (Washington, D.C.: July 11, 1997); The Results 

Act: Observations on the Department of Transportation’s Draft Strategic Plan, GAO/RCED-97-208R (Washington, D.C.: July 30, 1997); The Results Act: Observations on the Social Security Administration’s 

June 1997 Draft Strategic Plan, GAO/HEHS-97-179R (Washington, D.C.: July 22, 1997); The Results Act: Observations on the Small Business Administration’s Draft Strategic Plan, GAO/RCED-97-205R

(Washington, D.C.: July 11, 1997); and The Results Act: Observations on the Department of Housing and Urban Development’s Draft Strategic Plan, GAO/RCED-97-224R (Washington, D.C.: Aug. 8, 1997).

a

The 2003 plans for DOE and DOT were in draft form during the time of our review.

b

At the time of our review, the most recent SBA strategic plan was for fiscal years 2001-2008. SBA 

released a new strategic plan for fiscal years 2003-2008 in October 2003.

Element included in agency strategic plan?

Agency strategic plans Plan year

Mission 

statement

Long-term 

goals Strategies

Relationship 

between longterm goals and 

annual goals

External 

factors Evaluations

Department of Education 1997 X X X X X

2002 X X X X X

Department of Energy 1997 X X X

2003a X X X X X

Department of Housing and Urban 

Development

1997 X

2003 X X X X X

Small Business Administration 1997 X X X X

2001b X X X X X

Social Security Administration 1997 X X X X X X

2003 X X X X X X

Department of Transportation 1997 X X X

2003a X X X X X
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 49 GAO-04-38 Results-Oriented Government

Strategic Planning Strengths 

and Improvements from 

Initial Draft Plans

Consistent with our review of agencies’ 1997 strategic plans, the recent 

strategic plans we reviewed generally contained mission statements that 

were results oriented, distinct from other agencies, and covered the 

agencies’ major activities. DOT’s mission statement had improved by 

reflecting additional language from its enabling legislation that we 

recommended adding during our 1997 review. Still improvement could be 

made in this area as is shown by DOE’s mission statement. DOE’s mission 

was results oriented but did not address the department’s activities related 

to energy supply and conservation.

Our review of the current strategic plans also revealed improvements in the 

development of agencies’ long-term, strategic goals—essential for resultsoriented management. Although GPRA does not require that all of an 

agency’s long-term, strategic goals be results oriented, the intent of GPRA 

is to have agencies focus their strategic goals on results to the extent 

feasible. In addition, as required by GPRA, the goals should be expressed 

in a manner that could be used to gauge success in the future and should 

cover an agency’s major functions or activities. All of the strategic plans 

we reviewed contained long-term, strategic goals that demonstrated 

improvements in the quality of their 1997 goals. Agencies’ long-term 

strategic goals generally covered their missions, were results oriented, and 

were expressed in a manner that could be used to gauge future success. 

For example, SBA improved the quality of its long-term goals by focusing 

more on key outcomes to be achieved and less on process improvements, 

as was the case in its 1997 plan. In some cases, we observed strategic goals 

that addressed the agency’s organizational capacity to achieve results, such 

as SSA’s long-term goal to strategically manage and align staff to support its 

mission.
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 50 GAO-04-38 Results-Oriented Government

We also found improvements in how agencies’ current plans addressed 

performance and accountability challenges we had identified, a key 

weakness we identified in our earlier review. Each of the agency plans we 

reviewed discussed the long-term goals and strategies to address the 

challenges that we had identified. For example, Education’s strategic plan 

contained a long-term strategic goal to modernize the Federal Student 

Assistance programs and address identified problems in this area, which 

we have designated as high risk since 1990.3 SSA noted that it considered 

GAO-identified performance and accountability challenges when it 

determined its strategic goals and objectives, however not all of the 

challenges are clearly addressed in the plan.

A third area of improvement we observed was in the description of the 

strategies agencies planned to use to achieve their long-term strategic 

goals. In our review of agencies’ 1997 draft strategic plans, we found that 

many agencies did not fully develop strategies explaining how their longterm strategic goals would be achieved. In contrast, all six of the current 

strategic plans we reviewed contained strategies that appeared logically 

linked to achieving the agencies’ long-term goals.

Other strengths and improvements we observed in meeting GPRA’s basic 

requirements involved the reporting of external factors that could affect 

the achievement of the long-term goals and the identification of 

crosscutting activities, although as indicated below these discussions could 

be improved. The six agencies reviewed for this report each reported on 

external factors in current strategic plans. For example, for each of the 

strategic objectives in DOT’s strategic plan, DOT lists factors external to its 

control and how those factors could affect the achievement of its 

objectives. Although not a requirement, some of the better plans we 

reviewed discussed strategies to ameliorate the effect of external factors. 

For example, for an external factor on teacher certification under a goal on 

reading, Education’s plan states that the agency “will work with the states 

and national accreditation bodies to encourage the incorporation of 

research-based reading instruction into teacher certification 

requirements.”

3

Since 1990, GAO has periodically reported on government operations that it identifies as 

“high risk” because of the greater vulnerabilities to fraud, waste, abuse, and 

mismanagement. See U.S. General Accounting Office, High-Risk Series: An Update, GAO03-119 (Washington, D.C.: January 2003).
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 51 GAO-04-38 Results-Oriented Government

We have frequently reported that a focus on results, as envisioned by 

GPRA, implies that federal programs contributing to the same or similar 

results should be closely coordinated to ensure that goals are consistent 

and, as appropriate, program efforts are mutually reinforcing. This means 

that federal agencies are to look beyond their organizational boundaries 

and coordinate with other agencies to ensure that their efforts are aligned. 

During our 1997 review, we found that most agencies did not reflect in their 

draft plans the identification and planned coordination of activities and 

programs that cut across multiple agencies. In contrast, each of the six 

current agency strategic plans that we reviewed identified at least some 

activities and programs that cut across multiple agencies. For example, 

SBA’s 1997 plan contained no evidence of how the agency coordinated with 

other agencies, but the current plan contained a separate section 

describing crosscutting issues in the areas of innovation and research 

assistance, international trade assistance, business development 

assistance, veterans affairs, and disaster assistance.

Critical Strategic Planning 

Issues Needing Further 

Improvement

First, consistent with our 1997 review, the strategic plans we reviewed did 

not adequately link required elements in the plans. Although all of the 

agencies we reviewed provided some information on the relationship 

between their long-term and annual goals, the extent of information 

provided on how annual goals would be used to measure progress in 

achieving the long-term goals varied greatly. In the case of DOE, the plan 

provides a very brief description of the overall relationship between its 

long-term and annual goals with examples, but does not demonstrate how 

it will assess progress for each of its long-term goals and objectives. 

Another plan, DOT’s, refers the reader to the annual performance plan for 

information about annual goals. We have reported that this linkage is 

critical for determining whether an agency has a clear sense of how it will 

assess progress toward achieving its intended results.

Second, although the agencies’ descriptions of their strategies had 

improved since our initial reviews, with few exceptions, their strategies 

generally did not include information on how the agencies plan to align 

their activities, core processes, human capital, and other resources to 

support their mission-critical outcomes and whether they have the right 

mix of activities, skills, and resources to achieve their goals. Such 

information is critical to understanding the viability of the strategies. 

Furthermore, none of the agencies discussed alternative strategies they 

had considered in developing their plans. Without such discussions, it is 
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 52 GAO-04-38 Results-Oriented Government

unclear whether agency planning processes were truly strategic or simply a 

recasting of existing activities, processes, etc.

HUD was the only agency that provided any details of how it intended to 

coordinate with other agencies to achieve common or complementary 

goals for its crosscutting programs or activities. For example, to support 

its goal of “Equal Opportunity in Housing,” HUD’s plan states that HUD and 

the Department of Justice continue to coordinate their fair housing 

enforcement activities, especially with respect to responding quickly and 

effectively to Fair Housing Act complaints that involve criminal activity 

(e.g., hate crimes), a pattern and practice of housing discrimination, or the 

legality of state and local zoning or other land use laws or ordinances. We 

have reported that mission fragmentation and program overlap are 

widespread throughout the federal government.4

 As such, interagency 

coordination is important for ensuring that crosscutting programs are 

mutually reinforcing and efficiently implemented.

Finally, the draft strategic plans did not adequately address program 

evaluations. In combination with an agency’s performance measurement 

system, program evaluations can provide feedback to the agency on how 

well its activities and programs contributed to achieving strategic goals. 

For example, evaluations can be a potentially critical source of information 

for Congress and others in assessing (1) the appropriateness and 

reasonableness of goals; (2) the effectiveness of strategies by 

supplementing performance measurement data with impact evaluation 

studies; and (3) the implementation of programs, such as identifying the 

need for corrective action. Evaluations are important because they 

potentially can be critical sources of information for ensuring that goals are 

reasonable, strategies for achieving goals are effective, and that corrective 

actions are taken in program implementation. Five out of the six current 

plans that we reviewed included a discussion of program evaluations, 

however for most of these plans the discussions lacked critical information 

required by GPRA, such as a discussion of how evaluations were used to 

establish strategic goals or a schedule of future evaluations. For example, 

DOE’s plan stated that internal, GAO, and Inspector General (IG) 

evaluations were used as resources to develop its draft strategic plan, but 

specific program evaluations were not identified.

4

U.S. General Accounting Office, Managing for Results: Using the Results Act to Address 

Mission Fragmentation and Program Overlap, GAO/AIMD-97-146 (Washington, D.C.: 

Aug. 29, 1997).
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 53 GAO-04-38 Results-Oriented Government

Fiscal Year 2004 

Annual Performance 

Plans Addressed Some 

Weaknesses of Earlier 

Plans, but Still Have 

Room for Significant 

Improvement

According to our review of agencies’ first annual performance plans, which 

presented agencies’ annual performance goals for fiscal year 1999,5

 we 

found that substantial further development was needed for these plans to 

be useful in a significant way to congressional and other decision makers. 

Most of the fiscal year 1999 plans that we reviewed contained major 

weaknesses that undermined their usefulness in that they (1) did not 

consistently provide clear pictures of agencies’ intended performance, 

(2) generally did not relate strategies and resources to performance, and 

(3) provided limited confidence that agencies’ performance data will be 

sufficiently credible. Although all of the fiscal year 1999 plans contained 

valuable information for decision makers, their weaknesses caused their 

usefulness to vary considerably within and among plans.

As shown in table 2, our current review of agencies’ fiscal year 2004 

performance plans found that five agencies—Education, HUD, SBA, SSA, 

and DOT—improved their efforts to provide a clear picture of intended 

performance, with SSA and DOT being the clearest. Furthermore, the same 

five agencies improved the specificity of the strategies and resources they 

intended to use to achieve their performance goals, with DOT being the 

most specific. Finally, the same five agencies—Education, HUD, SBA, SSA, 

and DOT—made improvements in the area of greatest weakness—

reporting on how they will ensure performance data will be credible. 

However, only DOT’s plan provided a full level of confidence that the 

performance data the agency intended to collect would be credible. 

Appendix IV provides a more detailed discussion of (1) the required and 

other useful elements we reviewed to assess the clarity of the picture of 

intended performance, the specificity of the strategies and resources, and 

the level of confidence in the performance data and (2) changes in the 

quality of the six agencies’ annual performance plans we reviewed.

5

GAO/GGD/AIMD-98-228.
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 54 GAO-04-38 Results-Oriented Government

Table 2: Characterizations of Agencies’ Fiscal Year 1999 and 2004 Annual 

Performance Plans

Sources: GAO analysis of agencies’ fiscal year 2004 annual performance plans and U.S. General Accounting Office, Results Act: 

Observations on the Department of Education’s Fiscal Year 1999 Annual Performance Plan, GAO/HEHS-98-172R (Washington, D.C.: 

June 8, 1998); Results Act: Observations on DOE’s Annual Performance Plan for Fiscal Year 1999, GAO/RCED-98-194R (Washington, 

D.C.: May 28, 1998); Results Act: Observations on the Department of Housing and Urban Development’s Fiscal Year 1999 Annual 

Performance Plan, GAO/RCED-98-159R (Washington, D.C.: June 5, 1998); Results Act: Observations on the Small Business 

Administration’s Fiscal Year 1999 Annual Performance Plan, GAO/RCED-98-200R (Washington, D.C.: May 28, 1998); The Results Act: 

Observations on the Social Security Administration’s Fiscal Year 1999 Annual Performance Plan, GAO/HEHS-98-178R (Washington, 

D.C.: June 9, 1998); and Results Act: Observations on the Department of Transportation’s Annual Performance Plan for Fiscal Year 1999, 

GAO/RCED-98-180R (Washington, D.C.: May 12, 1998).

Picture of 

intended 

performance

(unclear, limited, 

general, clear)

Strategies and 

resources

(no, limited, 

general, specific)

Data credible

(no, limited, 

general, full)

Agency 1999 2004 1999 2004 1999 2004

Department of 

Education

Limited General Limited General Limited General

Department of Energy Limited Limited General General Limited Limited

Department of Housing 

and Urban Development

Limited General Limited General Limited General 

Small Business 

Administration

Limited General Limited General Limited General

Social Security 

Administration

Limited Clear Limited General No General

Department of 

Transportation

General Clear General Specific Limited Full
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 55 GAO-04-38 Results-Oriented Government

Plans Generally Provided a 

Clearer Picture of Intended 

Performance, Except for 

Crosscutting Areas

At the most basic level, an annual performance plan is to provide a clear 

picture of intended performance across the agency. Such information is 

important to Congress, agency managers, and others for understanding 

what the agency is trying to achieve, identifying subsequent opportunities 

for improvement, and assigning accountability. Our current review of 

agencies’ fiscal year 2004 performance plans found that five of the six 

agencies provided a clearer picture of intended performance than their 

fiscal year 1999 plans did, although only two of the 2004 plans—DOT’s and 

SSA’s—received the highest rating possible. As shown in table 2, except for 

DOT, the six agencies we reviewed for this report initially provided a 

limited picture of intended performance. Most of the fiscal year 1999 

performance plans we previously reviewed had at least some objective, 

quantifiable, and measurable goals, but few plans consistently included a 

comprehensive set of goals that focused on the results that programs were 

intended to achieve. Moreover, agencies did not consistently follow OMB’s 

guidance that goals for performance and accountability challenges be 

included in the plans. Agencies’ plans generally showed how their missions 

and strategic goals were related to their annual performance goals and 

covered all of the program activities in the agencies’ budget requests.6

 In 

addition, many agencies took the needed first step of identifying their 

crosscutting efforts, with some including helpful lists of other agencies 

with which they shared a responsibility for addressing similar national 

issues. However, the plans generally did not go further to describe how 

agencies expected to coordinate their efforts with other agencies.

The fiscal year 2004 plans improved the picture of performance by making 

annual goals and performance measures more results oriented, objective, 

and quantifiable. For example, Education’s plan included a measure for the 

number of states meeting their eighth grade mathematics achievement 

targets under the long-term goal to improve mathematics and science 

achievement for all students. We previously criticized Education’s 1999 

plan for lacking such outcome-oriented measures. Another overall 

improvement we observed was that all of the plans described intended 

efforts to address performance and accountability challenges we and 

others had previously identified. For instance, to address the 

governmentwide high-risk area of strategic human capital management, 

6

Program activity refers to the list of projects and activities in the appendix portion of the 

Budget of the United States Government. Program activity structures are intended to 

provide a meaningful representation of the operations financed by a specific budget 

account.
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 56 GAO-04-38 Results-Oriented Government

HUD states that to develop its staff capacity, it will complete a 

comprehensive workforce analysis in 2004 to serve as the basis to fill 

mission critical skill gaps through succession planning, hiring, and training 

initiatives in a 5-year human capital management strategy. The clarity of 

DOE’s plan remained limited because its annual goals were not clearly 

linked to its mission, the long-term goals in its strategic plan, or the 

program activities in its budget request.

Although five of the six agencies improved the clarity of the picture of 

intended performance, improvement is still needed in reporting on 

crosscutting efforts. In both the 1999 and 2004 plans, many agencies 

identified their crosscutting efforts, with some including helpful lists of 

other agencies with which they shared a responsibility for addressing 

similar national issues. Our review of fiscal year 2004 plans shows that the 

six agencies we reviewed still did not discuss how they expected to 

coordinate with other agencies to address common challenges or to 

achieve common or complementary performance goals. As we have 

reported previously, improved reporting on crosscutting efforts can help 

Congress use the annual performance plan to evaluate whether the annual 

goals will put the agency on a path toward achieving its mission and longterm strategic goals. In addition, the plans can aid in determining efforts to 

reduce significant program overlap and fragmentation that can waste 

scarce resources, confuse and frustrate program customers, and limit 

overall program effectiveness.

None of the six agencies’ plans indicated an intention to request waivers of 

specific administrative procedural requirements and controls that may be 

impeding an agencies’ ability to achieve results. This provision of GPRA 

allows agencies greater managerial flexibility in exchange for 

accountability for results. We previously reported on the results of the 

pilot project to implement this provision of GPRA and found that the pilot 

did not work as intended.7

 OMB did not designate any of the seven 

departments and one independent agency that submitted a total of 61 

waiver proposals as GPRA managerial accountability and flexibility pilots. 

For about three-quarters of the waiver proposals, OMB or other central 

management agencies determined that the waivers were not allowable for 

statutory or other reasons or that the requirement for which the waivers 

were proposed no longer existed. For the remaining proposals, OMB or 

other central management agencies approved waivers or developed 

7

GAO/GGD-97-36.
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 57 GAO-04-38 Results-Oriented Government

compromises by using authorities that were already available independent 

of GPRA.

Plans More Specifically 

Related Strategies and 

Resources to Performance 

Goals

To judge the reasonableness of an agency’s proposed strategies and 

resources, congressional and other decision makers need complete 

information on how the proposed strategies and resources will contribute 

to the achievement of agency goals. Agencies generally improved their 

plans by better relating strategies and resources to performance. 

Education’s, HUD’s, SBA’s, and SSA’s 1999 plans had a limited discussion, 

while DOE’s and DOT’s 1999 plans had a general discussion. In 2004, five of 

the six plans—Education’s, DOE’s, HUD’s, SBA’s, and SSA’s—provided 

general discussions of how their strategies and resources would contribute 

to achieving their performance goals. DOT’s 2004 plan improved to include 

a specific discussion. 

Our review of the 1999 plans found that most agencies’ performance plans 

did not provide clear strategies that described how performance goals 

would be achieved. In contrast, the 2004 performance plans we reviewed 

generally provided lists of the agencies’ current array of programs and 

initiatives. Several plans provided a perspective on how these programs 

and initiatives were necessary or helpful for achieving results. For 

example, DOE and HUD included in their plans a “means and strategies” 

section for each of their goals that described how the goal would be 

achieved. One strategy DOE identified to meet its goal of contributing 

unique, vital facilities to the biological environmental sciences was to 

conduct peer reviews of the facilities to assess the scientific output, user 

satisfaction, the overall cost-effectiveness of each facility’s operations, and 

their ability to deliver the most advanced scientific capability.

In addition, each of the agencies’ plans identified the external factors that 

could influence the degree to which goals are achieved. Some of the better 

plans, such as DOT’s and SBA’s, provided strategies to mitigate the negative 

factors or take advantage of positive factors, as appropriate. For example, 

for its transportation accessibility goals, DOT’s plan states that as the 

population ages, more people will require accessible public transit, for 

which states and local agencies decide how best to allocate federally 

provided resources. One of the strategies DOT intends to employ to 

address this external factor is the “Special Needs of Elderly Individuals and 

Individuals with Disabilities” grant program. The plan states the grant 

program will help meet transportation needs of the elderly and persons 
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 58 GAO-04-38 Results-Oriented Government

with disabilities when regular transportation services are unavailable, 

insufficient, or inappropriate to meet their needs.

Agencies’ 2004 plans did not consistently describe all the resources needed 

and how they would be used to achieve agency goals. Our review of 

agencies’ fiscal year 1999 plans found that most did not adequately 

describe—or reference other appropriate documents that describe—the 

capital, human, information, and financial resources needed to achieve 

their agencies’ performance goals. The 2004 plans we reviewed generally 

described the funding levels needed to achieve their performance goals 

overall and in some cases broke out funding needs by high-level 

performance goal. For example, SSA’s plan provides a general perspective 

on the financial resources needed to achieve its performance goals because 

it provides budget information by account and program activity. However, 

the plan is neither structured by budget program activity or account, nor 

does it provide a crosswalk between the strategic goals and budget 

program accounts. In contrast, HUD’s plan presented its requested funding 

and staffing levels at the strategic goal level, but did not present budget 

information at the level of its annual goals. In addition, although the plans 

make brief mention of nonfinancial resources, such as human capital, 

information technology, or other capital investments, little information is 

provided on how such resources would be used to achieve performance 

goals.

Plans Continue to Provide 

Less Than Full Confidence 

That Performance Data Will 

Be Credible

Credible performance information is essential for accurately assessing 

agencies’ progress towards the achievement of their goals and, in cases 

where goals are not met, identifying opportunities for improvement or 

whether goals need to be adjusted. Under GPRA, agencies’ annual 

performance plans are to describe the means that will be used to verify and 

validate performance data. To help improve the quality of agencies’ 

performance data, Congress amended GPRA through the Reports 

Consolidation Act of 2000 to require that agencies assess the completeness 

and reliability of the performance data in their performance reports. 

Agencies were also required to discuss in their report any material 

inadequacies in the completeness and reliability of their performance data 

and discuss actions to address these inadequacies. Meeting these new 

requirements suggests the need for careful planning to ensure that agencies 

can comment accurately on the quality of the performance data they report 

to the public. 
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 59 GAO-04-38 Results-Oriented Government

As shown in table 2, although five of the six agencies we reviewed 

improved in reporting how they plan to ensure that performance data will 

be credible, only one agency—DOT—improved enough over its 1999 plan 

to provide a full level of confidence in the credibility of its performance 

data. Four agencies—Education, HUD, SBA, and SSA—improved enough 

to provide a general level of confidence. However, DOE provided the same 

limited level of confidence in the credibility of the performance data as in 

its 1999 plan. Regarding all 24 of the fiscal year 1999 performance plans we 

reviewed, we found most provided only superficial descriptions of 

procedures that agencies intended to use to verify and validate 

performance data. Moreover, in general, agencies’ performance plans did 

not include discussions of documented limitations in financial and other 

information systems that may undermine efforts to produce high-quality 

data. As we have previously noted, without such information, and 

strategies to address those limitations, Congress and other decision makers 

cannot assess the validity and reliability of performance information.

We found that each of the 2004 plans we reviewed contained some 

discussion of the procedures the agencies would use to verify and validate 

performance information, although in some cases the discussion was 

inconsistent or limited. For example, the discussions of SBA’s verification 

and validation processes for its indicators in the 2004 plan were generally 

one- or two-sentence statements. SBA also noted that it does not 

independently verify some of the external data it gathers or that it does not 

have access to the data for this purpose. In contrast, the DOT plan referred 

to a separate compendium available on-line that provides source and 

accuracy statements, which give more detail on the methods used to 

collect performance data, sources of variation and bias in the data, and 

methods used to verify and validate the data.
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 60 GAO-04-38 Results-Oriented Government

In addition, all of the agencies except DOE discussed known limitations to 

performance data in their plans. These agencies’ plans generally provided 

information about the quality of each performance measure, including any 

limitations. According to DOE officials, DOE’s plan generally does not 

discuss data limitations because the department selected goals for which 

data are expected to be available and therefore did not anticipate finding 

any limitations. However, in our 2003 Performance and Accountability 

Series report on DOE, we identified several performance and 

accountability challenges where data were a concern, such as the need for 

additional information on the results of contractors’ performance to keep 

projects on schedule and within budget.8

 DOE’s contract management 

continues to be a significant challenge for the department and remains at 

high risk.

Finally, the remaining five agencies also discussed plans to address 

limitations to the performance data. For example, DOT’s plan provided a 

general discussion of the limitations to the internal and external sources of 

data used to measure performance. Detailed discussions were contained in 

an appendix to the plan and separate source and accuracy statements. This 

information had been lacking in its 1999 plan. Education, HUD, SBA, and 

SSA also provided information on limitations to their performance data and 

plans for improvement.

Strengths and 

Weaknesses of 

Selected Agencies’ 

Fiscal Year 2002 

Annual Performance 

Reports

Key to improving accountability for results as Congress intended under 

GPRA, annual performance reports are to document the results agencies 

have achieved compared to the goals they established. To be useful for 

oversight and accountability purposes, the reports should clearly 

communicate performance results, provide explanations for any unmet 

goals as well as actions needed to address them, and discuss known data 

limitations as well as how the limitations are to be addressed in the future. 

Compared to our reviews of the six agencies’ fiscal year 1999 performance 

reports, we identified a number of strengths and improvements as well as 

areas that continued to need improvement. Because the scope of our 

review of the fiscal year 2002 reports was broader than that for the fiscal 

year 1999 reports we previously reviewed, we were unable to make specific 

comparisons for the three characteristics we used to assess the fiscal year 

2002 reports. However, we discuss comparative information on aspects of 

8

U.S. General Accounting Office, Major Management Challenges and Program Risks: 

Department of Energy, GAO-03-100 (Washington, D.C.: January 2003).
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 61 GAO-04-38 Results-Oriented Government

the reports where available. Table 3 shows the results of our assessment of 

the six agencies’ annual performance reports for fiscal year 2002. 

Appendix V provides a more detailed discussion of (1) the required and 

other useful elements we reviewed to assess the clarity of the picture of 

performance, the clarity of the linkage between costs and performance, 

and the level of confidence in the performance data and (2) changes in the 

quality of the six agencies’ annual performance plans we reviewed.

Table 3: Characterizations of Agencies’ 2002 Annual Performance Reports 

Sources: GAO analysis of agencies’ fiscal year 2002 annual performance reports and U.S. General Accounting Office, Observations on 

the Department of Education’s Fiscal Year 1999 Performance Report and Fiscal Year 2001 Performance Plan, GAO/HEHS-00-128R

(Washington, D.C.: June 30, 2000); Observations on the Department of Energy’s Fiscal Year 1999 Accountability Report and Fiscal 

Years 2000 and 2001 Performance Plans, GAO/RCED-00-209R (Washington, D.C.: June 30, 2000); Observations on the Department of 

Housing and Urban Development’s Fiscal Year 1999 Performance Report and Fiscal Year 2001 Performance Plan, GAO/RCED-00-211R

(Washington, D.C.: June 30, 2000); Observations on the Small Business Administration’s Fiscal Year 1999 Performance Report and 

Fiscal Year 2001 Performance Plan, GAO/RCED-00-207R (Washington, D.C.: June 30, 2000); Observations on the Social Security 

Administration’s Fiscal Year 1999 Performance Report and Fiscal Year 2001 Performance Plan, GAO/HEHS-00-126R (Washington, D.C.: 

June 30, 2000); and Observations on the Department of Transportation’s Fiscal Year 1999 Performance Report and Fiscal Year 2001 

Performance Plan, GAO/RCED-00-201R (Washington, D.C.: June 30, 2000).

Progress in Providing a 

Clear Picture of 

Performance

The six agency reports that we reviewed contained a number of strengths, 

some of which we can describe as improvements over the reports on fiscal 

year 1999 performance. A key strength of four of the 2002 reports 

(Education, HUD, DOT, SSA) was a discussion of the relationship between 

the strategic plan, performance plan, and performance report. For 

example, SSA’s report identified relevant results that were linked to its 

strategic objective to deliver “citizen-centered, world-class service,” such 

as maintaining the accuracy, timeliness, and efficiency of service to people 

applying for its benefit programs. The clarity of the DOE and SBA reports 

Agency

Picture of 

performance

(unclear, limited, 

general, clear)

Resources linked 

to results

(no, limited, 

general, clear)

Data credible

(no, limited, 

general, full)

Department of Education Limited Clear General

Department of Energy General Limited Limited

Department of Housing 

and Urban Development

General No General

Small Business 

Administration

Limited General General

Social Security 

Administration

General Limited General

Department of 

Transportation

General No Full
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 62 GAO-04-38 Results-Oriented Government

was limited by not clearly relating agency performance results to strategic 

and annual performance goals. For example, the structure of SBA’s report 

reflected the objectives in its draft 2003 to 2008 strategic plan rather than 

those in its 2002 performance plan, making it difficult to assess progress 

against the original 2002 objectives. Furthermore, although there is no 

“right” number of performance measures to be used to assess progress, a 

number of the plans allowed for an easier review of results by limiting the 

overall number of measures presented or by highlighting key performance 

measures of greatest significance to their programs. For example, SBA 

discussed a total of 19 performance goals and DOT discussed a total of 40. 

Although SSA discussed a total of 69 performance goals, the report 

highlighted its progress in achieving 14 key goals. In contrast, Education, 

HUD, and DOE presented a total of 120, 184, and 260 measures, 

respectively. Furthermore, while Education and SSA each provided a table 

showing progress across all its measures, the other agencies did not 

provide such summary information.

As we found in our earlier reviews, the six agencies’ fiscal year 2002 reports 

generally allowed for an assessment of progress made in achieving agency 

goals. Some of the reports made this assessment easier than others by 

providing easy-to-read summary information. For example, SSA provided a 

table at the beginning of the report that summarized the results for each of 

its 69 indicators with the following dispositions: met, not met, almost met, 

and data not yet available. Other reports, such as HUD’s, required an 

extensive review to make this assessment. In addition, to place current 

performance in context, each of the agencies’ reports contained trend 

information, as required by GPRA, which allowed for comparisons between 

current year and prior year performance.

In addition, the majority of agencies maintained, or demonstrated 

improvements over, the quality of their 1999 reports in discussing the 

progress achieved in addressing performance and accountability 

challenges identified by agency IGs and GAO. For example, SBA’s report 

contained two broad overviews and an appendix describing the status of 

GAO audits and recommendations, as well as a description of the most 

serious management challenges SBA faces as identified by the agency’s IG.

Unfortunately, many of the weaknesses we identified in the agencies’ fiscal 

year 2002 reports were similar to those we found in their fiscal year 1999 

reports related to the significant number of performance goals (1) which 

were not achieved and lacked explanations or plans for achieving the goal 

in the future and (2) for which performance data were unavailable. Three 
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 63 GAO-04-38 Results-Oriented Government

of the six agencies we reviewed—HUD, SSA, and Transportation—did not 

consistently report the reasons for not meeting their goals. For example, 

Transportation provided explanations for only 5 of the 14 goals it did not 

meet. In addition, similar to our 1999 report findings, three of the six 

agencies we reviewed—HUD, SBA, and DOT—did not discuss their plans 

or strategies to achieve unmet goals in the future. For example, HUD 

reported “substantially meeting” only 47 percent of the performance targets 

in fiscal year 2002. However, although HUD provides various reasons for 

not meeting all its targets, it offers no information on plans or time frames 

to achieve the goals in the future. Finally, we continued to observe a 

significant number of goals for which performance data were unavailable. 

For example, performance data for 10 of SBA’s 19 performance goals were 

unavailable.

In addition, the majority of the reports we reviewed did not include other 

GPRA requirements. The reports generally did not evaluate the 

performance plan for the current year relative to the performance achieved 

toward the performance goals in the fiscal year covered by the report. The 

reports also did not discuss the use or effectiveness of any waivers in 

achieving performance goals. In addition, for two of the agencies—DOE 

and SBA—program evaluation findings completed during the fiscal year 

were not summarized. As we have previously noted, such evaluations 

could help agencies understand the relationship between their activities 

and the results they hope to achieve.
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 64 GAO-04-38 Results-Oriented Government

Progress in Linking 

Resources to Results

Although linking costs to performance goals is not a requirement of GPRA, 

both GPRA and the CFO Act emphasized the importance of linking 

program performance information with financial information as a key 

feature of sound management and an important element in presenting to 

the public a useful and informative perspective on federal spending. The 

committee report for GPRA suggested that developing the capacity to 

relate the level of program activity with program costs, such as cost per 

unit of result, cost per unit of service, or cost per unit of output, should be a 

high priority. In our survey of federal managers, this year we asked for the 

first time the extent to which federal managers had measures of costeffectiveness for the programs they were involved with. Only 31 percent of 

federal managers we surveyed reported having such measures to a great or 

very great extent, lower than any of the other types of measures associated 

with GPRA we asked about by at least 12 percent (see fig. 3 in ch. 2). Under 

the PMA, the current administration has set an ambitious agenda for 

performance budgeting, calling for agencies to better align budgets with 

performance goals and focus on capturing full budgetary costs and 

matching those costs with output and outcome goals. All this suggests that 

agencies will need to develop integrated financial and performance 

management systems that will enable the reporting of the actual costs 

associated with performance goals and objectives along with presentations 

designed to meet other budgetary or financial purposes, such as the 

accounts and program activities found in the President’s Budget and 

responsibility segments found in financial statements.9

9

According to OMB’s Statement of Federal Financial Accounting Standards No. 4—

Managerial Cost Accounting Standards, July 31, 1995, a responsibility segment is a 

component of a reporting entity that is responsible for carrying out a mission, conducting a 

major line of activity, or producing one or a group of related products or services. In 

addition, responsibility segments usually possess the following characteristics: (1) their 

managers report to the entity’s top management directly and (2) their resources and results 

of operations can be clearly distinguished from those of other segments of the entity. 

Managerial cost accounting should be performed to measure and report the costs of each 

segment’s outputs.
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 65 GAO-04-38 Results-Oriented Government

Of the six agencies we reviewed, only Education’s report clearly linked its 

budgetary information to the achievement of its performance goals or 

objectives. Education’s report laid out, using both graphics and text, the 

estimated appropriations associated with achieving each of its 24 

objectives. In addition the report provided the staffing in full-time 

equivalent employment (FTEs) and an estimate of the funds from salaries 

and expenses contributing to the support of each of these objectives. SBA’s 

report contained crosswalks that showed the relationship between SBA’s 

strategic goals, outcome goals, performance goals, and programs. Because 

SBA shows the resources for each program, a reader can infer a 

relationship between SBA’s resources and performance goals. However, 

the linkage between resources and results would be clearer if results and 

resources were presented by performance goal as well. SSA provided a 

limited view of the costs of achieving its performance goals by providing 

the costs associated with four out of five of its strategic goals.10 However, 

as reported by the IG, SSA needs to further develop its cost accounting 

system, which would help link costs to performance.11 DOE also provided 

a limited view of the costs of achieving its performance goals by organizing 

its performance information by budget program activity and associated net 

costs. According to DOE officials, the department plans to link its 

individual performance measures to the costs of program activities in 

future reports. Neither HUD nor DOT provided information on the cost of 

achieving individual performance goals or objectives.

Progress in Providing 

Confidence in the 

Credibility of Performance 

Data

To assess the degree to which an agency’s report provided full confidence 

that the agency’s performance information was credible, we examined the 

extent to which the reports discussed the quality of the data presented. As 

shown in table 3, only DOT’s report provided a full level of confidence in 

the quality of the data. The other agencies provided general or limited 

confidence in their data.

10SSA noted that its fifth strategic goal, “Valued Employees,” supports the accomplishment 

of all its basic functions, so its resources are inherently included in the other four goals.

11According to the IG, SSA began to implement an improved cost accounting system in fiscal 

year 2002, which was to be phased in over the next 3 to 4 years.
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 66 GAO-04-38 Results-Oriented Government

All six agencies in our current review complied with the Reports 

Consolidation Act of 2000 by including assessments of the completeness 

and reliability of their performance data in their transmittal letters. In 

contrast, we found that only 5 of the 24 CFO Act agencies included this 

information in their fiscal year 2000 performance reports.12 Of the six 

agencies in our current review, only DOE provided this assessment in its 

fiscal year 2000 report. For example, the Secretary of DOT stated in the 

transmittal letter that the 2002 report “contains performance and financial 

data that are substantially complete and reliable.” However, only two of 

the six agencies also disclosed material inadequacies in the completeness 

and reliability of their performance data and discussed actions to address 

the inadequacies in their transmittal letters. For example, SBA stated in its 

transmittal letter that it is “working to improve the completeness and 

reliability of the performance data for the advice provided to small 

business through SBA’s resource partners.” SBA explained that data for 

this aspect of its performance are collected through surveys, which are 

inconsistent and not comparable, and for which client responses are 

difficult to obtain. SBA stated that it is working to improve the survey 

instruments it uses to obtain performance data.

In addition to the requirements of the Reports Consolidation Act, we have 

previously reported on other practices that enhance the credibility of 

performance data that are not specifically required by GPRA. For instance, 

discussions of standards and methods used by agencies to assess the 

quality of their performance data in their performance reports provide 

decision makers greater insight into the quality and value of the 

performance data. None of the reports explicitly referred to a specific 

standard they used, however, DOE described its method for assuring data 

quality. The report states that the heads of DOE’s organizational elements 

certified the accuracy of their performance data. DOE subsequently 

reviewed the data for quality and completeness.

Other useful practices that help foster transparency to the public and assist 

decision makers in understanding the quality of an agency’s data include: 

(1) discussion of data quality, including known data limitations and actions 

to address the limitations, and (2) discussion of data verification and 

12U.S. General Accounting Office, Performance Reporting: Few Agencies Reported on the 

Completeness and Reliability of Performance Data, GAO-02-372 (Washington, D.C.: Apr. 26, 

2002).
Chapter 3

Agencies Have Addressed Many Critical 

Performance Planning and Reporting 

Challenges, but Weaknesses Persist

Page 67 GAO-04-38 Results-Oriented Government

validation procedures, including proposals to review data collection and 

verification and validation procedures. 

All six agencies’ reports described data limitations, although discussions 

were mostly brief and very high level. One exception was DOT, which 

directed readers to the DOT Web site to obtain an assessment of the 

completeness and reliability of its performance data and detailed 

information on the source, scope, and limitations of the performance data. 

HUD and SBA also discussed plans for addressing the limitations. For 

example, HUD stated that to address problems with its indicator on the 

number of homeowners who have been assisted with the Home Investment 

Partnership Program (HOME), HUD has established a team of managers, 

technical staff, and contractors to make a series of improvements to the 

Integrated Disbursement and Information System beginning in fiscal year 

2003 that should reduce the need to clean up the data. 

Each of the six agencies’ reports also discussed the procedures they used 

to verify and validate their performance data. However, these discussions 

ranged from the very general description of the DOE method (noted 

previously), to the very detailed discussions provided by DOT. DOT 

provides an on-line compendium that discusses the source and accuracy of 

its data. Furthermore, DOT’s 2002 report also describes strategies being 

undertaken to address the quality of its data. The report states that a DOT 

intermodal working group addressed data quality issues by developing 

departmental statistical standards and by updating source and accuracy 

statements for all of DOT’s data programs. The working group also worked 

to improve quality assurance procedures, evaluate sampling and 

nonsampling errors, and develop common definitions for data across 

modes.
Page 68 GAO-04-38 Results-Oriented Government

Chapter 4

Challenges to GPRA Implementation Persist Chapte4r

While a great deal of progress has been made in making federal agencies 

more results oriented, numerous challenges still exist to effective 

implementation of GPRA. The success of GPRA depends on the 

commitment of top leadership within agencies, OMB, and Congress. 

However, according to federal managers surveyed, top leadership 

commitment to achieving results has not grown significantly since our 1997 

survey. Furthermore, although OMB has recently shown an increased 

commitment to management issues, it significantly reduced its guidance to 

agencies on GPRA implementation compared to prior years, and it is not 

clear how the program goals developed through its PART initiative will 

complement and integrate with the long-term, strategic focus of GPRA. 

Obtaining leadership commitment to implement a strategic plan depends in 

part on the usefulness and relevance of agency goals and strategies to 

agency leaders, Congress, and OMB. However, GPRA’s requirement to 

update agency strategic plans every 3 years is out of sync with presidential 

and congressional terms and can result in updated plans that do not have 

the support of top administration leadership and key congressional 

stakeholders. 

As noted in chapter 2, more federal managers surveyed reported having 

results-oriented performance measures for their programs and we would 

expect to have seen similar increases in the use of this information for 

program management. However, we did not observe any growth in their 

reported use of this information for key management activities, such as 

adopting new program approaches or changing work processes. 

Additionally, managers noted human capital-related challenges that impede 

results-oriented management, including a lack of authority and training to 

carry out GPRA requirements, as well as a lack of recognition for the 

results achieved.

Consistent with our previous work, federal managers in our focus groups 

reported that significant challenges persist in setting outcome-oriented 

goals, measuring performance, and collecting useful data. However, our 

survey data suggested that federal managers do not perceive issues, such 

as “difficulty distinguishing between the results produced by the program 

and results caused by other factors” and “difficulty obtaining data in time to 

be useful,” to be substantial hindrances to measuring performance or using 

performance information. 

Additionally, mission fragmentation and overlap contribute to difficulties in 

addressing crosscutting issues, particularly when those issues require a 

national focus, such as homeland security, drug control, and the 
Chapter 4

Challenges to GPRA Implementation Persist

Page 69 GAO-04-38 Results-Oriented Government

environment. GAO has previously reported on a variety of barriers to 

interagency cooperation, such as conflicting agency missions, jurisdiction 

issues, and incompatible procedures, data, and processes. We have also 

reported that OMB could use the provision of GPRA that calls for OMB to 

develop a governmentwide performance plan to integrate expected agencylevel performance. Unfortunately, this provision has not been fully 

implemented and the federal government lacks a tool, such as a strategic 

plan, that could provide a framework for a governmentwide reexamination 

of existing programs, as well as proposals for new programs. Finally, 

federal managers in our focus groups and political appointees we 

interviewed believed that Congress does not use performance information 

to the fullest extent to conduct oversight and to inform appropriations 

decisions. While there is concern regarding Congress’ use of performance 

information, it is important to make sure that this information is initially 

useful. As a key user of performance information, Congress needs to be 

considered a partner in shaping agency goals at the outset. GPRA provides 

Congress opportunities to influence agency performance goals through the 

consultation requirement for strategic plans and through Congress’ 

traditional oversight role.

Top Leadership Does 

Not Consistently Show 

Commitment to 

Achieving Results

We have previously testified that perhaps the single most important 

element of successful management improvement initiatives is the 

demonstrated commitment of top leaders to change.1

 This commitment is 

most prominently shown through the personal involvement of top leaders 

in developing and directing reform efforts. Organizations that successfully 

address their long-standing management weaknesses do not “staff out” 

responsibility for leading change. Top leadership involvement and clear 

lines of accountability for making management improvements are critical 

to overcoming organizations’ natural resistance to change, marshalling the 

resources needed in many cases to improve management, and building and 

maintaining the organizationwide commitment to new ways of doing 

business.

Results from our surveys show that while the majority of managers 

continue to indicate top leadership demonstrates a strong commitment to 

achieving results, we have not seen a noteworthy improvement in the 

percentage of managers expressing this view. From our 1997 survey, we 

1

U.S. General Accounting Office, Management Reform: Elements of Successful 

Improvement Initiatives, GAO/T-GGD-00-26 (Washington, D.C.: Oct. 15, 1999).
Chapter 4

Challenges to GPRA Implementation Persist

Page 70 GAO-04-38 Results-Oriented Government

estimated about 57 percent of managers overall reported such commitment 

to a great or very great extent. On our 2003 survey, 62 percent of managers 

expressed a comparable view—a higher but not statistically significant 

increase. (See fig. 10.) 

Figure 10: Percentage of Federal Managers Who Reported to a Great or Very Great 

Extent Their Top Leadership Has a Strong Commitment to Achieving Results

As shown in figure 11, however, we continued to see a significant 

difference between the perceptions of SES and non-SES managers on this 

issue. That is, the percentage of SES managers reporting that top 

leadership demonstrated strong commitment to a great or very great extent 

in 2003 was 22 percent higher than for non-SES managers. 

57 53

62

0

10

20

30

40

50

60

70

80

90

100

Percent

Year

1997 2000 2003

Source: GAO.
Chapter 4

Challenges to GPRA Implementation Persist

Page 71 GAO-04-38 Results-Oriented Government

Figure 11: Percentage of SES and Non-SES Managers Who Reported to a Great or 

Very Great Extent Their Agency Top Leadership Demonstrated Strong Commitment 

to Achieving Results

a

There was a statistically significant difference between SES and non-SES.

We observed in our 1997 and 2000 reports on governmentwide 

implementation of GPRA that we would expect to see managers’ positive 

perceptions on items, such as the extent to which top leadership is 

committed to achieving results, become more prevalent and the gap 

between SES and non-SES managers begin to narrow as GPRA and related 

reforms are implemented; however, these changes do not appear to be 

happening as expected.2

Demonstrating the willingness and ability to make decisions and manage 

programs based on results and the ability to inspire others to embrace such 

2

U.S. General Accounting Office, The Government Performance and Results Act: 1997 

Governmentwide Implementation Will Be Uneven, GAO/GGD-97-109 (Washington, D.C.: 

June 2, 1997) and Managing for Results: Federal Managers’ Views Show Need for 

Ensuring Top Leadership Skills, GAO-01-127 (Washington, D.C.: Oct. 20, 2000).

72 78

82

0

10

20

30

40

50

60

70

80

90

100

Percent

Year

1997a

SES

2000a 2003a

Source: GAO.

56 52

60

Non-SES
Chapter 4

Challenges to GPRA Implementation Persist

Page 72 GAO-04-38 Results-Oriented Government

a model are important indicators of leadership commitment to resultsoriented management. However, in both our 1997 and 2000 surveys, only 

about 16 percent of managers reported that changes by management above 

their levels to the programs for which they were responsible were based on 

results or outcome-oriented performance information to a great or very 

great extent. In our 2003 survey, this indicator increased to 23 percent, a 

statistically significant increase from prior surveys. Twenty-eight percent 

of federal managers surveyed who expressed an opinion reported that the 

lack of ongoing top executive commitment or support for using 

performance information to make program/funding decisions hindered 

measuring performance or using performance information to a great or 

very great extent.

Our interviews with 10 top political appointees from the Clinton and 

current Bush administrations indicated a high level of support and 

enthusiasm for effectively implementing the principles embodied in GPRA. 

For example, one appointee noted that GPRA focused senior management 

on a set of goals and objectives to allow the organization to understand 

what is important and how to deal with accomplishment at a macro-level, 

as well as provided a structure for problem solving. Another political 

appointee noted that GPRA has made it important to look at what you get 

out of the budget, not just what you put into it, while another concluded 

that GPRA brought about a fundamental rethinking of how they managed 

their programs and processes. Such indications of support for GPRA are 

promising. However, to support the transition to more results-oriented 

agency cultures, top agency management will need to make a more 

concerted effort to translate their enthusiasm for GPRA into actions that 

communicate to employees that top management cares about performance 

results and uses the information in its decision making.

The need for strong, committed leadership extends to OMB as well. OMB 

has shown a commitment to improving the management of federal 

programs, both through its leadership in reviewing agency program 

performance using the PART tool as well as through the PMA, which calls 

for improved financial performance, strategic human capital management, 

competitive sourcing, expanded electronic government, and performance 

budget integration. Using the foundation of information generated by 

agencies in their strategic plans, annual performance plans, and program 

performance reports, OMB has used the PART tool to exercise oversight of 

selected federal programs by assessing program purpose and design, the 

quality of strategic planning, the quality of program management, and the 

extent to which programs can demonstrate results. PART provides OMB a 
Chapter 4

Challenges to GPRA Implementation Persist

Page 73 GAO-04-38 Results-Oriented Government

lens through which to view performance information for use in the budget 

formulation process. PART, and OMB’s use of performance data in the 

budget formulation process, potentially can complement GPRA’s focus on 

increasing the supply of credible performance information by promoting 

the demand for this information in the budget formulation process. As we 

reported in chapter 2, more federal managers noted that OMB was paying 

attention to their agencies’ efforts under GPRA. (See fig. 6.) Additionally, 

OMB convened a performance measurement workshop in April 2003 to 

identify practical strategies for addressing common performance 

measurement challenges. As a result of this workshop, it produced a paper 

in June 2003 that included basic performance measurement definitions and 

concepts and common performance measurement problems that were 

discussed at the workshop. This was part of OMB’s continuing efforts to 

improve PART as an evaluation tool.

However, there are areas where OMB could further enhance its leadership. 

OMB has stated that the PART exercise presents an opportunity to inform 

and improve on agency GPRA plans and reports and establish a 

meaningful, systematic link between GPRA and the budget process. OMB 

has instructed agencies that, in lieu of a performance plan, they are to 

submit a performance budget that includes information from the PART 

assessments, including all performance goals used in the assessment of 

program performance done under the PART process. The result is that 

program-specific performance measures developed through the PART 

review are to substitute for other measures developed by the agency 

through its strategic planning process. GPRA is a broad legislative 

framework that was designed to be consultative with Congress and other 

stakeholders and address the needs of many users of performance 

information—Congress to provide oversight and inform funding decisions, 

agency managers to manage programs and make internal resource 

decisions, and the public to provide greater accountability. Changing 

agency plans and reports for use in the budget formulation process may not 

satisfy the needs of these other users. Users other than OMB are not likely 

to find the information useful unless it is credible and valid for their 

purposes. PART’s program-specific focus may fit with OMB’s agency-byagency budget reviews, but it is not well suited to achieving one of the key 

purposes of strategic plans—to convey agencywide, long-term goals and 

objectives for all major functions and operations. PART’s focus on 

program-specific measures does not substitute for the strategic, long-term 

focus of GPRA on thematic goals and department- and governmentwide 

crosscutting comparisons.
Chapter 4

Challenges to GPRA Implementation Persist

Page 74 GAO-04-38 Results-Oriented Government

To reach the full potential of performance management, agency planning 

and reporting documents need to reflect the full array of uses of 

performance information, which may extend beyond those needed for 

formulating the President’s Budget. However, it is not yet clear whether the 

results of those reviews, such as changes to agencies’ program 

performance measures, will complement and be integrated with the longterm, strategic goals and objectives agencies have established in 

consultation with Congress and other stakeholders under GPRA. OMB has 

not yet clearly articulated how PART is to complement GPRA. Focus group 

participants suggested that the administration and OMB needed to 

reinforce GPRA’s usefulness as a management tool for agencies. They also 

emphasized the need for OMB to help agencies understand how to 

integrate GPRA with other management initiatives, such as PART.

As we noted in chapter 3, agencies’ plans and reports still suffer from 

persistent weaknesses and could improve in a number of areas, such as 

attention to issues that cut across agency lines, and better information 

about the quality of the data that underlie agency performance goals. 

However, OMB’s July 2003 guidance for the preparation and submission of 

strategic plans, annual performance plans, and annual performance reports 

is significantly shorter and less detailed than its 2002 guidance. For 

example, OMB no longer provides detailed guidance to agencies for the 

development of performance plan components. OMB’s 2002 guidance on 

the preparation and submission of annual performance plans is 

approximately 39 pages long; in its 2003 guidance, that discussion spans 

only 2 pages. The 2003 guidance in this area does not include entire 

sections found in the 2002 guidance, such as principles for choosing 

performance goals and indicators for inclusion in the annual plan, types of 

performance goals, crosscutting programs, and requirements for verifying 

and validating data.

OMB needs to maintain and strengthen its leadership role in working with 

agencies to help them produce the highest quality GPRA documents 

through its formal guidance and reviews of strategic plan and report 

submissions. Focus group participants discussed the need for consistent 

guidance on how to implement GPRA. Furthermore, there is no evidence 

that agencies have institutional knowledge of GPRA requirements that 

would obviate the need for OMB’s guidance. New managers will need a 

consistent resource that provides practical guidance on what agencies 

need to include in their planning and reporting documents to comply with 

GPRA and reflect best practices. Consistent, explicit OMB guidance on 

preparing GPRA documents can help ensure that gains in the quality of 
Chapter 4

Challenges to GPRA Implementation Persist

Page 75 GAO-04-38 Results-Oriented Government

GPRA documents are maintained and provide a resource for agencies to 

make further improvements in those documents. For example, guidance 

on how to discuss coordination of crosscutting programs or improvements 

to the credibility of performance data in agency performance plans goes 

hand-in-hand with OMB’s enhanced oversight of agency performance 

through the PART exercise.

The success of GPRA depends on the commitment of top leadership within 

agencies, OMB, and Congress. Obtaining such leadership commitment 

depends in part on the usefulness and relevance of agency goals and 

strategies to these parties. GPRA requires an agency to develop a strategic 

plan at least every 3 years to cover the following 5-year period. Thus, there 

have been two required updates of strategic plans since the initial strategic 

plans were submitted for fiscal year 1997—fiscal year 2000 and fiscal year 

2003. The fiscal year 2000 update occurred the year before a new 

presidential term began. According to our focus group participants—both 

the experts and federal managers—it makes little sense to require an 

update of a strategic plan shortly before a new administration is scheduled 

to take office. For example, changes in political leadership generally result 

in a new agenda with new objectives. Such changes force agencies to 

revise their plans, management initiatives, and strategies, which translates 

into additional GPRA-related work. A strategic plan that does not reflect 

the participation and buy-in of top administration leadership and key 

congressional stakeholders is unlikely to be successfully implemented. 

Therefore, GPRA’s requirement to update agency strategic plans according 

to a schedule that is out of sync with presidential and congressional terms 

means that effort may be wasted on plans that lack the support of top 

leadership.

Managers Report 

Mixed Results in Use of 

Performance 

Information

GPRA’s usefulness to agency leaders and managers as a tool for 

management and accountability was cited as a key accomplishment 

numerous times by focus group participants. However, a number of 

alternative views indicated use of performance information for key 

management decisions has been mixed. For example, one participant said 

they did not believe GPRA has been used as a tool yet, while another 

participant commented that only better managers take advantage of GPRA 

as a management tool. According to focus group participants, although 

many federal managers understand and use results-oriented management 

concepts in their day-to-day activities, such as strategic planning and 

performance measurement, they do not always connect these concepts to 

the requirements of GPRA.
Chapter 4

Challenges to GPRA Implementation Persist

Page 76 GAO-04-38 Results-Oriented Government

This view was strongly supported by our survey results. Prior to 

mentioning GPRA in our survey, we asked federal managers the extent to 

which they consider their agency’s strategic goals when engaging in key 

management tasks such as setting program activities, allocating resources, 

or considering changes in their programs. A relatively high percentage of 

managers—ranging from 66 to 79 percent—responded to a great or very 

great extent. However, when we asked similar questions about the extent 

to which they considered their agency’s annual performance goals as set 

forth in the agency’s GPRA annual performance plan for the same 

activities, the comparable responses were considerably lower, ranging 

from 22 to 27 percent.

Because the benefit of collecting performance information is only fully 

realized when this information is actually used by managers, we asked 

them about the extent to which they used the information obtained from 

measuring performance for various program management activities. As 

shown in figure 12, for seven of the nine activities we asked about, the 

majority of managers who expressed an opinion reported using 

performance information to a great or very great extent in 2003. Across all 

nine activities, the percentage of managers saying they used performance 

information to a great or very great extent ranged from 41 percent for 

developing and managing contracts to 60 percent for allocating resources, 

setting individual job expectations, and rewarding staff. While we had 

observed a decline in the reported use of performance information to this 

extent for many of these activities between 1997 and 2000, our 2003 results 

increased to levels not significantly different from 1997 for all but one 

category—adopting new program approaches or changing work processes. 

This category of use continued to be significantly lower at 56 percent in 

2003 than it was in 1997 at 66 percent. Although another category, 

coordinating program efforts with other internal or external organizations, 

shows a similar pattern of limited recovery, the difference between the 

1997 and 2003 results is not statistically significant.
Chapter 4

Challenges to GPRA Implementation Persist

Page 77 GAO-04-38 Results-Oriented Government

Figure 12: Percentage of Federal Managers Who Reported Using Information Obtained from Performance Measurement to a 

Great or Very Great Extent for Various Management Activities

Note: Percentages are based on those respondents answering on the extent scale.

a

There was a statistically significant difference between the 1997 and 2003 surveys.

b

This question was not asked in 1997.

0

10

20

30

40

50

62

66

56

59 58

53 53 53

60 60 60

38

41

N/Ab

66

43

51

56 52 51 51 51

49

57

61

44

60

70

80

90

100

Percent

Setting program

priorities

Allocating

resources

Adopting new program

approaches or changing

work processesa

Coordinating program efforts 

with other internal or external

organizations

Refining program

performance measures

Rewarding staff I manage

or supervise

Developing and

managing contracts

Setting new or revising

existing performance goals

Setting individual job

expectations for my staff

58

Source: GAO.

Key management activities

1997

2000

2003
Chapter 4

Challenges to GPRA Implementation Persist

Page 78 GAO-04-38 Results-Oriented Government

We have reported that involving program managers in the development of 

performance goals and measures is critical to increasing the relevance and 

usefulness of this information to their day-to-day activities.3

 Yet, our survey 

data indicate that participation in activities related to the development and 

use of performance information has also been mixed. In 2003, only 14 

percent of managers believed to a great or very great extent that their 

agencies considered their contributions to or comments on their agency’s 

GPRA plans or reports. However, significantly more SES managers (43 

percent) than non-SES managers (12 percent) expressed this view. Also, 

when compared to our 2000 survey when we first asked this question, the 

percentage of SES managers expressing this view in 2003 was significantly 

higher than in 2000 (32 percent). The percentage of non-SES managers was 

essentially unchanged from 2000 (10 percent).

Furthermore, as shown in figure 13, overall around half or fewer of 

managers responded “yes” on our 2003 survey to questions about being 

involved in developing ways to measure whether program performance 

goals are being achieved (46 percent), gathering and analyzing data to 

measure whether programs were meeting their specific performance goals 

(51 percent), or using measures for program performance goals to 

determine if the agency’s strategic goals were being achieved (43 percent). 

None of these overall results were significantly different from our 1997 

results. We did find, however, that significantly more SES managers 

responded “yes” on the 2003 survey (72 percent) than the 1997 survey (55 

percent) with regard to being involved in using performance measurement 

information to determine if the agency’s strategic goals were being 

achieved when compared to our 1997 results.

3

GAO/GGD-97-109 and GAO-01-127.
Chapter 4

Challenges to GPRA Implementation Persist

Page 79 GAO-04-38 Results-Oriented Government

Figure 13: Percentage of Federal Managers Responding “Yes” about Being Involved 

in the Following Activities

Managers Continue to 

Confront a Range of 

Human Capital 

Management 

Challenges

Managing people strategically and maintaining a highly skilled and 

energized workforce that is empowered to focus on results are critically 

important. Such human capital management practices are essential to the 

success of the federal government in the 21st century and to maximizing 

the value of its greatest asset—its people. Our survey results showed 

continuing challenges related to the adequacy of managerial decision 

making authority, training, and incentives.

55

51

0

10

20

30

40

50

60

70

80

90

100

Percent

Year

1997

Developing ways to measure whether program performance goals are being achieved

2003

49

46

35

43

Gathering and analyzing data to measure whether programs are meeting their specific 

performance goals

Source: GAO.

Using measurements for program performance goals to determine if the agency's strategic 

goals are being achieved
Chapter 4

Challenges to GPRA Implementation Persist

Page 80 GAO-04-38 Results-Oriented Government

Federal Managers Report 

That They Are Held 

Accountable for Program 

Results but Do Not Have the 

Decision-Making Authority 

They Need to Accomplish 

Agency Goals

High-performing organizations seek to shift the focus of management and 

accountability from activities and processes to contributions and achieving 

results. In each of our three surveys, we asked managers about the amount 

of decision-making authority they had and the degree to which they were 

held accountable for results.

As shown in figure 14, for 2003, an estimated 40 percent of federal 

managers overall reported that they had the decision-making authority they 

needed to help the agency accomplish its strategic goals to a great or very 

great extent. This was a statistically significant increase over our 1997 

estimate of 31 percent. While there were more SES and non-SES managers 

expressing this view on our 2003 survey than the 1997 survey, it was the 

non-SES managers that showed the significant increase. Despite this 

promising trend, however, there continued to be substantial differences in 

2003, as well as on the two previous surveys, between the responses of SES 

and lower-level managers on this question. Compared to the 57 percent of 

SES managers who reported having such authority to a great or very great 

extent in 2003, only 38 percent of non-SES managers reported having such 

authority to a great or very great extent.
Chapter 4

Challenges to GPRA Implementation Persist

Page 81 GAO-04-38 Results-Oriented Government

Figure 14: Percentage of Federal Managers Reporting to a Great or Very Great 

Extent That Managers/Supervisors at Their Levels Had the Decision-Making 

Authority They Needed to Help the Agency Accomplish Its Strategic Goals

a

There was a statistically significant difference between the 1997 and 2003 surveys.

b

There was a statistically significant difference between SES compared to non-SES for each survey.

However, when asked the extent to which managers or supervisors at their 

levels were held accountable for the accomplishment of agency strategic 

goals, 57 percent responded to a great or very great extent in 2003. Unlike 

in other areas, where SES managers had significantly different views from 

non-SES managers, there was little difference in the area of accountability. 

(See fig. 15.)

51

56 57

0

10

20

30

40

50

60

70

80

90

100

Percent

Year

1997

Federal managersa

2000 2003

31

36

40

SESb

Source: GAO.

Non-SESa,b

29

34

38
Chapter 4

Challenges to GPRA Implementation Persist

Page 82 GAO-04-38 Results-Oriented Government

Figure 15: Percentage of Federal Managers, SES, and Non-SES in 2003 Reporting to 

a Great or Very Great Extent That They Were Held Accountable for the 

Accomplishment of Agency Strategic Goals

This 57 percent is significantly higher than the 40 percent of managers 

overall who indicated that they had comparable decision-making authority. 

However, in contrast to the question on authority, as shown in figure 14, 

where more SES managers than non-SES managers expressed the view that 

they had the authority, there was little difference, as shown in figure 15, 

between the two groups in their views about being held accountable for 

achieving agency strategic goals to a great or very great extent. As figures 

14 and 15 further illustrate, roughly the same percentage of SES managers 

perceived to a great or very great extent that managers at their level had 

decision-making authority and accountability for achieving agency 

strategic goals. This result suggests that their authority was perceived to 

be on par with their accountability. In contrast, only 38 percent of non-SES 

managers perceived that managers at their levels had the decision-making 

authority they needed to a great or very great extent, while 57 percent 

perceived that they were held accountable to a comparable extent.

Managers are hard-pressed to achieve results when they do not have 

sufficient authority to act. In our report containing the results of our 1997 

survey, we noted that agencies needed to concentrate their efforts on areas 

57 57

61

0

10

20

30

40

50

60

70

80

90

100

Percent

Federal

managers

Non-SES SES

Source: GAO.
Chapter 4

Challenges to GPRA Implementation Persist

Page 83 GAO-04-38 Results-Oriented Government

where managers were not perceiving or experiencing progress, such as that 

concerning devolving decision-making authority to managers throughout 

their organizations. While authority for achieving results appears to be in a 

modestly upward trend, the balance between authority and accountability 

that fosters decision making to achieve results could be further improved, 

particularly among non-SES managers.

Fewer Than Half of 

Managers Reported Training 

on Key Tasks

We previously reported on the need for agencies to expend resources on 

effective training and professional development to equip federal employees 

to work effectively.4

 Among the resources focus group participants cited as 

lacking included federal managers and staff with competencies and skills 

needed to plan strategically, develop robust measures of performance, and 

analyze what the performance data mean. Our 2003 Guide calls for training 

and development efforts to be strategically focused on improving 

performance toward the agency’s goals and put forward with the agency’s 

organizational culture firmly in mind.5

 Throughout this process it is 

important that top leaders in the agencies communicate that investments in 

training and development are expected to produce clearly identified 

results. By incorporating valid measures of effectiveness into the training 

and development programs they offer, agencies can better ensure that they 

will adequately address training objectives and thereby increase the 

likelihood that desired changes will occur in the target population’s skills, 

knowledge, abilities, attitudes, or behaviors. Furthermore, if managers 

understand and support the objectives of training and development efforts, 

they can provide opportunities to successfully use the new skills and 

competencies on the job and model the behavior they expect to see in their 

employees.

In response to our 2003 survey, fewer than half of managers answered “yes” 

when we asked them whether, during the past 3 years, their agencies had 

provided, arranged, or paid for training that would help them accomplish 

any of seven critical results-oriented management-related tasks. However, 

progress is indicated in our survey results. As shown in figure 16, more 

managers answered “yes” in 2003 on all seven training areas than in 

4

GAO-01-127.

5

U.S. General Accounting Office, Human Capital: A Guide for Assessing Strategic 

Training and Development Efforts in the Federal Government (Exposure Draft) GAO-03-

893G (Washington, D.C.: July 1, 2003).
Chapter 4

Challenges to GPRA Implementation Persist

Page 84 GAO-04-38 Results-Oriented Government

previous surveys. These increases were statistically significant for five of 

the tasks—setting program performance goals, using program performance 

information to make management decisions, linking program performance 

to the achievement of agency strategic goals, and implementing the 

requirements of GPRA. 

Figure 16: Percentage of Federal Managers in Each Survey Year Who Reported That during the Past 3 Years Their Agencies 

Provided, Arranged, or Paid for Training That Would Help Them Accomplish Specific Tasks

a

This question was not asked in the 1997 survey.

b

There was a statistically significant difference between the 2000 and 2003 surveys.

c

There was a statistically significant difference between the 1997 and 2003 surveys.

As with our 2000 survey results, the 2003 survey results continued to 

demonstrate that there is a positive relationship between agencies 

providing training and development on setting program performance goals 

and the use of performance information when setting or revising 

0

10

N/Aa

20

30

40

27

35

43

38

36

47

45

39

32

27

19

41

33

28

41

33 31

49

44

35

50 Percent

Set program

performance

goalsc

Use program

performance 

information to 

make decisionsc

Link the

performance of

programs/operations/

projects to the

achievement of agency 

strategic goalsc

Implement the

requirements

of GPRAc

Conduct

strategic

planning

Develop program

performance 

measures

Assess the quality 

of performance 

datab

Source: GAO.

1997

2000

2003
Chapter 4

Challenges to GPRA Implementation Persist

Page 85 GAO-04-38 Results-Oriented Government

performance goals. For those managers who responded “yes” to training 

on setting performance goals, 60 percent also reported that they used 

information obtained from performance measurement when setting new or 

revising existing performance goals to a great or very great extent. In 

contrast, for those managers who responded “no” to training on setting 

performance goals, only 38 percent reported that they used information 

obtained from performance measurement for setting new or revising 

existing performance goals to a great or very great extent. The difference 

between these percentages is statistically significant. Effective training 

and development programs are an integral part of a learning environment 

that can enhance the federal government’s ability to attract and retain 

employees with the skills and competencies needed to achieve results. 

Training and developing new and current staff to fill new roles and work in 

different ways will be a crucial part of the federal government’s endeavors 

to meet its transformation challenges. Ways that employees learn and 

achieve results will also continue to transform how agencies do business 

and engage employees in further innovation and improvements.

Managers Perceive a Lack of 

Positive Recognition for 

Helping Agencies Achieve 

Results

Another fundamental aspect of the human capital management challenge 

agencies face is providing the incentives to their employees to encourage 

results-oriented management. Monetary and nonmonetary incentives can 

be used as a method for federal agencies to reward employees and to 

motivate them to focus on results.

Overall, an increasing but still small percentage of managers reported in 

1997, 2000, and 2003 that employees in their agencies received positive 

recognition to a great or very great extent for helping agencies accomplish 

their strategic goals. In 1997, 26 percent of federal managers reported such 

an extent of positive recognition as compared to 37 percent in 2003, a 

statistically significant increase. Interestingly, this improvement is seen in 

the responses of non-SES managers. As shown in figure 17, the percentage 

of SES managers expressing this view stayed at about the same level over 

the three surveys, while the percentage of non-SES managers holding this 

view was significantly higher in 2003 than in 1997. Even with this 

improvement on the part of the responses from non-SES managers, 

significantly more SES managers (47 percent) than non-SES managers (36 

percent) expressed this perception to a comparable extent in 2003.
Chapter 4

Challenges to GPRA Implementation Persist

Page 86 GAO-04-38 Results-Oriented Government

Figure 17: Percentage of Federal Managers Who Reported to a Great or Very Great 

Extent That Employees in Their Agencies Received Positive Recognition for Helping 

Their Agencies Accomplish Their Strategic Goals

a

There was a statistically significant difference between the 1997 and 2003 surveys.

Unfortunately, most existing federal performance appraisal systems are not 

designed to support a meaningful performance-based pay system in that 

they fail to link institutional, program, unit, and individual performance 

measurement and reward systems. In our view, one key need is to 

modernize performance management systems in executive agencies so that 

they link to the agency’s strategic plan, related goals, and desired outcomes 

and are therefore capable of adequately supporting more performancebased pay and other personnel decisions.

We have reported federal agencies can develop effective performance 

management systems by implementing a selected, generally consistent set 

of key practices. These key practices helped public sector organizations 

both in the United States and abroad create a clear linkage—“line of 

sight”—between individual performance and organizational success and, 

48

52

47

0

10

20

30

40

50

60

70

80

90

100

Percent

Year

1997

Federal managersa

2000 2003

26

31

37

SES

Source: GAO.

Non-SESa

25

29

36
Chapter 4

Challenges to GPRA Implementation Persist

Page 87 GAO-04-38 Results-Oriented Government

thus, transform their cultures to be more results oriented, customerfocused, and collaborative in nature. Examples of such practices include

• aligning individual performance expectations with organizational goals,

• connecting performance expectations to crosscutting goals,

• linking pay to individual and organizational performance, and

• making meaningful distinctions in performance.6

Beyond implementing these key practices, high-performing organizations 

understand that their employees are assets whose value to the organization 

must be recognized, understood, and enhanced. They view an effective 

performance management system as an investment to maximize the 

effectiveness of people by developing individual potential to contribute to 

organizational goals. To maximize this investment, an organization’s 

performance management system is designed, implemented, and 

continuously assessed by the standard of how well it helps the employees 

help the organization achieve results and pursue its mission.

6

For a complete list and discussion of the practices, see U.S. General Accounting Office, 

Results oriented Cultures: Creating a Clear Linkage between Individual Performance 

and Organizational Success, GAO-03-488 (Washington, D.C.: Mar. 14, 2003).
Chapter 4

Challenges to GPRA Implementation Persist

Page 88 GAO-04-38 Results-Oriented Government

Persistent Challenges 

in Setting OutcomeOriented Goals, 

Measuring 

Performance, and 

Collecting Useful Data

In prior reports, we have described difficulties faced by federal managers 

in developing useful, outcome-oriented measures of performance and 

collecting data indicating progress achieved.7

 One of the most persistent 

challenges has been the development of outcome-oriented performance 

measures. Additionally, it is difficult to distinguish the impact of a 

particular federal program from the impact of other programs and factors, 

thus making it difficult to attribute specific program performance to 

results. The lack of timely and useful performance information can also 

hinder GPRA implementation.

Meaningful, OutcomeOriented Performance 

Measures Are Sometimes 

Hard to Develop

In the past, we have noted that federal managers found meaningful 

performance measures difficult to develop. Focus group participants and 

survey respondents noted that outcome-oriented performance measures 

were especially difficult to establish when the program or line of effort was 

not easily quantifiable. The challenge of the “complexity of establishing 

outcome-oriented goals and measuring performance” was cited by six of 

the eight focus groups as one of the key challenges that managers face in 

implementing GPRA. Focus group participants agreed that they often felt 

as if they were trying to measure the immeasurable, not having a clear 

understanding of which performance indicators could accurately inform 

the agency how it is carrying out a specific activity. Managers from 

agencies engaged in basic science research and development and grantmaking functions noted that this effort was particularly difficult for them 

because federal programs, especially those that are research-based, often 

take years to achieve the full scope of their goals. On our most recent 

survey, we estimated that 36 percent of federal managers who had an 

opinion indicated that the determination of meaningful measures hindered 

the use of performance information or performance measurement to a 

great or very great extent. While this number was significantly lower than 

the percentage of managers expressing the comparable view on the 1997 or 

2000 survey and may reflect some lessening of this as a hindrance to some 

7

See for example, U.S. General Accounting Office, Managing for Results: Analytic 

Challenges in Measuring Performance, GAO/HEHS/GGD-97-138 (Washington, D.C.: May 30, 

1997); Program Evaluation: Agencies Challenged by New Demand for Information on 

Program Results, GAO/GGD-98-53 (Washington, D.C.: Apr. 24, 1998); Managing for Results: 

Measuring Program Results That Are Under Limited Federal Control, GAO/GGD-99-16

(Washington, D.C.: Dec. 11, 1998); and Managing for Results: Challenges Agencies Face in 

Producing Credible Performance Information, GAO/GGD-00-52 (Washington, D.C.: Feb. 4, 

2000).
Chapter 4

Challenges to GPRA Implementation Persist

Page 89 GAO-04-38 Results-Oriented Government

managers, it nonetheless continues to be among those items having the 

largest percentage of managers citing it as a substantial hindrance.

Impact of Federal Programs 

Difficult to Discern

In our June 1997 report on GPRA, we noted that “the often limited or 

indirect influence that the federal government has in determining whether 

a desired result is achieved complicates the effort to identify and measure 

the discrete contribution of the federal initiative to a specific program 

result.”8

 This occurs primarily because many federal programs’ objectives 

are the result of complex systems or phenomena outside the program’s 

control. In such cases, it is particularly challenging for agencies to 

confidently attribute changes in outcomes to their program—the central 

task of program impact evaluation. This is particularly challenging for 

regulatory programs, scientific research programs, and programs that 

deliver services to taxpayers through third parties, such as state and local 

governments.

We have reported that determining the specific outcomes resulting from 

federal research and development has been a challenge that will not be 

easily resolved.9

 Due to the difficulties in identifying outcomes, research 

and development agencies typically have chosen to measure a variety of 

proxies for outcomes, such as the number of patents resulting from 

federally funded research, expert review and judgments of the quality and 

importance of research findings, the number of project-related publications 

or citations, and contributions to expanding the number of research 

scientists.

8

GAO/GGD-97-109, 6.

9

U.S. General Accounting Office, Managing for Results: Key Steps and Challenges in 

Implementing GPRA in Science Agencies, GAO/T-GGD/RCED-96-214 (Washington, D.C.: 

July 10, 1996).
Chapter 4

Challenges to GPRA Implementation Persist

Page 90 GAO-04-38 Results-Oriented Government

We have also reported that implementing GPRA in a regulatory 

environment is particularly challenging.10 Although federal agencies are 

generally required to assess the potential benefits and costs of proposed 

major regulatory actions, they generally do not monitor the benefits and 

costs of how these and other federal programs have actually performed. 

For example, in the case of the Environmental Protection Agency (EPA), to 

determine if existing environmental regulations need to be retained or 

improved, we previously recommended that EPA study the actual costs and 

benefits of such regulations.11

In the past, regulatory agencies have cited numerous barriers to their 

efforts to establish results-oriented goals and measures. These barriers 

included problems in obtaining data to demonstrate results, accounting for 

factors outside of the agency’s control that affect results, and dealing with 

the long time periods often needed to see results. Our prior work 

discussed best practices for addressing challenges to measuring the results 

of regulatory programs. In particular, to address the challenge of 

discerning the impact of a federal program, when other factors also affect 

results, we suggested agencies “establish a rationale of how the program 

delivers results.” Establishing such a rationale involves three related 

practices: (1) taking a holistic or “systems” approach to the problem being 

addressed, (2) building a program logic model that described how activities 

translated to outcomes, and (3) expanding program assessments and 

evaluations to validate the model linkages and rationale.

We have also reported on the difficulties encountered in meeting GPRA 

reporting requirements for intergovernmental grant programs.12 Programs 

that do not deliver a readily measurable product or service are likely to 

have difficulty meeting GPRA performance measurement and reporting 

requirements. Intergovernmental grant programs, particularly those with 

the flexibility inherent in classic block grant design, may be more likely to 

10U.S. General Accounting Office, Managing for Results: Strengthening Regulatory 

Agencies’ Performance Management Practices, GAO/GGD-00-10 (Washington, D.C.: Oct. 28, 

1999).

11U.S. General Accounting Office, Environmental Protection: Assessing the Impacts of 

EPA’s Regulations Through Retrospective Studies, GAO/RCED-99-250 (Washington, D.C.: 

Sept. 14, 1999).

12U.S. General Accounting Office, Grant Programs: Design Features Shape Flexibility, 

Accountability, and Performance Information, GAO/GGD-98-137 (Washington, D.C.: 

June 22, 1998).
Chapter 4

Challenges to GPRA Implementation Persist

Page 91 GAO-04-38 Results-Oriented Government

have difficulty producing performance measures at the national level and 

raise delicate issues of accountability. Although most flexible grant 

programs we reviewed reported simple activity or client counts, relatively 

few of them collected uniform data on the outcomes of state or local 

service activities. Collecting such data requires conditions (such as 

uniformity of activities, objectives, and measures) that do not exist under 

many flexible program designs, and even where overall performance of a 

state or local program can be measured, the amount attributable to federal 

funding often cannot be separated out.

Focus group participants also suggested that they faced challenges in 

obtaining timely performance data from relevant partner organizations and 

in identifying what the federal government’s contribution has been to a 

specific outcome. Furthermore, survey respondents provided some 

corroboration for these views. Across all three of our surveys, we estimate 

that roughly a quarter of all federal managers reported this difficulty—

distinguishing between the results produced by the program they were 

involved with and results caused by other factors—as a substantial 

hindrance. In response to a survey question about what the federal 

government could do to improve its overall focus on managing for results, 

one respondent noted: “Defining meaningful measures for the work we do 

is extremely difficult; and even if they could be defined, performance and 

accomplishment is (sic) dependent on so many factors outside our control 

that it is difficult, if not impossible, to make valid conclusions.”

Timely, Useful Performance 

Information Not Always 

Available

In February 2000, we reported that intergovernmental programs pose 

potential difficulties in collecting timely and consistent national data.13 We 

also noted that agencies had limited program evaluation capabilities and 

weaknesses in agencies’ financial management capabilities make it difficult 

for decision makers to effectively assess and improve many agencies’ 

financial performance. On the basis of our current findings, these issues 

still exist. Federal managers who participated in our focus groups cited 

difficulties in gathering data from state or local entities, as well as statutory 

limitations regarding the nature and breadth of data that they were 

permitted to collect. However, in our 2003 survey, only 27 percent of 

federal managers indicated that obtaining data in time to be useful was a 

13GAO/GGD-00-52.
Chapter 4

Challenges to GPRA Implementation Persist

Page 92 GAO-04-38 Results-Oriented Government

substantial hindrance; 31 percent expressed a comparable view with regard 

to obtaining valid or reliable data.

Focus group participants also noted that OMB’s accelerated time frames 

for reporting performance information will contribute to the challenge of 

producing complete, timely information in their agencies’ performance 

reports. Over the past 2 fiscal years, OMB has moved the deadline for 

submission of agencies’ performance reports (now performance and 

accountability reports) back from the statutory requirement of March 31; 

for fiscal year 2003 data, the deadline is January 30, 2004. In fiscal year 

2004, these reports will be due on November 15, 2004. According to the 

managers, individual agencies may work on different time frames based 

partially on the population they serve or the stakeholders they must work 

with, such as state or local agencies. This “one size fits all” approach does 

not take such differences into account.

Additionally, OMB requires agencies to report on their performance data 

quarterly; managers noted that this was particularly difficult for outcomes 

that may be achieved over extended periods of time, such as outcomes 

associated with basic science. As we have previously reported, measuring 

the performance of science-related projects can be difficult because a wide 

range of factors determine if and how a particular research and 

development project will result in a commercial application or have other 

benefits. Efforts to cure diseases or pursue space exploration are difficult 

to quantify and break down into meaningful quarterly performance 

measures.

Crosscutting Issues 

Hinder Successful 

GPRA Implementation

Crosscutting issues continue to be a challenge to GPRA implementation. 

Mission fragmentation and program overlap are widespread across the 

federal government. Moreover, addressing this challenge is essential to the 

success of national strategies in areas such as homeland security, drug 

control, and the environment.

We have reported that agencies could use the annual performance planning 

cycle and subsequent annual performance reports to highlight crosscutting 

program efforts and to provide evidence of the coordination of those 

efforts. Our review of six agencies’ strategic and annual performance plans 

showed some improvement in addressing their crosscutting program 

efforts, but a great deal of improvement is still necessary. Few of the plans 

we reviewed attempted the more challenging task of discussing planned 

strategies for coordination and establishing complementary performance 
Chapter 4

Challenges to GPRA Implementation Persist

Page 93 GAO-04-38 Results-Oriented Government

goals and complementary or common performance measures. For 

example, SSA’s 2004 performance plan makes some mention of the agency’s 

efforts to coordinate with other agencies to preserve the integrity of the 

Social Security number as a personal identifier, but there are very few 

details about this important component of its mission.

Previous GAO reports and agency managers identified several barriers to 

interagency coordination. First, missions may not be mutually reinforcing 

or may even conflict, making reaching a consensus on strategies and 

priorities difficult. In 1998 and 1999, we found that mission fragmentation 

and program overlap existed in 12 federal mission areas, ranging from 

agriculture to natural resources and the environment. Implementation of 

federal crosscutting programs is often characterized by numerous 

individual agency efforts that are implemented with little apparent regard 

for the presence of related activities. Second, we reported on agencies’ 

interest in protecting jurisdiction over missions and control over 

resources.14 Focus group participants echoed this concern, noting that 

there can be “turf battles” between agencies, where jurisdictional 

boundaries, as well as control over resources, are hotly contested. Finally, 

incompatible procedures, processes, data, and computer systems pose 

difficulties for agencies to work across agency boundaries. For example, 

we reported how the lack of consistent data on federal wetlands programs 

implemented by different agencies prevented the government from 

measuring progress toward achieving the governmentwide goal of no net 

loss of the nation’s wetlands.15

14U.S. General Accounting Office, Managing for Results: Barriers to Interagency 

Coordination, GAO/GGD-00-106 (Washington, D.C.: Mar. 29, 2000).

15U.S. General Accounting Office, Wetlands Overview: Problems With Acreage Data Persist, 

GAO/RCED-98-150 (Washington, D.C.: July 1, 1998) and Results-Oriented Management: 

Agency Crosscutting Actions and Plans in Border Control, Flood Mitigation and 

Insurance, Wetlands, and Wildland Fire Management, GAO-03-321 (Washington, D.C.: 

Dec. 20, 2002).
Chapter 4

Challenges to GPRA Implementation Persist

Page 94 GAO-04-38 Results-Oriented Government

We have previously reported and testified that GPRA could provide OMB, 

agencies, and Congress with a structured framework for addressing 

crosscutting program efforts.16 OMB, for example, could use the provision 

of GPRA that calls for OMB to develop a governmentwide performance 

plan to integrate expected agency-level performance. Unfortunately, this 

provision has not been fully implemented. OMB issued the first and only 

such plan in February 1998 for fiscal year 1999. In our review of the plan,17

we found that it included a broad range of governmentwide management 

objectives and a mission-based presentation of key performance goals 

based on agency performance plans and the plan’s framework should 

ultimately allow for a cohesive presentation of governmentwide 

performance. However, the specific contents of this initial plan did not 

always deliver an integrated, consistent, and results-oriented picture of 

fiscal year 1999 federal government performance goals.

OMB officials we interviewed at the time stressed that developing the 

governmentwide plan was viewed as an essential and integral component 

of the President’s budget and planning process. From OMB’s perspective, 

both the plan and the budget submission were intended to serve as 

communication tools for a range of possible users. In their opinion, the 

plan added value by reflecting a governmentwide perspective on policy 

choices made throughout the budget formulation process. OMB 

acknowledged that the plan itself did not serve to change the process 

through which decisions on government priorities were made, but 

enhanced it by placing a greater emphasis on results. As one official 

described it, the governmentwide performance plan was a derivative 

document, reflecting the budget and management decisions made 

throughout the process of formulating the President’s budget submission. 

However, we found that focusing broadly on governmentwide outcomes 

should be a central and distinguishing feature of the federal government 

performance plan. To be most effective and supportive of the purposes of 

GPRA, the governmentwide plan must be more than a compilation of 

agency-level plans; integration, rather than repetition, must be its guiding 

principle.

16GAO/GGD-00-106 and U.S. General Accounting Office, Results-Oriented Government: 

Using GPRA to Address 21st Century Challenges, GAO-03-1166T (Washington, D.C.: 

Sept. 18, 2003).

17U.S. General Accounting Office, The Results Act: Assessment of the Governmentwide 

Performance Plan for Fiscal Year 1999, GAO/AIMD/GGD-98-159 (Washington, D.C.: Sept. 8, 

1998).
Chapter 4

Challenges to GPRA Implementation Persist

Page 95 GAO-04-38 Results-Oriented Government

OMB has not issued a distinct governmentwide performance plan since 

fiscal year 1999. Most recently, the President’s fiscal year 2004 budget 

focused on describing agencies’ progress in addressing the PMA and the 

results of PART reviews of agency programs. Although such information is 

important and useful, it does not provide a broader and more integrated 

perspective of planned performance on governmentwide outcomes. 

Additionally, the fiscal year 2004 budget identified budget requests and 

performance objectives by agency, such as the U.S. Department of Defense, 

as opposed to crosscutting governmentwide themes. From this 

presentation, one could assume that the only activities the U.S. government 

planned to carry out in support of national defense were those listed under 

the chapter “Department of Defense.” However, the chapter of the fiscal 

year 2004 budget discussing “the Department of State and International 

Assistance Programs,” contains a heading titled, “Countering the Threat 

from Weapons of Mass Destruction.” And while OMB may have a technical 

reason for not classifying this task as being related to national defense or 

homeland security, it is unclear that a lay reader could make that 

distinction. The fiscal year 2005 budget also identified budget requests by 

agency, not by crosscutting theme. Without such a governmentwide focus, 

OMB is missing an opportunity to assess and communicate the relationship 

between individual agency goals and outcomes that cut across federal 

agencies and more clearly relate and address the contributions of 

alternative federal strategies. The governmentwide performance plan also 

could help Congress and the executive branch address critical federal 

performance and management issues, including redundancy and other 

inefficiencies in how we do business. It could also provide a framework for 

any restructuring efforts.

A strategic plan for the federal government, supported by key national 

indicators to assess the government’s performance, position, and progress, 

could provide an additional tool for governmentwide reexamination of 

existing programs, as well as proposals for new programs. If fully 

developed, a governmentwide strategic plan could potentially provide a 

cohesive perspective on the long-term goals of the federal government and 

provide a much needed basis for fully integrating, rather than merely 

coordinating, a wide array of federal activities. Successful strategic 

planning requires the involvement of key stakeholders. Thus, it could serve 

as a mechanism for building consensus. Further, it could provide a vehicle 

for the President to articulate long-term goals and a road map for achieving 

them. In addition, a strategic plan could provide a more comprehensive 

framework for considering organizational changes and making resource 

decisions.
Chapter 4

Challenges to GPRA Implementation Persist

Page 96 GAO-04-38 Results-Oriented Government

Developing a strategic plan for the federal government would be an 

important first step in articulating the role, goals, and objectives of the 

federal government. It could help provide critical horizontal and vertical 

linkages. Horizontally, it could integrate and foster synergies among 

components of the federal government as well as help to clarify the role of 

the federal government vis-à-vis other sectors of our society. Vertically, it 

could provide a framework of federal missions and goals within which 

individual federal agencies could align their own missions and goals that 

would cascade down to individual employees. The development of a set of 

key national indicators could be used as a basis to inform the development 

of the governmentwide strategic and annual performance plans. The 

indicators could also link to and provide information to support outcomeoriented goals and objectives in agency-level strategic and annual 

performance plans.

Managers View 

Congress’ Use of 

Performance 

Information as Limited

Focus group members believed that one of the main challenges to GPRA 

implementation was the reluctance of Congress to use that information 

when making decisions, especially appropriations decisions. This concern 

was cited as a significant challenge in each of the focus groups, and was 

one of the top three “challenges” in five of the eight focus groups. In some 

cases, managers in our focus groups noted that this lack of usage was a 

significant disincentive to doing a good job in preparing GPRA plans and 

reports. Agency managers made the following criticisms regarding the 

perceived lack of congressional use of performance information:

• appropriators have not bought into GPRA, so there is no incentive to do 

this well,

• failure of congressional leadership in developing and using performance 

measures,

• appropriators do not use performance data or tools to make decisions, 

and

• GPRA does not drive public policy decisions.

Results from our survey provide some further information in support of 

this view. On our 2003 survey, when we asked federal managers about the 

extent to which they thought congressional committees paid attention to 

agency efforts under GPRA, only 22 percent of federal managers responded 

in the great to very great categories. This result was not significantly 
Chapter 4

Challenges to GPRA Implementation Persist

Page 97 GAO-04-38 Results-Oriented Government

different from the results we observed on our 2000 survey when we asked 

this question about three specific types of congressional committees—

authorization, appropriation, and oversight. On the 2000 survey, only 18 

percent of federal managers held a similar view concerning authorizing 

committees, 19 percent for appropriations committees, and 20 percent for 

oversight committees. As we noted earlier, when this item was asked in 

relation to OMB, there was a significant increase in the percentage of 

managers responding to a great or very great extent from 2000 to 2003. The 

31 percent of managers who viewed OMB as paying attention to a great or 

very great extent in 2003 was significantly higher than the 22 percent 

holding a comparable view of congressional committees.

Although managers expressed these concerns about the use of this 

information, a recent review by the CRS suggested that Congress uses 

performance information to some extent, as evidenced by citations in 

legislation and committee reports.18 For example, in the 106thCongress 

(1999-2000), 42 public laws contained statutory language relating to GPRA 

and performance measures, and 118 legislative reports19 contained GPRAassociated passages. As shown in figure 18, across all three of our surveys, 

only a minority of federal managers governmentwide viewed the lack of 

ongoing congressional commitment for using performance information as a 

hindrance to a great or very great extent.

18Congressional Research Service, Government Performance and Results Act: Overview of 

Associated Provisions in the 106th Congress, (Washington, D.C.: 2002).

19This included reports that accompanied bills passed by both the House and Senate that 

were either enacted into law or vetoed by the President.
Chapter 4

Challenges to GPRA Implementation Persist

Page 98 GAO-04-38 Results-Oriented Government

Figure 18: Percentage of Federal Managers Reporting to a Great or Very Great 

Extent That a Lack of Ongoing Congressional Commitment or Support for Using 

Performance Information in Making Program/Funding Decisions Is a Hindrance

Note: Percentages are based on those respondents answering on the extent scale.

While there is concern regarding Congress’ use of performance 

information, it is important to make sure that this information is initially 

useful. One of GPRA’s purposes is to respond to a need for accurate, 

reliable information for congressional decision making. In 2000, we 

reported that congressional staffs stated that they were looking for 

recurring information on spending priorities within programs; the quality, 

quantity, and efficiency of program operations; the populations served or 

regulated; as well as programs’ progress in meeting their objectives.20 For 

example, learning who benefits from a program can help in addressing 

questions about how well services are targeted to those most in need. 

Some of these recurring needs were met through formal agency 

documents, such as annual performance plans. However, some 

information the agencies provided did not fully meet the congressional 

20U.S. General Accounting Office, Managing for Results: Views on Ensuring the Usefulness 

of Agency Performance Information to Congress, GGD-00-35 (Washington, D.C.: Jan. 26, 

2000).

33 34 31

0

10

20

30

40

50

60

70

80

90

100

Percent

Year

1997 2000 2003

Source: GAO.
Chapter 4

Challenges to GPRA Implementation Persist

Page 99 GAO-04-38 Results-Oriented Government

staffs’ needs because the presentation was not clear, directly relevant, or 

sufficiently detailed. For example, congressional staffs wanted to see more 

direct linkages among the agencies’ resources, strategies, and goals. In 

other cases, the information was not readily available to the congressional 

staffs, either because it had not been requested or reported, or because 

staff were not informed that it was available.

As a key user of performance information, Congress also needs to be 

considered a partner in shaping agency goals at the outset. For example, 

through the strategic planning requirement, GPRA requires federal 

agencies to consult with Congress and key stakeholders to reassess their 

missions and long-term goals as well as the strategies and resources they 

will need to achieve their goals. GPRA also provides a vehicle for Congress 

to explicitly state its performance expectations in outcome-oriented terms 

when establishing new programs or in exercising oversight of existing 

programs that are not achieving desired results. Congress could use 

authorizing and appropriations hearings to determine if agency programs 

have clear performance goals, measures, and data with which to track 

progress and whether the programs are achieving their goals. If goals and 

objectives are unclear or not results oriented, Congress could use 

legislation to articulate the program outcomes it expects agencies to 

achieve. This would provide important guidance to agencies that could 

then be incorporated in agency strategic and annual performance plans.
Page 100 GAO-04-38 Results-Oriented Government

Chapter 5

Conclusions and Recommendations Chapte5r

Agenda for Achieving a 

Sustainable, 

Governmentwide 

Focus on Results

As we have shown in this report, in the 10 years since the enactment of 

GPRA, significant progress has been made in instilling a focus on results in 

the federal government. First, GPRA statutory requirements laid a 

foundation for results-oriented management in federal agencies. Expert 

and agency focus group participants cited the creation of this statutory 

foundation as one of the key accomplishments of GPRA. Since GPRA 

began to be implemented governmentwide in fiscal year 1997, we have 

observed significant increases in the percentage of federal managers who 

reported having results-oriented performance measures for their programs. 

Focus group participants’ views on whether GPRA has had a positive effect 

on the federal government’s ability to deliver results to the American public 

were mixed. For example, the information gathered and reported for 

GPRA allows agencies to make better-informed decisions, which improves 

their ability to achieve results. In addition, GPRA has made the results of 

federal programs more transparent to the public. Other participants stated 

that while certain aspects of GPRA-related work have been positive, 

agencies’ ability to deliver results and public awareness of their activities 

cannot be exclusively attributed to GPRA.

Second, GPRA has increased the connection between resources and results 

by creating more formal linkages between agency performance goals and 

objectives and the program activities in the budget. Over the first 4 years of 

agency efforts to implement GPRA, we observed that agencies continued to 

tighten the required linkage between their performance plans and budget 

requests. However, much remains to be done in this area. For example, we 

have not observed notable increases in federal managers’ perceptions 

about their personal use of plans or performance information when 

allocating resources, or about the use of performance information when 

funding decisions are made about their programs. However, it should be 

noted that we estimate a majority have positive perceptions about the use 

of performance information to allocate resources.

Third, GPRA has provided a foundation for examining agency missions, 

performance goals and objectives, and the results achieved. We have seen 

improvements in the quality of agency strategic plans, annual performance 

plans, and performance reports since initial efforts. However, few of the 

six agencies we reviewed in this report produced GPRA planning and 

reporting documents that met all of our criteria for the highest level of 

quality. Most of these agencies continued to miss opportunities to present 

clear pictures of their intended and actual performance results in their 

GPRA plans and reports and to show how resources are aligned with actual 
Chapter 5

Conclusions and Recommendations

Page 101 GAO-04-38 Results-Oriented Government

performance results. Furthermore, most of the agencies we reviewed did 

not provide a full level of confidence in the credibility of their performance 

data.

Performance-based management, as envisioned by GPRA, requires 

transforming organizational cultures to improve decision making, 

maximize performance, and assure accountability. This transformation is 

not an easy one and requires investments of time and resources as well as 

sustained leadership commitment and attention. Challenges to successful 

implementation of GPRA include inconsistent top leadership commitment 

to creating a focus on results; an approach to setting goals and developing 

strategies for achieving critical outcomes that creates individual agency 

stovepipes rather than an integrated, holistic governmentwide approach; 

getting federal managers to make greater use of performance information 

to manage their programs and providing them authority to act that is 

commensurate with their accountability for results; difficulty in 

establishing meaningful measures of outcomes and assessing results of 

federal programs that are carried out by nonfederal entities; and untimely 

performance data.

The challenges identified in this report are not new—most have not 

changed significantly since we first reported on governmentwide 

implementation of GPRA. However, we have frequently reported on 

approaches that agencies, OMB, and Congress could use to address the 

challenges. These approaches include strengthening the commitment of 

top leadership to creating and sustaining a focus on results; taking a 

governmentwide approach to achieving outcomes that are crosscutting in 

nature; improving the usefulness of performance information to managers, 

Congress, and the public; and improving the quality of performance 

measures and data. Collectively, these approaches form the agenda that 

federal agencies, OMB, and Congress will need to follow to bring about a 

more sustainable, governmentwide focus on results.

Strengthening Top 

Leadership Commitment to 

Creating and Sustaining 

Results-Oriented Cultures

Successfully addressing the challenges that federal agencies face requires 

leaders who are committed to achieving results, who recognize the 

importance of using results-oriented goals and quantifiable measures, and 

who integrate performance-based management into the culture and day-today activities of their organizations. Top leadership must play a critical 

role in creating and sustaining high-performing organizations. Without the 

clear and demonstrated commitment of agency top leadership—both 
Chapter 5

Conclusions and Recommendations

Page 102 GAO-04-38 Results-Oriented Government

political and career—organizational cultures will not be transformed, and 

new visions and ways of doing business will not take root.

To be positioned to address the array of challenges faced by our national 

government, federal agencies will need to transform their organizational 

cultures so that they are more results oriented, customer-focused, and 

collaborative. Leading public organizations here in the United States and 

abroad have found that strategic human capital management must be the 

centerpiece of any serious change management initiative and efforts to 

transform the cultures of government agencies. Performance management 

systems are integral to strategic human capital management. Such systems 

can be key tools to maximizing performance by aligning institutional 

performance measures with individual performance and creating a “line of 

sight” between individual and organizational goals. Leading organizations 

use their performance management systems as a key tool for aligning 

institutional, unit, and employee performance; achieving results; 

accelerating change; managing the organization day to day; and facilitating 

communication throughout the year so that discussions about individual 

and organizational performance are integrated and ongoing.1

 

Furthermore, achieving this cultural transformation requires people to 

have the knowledge and skills to develop and use performance information 

to improve program performance. Our survey data indicated a significant 

relationship between those managers who reported they received training 

on setting performance goals and those who used performance information 

when setting or revising performance goals. However, federal agencies 

have not consistently showed a commitment to investing in needed training 

and development opportunities to help ensure that managers and 

employees have the requisite skills and competencies to achieve agency 

goals.

The commitment to focusing on and using performance information needs 

to extend to OMB and Congress as well. Through the administration’s PMA 

and PART initiatives, OMB has clearly placed greater emphasis on 

management issues over the past several years. However, the focus of such 

oversight needs to extend beyond the emphasis on formulating the 

President’s Budget to include an examination of the many challenges 

agencies face that may be contributing to poor performance. In spite of the 

1

U.S. General Accounting Office, Human Capital: Key Principles From Nine Private 

Sector Organizations, GAO/GGD-00-28 (Washington, D.C.: Jan. 31, 2000).
Chapter 5

Conclusions and Recommendations

Page 103 GAO-04-38 Results-Oriented Government

persistent weaknesses we found in agencies’ strategic plans and annual 

performance plans and reports, OMB significantly reduced the scope of its 

guidance to agencies on how to prepare these documents. By emphasizing 

a focus on resource allocation through its PART exercise and providing less 

information on how to comply with GPRA, OMB may be sending a message 

to agencies that compliance with GPRA is not important. Without strong 

leadership from OMB, the foundation of performance information that has 

been built could deteriorate.

OMB leadership is critical to addressing the continuing challenges 

presented in GPRA implementation and the transformation of the federal 

government to an increasingly results-oriented culture. OMB, as the 

primary focal point for overall management in the federal government, can 

provide the needed impetus by providing guidance, fostering 

communication among agencies, and forming intragovernmental councils 

and work groups tasked with identifying potential approaches and 

solutions to overcoming the persistent challenges to results-oriented 

management.

Congress can also play a decisive role in fostering results-oriented cultures 

in the federal government by using information on agency goals and results 

at confirmation, oversight, authorization, and appropriation hearings. 

Consistent congressional interest in the status of an agency’s GPRA efforts, 

performance measures, and uses of performance information to make 

decisions, will send an unmistakable message to agencies that Congress 

expects GPRA to be thoroughly implemented.

We also found that timing issues may affect the development of agency 

strategic plans that are meaningful and useful to top leadership. The 

commitment of top leadership within agencies, OMB, and Congress is 

critical to the success of strategic planning efforts. A strategic plan should 

reflect the policy priorities of an organization’s leaders and the input of key 

stakeholders if it is to be an effective management tool. However, GPRA 

specifies time frames for updating strategic plans that do not correspond to 

presidential or congressional terms. As a result, an agency may be required 

to update its strategic plan a year before a presidential election and without 

input from a new Congress. If a new president is elected, the updated plan 

is essentially moot and agencies must spend additional time and effort 

revising it to reflect new priorities. Our focus group participants, including 

GPRA experts, strongly agreed that this timing issue should be addressed 

by adjusting time frames to correspond better with presidential and 

congressional terms.
Chapter 5

Conclusions and Recommendations

Page 104 GAO-04-38 Results-Oriented Government

Addressing 

Governmentwide Needs

Mission fragmentation and program overlap are widespread throughout the 

federal government.2 We have noted that interagency coordination is 

important for ensuring that crosscutting program efforts are mutually 

reinforcing and efficiently implemented. Our review of six agencies’ 

strategic and annual performance plans along with our previous work on 

crosscutting issues has demonstrated that agencies’ still present their goals 

and strategies in a mostly stovepiped manner. They have generally not 

used their plans to communicate the nature of their coordination with 

other agencies, in terms of the development of common or complementary 

goals and objectives or strategies jointly undertaken to achieve those goals.

We have also reported that GPRA could provide a tool to reexamine federal 

government roles and structures governmentwide. GPRA requires the 

President to include in his annual budget submission a federal government 

performance plan. Congress intended that this plan provide a “single 

cohesive picture of the annual performance goals for the fiscal year.” The 

governmentwide performance plan could help Congress and the executive 

branch address critical federal performance and management issues, 

including redundancy and other inefficiencies in how we do business. It 

could also provide a framework for any restructuring efforts. 

Unfortunately, this provision has not been fully implemented. Instead, 

OMB has used the President’s Budget to present high-level information 

about agencies and certain program performance issues. The agency-byagency focus of the budget does not provide the integrated perspective of 

government performance envisioned by GPRA.

If the governmentwide performance plan were fully implemented, it could 

provide a framework for such congressional oversight. For example, in 

recent years, OMB has begun to develop common measures for similar 

programs, such as job training. By focusing on broad goals and objectives, 

oversight could more effectively cut across organization, program, and 

other traditional boundaries. Such oversight might also cut across existing 

committee boundaries, which suggests that Congress may benefit from 

using specialized mechanisms to perform oversight (i.e., joint hearings and 

special committees).

A strategic plan for the federal government, supported by key national 

indicators to assess the government’s performance, position, and progress, 

2

GAO/AIMD-97-146.
Chapter 5

Conclusions and Recommendations

Page 105 GAO-04-38 Results-Oriented Government

could provide an additional tool for governmentwide reexamination of 

existing programs, as well as proposals for new programs. If fully 

developed, a governmentwide strategic plan can potentially provide a 

cohesive perspective on the long-term goals of the federal government and 

provide a much needed basis for fully integrating, rather than merely 

coordinating, a wide array of federal activities. Successful strategic 

planning requires the involvement of key stakeholders. Thus, it could serve 

as a mechanism for building consensus. Further, it could provide a vehicle 

for the President to articulate long-term goals and a road map for achieving 

them. In addition, a strategic plan can provide a more comprehensive 

framework for considering organizational changes and making resource 

decisions.

Developing a strategic plan for the federal government would be an 

important first step in articulating the role, goals, and objectives of the 

federal government. It could help provide critical horizontal and vertical 

linkages. Horizontally, it could integrate and foster synergies among 

components of the federal government as well as help to clarify the role of 

the federal government vis-à-vis other sectors of our society. Vertically, it 

could provide a framework of federal missions and goals within which 

individual federal agencies could align their own missions and goals that 

would cascade down to individual employees. The development of a set of 

key national indicators could be used as a basis to inform the development 

of governmentwide strategic and annual performance plans. The 

indicators could also link to and provide information to support outcomeoriented goals and objectives in agency-level strategic and annual 

performance plans.

Improving Usefulness of 

Performance Information

We have found that leading organizations that progressed the farthest to 

results-oriented management did not stop after strategic planning and 

performance measurement. They applied their acquired knowledge and 

data to identify gaps in their performance, report on that performance, and 

finally use that information to improve their performance to better support 

their missions.
Chapter 5

Conclusions and Recommendations

Page 106 GAO-04-38 Results-Oriented Government

Performance data can have real value only if they are used to identify the 

gap between an organization’s actual performance level and the 

performance level it has identified as its goal. Once the performance gaps 

are identified for different program areas, managers can determine where 

to target their resources to improve overall mission accomplishment. 

When managers are forced to reduce their resources, the same analysis can 

help them target reductions to keep to a minimum the impact on their 

organization’s overall mission.3

Under GPRA, agencies produce a single strategic plan, annual performance 

plan, and annual performance report. However, there are many potential 

consumers of agencies’ performance information—Congress, the public, 

and the agency itself. One size need not fit all. Clearly, an agency will need 

more detailed information on its programs for operational purposes than 

would be suitable for external audiences. Of the six agencies’ performance 

reports we reviewed, some of them provided useful summary tables or 

information that provided overall snapshots of performance or highlighted 

progress in achieving key goals. Other reports that lacked such a summary 

made it difficult to assess the progress achieved.

To improve the prospect that agency performance information will be 

useful to and used by these different users, agencies need to consider the 

different information needs and how to best tailor their performance 

information to meet those needs. For example, we have reported that, 

although many information needs were met, congressional staff also 

identified gaps in meeting their information needs.4

 Key to addressing 

these information gaps was improving communication between 

congressional staff and agency officials to help ensure that congressional 

information needs are understood, and that, where feasible, arrangements 

are made to meet them. Improved two-way communication might also 

make clear what information is and is not available, as well as what is 

needed and what is not needed. This might entail the preparation of 

simplified and streamlined plans and reports for Congress and other 

external users.

3

U.S. General Accounting Office, Executive Guide: Effectively Implementing the 

Government Performance and Results Act, GAO/GGD-96-118 (Washington, D.C.: June 1, 

1996).

4

GAO/GGD-00-35.
Chapter 5

Conclusions and Recommendations

Page 107 GAO-04-38 Results-Oriented Government

Another challenge that limits the usefulness of agency performance reports 

is the lack of timely data on performance. For the six performance reports 

we reviewed we continued to observe a significant number of goals for 

which performance data were unavailable. Policy decisions made when 

designing federal programs, particularly intergovernmental programs, may 

make it difficult to collect timely and consistent national data. In 

administering programs that are the joint responsibility of state and local 

governments, Congress and the executive branch continually balance the 

competing objectives of collecting uniform program information to assess 

performance with giving states and localities the flexibility needed to 

effectively implement intergovernmental programs.

Improving Performance 

Measures and Data Quality

Another key challenge to achieving a governmentwide focus on results is 

that of developing meaningful, outcome-oriented performance goals and 

collecting performance data that can be used to assess results. 

Performance measurement under GPRA is the ongoing monitoring and 

reporting of program accomplishments, particularly progress toward 

preestablished goals. It tends to focus on regularly collected data on the 

level and type of program activities, the direct products and services 

delivered by the programs, and the results of those activities. For programs 

that have readily observable results or outcomes, performance 

measurement may provide sufficient information to demonstrate program 

results. In some programs, however, outcomes are not quickly achieved or 

readily observed, or their relationship to the program is uncertain. In such 

cases, more in-depth program evaluations may be needed, in addition to 

performance measurement, to examine the extent to which a program is 

achieving its objectives.
Chapter 5

Conclusions and Recommendations

Page 108 GAO-04-38 Results-Oriented Government

Given the difficult measurement challenges we have identified, it is all the 

more important that agency strategic planning efforts include the 

identification of the most critical evaluations that need to take place to 

address those challenges. However, our previous work has raised concerns 

about the capacity of federal agencies to produce evaluations of program 

effectiveness.5

 Few of the agencies we reviewed deployed the rigorous 

research methods required to attribute changes underlying outcomes to 

program activities. Yet we have also seen how some agencies have 

profitably drawn on systematic program evaluations to improve their 

measurement of program performance or understanding of performance 

and how it might be improved.6

 Our review of six agencies’ strategic plans 

and performance reports in this report revealed weaknesses in their 

discussions of program evaluation. Most of the strategic plans lacked 

critical information required by GPRA, such as a discussion of how 

evaluations were used to establish strategic goals or a schedule of future 

evaluations. Furthermore, two of the six performance reports did not 

summarize the results of program evaluations completed that year, as 

required.

Our work has also identified substantial, long-standing limitations in 

agencies’ abilities to produce credible data and identify performance 

improvement opportunities that will not be quickly or easily resolved.7

 

According to our review, five of six agencies’ annual performance plans 

showed meaningful improvements in how they discussed the quality of 

performance data. However, only DOT’s performance plan and report 

contained information that provided a full level of confidence in the 

credibility of its performance data. In particular, the plans and reports did 

not always provide detailed information on how the agencies verified and 

validated their performance data.

5

U.S. General Accounting Office, Program Evaluation: Agencies Challenged by New 

Demand for Information on Program Results, GAO/GGD-98-53 (Washington, D.C.: Apr. 24, 

1998).

6

U.S. General Accounting Office, Program Evaluation: Studies Helped Agencies Measure 

or Explain Program Performance, GAO/GGD-00-204 (Washington, D.C.: Sept. 28, 2000).

7

GAO/GGD-00-52.
Chapter 5

Conclusions and Recommendations

Page 109 GAO-04-38 Results-Oriented Government

Recommendations for 

Executive Action

To provide a broader perspective and more cohesive picture of the federal 

government’s goals and strategies to address issues that cut across 

executive branch agencies, we recommend that the Director of OMB fully 

implement GPRA’s requirement to develop a governmentwide performance 

plan.

To achieve the greatest benefit from both GPRA and PART, we recommend 

that the Director of OMB articulate and implement an integrated and 

complementary relationship between the two. GPRA is a broad legislative 

framework that was designed to be consultative with Congress and other 

stakeholders, and allows for varying uses of performance information. 

PART looks through a particular lens for a particular use—the executive 

budget formulation process.

To improve the quality of agencies’ strategic plans, annual performance 

plans, and performance reports and help agencies meet the requirements of 

GPRA, we recommend that the Director of OMB provide clearer and more 

consistent guidance to executive branch agencies on how to implement 

GPRA. Such guidance should include standards for communicating key 

performance information in concise as well as longer formats to better 

meet the needs of external users who lack the time or expertise to analyze 

lengthy, detailed documents.

To help address agencies’ performance measurement challenges, we 

recommend that the Director of OMB engage in a continuing dialogue with 

agencies about their performance measurement practices with a particular 

focus on grant-making, research and development, and regulatory 

functions to identify and replicate successful approaches agencies are 

using to measure and report on their outcomes, including the use of 

program evaluation tools. Additionally, we recommend that the Director of 

OMB work with executive branch agencies to identify the barriers to 

obtaining timely data to show progress against performance goals and the 

best ways to report information where there are unavoidable lags in data 

availability. Interagency councils, such as the President’s Management 

Council and the Chief Financial Officers’ Council, may be effective vehicles 

for working on these issues.

To facilitate the transformation of agencies’ management cultures to be 

more results-oriented, we recommend that the Director of OMB work with 

agencies to ensure they are making adequate investments in training on 
Chapter 5

Conclusions and Recommendations

Page 110 GAO-04-38 Results-Oriented Government

performance planning and measurement, with a particular emphasis on 

how to use performance information to improve program performance.

Matters for 

Congressional 

Consideration

To ensure that agency strategic plans more closely align with changes in 

the federal government leadership, Congress should consider amending 

GPRA to require that updates to agency strategic plans be submitted at 

least once every 4 years, 12-18 months after a new administration begins its 

term. Additionally, consultations with congressional stakeholders should 

be held at least once every new Congress and interim updates made to 

strategic and performance plans as warranted. Congress should consider 

using these consultations along with its traditional oversight role and 

legislation as opportunities to clarify its performance expectations for 

agencies. This process may provide an opportunity for Congress to 

develop a more structured oversight agenda.

To provide a framework to identify long-term goals and strategies to 

address issues that cut across federal agencies, Congress also should 

consider amending GPRA to require the President to develop a 

governmentwide strategic plan.

Agency Comments We provided a copy of the draft report to OMB for comment. OMB’s 

written comments are reprinted in appendix VIII. In general, OMB agreed 

with our findings and conclusions. OMB agreed to implement most of our 

recommendations, noting that these recommendations will enhance its 

efforts to make the government more results oriented. OMB agreed to 

(1) work with agencies to ensure they are provided adequate training in 

performance management, (2) revise its guidance to clarify the integrated 

and complementary relationship between GPRA and PART, and 

(3) continue to use PART to improve agency performance measurement 

practices and share those practices across government. 

In response to our recommendation that OMB fully implement GPRA’s 

requirement to develop a governmentwide performance plan, OMB stated 

that the President’s Budget represents the executive branch’s 

governmentwide performance plan. However, the agency-by-agency focus 

of the budget over the past few years does not provide an integrated 

perspective of government performance, and thus does not meet GPRA’s 

requirement to provide a “single cohesive picture of the annual 

performance goals for the fiscal year.” In response to our matter for 
Chapter 5

Conclusions and Recommendations

Page 111 GAO-04-38 Results-Oriented Government

congressional consideration that Congress should consider amending 

GPRA to require the President to develop a governmentwide strategic plan, 

OMB noted that the budget serves as the governmentwide strategic plan. 

However, in our opinion, the President’s Budget focuses on establishing 

agency budgets for the upcoming fiscal year. Unlike a strategic plan, it 

provides neither a long-term nor an integrated perspective on the federal 

government’s activities. A governmentwide strategic plan should provide a 

cohesive perspective on the long-term goals of the federal government and 

provide a basis for fully integrating, rather than primarily coordinating, a 

wide array of federal activities.

We provided relevant sections of the draft report to Education, DOE, HUD, 

SBA, SSA, and DOT. Education and SBA did not provide any comments, 

while DOT provided minor technical comments. Written comments from 

DOE, HUD, and SSA are reprinted in appendixes IX, X, and XI, respectively, 

along with our responses.

DOE disagreed with portions of our analyses of its 2004 Annual 

Performance Plan and its 2002 Performance and Accountability Report. 

Our analysis of DOE’s documents was based on specific criteria (see 

appendixes IV and V for details) and was characterized in relation to our 

reviews of the other five agencies’ documents. We modified or clarified 

certain characterizations in response to DOE comments, but for the most 

part found that our characterizations were appropriate.

SSA generally agreed with our observations and agreed to incorporate 

them in its future planning efforts. SSA made several points of clarification 

and disagreed with our observation that its performance and accountability 

report does not clearly state how program evaluations were used to answer 

questions about program performance and results and how they can be 

improved. SSA noted that its evaluations rely on surveys, and these 

surveys form the basis for its efforts to deliver high-quality service. SSA 

also noted that it lists other evaluations that are of great importance to its 

ongoing operations. We do not discount the usefulness of SSA’s surveys in 

assessing its day-to-day management of programs. Rather, we believe that 

it would be helpful for SSA to clearly identify the range of evaluations 

conducted and how each of them contributed to improved program 

performance.

HUD noted that all of the areas we suggested for further improvement are 

under consideration for improvement. However, they disagreed with us on 

two observations related to the strategic plan: (1) that the link between its 
Chapter 5

Conclusions and Recommendations

Page 112 GAO-04-38 Results-Oriented Government

long-term and intermediate goals is difficult to discern and (2) that it did 

not explain how it used the results of program evaluations to update the 

current plan and did not include a schedule for future evaluations. On the 

basis of OMB guidance for preparing strategic plans and the criteria we 

used to evaluate all six agencies’ strategic plans (see app. III for more 

detail), we maintain that these two observations are valid and require 

further attention. HUD also disagreed with how we presented the 

performance information in its summary report cards (see fig. 22). HUD 

noted that many of the results were explained in the individual indicator 

write-ups that followed the summary information. Our analysis of HUD’s 

information included qualitative aspects of how the information was 

presented, such as its usefulness to inform the average reader with little or 

no exposure to the subject matter, and the extent to which HUD presented 

a complete summary of performance information in a user-friendly format.

Technical comments from DOE, HUD, and SSA were incorporated, as 

appropriate.
Page 113 GAO-04-38 Results-Oriented Government

Appendix I

Objectives, Scope, and Methodology AAppenednxidexiIs

As agreed with your offices, our objectives for this report were to 

determine (1) the effect of the Government Performance and Results Act 

(GPRA) over the last 10 years in creating a governmentwide focus on 

results and the government’s ability to deliver results to the American 

public, including an assessment of changes in the overall quality of 

agencies’ strategic plans, annual performance plans, and annual 

performance reports; (2) the challenges that agencies face in measuring 

performance and using performance information in management decisions; 

and (3) how the federal government can continue to shift toward a more 

results-oriented focus. To meet our objectives, we collected 

governmentwide data to assess the government’s overall focus on results. 

We conducted a governmentwide survey of federal managers, focus groups 

with federal managers and GPRA experts, and interviews with top 

appointed officials. We identified and reviewed previously published 

reports on GPRA. Finally, we selected a sample of agencies to review for 

changes in the quality of their strategic plans, performance plans, and 

performance reports since their initial efforts.

We conducted our work from January through November 2003 in 

Washington, D.C. in accordance with generally accepted government 

auditing standards. We provided drafts of the relevant sections of this 

report to officials from each of the agencies whose GPRA reports we 

reviewed. We also provided a draft of this report to OMB.

Methodology for 

Governmentwide 

Survey

A Web-based questionnaire on performance and management issues was 

administered to a stratified random probability sample of 800 persons from 

a population of approximately 98,000 mid-level and upper-level civilian 

managers and supervisors working in the 24 executive branch agencies 

covered by the Chief Financial Officers Act of 1990 (CFO). The sample was 

drawn from the Office of Personnel Management’s (OPM) Civilian 

Personnel Data File as of December 31, 2002, using file designators 

indicating performance of managerial and supervisory functions.
Appendix I

Objectives, Scope, and Methodology

Page 114 GAO-04-38 Results-Oriented Government

The questionnaire was designed to obtain the observations and perceptions 

of respondents on various aspects of GPRA as well as such results-oriented 

management topics as the presence, use, and usefulness of performance 

measures, hindrances to measuring and using performance information, 

and agency climate. Most of the items on the questionnaire were closedended, meaning that depending on the particular item, respondents could 

choose one or more response categories or rate the strength of their 

perception on a 5-point extent scale. Almost all the items on this 

questionnaire were asked in two earlier mail-out surveys. One survey was 

conducted between November 1996 and January 1997 as part of the work 

we did in response to a GPRA requirement that we report on 

implementation of the act. The other survey was conducted between 

January and August 2000.1

This survey covered the same CFO Act agencies and was designed to 

update the results from the two earlier surveys. Similar to the two earlier 

surveys, the sample was stratified by whether the manager or supervisor 

was Senior Executive Service (SES) or non-SES. The management levels 

covered general schedule (GS), general management (GM), or equivalent 

schedules at levels comparable to GS/GM-13 through career SES or 

equivalent levels of executive service. The sample also included the same 

or equivalent special pay plans that were covered in our 2000 survey, e.g., 

Senior Foreign Service executives.

We sent an e-mail to members of the sample that notified them of the 

survey’s availability on the GAO Web site and included instructions on how 

to access and complete the survey. Members of the sample who did not 

respond to the initial notice were sent up to two subsequent reminders 

asking them to participate in the survey. The survey was administered from 

June through August 2003.

During the course of the survey, we deleted 26 persons from our sample 

who had either retired, separated, died, or otherwise left the agency or had 

some other reason that excluded them from the population of interest. We 

received useable questionnaires from 503 sample respondents, about 65 

percent of the eligible sample. The eligible sample includes 39 persons that 

1

For information on the design and administration of the two earlier surveys, see GAO/GGD97-109, GAO-01-127, and U.S. General Accounting Office, Managing For Results: Federal 

Managers’ Views on Key Management Issues Vary Widely Across Agencies, GAO-01-592

(Washington, D.C.: May 25, 2001).
Appendix I

Objectives, Scope, and Methodology

Page 115 GAO-04-38 Results-Oriented Government

we were unable to locate and therefore unable to request that they 

participate in the survey.

To assess whether the views of those individuals who chose not to 

participate in our survey might be different than those who did, we made 

an effort to administer a brief survey over the telephone to those 

individuals who still had not responded about a month or more after the 

survey had been available to them despite being contacted twice after the 

initial notification e-mail had been sent out. This telephone survey 

consisted of four items from the full survey. There were 58 persons who 

agreed to answer these questions over the telephone. This was 41 percent 

of those individuals who had not responded at the time we attempted to 

contact them for the purpose of asking these four questions. 

We analyzed the responses of this group on the four selected items 

compared to the responses received from all other respondents. Our 

analyses of the items showed very few differences between nonresponders 

and responders. There was no sufficient or consistent pattern of 

responding that would warrant a conclusion that the views of 

nonresponders were notably different than responders. The responses of 

each eligible sample member who provided a useable questionnaire were 

subsequently weighted in the analysis to account statistically for all the 

members of the population.

The overall survey results are generalizable to the population of managers 

as described above at the CFO Act agencies. All results are subject to some 

uncertainty or sampling error as well as nonsampling error. As part of our 

effort to reduce nonsampling sources of error in survey results, we 

checked and edited (1) the survey data for responses that failed to follow 

instructions and (2) the programs used in our analyses. In general, 

percentage estimates in this report for the entire sample have confidence 

intervals ranging from about ± 4 to ±11 percentage points at the 95 percent 

confidence interval. In other words, if all CFO Act agency managers and 

supervisors in our population had been surveyed, the chances are 95 out of 

100 that the result obtained would not differ from our sample estimate in 

the more extreme cases by more than ±11 percent. Appendix VI shows the 

questions asked with the weighted percentage of managers responding to 

each item.

Because a complex sample design was used in the current survey as well as 

the two previous surveys and different types of statistical analyses are 

being done, the magnitude of sampling error will vary across the particular 
Appendix I

Objectives, Scope, and Methodology

Page 116 GAO-04-38 Results-Oriented Government

surveys, groups, or items being compared due to differences in the 

underlying sample sizes and associated variances. The number of 

participants in the current survey is only about one fifth of the number in 

the 2000 survey (2,510) and slightly more than half of those in the first 

survey (905). The 2000 survey was designed with a larger sample than the 

other two surveys in order to provide estimates for each individual agency 

as well as all the CFO Act agencies collectively. Consequently, in some 

instances, a difference of a certain magnitude may be statistically 

significant. In other instances, depending on the nature of the comparison 

being made, a difference of equal or even greater magnitude may not 

achieve statistical significance. For example, when comparing a result 

from the current survey to the larger 2000 survey with its relatively smaller 

confidence interval, a difference of a certain magnitude may be significant. 

However, when comparing the current survey with the first survey, that 

difference may not be significant given the greater imprecision in the 

estimates due to both surveys’ smaller sample sizes. We note throughout 

the report when differences are significant at the .05 probability level.

Methodology for Focus 

Groups

We held a series of focus groups as one of our methods for obtaining 

information about the accomplishments and challenges agencies face in 

implementing and overseeing GPRA-related activities. Focus groups are a 

form of qualitative research in which a specially trained leader, a 

moderator, meets with a small group of people (usually 8 to 10) who are 

knowledgeable about the topics to be discussed.

In all, we conducted eight focus groups—one with experts on GPRA and 

performance management and seven with federal managers. For our focus 

group with experts, we invited individuals from the private sector, academia, 

the National Academy of Public Administration, and OMB. These 

individuals were involved either in drafting GPRA, overseeing its 

implementation, or studying and critiquing implementation, from outside 

government. Out of the 14 experts we invited, a total of 11 attended the 

focus group.
Appendix I

Objectives, Scope, and Methodology

Page 117 GAO-04-38 Results-Oriented Government

For our focus groups with agency managers, we obtained a list of potential 

participants for our focus groups by contacting all 24 CFO Act agencies and 

requesting that they submit a list of candidates and their profiles based on 

the following criteria: federal managers (1) in the GS-13 pay grade and 

above, including members of the SES; (2) having at least 3 years of 

managerial experience; (3) currently located in the Washington, D.C., 

metropolitan area; (4) having hands-on experience with GPRA or 

performance management;2 and (5) representing both departments and 

their component bureaus. We received profiles of candidates from all 

agencies; however, no managers from the OPM chose to participate in the 

focus groups.

To select focus group participants, we reviewed the profiles submitted by 

agencies and selected candidates with diverse experience who held a 

variety of different positions within the agency in order to capture a broad 

range of perspectives. For example, we invited a comptroller; a deputy 

director of management, administration, and planning; budget directors; 

budget officers; management analysts; and program managers; among 

others. We contacted the candidates and provided them with the list of 

questions to be discussed at the focus group in advance so they would be 

aware of our interests and be better able to provide us, where possible, 

with examples to illustrate their responses to our questions. Out of 104 

agency officials we invited, 70 participated in the focus groups.3

 

2

For example, candidates could be operations managers with hands-on experience 

managing a federal program or agency officials directly involved in carrying out the 

activities required under GPRA, such as developing a strategic or annual performance plan 

or annual performance report.

3

Due to last minute circumstances, a federal manager participated via teleconference from 

an agency’s field office and another was unable to attend the focus group, but mailed his 

answers to questions we sent to all participants in advance of the focus groups.
Appendix I

Objectives, Scope, and Methodology

Page 118 GAO-04-38 Results-Oriented Government

During each session, the moderator explained the scope of our work and 

elaborated on how the focus groups were one of several methods we were 

using to collect information relevant to our objectives. As part of the focus 

group process, the moderator asked participants at each session to identify 

the main accomplishments and challenges that, in their view, could be 

attributed to GPRA and to mention possible solutions to these challenges. 

During the sessions, we created lists of the accomplishments, challenges, 

and solutions identified by group participants and posted these lists around 

the meeting room. Participants were then asked to review the lists and 

vote on the three most important accomplishments and the top three 

challenges.4

 

To organize the information collected during the focus groups, we reviewed 

the statements made by participants in response to our questions. We 

identified related sets of statements and summarized them as a general 

theme. We noted how often a theme was expressed both within and across 

each focus group. We also examined the number of votes each posted 

statement obtained. Our analysis focused on those themes that were 

supported by statements that obtained a high number of votes and were 

mentioned frequently within and across the majority of focus groups.

The focus group results discussed in this report are summary descriptions 

reflecting the range of views and perceptions held by employees, 

supervisors, or project managers who participated in the focus groups. 

Although we cannot assume all federal managers share these views, the 

extent to which certain opinions or perceptions were repeatedly expressed 

or endorsed by many participants from multiple focus groups provides a 

rough gauge of the significance of these views.

Methodology for 

Interviews with 

Political Appointees 

To obtain an additional perspective from top political managers of federal 

agencies on GPRA, we held telephone or in-person interviews with 10 highlevel officials serving under political appointments with CFO Act agencies. 

Five former officials from the Clinton administration and five serving under 

the current Bush administration were interviewed. For example, we 

interviewed deputy secretaries, chief financial officers, and deputy 

assistant secretaries for management. We asked them to provide their 

perspective on the main accomplishments or other effects of GPRA, the 

4

We read the list of comments to the manager who participated via teleconference and voted 

on his behalf based on his preferences.
Appendix I

Objectives, Scope, and Methodology

Page 119 GAO-04-38 Results-Oriented Government

key challenges to implementation, and possible improvements to GPRA. 

We summarized the interviewees’ answers and identified recurring themes 

or observations for our analysis.

Methodology for 

Selecting Agencies to 

Review for Changes in 

the Quality of Their 

Strategic Plans, Annual 

Performance Plans, 

and Annual 

Performance Reports 

To address how the quality of agency strategic plans, performance plans, 

and performance reports have changed since their initial efforts, we 

reviewed a sample of six agencies’ current strategic plans, annual 

performance plans, and annual performance reports and compared the 

results to our findings from prior reviews of the agencies’ initial efforts in 

producing these documents. We did not independently verify or assess the 

information we obtained from agency plans and reports. If an agency 

chose not to discuss its efforts concerning elements in the plans and 

reports, it does not necessarily mean that the agency is not implementing 

those elements.

We selected the departments and agencies to review based on the extent to 

which they collectively represented the full range of characteristics in the 

following four areas: (1) agency size (small, medium, large); (2) primary 

program types (direct service, research, regulatory, transfer payments, and 

contracts or grants); (3) quality of fiscal year 2000 performance plan based 

on our previous review (low, medium, high);5

 and (4) type of agency 

(cabinet department and independent agency).

Based on these characteristics, we selected the following departments and 

agencies:

• Department of Education (Education),

• Department of Energy (DOE),

• Department of Housing and Urban Development (HUD), 

• Small Business Administration (SBA), 

5

GAO/GGD/AIMD-99-215. Based on how we had rated agencies’ annual performance plans 

on their picture of performance, specificity of strategies and resources, and the degree of 

confidence that performance information will be credible, we assigned numeric values to 

each agencies’ rating (e.g., clear=3, general=2, limited=1, unclear=0) and added them up to 

determine overall quality of high, medium, or low. An agency’s plan was considered high 

quality if its score was between 7-9, a score of 5-6 was considered medium quality, and a 

score of 3-4 was low. No agencies received a score lower than 3.
Appendix I

Objectives, Scope, and Methodology

Page 120 GAO-04-38 Results-Oriented Government

• Social Security Administration (SSA), and 

• Department of Transportation (DOT).

Table 4 shows the characteristics represented by each of these agencies.

Table 4: Summary of Characteristics of Agencies Selected for Review of Strategic Plans, Annual Performance Plans, and Annual 

Performance Reports

Source: GAO.

a

The size of the agencies is based on data from December 2002.

b

In March 2003, the U.S. Coast Guard and the Transportation Security Administration (TSA) were 

transferred from DOT to the Department of Homeland Security. According to the fiscal year 2005 

President’s Budget, in fiscal year 2003, the Coast Guard and TSA had 43,702 and 57,324 full-time 

equivalent positions, respectively.

A more detailed discussion of the criteria we used to assess the quality of 

the agencies’ planning and reporting documents and the results of our 

review is contained in appendixes III, IV, and V.

Sizea Functions

Quality of fiscal 

year 2000 

performance 

plans

Departments 

Full time 

equivalent 

positions

Small (S), 

medium (M), 

large (L) Research

Direct 

service Regulatory

Transfer

payments

Grants/

contracts

Low (L), 

medium (M), 

high (H)

Education 4,756 S X X X X X M

DOE 16,067 M X X X L

HUD 10,752 M X X X X M

DOT 135,022b L X X X X H

Agencies

SBA 4,005 S X X X L

SSA 64,418 L X X H
Page 121 GAO-04-38 Results-Oriented Government

Appendix II

Focus Group Participants Agreed GPRA 

Provides a Framework for Federal Agencies 

to Become More Results Oriented ApendxiI

While GPRA’s goal is to make the federal government more results 

oriented, work carried out in support of this effort, such as planning 

activities, implementing programs, reporting on outcomes, and evaluating 

performance, generally lies in the hands of federal managers. To get a 

better appreciation for the main accomplishments and challenges agencies 

face in implementing and overseeing GPRA-related activities, we 

conducted a total of seven focus groups comprised of federal managers 

from 23 CFO Act agencies, and an eighth focus group comprised of 11 

experts on GPRA and performance management and budgeting.

For our focus groups, we asked participants to discuss (1) the key 

accomplishments of GPRA to date, (2) whether GPRA has created a focus 

on achieving results across the federal government, (3) the effect of GPRA 

on the government’s ability to deliver results to the American public, (4) the 

persistent and prevalent challenges agencies face in implementing and 

overseeing GPRA-related activities, and (5) suggestions to address these 

challenges.

We recorded the views expressed by participants and categorized them into 

themes that were most commonly expressed or endorsed both within and 

across the groups. The focus group results discussed in this report are 

organized according to the themes we identified and are summary 

descriptions reflecting the range of views and perceptions expressed by the 

experts, supervisors, and project managers. A more complete discussion 

of our scope and methodology can be found in appendix I.

Focus group participants indicated that GPRA has helped to make the 

federal government more results oriented. However, they noted that a 

number of obstacles have made GPRA implementation challenging, such as 

difficulty in establishing results-oriented goals and measuring 

performance, addressing frequently changing priorities resulting from 

changes in administration, and lack of top leadership support for GPRA. In 

all, participants generally perceive the information contained in GPRA 

reports to be important and useful; however, they do not believe that 

lawmakers use this information to make resource decisions or conduct 

oversight. To address these problems and concerns, focus group 

participants stated that, among other things, Congress should provide 

guidance to agencies on how to make GPRA reports more useful, OMB 

should reinforce its value as a management tool, and agencies need to 

commit the resources necessary to carry out GPRA-related activities.
Appendix II

Focus Group Participants Agreed GPRA 

Provides a Framework for Federal Agencies 

to Become More Results Oriented

Page 122 GAO-04-38 Results-Oriented Government

GPRA 

Accomplishments 

Overall, focus group participants stated that GPRA has had a positive effect 

on federal agencies’ efforts to become more results oriented. Based in 

statute, GPRA has created a framework for agencies to focus on achieving 

results by requiring them to establish program goals and objectives, 

develop performance indicators, and measure the extent to which they 

have made progress towards achieving program goals. As a result, federal 

managers noted that they have been increasingly able to view their 

programs in terms of outcomes, not outputs, and have been generally 

learning how to use this framework as a management tool. Participants 

also attributed a series of cultural changes within federal agencies to 

GPRA, where problem solving, creative thinking, and agencywide 

discussions on budget and performance have become more common. The 

strategic and annual performance plan and performance reports that 

federal agencies are required to submit to OMB and Congress under GPRA 

have increased the transparency of government activities. These 

documents have also helped agencies justify their budget requests based on 

their performance.

Creating a Framework in 

Statute and a Management 

Tool for Agency Leadership

Participants agreed that GPRA created a framework in statute for federal 

agencies to plan their activities in order to become more results oriented 

and provided a managerial tool for program accountability. Using this 

framework, agencies can develop and focus on strategies to carry out the 

programs they administer; set goals and identify performance indicators 

that will inform them whether or not they achieved the performance they 

expected; and determine what impact, if any, their programs have had on 

the American public. According to the experts in one of our focus groups, 

comparing federal agencies’ current mission statements contained in their 

strategic plans to what they were in the past demonstrates that agencies 

have done some “soul searching” to get a better sense of what their role is 

(or should be) and how they can achieve it. Given that GPRA is in statute, 

the use of this planning framework is likely to be sustained within agencies.

Participants also mentioned that GPRA has encouraged federal managers 

to view their programs in terms of results, not just inputs and outputs. 

Such a change is important, as it has encouraged federal managers to 

reflect on the statutory intent of their programs and use this framework as 

a management tool for establishing accountability within their programs.
Appendix II

Focus Group Participants Agreed GPRA 

Provides a Framework for Federal Agencies 

to Become More Results Oriented

Page 123 GAO-04-38 Results-Oriented Government

Cultural Change within 

Federal Agencies

Participants in the majority of focus groups agreed that GPRA has been a 

driving force behind many cultural changes that have occurred within 

federal agencies. Highlighting the focus on results, participants stated that 

GPRA has stimulated a problem-solving approach within federal agencies 

and encouraged them to think creatively when developing performance 

indicators for their programs. GPRA has also changed the dialogue within 

federal agencies; front-line managers and staff at lower levels of the 

organization now discuss budget issues in connection with performance. 

Similarly, experts noted that performance management and resource 

investments are more frequently communicated between agency officials 

and Congress than in the past. Within agencies, GPRA documents can 

provide a context of missions, goals, and strategies that political 

appointees can use to articulate agencies’ priorities.

Increased Transparency of 

Government Results

Some participants agreed that GPRA has increased federal agencies’ ability 

to present their results to the American public, benefiting both 

stakeholders and agency staff. On the one hand, GPRA reports enable 

federal agencies to communicate the results of government programs and 

activities to a broad public. For example, GPRA reports are available on 

agencies’ Web sites and provide information to OMB, Congress, and the 

American public on what agencies plan to do, how they plan to do it, and, 

as summarized in the performance and accountability reports, what 

agencies have accomplished and how much money it cost them to do it.

Similarly, some participants agreed that GPRA allows federal employees to 

see exactly how their work can produce a positive outcome, increasing 

employee morale. Using information contained in GPRA reports, agency 

employees can compare department goals to the results of their activities, 

and see how their work contributes to these goals. For example, a focus 

group participant from the Indian Health Service in California stated that 

he was pleased to learn that health care indicators of some Native 

American tribes had already exceeded levels originally projected by his 

agency to be reached by the year 2010.

Link between Budget and 

Performance

Participants also agreed that the GPRA framework has had a positive effect 

on agencies’ ability to link their performance to their budget. By focusing 

on their mission and outcomes, agencies are learning to prioritize activities 

and align their resources to ensure that they will be able to achieve results. 

For example, a participant stated that the National Wild Horse and Burro 
Appendix II

Focus Group Participants Agreed GPRA 

Provides a Framework for Federal Agencies 

to Become More Results Oriented

Page 124 GAO-04-38 Results-Oriented Government

Program, managed by the Bureau of Land Management in the U.S. 

Department of the Interior, recently developed a model for its GPRArelated work which divided their program into discrete components, 

identifying the results it could accomplish in the short- and long-term and 

specifying what additional resources were needed. According to the 

participant, the program received additional funding based on this needs 

assessment.

In addition, a few managers noted that the link between budget and 

performance has given agencies an incentive to commit the resources 

necessary to modernize information systems. Having the right information 

on time enables agencies to integrate budget requests and performance 

information in ways that are more meaningful. This information also 

increases Congress’s ability to make informed budget decisions based on 

agency performance.

Views on Delivering 

Results to the 

American Public Were 

Mixed

Participants’ views on whether GPRA has helped agencies deliver results to 

the American public were generally mixed. Some federal managers in our 

focus groups agreed that GPRA has had a positive effect on raising 

awareness on many issues, and that in and of itself is a way of delivering 

results. The information gathered and reported for GPRA allows agencies 

to make better-informed decisions, which improves their ability to achieve 

results. Of note, GPRA has allowed agencies to move towards 

performance-based budgeting, which helps agencies identify resources 

available to use in programs where outcomes can be achieved. For 

example, programs and expenses that do not add value to the agency’s 

mission could be eliminated. Having performance data readily available is 

another key area where GPRA has enabled agencies to deliver results to the 

American public.

Other participants stated that while certain aspects of GPRA-related work 

have been positive, agencies’ ability to deliver results and public awareness 

of their activities cannot always be exclusively attributed to GPRA. For 

example, while measuring performance is a move in the right direction, 

GPRA reports provide too many indicators and it is unclear how this has 

led to better performance among federal agencies. A few participants also 

stated that agencies deliver results in ways that the American public does 

not fully recognize. For example, a participant stated that agencies 

working in the area of international relations generally lack recognition for 

the results of their activities, as the American public is generally unfamiliar 

with the nuances of foreign policy and all the programs the U.S. 
Appendix II

Focus Group Participants Agreed GPRA 

Provides a Framework for Federal Agencies 

to Become More Results Oriented

Page 125 GAO-04-38 Results-Oriented Government

government finances overseas. And while GPRA has helped these agencies 

prioritize their work to achieve results, it is unclear that GPRA has 

improved the visibility and understanding in the public eye of what these 

agencies do because the American public does not see the results of their 

work. A participant stated that research-based agencies have also used 

GPRA to plan activities that benefit the American public in ways they are 

not fully aware of. For example, National Aeronautics and Space 

Administration’s (NASA) space program includes, among other things, 

predicting whether or not an asteroid will strike the earth, although on a 

daily basis the American public is probably not worried about such a rarity.

For other participants, the link between GPRA and agencies’ service 

delivery was not clear. Participants characterized GPRA-related activities 

as time consuming, and there is no real evidence that this work has 

improved their ability to deliver results to the American public. Other 

participants stated that many agencies rely on grant recipients to carry out 

their work, and delivering results to the American public depends, to a 

large extent, on the diligence of these organizations to implement their 

programs. Overall, they held the view that performance would not change 

dramatically if GPRA were no longer a requirement for federal agencies.

Alternate Views on 

GPRA’s Effect

Participants in one of our focus groups stated that GPRA, per se, had not 

led federal agencies to achieve specific accomplishments. These 

participants believed that managers’ initiative, not the framework 

established by GPRA, has been key to improving agencies’ planning efforts 

and focus on results. A few participants also mentioned that the results 

framework established by GPRA is somewhat intangible unless managers 

can use it effectively; without the adequate infrastructure to implement 

GPRA, an agency’s compliance with the law is simply paperwork, as GPRA 

does not allow for a systematic and thorough approach to performance 

management. For example, while agencies can develop performance 

indicators to gauge progress towards a program goal, they often encounter 

problems in collecting relevant data and measuring the agencies’ 

contribution to a specific outcome. In addition, agencies are not able to 

make changes to the programs they manage, limiting their ability to deliver 

results.
Appendix II

Focus Group Participants Agreed GPRA 

Provides a Framework for Federal Agencies 

to Become More Results Oriented

Page 126 GAO-04-38 Results-Oriented Government

Challenges in 

Implementing and 

Overseeing GPRA 

Activities

Participants stated that agencies face significant challenges in complying 

with GPRA-related activities. Focus group participants also agreed 

Congress does not appear to use agencies’ performance information when 

making budget decisions. In carrying out GPRA-related activities, 

managers find it difficult to identify performance indicators and establish 

program goals that are results oriented, as required by GPRA. In addition, a 

few participants stated that they lack the support from senior management 

to work on GPRA activities. Changes in administration also tend to disrupt 

progress made by agencies in support of GPRA.

Performance Information 

Not Used

Participants strongly believed that Congress does not take into account 

agencies’ performance information when overseeing agency activities and 

making budget decisions. As a result, there is a negative effect on how 

agencies view their efforts in producing GPRA plans and reports—many 

believe that they are less worthwhile than the effort and resources 

invested. On the other hand, participants perceive the budget process 

strictly as a political exercise, and it is unclear how useful performance 

information can be in this context.

Complexity of Establishing 

Results-Oriented Goals and 

Measuring Performance

Participants stated that establishing results-oriented goals and identifying 

performance indicators are generally complex undertakings. Participants 

agreed that they often feel as if they were trying to measure the 

immeasurable, not having a clear understanding of which performance 

indicators could accurately inform the agency how it is carrying out a 

specific activity. And while agencies generally try to improve the indicators 

from one year to another, in doing so, they generally lose their ability to 

develop trend information to track progress made over time.

Participants also mentioned that there appears to be a disconnect between 

some federal programs that generally produce results over a longer time 

period and GPRA’s requirement that agencies report annually on their 

progress towards achieving their goals. Participants stated that federal 

programs, especially those that are research-based, often take years to 

achieve the full scope of their work. Consequently, and for reasons that 

range from lack of performance data to an absence of short-term 

outcomes, it could appear as though resources invested in carrying out 

their activities led to no results in the short run.
Appendix II

Focus Group Participants Agreed GPRA 

Provides a Framework for Federal Agencies 

to Become More Results Oriented

Page 127 GAO-04-38 Results-Oriented Government

Focus group participants generally agreed that in cases where third parties, 

such as states or localities, implement federal programs, some agencies 

face challenges in obtaining timely performance data from relevant partner 

organizations. In some instances, federal managers have trouble 

identifying what the government’s contribution has been to a specific 

outcome. While the experts generally attributed this to agencies not doing 

a good job at setting appropriate goals and the corresponding measures 

and objectives not being clear enough, managers stated that their lack of 

control over grantees and the strict reporting deadlines imposed by GPRA 

were factors that worked against their efforts to deliver results.

Managing Strategically with 

Frequently Changing 

Priorities

Some participants stated that it is difficult for them to manage strategically, 

given the frequently changing priorities that come with changes in 

administrations. While GPRA requires an agency to develop a strategic 

plan at least every 3 years to cover the following 5-year period, participants 

agreed that it makes little sense to update it shortly before a new 

administration is scheduled to take office. In addition, changes in political 

leadership generally result in a new agenda with new objectives. These 

changes force agencies to revise their plans, management initiatives, and 

strategies, which translate into additional GPRA-related work, generally 

characterized by focus group participants as a burden agency staff must 

add to their normal work load.

Lack of Top Leadership 

Support

Some participants stated that they often encounter resistance from agency 

leadership to endorse GPRA-related activities. In some instances, seniorlevel support for GPRA falters or is nonexistent. Participants attributed 

this to the reluctance within some federal agencies to think about 

outcomes and performance. Some focus group participants stated that in 

some instances high-level managers are somewhat averse to being held 

accountable for the results of programs they run.

Suggestions to Address 

GPRA Challenges

To address these challenges, focus group participants made the following 

suggestions.
Appendix II

Focus Group Participants Agreed GPRA 

Provides a Framework for Federal Agencies 

to Become More Results Oriented

Page 128 GAO-04-38 Results-Oriented Government

Congress Should Clarify 

How Performance 

Information Could Be More 

Useful 

Congressional staff should provide guidance on agencies’ GPRA 

submissions—specifically, how information could be presented in the 

reports to make it more useful in the decision-making process. They could 

also communicate to agencies how the performance information is being 

used, so that agencies do not perceive their data gathering efforts as 

inconsequential. Additionally, in using performance information to make 

budget decisions, Congress should consider the results achieved by 

agencies in addition to results not achieved.

The Administration and 

OMB Need to Reinforce the 

Value of GPRA as a 

Management Tool

Agencies should embrace GPRA as a management tool, not just an external 

requirement that is separate from their day-to-day activities. To this end, 

the administration and OMB need make sure agency officials understand 

how GPRA can be further integrated with other management initiatives.

Agency Guidance on GPRA 

Should Recognize Diversity 

of Federal Agencies 

OMB’s guidance to agencies on how to implement GPRA should recognize 

that one-size-fits-all approaches are unlikely to be useful. OMB should also 

afford agencies some flexibility by simplifying the reporting process. For 

example, some participants believed that not everything needed to be 

measured. OMB should make exceptions for unique situations and 

programs, e.g., science programs, and it could consider providing multiyear 

funding for them.

OMB Should Publish a 

Governmentwide 

Performance Report 

OMB should commit to regularly publishing a governmentwide 

performance report that would articulate the government’s 

accomplishments to the public. It would also be useful if it singled out 

higher-performing programs so agencies could use them as models to guide 

their planning efforts.

Agencies Need to Obtain 

and Commit Resources to 

Carry Out GPRA-Related 

Activities 

Agency leadership needs to ensure that staff members have the necessary 

resources to work on GPRA-related activities. In addition, they need to 

invest resources to train staff, including political appointees, on GPRA and 

the benefits of linking budget to performance.
Appendix II

Focus Group Participants Agreed GPRA 

Provides a Framework for Federal Agencies 

to Become More Results Oriented

Page 129 GAO-04-38 Results-Oriented Government

Timing of GPRA Reports 

Should Take into Account 

Changes of Administration

Under GPRA, agencies are to update their strategic plans every 3 years. 

However, this effort can be wasted. Given that federal administrations 

generally span at least 4 years, participants suggested that the required 

update be changed to every 4 years to correspond with new presidential 

terms.

Agencies Should Share 

Experiences on How to 

Address Common Problems

Agencies should collaborate more to develop strategies to address difficult 

issues, such as how to identify performance indicators and measure agency 

contributions to specific outcomes. It would be useful if more agencies 

created structured forums for managers to share experiences, talk about 

effective practices, and share solutions.
Page 130 GAO-04-38 Results-Oriented Government

Appendix III

Observations on Agencies’ Strategic Plans ApendxiI

Under GPRA, strategic plans are the starting point and basic underpinning 

for results-oriented management. One of our objectives was to assess the 

changes in the overall quality of agencies’ goals, strategies, and data 

articulated in their strategic plans. To meet this objective, we judgmentally 

selected six agencies—Education, DOE, HUD, SBA, SSA, and DOT—using 

criteria such as agency size, primary program types, and previous GAO 

reviews. To assess the overall quality and improvements made to the 

agencies’ strategic plans, we relied on requirements contained in GPRA and 

accompanying committee report language,1

 guidance to agencies from 

OMB for developing strategic plans,2 previous GAO reports and 

evaluations,3 and our knowledge of agencies’ operations and programs. In 

conducting our reviews, we compared our assessments of agencies’ 

current strategic plans to our assessments of draft plans from fiscal year 

1997.4

 A more detailed discussion of our scope and methodology can be 

found in appendix I.

Required Elements of 

Agency Strategic Plans

GPRA requires an agency’s strategic plan to contain six key elements: 

1. A comprehensive agency mission statement. The agency mission 

statement should concisely summarize what the agency does, as 

required by law, presenting the main purposes for all its major 

functions and operations. According to OMB guidance issued in 2002, a 

mission statement is brief, defining the basic purpose of the agency, and 

corresponds directly with an agency’s core programs and activities. 

The program goals should flow from the mission statement as well. 

The federal government’s adaptive responses over time to new needs 

and problems have contributed to fragmentation and overlap in a host 

of program areas, such as food safety, employment training, early 

childhood development, and rural development, which could limit the 

1

Government Performance and Results Act of 1993, Committee on Governmental Affairs, 

United States Senate, S. Rpt. No. 58, 103d Cong. 1st Sess. (1993).

2

Office of Management and Budget, Circular No. A-11, Part 6, Preparation and Submission 

of Strategic Plans, Annual Performance Plans, and Annual Program Performance Reports

(Washington, D.C.: June 2002). 

3

U.S. General Accounting Office, Agencies’ Strategic Plans Under GPRA: Key Questions to 

Facilitate Congressional Review, GAO/GGD-10.1.16 (Washington, D.C.: May 1, 1997).

4

GAO/HEHS-97-176R; GAO/RCED-97-199R; GAO/RCED-97-208R; GAO/HEHS-97-179R;

GAO/RCED-97-205R; and GAO/RCED-97-224R.
Appendix III

Observations on Agencies’ Strategic Plans

Page 131 GAO-04-38 Results-Oriented Government

overall effectiveness of the federal effort. The mission statement helps 

to distinguish agencies’ roles from one another and reduce the overlap 

and identify areas needing coordination and collaboration.

2. Agencywide long-term goals and objectives. General goals and 

objectives—or strategic goals—explain what results are expected from 

the agency’s major functions and when to expect those results. Thus, 

such goals are an outgrowth of the mission and are very often results 

oriented. OMB guidance states that the goals should be defined in a 

manner that allows a future assessment to be made on whether the goal 

was or is being achieved. General goals should predominately be 

outcomes, and are long-term in nature.

3. Approaches or strategies to achieve goals and objectives. 

Strategies help in aligning an agency’s activities, core processes, and 

resources to support achievement of the agency’s strategic goals and 

mission. Under GPRA, strategies are to briefly describe the operational 

processes, staff skills, and technologies, as well as the human, capital, 

information, and other resources needed. According to OMB guidance, 

descriptions should be brief, but more detailed data should be provided 

if a significant change in a particular means or strategy would be 

essential for goal achievement. In addition, the plan should summarize 

agencies’ efforts to provide high-quality and efficient training and skill 

improvement opportunities for employees. As we have reported 

previously, agencies’ planning processes should support making 

intelligent resource allocation decisions that minimize, to the extent 

possible, the effect of funding reductions on mission accomplishment.

4. A description of the relationship between long-term and annual 

goals. Under GPRA, agencies’ long-term strategic goals and objectives 

are to be linked to their annual performance plans and the day-to-day 

activities of their managers and staff. Without this linkage, Congress 

may not be able to judge whether an agency is making progress toward 

achieving its long-term goals. OMB guidance states that an updated and 

revised strategic plan should briefly outline (1) the type, nature, and 

scope of the performance goals being included in annual performance 

plans and (2) how these annual performance goals relate to the longterm, general goals and their use in helping determine the achievement 

of the general goals.

5. An identification of key external factors. Identification of key 

factors, external to the agency and beyond its control that could 
Appendix III

Observations on Agencies’ Strategic Plans

Page 132 GAO-04-38 Results-Oriented Government

significantly affect the achievement of the strategic goals, are important 

for Congress or the agencies to judge the likelihood of achieving the 

strategic goals and actions needed to better meet those goals. Such 

external factors could include economic, demographic, social, 

technological, or environmental factors. Information on these factors 

can be useful for goal setting and also for explaining results in the 

agency’s annual performance reports, including, when applicable, the 

reasons annual performance goals were not met. According to OMB 

guidance, if key factors cannot be identified, a statement of explanation 

should be included in the plan.

6. A description of program evaluations. Finally, strategic plans 

should include a description of completed program evaluations that 

were used in developing the strategic plan, and a schedule for future 

program evaluations. Program evaluations can be a potentially critical 

source of information for Congress and others in ensuring the validity 

and reasonableness of goals and strategies, as well as for identifying 

factors likely to affect performance. Such information can also be 

useful in explaining results in an agency’s annual performance report, 

including, when applicable, the reasons annual performance goals were 

not met, and identifying appropriate strategies to meet unmet goals. 

Program evaluations are defined in the act as objective and formal 

assessments of the results, impact, or effects of a program or policy. 

The evaluations include assessments of the implementation and results 

of programs, operating policies, and practices.

In addition to the six key elements, OMB guidance also states that agencies 

participating in crosscutting programs should describe in their strategic 

plans how the programs are related and how coordination will occur to 

support common efforts. Uncoordinated program efforts can waste scarce 

funds, confuse and frustrate program customers, and limit the overall 

effectiveness of the federal effort. OMB guidance also states that the 

strategic plan should include a brief description of any steps being taken to 

resolve mission-critical management problems. One purpose of GPRA is to 

improve the management of federal agencies. Therefore, it is particularly 

important that agencies develop strategies to address management 

challenges that threaten their ability to meet long-term strategic goals as 

well as this purpose of GPRA.

As shown in table 5, the majority of agencies have made progress in 

addressing the required elements of strategic planning under GPRA.
Appendix III

Observations on Agencies’ Strategic Plans

Page 133 GAO-04-38 Results-Oriented Government

Table 5: Agencies’ Progress in Addressing Required Elements of Strategic Planning under GPRA

Sources: GAO/GGD-10.1.16; GAO/HEHS-97-176R; GAO/RCED-97-199R; GAO/RCED-97-208R; GAO/HEHS-97-179R; GAO/RCED-97-205R; GAO/RCED-97-224R; and GAO analysis of U.S. Department of 

Education, Office of the Deputy Secretary, Planning and Performance Management Service, U.S. Department of Education Strategic Plan 2002-2007 (Washington, D.C.: 2002); Department of Energy, 2003 

Strategic Plan (Draft) (Washington, D.C.: 2003); Department of Transportation, U.S. Department of Transportation Draft Strategic Plan for Fiscal Years 2003-2008 (Washington, D.C.: 2003); Social Security 

Administration, Social Security Administration Strategic Plan 2003-2008 (Washington, D.C.: 2003); U.S. Department of Housing and Urban Development, Strategic Plan FY 2003-FY 2008 (Washington, D.C.: 

2003); and Small Business Administration, SBA Strategic Plan, FY 2001-FY 2006 (Washington, D.C.: 2000).

a

The 2003 plans for DOE and DOT were in draft form during the time of our review.

b

At the time of our review, the most recent SBA strategic plan was for fiscal years 2001-2008. SBA 

released a new strategic plan for fiscal years 2003-2008 in October 2003.

The remainder of this appendix discusses our observations on how the 

quality of each of the agencies’ strategic plans we reviewed has changed 

since the agencies submitted their first draft strategic plans in 1997. We did 

not independently verify or assess the information we obtained from 

agency strategic plans. If an agency chose not to discuss its efforts 

concerning elements in the strategic plan, it does not necessarily mean that 

the agency is not implementing those elements.

Element included in agency strategic plan?

Agency strategic plans Plan year

Mission 

statement

Long-term 

goals Strategies

Relationship 

between longterm goals and 

annual goals

External 

factors Evaluations

Department of Education 1997 X X X X X

2002 X X X X X

Department of Energy 1997 X X X

2003a X X X X X

Department of Housing and 

Urban Development

1997 X

2003 X X X X X

Small Business Administration 1997 X X X X

2001b X X X X X

Social Security Administration 1997 X X X X X X

2003 X X X X X X

Department of Transportation 1997 X X X

2003a X X X X X
Appendix III

Observations on Agencies’ Strategic Plans

Page 134 GAO-04-38 Results-Oriented Government

Observations on 

Changes in the Quality 

of Education’s 

Strategic Plan 

In our review of Education’s June 1997 draft strategic plan,5 we found that 

the plan generally complied with GPRA and included all but one of the six 

elements required by GPRA; it did not discuss how Education’s long-term 

goals and objectives would be related to its annual performance goals. 

Also, we observed that the plan presented a logical, fairly complete 

description of how Education intended to achieve its mission, but a few 

areas could have been improved. In comparison, Education’s 2002-2007 

strategic plan has improved on several areas we identified in our 1997 

review. However, we still found areas where Education could improve. 

Table 6 summarizes these findings. 

Table 6: Education’s Progress in Addressing Required Elements of Strategic Planning under GPRA

5

GAO/HEHS-97-176R.

Element of strategic planning Included in initial draft strategic plan Included in current strategic plan

Mission statement Yes. Mission statement clearly and briefly 

explained why the agency exists, what the 

agency does, and how it performs its work.

Yes. Mission statement is outcome 

oriented, comprehensive, covers all of the 

agency’s functions and activities, and is the 

same as in the 1997 draft plan.

Long-term goals Yes. These goals were related to the 

mission and were results oriented, but did 

not appear to reflect civil rights enforcement 

and monitoring responsibilities.

Yes. The goals have changed, but are 

related to each other and the mission and 

are results oriented. They now additionally 

reflect civil rights enforcement and 

monitoring responsibilities.

Strategies Yes. The plan outlined strategies to achieve 

goals overall and to hold managers 

accountable for achieving objectives, and 

generally described some of its resource 

requirements throughout the plan.

Yes. The plan includes strategies that are 

linked to the goals. Several strategies relate 

to resource alignment to achieve outcomes, 

but the actual resources required are not 

always specified.

Relationship between long-term goals and 

annual goals

No. Education did not discuss the 

relationship between its strategic plan goals 

and those to be included in its annual plan, 

but indicated this would be done once its 

annual plan was prepared.

Yes. Plan contains the annual performance 

measures and targets, which represent the 

annual performance goals, and aligns them 

with long-term goals, with which they have a 

logical relationship.

External factors Yes. The plan generally described factors 

outside program scope and responsibilities 

that could negatively affect Education’s 

ability to achieve goals, but factors were not 

directly linked to particular goals.

Yes. The plan adequately describes 

external factors that could affect achieving 

its goals and they are linked to the particular 

goals. The plan also briefly discusses 

strategies to ameliorate the effects of a 

number of these factors.
Appendix III

Observations on Agencies’ Strategic Plans

Page 135 GAO-04-38 Results-Oriented Government

Sources: GAO/HEHS-97-176R; U.S. Department of Education, Office of the Deputy Secretary, Planning and Performance Management Service, U.S. Department of Education Strategic Plan 2002-2007

(Washington, D.C.: 2002).

a

According to OMB Circular No. A-11, Part 6, Preparation and Submission of Strategic Plans, Annual 

Performance Plans, and Annual Program Performance Reports, June 2002, general goals, which are 

multiyear and long term, are synonymous with general objectives, and either term can be used. The 

objectives in Education’s strategic plan are such multiyear, long-term objectives, and are referred to in 

our report as the agency’s “long-term goals.” OMB’s Circular A-11 also indicates that some agencies 

include strategic goals in their strategic plan, which represent overarching statements of aim or 

purpose whose achievement cannot be determined and which can be used to group outcome or 

output goals. Education’s strategic goals meet this description.

Current Strategic Plan 

Strengths and 

Improvements from Fiscal 

Year 1997 Draft Strategic 

Plan 

Education’s current mission, “to ensure equal access to education and to 

promote educational excellence throughout the nation,” is the same 

comprehensive, outcome-oriented mission that was included in its 1997 

draft plan. All of Education’s functions and activities are covered by it. The 

plan’s long-term goals6

 are expressed so as to allow Education and 

Congress to assess whether they are being achieved. Moreover, in contrast 

to findings in our review of Education’s 1997 Draft Strategic Plan, 

Education’s civil rights responsibilities—including enforcing five civil right 

statutes that ensure equal educational opportunity for all students, 

regardless of race, color, national origin, sex, disability, or age—appear to 

be reflected, at least in part, in the current plan’s goals. For example, one 

long-term goal is to reduce the gaps in college access and completion 

among student populations differing by race/ethnicity, socioeconomic 

status, and disability while increasing the educational attainment of all. 

Another is to enhance the literacy and employment skills of American 

adults. Under the latter, the plan includes a strategy to work with state 

vocational rehabilitation agencies to ensure implementation of standards 

that will assist individuals with disabilities in obtaining high-quality 

Evaluations Yes. Education said it would provide 

detailed descriptions of supporting 

evaluations once it consulted with Congress, 

completed the strategic plan, and agreed on 

performance indicators. The plan indicated 

a commitment to using evaluations, listing 

evaluations it intended to use to develop 

sound measures, but did not describe the 

evaluations.

No. Because of the comprehensive 

revamping of Education’s strategic plan in 

2002, its program evaluation plan was 

completely restructured and was not 

released until it was included in the 2004 

annual plan,a

 which contains information on 

its new directions for program evaluation.

(Continued From Previous Page)

Element of strategic planning Included in initial draft strategic plan Included in current strategic plan

6

U.S. Department of Education, Office of the Deputy Secretary, Strategic Accountability 

Service, U.S. Department of Education FY 2004 Annual Plan (Washington, D.C.: March 

2003). This document represents Education’s performance plan for GPRA and will be 

referred to henceforth as the “annual plan.”
Appendix III

Observations on Agencies’ Strategic Plans

Page 136 GAO-04-38 Results-Oriented Government

employment outcomes. As in the past, some goals are targeted at results 

for which Education has limited direct control and are, instead, greatly 

influenced by third parties. However, Education recognizes this situation 

and shows its intent to work closely with its partners.

Education’s current plan provides some information on linking results and 

day-to-day activities within the department. For example, the plan says 

employees will be held accountable for implementation and success from 

top to bottom and senior officers will have performance contracts linked to 

the plan and be recognized for achieving results. In addition, the strategy 

to foster a customer service orientation by ensuring that states, districts, 

and other partners receive timely responses to inquiries; to assign senior 

officers to develop relationships with individual states; and to create a 

customer support team to respond to issues, seems logically linked to the 

day-to-day activities of managers and staff. However, the link between 

results and day-to-day activities is not apparent for most of the goals and 

their strategies.

The current plan includes several annual performance measures that are at 

least related to how well information technology is supporting strategic 

and program goals, as required by the Clinger-Cohen Act. For example, for 

its long-term goal to modernize the Federal Student Assistance (FSA) 

Programs and reduce their high-risk status, the plan contains a measure on 

the integration of FSA processes and systems that work together to support 

FSA program delivery functions. Commendably, Education includes some 

goals related to reducing agency program unintended negative effects, such 

as a goal and related measure to reduce the data collection and reporting 

burden.

Education’s current plan shows great improvement on recognizing and 

addressing external factors. Beyond adequately describing external factors 

that could affect achieving its goals and directly linking these factors to 

particular goals, the plan also briefly describes strategies to ameliorate the 

effects of a number of factors. For example, for an external factor on 

teacher certification under a goal on reading, the plan says that Education 

“will work with the states and national accreditation bodies to encourage 

the incorporation of research-based reading instruction into teacher 

certification requirements.” In addition, the plan includes strategies 

indicating that Education monitors some internal factors that could affect 

achievement of long term goals. Moreover, the plan recognizes the 

overarching critical external factor for Education—that it depends greatly 

on third parties who often control the results the department is trying to 
Appendix III

Observations on Agencies’ Strategic Plans

Page 137 GAO-04-38 Results-Oriented Government

achieve—and its strategies related to the goal-specific external factors 

often reflect this.

In our 1997 assessment of Education’s draft plan, we commented that 

although the plan identified several management challenges the 

department would face in the coming years, it provided little detail about 

them and how they would be addressed. In our January 2001 performance 

and accountability series, we identified four department-specific and two 

governmentwide challenges that we said Education needed to meet.7

 

Education’s current strategic plan includes some goals, measures, and 

strategies that could be used to address these challenges. For example, 

Education’s goal to “develop and maintain financial integrity and 

management and internal controls” and this goal’s strategies and measures 

are directly related to the management challenge we identified on 

improving financial management. One of the measures under this goal is 

“the achievement of an unqualified audit opinion,” an important issue in 

our identification of financial management weaknesses as a challenge in 

2001, and Education received an unqualified audit opinion in early 2003. 

Moreover, the current strategic plan includes goals and strategies meant to 

improve Education’s strategic human capital management and strengthen 

information technology security, two important governmentwide high-risk 

areas we identified in 2001.

Critical Strategic Planning 

Issues Needing Further 

Improvement

In our report on Education’s 1997 draft strategic plan, we found that, 

although the department had numerous crosscutting programs and 

activities, the plan had identified key interagency activities for some 

programs but not for others. For example, we said that the plan did not 

identify or discuss activities for postsecondary programs for which the 

Higher Education Act of 1965 required coordination. Education’s current 

plan includes an appendix, which highlights collaborative initiatives under 

each of the department’s strategic goals, with some activities related to 

7

U.S. General Accounting Office, Performance and Accountability Series: Major 

Management Challenges and Program Risks, Department of Education, GAO-01-245

(Washington, D.C.: January 2001). The January 2001 assessment was the last time we 

assessed the Department of Education under our Performance and Accountability Series 

before the release of the agency’s 2002-2007 Strategic Plan. We further reported in October 

2002 on how Education and other agencies reported responding to their management 

challenges and program risks. (U.S. General Accounting Office, Performance and 

Accountability: Reported Agency Actions and Plans to Address 2001 Management 

Challenges and Program Risks, GAO-03-225 (Washington, D.C.: Oct. 31, 2002).
Appendix III

Observations on Agencies’ Strategic Plans

Page 138 GAO-04-38 Results-Oriented Government

postsecondary education, including most of those mentioned in our 1997 

report. However, as the plan states, the appendix presents a brief overview 

of the highlights of some of its collaborative initiatives with other agencies.

In our 1997 review, we stated that some resource requirements were 

described throughout the plan. In the current plan, while a number of 

strategies under the department’s strategic goal to establish management 

excellence are related to aligning resources to achieve outcomes, the 

actual resources required—such as human, capital, and information—are 

not always specifically indicated. The exception is for information 

resources, for which a number of strategies discuss the information 

resources that will be required to address the related goal. For example, 

under the goal to develop and maintain financial integrity and management 

and internal controls, the plan says that Education will “implement a new 

financial system capable of producing timely and reliable financial data and 

reconcile systems to the general ledger.” Moreover, while the plan stresses 

accountability throughout, it only refers to providing the authority needed 

to achieve results once.8

 In addition, consideration of alternate strategies 

for achieving goals is not discussed.

In our 1997 review, we reported that Education said it would provide 

detailed descriptions of supporting evaluations once it consulted with 

Congress, completed the strategic plan, and agreed on performance 

indicators. The draft plan indicated Education’s commitment to using 

evaluations and listed evaluations and strategies it intended to use to 

develop sound measures, but did not describe the evaluations. The current 

plan does not include descriptions of supporting evaluations or a future 

program evaluation schedule. According to Education officials, this was 

not done because, with the complete revamping of the strategic plan based 

on passage of the No Child Left Behind Act of 2001,9

 Education was set on a 

course to totally restructure its evaluation program, but could not do so in 

time to include it in the strategic plan. Consequently, Education instead 

included information about its new directions for program evaluation 

studies and a schedule of evaluations in its 2004 annual plan. The schedule, 

however, lacked a timetable, except for stating whether the evaluations 

8

Within one of its strategies, the plan states that “managers will be given the freedom to 

manage and will be held accountable for results.”

9

Pub. L. No. 107–110, January 8, 2002. The No Child Left Behind Act of 2001 is a 

reauthorization of the Elementary and Secondary Education Act, one of the major pieces of 

authorizing legislation for the Department of Education.
Appendix III

Observations on Agencies’ Strategic Plans

Page 139 GAO-04-38 Results-Oriented Government

were new or continuing. The current strategic plan indicates, in some 

cases, the use or planned use of program evaluation findings to develop or 

revise components of the plan. For example, for long-term goals on various 

types of academic achievement, data from the National Assessment of 

Educational Progress were identified as having been used to set related 

targets.

Observations on 

Changes in the Quality 

of DOE’s Strategic Plan

Overall, DOE’s draft 2003 strategic plan meets the required elements of 

GPRA and has improved greatly over its 1997 draft strategic plan, as shown 

in table 7. DOE made improvements to its plan by establishing resultsoriented and measurable objectives, and addressing elements that were not 

included in the department’s 1997 plan, such as reporting on external 

factors. Although DOE has shown improvement, a few elements in the plan 

could still be enhanced. For instance, further improvement could be made 

to the mission statement so that it better addresses the department’s major 

activities, and additional information could be included in DOE’s strategies 

to achieve its goals, such as management accountability.

Table 7: DOE’s Progress in Addressing Required Elements of Strategic Planning under GPRA

Element of strategic planning Included in initial draft strategic plan Included in current draft strategic plan

Mission statement Yes. DOE’s mission was results oriented, 

met a public need, and covered the agency’s 

major activities.

Yes. DOE’s mission is results oriented and 

meets a public need, but it does not address 

the department’s major activities related to 

energy supply and conservation.

Long-term goals Yes. Long-term goals covered mission and 

major functions of the agency. Goals and 

objectives were results oriented but not 

measurable.

Yes. Long-term goals cover the mission and 

major functions of the agency. Objectives, 

referred to as intermediate goals, are results 

oriented and measurable.

Strategies Yes. The plan included strategies and 

measures to evaluate the results of the 

strategies, but was missing information on 

linkages to day-to-day activities, and the 

extent to which managers have the 

knowledge, skills, and abilities to implement 

the strategies.

Yes. Strategies to achieve each goal are 

included in the plan, along with intermediate 

goals to measure success, but information 

on linkages to day-to-day activities, and the 

extent to which managers have the 

knowledge, skills, and abilities to implement 

the strategies, is not included.

Relationship between long-term goals and 

annual goals

No. Relationship between the long-term 

goals and annual performance goals was 

missing in the draft plan.

Yes. The draft plan provides a brief 

description of the relationship between the 

long-term strategic goals and the annual 

performance goals.
Appendix III

Observations on Agencies’ Strategic Plans

Page 140 GAO-04-38 Results-Oriented Government

Sources: GAO/RCED-97-199R and Department of Energy, 2003 Strategic Plan (Draft), (Washington, D.C.: 2003).

Strategic Plan Strengths and 

Improvements from Fiscal 

Year 1997 Plan

In its 2003 draft strategic plan, DOE has made significant improvements in 

developing measurable objectives, referred to in its plan as intermediate 

goals. GAO’s review of DOE’s 1997 draft strategic plan found that 

objectives related to DOE’s long-term goals were stated in ways that would 

make it difficult to measure whether they were being achieved. In the 2003 

draft plan, the objectives are stated in ways that will enable DOE to 

measure its progress in achieving goals. For example, to meet the goal of 

enhancing energy security through various means, one of DOE’s affiliated 

objectives is to ensure that throughout DOE’s 25-year planning period, the 

Strategic Petroleum Reserve is ready to supply oil at a sustained rate of 4.3 

million barrels per day for 90 days within 15 days notice by the President.

In addition, DOE has improved its draft 2003 strategic plan by including 

elements that were not included in its 1997 draft plan. These elements 

consisted of (1) identifying external factors and (2) describing the 

relationship between long-term and annual goals. For each of its long-term 

goals, DOE identified external factors, such as reduced funding, lack of 

scientific talent, and unpredictable technological developments, that could 

affect its ability to achieve its goals. The strategic plan also included a 

description of the relationship between the long-term strategic goals and 

the annual performance goals. The plan included a diagram that showed an 

example of a strategic goal, its associated objectives, and how they are 

related to the annual performance goals and targets. However, the plan 

could be improved if all annual performance goals were discussed in the 

plan and linked to each strategic goal so that it would be clear how annual 

performance goals would be used to gauge performance. DOE staff stated 

that a description of all actual annual performance goals was not 

something that they thought should be included in the strategic plan 

because the annual goals differ each year.

Finally, our past and current reviews of DOE’s 1997 and 2003 draft strategic 

plans found that DOE addressed performance and accountability 

External factors No. Key external factors were not 

addressed in the draft plan.

Yes. Key external factors and how they 

could affect the ability to achieve each goal 

are identified in the draft plan.

Evaluations No. The impact of program evaluations on 

the development of strategic goals was not 

included in the draft plan.

No. The impact of program evaluations on 

the development of strategic goals is not 

discussed thoroughly.

(Continued From Previous Page)

Element of strategic planning Included in initial draft strategic plan Included in current draft strategic plan
Appendix III

Observations on Agencies’ Strategic Plans

Page 141 GAO-04-38 Results-Oriented Government

challenges that we had previously identified. In January 2003, we identified 

six areas where DOE’s management attention was needed: (1) addressing 

security concerns, (2) revitalizing infrastructure, (3) improving contract 

management, (4) managing the nuclear weapons stockpile, (5) cleaning up 

radioactive and hazardous wastes, and (6) enhancing leadership in meeting 

energy needs. All areas were addressed in the 2003 draft strategic plan, 

with the exception of improving contract management. For example, for 

the challenge of enhancing leadership in meeting energy needs, one of the 

intermediate goals requires DOE to develop and demonstrate technologies 

that can reduce emissions by more than 70 metric tons of carbon and 

equivalent greenhouse gases by 2012. 

Critical Strategic Planning 

Issues Needing Further 

Improvement

There are three elements in DOE’s draft 2003 strategic plan that requires 

further improvement. To begin with, although DOE’s mission is results 

oriented, it was revised from the 1997 draft strategic plan and does not 

address the department’s major activities related to energy supply and 

conservation. These activities account for approximately 10 percent of 

DOE’s $23.4 billion fiscal year 2004 budget request. Our review of the 1997 

draft strategic plan found that DOE’s mission addressed all of its major 

activities.

In addition, the impact of program evaluations on the development of 

strategic goals could be discussed in greater detail. The strategic plan 

stated that internal, GAO, and the Inspector General (IG) evaluations were 

used as resources to develop the draft strategic plan, but specific program 

evaluations were not identified. A schedule of future program evaluations 

was also not discussed in the strategic plan. According to DOE, there is no 

plan to include a table of reports and evaluations that were used to develop 

the goals for the strategic plan because the evaluations were from a prior 

administration, and when the administration changes, it usually does not 

use evaluations from past administrations.
Appendix III

Observations on Agencies’ Strategic Plans

Page 142 GAO-04-38 Results-Oriented Government

DOE could also enhance the strategies included in its plan by providing 

information on how goals are linked to the department’s day-to-day 

activities, and the extent to which managers have the knowledge, skills, 

and abilities to implement the strategies. None of this information was 

included in the 1997 and 2003 draft strategic plans. According to DOE 

officials, in drafting the plan, their goal was to keep the plan at a high level 

and this additional information would require more detail than what is 

needed for a strategic plan. For example, one official stated that a 

description of linkages to day-to-day activities was not discussed in the 

strategic plan because it would conflict with the performance plan and 

budget justification, which is where the information can be found. As we 

have stated previously, without this information, it is difficult to judge 

DOE’s likelihood of success in achieving the goals or the appropriateness 

of the strategies.10

Finally, DOE’s draft strategic plan could be improved by identifying 

programs and activities that are crosscutting or similar to those of other 

agencies. In the 2003 draft plan, crosscutting activities are only identified 

for one goal related to science. According to one DOE official, as the 

strategic goals were being developed, DOE staff took crosscutting 

activities into consideration. As we stated in 1997, unless DOE addresses 

crosscutting issues in its plan, Congress cannot be assured that federal 

programs are working effectively.

Observations on 

Changes in the Quality 

of HUD’s Strategic Plan 

Overall, HUD has made progress in developing its strategic plan for fiscal 

years 2003 through 2008 as required under GPRA. In 1997, we stated that 

HUD’s draft strategic plan did not cover the six components required under 

GPRA.11 HUD’s fiscal year 2003-2008 strategic plan addressed several 

issues we had previously identified, such as making sure the mission 

statement is linked to the department’s major operations and functions; 

including outcome-related goals and objectives for the department’s 

functions and operations; generally providing a description of how it will 

achieve its goals and objectives; identifying key external factors affecting 

the achievement of departmental goals; and explaining how HUD will 

coordinate with other agencies to address crosscutting issues. However, 

HUD could improve its strategic plan by explaining the relationship 

10GAO/RCED-97-199R.

11GAO/RCED-97-224R.
Appendix III

Observations on Agencies’ Strategic Plans

Page 143 GAO-04-38 Results-Oriented Government

between the long-term and intermediate performance measures listed for 

each strategic goal and discussing how it used program evaluations to 

develop the plan. Table 8 summarizes these findings.

Table 8: HUD’s Progress in Addressing Required Elements of Strategic Planning under GPRA

Sources: GAO/RCED-97-224R and U.S. Department of Housing and Urban Development, Strategic Plan FY 2003-FY 2008. (Washington, D.C.: 2003).

Strategic Plan Strengths and 

Improvements from Fiscal 

Year 1997 Draft Plan

In its most recent strategic plan for fiscal years 2003-2008, HUD has made 

progress in crafting a mission statement that generally covers its major 

program functions, operations, and relevant statutes that authorize its 

programs. In addition, the current plan builds upon the strength of its first 

draft strategic plan by including the strategic objectives that cover the 

department’s major functions and operations. 

Element of strategic planning Included in initial draft strategic plan Included in current strategic plan

Mission statement No. HUD’s mission statement did not cover 

the major program functions and operations 

of the agency.

Yes. HUD’s mission statement covers the 

agency’s major program functions and 

operations and relevant statutes.

Long-term goals Yes. The plan included eight strategic 

objectives, and they generally covered the 

department’s major functions and 

operations.

Yes. The plan includes long-term goals that 

cover the major programs and functions of 

the agency.

Strategies No. The strategic plan lacked an adequate 

description of how its strategic objectives 

would be achieved.

Yes. The strategic plan discusses the 

means and strategies to address each 

strategic goal, including plans to address 

human capital issues critical to carrying out 

its mission.

Relationship between long- term goals and 

annual goals

No. HUD’s strategic plan provided limited 

examples of annual performance goals 

under each of its strategic objectives, but it 

did not describe the relationship between 

them.

Yes. However, some long-term performance 

measures do not appear to have 

corresponding intermediate measures, and 

in other instances it is not clear how HUD 

will measure progress towards its goals

External factors No. HUD briefly discussed the external 

factors in its draft strategic plan without 

linking them to specific strategic objectives.

Yes. HUD describes the external factors 

that could affect achieving its strategic 

objectives.

Evaluations No. HUD’s strategic plan did not include 

information on program evaluations.

No. HUD does not describe how program 

evaluations were used to prepare its 

strategic plan. Although it mentions that 

program evaluations will be used to advance 

key policy objectives, it states that there are 

no fixed timetables for when these 

evaluations will take place.
Appendix III

Observations on Agencies’ Strategic Plans

Page 144 GAO-04-38 Results-Oriented Government

In contrast to its draft strategic plan of 1997, HUD’s most recent plan 

provides a general description of how its strategic objectives will be 

achieved. For example, HUD lists the means and strategies following each 

strategic goal and supporting objectives to describe how it will carry out its 

activities. In addition, the strategic plan identifies a few legislative and 

regulatory changes HUD will pursue to meet its objectives, such as a new 

tax credit for developers of affordable housing and expanded eligibility for 

the Assisted Living Conversion Program. To carry out its work, HUD 

acknowledges in its plan that it needs to recruit and retain its current 

workforce to ensure the proper skills and abilities needed to carry out its 

mission.

HUD’s most recent strategic plan also describes key factors external to the 

department and beyond its control that could significantly affect the 

achievement of its objectives. For example, for its goal of Promoting 

Decent and Affordable Housing, HUD identifies the impact of broad 

economic factors on opportunities for low-income workers as a factor that 

will affect the department’s ability to assist renters that depend on HUD’s 

programs to make progress towards self sufficiency. HUD’s strategic plan 

also provides a general description of current program evaluations and 

mentions that the results of these evaluations will support key policy 

objectives within the department.

Addressing a shortcoming of its draft strategic plan in 1997, HUD’s current 

plan generally explains how it will coordinate with other agencies to 

address crosscutting problems. For example, the Interagency Council on 

the Homeless (ICH), created by the Secretaries of HUD, the Department of 

Health and Human Services (HHS), and Veterans Affairs (VA), will continue 

to work to identify the obstacles homeless people face to enroll in the main 

service programs and recommend specific changes—legislative, policy, and 

procedural—that would make federal supportive service programs more 

accessible to them.
Appendix III

Observations on Agencies’ Strategic Plans

Page 145 GAO-04-38 Results-Oriented Government

Lastly, HUD improved its discussion of how it plans to address 

performance and accountability challenges we have raised. In January 

2003, we reported that programmatic and financial management 

information systems and human capital issues were HUD’s performance 

and accountability challenges.12 While in the past HUD acknowledged 

some of these problems and described how they would be addressed, its 

plans for management reform were not fully integrated into its draft 

strategic plan. In contrast, one of HUD’s strategic goals in its current 

strategic plan, “Embrace High Standards of Ethics, Management, and 

Accountability,” lists specific actions the department will take to address 

these challenges.

Critical Strategic Planning 

Issues Needing Further 

Improvement

HUD has made progress in linking its strategic objectives to both long-term 

goals and intermediate measures in its strategic plan; however, long-term 

performance measures are not consistently linked to a corresponding 

intermediate measure, making it difficult for the reader to understand how 

the department will measure progress towards its strategic goals. For 

example, HUD projects that the percentage of architects and builders 

indicating awareness of the design and construction requirements of the 

Fair Housing Act will increase through fiscal year 2008. However, HUD 

does not mention in the plan how many architects or engineers it will 

survey to establish a baseline of awareness or to gauge progress made 

towards this goal in subsequent years. HUD officials explained that only 

those long-term goals that are critical to being achieved within the next 1-2 

years have corresponding intermediate performance measures. Therefore, 

it is somewhat unclear how HUD plans to assess progress made towards its 

broader goal of “Ensure Equal Opportunity in Housing.”

Similarly, the strategic plan does not always provide a clear picture of how 

HUD will be able to measure progress towards its strategic goals. For 

example, the plan states that HUD’s Community Development Block 

Grants (CDBG) Program will create or retain 400,000 jobs by fiscal year 

2008; HUD’s intermediate measure for fiscal year 2004 is 84,000 jobs. These 

estimates are based on the average jobs created per grant dollar reported 

by grantees. However, HUD does not mention how it will be able to discern 

between those jobs created by CDBG and those created by other means.

12U.S. General Accounting Office, Major Management Challenges and Program Risks: 

Department of Housing and Urban Development, GAO-03-103 (Washington, D.C.: January 

2003).
Appendix III

Observations on Agencies’ Strategic Plans

Page 146 GAO-04-38 Results-Oriented Government

Similar to our previous finding, HUD’s current strategic plan does not 

describe how program evaluations were used to develop its strategic goals 

or other components of its strategic plan, and does not include a schedule 

for future evaluations. While the plan mentions that program evaluations 

will be used to advance key policy objectives, it states that there are no 

fixed timetables for when these evaluations will take place.

Observations on 

Changes in the Quality 

of SBA’s Strategic Plan

In our review of SBA’s March 1997 draft strategic plan,13 we found that the 

plan did not meet two of the six requirements set forth by GPRA: (1) a 

discussion of the relationship between the long-term goals and objectives 

and the annual performance goals and (2) a description of how program 

evaluations were used to establish or revise strategic goals and a schedule 

for future program evaluations. In addition, the four elements the plan 

contained could have better conformed to GPRA’s requirements and OMB 

guidance. In comparison, SBA’s 2001-2006 strategic plan has improved on 

several areas we identified in our 1997 review.14 However, we still found 

areas that SBA could improve. Table 9 summarizes these findings.

Table 9: SBA’s Progress in Addressing Required Elements of Strategic Planning under GPRA

13GAO/RCED-97-205R.

14At the time of our review, the most recent SBA strategic plan was for fiscal years 2001-

2008. SBA released a new strategic plan for fiscal years 2003-2008 in October 2003.

Element of strategic planning Initial draft strategic plan Current strategic plan

Mission statement Yes. The mission statement was results 

oriented. However, it did not directly 

incorporate key aspects of SBA’s legislative 

mandate and it did not encompass one of 

SBA’s significant activities.

Yes. SBA’s mission statement now 

specifically mentions its mandate and 

encompasses SBA’s disaster loan program 

to families and businesses.

Long-term goals Yes. Generally the strategic goals covered 

the major functions and operations of SBA. 

Most of SBA’s strategic goals were 

expressed as processes, not as outcomes.

Yes. The strategic goals are outcome 

oriented and cover the major functions and 

operations of SBA.

Strategies Yes. However, the strategies were too 

vague or general to assess whether or not 

they would help achieve SBA’s goals. Also, 

the strategies and goals could have been 

linked more explicitly.

Yes. The strategies will generally help SBA 

achieve its goals. The strategies are now 

listed by strategic goal and related objective. 

The plan discusses some of the resources 

SBA needs to achieve its goals.
Appendix III

Observations on Agencies’ Strategic Plans

Page 147 GAO-04-38 Results-Oriented Government

Sources: GAO/RCED-97-205R and Small Business Administration, SBA Strategic Plan, FY 2001 – FY 2006 (Washington, D.C.: 2000).

Strategic Plan Strengths and 

Improvements from Fiscal 

Year 1997 Plan

SBA’s current mission statement is an improvement from the one contained 

in its 1997 draft strategic plan. In our July 1997 report on SBA’s draft 

strategic plan, we noted that the mission statement could be improved by 

more directly incorporating key aspects of SBA’s legislative mandate to aid, 

counsel, assist, and protect the interests of small businesses. In addition, 

the mission statement did not encompass one of SBA’s significant 

activities–making loans to individuals. The mission statement in the 2001-

2006 strategic plan now includes both of these items.

The long-term, or strategic, goals of SBA’s 2001-2006 strategic plan are 

(1) helping small businesses succeed, (2) helping Americans recover from 

disasters, and (3) modernizing SBA. These three strategic goals are 

outcome oriented. We stated in our previous report on SBA’s draft strategic 

plan that only two of SBA’s seven strategic goals described outcomes. The 

rest of the goals were expressed as processes.

SBA’s 2001-2006 strategic plan shows significant improvement in its 

strategies. In 1997 we stated that the plan could be improved by making 

the linkage between the strategies and goals/objectives more explicit. 

Objectives were listed as a group, followed by the strategies, which were 

also listed as a group. The 2001-2006 plan describes the strategies, 

objective by objective, making the linkage clear. The strategies contained 

in SBA’s 1997 plan consisted entirely of one-line statements and most were 

too vague or general to enable an assessment of whether or not they would 

Relationship between long-term goals and 

annual goals

No. The linkage between proposed 

performance measures, strategies, and 

objectives was unclear.

Yes. The plan lists performance measures 

by objectives for each strategic goal. Logic 

models show the relationship between 

measures and outcomes.

External factors Yes. The plan listed a number of external 

factors. A discussion of how the external 

factors would be taken into account when 

assessing progress toward goals was not 

included in the plan.

Yes. The plan lists several external factors. 

Generally, strategies to ameliorate the 

effects of these factors are included. 

However, SBA does not discuss how 

external factors will be taken into account 

when assessing progress toward goals.

Evaluations No. The plan did not describe how program 

evaluations were used to establish or revise 

strategic goals or include a schedule for 

future program evaluations.

No. The plan states that lessons learned 

from SBA’s program evaluations have 

influenced its strategic direction. Examples 

of future program evaluations are given, but 

a specific schedule of when these 

evaluations are to occur is not included.

(Continued From Previous Page)

Element of strategic planning Initial draft strategic plan Current strategic plan
Appendix III

Observations on Agencies’ Strategic Plans

Page 148 GAO-04-38 Results-Oriented Government

help achieve the goals and objectives in the plan. While many of the 

strategies listed in SBA’s 2001-2006 plan are only one- or two-sentence 

statements, all of the strategies seem to aid in achieving SBA’s goals and 

objectives. For example, one strategy for the objective of providing 

entrepreneurial development assistance is to train SBA personnel in 

outreach and business development approaches. This strategy would 

presumably aid in making staff more competent at helping small businesses 

develop. The current plan also includes some of the resources needed to 

achieve SBA’s goals. These resources include a table showing SBA’s fiscal 

year 2001 budget request and its plans to modernize its information 

technology systems.

The 2001-2006 strategic plan also makes a clearer connection between 

strategic goals and annual performance measures that will be used to gauge 

progress in achieving strategic goals. SBA shows this relationship by 

linking its performance output measures with the intended outcomes of 

SBA’s programs for two of the strategic goals. For the other strategic goal, 

“Modernizing SBA,” SBA lists the performance measures it will use by 

objective. For example, the performance measure “personnel trained or 

retrained” will gauge progress in achieving SBA’s human capital 

investments objective. This shows improvement over the draft strategic 

plan, which listed the performance measures as a group without showing 

how they were related to SBA’s strategic goals.

Similar to its 1997 draft plan, SBA’s 2001-2006 plan specifies external 

factors that could affect the achievement of its strategic goals. The plan 

lists eight external factors associated with two of its three strategic goals; 

no external factors were associated with the third goal. The plan includes 

strategies that seem to be intended to mitigate the effect of six of these 

external factors, while two of the external factors, congressional support 

and public support, do not seem to be addressed. An example of an 

external factor that seems to be addressed by a strategy is economic 

conditions, listed for the strategic goal “helping small businesses succeed.” 

One of the strategies for this strategic goal is to determine economic trends 

and conditions. SBA states that it tracks economic trends that affect small 

businesses and the contribution small businesses make to the economy. 

SBA then brings these data to the attention of policymakers.

We noted in our 1997 report that SBA did not explicitly address the 

relationship of SBA’s activities to similar activities in other agencies and 

provided no evidence of coordination. In contrast, SBA’s current strategic 

plan includes a section on crosscutting issues. This section contains 
Appendix III

Observations on Agencies’ Strategic Plans

Page 149 GAO-04-38 Results-Oriented Government

discussions of innovation and research assistance, international trade 

assistance, business development assistance, veterans affairs, and disaster 

assistance. One example of coordination is the U.S. Export Assistance 

centers, which combine the trade-promotion and export-finance resources 

of SBA, the Department of Commerce, the Export-Import Bank, and in 

some locations, the Agency for International Development and the 

Department of Agriculture.

Finally, SBA’s 2001-2006 strategic plan generally addresses performance 

and accountability challenges that we have previously identified. For 

example, we noted in our January 2001 report on major management 

challenges and program risks15 that SBA needed to continue to improve 

oversight of its lending partners to correct oversight weaknesses. The plan 

identifies lender oversight as a management challenge and states that SBA 

has developed and implemented a safety and soundness oversight program 

for Small Business Lending companies, institutionalizing the process 

through the Office of Lender Oversight. This is an improvement over SBA’s 

draft strategic plan, which we reported did not clearly address previously 

identified management problems.

Critical Strategic Planning 

Issues Needing Further 

Improvement

In 1997, we noted that SBA’s draft strategic plan did not include a 

discussion of how the external factors would be taken into account when 

assessing progress toward goals. This observation holds true for the 

current strategic plan. For the external factor mentioned above, economic 

conditions, the plan states that if the economy remains strong, surety bond 

guaranties will remain constant or decrease, but if the economy 

deteriorates, demand will increase. However, the plan does not state how 

SBA will assess success or failure in meeting its goals in relation to this 

factor.

In its 1997 draft plan, SBA did not describe how program evaluations were 

used to establish or revise strategic goals or include a schedule for future 

program evaluations. In the 2001-2006 plan, SBA states, “We have used 

lessons learned in our performance monitoring and program evaluations to 

influence the strategic direction contained in this plan.” The plan includes 

findings from six completed program evaluations; however, no further 

detail is given as to how these program evaluations were used to establish 

15U.S. General Accounting Office, Major Management Challenges and Program Risks: 

Small Business Administration, GAO-01-260 (Washington, D.C.: January 2001).
Appendix III

Observations on Agencies’ Strategic Plans

Page 150 GAO-04-38 Results-Oriented Government

or revise the strategic goals. While the current plan gives examples of 

future program evaluations, such as conducting a benchmark study on the 

HUBZone program to assess the changes in employment and investment in 

distressed urban and rural communities, it does not include a schedule of 

future evaluations. SBA states that for the next several years, the agency 

plans to systematically review programs that offer the most financial risk to 

the government and also the programs that can offer tips on how to 

improve efforts.

Observations on 

Changes in the Quality 

of SSA’s Strategic Plan

SSA’s strategic plan for 2003-2008 is well structured and contains all of the 

required elements under GPRA. In 1997, we noted that SSA’s draft strategic 

plan contained all six required components, but suggested a number of 

ways it could be strengthened. SSA has addressed some of the issues we 

previously identified, such as associating specific programs with goals and 

identifying external factors that may affect goal achievement. SSA could 

further improve its strategic plan through (1) ensuring that its strategic 

objectives will assist SSA in achieving its strategic goals, (2) explicitly 

describing the effect of external factors on goal attainment, (3) providing 

timetables or schedules for achieving results, (4) providing details on how 

each performance and accountability challenge will be addressed, 

(5) clearly explaining how program evaluations were used in formulating 

the strategic plan, and (6) discussing the manner in which SSA has 

coordinated with other agencies, especially those that serve the same 

beneficiaries. Table 10 summarizes SSA’s progress in addressing the 

required elements of GPRA.

Table 10: SSA’s Progress in Addressing Required Elements of Strategic Planning under GPRA

Element of strategic planning Included in initial draft strategic plan Included in current strategic plan

Mission statement Yes. SSA’s mission statement was 

appropriate and reflective of its new status 

as an independent agency.

Yes. SSA’s mission statement has not 

changed substantially.

Long-term goals Yes. Long-term goals were established. 

However, the relationship between longterm goals and specific programs was 

unclear and did not identify the results to be 

achieved.

Yes. The goals’ relationship to specific 

programs is more defined, but SSA’s goal for 

achieving solvency of the social security 

system is ambitious, given SSA’s mission 

and responsibilities. Key outcomes are 

identified for each goal. SSA’s major 

management challenges are not all clearly 

linked to individual goals or objectives.
Appendix III

Observations on Agencies’ Strategic Plans

Page 151 GAO-04-38 Results-Oriented Government

Sources: GAO/HEHS-97-179R) and Social Security Administration, Social Security Administration Strategic Plan 2003-2008, (Washington, D.C.: 2000).

Strategic Plan Strengths and 

Improvements from Fiscal 

Year 1997 Plan

SSA’s mission statement changed very little between 1997 and 2003. OMB 

Circular A-11 notes that the mission statement should be brief and define 

the basic purpose of the agency, with particular focus on its core programs 

and activities. SSA’s statement conforms to this guidance—it reads, “To 

advance the economic security of the nation’s people through 

compassionate and vigilant leadership in shaping and managing America’s 

social security programs.”16 

Strategies Yes. The strategic plan was generally 

complete with regard to processes and 

technology, but did not include timetables or 

schedules. Some of the success was 

predicated on changes in processes or 

technology improvements.

Yes. SSA added some timetables, but the 

required resources are not specified. Some 

of the success is still predicated on 

technological improvements.

Relationship between long-term goals and 

annual goals

Yes. SSA provided numerous performance 

measures relating to strategic goals and 

objectives, and plans for developing new 

measures were discussed. It was 

sometimes difficult to link measures with 

their appropriate objectives. It was also 

hard to discern which objectives did not yet 

have performance goals. Some data were 

expressed by program, while other data 

were aggregated.

Yes. SSA provided one or more key 

outcomes for each strategic objective. SSA 

notes that success in meeting the objectives 

will be measured in the annual performance 

plans by progress in achieving the key 

outcomes.

External factors Yes. The report mentioned several key 

external factors, but did not explicitly link 

factors to general goals and state how they 

could have affected goal attainment. Also, 

the report did not discuss needed changes 

(by Congress) to ensure solvency.

Yes. External (environmental) factors are 

listed in a separate section. However, the 

plan does not explicitly link factors to general 

goals and state how they could affect goal 

attainment. Specific effects are not 

discussed—most examples are vague.

Evaluations Yes. The report contained a broad 

discussion of program evaluations, but the 

evaluations were not clearly described. 

Also, SSA did not describe how the 

evaluations were used to establish or revise 

specific goals/objectives. Finally, there was 

no schedule for completing future 

evaluations or methodologies.

Yes. Future evaluations (with brief 

descriptions) are listed, but there is no 

discussion of how current evaluations are 

used to establish or revise specific 

goals/objectives. The plan states that SSA 

used internal and external (GAO, IG) 

evaluations to determine strategic plans and 

objectives.

(Continued From Previous Page)

Element of strategic planning Included in initial draft strategic plan Included in current strategic plan

16In the 2003-2008 plan, the word “advance” replaced the word “promote.”
Appendix III

Observations on Agencies’ Strategic Plans

Page 152 GAO-04-38 Results-Oriented Government

In 1997, the relationship between SSA’s long-term goals and specific 

programs was unclear and did not identify the specific results to be 

achieved. Since that time, SSA has improved this linkage and better 

articulated intended results, including quantifiable goals. For example, as 

part of its strategic goal to ensure superior stewardship of Social Security 

programs and resources, SSA notes that one of its key outcomes is to 

increase Supplemental Security Income (SSI) payment accuracy to 96 

percent (free of preventable error) by 2008.

SSA improved upon its linkage between long-term goals and annual goals in 

its fiscal year 2003-2008 strategic plan. Under each strategic goal in this 

plan, SSA provided one or more key outcomes for each strategic objective; 

in its 1997 draft strategic plan, we found it difficult to link measures with 

the appropriate objectives and discern which objectives did not yet have 

performance goals.

Critical Strategic Planning 

Issues Needing Further 

Improvement

Not all of SSA’s strategic objectives and associated performance measures 

will allow SSA to achieve its related strategic goals. Specifically, the 

solvency goal in SSA’s current strategic plan reads, “To achieve sustainable 

solvency and ensure Social Security programs meet the needs of current 

and future generations,” but the sole associated objective—through 

education and research efforts, support reforms to ensure sustainable 

solvency and more responsive retirement and disability programs—will not 

allow SSA, on its own, to reach that goal. While SSA’s mission is to advance 

the economic security of the nation’s people, it is not unilaterally 

responsible for achieving solvency in social security programs.

An agency’s strategic plan is expected to contain strategies for achieving 

the goals articulated. In 1997, SSA’s strategic plan was generally complete 

with regard to processes and technology, but did not include timetables or 

schedules for results. While the current strategic plan contains processes, 

anticipated progress in technology, and some timetables, it does not 

contain timetables or schedules for all of the results. For example, as part 

of its strategic objective to “efficiently manage Agency finances and assets 

and effectively link resources to performance outcomes,” SSA’s key 

outcomes include (1) competing or converting 50 percent of commercial 

positions and (2) to “get to green” on all five President’s Management 

Agenda (PMA) items. SSA has neither identified the required resources to 

achieve these goals nor has it identified a time frame for achieving them.
Appendix III

Observations on Agencies’ Strategic Plans

Page 153 GAO-04-38 Results-Oriented Government

In our review of SSA’s 1997 draft strategic plan, we noted that SSA 

described several key external factors that may affect its programs, but did 

not explicitly link such factors to its general goals and state how these 

factors could affect goal attainment. In the current strategic plan, SSA 

identifies environmental (external) factors: demographics, health and 

disability trends, technological advances, and workforce trends. However, 

as we found in our earlier review, the effects of these factors on specific 

performance goals are not specified, even though SSA notes that they drive 

the development of such goals.

SSA noted that it considered major management challenges identified by 

GAO when it determined its strategic goals and objectives, but not all of 

these challenges are clearly addressed in the plan. While these challenges 

are not clearly identified, SSA addresses them to some degree throughout 

the plan. For example, SSA’s strategic goal to “Strategically manage and 

align staff to support SSA’s mission” addresses the governmentwide 

challenge of strategic human capital management.17

SSA includes a list of major strategic process and program evaluations 

scheduled for the fiscal years 2003-2008 time period, organized by strategic 

goal. However, SSA does not list ongoing evaluations or mention how the 

results of these evaluations were used to prepare the current strategic plan. 

SSA notes that many of the hundreds of process and program evaluations 

conducted annually were designed to evaluate and improve internal 

processes falling below the strategic level. However, some of the ongoing 

evaluations are associated with specific strategic goals; thus, their 

outcomes could be discussed in the context of the strategic goals with 

which they are affiliated.

SSA’s strategic plan contains a very limited discussion of its interactions 

with other agencies that have similar goals or serve the same beneficiaries. 

In an interview, SSA officials noted that SSA has extensive interactions 

with other agencies on such issues as earnings accuracy and medical 

information, but the level of interaction varies by initiative. SSA’s strategic 

plan would benefit from a broader discussion of these interactions, 

especially if they were broken down by initiative. For example, as part of 

the objective to increase the accuracy of earnings records, SSA notes that it 

will collaborate with the Internal Revenue Service (IRS) to achieve more 

17U.S. General Accounting Office, Major Management Challenges and Program Risks: 

Social Security Administration, GAO-03-117 (Washington, D.C.: January 2003).
Appendix III

Observations on Agencies’ Strategic Plans

Page 154 GAO-04-38 Results-Oriented Government

accurate wage reporting as part of its means and strategies to reduce the 

size of the suspense file.18 It would be helpful if SSA offered more details as 

to the nature and extent of its collaboration with IRS.

Observations on 

Changes in the Quality 

of DOT’s Strategic Plan 

In our review of DOT’s July 1997 draft strategic plan, we found that the plan 

only met three of the six elements required by GPRA.19 The plan did not 

meet GPRA’s requirements to describe (1) strategies for achieving the 

goals, (2) a linkage between DOT’s long-term goals and annual 

performance goals, and (3) the external factors that could significantly 

affect DOT’s ability to achieve its goals. Further, for the three elements that 

the plan did meet, each had weaknesses. In comparison, DOT’s 2003-2008 

draft strategic plan has improved on several areas we identified in our 1997 

review.20 However, we still found areas where DOT could improve. Table 

11 summarizes these findings.

Table 11: DOT’s Progress in Addressing Required Elements of Strategic Planning under GPRA

18The suspense file contains information on earnings that cannot be matched to an 

individual’s record due to an invalid name/Social Security number combination.

19GAO/RCED-97-208R.

20At the time of our review, the Department of Transportation was in the process of revising 

its strategic plan. A draft copy of the updated strategic plan, dated July 1, 2003, was used for 

this review.

Element of strategic planning Included in initial draft strategic plan Included in current draft strategic plan

Mission statement Yes. DOT’s mission statement was 

comprehensive and covers its major 

functions and operations.

Yes. The mission statement continues to 

cover its major functions and operations and 

more explicitly states DOT’s statutory 

authority.

Long-term goals Yes. Five long-term goals encompassed 

DOT’s major functions and operations. 

However, it was not clear as to how DOT 

would measure success for most of its 

goals.

Yes. Five strategic objectives encompass 

DOT’s major functions and operations. 

Outcome goals and candidate performance 

measures for each strategic goal help show 

how DOT will measure success.

Strategies No. While the plan listed six corporate 

management strategies for achieving its 

long-term goals, it did not describe the 

operational processes, the skills, the 

technology, and the resources required to 

meet them.

Yes. Strategies are listed by strategic 

objective and include discussions on 

leadership, building expertise, and 

technology.
Appendix III

Observations on Agencies’ Strategic Plans

Page 155 GAO-04-38 Results-Oriented Government

Sources: GAO/RCED-97-208R and U.S. Department of Transportation, U.S. Department of Transportation Draft Strategic Plan for Fiscal Years 2003-2008 (Washington, D.C.: 2003).

Strategic Plan Strengths and 

Improvements from Fiscal 

Year 1997 Plan

The mission statement contained in DOT’s 2003-2008 draft strategic plan is 

an improvement over the one contained in the 1997 draft plan. DOT’s 

mission, as stated in the 2003-2008 draft strategic plan, is “to develop and 

administer policies and programs that contribute to providing fast, safe, 

efficient, and convenient transportation at the lowest cost consistent with 

the national objectives of general welfare, economic growth and stability, 

the security of the United States, and the efficient use and conservation of 

the resources of the United States.” The mission statement covers the 

major functions and operations of the department. In our July 1997 report 

on DOT’s 1997 draft strategic plan, we noted that the mission statement 

could be improved by including language from the department’s enabling 

legislation to focus the mission statement more directly on DOT’s core 

activities. We gave an example of adding the department’s purpose to 

develop transportation policies and programs that “contribute to providing 

fast, safe, efficient, and convenient transportation at the lowest cost” from 

DOT’s enabling legislation. The mission statement in the 2003-2008 plan 

includes such language.

As in its 1997 draft strategic plan, DOT’s 2003-2008 draft strategic plan 

meets the requirement of GPRA to include long-term goals and objectives 

for the major functions and operations of the department. The 2003-2008 

draft strategic plan contains five strategic objectives (long-term goals) that 

cover the major functions and activities of the department and are results 

Relationship between long term-goals and 

annual goals

No. The plan did not describe how 

performance goals would be related to the 

long-term goals.

Yes. The plan includes performance 

measures and refers to the performance 

plan for further information on annual 

performance goals.

External factors No. Four external factors were identified, 

but other key factors were not included in the 

plan. Only one external factor was 

discussed in terms of how it could have 

affected DOT’s ability to accomplish its 

goals.

Yes. Several external factors are listed for 

each strategic objective. Generally the plan 

gives descriptions of how these factors 

could affect the achievement of goals.

Evaluations Yes. Program evaluations used in 

establishing goals and a schedule of future 

evaluations were discussed. However, the 

plan did not provide enough information to 

determine the scope and methodology or 

the key issues to be addressed in future 

evaluations.

No. An extensive table describes the scope 

and methodology and the completion date of 

program evaluations for fiscal years 2003-

2008. However, the plan does not 

specifically mention which or how previous 

evaluations were used in developing the 

plan.

(Continued From Previous Page)

Element of strategic planning Included in initial draft strategic plan Included in current draft strategic plan
Appendix III

Observations on Agencies’ Strategic Plans

Page 156 GAO-04-38 Results-Oriented Government

oriented. Besides the strategic objectives of “safety,” “mobility,” “global 

connectivity,” “environmental stewardship” and “security,” the current draft 

also contains an “organizational excellence” objective to “advance the 

department’s ability to manage for results and achieve the goals of the 

PMA.”

Each strategic objective section in the 2003-2008 draft plan contains 

strategies for attaining DOT’s outcomes and objectives. The strategies for 

each strategic objective are listed in the categories of “leadership,” 

“building expertise,” and “technology.” For example, a “technology” 

strategy for the “mobility” strategic objective is to “examine ways to 

encourage cargo movements by water through the development of barge 

and fast vessel technologies to bring new capacity to our intermodal 

transportation system.” The plan states that this strategy supports DOT’s 

outcomes of reduced congestion in all modes of transportation and 

increased reliability throughout the system. The strategies for each 

strategic objective generally describe the operational processes, the skills, 

and the technology required to meet DOT’s goals and objectives. The 

current draft strategic plan also states that the resources and programs 

listed in DOT’s annual performance plans and budgets are necessary to 

achieve DOT’s outcomes and to execute the strategies. In contrast, the 

1997 draft strategic plan provided insufficient information to describe the 

operational processes, the skills, the technology, and the resources 

required to meet DOT’s long-term goals, as required by GPRA.

Also, each strategic objective section in the current draft plan contains 

crosswalks between outcomes in the strategic plan and performance 

measures in the annual performance plans and reports. These crosswalks 

show the measures that will be used to measure progress in achieving most 

of DOT’s outcomes and strategic objectives. For example, the performance 

measure “number of passengers in international markets with open skies 

aviation agreements” is related in a crosswalk to the outcome “reduced 

barriers to trade in transportation goods and services.” Together, the 

measure and outcome will show progress toward DOT’s global connectivity 

strategic objective. This is an improvement from DOT’s 1997 draft strategic 

plan when we noted that although supporting documents showed that DOT 

had developed information on how to measure each outcome goal, this 

information was not included in the draft.

In contrast to DOT’s 1997 draft strategic plan, the 2003-2008 draft plan lists 

several external factors for each strategic objective and generally discusses 

how these factors could affect the department’s ability to achieve its 
Appendix III

Observations on Agencies’ Strategic Plans

Page 157 GAO-04-38 Results-Oriented Government

outcomes and objectives. For example, one of the external factors for 

DOT’s environmental stewardship strategic objective is that DOT faces a 

significant challenge to control and minimize air, water, and noise 

pollution. The plan states that if DOT cannot control and minimize this 

pollution, the department may encounter a public backlash that may 

impede system improvement. For the external factors relating to the safety 

and mobility strategic objectives, the plan lists both positive and negative 

consequences the factors could have on achieving goals. One example 

would be the possible effects the expansion and integration of the 

telecommunications and e-commerce industry sectors could have upon 

transportation safety. The plan states that this could affect the 

achievement of DOT’s safety objective by leading to unsafe practices, such 

as the use of cell phones and other personal devices while driving. On the 

other hand, these technologies could also contribute to safety by alerting 

responders to the location of crashes and vehicles in distress. The 1997 

draft plan identified four external factors and only discussed how one of 

those factors could affect DOT’s ability to accomplish its goals.

The current draft strategic plan includes an extensive table listing program 

evaluations to be completed during fiscal years 2003-2008. The table 

includes the name of the program to be evaluated, which strategic goal(s) 

the program supports, the scope and methodology of the evaluation, and 

the fiscal year during which the evaluation will be completed. DOT’s 1997 

draft strategic plan only listed the titles for the evaluations scheduled for 

1997 and 1998, which was insufficient to determine the scope and 

methodology.

DOT’s current draft plan lists crosscutting programs by strategic objective. 

The discussions of crosscutting programs include the goal of each 

program, which of DOT’s outcomes each supports, and the agencies 

involved. For example, the goal of aviation security is to prevent 

explosives, weapons, and other dangerous items from being placed aboard 

aircraft. This program supports DOT’s security outcome of rapid recovery 

of transportation in all modes from intentional harm and natural disasters. 

DOT, through the Federal Aviation Administration, leads this program, 

which involves the Transportation Security Administration, Federal Bureau 

of Investigation, U.S. Customs Service, and U.S. Postal Service, among 

others. Previously, the 1997 draft did not provide evidence that DOT 

coordinated with other agencies that had programs and activities that were 

crosscutting or similar to DOT’s.
Appendix III

Observations on Agencies’ Strategic Plans

Page 158 GAO-04-38 Results-Oriented Government

DOT’s major management challenges, which we identified, are generally 

discussed in the 2003-2008 draft strategic plan, organized by related 

strategic objective. For example, we noted in our January 2003 report that 

one major management challenge that DOT faces is building human capital 

strategies.21 The current draft strategic plan includes a discussion of 

human capital in DOT’s organizational excellence objective. A separate 

section of this discussion addresses our concerns regarding human capital 

strategies and includes several milestones to address these concerns. 

These milestones include conducting workforce planning for missioncritical occupations in fiscal year 2003, and implementing a 

departmentwide performance management system beginning in fiscal year 

2003, and a uniform branding and marketing approach to attract, acquire, 

and retain diverse high-quality talent. DOT’s 1997 draft strategic plan did 

not adequately address the major management challenges we had 

previously identified.

Critical Strategic Planning 

Issues Needing Further 

Improvement

As stated above, the 2003-2008 plan provides a clear picture of how success 

will be measured for most of DOT’s strategic objectives and outcomes. 

However, this clarity is not provided for a few strategic objectives and 

outcomes. We noted the same issue in our review of DOT’s 1997 draft 

strategic plan. For example, in the current plan three of the outcome goals 

for the global connectivity strategic objective lack corresponding 

performance measures. These outcomes are enhanced international 

competitiveness of U.S. transport providers and manufacturers, 

harmonized and standardized regulatory and facilitation requirements, and 

the most competitive, cost-effective and efficient environments for 

passenger travel. The plan states that the measures are to be determined. 

However, without these measures it is unclear how progress will be 

measured because the outcomes themselves do not lend themselves to 

measurement.

While the current strategic plan shows improvement in the schedule for 

future program evaluations, it does not sufficiently describe the 

evaluations used in establishing or revising DOT’s strategic objectives. 

DOT states that detailed descriptions of completed program evaluations 

are presented in its 2002 performance and accountability report. Further, 

the plan states that DOT considered the results of completed program 

21U.S. General Accounting Office, Major Management Challenges and Program Risks: 

Department of Transportation, GAO-03-108 (Washington, D.C.: January 2003).
Appendix III

Observations on Agencies’ Strategic Plans

Page 159 GAO-04-38 Results-Oriented Government

evaluations, as well as reports from DOT’s Inspector General and GAO, in 

writing the strategies to achieve its strategic objectives and outcomes. The 

plan does not describe which or how program evaluations were used to 

write the strategies.
Page 160 GAO-04-38 Results-Oriented Government

Appendix IV

Observations on Agencies’ Annual 

Performance Plans ApendxiIV

Under GPRA, agencies are to prepare annual performance plans after the 

development of their strategic plans. These annual plans are to establish 

the connections between the long-term strategic goals outlined in the 

strategic plans and the day-to-day activities of managers and staff. One of 

our objectives was to assess the changes in the overall quality of agencies’ 

goals, strategies, and data articulated in their annual performance plans. 

To meet this objective, we judgmentally selected six agencies—Education, 

DOE, HUD, SBA, SSA, and DOT—using criteria such as agency size, 

primary program types, and previous GAO reviews. To assess the overall 

quality and improvements made to the agencies’ performance plans, we 

relied on requirements contained in GPRA and accompanying committee 

report language,1

 guidance to agencies from the OMB for developing 

performance plans,2 best practices identified in our published work,3

previous GAO evaluations,4 interviews with agency officials, and our 

knowledge of agencies’ operations and programs.

Key Elements of 

Information for Annual 

Performance Plans

Although GPRA does not require a specific format for the performance 

plan, it does require the plan to (1) identify annual performance goals and 

measures for each of an agency’s program activities, (2) discuss the 

strategies and resources needed to achieve annual performance goals, and 

(3) provide an explanation of the procedures the agency will use to verify 

and validate its performance data. We categorized each agency’s plan 

based on the degree to which it collectively addressed these three 

characterizations.

To assess the degree to which an agency’s plan provides a clear picture of 

intended performance across the agency, we examined whether it included 

(1) sets of performance goals and measures that address program results, 

(2) baseline and trend data for past performance, (3) performance goals or 

1

Government Performance and Results Act of 1993, Committee on Governmental Affairs, 

United States Senate, S. Rpt. No. 58, 103d Cong. 1st Sess. (1993).

2

OMB Circular No. A-11: Part 6, Preparation and Submission of Strategic Plans, Annual 

Performance Plans, and Annual Program Performance Reports (Washington, D.C.: June 

2002).

3

GAO/GGD/AIMD-99-215 and The Results Act: An Evaluator’s Guide to Assessing Agency 

Annual Performance Plans, GAO/GGD-10.1.20 (Washington, D.C.: April 1998).

4

GAO/HEHS-98-172R, GAO/RCED-98-194R, GAO/RCED-98-159R, GAO/RCED-98-180R, 

GAO/RCED-98-200R, and GAO/HEHS-98-178R.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 161 GAO-04-38 Results-Oriented Government

strategies to resolve mission-critical management problems, and

(4) identification of crosscutting programs (i.e., those programs that 

contribute to the same or similar results), common or complementary 

performance goals and measures to show how differing program strategies 

are mutually reinforcing, and planned coordination strategies.

To assess the degree to which an agency’s plan provides a specific 

discussion of strategies and resources the agency will use to achieve 

performance goals, we examined whether it included (1) budgetary 

resources related to the achievement of performance goals, (2) strategies 

and programs linked to specific performance goals and descriptions of how 

the strategies and programs will contribute to the achievement of those 

goals, (3) a brief description or reference to a separate document of the 

human capital, information, and other resources required to achieve 

results,5

 and (4) strategies to leverage or mitigate the effects of external 

factors on the accomplishment of performance goals.

Finally, to assess the degree to which an agency provides confidence that 

its performance information will be credible, we examined how each 

report discussed the quality of the data presented. To help improve the 

quality of agencies’ performance data, Congress included a requirement in 

the Reports Consolidation Act of 2000 that agencies assess the 

completeness and reliability of their performance data. Under the Act, 

agencies were to include this assessment in the transmittal letter with their 

fiscal year 2000 performance reports. Agencies were also required to 

discuss in their report any material inadequacies in the completeness and 

reliability of their performance data and discuss actions to address these 

inadequacies.

5

The Homeland Security Act (Pub. L. No. 107-296), also requires that agencies provide a 

description of how the performance goals and objectives are to be achieved, including the 

operations, processes, training, skills and technology, and the human capital, information, 

and other resources and strategies required to meet those performance goals and 

objectives.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 162 GAO-04-38 Results-Oriented Government

For each of these elements, we characterized each agency’s fiscal year 1999 

and fiscal year 2004 plan in one of four ways, based on the degree to which 

the plan contained informative practices associated with that element. 

Thus, to address the first element concerning the degree to which the plan 

provided a clear picture of performance, we characterized each plan in one 

of four ways: (1) clear, (2) general, (3) limited, or (4) unclear. To address 

the second element, on the extent to which a plan includes specific 

discussions of strategies and resources, we characterized each plan as 

(1) containing specific discussions of strategies and resources, (2) general 

discussions, (3) limited discussions, or (4) no discussions. Finally, to 

address the third element on the extent to which a plan provides 

confidence that performance information will be credible, we 

characterized each plan as providing (1) full confidence, (2) general 

confidence, (3) limited confidence, or (4) no confidence. In conducting our 

reviews, we compared our assessments of agencies’ fiscal year 2004 plans 

to our assessments of plans from fiscal year 1999 using similar criteria.6

 A 

more detailed discussion of our scope and methodology and the criteria we 

used can be found in appendix I.

Table 12 summarizes our characterizations of the six agencies’ annual 

performance plans based on our current review of fiscal year 2004 plans 

and our previously published reviews of 1999 plans. Although the 

characterization of agency performance plans did not change significantly 

between the 1999 and the 2004 plans, the majority of agencies’ plans 

showed some improvement.

6

GAO/HEHS-98-172R, GAO/RCED-98-194R, GAO/RCED-98-159R, GAO/RCED-98-180R,

GAO/RCED-98-200R, and GAO/HEHS-98-178R.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 163 GAO-04-38 Results-Oriented Government

Table 12: Characterizations of Agencies’ Annual Performance Plans 

Sources: GAO/HEHS-98-172R; GAO/RCED-98-194R; GAO/RCED-98-159R; GAO/RCED-98-180R; GAO/RCED-98-200R; GAO/HEHS98-178R; and Department of Education, FY 2004 Annual Performance Plan (Washington, D.C.: 2003); Department of Energy, Annual 

Performance Plan, Fiscal Year 2004 (Washington, D.C.: 2003); Housing and Urban Development, Annual Performance Plan, Fiscal Year 

2004 (Washington, D.C.: 2003); Small Business Administration, Budget Request & Performance Plan: FY 2004 Congressional 

Submission (Washington, D.C.: 2003); Social Security Administration, Annual Performance Plan, Fiscal Year 2004 (Washington, D.C.: 

2003); and Department of Transportation, Fiscal Year 2004 Performance Plan (Washington, D.C.: 2003).

The remainder of this appendix discusses our observations on how the 

quality of each of the agencies’ annual performance plans we reviewed has 

changed since the agencies submitted their first performance plans in 1999. 

We did not independently verify or assess the information we obtained 

from agency annual performance plans. If an agency chose not to discuss 

its efforts concerning elements in the plan, it does not necessarily mean 

that the agency is not implementing those elements.

Agency

Characterizations 

Picture of 

intended 

performance

(unclear, limited, 

general, clear)

Strategies and 

resources

(no discussions, 

limited, general, 

specific)

Data credible 

(no, limited, 

general, full)

1999 2004 1999 2004 1999 2004

Department of 

Education

Limited General Limited General Limited General

Department of Energy Limited Limited General General Limited Limited

Department of Housing 

and Urban 

Development

Limited General Limited General Limited General 

Small Business 

Administration

Limited General Limited General Limited General

Social Security 

Administration

Limited Clear Limited General No General

Department of 

Transportation

General Clear General Specific Limited Full
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 164 GAO-04-38 Results-Oriented Government

Observations on 

Changes in the Quality 

of Education’s Annual 

Performance Plan

Education’s fiscal year 2004 annual plan7 provides a general picture of 

intended performance across the agency—an improvement over the 1999 

plan—because the measures and indicators adequately indicate progress 

toward meeting annual targets; the measures are objective, measurable, 

and quantifiable; and baseline and trend data are included where available. 

However, the relationship between the goals and measures in volume 2 and 

the long-term goals in volume 1 is not clear and the plan does not make 

clear whether, and, if so, how, all of the program activities in the 

department’s budget are covered by the annual performance goals.8

 In 

another improvement over the 1999 plan, volume 1 of the 2004 plan 

provides a general discussion of Education’s strategies and resources to 

achieve its goals by presenting strategies and resources and the projected 

distribution of fiscal year 2004 funding and staffing for each long-term goal. 

The plan also adequately recognizes and discusses external factors that 

could affect the department’s performance. However, the resources and 

many of the strategies are not directly linked to the achievement of 

individual annual performance goals and no strategies or resources are 

designated for the goals and measures in the program performance plans in 

volume 2. Lastly, the 2004 plan provides general confidence that agency 

performance information will be credible. The 2004 plan contains 

information on data sources for most of its measures, and for some, 

identifies limitations. The plan also includes an appendix entitled 

“Information Quality Guidelines” which recognizes data quality as a major 

challenge for the agency and says that its improvement is a top priority.

7

Education’s plan states that its fiscal year 2004 annual plan includes both department-level 

measures and program performance plans. These are organized into two volumes: the 

Annual Plan Fiscal Year 2004 U.S. Department of Education includes the departmentlevel measures and the FY 2004 Program Performance Plan: U.S. Department of 

Education includes the program performance plans with their individual program 

measures. These volumes are presented in a slightly different electronic format for the 

public and other parties in general, which is available at Education’s Web site. The two 

volumes will henceforth be referred to as Education’s 2004 annual plan, or, where 

applicable, volume 1 and volume 2.

8

Education’s 2004 annual plan represents its annual performance goals as “targets.” 

According to GPRA, the definition for “performance goal” is “a target level of performance.”
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 165 GAO-04-38 Results-Oriented Government

Education’s Fiscal Year 2004 

Performance Plan Provides 

a General Picture of 

Intended Performance

Education’s 2004 annual plan generally defines expected performance. 

Measures and indicators are formulated so as to adequately indicate 

progress towards meeting annual targets, seem to sufficiently cover key 

performance aspects, and adequately capture important program 

distinctions. The plan contains about 360 measures between volumes 1 

and 2, an improvement over the 860 contained in its 1999 plan, which we 

judged to be potentially excessive for an annual performance plan and 

possibly interfering with Education’s ability to assess its performance. 

Unlike in our review of the department’s 1999 annual plan, the measures 

and indicators in the 2004 plan are objective, measurable, and quantifiable. 

For example, most measures and indicators are set up to measure 

percentages, cost, counts, or other numerical values with measurable, 

quantifiable 2004 targets. In most cases where a measurable target is not 

given, the plan provides a reasonable explanation, such as a new program 

or the measure being new, or a case where data are not collected or 

available each year. In most cases, the plan provides trend data for 

measures, which provides a helpful context for assessing the relevance of 

the 2004 targets, or an explanation of why such data were not provided 

(e.g., the baseline has not yet been established because the measure and/or 

program are new).

In our review of Education’s 1999 annual plan, we said that greater 

outcome measure use would make future annual plans more useful. The 

2004 plan frequently includes outcome goals and measures, such as a 

measure for the number of states meeting their eighth-grade mathematics 

achievement targets under the long-term goal to improve mathematics and 

science achievement for all students.

Volume 1 of Education’s 2004 annual plan directly aligns strategies, action 

steps, measures, and targets with each of Education’s long-term goals and 

six strategic goals,9

 containing the same strategic goals, long-term goals, 

and mission as the 2002-2007 strategic plan. In our review of the 1999 plan, 

we also found that the plan had performance goals in volume 1 that were 

directly linked to its mission, strategic goals, and objectives. However, the 

relationship between the goals and measures in volume 2 and the long-term 

goals in volume 1 was not made clear in the department’s 2004 annual plan, 

which was similar to what we found in our review of Education’s 1999 

9

In this report, we refer to the multiyear, long-term objectives in Education’s annual plan as 

“long-term goals.” The strategic goals included in the plan represent overarching statements 

of aim or purpose that are used to group Education’s long-term goals.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 166 GAO-04-38 Results-Oriented Government

plan—that the department could more directly link the goals and measures 

in volume 2 with the strategic objectives (long-term goals). Education’s 

fiscal year 2002 Performance and Accountability Report includes a table 

making it clear that these programs and their goals and measures are 

aligned across Education’s strategic and long-term goals. By including 

such a table in its annual performance plan, Education could clearly show 

the link between the goals and measures in volume 2 and the long-term 

goals in volume 1.

Although volume 2 of the 2004 annual plan states that it contains individual 

program performance plans for all major programs and many smaller 

programs, the annual plan does not make clear whether, and, if so, how all 

of the program activities in the department’s budget are covered by 

performance goals. In contrast, we found that the 1999 plan provided 

sufficient information to determine which performance goals and measures 

in volume 2 covered which program activities in the budget and whether all 

were covered. For example, the 1999 plan contained tables indicating the 

funding levels for the program activities in the department’s budget and 

how those activities related to the programs in volume 2.

In our review of Education’s 1999 annual plan, we gave the agency credit 

for addressing the need to coordinate with other federal agencies having 

related strategic goals or performance goals. However, we further noted 

that Education could build on its foundation by identifying performance 

goals that reflect activities being undertaken to support programs of a 

crosscutting nature and specifying the activities each agency would 

undertake and what it expects to achieve within the fiscal year. While 

selected action steps in the 2004 plan refer to instances where the 

department will coordinate or cooperate with other federal agencies, the 

plan does not include steps or goals for most crosscutting issues identified 

in Education’s strategic plan. For example, for a crosscutting issue 

identified in the strategic plan on safe and drug-free schools and 

communities, Education said it partners with the Departments of Justice 

(Justice) and HHS to promote drug and alcohol education programs and to 

disseminate information to schools and private organizations. The 

department also coordinates closely with the Office of National Drug 

Control Policy, and works closely with the Office of Juvenile Justice and 

Delinquency Prevention Programs to share innovative ideas and promote 

prevention strategies and programs. The relevant annual plan sections in 

both volumes 1 and 2 do not identify goals or action steps related to these
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 167 GAO-04-38 Results-Oriented Government

interactions. Additionally, according to our report on Education’s June 

1997 draft strategic plan,10 the department has numerous crosscutting 

programs and activities, such as those related to early childhood and 

employment training, and the 2004 annual plan does not address them all.

Education’s 2004 annual plan discusses applicable goals, measures, and 

strategies for two governmentwide major management challenges 

regarding strategic human capital management and information security, as 

well as three of the four major management challenges we identified for 

Education in our January 2001 Performance and Accountability Series.

11 

For example, for its student financial assistance programs, the department 

has developed performance targets for fiscal year 2004 related to being 

removed from our high-risk list, increasing the default recovery rate, and 

decreasing grant overpayments to students. Also, a key strategy under its 

goal to improve the strategic management of the department’s human 

capital is to develop a 5-year human capital plan, including developing a 

recruitment plan and relevant training programs. For the fourth of 

Education’s major management challenges—promoting coordination with 

other federal agencies and school districts to help build a solid foundation 

of learning for all children—the 2004 plan did not include specific goals or 

measures, but it did discuss some related strategies and steps, such as 

using partnerships with other federal programs to promote development of 

intervention strategies and methods to address the high incidence of 

learning disabilities and illiteracy among adolescents attending high 

schools.

Education’s Fiscal Year 2004 

Performance Plan Provides 

a General Discussion of 

Strategies and Resources

In our review of Education’s 1999 annual plan, we found that the plan had a 

limited discussion of how the department’s strategies and resources would 

help achieve its annual performance goals. The 2004 plan includes 

strategies and resources under each of its long-term goals to be used to 

achieve its annual performance goals in volume 1, including the projected 

distribution of fiscal year 2004 funding and staffing, in both dollars and fulltime-employees (FTE), for each long-term goal, under which the annual 

performance goals are organized. However, the resources and many of the 

10GAO/HEHS-97-176R.

11U.S. General Accounting Office, Performance and Accountability Series—Major 

Management Challenges and Program Risks: A Governmentwide Perspective, GAO-01-241

(Washington, D.C.: January 2001) and GAO-01-245.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 168 GAO-04-38 Results-Oriented Government

strategies are not directly linked to the achievement of individual annual 

performance goals and no strategies or resources are designated for the 

goals and measures in the program performance plans in volume 2. 

Overall, the plan does not discuss how resources were allocated to each 

goal, a rationale for how the resources will contribute to improving 

performance, or the relationship of capital asset investments, including 

those for information technology (IT), to the achievement of specific goals. 

However, the plan does include a performance measure and goal for the 

cost and schedule of IT investments and a strategy for completing the 

department’s enterprise architecture, which is to be used to guide IT capital 

decisions. In addition, the department has a plan for human capital 

management and a new performance appraisal system that is meant to link 

employee performance standards to the department’s strategic priorities, 

but neither had been fully implemented.

In our review of the 1999 plan, we said that external factors that could 

affect performance were not discussed and that such factors are important 

for a department like Education because much of what it hopes to achieve 

depends on others and external events. In its 2004 plan, Education clearly 

acknowledges that improving support for its state, local, and institutional 

partners, who have the direct ability to influence outcomes the department 

seeks, is a major challenge. The plan contains numerous activities to 

handle this challenge, including, for example, to provide support and 

technical assistance, improve grant monitoring, and fund an annual survey 

of states’ efforts. Moreover, although not labeled as addressing external 

factors, the plan has specifically related strategies and/or action steps for 

most external factors identified in Education’s 2002-2007 strategic plan.

Education’s Fiscal Year 2004 

Performance Plan Provides 

General Confidence That 

Performance Data Will Be 

Credible

In our review of Education’s 1999 annual plan, we found that it did not 

provide sufficient confidence that its performance information would be 

credible. For example, the 1999 plan did not sufficiently recognize 

limitations in Education’s data for its elementary and secondary education 

programs. In comparison, Education’s 2004 plan recognizes limitations in 

Education’s data for many of its elementary and secondary education 

programs, as well as for other programs. In many of these cases, the plan 

also discusses plans to address these limitations. Also, the plan includes an 

appendix containing an abbreviated form of its “Information Quality 

Guidelines” and a sample checklist for statistical data from its complete 

guidelines. The appendix recognizes data quality as a major challenge to 

the department’s successful implementation of GPRA and says that the 

improvement of data quality is a top priority. The checklist includes 
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 169 GAO-04-38 Results-Oriented Government

several steps related to the verification and validation of data, such as 

evaluating data quality, including known limitations; addressing the 

reliability of data sources; and ensuring reproducibility of findings using 

the same data and methods of analysis. In addition, the plan usually 

identifies the sources of data and, although not in volume 1, includes a 

column on sources and data quality for each measure in volume 2, usually 

with an item entitled “Validated By” and, in some cases, “Limitations.” In 

the end, the lack of direct control over the implementation of its programs, 

including the collection of data, is a significant data quality challenge that 

Education must face.

The 2004 annual plan also contains several action steps on new or changing 

information systems that relate to improving the collection of information 

for measuring performance. For example, under its strategy to reduce 

Education’s partners’ data reporting burden, the plan includes an action 

step to develop and implement the Performance-Based Data Management 

Initiative collection system. This step is directly related to the plan’s 

measure to reduce the burden hours of Education program data collections 

per year.

Observations on 

Changes in the Quality 

of DOE’s Annual 

Performance Plan

Compared to the fiscal year 1999 plan we reviewed, DOE’s fiscal year 2004 

performance plan continued to provide a limited picture of intended 

performance. Although the plan included more results-oriented annual 

performance measures, it still provided a limited linkage between its 

reported annual goals and its mission, strategic plan goals, and program 

activities within its budget request. Furthermore, the 2004 plan provided a 

general discussion of strategies and resources, similar to our 1999 findings. 

Finally, the 2004 plan provided a limited level of confidence that data will 

be credible by making little progress in reporting on the procedures it uses 

to ensure data quality or identifying significant data limitations, which is 

consistent with our 1999 findings.

DOE’s Fiscal Year 2004 

Performance Plan Provides 

a Limited Picture of 

Intended Performance

While DOE has improved its development of annual performance 

measures—referred to as targets—by making them more results oriented, 

the overall picture of performance is limited by the lack of alignment 

between its annual and strategic goals and minimal discussion of 

coordination with other agencies. Our review of DOE’s 1999 plan found 

that many measures were unclear, appeared limited in scope, or were not 

very useful indicators of performance. We found these problems in the 
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 170 GAO-04-38 Results-Oriented Government

performance plans for subsequent years as well. For the 2004 plan, 

however, the majority of the performance measures related to each goal 

were results oriented and pertained specifically to the performance of 

fiscal year 2004. An example of one measure requires DOE to train 4,000 

federal employees by the end of fiscal year 2004 in energy management 

best practices that support National Energy Policy education goals.

DOE provided a limited link between its reported annual goals and its 

mission, strategic plan goals, and program activities within its budget 

request. While the 2004 annual performance plan goals address all of the 

major program activities in DOE’s budget, the goals and mission of the 2004 

plan do not align with the mission and goals for the 2003 draft strategic 

plan. This represents a set back because in our review of DOE’s 1999 

annual performance plan, we found that DOE clearly linked its annual 

goals to the agency’s mission, strategic plan goals, and its program 

activities within its budget request. DOE officials told us the lack of 

linkage between the performance plan and the strategic plan was a matter 

of timing. According to these officials, the department originally updated 

its strategic plan at the same time as the annual performance plan, which 

was finalized in the early months of 2003, and the goals of each plan 

coincided, but the draft strategic plan goals were revised in the latter part 

of the year and no longer align with the 2004 performance plan.

DOE’s ability to show coordination with other agencies is also limited. In 

1999, we reported that DOE did not adequately show that it coordinated 

with other agencies that have related strategic or performance goals. 

DOE’s 1999 plan contained very little evidence of specific goals and 

measures that addressed crosscutting programs and only briefly described 

coordination with other agencies. The 2004 plan does not specifically 

describe how coordination is taking place among crosscutting programs, 

but does identify groups that it is collaborating with on certain programs. 

In response, DOE officials told us the plan does not discuss what specific 

collaboration activities are taking place because it would require reporting 

too much detail for a performance plan. DOE officials stated that it 

collaborates at the program level, rather than the agency level, because the 

program plans pertain to an organizational layer lower than the annual 

performance plan.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 171 GAO-04-38 Results-Oriented Government

Finally, the plan briefly mentions that the department has been identifying 

challenges and working on ways to address them. According to DOE 

officials, when developing annual targets for the department, management 

challenges are considered but not mentioned specifically in the report. Our 

review of management challenges in 2002 found that DOE had addressed 

all eight of its challenges in its 2003 annual performance plan.12 In 

comparing these challenges to the 2004 plan, we found that DOE continues 

to have goals that address the eight challenges we identified in 2002 and the 

additional challenges that we identified in our 2003 performance and 

accountability report.13

DOE’s Fiscal Year 2004 

Performance Plan Provides 

a General Discussion of 

Strategies and Resources

DOE provided a general discussion of the strategies and resources that it 

will use to achieve its annual performance goals. DOE’s 1999 plan partially 

provided clear and reasonable strategies for achieving performance goals, 

how strategies would contribute to achieving the performance goals, and 

key external factors that might affect performance. For each of the 2004 

annual performance goals, DOE included a “Means and Strategies” section 

in the plan that described how each goal will be achieved. For example, 

one strategy identified to meet its goal of contributing unique, vital 

facilities to the biological and environmental sciences is to conduct peer 

reviews of the facilities to assess the scientific output, user satisfaction, 

and the overall cost-effectiveness of each facility’s operations, and their 

ability to deliver the most advanced scientific capability. The 2004 plan 

includes a brief discussion of the department’s overall needs, particularly in 

the areas of human capital, financial, and logistical resources. The plan 

also identified budget amounts for each of its goals. DOE’s 1999 plan 

partially identified the resources needed to accomplish annual 

performance goals.

The plan also provided a general discussion of the external factors that 

could affect achievement of the goals, but it did not specifically discuss 

actions on how the external factors will be addressed. For example, the 

plan states that external factors related to DOE’s goal of achieving reliable, 

affordable, and environmentally sound energy supplies, such as renewable 

fuels, include program funding, the state of the economy, the availability of 

12GAO-03-225.

13GAO-03-100.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 172 GAO-04-38 Results-Oriented Government

conventional supplies, the cost of competing technologies, and the 

continuation of federal tax incentives and other national-level policies.

DOE’s Fiscal Year 2004 

Performance Plan Provides 

Limited Confidence That 

Performance Data Will Be 

Credible

DOE has made limited progress on reporting the procedures it uses to 

ensure data quality. Its 1999 plan described how DOE would ensure that its 

performance information is sufficiently complete, accurate, and consistent, 

but did not discuss in detail DOE procedures on how to help ensure the 

quality of data or the process of collecting the data. The plan also did not 

identify significant data limitations and how they may affect DOE’s ability 

to achieve performance goals. However, the 2004 plan showed some 

improvement over the 1999 plan by describing credible procedures to 

verify and validate performance information and specific program 

evaluations are mentioned for each goal. The plan also discusses that DOE 

acquired new commercial software for performance tracking through 

remote data entry, monitoring, and oversight by program offices and 

managers. The 2004 plan only identifies data limitations and any new or 

modified systems very briefly for a few relevant goals. According to DOE 

officials, with a few exceptions, its plans do not discuss data limitations 

because DOE writes goals that are not affected by data limitations. The 

goals are written to ensure that the data will be there to meet performance 

targets. However, our 2003 performance and accountability series 

identified several DOE management challenges where data quality was a 

concern, such as further upgrades needed for cyber security to ensure 

adequate protection of data and information systems and additional 

information on the results of contractors’ performance to keep projects on 

schedule and within budget.

Observations on 

Changes in the Quality 

of HUD’s Annual 

Performance Plan

HUD’s annual performance plan for fiscal year 2004 improves upon areas 

where we previously reported shortcomings and generally meets the 

criteria set forth in GPRA. HUD’s 2004 plan provides a general picture of 

intended performance by covering all the programs contained in HUD’s 

budget and linking program activities to strategic goals and objectives. The 

plan also improved by providing specific information on HUD’s strategies 

and activities along with performance measures it will use to assess 

progress toward its goals and discussing relevant external factors that 

could affect the attainment of certain program objectives. HUD also 

provides greater confidence that performance data will be credible by 

thoroughly discussing the data it will use for measuring progress toward its 

goals. Nevertheless, the plan could be further enhanced if it included more 
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 173 GAO-04-38 Results-Oriented Government

specific information on how funds will be allocated to achieve program 

objectives, explain how HUD will contribute to crosscutting efforts along 

with other agencies, and what steps it will take to mitigate the impact of 

external factors on its programmatic objectives.

HUD’s Fiscal Year 2004 

Performance Plan Provides 

a General Picture of 

Intended Performance

Since our review of HUD’s annual performance plan for fiscal year 1999,14

HUD has made progress in developing an annual performance plan that 

generally reflects the department’s mission and provides a general picture 

of intended performance. HUD’s most recent performance plan covers the 

program activities contained in its budget, and generally links program 

activities to strategic goals and objectives, key items missing from its plan 

for fiscal year 1999. HUD has also improved the quality of its performance 

plan by including performance measures that generally indicate how the 

department will gauge progress toward achieving its goals. For example, 

the performance plan lists a series of performance measures for each 

objective that can be used to indicate progress towards the department’s 

goals and expected performance. These measures are also objective and a 

number of them have been quantified, another key area where HUD has 

improved since its first performance plan. For example, activities 

supporting HUD’s long-term strategic objective to “Improve the Physical 

Quality and Management Accountability of Public and Assisted Housing” 

include, among other things, eliminating 100,000 units of the worst public 

housing. According to the current plan, the department intends to 

demolish 10,000 of these units in fiscal year 2004.

While HUD’s most recent annual performance plan generally identifies 

other agencies it will coordinate with to address crosscutting efforts, it 

does not discuss how it plans to work with these agencies to address these 

crosscutting activities. For example, the plan states that the Interagency 

Working Group on Limited English Proficiency will ensure that persons 

with limited English proficiency will have meaningful access to funded and 

federally conducted programs and activities. However, the plan does not 

discuss what HUD’s contribution to this multiagency effort will be, what 

strategies it will employ, or how it will measure progress toward achieving 

the strategies of this multiagency effort.

14GAO/RCED-98-159R.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 174 GAO-04-38 Results-Oriented Government

HUD’s current performance plan is an improvement compared to its fiscal 

year 1999 plan as it describes steps HUD will take to address major 

management challenges. One of HUD’s strategic goals, “Embrace High 

Standards of Ethics, Management, and Accountability,” identifies five 

objectives that cover management challenges, some of which have been 

raised by GAO and the HUD IG. These objectives discuss plans to rebuild 

HUD’s human capital and diversify its workforce; improve HUD’s 

management, internal controls, and resolve audit issues; improve 

accountability, service delivery, and customer service; ensure program 

compliance; and improve internal communication and employee 

involvement.

HUD’s Fiscal Year 2004 

Performance Plan Provides 

a General Discussion of 

Strategies and Resources

HUD’s most recent performance plan also improves upon earlier plans we 

reviewed in providing readers an idea of the strategies that HUD will 

employ to carry out its goals. Each strategic goal in the annual 

performance plan contains a section titled “Means and Strategies,” which 

describes activities HUD will pursue to support that goal. For example, to 

support its goal of “Increasing Homeownership Opportunities,” HUD will 

fund low-income homeowner assistance programs to provide 

approximately 40,000 families with down payments and closing costs on 

their homes, 473,199 families with home purchase and homeownership 

counseling, and about 232,370 families with rental counseling.

HUD’s performance plan also discusses the strategies it will employ to 

address the department’s human capital issues, such as the upcoming 

potential wave of employees planning to retire and the need to equip staff 

with the desired knowledge and skills. For example, HUD completed a 

staff resource estimation and allocation system in 2002, and it will conduct 

a comprehensive workforce analysis in 2004 to serve as the basis to fill 

mission-critical skill gaps through succession planning, hiring, and training 

initiatives in its Five-Year Human Capital Management Strategy.

Although HUD has made progress in linking its resources to strategies, it 

could improve the discussion by linking funding allocations to specific 

performance goals, thus making the plan more informative. The plan 

discusses budget and staff allocations for programs supporting each 

strategic goal. For instance, portions of HUD’s Community Development 

Block Grants Fund and Home Investment Partnership Program, with a 

combined budget authority for fiscal year 2004 of more than $2.5 billion 

and staff of 203, support the strategic goal of promoting “Decent Affordable 

Housing.” However, it is unclear what resources will be used to pursue 
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 175 GAO-04-38 Results-Oriented Government

specific performance targets for each program. Additionally, HUD does not 

mention in its plan how IT and capital resources will be used to support its 

programs.

Anticipating that some aspects of the department’s strategic goals are 

intertwined with broader phenomena, the performance plan also discusses 

several external factors relevant to each strategic goal that could affect 

HUD’s ability to meet its objectives. For example, for its strategic goal 

“Promote Decent and Affordable Housing,” HUD states that broad 

economic factors can affect opportunities for low-income workers relying 

on the department for rent assistance to make progress towards selfsufficiency. However, it is unclear from the performance plan what actions, 

if any, HUD has put in place to mitigate the effect of these external factors.

HUD’s Fiscal Year 2004 

Performance Plan Provides 

General Confidence That 

Performance Data Will Be 

Credible 

HUD has also made significant progress in providing assurance that the 

department will be able to use credible data to gauge progress towards 

achieving its goals. HUD identifies the steps it (or others) will take to 

verify and validate the performance data to ensure that what is reported on 

HUD’s performance will be credible. For example, for its objective 

“Increasing Minority Homeownership,” HUD will rely on, among other 

indicators, the rate of minority homeownership from the Current 

Population Survey conducted monthly by the U.S. Census Bureau. HUD 

will not verify the data because the Bureau performs that task. 

Additionally, HUD also includes in its most recent performance plan a 

discussion of the inherent limitations of the data it will use and generally 

discusses steps it will take to improve the measure, providing the reader 

with a clearer expectation of what HUD will be able to report.

Observations on 

Changes in the Quality 

of SBA’s Annual 

Performance Plan

SBA’s 2004 performance plan shows progress made over the agency’s 1999 

performance plan. In contrast to our review of SBA’s 1999 plan,15 the 2004 

plan provides a general picture of intended performance by discussing 

coordination between SBA and other federal agencies on crosscutting 

activities. Resource analysis sections throughout the plan provide a 

general discussion of how SBA has previously used its resources to achieve 

its goals and how it intends to use future resources for the same purposes. 

The 2004 plan also provides general confidence that SBA’s performance 

15GAO/RCED-98-200R.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 176 GAO-04-38 Results-Oriented Government

data will be credible by including more detail on how SBA verifies and 

validates its data, as well as by identifying data limitations. However, 

several areas of the plan could be improved, such as clearly linking SBA’s 

performance indicators, performance goals, and programs.

SBA’s Fiscal Year 2004 

Performance Plan Provides 

a General Picture of 

Intended Performance

SBA’s fiscal year 2004 performance plan provides a general picture of 

intended performance. The performance goals and performance indicators 

in the plan are generally objective, measurable, and quantified. 

Performance indicators are listed throughout the plan by related programs 

and strategic programmatic goals. Performance goals and outcome goals 

are listed for each strategic programmatic goal. In our review of SBA’s 

fiscal year 1999 performance plan, we noted that SBA’s performance goals 

were objective and measurable, its performance measures were generally 

objective and quantified, and that the performance goals in the plan were 

clearly linked to SBA’s strategic goals and objectives.

Like the performance measures contained in its 1999 plan, the 2004 plan’s 

performance indicators will be useful in assessing progress towards SBA’s 

performance goals. For example, the performance indicator “Regulatory 

Cost Savings to Small Business” will adequately show progress for the 

corresponding performance goal “Cost savings for small business due to 

the efforts of the Office of Advocacy.” In this example, the performance 

indicator, which is listed under the Advocacy Program, can be linked to a 

performance goal because of a crosswalk that relates outcome goals, 

performance goals, and programs. However, there is not always such a 

clear link between all of SBA’s performance indicators and performance 

goals because indicators are listed by program instead of by performance 

goal. The BusinessLaw.gov program is linked to three performance goals: 

“number of users of BusinessLaw.gov,” “reduced cost to businesses and 

regulatory agencies,” and “increased rate of compliance.” While the first 

two performance goals appear related to the first two indicators listed in 

the BusinessLaw.gov program section, there is no clear relationship 

between any of the other performance indicators for this program and the 

third performance goal, “increased rate of compliance.”

SBA’s 2004 performance plan contains annual performance goals that 

generally cover the agency’s budget activities. The 2004 performance plan 

contains a budget crosswalk that “shows how the goals relate to specific 

and general program areas.” This is an improvement over the 1999 plan, 

which we noted contained a budget crosswalk, but the categories in it did 

not match SBA’s budget accounts or activities by name or account number. 
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 177 GAO-04-38 Results-Oriented Government

However, the performance plan does not seem to cover all of SBA’s 

programs. Three “advocacy” programs listed in the crosswalk do not seem 

to be contained in the plan: Business.gov, Disability Initiative, and National 

Women’s Business Council.

Each strategic programmatic goal section in the 2004 plan contains a 

discussion of crosscutting issues. Several examples of coordination efforts 

are given, such as SBA working with the Department of Defense to 

integrate the PRO-Net system with the Central Contractor Registry and 

SBA partnering with the Federal Acquisition Institute to develop on-line 

training courses for small business programs. In contrast, SBA’s 1999 

performance plan provided little information on SBA’s coordination efforts 

with other entities whose programs and activities crosscut those of SBA.

SBA’s 2004 performance plan generally addresses performance and 

accountability challenges we have previously identified. For example, we 

have previously stated that SBA needs to strengthen its performance in 

human capital management. The 2004 plan includes outcome goals, 

performance goals, and programs to address SBA’s strategic management 

of human capital in a section on the PMA.

SBA’s Fiscal Year 2004 

Performance Plan Provides 

a General Discussion of 

Strategies and Resources

The 2004 performance plan provides a general discussion of the strategies 

and resources SBA will use to achieve its goals. Each strategic 

programmatic goal and each of the goals for the PMA contains a discussion 

of the strategies for accomplishing the goals. These discussions provide a 

broad overview of the strategies used at the strategic programmatic goal 

level. For example, the plan includes a strategy for SBA’s strategic 

management of human capital. The strategy lays out SBA’s Transformation 

and Human Capital plans, which will be used to implement a new vision of 

SBA. In its 1999 performance plan, SBA discussed strategies for most of its 

performance goals, although for some of the goals the strategies were 

missing.

Each strategic programmatic goal and several of the PMA goals contain 

brief discussions of external factors that could affect the achievement of 

SBA’s goals. These discussions include actions to address external factors, 

such as working with an Interagency Acquisition Working Group under the 

Procurement Executives Council to develop supplemental performance 

measures to better evaluate the success of its programs. In 1998, we noted 

that SBA’s 1999 plan recognized certain external factors and contained a 

discussion of actions SBA could take to mitigate the effects of such factors 
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 178 GAO-04-38 Results-Oriented Government

for one of its strategic goals. We stated that it would be useful for SBA to 

include a similar discussion of external factors and mitigation strategies for 

its other strategic goals.

Each program listed throughout the plan has a resource analysis section 

that describes how resources were used in fiscal year 2002. Some of these 

analyses also include planned resources for fiscal year 2004. For example, 

the resource analysis section for the Small Business Development Centers 

(SBDC) program states that for fiscal year 2004 SBA requested 

approximately the same level of funding as in fiscal year 2002. In 2002, 85 

percent of the funds were for grants, while the other 15 percent covered 

field support, program management, and overhead costs such as rent, legal 

services, human resources, and information technology support. Some of 

the resource analyses also contained pie charts of the breakdown of costs. 

This is an improvement over SBA’s 1999 performance plan, which we found 

did not specifically identify the human or technological resources that SBA 

would need to achieve its performance goals.

SBA’s Fiscal Year 2004 

Performance Plan Provides 

General Confidence That 

Performance Data Will Be 

Credible

SBA’s 2004 performance plan provides general confidence that its 

performance data will be credible. An appendix of the performance plan 

contains verification and validation information, as well as data limitations 

and remedies for these limitations for most of SBA’s performance 

indicators. However, the appendix does not include this information for 

the performance indicators of the disaster loan program, nor are any of the 

PMA performance indicators discussed in the appendix.

Generally, the discussions of SBA’s verification and validation processes for 

its indicators in the 2004 plan are one- or two-sentence statements. For one 

of the indicators, “number of jobs created and retained by the 7(a) loan 

program,” SBA states that it does not have access to the data for 

verification purposes. SBA also notes that it does not independently verify 

some of the external data it gathers, as is stated in the verification 

discussion of the indicator, “504 loans to emerging market firms.” This is 

an improvement over SBA’s 1999 performance plan, which included brief 

descriptions, often only one or two words, on the means it used to verify 

and validate its data. We noted in our report on the 1999 plan that these 

appeared to be sources of data for the measures rather than means to verify 

and validate the data.

The data limitations contained in SBA’s 2004 performance plan are 

generally one-sentence statements and the same limitations are used for 
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 179 GAO-04-38 Results-Oriented Government

multiple indicators. For example, several limitations, such as “the measure 

is based on the number of approved loans” or “information is derived from 

loan approval data,” are used for multiple indicators. The appendix also 

lists remedies for the data limitations for each indicator. For example, a 

limitation of SBA’s indicator “small business appointments conducted with 

procurement officials” is that the indicator may not capture unscheduled 

appointments. The remedy for this limitation is to keep track of both 

scheduled and unscheduled appointments. The discussion of data 

limitations and their remedies in the 2004 plan shows progress over SBA’s 

1999 plan, which did not contain a discussion of data limitations.

Observations on 

Changes in the Quality 

of SSA’s Annual 

Performance Plan

Compared to the fiscal year 1999 plan we reviewed, SSA’s performance plan 

provided a clear picture of intended performance by (1) defining expected 

performance, (2) offering trend data, which helps track progress toward 

performance goals, and (3) using objective, measurable, and quantifiable 

performance measures. SSA also provided general information on its 

strategies and resources, somewhat better than our 1999 findings. Finally, 

the plan provided a general level of confidence that data will be credible by 

describing the Inspector General’s (IG) involvement in data testing, 

providing data sources and definitions, and identifying some data 

weaknesses, an improvement over the 1999 plan. However, SSA’s 

performance plan still does not fully discuss the agency’s coordination with 

other agencies, identify performance goals that clearly cover all the 

program activities, address how SSA plans to use the information from the 

evaluations to improve program results, identify the resources needed to 

address each performance goal, and discuss data verification and 

validation procedures for its internal systems.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 180 GAO-04-38 Results-Oriented Government

SSA’s Fiscal Year 2004 

Performance Plan Provides 

a Clear Picture of Intended 

Performance

Overall, SSA’s fiscal year 2004 performance plan has improved over its 1999 

plan.16 The 1999 plan only provided a partial picture of SSA’s intended 

performance across the agency. In June 1998, we reported that SSA’s 1999 

Annual Performance Plan contained performance goals, many of which 

were measurable and linked to the agency’s strategic goals17 and objectives; 

some of the performance goals related to particular strategic goals were 

objective, measurable, and quantifiable. However, other goals were not 

measurable or quantifiable and did not define the level of performance to 

be achieved, thus making it difficult to see how SSA would assess success.

SSA’s fiscal year 2004 plan provides a much clearer picture of intended 

performance through (1) defining expected performance, (2) offering trend 

data, which helps track progress toward performance goals, and (3) the use 

of objective, measurable, and quantifiable performance measures. For 

example, as part of the strategic objective to “Prevent fraudulent and 

erroneous payments and improve debt management,” SSA provided 

historical data on the outcome measure “Percent of SSI payments free of 

preventable error (overpayments and underpayments)” from fiscal years 

1999-2001 and projected goals for fiscal years 2002-2004.

While we found significant improvements in SSA’s 2004 annual 

performance plan over its 1999 plan, we also found some weaknesses. For 

example, coordination efforts with other entities, such as federal agencies, 

state and local entities, and others, are not well identified. According to 

SSA officials, SSA coordinates with other federal agencies, such as the IRS 

and the Immigration and Naturalization Service, as well as Veterans 

Administration, on information-sharing initiatives. However, these types of 

coordination efforts are mentioned only briefly, if at all, in the 2004 annual 

performance plan.

In its 2004 plan, SSA includes a list of major program evaluations it plans to 

conduct during 2003-2004, with a brief description of the evaluations, their 

associated strategic goals, and projected completion dates. However, there 

is no indication how SSA plans to use the information from the evaluations 

16Much of this improvement took place between the 1999 and 2000 plans. We reported that 

SSA’s fiscal year 2000 performance plan showed significant improvement over its 1999 plan 

in U.S. General Accounting Office, Observations on the Social Security Administration’s 

Fiscal Year 2000 Performance Plan, GAO/HEHS-99-162R (Washington, D.C.: July 20, 1999).

17SSA reduced its strategic goals from five to four in its 2003-2008 strategic plan.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 181 GAO-04-38 Results-Oriented Government

to improve program results. The plan could be enhanced if the 

descriptions of these evaluations included the manner in which SSA 

planned to use the information gathered to improve its programs.

SSA’s Fiscal Year 2004 

Performance Plan Provides 

a General Discussion of 

Strategies and Resources

SSA’s 1999 plan had little discussion of the relationship between SSA’s 

mission, goals, and budget activities. Throughout the document, the fiscal 

year 2004 plan included clearer discussions of the linkage between SSA’s 

mission and goals. It also provided performance data dating back to fiscal 

year 1999, essential to making comparisons between prior and proposed 

levels of performance. The 2004 performance plan noted that the 

Limitation on Administrative Expenses account, SSA’s basic administrative 

account, is an annual appropriation that covers everything from salaries 

and benefits of SSA federal employees (excluding IG) to systems and 

telecommunications activities. SSA provided information on the funding 

sources of this account, including some of its budget accounts.

In its fiscal year 2004 plan, SSA provided information on the strategies it 

plans to use in addressing its key strategic objectives. The plan included a 

summary chart, showing the strategic objectives associated with each 

strategic goal, as well as the performance measures under each objective. 

In addition, the “means and strategies” section associated with each 

strategic objective identified strategies that support items in the PMA, GAO 

and IG major management challenges, and Social Security Advisory Board 

recommendations. In our October 2002 report Performance and 

Accountability: Reported Agency Actions and Plans to Address 2001 

Management Challenges and Program Risks,

18 we noted that SSA 

identified directly related goals and measures for five of its six challenges, 

and had strategies (without goals or measures) for the sixth challenge.

It is difficult to determine whether or not the annual performance plan 

identifies annual performance goals that cover all of the program activities 

in the agency’s budget, as well as the financial, human capital, and 

information technology resources needed to address each individual goal. 

General human capital requirements and goals are identified as part of 

SSA’s strategic goal to strategically manage and align staff to support SSA’s 

mission. The plan is neither structured by program activity nor account. 

SSA noted that it aligned its strategic goals, performance measures, and 

18GAO-03-225.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 182 GAO-04-38 Results-Oriented Government

budget with its major functional responsibilities rather than by program 

accounts since direct service and support employees provide services 

linked to these functional responsibilities, as opposed to a specific 

program. However, SSA does not indicate what it means by “functional 

responsibilities,” nor does it show a clear link between its strategic goals 

and such responsibilities.

As in the fiscal year 1999 plan, the fiscal year 2004 plan included a 

discussion of external factors that could affect the achievement of its 

goals.19 SSA identified strategies to alleviate some, but not all of, the 

factors. For example, SSA plans to mitigate the loss of institutional 

knowledge through SSA’s “retirement wave” through the use of employee 

development programs, redeploying positions to direct service, hiring 

Presidential Management Interns, and the increased use of hiring 

flexibilities. However, the discussion of factors affecting SSA’s solvency 

strategic goal merely notes that Social Security programs must respond to 

related developments.

SSA’s Fiscal Year 2004 

Performance Plan Provides 

General Confidence That 

Performance Data Will Be 

Credible

SSA’s 1999 plan stated that the Office of the Inspector General was 

responsible for reviewing the data systems underlying its performance 

measures, but did not provide further details that would assure the reader 

that SSA is taking the steps necessary to ensure data integrity. In contrast, 

SSA’s fiscal year 2004 plan provided data sources and definitions for each 

performance measure. SSA’s fiscal year 2004 plan identified data 

limitations related to performance measures, as well as some efforts to 

correct or address data weaknesses. When performance indicators and 

goals are not quantified, SSA describes its benchmarks for goal 

achievement. For example, for the outcome measure “Developing new 

performance management systems,” SSA defines “Implementing the new 

SES system” as its goal for 2003.

As in the fiscal year 1999 plan, SSA notes that the IG’s office is involved in 

the data system reliability process. In the fiscal year 2004 plan, SSA went 

further to explain the IG’s four-point approach to reviewing performance 

measures, including assessing whether the reported performance measure 

data are valid. SSA also noted that performance data for its quantifiable 

measures are generated by automated management information and 

19SSA refers to external factors as environmental factors.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 183 GAO-04-38 Results-Oriented Government

workload measurement systems, as a by-product of routine operations. 

However, there is no discussion of verification and validation procedures 

for data generated by these systems.

Observations on 

Changes in the Quality 

of DOT’s Annual 

Performance Plan

DOT’s annual performance plan for fiscal year 2004 showed evidence of 

improvements in areas that we previously identified had shortcomings in 

our 1998 review of DOT’s 1999 performance plan.20 The 2004 plan provides 

a clear picture of intended performance with DOT’s measures and 

performance goals now being clearly linked to the department’s strategic 

objectives. A specific discussion of DOT’s strategies and resources in the 

plan includes numerous and detailed strategies for achieving DOT’s 

performance goals, and the resources needed for those strategies. 

Procedures to verify and validate data, as well as known data limitations, 

are described for each performance measure providing full confidence in 

the credibility of DOT’s performance data. Still, the performance plan 

could be improved by including a discussion of, and performance measures 

for, each of DOT’s program activities and by more consistently describing 

DOT’s role in crosscutting programs.

DOT’s Fiscal Year 2004 

Performance Plan Provides 

a Clear Picture of Intended 

Performance

DOT’s fiscal year 2004 performance plan shows evidence of many of the 

same strengths as, and a few improvements over, its fiscal year 1999 

performance plan and provides a clear picture of intended performance. 

The 2004 plan lists outcome goals, performance goals, and measures by 

strategic objective, all of which are generally objective, quantifiable, and 

can show progress toward DOT’s strategic objectives. For example, the 

measure “fatalities per 100 million vehicle-miles of travel” will gauge 

progress toward the performance goal “reduce highway fatalities per 100 

million vehicle-miles traveled to no more than 1.0 in 2008, from 1.7 in 1996.” 

The data gathered by the measure will also show progress toward the 

related outcome of “reduce the number of transportation-related deaths” 

for DOT’s “safety” strategic objective. This is an improvement over DOT’s 

1999 plan in which we found that DOT’s performance goals typically 

covered only a portion of the strategic goals and the link between annual 

performance goals and strategic goals could be improved.

20GAO/RCED-98-180R.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 184 GAO-04-38 Results-Oriented Government

DOT’s plan also presents trend and baseline data for each performance 

measure and goal. For the example given above, the plan indicates the 

performance targets from 1999 to 2004 and also presents actual data for 

these targets for 1999 to 2002. In its 1999 plan, DOT had provided baseline 

data for most of its performance goals and measures as well. This 

information, along with the clearly linked performance goals and strategic 

objectives, helps to show DOT’s progress in achieving its goals.

As in the 1999 plan, DOT’s 2004 performance plan generally covers each 

program activity in its budget request for fiscal year 2004. An appendix to 

the performance plan lists DOT’s program activities and indicates the 

proposed funding level for each program by strategic objective. However, 

as in its 1999 plan, a few programs do not seem to be linked to the strategic 

objectives elsewhere in the plan. Capital grants to the National Passenger 

Rail Corporation (Amtrak) and the Bureau of Transportation Statistics’ 

Office of Airline Information are both linked to the “mobility & economic 

growth” strategic objective in the budget crosswalk, but they do not appear 

in the discussions contained within that strategic objective section. When 

the 2004 plan was published in February 2003, DOT had not yet released its 

new reform strategy for Amtrak, which was made public in July 2003. Still, 

the inclusion of information on Amtrak, as well as a discussion of the 

Bureau of Transportation Statistics’ Office of Airline Information, would 

provide a clearer picture of how DOT intends to achieve its goals.

The discussions of each performance goal have sections entitled “Other 

Federal Programs with Common Outcomes.” In this section, the plan 

describes crosscutting programs and other agencies with which DOT 

works. For example, the plan states that the Research and Special 

Programs Administration of DOT continues to develop the National 

Pipeline Mapping System with the Federal Energy Regulatory Commission, 

the National Oceanic and Atmospheric Administration, the Department of 

Energy, the U.S. Geological Survey and others, in order to help analyze 

risks to environmentally sensitive and populated areas. This supports 

DOT’s efforts to reduce pipeline incidents. Yet for several goals, 

coordination efforts are not described. One example of this is in the 

highway congestion section where the plan states that the Federal Highway 

Administration works closely with the Department of the Interior, 

Department of Agriculture, and Department of Defense agencies to 

improve mobility on federally owned lands. However, the plan does not 

describe the specific actions that are being taken to improve mobility. Our 

1998 report stated that DOT’s contribution or role was not described in 

many of the crosscutting programs listed in DOT’s 1999 performance plan.
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 185 GAO-04-38 Results-Oriented Government

The 2004 performance plan generally addresses performance and 

accountability challenges we previously identified. The discussions of 

these management challenges are included in the plan by the performance 

goal and programs to which they are related. For example, in discussing 

highway safety, DOT addresses our concerns on transportation safety, 

specifically through the use of safety belts. The strategies include 

continuing the National Highway Traffic Safety Administration’s safety belt 

outreach to high-risk populations and encouraging states to embrace “Click 

It or Ticket” as the message or theme for their Buckle Up Campaigns. A 

performance measure related to this management challenge included in the 

plan is “percentage of front seat occupants using safety belts.” However, 

not all of the management challenges have related measures and goals. For 

example, we have previously identified building human capital strategies as 

a management challenge for DOT. A section within the plan focuses on an 

“organizational excellence” objective to implement the PMA. Strategic 

management of human capital strategies is discussed in this section but no 

goals or measures are given to show DOT’s progress with these strategies. 

Still, this shows some improvement over the 1999 plan, which generally 

covered management challenges, but did so in a separate appendix without 

explaining how the challenges were related to the rest of the plan. We 

noted that this area could be improved by including goals and measures 

related to resolving these challenges.

DOT’s Fiscal Year 2004 

Performance Plan Provides 

a Specific Discussion of 

Strategies and Resources

DOT’s 2004 performance plan shows several improvements over its 1999 

performance plan, providing a specific discussion of strategies and 

resources. Discussions of each performance goal include a section titled 

“Strategies and Initiatives to Achieve 2004 Target.” These sections include 

a variety of means by which DOT intends to accomplish its performance 

goals. One example would be for DOT’s performance goal to “reduce 

pipeline hazmat (hazardous materials) spilled 30 percent by 2006, from the 

last five years’ average spill rate.” The strategies for this goal include 

enforcing operator qualification requirements, expanding monitoring 

technology that can help prevent construction-related damage to pipelines, 

and developing regulatory standards for leak detection technology. This 

shows progress from when we reported that DOT’s 1999 performance plan 

lacked sufficient information to clearly link the strategies to performance 

goals in many cases.

In contrast to its 1999 performance plan, DOT’s 2004 performance plan 

generally discusses the human, capital, information, and other resources 

needed to meet its performance goals. Each performance goal section of 
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 186 GAO-04-38 Results-Oriented Government

the performance plan includes a graph showing the enacted funding for 

fiscal year 2002, and the proposed funding for fiscal years 2003 and 2004. 

The organizational excellence objective for DOT describes DOT’s human 

capital and information technology resources and strategies. For example, 

one of DOT’s strategies for strategic management of human capital is to 

“establish a corporate approach to target recruitment efforts, with special 

emphasis on cross-modal, mission-critical occupations,” which includes a 

pilot program for centrally recruiting and training entry-level employees for 

one or more mission-critical occupations.

The 2004 plan also discusses external factors that could hamper DOT’s 

ability to achieve its performance goals. In our review of DOT’s 1999 

performance plan, we noted that the plan could be improved by 

recognizing more external factors and by discussing actions that DOT 

could take to mitigate the effects of these factors. In contrast, external 

factors are listed for most of the performance goals in the 2004 plan. For 

its transportation accessibility goals, DOT states that as the population 

ages, more people will require accessible public transit, for which states 

and local agencies decide how best to allocate federally provided 

resources. One of the strategies that addresses this external factor is the 

Special Needs of Elderly Individuals and Individuals with Disabilities 

grants, which DOT states will help meet the transportation needs of the 

elderly and persons with disabilities when regular transportation services 

are unavailable, insufficient, or inappropriate to meet their needs.

DOT’s Fiscal Year 2004 

Performance Plan Provides 

Full Confidence That 

Performance Data Will Be 

Credible

The 2004 plan provides full confidence that DOT’s performance data will be 

credible. As in the 1999 performance plan, the 2004 performance plan 

contains a section, entitled “Performance Data and Performance 

Measurement,” that discusses the means that DOT uses to verify and 

validate its data. But unlike the 1999 plan in which this discussion was 

broad and not linked to specific goals and measures, the 2004 plan also 

contains an appendix that provides the following for each of DOT’s 

measures: the source of the data, limitations of the data, observations on 

the quality of the data, work planned or ongoing to improve data quality, 

and any known biases. Finally, DOT has compiled source and accuracy 

statements,21 which provide more detail on the methods used to collect the 

21Bureau of Transportation Statistics, Source & Accuracy Compendium, 

http://www.bts.gov/statpol/SAcompendium.html (Washington, D.C.: Aug. 15, 2003).
Appendix IV

Observations on Agencies’ Annual 

Performance Plans

Page 187 GAO-04-38 Results-Oriented Government

data, sources of variation and bias in the data, and methods used to verify 

and validate the data.

The presentation of data limitations in DOT’s 2004 performance plan also 

shows progress from its 1999 plan. The Performance Data and 

Performance Measurement section includes a general discussion of DOT’s 

data limitations. This discussion includes limitations for the internal and 

external data used by the department. Specific limitations for internal data 

can be found in the aforementioned source and accuracy compendium, 

while details on the limitations of external data are given in the appendix 

on performance measures. In our report on DOT’s 1999 performance plan, 

we stated that information on data limitations was lacking for most 

measures and that the plan could be improved by more consistently 

addressing the data limitations throughout the plan.
Page 188 GAO-04-38 Results-Oriented Government

Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports ApendxiV

To help Congress and the President determine agencies’ actual 

performance and progress in achieving strategic plan goals, GPRA requires 

each agency to prepare a report on program performance for the previous 

fiscal year.1

 One of our objectives was to assess the overall quality of 

agencies’ annual performance and accountability reports and the extent to 

which selected elements of agency reporting have improved. To meet this 

objective, we judgmentally selected six agencies—Education, DOE, HUD, 

SBA, SSA, and DOT—using criteria, such as agency size, primary program 

type, and previous GAO reviews. To assess the overall quality and 

improvements made to the agencies’ performance and accountability 

reports, we relied on requirements and guidance contained in GPRA and 

accompanying committee report language,2

 guidance to agencies from 

OMB for developing performance reports,3 interviews with agency officials, 

the Chief Financial Officers Act,4 our previous reports,5 and our knowledge 

of agencies’ operations and programs. To assess the quality of the six 

agencies’ performance and accountability reports, we categorized each 

report based on the degree to which it addressed three characterizations: 

(1) picture of performance, (2) link between resources and results, and 

(3) credibility of performance information.

To assess the degree to which an agency’s report provided a clear picture of 

performance across the agency, we reviewed the extent to which the report 

addressed elements required by GPRA. The annual performance report 

should:

• describe the performance indicators established in the agency’s annual 

performance plan, along with the actual program performance achieved 

compared with the performance goals expressed in the plan for that 

fiscal year;

1

Office of Management and Budget, Memorandum: Program Assessment Rating Tool 

(PART) – Presentation in Congressional Justifications, M-03-06 (Washington, D.C.: 2003).

2

Government Performance and Results Act of 1993, Committee on Governmental Affairs, 

United States Senate, S. Rpt. No. 58, 103d Cong. 1st Sess. (1993).

3

OMB Circular No. A-11, Part 6, Preparation and Submission of Strategic Plans, Annual 

Performance Plans, and Annual Program Performance Reports (Washington, D.C.: June 

2002).

4

Chief Financial Officers Act of 1990 (Pub. L. No. 101-576).

5

GAO-02-372 and Executive Guide: Creating Value Through World-class Financial 

Management, GAO/AIMD-00-134 (Washington, D.C.: Apr. 1, 2000).
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 189 GAO-04-38 Results-Oriented Government

• review the success of achieving the performance goals of the fiscal year;

• provide actual results for the 3 preceding fiscal years;

• evaluate the performance plan for the current fiscal year relative to the 

performance achieved toward the performance goals in the fiscal year 

covered by the report; 

• explain and describe where a performance goal has not been met or a 

corresponding level of achievement if an alternative form is used, as 

well as why the goal was not met, plans and schedules for achieving the 

established performance goal, and if the performance goal is impractical 

or infeasible;

• describe the use and assess the effectiveness of achieving performance 

goals of any waivers; and 

• include the summary findings for those program evaluations completed 

during the fiscal year covered by the report.6

We also looked at the extent to which the reports clearly discussed 

progress achieved in addressing the major management challenges 

previously identified by us or others. For agencies that choose to issue a 

performance and accountability report, the Reports Consolidation Act of 

2000 requires that the report include a summary of the most serious 

management and performance challenges facing the agency, as identified 

by their IG, and a brief assessment of the agency’s progress in addressing 

those challenges.

In assessing the clarity of the performance information, we also looked at 

selected qualitative characteristics used by the Association of Government 

Accountants, in conjunction with the Chief Financial Officers Council, in 

assessing performance and accountability reports for the Certificate of 

Excellence in Accountability Reporting.7

 These characteristics included 

(1) whether there was a clear relationship between the performance 

6

The Homeland Security Act (Pub. L. No. 107-296) requires agencies to include a review of 

the performance goals and evaluation of the performance plan relative to the agency’s 

strategic human capital management.

7

Association of Government Accountants, Certificate of Excellence in Accountability 

Reporting: Reviewers Checklist, Fiscal Year 2001. (Washington, D.C.).
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 190 GAO-04-38 Results-Oriented Government

information in the report and the goals and objectives contained in the 

strategic and annual performance plans, (2) the extent to which the agency 

limited the measures it discussed to those that were most significant for its 

programs, and (3) the extent to which the report was user friendly by being 

well-organized, concise, readable, and making effective use of graphics to 

ease understanding of narrative information. We characterized the clarity 

of each report in one of four ways: (1) clear, (2) general, (3) limited, or 

(4) unclear, based on the extent to which the 2002 report addressed the 

elements required by GPRA and the other informative practices we 

described.

Both GPRA and the CFO Act emphasized the importance of linking 

program performance information with financial information as a key 

feature of sound management and an important element in presenting to 

the public a useful and informative perspective on federal spending. 

Similarly, the current administration’s ambitious agenda for performance 

budgeting, calling for agencies to better align budgets with performance 

goals and focus on capturing full budgetary costs and matching these costs 

with output and outcome goals, suggests that agencies need to develop 

integrated financial and performance management systems that will enable 

the reporting of the actual costs associated with performance results. 

Although linking resources to performance goals is not a requirement of 

GPRA, the committee report for GPRA suggested that developing the 

capacity to relate the level of program activity with program costs, such as 

costs per unit of result, costs per unit of service, or costs per unit of output, 

should be a high priority. We have reported that world-class financial 

management practices call for enterprisewide systems to integrate 

financial and operating data to support both management decision making 

and external reporting requirements. To assess the degree to which an 

agency’s report discussed the relationship between resources and results, 

we characterized each report as having a (1) clear relationship, (2) general 

relationship, (3) limited relationship, or (4) no relationship.

Finally, to assess the degree to which an agency’s plan provided confidence 

that the agency’s performance information would be credible, we examined 

how each report discussed the quality of the data presented. To help 

improve the quality of agencies’ performance data, Congress included a 

requirement in the Reports Consolidation Act of 2000 that agencies assess 

the completeness and reliability of their performance data. Under the act, 

agencies were to begin including this assessment in the transmittal letter 

with their fiscal year 2000 performance reports. Agencies were also 

required to discuss in their report any material inadequacies in the 
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 191 GAO-04-38 Results-Oriented Government

completeness and reliability of their performance data and discuss actions 

to address these inadequacies.

We have previously reported on other practices that enhance the credibility 

of performance data that are not specifically required by GPRA.8

 For 

instance, discussions of standards and methods used by agencies to assess 

the quality of their performance data in their performance reports provides 

decision makers greater insight into the quality and value of the 

performance data. We also reported on additional practices, in several 

agencies’ performance reports, that would help foster transparency to the 

public and assist decision makers in understanding the quality of an 

agency’s data. The additional practices we observed included 

(1) discussions of data quality, including known data limitations and 

actions to address the limitations and (2) discussions of data verification 

and validation procedures. To address the extent to which a report 

provided confidence that performance information was credible, we 

characterized each report as providing (1) full confidence, (2) general 

confidence, (3) limited confidence, or (4) no confidence.

In conducting our reviews, to the extent information was available in prior 

assessments, we compared our findings of agencies’ fiscal year 2002 

reports to our assessments of reports for fiscal year 1999.9

 A more detailed 

discussion of our scope and methodology and the criteria we used can be 

found in appendix I. Table 13 shows the results of our assessment of the 

six agencies’ reports.

8

GAO-02-372.

9

GAO/HEHS-00-128R, GAO/RCED-00-209R, GAO/RCED-00-211R, GAO/RCED-00-207R, 

GAO/HEHS-00-126R, and GAO/RCED-00-201R.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 192 GAO-04-38 Results-Oriented Government

Table 13: Characterizations of Agencies’ Fiscal Year 2002 Annual Performance and 

Accountability Reports

Sources: U.S. Department of Education, U.S. Department of Education FY 2002 Performance and Accountability Report (Washington, 

D.C.: 2003); U.S. Department of Energy, Performance and Accountability Report, Fiscal Year 2002 (Washington, D.C.: 2003); U.S. 

Department of Housing and Urban Development, Fiscal Year 2002 Performance and Accountability Report (Washington, D.C.: 2003); 

Small Business Administration, Fiscal Year 2002 Performance and Accountability Report (Washington, D.C.: 2003); Social Security 

Administration, Performance and Accountability Report, Fiscal Year 2002 (Washington, D.C.: 2002); and U.S. Department of 

Transportation, Fiscal Year 2002 Performance and Accountability Report (Washington, D.C.: 2003).

The remainder of this appendix discusses our observations on the quality 

of the agencies’ annual performance and accountability reports we 

reviewed and, to the extent information was available from our prior 

reviews, how the quality has changed since the agencies submitted their 

first reports on fiscal year 1999 performance. We did not independently 

verify or assess the information we obtained from agency annual 

performance reports. If an agency chose not to discuss its efforts 

concerning elements in the report, it does not necessarily mean that the 

agency is not implementing those elements.

Observations on the 

Quality of Education’s 

Fiscal Year 2002 

Performance and 

Accountability Report

Education’s fiscal year 2002 Performance and Accountability Report 

comprises two volumes—the main volume and a second volume including 

performance reports for the agency’s individual programs. In our 

assessment, we did not review the second volume, which includes very 

detailed, discrete, and disaggregated performance information with over 

350 individual measures for the Office of Civil Rights, IG, and 117 

Education programs in 60 clusters.

Although Education’s report included many features designed to present its 

performance information clearly, the overall clarity was limited by the 

Department/agency

Characterizations 

Picture of 

performance

(unclear, limited, 

general, clear)

Resources linked 

to results

(no, limited, 

general, clear)

Data credible 

(no, limited, 

general, full)

Department of Education Limited Clear General

Department of Energy General Limited Limited 

Department of Housing and 

Urban Development

General No General

Small Business Administration Limited General General

Social Security Administration General Limited General

Department of Transportation General No Full
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 193 GAO-04-38 Results-Oriented Government

significant amount of performance information that was unavailable to 

show Education’s performance results. In contrast, Education’s report very 

clearly related its performance to its costs by using both graphics and text 

to provide the agency’s estimate of appropriations associated with 

achieving each of its six strategic goals, 24 objectives (long-term goals), 

and individual programs. Finally, Education provided a general level of 

confidence in the quality of its data, primarily because of its recognition of 

the challenges it faces on the timeliness, reliability, and validity of its data. 

Education’s recent efforts in undertaking a performance-based data 

management initiative in partnership with state leaders to allow timely and 

ready access to high-quality achievement and other performance data, 

which the IG said would address many of the related concerns identified 

during IG audits, also aided in the level of confidence in the data. 

Education’s Fiscal Year 2002 

Report Provided a Limited 

Picture of Performance

Education’s 2002 performance report is directly aligned with the goals and 

measures in the agency’s 2002-2007 strategic plan and its 2002-2003 annual 

plan. Of the 210 measures included in the agency’s strategic plan and 

annual plan, 120 were identified for measurement in fiscal year 2002, and 

all of these are addressed in the performance report. The report contains 

sections on changes planned to enhance performance on the basis of 

results. For each measure, the performance report includes trend data, 

with a table showing actual data from fiscal years 1999 through 2002; in 

some cases, the table also includes data from fiscal year 1998. When data 

are not provided, the table indicates that they were not applicable or not 

available. Overall, the report contains clear, succinct figures and a table 

summarizing the status of all 120 measures, as summarized in figure 19.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 194 GAO-04-38 Results-Oriented Government

Figure 19: Summary of Education’s Performance Indicators for Fiscal Year 2002

However, while Education’s 2002 report does review the levels of success 

for its performance goals10 for fiscal year 2002, there is a critical limitation. 

As we observed in our review of Education’s 1999 report,11 data were not 

yet available for many measures in the 2002 report. Specifically, 2002 data 

were available for only 41 of the 120 measures; the rest were characterized 

as “Pending: Data Not Yet Available” (63) or “Incomplete: Data Not 

Expected” (16). Despite the numerous strengths in Education’s 2002 

performance report, the picture of performance for Education presented in 

this report is limited mainly because of the lack of data for so many of its 

2002 targets. However, Education recognizes the challenges created by its 

limited access to timely, reliable data:

We still face significant challenges to meeting our national education goals. Primary among 

these challenges is access to timely, reliable data on our performance in meeting our goals 

and implementing our programs. Our efforts to identify effective and ineffective programs 

10Education’s annual performance goals are represented by its targets.

11GAO/HEHS-00-128R.

52.5%

17.5%

13.3%

1.7%

Target almost met

4.2%

Set baseline

Data not expected

Target met or exceeded

Data not yet available

10.8%

Target not met

Source: GAO analysis of U.S. Department of Education's FY 2002 Performance and Accountability Report.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 195 GAO-04-38 Results-Oriented Government

in the Department are severely limited by the frequent absence of useful data about them. 

In FY 2002 we designed a performance-based data management initiative which will provide 

much more robust information about our programs and strategic objectives, as well as 

provide a strong foundation for educational research and evaluation.

This data management initiative is being undertaken in partnership with 

state leaders and the software industry and is expected to result in an 

electronic data system that will allow ready access to high-quality 

achievement and other performance data in a timely and seamless manner 

in the future. Because the lack of data for so many of its targets blurs the 

picture of performance, Education should take every possible step to 

complete, as quickly as possible, its newly established performance-based 

data management initiative.

While lacking data for so many of its measures, Education’s 2002 report 

provides an explanation for measures with pending or incomplete data. 

For pending data, the report states that comparisons to targets will be 

made in the subsequent performance and accountability report, in addition 

to citing the department’s performance-based data management initiative. 

The report further indicates that measures with incomplete data were so 

characterized because methods to collect data were not ready in time to 

measure fiscal year 2002 results, data collection did not occur, or data 

collection was delayed. The report goes on to say that, for these measures, 

Education will put methods in place to measure fiscal year 2003 results, 

develop other data sources, or revise its measures to measure results 

differently. In addition, for each incomplete measure, the report clearly 

describes why data are incomplete and what will be done to address the 

situation. For example, for its measure on the percentage of states with 

complete school accountability systems in place, as required by the No 

Child Left Behind Act,12 the report explains that the requirements under 

this act are more extensive than in the past, that states that had met prior 

requirements may not yet meet new requirements, that the department had 

decided regulation would be necessary, and that regulations had not been 

finalized to define a complete school accountability system.

In addition, the report almost always included explanations of performance 

and sometimes provided information on why targets were not met when 

that was the case. However, such information was not always easy to find, 

12Pub. L. No. 107–110, January 8, 2002. The No Child Left Behind Act of 2001 is a 

reauthorization of the Elementary and Secondary Education Act, one of the major pieces of 

authorizing legislation for Education.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 196 GAO-04-38 Results-Oriented Government

as the report did not always include it in the same area as the related 

measure.

For most measures, including those that did not meet their targets, the 

report provided information on the steps Education is taking or plans to 

take to improve or enhance performance. For example, the 2002 target for 

a measure on the number of children attending charter schools was 690,000 

and the actual result was 575,000. To improve performance on this 

measure, Education’s report says that it is distributing guidance and 

information to encourage parents to consider charter schools, using both 

publications and its Web site to promote charter school enrollment, and 

sponsoring a charter schools Web site with information on federal 

assistance for charter schools. However, it was not always clear how the 

steps cited would improve or enhance performance. For example, for four 

measures on advanced placement (AP) achievement that were not met in 

2002, the strategy given for improving performance is to continue to 

support increasing AP achievement through the Advanced Placement 

Incentives program, but the report does not include an explanation of, or 

any information on, this incentives program.

Education’s report included an appendix entitled “Findings from FY 2002 

Evaluations” and included summaries of the findings from nine GAO 

reports and eight other studies completed in fiscal year 2002. For example, 

the Education for Homeless Children and Youth Program: Learning to 

Succeed report comprised two studies that found that homeless students 

are best served when promising practices are implemented as part of a 

comprehensive and coordinated education program for the homeless.

The 2002 performance and accountability report also contained an 

appendix consisting of the IG’s summary of serious management 

challenges, including financial management, federal student aid programs, 

information technology, program performance and accountability, and 

human capital. For each of these challenges, the IG provided information 

on Education’s progress in addressing them. Under program performance 

and accountability, for example, the IG pointed out that a significant 

amount of the data used to measure education programs were provided by 

state and local education entities and that it is imperative that these data 

are accurate, so as to provide Congress, OMB, and the public with an 

objective measure of the success of education programs. The IG said that 

Education has recognized the importance of improving data quality and 

addressed this issue in its performance plan.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 197 GAO-04-38 Results-Oriented Government

The IG’s summary of serious management challenges also included 

references to GAO’s high-risk list with respect to federal student aid 

programs and human capital management, and Education’s report included 

measures related to both of these areas. Specifically, for its 2002 measure 

to have Federal Student Aid (FSA) leave the GAO high-risk list by 2003 and 

not return, the report states that the department “almost met” its 2002 

target by achieving 94 percent of its FSA High Risk Plan, and it described 

the shortfall as not significant or material. In contrast, in our review of 

Education’s 1999 Performance Report,13 we noted that the department did 

not have goals, objectives, or measures related to problems with its student 

assistance programs. In addition, for the six measures in Education’s 2002 

report under its strategic goal to improve the strategic management of its 

human capital, the department reports that four targets were pending, one 

was incomplete, and one had set the baseline. With respect to the GPRA 

requirement14 that agencies’ performance reports include a review of the 

performance goals and evaluation of the performance plan relative to the 

department’s strategic human capital management, Education’s report 

discusses its human capital management strategic goal and related 

performance goals in the context of its human capital management plan, 

One-Ed.

Education’s 2002 Report 

Showed a Clear 

Relationship between 

Resources and Results

Education’s 2002 report included information for each of its six strategic 

goals and 24 objectives that clearly linked the department’s resources with 

its efforts to achieve specific results. While the department was not able to 

break the costs down by each of its measures and targets, the report used 

both graphics and text to provide the department’s estimate of 

appropriations associated with achieving each of its six strategic goals, 24 

objectives (long-term goals), and individual programs. For example, for 

each of its objectives, the report used a pie chart to show the percentage of 

the department’s appropriation that supports the objective, the rest of the 

strategic goal the objective falls under, and the other five strategic goals. 

An example is shown in figure 20 for the objective to ensure that all 

students read on grade level by the third grade.

13GAO/HEHS-00-128R.

14As amended by the Homeland Security Act of 2002.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 198 GAO-04-38 Results-Oriented Government

Figure 20: Inputs: Allocating Funds for Education’s Objective to Ensure That All 

Students Read on Grade Level by the Third Grade

The text accompanying each chart listed the dollar amount supporting the 

objective’s activities, the percentage of the strategic goal’s allocation that 

amount represented, the individual programs that supported the objective, 

and the dollar amount from salaries and expenses that was included in the 

dollar amount for the objective. In addition, in the report’s appendixes, a 

table summarizing fiscal year 2002 appropriations and staffing allocated by 

goal and objective also included the FTEs under staffing for each strategic 

goal and objective. Another table provided a percentage breakdown of 

each objective’s appropriations by 146 agency programs.

Education’s Fiscal Year 2002 

Report Provided General 

Confidence That 

Performance Data Were 

Credible

Education’s 2002 performance report provided general confidence that the 

agency’s data were credible because of its recognition of the challenges it 

faces on the timeliness, reliability, and validity of its data; its 

straightforward disclosure of these challenges; and its recent efforts to 

address them. In Education’s transmittal letter, the department secretary 

said that the information contained in the report is “as complete and 

reliable as we have available.” However, Education’s report recognized 

that one of the agency’s significant challenges to meeting its national 

education goals is access to timely, reliable data on performance and that 

the lack of useful data severely limits efforts to identify effective and 

ineffective programs. The report further explained that 97 percent of the 

department’s funding is awarded to third parties, including, for example, 

Source: U.S. Department of Education’s FY 2002 Performance and Accountability Report (Washington, D.C. January 2003).

50%

18%

32% Other Goal 2

Objective 2.1

Other Goals
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 199 GAO-04-38 Results-Oriented Government

state and local agencies, that have an impact on the measurement of 

results, especially the timing of data collection. Thus, Education 

recognized in its report that it had limited control over the data it must use 

to report results. Similarly, in the IG’s summary of serious management 

challenges, the report noted that Education needs to improve its controls 

over the timeliness, reliability, and validity of data.

Moreover, Education’s report included information on recent steps it has 

taken to address its data challenges. In addition to a discussion of its data 

management initiative to develop an electronic data system providing 

access to timely, high-quality data, the report included an appendix with the 

agency’s Information Quality Guidelines and draft Data Quality 

Standards presented in an abbreviated format. The discussion of data 

quality standards recognizes the importance of data quality concepts to the 

process of developing high-quality performance measures. The eight 

standards provided are: validity, accurate definitions, accurate counts, 

editing, calculation, timeliness, reporting, and burden reduction. To 

facilitate the use of the standards, Education reported that it created a data 

quality checklist and regularly held classes to teach staff how to apply the 

standards. The IG’s summary of serious management challenges gave the 

department credit for this effort, pointing out that these guidelines address 

many of the concerns identified during IG audits and that the department 

plans to disseminate these guidelines to the chief state school officers.

The report also provided explanations of data sources and data quality for 

most measures. For example, for the measures on the percentages of 12th 

grade students scoring at or above the basic and proficient levels on the 

National Assessment for Educational Progress (NAEP) reading test, the 

source is given as: “U.S. Department of Education; National Center for 

Education Statistics, (NAEP); The Nation’s Report Card, Reading.” Under 

data quality, the report states that NAEP data are validated using rigorous 

National Center for Education Statistics statistical standards. For most of 

the measures, the explanations of data quality contain similar information 

on data validation. However, the data quality information only sometimes 

identifies what limitations are relevant, if any. For example, for a measure 

on the percentage of managers satisfied with services received from 

Education’s Office of Management when hiring staff, the department relied 

on an internal survey of managers for its data. Although the response rate 

for this survey was 22 percent, the report did not say whether this was a 

limitation to the data collected. For a few of Education’s measures, the 

report stated that no data limitations had been noted. It would be better if 
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 200 GAO-04-38 Results-Oriented Government

the agency clearly stated whether the data for each measure had limitations 

or not, and, if so, what they were.

Observations on the 

Quality of DOE’s Fiscal 

Year 2002 Annual 

Performance and 

Accountability Report

DOE’s 2002 Annual Performance and Accountability Report provided a 

general picture of intended performance by explaining in detail the 

progress made in meeting performance measures and addressing 

management challenges. It also provided a limited discussion of the costs 

incurred to achieve DOE’s performance goals by organizing its report by 

major program activities, the costs of these activities, and their 

corresponding goals. Finally, the report provided a limited level of 

confidence that data will be credible because the report did not include a 

discussion on data limitations.

DOE’s Fiscal Year 2002 

Report Provided a General 

Picture of Performance

DOE’s 2002 Performance and Accountability Report provided a general 

picture of performance in meeting its goals and measures. The report 

contained a detailed explanation of progress for each of its performance 

measures, which were referred to as “targets,” by identifying whether each 

measure was met, not met, or had mixed results, as shown in figure 21. In 

addition, the report identified the operational processes, technology, 

human capital, and other resources used to achieve each performance 

measure. The results for the past 3 years of performance measures related 

to each goal were also reported so that performance trends could be 

identified. However, the report did not clearly explain how the results of 

the performance measures reported contributed to achieving the 

performance goals in DOE’s annual performance plan and strategic plan.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 201 GAO-04-38 Results-Oriented Government

Figure 21: Summary of DOE’s Performance Indicators for Fiscal Year 2002

Each performance measure that was not met or had mixed results 

contained a plan of action to achieve the measure in the future. In addition, 

the majority of fiscal year 2002 measures that were not met contained a 

clear explanation as to why they were not met or had mixed results. For 

example, a measure that required Southeastern Power Administration to 

meet its planned repayment of principal of federal investment was not met 

due to severe drought. DOE’s report explained that to achieve the measure 

in the future, Southeastern plans to change its rate design, propose rate 

increases to obtain greater revenue, and increase cost recovery from fixed 

charges.

83.5%

5%

11.5% Mixed

Not met

Met

Source: GAO analysis of DOE’s Fiscal Year 2002 Annual Performance and Accountability Report.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 202 GAO-04-38 Results-Oriented Government

DOE also discussed the progress it made in addressing performance and 

accountability challenges. The report identifies significant issues for fiscal 

year 2002 that, as stated in the report, merit a higher level of attention and 

focus in the department. Each of the challenges were linked to a goal and 

its related performance measure(s). In addition, actions taken to address 

each challenge and the progress made on those actions were identified. 

For example, one of the challenges identified was the need for DOE to 

meet federal requirements for improved and more cost-effective use of 

information technology. A related goal to deal with this challenge was for 

DOE to promote the effective management of information technology 

resources in the department. To address this challenge, the report stated 

that DOE realigned its management structure for information technology 

issues, established an enterprisewide license for Microsoft software, and 

launched an e-government applications task force to identify high-priority 

e-government investments, among other actions. In our prior review of 

DOE’s 2001 performance report, we also found that DOE had made 

progress in addressing all eight of its major management challenges.15 

It is unclear, however, how program evaluations were used to assess 

performance because DOE did not include a summary of program 

evaluation findings in either the fiscal year 1999 or 2002 reports. According 

to DOE officials, a section on program evaluations was not included in 

fiscal year 2002 and one is not planned for fiscal year 2003 in order to limit 

the amount of detail included in the report.

DOE’s Fiscal Year 2002 

Report Showed a Limited 

Relationship between 

Resources and Results

DOE’s 2002 performance report provided a limited discussion of how its 

resources were related to its performance. As in our review of the 1999 

report, DOE’s 2002 report, which also includes DOE’s financial information, 

continued its practice linking the department’s performance information to 

the costs of its program activities. For the majority of its program 

activities, the 2002 report included the program activity’s net costs and 

related performance information for fiscal years 1999 through 2002. During 

our review of DOE’s 1999 performance report, we were supportive of 

DOE’s efforts to link its performance goals and measures to the program 

activities in the President’s Budget. However, DOE has not moved beyond 

presenting its performance and cost information by program activity, 

instead of by strategic or annual performance goal or objective. A report 

15GAO-03-225.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 203 GAO-04-38 Results-Oriented Government

that presented cost and performance information by performance goals in 

addition to other presentations would more clearly identify the costs 

associated with the achievement of each goal. According to DOE officials, 

the department plans to link its individual performance measures to the 

costs of program activities in future reports.

DOE’s Fiscal Year 2002 

Report Provides Limited 

Confidence That 

Performance Data Are 

Credible

DOE’s reporting on data credibility has improved but is still limited. Based 

on our review of DOE’s 1999 performance report, a key improvement made 

to the 2002 performance report was the department’s ability to report on its 

data validation and verification processes. DOE’s 1999 report did not 

discuss the implementation of DOE’s verification and validation plan or 

provide any evidence that the data quality was sufficient for assessing the 

department’s performance. The 2002 report met some requirements of the 

Reports Consolidation Act by including a statement in the report’s 

transmittal letter assessing the completeness and reliability of the data. 

The letter did not discuss any material inadequacies with DOE’s 

performance data. The report also included a high-level discussion on how 

DOE will validate and verify its data and refers the reader to its 2003 annual 

performance plan for further details. For example, the report stated that 

DOE’s end-of-year reporting process includes certification by heads of 

organizational elements on the accuracy of reported results, followed by a 

review for quality and completeness by DOE’s Office of Program Analysis 

and Evaluation.

Although the department has improved on reporting its data verification 

and validation processes, it has not improved on reporting any existing 

data limitations. Neither the 1999 nor the 2002 report included an overall 

discussion of the limitations to the data or steps DOE would take to 

address those limitations, although the 2002 performance report did 

identify minor data limitations for a few specific goals. DOE officials 

stated that a discussion on data limitations was not included in the 2002 

report because the department already reports on this information in the 

annual performance plan and they thought it was redundant to put it in the 

report.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 204 GAO-04-38 Results-Oriented Government

Observations on the 

Quality of HUD’s Fiscal 

Year 2002 Annual 

Performance and 

Accountability Report

Compared to its fiscal year 1999 performance report, HUD’s fiscal year 

2002 Performance and Accountability Report provides a general picture of 

what the department accomplished by including, among other things, a 

report card listing its performance indicators with the corresponding 

achievement, a list of program evaluations concluded during the fiscal year, 

trend information for some of its performance indicators, a discussion of 

the department’s attempts to address its performance and accountability 

challenges, and visual aids to illustrate information on its performance. In 

a few instances, the report mentions whether certain performance goals 

are impractical or unfeasible. However, the report is not as clear as it could 

be because it does not (1) explain how it plans to address performance 

targets that were not met during the fiscal year; (2) include an evaluation of 

the fiscal year 2003 performance plan relative to the performance 

information presented in the performance report for the fiscal year; or 

(3) include an evaluation of the fiscal year 2003 performance plan relative 

to the performance information presented in the performance report for 

fiscal year 2002. The report does not show the relationship between 

resources and results by linking expended dollar amounts to specific 

program objectives. The report provides general confidence to the reader 

that the data presented are credible by providing background information 

on each performance indicator and discussing the results and analysis of 

the most recent data.

HUD’s 2002 Report Provided 

a General Picture of 

Performance

Overall, HUD’s fiscal year 2002 Performance and Accountability Report 

provides a general understanding of what the department’s mission is and 

what it accomplished during the previous fiscal year.16 Since we first 

reviewed its report for fiscal year 1999, HUD has made progress in 

developing its performance report to comply with GPRA.17 In reviewing 

HUD’s fiscal year 1999 performance report, we noted that it only contained 

performance information for three of the department’s four outcome 

measures. HUD’s report for fiscal year 2002 includes a report card for each 

strategic goal listing performance targets that were met. The report card 

also provides an explanation for many performance targets that were not 

marked as being met during the fiscal year. Although the report includes 

16U.S. Department of Housing and Urban Development, Fiscal year 2002 Performance and 

Accountability Report (Washington, D.C.: 2003).

17GAO/RCED-00-211R. 
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 205 GAO-04-38 Results-Oriented Government

visual aids to enhance readers’ understanding of progress made toward 

attaining performance targets, HUD could enhance the report by providing 

a summary of performance targets for all strategic objectives met (or not) 

during the fiscal year.

HUD’s performance report does not meet GPRA’s requirement of including 

an evaluation of its fiscal year 2003 performance plan relative to the 

performance attained by the department in fiscal year 2002. Including this 

evaluation could provide readers some assurance that HUD takes into 

account prior performance information, such as unmet goals, to manage its 

performance in the fiscal year already under way.

The report suggests that some of the performance targets that were not met 

during the fiscal year were impractical or unfeasible. For example, for its 

goal of “Increase the Rate of Homeownership,” HUD mentions that the 

indicator can be resistant to increases above an undetermined level 

because homeownership is not practical or desirable for all households. 

Broad economic conditions, including employment, incomes and interest 

rates can affect homeownership rates. Likewise, HUD will no longer track a 

performance indicator that measures the percentage of low-income 

housing units containing threats to health and safety, such as exposed 

wiring, unvented heaters, holes in the floor, and rodents. HUD mentions 

that this indicator is not included in the fiscal year 2003 annual 

performance plan because of the difficulty of attributing the results to its 

programs.

In several instances, HUD’s annual performance report lacks a discussion 

of how it plans to address unmet performance targets as required by GPRA. 

For example, while HUD substantially met almost half of its performance 

targets in fiscal year 2002, the report does not mention what steps the 

department will take to address some of its unmet performance targets 

(see fig. 22).
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 206 GAO-04-38 Results-Oriented Government

Figure 22: Summary of HUD’s Performance Indicators for Fiscal Year 2002

a

Rather than stating if some performance targets were met or not, HUD provided the following 

explanations: data not available; no performance goal for this fiscal year; third quarter of calendar year 

(last quarter of fiscal year, not entire fiscal year); calendar year ending in the current fiscal year; 

calendar year ending the previous fiscal year; other reporting period; results too complex to 

summarize; and baseline newly established.

HUD continued to build upon the strengths of its earlier report by including 

trend information for some performance indicators it used to measure 

progress toward its targets during the past fiscal year. While not presented 

consistently throughout the report, trend information provides a context to 

understand HUD’s performance and helps to show the extent to which 

HUD exceeded or fell short of expectations set for its performance targets. 

For instance, for HUD’s performance indicator that tracks increases in the 

share of welfare families residing in public housing that move from welfare 

to work each year, the report mentions that in fiscal year 2002 the rate was 

13.1 percent compared to 19.9 percent in fiscal year 2001. In preparing to 

implement the Temporary Assistance for Needy Families program, HUD 

originally estimated this indicator to be around 6.5 percent in fiscal year 

1997.

47%

29%

20%

2%

Program not funded

2%

Data not reliable

No explanation

Undetermineda

Substantially met

Source: GAO analysis of HUD’s Fiscal Year 2002 Performance and Accountability Report.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 207 GAO-04-38 Results-Oriented Government

HUD’s fiscal year 2002 report also mentions that it concluded several 

program evaluations during the fiscal year, a key requirement absent from 

its performance report for fiscal year 1999. The report also provides a brief 

summary of the main findings of these program evaluations. In fiscal year 

2002, HUD concluded and published reports on 21 program evaluations 

covering five of its strategic goals.

Similar to our findings on HUD’s previous performance reports,18 HUD’s 

fiscal year 2002 report discusses the steps the department took to address 

decade-long management challenges. For example, while HUD’s report 

mentions that deficiencies remain in its financial management systems, in 

fiscal year 2002 the department initiated a project to design and implement 

an integrated financial system. Similarly, to address staffing imbalances 

and human capital challenges, HUD implemented the last phase of its 

Resource Estimation and Allocation Process in January 2002 and started to 

implement the Total Estimation and Allocation Mechanism, a tool that 

collects actual workload accomplishments and staff usage within the 

various operating components at HUD.

HUD’s 2002 Report Showed 

No Relationship between 

Resources and Results

While HUD has made some improvements in how it presents cost 

information in its report, it is still not useful for linking program objectives 

to specific dollar expenditures. HUD’s report provides a summary of the 

cost of operations by each reporting segment, such as the total amount of 

money spent by the Federal Housing Authority and the Public and Indian 

Housing programs, and states that the total cost for fiscal year 2002 

operations was $33 billion. However, the report does not reconcile these 

costs to specific program performance objectives, limiting the reader’s 

ability to understand how HUD used its resources to carry out its 

objectives during the fiscal year.

HUD’s 2002 Report Provided 

General Confidence That 

Performance Data Are 

Credible

HUD’s fiscal year 2002 performance report generally informs the reader on 

critical issues about the reliability of its performance data, an issue that 

was not discussed in detail in its earlier report. In its transmittal letter, 

HUD briefly discusses that in some instances the data used in the report 

were either incomplete and/or unreliable. The report includes background 

information, results, analysis, and a discussion of the data used for each 

18GAO-03-225.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 208 GAO-04-38 Results-Oriented Government

performance indicator during the fiscal year. In discussing the data, in 

some instances HUD points out issues concerning data validity and 

accuracy and mentions steps HUD will take to correct problems. For 

example, to address problems with its indicator on the number of 

homeowners who have been assisted with the HOME program, HUD has 

established a team of managers, technical staff, and contractors to make a 

series of improvements to the Integrated Disbursement and Information 

System beginning in fiscal year 2003, which should reduce the need for data 

cleanup.

Observations on the 

Quality of SBA’s Fiscal 

Year 2002 Annual 

Performance and 

Accountability Report

SBA’s fiscal year 2002 annual performance report shows several 

improvements over the agency’s initial report. The report shows a general 

relationship between resources and results by including an analysis of 

resources used by each program in fiscal year 2002. A section on data 

validation and verification, which includes data limitations and remedies 

for those limitations, provides a general level of confidence in the 

credibility of SBA’s performance data. While the 2002 report includes a 

scorecard to show the agency’s overall performance, the report provides a 

limited picture of performance due to a lack of plans to meet unmet goals 

in the future and data that were unavailable to show progress towards a 

large share of SBA’s performance goals, among other reasons.

SBA’s 2002 Report Provided 

a Limited Picture of 

Performance

SBA’s 2002 performance report19 provides a limited picture of its 

performance. SBA’s 2002 report includes a scorecard that shows overall 

agency performance for its 2002 goals, including trend data (when 

available) from fiscal year 1999 to fiscal year 2002, the fiscal year 2002 goal, 

and a column showing the percentage of the fiscal year 2002 goal achieved. 

However, based on the performance information provided in the fiscal year 

2002 performance report, it can be difficult to gauge SBA’s progress in 

achieving its goals. This is similar to our findings on SBA’s 1999 report, 

which we noted was unclear as to how well SBA performed in achieving 

several of its performance goals for two of the key outcomes addressed in

19Small Business Administration, SBA’s Performance & Accountability Report for Fiscal 

Year 2002 (Washington, D.C.: 2003).
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 209 GAO-04-38 Results-Oriented Government

our 2000 report.20 Figure 23 summarizes SBA’s progress on its 19 

performance goals for fiscal year 2002.

Figure 23: Summary of SBA’s Performance Goals for Fiscal Year 2002

Data were unavailable for 10 of SBA’s 19 performance goals in 2002. For 

the nine goals that had performance data available, SBA met seven. SBA’s 

2002 performance report included explanations for all of its goals that were 

unmet, deemed infeasible, or for which data were not available. For 

example, the report states that “homes restored to pre-disaster conditions” 

and “businesses restored to pre-disaster conditions” are no longer goals 

because SBA is reviewing its outcome measures for the disaster loan 

program. Also, data were not available for the “customer satisfaction” goal 

in the disaster assistance program because a Customer Service Survey for 

disaster loan recipients was not issued during the fiscal year due to having 

not received final clearance from OMB. This contrasts to our findings on 

SBA’s 1999 report when the agency did not provide explanations for not 

20Our review of SBA’s fiscal year 1999 report, GAO/RCED-00-207R, focused on our 

observations on only three of SBA’s key outcomes, as well as the major management 

challenges addressed in the performance report. Since our review of SBA’s 2002 

performance report used somewhat different assessment criteria, we could not make valid 

comparisons on all aspects of the reports.

53%

10%

37%

Goal not achieved

n=19

Goal achieved

Data not available

Source: GAO analysis of SBA's FY 2002 Annual Performance and Accountability Report.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 210 GAO-04-38 Results-Oriented Government

meeting several of its performance goals. However, for the two goals that 

were unmet in 2002, “start-ups receiving 7(a) and 504 financing” and “jobs 

created and retained by SBIC clients,” the report does not describe any 

plans for achieving the goals in the future. The lack of information for over 

half of SBA’s performance goals and the absence of plans for achieving 

unmet goals limits our ability to assess the overall progress the agency 

made in fiscal year 2002, as well as the likelihood that it will improve its 

performance in the future.

Several other factors limited our ability to evaluate SBA’s performance in 

its fiscal year 2002 report. The report presents performance data in 

accordance with the goal structure of SBA’s 2003-2008 draft strategic plan. 

The goal structure contained in the fiscal year 2002 performance report 

does not directly correspond with the goal structure presented in the 2002 

performance plan. Only one of the report’s strategic goals, “Help Families 

and Businesses Recover from Disasters,” directly corresponds to the 2002 

performance plan. Similarly, not all of the performance goals listed in 

performance scorecards in both documents correspond. For example, the 

performance goal “start-ups receiving 7(a) and 504 loans viable 3 years 

after receiving loan,” listed in the scorecard in the 2002 report, is not listed 

in the 2002 performance plan. The report states that based on 2002 results 

SBA made “substantial modifications” to its fiscal year 2003 goals, but the 

report does not specifically discuss how the performance achieved in 2002 

could affect the achievement of the 2003 goals. Finally, the report does not 

include the findings of program evaluations completed in fiscal year 2002. 

SBA states that it was unable to conduct program evaluations in 2002 due 

to a lack of funding and that the agency has requested funding for program 

evaluations in fiscal years 2003 and 2004.

Similar to our findings on SBA’s 1999 performance report, the agency 

continues to provide information indicating the agency’s progress in 

addressing management challenges that have been previously identified. 

For example, we have previously observed that SBA needs to improve the 

quality of the performance measures that it uses for the disaster loan 

program. SBA states in its 2002 report that the agency is in the process of 

reevaluating its measures for the disaster loan program, and specifically 

that the methodology for measuring the number or percentage of homes 

and businesses restored through the program will be addressed by this 

review.

SBA’s 2002 performance report also discusses the agency’s strategic 

management of human capital. One section of the report relating to the 
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 211 GAO-04-38 Results-Oriented Government

PMA describes SBA’s transformation plan, which is to realign the agency’s 

“organization, operation, and workforce to better serve its small business 

customers.” An appendix to the report identifies fully developing and 

implementing the agency’s human capital management strategy as one of 

SBA’s major challenges. This section lists the actions that SBA needs to 

take to address this challenge as well as the progress the agency has made 

in implementing these actions. However, the report does not include a 

review of the performance goals and evaluation of the performance plan 

relative to the agency’s strategic human capital management, as required by 

the Homeland Security Act of 2002.

The report also contains two broad overviews and an appendix of GAO 

audits and recommendations, as well as a description of management 

challenges identified by the agency’s Inspector General. One chart, entitled 

“Status of GAO Reviews Conducted at SBA in FY 2002,” shows the review 

title, status of the review (open or closed), and the number of 

recommendations that came from these reviews. Another chart, entitled 

“Number of Open GAO Recommendations at End of FY 2002,” lists the 

GAO report number and title, the year it was issued, and the number of 

recommendations remaining open. Further detail is provided in an 

appendix to the performance report, which lists GAO’s outstanding 

recommendations, the status of the recommendations, and the estimated 

date of completion. Another appendix includes a report from SBA’s Acting 

IG that describes the most serious management challenges SBA faced in 

fiscal year 2002.

SBA’s 2002 Report Showed a 

General Relationship 

between Resources and 

Results

SBA’s 2002 performance report contains analyses of resources and results 

for SBA’s programs that show a general relationship between resources and 

results. In the description of each program’s performance, the report 

includes an analysis of the resources used by each program. For example, 

the fiscal year 2002 cost of the Advocacy Program was estimated to be 

$8 million, with 50 percent of the funds going to support the Office of 

Advocacy, 14 percent funding economic research, 11 percent for SBA’s 

executive direction support, 16 percent for fixed costs, and 9 percent going 

to human resources, information technology, and procurement. The report 

contains crosswalks that show the relationship between SBA’s strategic 

goals, outcome goals, performance goals, and programs. The resources 

used by each program can then be linked through this crosswalk to 

performance goals to generally show the resources needed for the results 

achieved towards the goals. However, the connection of resources to 
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 212 GAO-04-38 Results-Oriented Government

results could be more explicitly stated in the report if results and resources 

were also presented by performance goal.

SBA’s 2002 Report Provided 

General Confidence That 

Performance Data Are 

Credible

SBA’s 2002 performance report provides general confidence that the 

agency’s performance data are credible. In the letter transmitting its 2002 

performance report, SBA states that the performance data for its credit and 

procurement assistance programs are complete and reliable, based on a 

systematic review of these data. SBA further states that it is “working to 

improve the completeness and reliability of the performance data for the 

advice provided to small business through SBA’s resource partners.” Data 

for this aspect of SBA’s performance are collected through surveys, which 

the agency notes are neither consistent nor comparable, and from which 

client responses are difficult to obtain. This could be seen as a material 

inadequacy, of which a discussion is required by the Reports Consolidation 

Act. SBA discusses the actions it will take to address the quality of the 

surveys by stating in the transmittal letter that it is working to improve the 

survey instruments it uses to obtain performance data.

SBA provides a detailed discussion of each performance indicator in a 

section of the report on data validation and verification. For each 

indicator, the report provides a definition, source, information on 

validation, and means for data verification. The verification process for 

several measures includes audits, independent reviews, and consistency 

checks. However, for these measures this is the only discussion of 

verification procedures and no further details are provided. Also, for 

several measures, such as “number of start-up firms financed by 7(a) & 

504” and “regulatory cost savings to small businesses,” SBA states that it 

does not independently verify the data.

The report also addresses limitations to its data in the section on data 

validation and verification. In this section SBA states that it faces many 

challenges in acquiring high-quality data on both outputs and outcomes, 

from both internal and external sources. The strategies that SBA will use 

to address the shortcomings of its data quality are contained in this section 

as well, which include ensuring the validity of performance measures and 

data, fostering organizational commitment and capacity for data quality, 

assessing the quality of existing data, responding to data limitations, and 

building quality into the development of performance data.

The section on data validation and verification in the 2002 report discusses 

how SBA plans to remedy the limitations for each indicator. For example, 
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 213 GAO-04-38 Results-Oriented Government

for the “customer satisfaction rate” measure of the disaster loan program, 

the report states that the surveys used for this measure only determine the 

satisfaction of those who received disaster loans and therefore do not 

address the satisfaction of those who did not receive the loans. The 

remedy listed for this limitation is to expand the survey to include all 

applicants. This is an improvement from SBA’s 1999 report, which we 

noted did not discuss data limitations that could affect the quality of data 

used by SBA to assess its performance.

Observations on the 

Quality of SSA’s Fiscal 

Year 2002 Annual 

Performance and 

Accountability Report

On the whole, SSA’s 2002 performance report generally showed the 

agency’s progress towards its annual goals for fiscal year 2002.21 It showed 

continued emphasis on outcome-oriented goals and identified relevant 

results that were linked to individual strategic objectives. It also provided 

trend information, typically back to fiscal year 1999, and contained a brief 

discussion of the program evaluations completed during fiscal year 2002. 

SSA’s strategic goals were linked to financial resources at a very high level, 

but none of the performance goals were associated with costs; thus, the 

cost of achieving (or not achieving) a particular goal was not clear. 

Additionally, the SSA Commissioner certified that SSA’s data presentation 

was credible, but missing data and a lack of documentation of the methods 

and data used to measure its performance reduced the overall quality of the 

document.

SSA’s 2002 Report Provided 

a General Picture of 

Performance

SSA’s 2002 performance report exhibited a general description of 

performance, including the identification of 14 key performance indicators 

out of a total of 69 indicators. Similar to our review of SSA’s fiscal year 

1999 report,22 we found that many of SSA’s fiscal year 2002 goals and 

indicators were outcome oriented. In the fiscal year 2002 report, SSA 

plainly summarized progress on its 69 performance indicators, as shown in 

figure 24.

21Social Security Administration, Performance and Accountability Report, Fiscal Year 

2002 (Washington, D.C.: 2002).

22Our review of SSA’s fiscal year 1999 performance report (GAO/HEHS-00-126R) focused on 

our observations on five of the agency’s key outcomes, as well as the major management 

challenges addressed in the performance report. Since our review of SSA’s 2002 

performance report used different assessment criteria, we could not make valid 

comparisons on all aspects of the reports.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 214 GAO-04-38 Results-Oriented Government

Figure 24: Summary of SSA’s Performance Goals for Fiscal Year 2002

In SSA’s 1999 annual performance report, performance measures focused 

on activities rather than results, so it was difficult to determine the agency’s 

real progress in achieving results. For example, the measures directly 

related to the outcome “long-term disability benefits are reduced because 

people return to the workplace” did not sufficiently track progress toward 

this key outcome. One of the measures was to “begin implementation of 

the ‘Ticket to Independence’ program, contingent upon enactment of 

supporting legislation in FY 1998.”23 This measure was neither quantifiable 

nor measurable, and did not measure the number of beneficiaries achieving 

this outcome.

In the 2002 report, SSA identified relevant results that are linked to 

strategic objectives. For example, one of SSA’s objectives related to the 

strategic goal “deliver citizen-centered, world-class service” was to 

“maintain the accuracy, timeliness, and efficiency of service to people 

applying for Old Age and Survivors Insurance (OASI) and Supplemental 

Security Income (SSI) Aged benefits.” SSA reported on the timeliness of 

OASI and SSI claims, as well as the implementation of software and 

infrastructure for paperless processing of claims, as the relevant results.

23This program was an administration proposal to test allowing disabled beneficiaries to 

choose their own public or private vocational rehabilitation provider.

17%

63%

10%

10%

Data not yet available

Not met

Almost met

Met

Source: SSA's FY 2002 Annual Performance and Accountability Report.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 215 GAO-04-38 Results-Oriented Government

In its 1999 performance report, SSA noted that a number of its goals were 

not met, such as those relating to accurate and timely disability 

determinations. Also, data on the accuracy of decisions at the initial level 

were not available, and accuracy at the appellate level was not measured. 

In its 2002 report, 10 percent of the goals were not met and 10 percent were 

almost met. SSA’s report provided explanations for 17 performance goals 

SSA did not meet. However, not all of the explanations actually identified 

the reasons for SSA’s not meeting its goals. For example, SSA did not meet 

its goals for the performance indicators “percent of 800-number calls 

handled accurately—payment” and “percent of 800-number calls handled 

accurately—service.” The explanation noted that several quality initiatives 

were implemented, but SSA did not provide explanations as to why the 

goals were not met. SSA also noted that some of its performance 

indicators were being eliminated in favor of more focused and outcomebased goals. In some cases, SSA identified future plans to improve 

performance.

In SSA’s 2002 performance and accountability report, trend information 

was generally available for comparison of data back to fiscal year 1999. 

This information was helpful in making an assessment whether SSA was 

making progress towards its goals. SSA noted that it addressed all the IG’s 

list of major management challenges in its report, and that it addressed the 

major management challenges we identified in its annual performance 

plan. SSA also addresses the progress it made against certain challenges 

GAO and the IG identified during fiscal year 2002 in its performance and 

accountability report.24 For example, SSA highlights components of its SSI 

Corrective Action Plan that are geared to improve the administration of the 

SSI program and get it removed from our high-risk list. The IG’s report 

noted that SSA needs to have performance goals and measures that address 

the major management challenges facing SSA, as they are not all addressed. 

For example, performance measures were not established to address 

problems with the Earnings Suspense File and the integrity of the 

representative payee process.

24In our October 2002 report GAO-03-225, we noted that SSA reported progress on all six of its 

major management challenges in its fiscal year 2001 annual performance report.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 216 GAO-04-38 Results-Oriented Government

Finally, SSA’s performance and accountability report contained a 

discussion of the program evaluations conducted, organized by strategic 

goal. However, the program evaluations SSA identified were typically 

surveys of people who did business with SSA or assessments of internal 

needs, such as a survey of training effectiveness and water/air quality 

surveys. While this is a slight improvement over its 1999 report, where 

there was only a brief summary of program evaluations, it would be helpful 

for SSA to report on whether and how its evaluations have helped answer 

questions about program performance and results. We have previously 

reported that evaluations can help agencies improve their measurement of 

program performance and/or understanding of performance and how it 

might be improved.25

SSA’s 2002 Report Showed a 

Limited Relationship 

between Resources and 

Results

In the fiscal year 2002 performance and accountability report, SSA’s 

performance goals were not aligned by budget account—rather, they were 

associated with strategic goals, which in turn cross budget accounts and 

programs. Thus, the monetary, human capital, and technological resources 

necessary to achieve many performance goals were not adequately 

described. The financial statements show a schedule of financing and a 

schedule of budgetary resources for each of SSA’s major programs, and 

operating expenses were associated with four out of the five strategic 

goals.26 However, these resources were not broken down by performance 

goal, and were not linked to outcomes. Additionally, as reported by the IG, 

SSA needs to further develop its cost accounting system, which it began to 

use in fiscal year 2002; such a system would help to link costs with 

performance.27

SSA’s 2002 Report Provided 

General Confidence That 

Performance Data Are 

Credible

While SSA provides data sources and definitions for the data supporting its 

performance indicators, some data issues continue to detract from SSA’s 

performance report. SSA’s transmittal letter noted that the performance 

and financial data presented are fundamentally complete and reliable, and 

that no material inadequacies were identified. Data sources are identified 

25We characterized program evaluations and their uses in GAO/GGD-00-204.

26SSA noted that its fifth strategic goal, “Valued Employees,” supports the accomplishment 

of all its basic functions, so its resources are inherently included in the other four goals.

27SSA began to implement an improved cost accounting system in fiscal year 2002, which 

will be phased in over the next 3 to 4 years.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 217 GAO-04-38 Results-Oriented Government

for many of the performance indicators, such as findings from evaluations 

and quality assurance reports. In certain cases, data limitations are 

identified; for example, SSA noted that data to support the “Percent of 

SSNs issued accurately” goal does not include SSNs (social security 

numbers) assigned via the Enumeration-at-Birth process and major errors 

identified by the Office of Quality Assurance that do not include these SSNs 

result in SSN cards being issued erroneously. Some data verification 

procedures are noted in the report, but verification procedures are not 

consistently discussed and data reliability and completeness is not 

ensured. The IG noted that of the 21 measures it reviewed, 16 were 

reliable; data or documentation of the methods used to measure SSA’s 

performance were not available for the other five measures. The IG went 

further to say that even for the performance measures found to be reliable, 

SSA lacks documentation of the methods and data used to measure its 

performance. Finally, data were not available for 17 percent of the 

performance goals, so it was difficult to assess whether or not progress had 

been made in those areas.

Observations on the 

Quality of DOT’s Fiscal 

Year 2002 Annual 

Performance and 

Accountability Report

DOT’s fiscal year 2002 performance report provided information that 

generally showed the department’s performance and progress toward its 

goals. Summary tables within the report showed when DOT met or did not 

meet its targets and the report supplied brief analyses as to whether or not 

DOT would likely meet its targets for fiscal year 2003 based on actual 

performance in 2002. A separate section of the report on performance data 

completeness and reliability, along with an on-line compendium, provides a 

full level of confidence in DOT’s performance data. However, the report 

does not clearly show the relationship between resources and results.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 218 GAO-04-38 Results-Oriented Government

DOT’s 2002 Report Provided 

a General Picture of 

Performance

DOT’s fiscal year 2002 performance report28 provides a general picture of 

the department’s performance. The report includes performance summary 

tables, which show the progress made toward each strategic objective. 

These performance summaries include actual performance data from fiscal 

years 1996 to 2002 when possible, as well as the performance targets for 

fiscal year 2002 and whether or not the target was met. Similarly, we noted 

in our 2000 report29 reviewing DOT’s 1999 performance report that 

performance information was clearly articulated, with summary tables 

listing the fiscal year 1999 goals and trend data, and checkmarks to indicate 

whether or not goals were met. Figure 25 summarizes DOT’s overall 

performance on its 40 performance targets, as reported in its 2002 report.

Figure 25: Summary of DOT’s Performance Indicators for Fiscal Year 2002

According to the report, DOT met 24 (60 percent) of its performance 

targets. Fourteen (35 percent) of DOT’s performance targets were not met. 

28U.S. Department of Transportation, Fiscal Year 2002 Performance and Accountability 

Report (Washington, D.C.: 2003).

29Our review of DOT’s fiscal year 1999 report, GAO/RCED-00-201R, focused on our 

observations regarding only four of the department’s key outcomes, as well as the major 

management challenges addressed in the performance report. Since our review of DOT’s 

2002 performance report used different assessment criteria, we could not make valid 

comparisons on all aspects of the reports.

60%

5%

35% Not met

Data not available

Met

Source: GAO analysis of DOT’s Fiscal Year 2002 Annual Performance and Accountability Report.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 219 GAO-04-38 Results-Oriented Government

The report provides explanations for why five of these targets were unmet. 

For example, the target for the measure “number of passengers (in 

millions) in international markets with open skies aviation agreements” of 

the “mobility and economic growth” strategic objective was unmet. The 

target was set at 59.7 million passengers, while DOT’s preliminary estimate 

for this measure indicated there were 57 million passengers. The report 

states that this target was unmet because passenger travel diminished in 

fiscal year 2002 due to the impact that the events of September 11, 2001, 

had on air travel. However, the report did not provide explanations 

describing why the nine other targets were not met. A DOT official stated 

that explanations for these unmet targets were not included in the 2002 

report due, in part, to time constraints. In our 2000 report of DOT’s 1999 

performance report, we stated that for all of its unmet goals except transit 

fatalities, the department provided explanations for not meeting the goals 

related to the outcomes we observed.

We noted in our report on DOT’s 1999 performance report that the 

department supplied strategies to achieve its unmet goals in the future, for 

the areas we reviewed. However, of the 14 unmet targets in the fiscal year 

2002 report, DOT provided future plans to achieve only two. For example, 

the report provided a plan for future achievement of the unmet target 

“percent of environmental justice cases unresolved after one year.” The 

report stated that DOT’s External Complaint Tracking System was being 

revised “to track complaints more closely, in a more timely way, and with a 

higher level of data quality.” DOT is also developing guidance requiring 

more intensive legal staff involvement in external civil rights complaints, 

especially environmental justice cases. A DOT official stated that future 

plans were not included in the 2002 report for the other unmet targets due, 

in part, to time constraints.

Data were unavailable for two of DOT’s measures, “cumulative average 

percent change in transit passenger-miles traveled per transit market” and 

“employment sites (in thousands) made accessible by Job Access and 

Reverse Commute (JARC) transportation services.” The report explains 

the reasons why data were unavailable for both of these measures and 

includes plans to provide these data in the future. Data were unavailable 

for the JARC program performance measure because DOT had not 

received data from JARC grantees to verify that fiscal year 2002 program 

targets had been achieved. DOT states that a new reporting system is being 

implemented, which should improve data gathering performance.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 220 GAO-04-38 Results-Oriented Government

Although the report does not identify any performance goals that were 

impractical or infeasible, it states that the measure on transit passengermiles traveled had been changed in fiscal year 2002 because “it placed 

excessive emphasis on increasing ridership in the Nation’s very largest 

urban areas.” However, after using the measure for one year, DOT 

concluded that the measure should once again be modified to account for 

changes in the level of employment in each urban area. The report states 

that a recent study found that changes in the level of employment are a key 

economic factor related to changes in the level of transit ridership.

Another strength of DOT’s fiscal year 2002 performance report is an 

analysis of whether or not DOT will likely meet its planned performance 

targets for fiscal year 2003. Each discussion of performance goals contains 

an evaluation of the fiscal year 2003 performance plan target and whether 

or not it will likely be met based on the fiscal year 2002 data. For example, 

for its highway fatality rate targets, DOT says that the National Highway 

Traffic Safety Administration and the Federal Motor Carrier Safety 

Administration will be challenged to meet the established fiscal year 2003 

targets because targets had not been met for fiscal year 2002. In other 

instances where DOT is sure that it will meet its targets for fiscal year 2003, 

it simply states so, as in the case of its aviation safety targets.

DOT’s 2002 performance report also includes information on completed 

program evaluations. There is a separate section in the report that 

discusses DOT’s fiscal year 2002 program evaluations. Summaries of the 

findings of these evaluations are discussed in this section. For example, an 

evaluation of the Noise Set-Aside Portion of the FAA (Federal Aviation 

Administration) Airport Improvement Program found that funding for the 

program’s noise compatibility projects was variable from year to year, 

making it difficult to forecast annual population benefits.

As we found in the 1999 report, the major management challenges that 

DOT faces are generally discussed in the 2002 report. These discussions 

were contained within the program section to which they relate. The 

report also showed the progress DOT has made in addressing its 

management challenges. For example, we noted in our January 2003 

report30 on DOT’s major management challenges that FAA’s financial 

management systems remained at high risk. DOT’s 2002 report states that 

FAA created an Interim Fixed Asset System to centrally control and 

30GAO-03-108.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 221 GAO-04-38 Results-Oriented Government

account for its property and that in fiscal year 2003, FAA will convert to use 

Delphi, DOT’s financial accounting system.

The 2002 performance report included a discussion on strategic human 

capital management as part of DOT’s “organizational excellence” strategic 

objective. This discussion included a brief overview of DOT’s human 

capital plan as well as strategies for strategic human capital management. 

For example, the report noted that FAA was redirecting 37,300 employees 

into a results-oriented Air Traffic Organization, “freeing most of the FAA to 

manage better and modernize more efficiently.” However, the report did 

not include a review of the performance goals and evaluation of the 

performance plan relative to the agency’s strategic human capital 

management, as required by the Homeland Security Act of 2002.

DOT’s 2002 Report Showed 

No Relationship between 

Resources and Results

DOT’s 2002 performance report did not show the relationship between the 

resources it used and the results it achieved in fiscal year 2002. The 

financial portion of the report provided a statement of net cost for each of 

DOT’s programs in fiscal year 2002. The report could be improved by 

providing net cost information for DOT’s performance goals in the 

performance section of the report, similar to the funding information 

provided in the 2004 performance plan.

DOT’s 2002 Report Provided 

Full Confidence That 

Performance Data Are 

Credible

DOT’s 2002 performance report provided a full level of confidence that the 

department’s performance data were credible. In his transmittal letter, the 

Secretary of DOT stated that the 2002 report “contains performance and 

financial data that are substantially complete and reliable.” The letter also 

stated that a section of the report assessed the inadequacies of DOT’s 

performance data and provided plans to remedy those inadequacies.

The “Performance Data Completeness and Reliability” section of the 2002 

report generally discussed data completeness, reliability, and limitations. 

This section discussed an overall limitation in DOT’s performance data. 

The report stated that much of DOT’s performance data came from 

external sources, and therefore, the department had no direct control over 

the quality of these data. The report continues by stating that DOT takes 

limitations to its external data into account when it uses these data.
Appendix V

Observations on Agencies’ Annual 

Performance and Accountability Reports

Page 222 GAO-04-38 Results-Oriented Government

The 2002 report noted that DOT has compiled source and accuracy 

statements that provide detail on the methods used to collect performance 

data, sources of variation and bias in the data, methods used to verify and 

validate the data, as well as data limitations.31 However, the online Source 

and Accuracy Compendium does not include this information for the goals 

and measures related to the department’s organizational excellence 

objective. The compendium states that a small number of source and 

accuracy statements are not yet completed and that they will be added 

upon completion.

Finally, the 2002 report also described strategies being undertaken to 

address the quality of data used by DOT. The report stated that a DOT 

intermodal working group addressed data quality issues by developing 

departmental statistical standards and by updating source and accuracy 

statements for all of DOT’s data programs. The working group also worked 

to improve quality assurance procedures, evaluate sampling and 

nonsampling errors, and develop common definitions for data across 

modes.

31Bureau of Transportation Statistics, Source & Accuracy Compendium, 

http://www.bts.gov/statpol/SAcompendium.html (Washington, D.C.: Aug. 15, 2003).
Page 223 GAO-04-38 Results-Oriented Government

Appendix VI

GAO Federal Managers’ Survey Data ApendxiVI

Q1. What is your current grade level?

Q1a. If you answered "Other" in question 1 above, please enter your response below.

Q2. In total, for how many years have you been a supervisor and/or a manager in the federal government?

Q2no. Or, If you have never been a supervisor or a manager in the federal government, please check the box below.

Q3. In your current role, approximately how many government employees are you responsible for? (Please answer for your permanent 

position. Please specify the total number. If none, enter 0.) Enter numeric digits only. Employees:

Q4. Please indicate where you currently work. (If you are currently on temporary assignment or on detail, please answer for your 

permanent work location.)

GS/GM-13 or 

equivalent

(percent)

GS/GM-14 or 

equivalent 

(percent)

GS/GM-15 or 

equivalent 

(percent)

Senior 

Executive 

Service or 

equivalent 

(percent)

Other - please 

specify - 

Continue with 

next question 

(percent)

Number of 

respondents

31.4 35.2 24.6 7.9 0.9 500

Writing 

comment

(percent)

Number of 

respondents

0.9 3

Mean Median Minimum Maximum

Number of 

respondents

13.1 13 1 43 497

Percent

Number of 

respondents

0.6 3

Mean Median Minimum Maximum

Number of 

respondents

77.7 15 1 11,000 475

Headquarters of my 

department or agency

(percent)

A field office of 

my department 

or agency 

(percent)

Other - please 

specify - Continue 

with next question 

(percent)

No answer 

(percent)

Number of 

respondents

28.9 60.7 9.5 1.0 503
Appendix VI

GAO Federal Managers’ Survey Data

Page 224 GAO-04-38 Results-Oriented Government

Q4a. If you answered "Other" in question 4 above, please enter your response below.

Q5. For those program(s)/operation(s)/project(s) that you are involved with, to what extent, if at all, do you consider your agency’s strategic 

goals when participating in the following activities? (Check one box in each row.)

Q6. Are there performance measures for the program(s)/operation(s)/ project(s) that you are involved with?

Q7. To what extent, if at all, do you agree with the following statements as they relate to performance measures for the 

program(s)/operation(s)/ project(s) that you are involved with? (Check one box in each row.) We have performance measures that...

Writing 

comment

(percent)

Number of 

respondents

9.5 38

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents

a. Setting program 

priorities 33.9 45.3 11.4 4.7 1.8 2.2 0.7 503

b. Allocating resources 30.3 39.9 18.9 4.8 1.8 3.1 1.3 503

c. Adopting new program 

approaches or changing 

work processes 33.6 39.2 16.5 4.6 2.4 1.9 1.8 503

d. Developing or refining 

program performance 

measures 29.0 37.0 16.3 8.3 4.1 4.4 0.9 503

Yes

(percent)

No 

(percent)

Do not know 

(percent)

No answer 

(percent)

Number of 

respondents

89.0 6.2 3.8 1.0 503

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents

a....tell us how many things 

we produce or services we 

provide. (Output 

measures) 25.8 28.7 19.4 8.4 4.8 1.8 11.0 503

b....tell us if we are 

operating efficiently. 

(Efficiency measures) 15.2 27.5 25.8 13.4 5.5 1.5 11.0 503

c....tell us whether or not 

we are satisfying our 

customers. (Customer 

service measures) 16.8 29.8 21.6 12.6 6.1 1.6 11.4 503
Appendix VI

GAO Federal Managers’ Survey Data

Page 225 GAO-04-38 Results-Oriented Government

Q8. For those program(s)/operation(s)/project(s) that you are involved with, to what extent, if at all, do you use the information obtained 

from performance measurement when participating in the following activities?

d....tell us about the quality 

of the products or services 

we provide. (Quality 

measures) 16.0 30.3 23.5 12.6 5.6 0.7 11.3 503

e....demonstrate to 

someone outside of our 

agency whether or not we 

are achieving our intended 

results. (Outcome 

measures) 19.1 35.9 19.0 10.5 2.6 1.5 11.3 503

f....link our product or 

service costs with the 

results we achieve. (Costbenefit measures) 12.5 18.7 19.7 20.0 11.7 6.0 11.3 503

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents

a. Setting program 

priorities 13.0 38.7 21.7 10.8 3.4 1.2 11.0 503

b. Allocating resources 14.9 36.6 20.3 10.8 3.7 1.8 12.0 503

c. Adopting new program 

approaches or changing 

work processes 13.9 34.8 24.2 10.2 4.1 0.9 11.9 503

d. Coordinating program 

efforts with other internal 

or external organizations 10.1 32.2 29.4 10.9 3.6 2.7 11.1 503

e. Refining program 

performance measures 14.4 28.8 24.7 13.2 3.4 4.1 11.3 503

f. Setting new or revising 

existing performance goals 16.1 32.4 21.0 11.8 2.9 3.5 12.3 503

g. Setting individual job 

expectations for the 

government employees I 

manage or supervise 18.4 33.6 22.0 9.3 2.8 2.4 11.4 503

h. Rewarding government 

employees I manage or 

supervise 17.7 33.3 21.8 8.2 4.0 3.0 11.9 503

i. Developing and 

managing contracts 7.3 18.8 19.3 10.4 8.1 24.4 11.7 503

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents
Appendix VI

GAO Federal Managers’ Survey Data

Page 226 GAO-04-38 Results-Oriented Government

Q9. Based on your experience with the program(s)/operation(s)/project(s) that you are involved with, to what extent, if at all, have the 

following factors hindered measuring performance or using the performance information?

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents

a. Setting program 

priorities 14.2 20.3 32.1 19.7 9.1 3.9 0.7 503

b. Different parties are 

using different definitions 

to measure performance 10.3 23.5 24.6 24.6 10.4 5.6 1.0 503

c. Difficulty obtaining 

valid or reliable data 7.5 22.1 26.0 27.2 12.0 4.1 1.0 503

d. Difficulty obtaining 

data in time to be useful 7.6 17.5 23.9 26.0 18.3 4.4 2.2 503

e. Lack of incentives 

(e.g., rewards, positive 

recognition) 12.9 15.8 25.0 24.1 16.6 4.8 0.8 503

f. Difficulty resolving 

conflicting interests of 

stakeholders, either 

internal or external 9.5 19.3 25.5 22.0 14.5 8.1 1.0 503

g. Difficulty distinguishing 

between the results 

produced by the program 

and results caused by 

other factors 7.4 18.7 24.3 28.6 13.7 6.3 1.0 503

h. Existing information 

technology and/or 

systems not capable of 

providing needed data 9.7 19.4 20.6 26.6 17.0 5.4 1.3 503

i. Lack of staff who are 

knowledgeable about 

gathering and/or 

analyzing performance 

information 10.1 17.5 23.4 26.2 16.5 5.0 1.3 503

j. Lack of ongoing top 

executive commitment or 

support for using 

performance information 

to make program/funding 

decisions 9.7 16.0 18.4 24.0 23.1 7.9 1.0 503
Appendix VI

GAO Federal Managers’ Survey Data

Page 227 GAO-04-38 Results-Oriented Government

Q10. To what extent, if at all, do you agree with the following statements? (Check one box in each row.)

k. Lack of ongoing 

Congressional 

commitment or support 

for using performance 

information to make 

program/funding 

decisions 7.1 16.6 18.1 16.5 17.1 23.8 0.7 503

l. Difficulty determining 

how to use performance 

information to improve 

the program 5.3 12.8 30.1 26.6 18.8 5.6 0.7 503

m. Concern that OMB will 

micromanage programs 

in my agency 7.9 10.8 14.7 19.2 26.6 19.7 1.0 503

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents

a. Agency managers/ 

supervisors at my level 

have the decision making 

authority they need to 

help the agency 

accomplish its strategic 

goals. 9.5 30.1 28.1 25.2 5.6 0.6 1.0 503

b. Agency managers/ 

supervisors at my level 

are held accountable for 

agency accomplishment 

of its strategic goals. 14.6 42.9 24.1 12.7 3.3 1.5 1.0 503

c. Agency managers/ 

supervisors at my level 

are held accountable for 

the results of the 

program(s)/ 

operation(s)/project(s) 

they are responsible for. 24.2 46.5 17.2 7.2 3.0 0.9 1.0 503

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents
Appendix VI

GAO Federal Managers’ Survey Data

Page 228 GAO-04-38 Results-Oriented Government

Q11. To what extent, if at all, do you agree with the following statements? (Check one box in each row.) The following items focus on the 

program(s)/operation(s)/project(s) that you are responsible for.

d. Employees in my 

agency receive positive 

recognition for helping the 

agency accomplish its 

strategic goals. 9.7 27.1 31.2 22.0 7.2 1.5 1.3 503

e. My agency’s top 

leadership demonstrates 

a strong commitment to 

achieving results. 24.4 37.1 21.8 9.9 2.9 2.6 1.3 503

f. My agency is investing 

the resources needed to 

ensure that its 

performance data is of 

sufficient quality. 9.5 21.4 29.3 21.9 7.4 9.3 1.0 503

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents

a. The individual I report to 

periodically reviews with 

me the results or outcomes 

of the program(s) 

operation(s)/project(s) that 

I am responsible for. 18.8 36.1 23.1 15.3 5.3 0.4 1.0 503

b. Funding decisions for 

the program(s)/ 

operation(s)/project(s) I am 

responsible for are based 

on results or outcomeoriented performance 

information. 4.5 20.9 23.1 26.4 16.5 7.5 1.0 503

c. Staffing and personnel 

decisions for the 

program(s)/ 

operation(s)/project(s) I am 

responsible for are based 

on results or outcomeoriented performance 

information. 4.2 20.9 27.8 23.2 19.0 3.7 1.3 503

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents
Appendix VI

GAO Federal Managers’ Survey Data

Page 229 GAO-04-38 Results-Oriented Government

Q12. During the past 3 years, has your agency provided, arranged, or paid for training that would help you to accomplish the following 

tasks? (Check one box in each row.)

d. Changes by 

management above my 

level to the program(s)/ 

operation(s)/project(s) I am 

responsible for are based 

on results or outcomeoriented performance 

information. 2.6 20.5 25.2 26.5 15.3 8.2 1.6 503

e. It is easy to motivate 

employees to be more 

results-oriented in the 

program(s)/operation(s)/ 

project(s) I am responsible 

for. 3.5 22.0 33.6 29.0 7.8 2.8 1.3 503

f. I have sufficient 

information on the validity 

of the performance data I 

use to make decisions. 4.7 32.6 30.2 20.3 7.1 3.2 1.8 503

Yes 

(percent)

No 

(percent)

No answer 

(percent)

Number of 

respondents

a. Conduct strategic planning 46.7 52.3 1.0 503

b. Set program performance goals 48.8 50.2 1.0 503

c. Develop program performance 

measures 42.9 55.5 1.6 503

d. Assess the quality of performance 

data 35.3 63.5 1.3 503

e. Use program performance 

information to make decisions 40.7 56.8 2.4 503

f. Link the performance of 

program(s)/operation(s)/project(s) to 

the achievement of agency strategic 

goals 40.8 57.0 2.2 503

g. Implement the requirements of the 

Government Performance and Results 

Act (GPRA or the Results Act) 31.9 66.7 1.3 503

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents
Appendix VI

GAO Federal Managers’ Survey Data

Page 230 GAO-04-38 Results-Oriented Government

Q13. In your opinion, to what extent, if at all, do you believe you need training (or additional training) in order to help you to accomplish the 

following tasks? (Check one box in each row.)

Q14. What, in your opinion, can the Federal government do to improve its overall focus on managing for results?

Q15. Prior to receiving this questionnaire , which of the following statements best describes your awareness of GPRA?

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents

a. Conduct strategic 

planning 8.2 18.3 27.8 30.8 11.2 2.8 0.9 503

b. Set program 

performance goals 8.6 19.7 27.6 27.2 12.7 3.0 1.2 503

c. Develop program 

performance measures 9.6 21.1 29.2 23.7 11.2 3.6 1.6 503

d. Assess the quality of 

performance data 8.9 22.1 25.7 25.7 13.3 2.5 1.9 503

e. Use program 

performance information 

to make decisions 9.5 22.1 23.7 26.9 13.8 2.2 1.9 503

f. Link the performance of 

program(s)/operation(s)/p

roject(s) to the 

achievement of agency 

strategic goals 12.0 21.6 24.2 27.3 11.2 2.8 0.9 503

g. Implement the 

requirements of the 

Government Performance 

and Results Act (GPRA or 

the Results Act) 12.0 25.0 26.4 17.6 11.1 6.9 1.0 503

Writing 

comment

(percent)

Number of 

respondents

65.4 503

I had never 

heard of GPRA.

(percent)

I had heard of 

GPRA but had no 

knowledge of its 

requirements. 

(percent)

I had heard of 

GPRA and had 

a low level of 

knowledge of its 

requirements. 

(percent)

I had heard of 

GPRA and had 

moderate 

knowledge of its 

requirements. 

(percent)

I had heard of 

GPRA and 

had extensive 

knowledge of 

its requirements. 

(percent)

No answer 

(percent)

Number of 

respondents

19.5 13.5 24.5 35.4 5.6 1.5 503
Appendix VI

GAO Federal Managers’ Survey Data

Page 231 GAO-04-38 Results-Oriented Government

Q16. For those program(s)/operation(s)/project(s) that you are involved with, to what extent,if at all, do you consider the annual 

performance goals set forth in your agency’s GPRAannual performance plan when participating in the following activities?

Q17. During the past 3 years, have you been involved in these activities? (Check one box in each row.) GPRA-related activities

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents

a. Setting program 

priorities 7.5 19.8 24.4 12.7 13.5 20.8 1.2 503

b. Allocating resources 5.2 16.8 23.9 15.7 14.7 22.5 1.2 503

c. Adopting new program 

approaches or changing 

work processes 5.9 21.3 23.8 13.5 13.3 21.1 1.2 503

d. Coordinating program 

efforts with other internal 

or external organizations 5.4 16.2 26.4 15.7 12.8 22.3 1.2 503

e. Developing or refining 

program performance 

measures 5.7 18.1 21.5 15.9 14.7 22.9 1.2 503

f. Setting individual job 

expectations for the 

government employees I 

manage or supervise 5.7 22.0 19.7 15.4 15.0 20.5 1.6 503

g. Rewarding 

government employees I 

manage or supervise 5.9 19.5 20.4 16.1 16.2 20.5 1.5 503

h. Developing and 

managing contracts 3.3 12.3 14.9 13.3 16.4 37.1 2.7 503

Yes 

(percent)

No 

(percent)

No answer 

(percent)

Number of 

respondents

a. Developing ways to measure 

whether program performance goals 

are being achieved. 45.8 52.7 1.5 503

b. Gathering and analyzing data to 

measure whether programs are 

meeting their specific performance 

goals. 50.6 47.9 1.5 503

c. Using measures for program 

performance goals to determine if the 

agency’s strategic goals are being 

achieved. 42.6 55.9 1.5 503

d. Assessing the quality of data used 

in measuring performance. 39.7 58.2 2.1 503
Appendix VI

GAO Federal Managers’ Survey Data

Page 232 GAO-04-38 Results-Oriented Government

Q18. To what extent, if at all, do you agree with the following statements? (Check one box in each row.) Extent I agree with the following 

statements:

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents

a. The objectives of my 

program(s)/operation(s)/pr

oject(s) are in alignment 

with my agency’s strategic 

plan under GPRA. 10.1 29.4 15.8 7.2 1.2 35.0 1.2 503

b. The costs associated 

with implementing GPRA 

have taken time or funds 

away from other important 

activities or projects. 4.8 7.7 13.1 15.1 10.6 46.1 2.5 503

c. The benefits to my 

agency that are achieved 

by implementing GPRA 

are worth the costs 

incurred in doing so (e.g., 

in time, money, and effort). 3.8 11.3 15.8 12.4 7.4 48.0 1.3 503

d. GPRA strategic and 

annual performance plans 

are mostly a repackaging 

of goals, measures, and 

objectives that were 

already being used within 

my agency. 5.7 22.7 19.1 7.7 2.8 40.1 1.9 503

e. Managerial 

effectiveness is impeded 

by the lack of integration 

between GPRA and other 

federal management 

programs. 3.8 7.7 16.3 13.4 6.8 50.5 1.5 503

f. My agency considers 

contributions to and 

comments on GPRA plans 

or reports from 

managers/supervisors at 

my level: 3.5 10.6 15.6 15.0 11.7 42.1 1.5 503
Appendix VI

GAO Federal Managers’ Survey Data

Page 233 GAO-04-38 Results-Oriented Government

Q19. To what extent, if at all, do you believe that GPRA has improved your agency’s ability to deliver results to the American public.

Q19a. Please briefly describe how GPRA has improved your agency’s ability to deliver results to the American public.

g. Agency managers/ 

supervisors at my level 

use GPRA annual 

performance plans to 

manage their 

program(s)/operation(s)/pr

oject(s). 3.6 7.8 18.5 16.4 17.7 33.9 2.1 503

h. GPRA’s planning and 

reporting requirements 

impose a significant 

paperwork burden. 2.9 7.4 17.1 15.9 7.3 47.7 1.8 503

i. GPRA has caused 

agency managers/ 

supervisors at my level to 

place a greater emphasis 

on getting input from 

appropriate stakeholders 

on their interests and 

expectations. 2.5 7.0 20.8 13.1 12.2 42.2 2.1 503

To a very great 

extent 

(Continue with 

question 19a.)

(percent)

To a great 

extent 

(Continue with 

question 19a.) 

(percent)

To a moderate 

extent (Continue 

with question 

19a.) 

(percent)

To a small 

extent 

(percent)

To no extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents

1.6 6.9 14.5 25.2 11.6 38.0 2.1 503

Writing 

comment

(percent)

Number of 

respondents

23.0 139

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents
Appendix VI

GAO Federal Managers’ Survey Data

Page 234 GAO-04-38 Results-Oriented Government

Q20. To what extent, if at all, do you believe the following persons or entities pay attention to your agency’s efforts under GPRA? (Check 

one box in each row.)

Q21. To what extent, if at all, do you believe that efforts to implement GPRA to date have improved the program(s)/operation(s)/project(s) 

in which you are involved?

Q22. To what extent, if at all, do you believe that efforts to implement GPRA to date have improved your agency’s 

programs/operations/projects?

To a very 

great 

extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no 

extent 

(percent)

No basis to 

judge/Not 

applicable 

(percent)

No answer 

(percent)

Number of 

respondents

a. Department Secretary 

(if applicable) 10.3 18.3 10.6 6.2 3.6 47.1 3.9 503

b. Agency head other 

than Department 

Secretary (if applicable) 11.4 20.6 12.9 6.9 2.1 42.9 3.2 503

c. The individual I report 

to 8.5 16.0 19.3 14.0 13.2 26.6 2.4 503

d. Managers and 

supervisors at my level 5.1 13.2 19.7 19.5 16.0 24.1 2.4 503

e. Employees who report 

to me 2.5 8.3 13.8 18.1 30.7 24.0 2.7 503

f. Office of Management 

and Budget 14.1 16.7 11.4 4.0 1.8 49.0 2.9 503

g. Congressional 

committees 7.8 14.5 13.0 8.2 3.5 50.3 2.6 503

h. The audit community 

(e.g., GAO, Inspectors 

General) 11.7 16.9 11.5 7.3 1.2 48.2 3.2 503

i. The general public 1.5 2.5 6.1 17.5 26.4 42.2 3.9 503

I have not been 

sufficiently involved in 

GPRA to have an 

opinion.

(percent)

To a very 

great extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no extent 

(percent)

No answer 

(percent)

Number of 

respondents

48.7 2.9 5.2 14.7 16.2 10.0 2.4 503

I have not been 

sufficiently involved in 

GPRA to have an 

opinion.

(percent)

To a very 

great extent 

(percent)

To a great 

extent 

(percent)

To a 

moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no extent 

(percent)

No answer 

(percent)

Number of 

respondents

47.9 2.3 6.2 17.0 16.3 8.7 1.5 503
Appendix VI

GAO Federal Managers’ Survey Data

Page 235 GAO-04-38 Results-Oriented Government

Q23. To what extent, if at all, do you believe implementing GPRA can improve your agency’s programs/operations/projects in the future ?

Q24. If you have been involved to any extent in implementing GPRA for the program(s)/operation(s)/project(s) you are involved with, what 

has been your greatest difficulty, and in what ways, if any, do you think this difficulty could be addressed?

Q25. If you have additional comments regarding any previous question or any comments/suggestions concerning GPRA, please use the 

space provided below.

Note: Percents reported are weighted percents based on the population size. Unweighted N reported 

for each item.

To a very 

great extent

(percent)

To a great 

extent 

(percent)

To a moderate 

extent 

(percent)

To a small 

extent 

(percent)

To no extent 

(percent)

No basis 

to judge 

(percent)

No answer 

(percent)

Number of 

respondents

3.4 11.9 23.7 16.7 5.6 35.7 3.0 503

Writing 

comment

(percent)

Number of 

respondents

26.0 503

Writing 

comment

(percent)

Number of 

respondents

16.5 503
Page 236 GAO-04-38 Results-Oriented Government

Appendix VII

Agencies Subject to the Chief Financial 

Officers Act ApendxiVI

The Chief Financial Officers Act of 1990 (the CFO Act) created the position 

of Chief Financial Officer in each executive department and in each major 

executive agency in the federal government. The agencies covered by the 

CFO Act are:

1. Agency for International Development 

2. Department of Agriculture

3. Department of Commerce

4. Department of Defense

5. Department of Education

6. Department of Energy

7. Department of Health and Human Services

8. Department of Housing and Urban Development 

9. Department of the Interior

10. Department of Justice

11. Department of Labor

12. Department of State

13. Department of Transportation 

14. Department of the Treasury

15. Department of Veterans Affairs 

16. Environmental Protection Agency

17. Federal Emergency Management Agency1

1

The Federal Emergency Management Agency became part of the Department of Homeland 

Security in March 2003.
Appendix VII

Agencies Subject to the Chief Financial 

Officers Act

Page 237 GAO-04-38 Results-Oriented Government

18. General Services Administration

19. National Aeronautics and Space Administration

20. National Science Foundation

21. Nuclear Regulatory Commission

22. Office of Personnel Management

23. Small Business Administration

24. Social Security Administration2

2

Formerly part of the Department of Health and Human Services, the Social Security 

Administration once again became an independent agency on March 31, 1995. Congress 

established the position of Chief Financial Officer within SSA in the Social Security 

Independence and Program Improvements Act of 1994.
Page 238 GAO-04-38 Results-Oriented Government

Appendix VIII

Comments from the Office of Management 

and Budget ApendxiVI
Appendix VIII

Comments from the Office of Management 

and Budget

Page 239 GAO-04-38 Results-Oriented Government
Page 240 GAO-04-38 Results-Oriented Government

Appendix IX

Comments from the Department of Energy ApendxiIX

Note: GAO comments 

supplementing those in 

the report text appear 

at the end of this 

appendix.

Now on pp. 53-54.

See comment 1.

See comment 2.

See comment 3.

See comment 1.

Now on p. 56.
Appendix IX

Comments from the Department of Energy

Page 241 GAO-04-38 Results-Oriented Government

See comment 4.

See comment 5.
Appendix IX

Comments from the Department of Energy

Page 242 GAO-04-38 Results-Oriented Government

The following are our comments on DOE’s letter dated January 15, 2004.

GAO Comments The Director of the Office of Program Assessment and Evaluation 

forwarded written comments from DOE on a draft of this report. DOE 

disagreed with several of our conclusions concerning its 2004 Annual 

Performance Plan and 2002 Performance and Accountability Report. We 

incorporated the additional information and perspectives of DOE into our 

report as appropriate.

1. We stated that, when compared to the 1999 Annual Performance Plan, 

DOE’s 2004 Annual Performance Plan continued to provide a limited 

picture of intended performance. DOE disagreed, stating that the plan 

provided a clear picture of intended performance because a description 

of each program was contained in each of the general goals. We agree 

that a description of programs is provided for each goal, and we also 

stated in our draft report that improvement was made in developing 

results-oriented performance measures that pertained specifically to 

fiscal year 2004. However, in our view, describing each program does 

not sufficiently explain DOE’s expectations for intended performance. 

More specifically, we found the overall picture of intended performance 

was limited because DOE did not specifically describe how it 

coordinates with other agencies to accomplish crosscutting programs 

and did not provide a clear link between its annual goals and its mission 

and strategic goals. In our draft report we acknowledged that a link did 

not exist between the performance plan and the strategic plan because 

the strategic plan was revised after the performance plan was finalized. 

Nevertheless, DOE did not revise the final 2004 performance plan to 

reflect its changes in strategic goals. The lack of alignment between the 

performance plan and strategic plan goals limits the usefulness of the 

performance plan to support managers and staff in their day-to-day 

activities in achieving DOE’s long-term strategic goals.

2. In response to our observation that DOE provided a general discussion 

of the strategies and resources needed to achieve its performance 

goals, DOE stated that its 2004 annual performance plan provided 

specific strategies and resources that will be used to achieve 

performance goals. DOE also noted that funding was included at the 

general goal level. We agree that funding was included at the general 

strategic goal level. However, better plans relate resources to the 

achievement of performance goals. DOE did not provide resource 

information at the performance goal level. Furthermore, while DOE 
Appendix IX

Comments from the Department of Energy

Page 243 GAO-04-38 Results-Oriented Government

discussed external factors that could affect its ability to achieve its 

performance goals at a high level, it did not discuss any specific 

strategies to mitigate those factors. 

3. DOE disagreed with our characterization that its 2004 annual 

performance plan provided limited confidence that performance data 

will be credible. The department stated that the introduction section of 

its plan contained specific sections on assessing results and validating 

and verifying data, as well as discussed a software package used to 

document and track performance. In our draft report, we stated that 

DOE’s plan showed some improvement over its 1999 plan by describing 

credible procedures to verify and validate performance information 

and by mentioning specific program evaluations for each goal. We also 

noted that DOE acquired new commercial software for performance 

tracking through remote data entry, monitoring, and oversight by 

program offices and managers. However, we concluded that DOE’s 

reporting of credible performance data was limited because its plan 

does not specifically identify data limitations overall or for each of its 

goals. As we stated in our report, we found this to be of particular 

concern because, as we mentioned in our 2003 performance and 

accountability series, DOE has several management challenges where 

data quality is a concern.

4. Concerning its 2002 Annual Performance and Accountability Report, 

DOE stated that it provided a plan of action for addressing the causes 

of targets that were not met. We agree, and in our draft report we state 

that all targets that were not met or had mixed results contained a plan 

of action to achieve the target in the future. We also found that the 

majority of DOE targets that were not met or had mixed results 

contained clear explanations. We revised our text in the final version of 

this report to make these findings more evident.

5. Finally, DOE disagreed with our finding that it did not provide a 

discussion of the relationship between the strategic plan, performance 

plan, and the performance report in its 2002 Performance and 

Accountability Report. DOE stated that the introductory section of the 

report contains a paragraph that discusses the linkages between these 

three reports. However, although the performance and accountability 

report links these documents by organizing its results section 

according to strategic goals, associated program performance goals 

and targets, it did not succinctly demonstrate how the results relate to 
Appendix IX

Comments from the Department of Energy

Page 244 GAO-04-38 Results-Oriented Government

the annual and long-term strategic goals. We modified the draft 

accordingly to clarify this point.
Page 245 GAO-04-38 Results-Oriented Government

Appendix X

Comments from the Department of Housing 

and Urban Development ApendxiX

Note: GAO comments 

supplementing those in 

the report text appear 

at the end of this 

appendix.
Appendix X

Comments from the Department of Housing 

and Urban Development

Page 246 GAO-04-38 Results-Oriented Government

See comment 1.
Appendix X

Comments from the Department of Housing 

and Urban Development

Page 247 GAO-04-38 Results-Oriented Government

See comment 2.
Appendix X

Comments from the Department of Housing 

and Urban Development

Page 248 GAO-04-38 Results-Oriented Government

See comment 3.
Appendix X

Comments from the Department of Housing 

and Urban Development

Page 249 GAO-04-38 Results-Oriented Government

See comment 4.

See comment 5.
Appendix X

Comments from the Department of Housing 

and Urban Development

Page 250 GAO-04-38 Results-Oriented Government

See comment 6.
Appendix X

Comments from the Department of Housing 

and Urban Development

Page 251 GAO-04-38 Results-Oriented Government

The following are our comments on HUD’s letter dated January 9, 2004.

GAO Comments HUD provided written comments and disagreed with several of our 

observations, which we address below. HUD also mentioned that all of the 

areas we suggested for further improvement were already in process or 

being considered. Where appropriate, we incorporated HUD’s comments 

and perspectives to clarify our report.

1. HUD did not agree with our observation that the link between longterm and intermediate goals in its strategic plan is difficult to discern. 

The department mentioned that the need for a direct link between longterm and intermediate goals is not apparent, as they are aligned with 

strategic goals. GPRA requires that an agency’s strategic plan contain, 

among other things, a description of the relationship between the longterm goals and objectives and the annual performance goals. In 

addition, OMB’s June 2002 Circular A-11 states that the strategic plan 

should briefly outline how annual performance goals relate to the longterm, general goals, and how they help determine the achievement of 

the general goals. Federal agencies can help readers understand how 

they move from general goals to specific, measurable outcomes by 

discussing how they plan to measure progress in achieving the long 

term-goals in their strategic plan. For example, for its strategic goal of 

“Increase Homeownership Opportunities,” HUD mentions that one of 

its long-term performance measures is to combat predatory lending. 

Readers can review the intermediate measures listed under that goal to 

get a sense of how HUD plans to accomplish this objective. For 

example, HUD mentions that beginning in the third quarter of fiscal 

year 2003, field offices will report all activities related to predatory 

lending to headquarters each quarter. However, not all long-term 

measures listed in the strategic plan have a corresponding intermediate 

performance measure.
Appendix X

Comments from the Department of Housing 

and Urban Development

Page 252 GAO-04-38 Results-Oriented Government

2. HUD disagreed with our observation that it did not explain in its 

strategic plan how it used the results of program evaluations to update 

the current plan and did not include a schedule for future evaluations. 

As we have previously reported, program evaluations are individual, 

systematic studies that use objective measurement and analysis to 

answer specific questions about how well a program is working and, 

thus, may take many forms. Where a program aims to produce changes 

that result from program activities, outcome or effectiveness 

evaluations assess the extent to which those results were achieved. 

Where complex systems or events outside a program’s control also 

influence its outcomes, impact evaluations use scientific research 

methods to establish the causal connection between outcomes and 

program activities and isolate the program’s contribution to those 

changes. A program evaluation that also systematically examines how 

a program was implemented can provide important information about 

why a program did or did not succeed and suggest ways to improve it.1

 

In its strategic plan, HUD provides a few examples of how it modified 

performance measures as a result of program evaluations. However, 

we found that 38 of the 41 performance measures discussed in the 

strategic plan did not mention how, if at all, HUD revised and/or 

updated them as the result of program evaluations. Elsewhere in the 

plan, HUD discussed program evaluation activities carried out by its 

Office of Policy Development and Research; however, a significant 

number of those evaluations will take place in the future and there is no 

fixed timetable for when HUD will issue reports on its findings.

3. HUD questioned an example we used to show that its strategic plan did 

not always provide a clear picture of how it will be able to measure 

progress toward its strategic goals. We chose this example because 

HUD used the number of jobs created or retained to measure its 

progress in achieving the results of the Community Development Block 

Grant (CDBG) program. As HUD discusses in it strategic plan, there 

are factors external to the CDBG program, such as broad macroeconomic trends and HUD’s limited control over how grant recipients 

use the funding, which can significantly affect job creation in a 

community. Therefore, it is difficult to establish the contribution of the 

CDBG program—apart from the other factors—to HUD’s stated goal.

1

GAO/GGD-00-204.
Appendix X

Comments from the Department of Housing 

and Urban Development

Page 253 GAO-04-38 Results-Oriented Government

4. HUD also disagreed with our observation that in its annual 

performance report it did not state the steps it would take to address 

unmet performance goals. We recognize that in some instances HUD 

mentioned how it would address unmet goals. GPRA requires that 

agencies explain and describe, where a performance goal has not been 

met, why the goal was not met, schedules for achieving the established 

performance goal, and whether or not the performance goal is 

impractical or unfeasible. However, our review of HUD’s performance 

report found that of the 93 unmet performance targets for fiscal year 

2002, 74 lacked an explanation of how HUD would address them in 

fiscal year 2003.

5. In commenting on our observation that HUD did not include an 

evaluation of its fiscal year 2003 performance plan relative to the 

performance attained by the department in fiscal year 2002, HUD 

mentioned that we should consider the impact of the acceleration of 

reporting deadlines on the department’s ability to include an evaluation 

of the new fiscal year’s performance plan relative to the performance 

attained in the just completed fiscal year and reasonable alternative 

actions to fulfill this requirement. While we acknowledge that changes 

in the reporting deadlines can create challenges for federal agencies, 

these deadlines are governmentwide and not specific to HUD. In our 

review of agency plans we found that some agencies, such as DOT, 

were able to collect performance information for 95 percent of their 

performance indicators and were able to predict future performance, 

despite not having complete performance information and facing the 

same deadlines. DOT provided an evaluation of whether or not fiscal 

year 2003 performance targets would be met for each of its 40 

performance goals based on fiscal year 2002 results. These evaluations 

were included for the two performance goals for which data were 

unavailable. For example, for the measure “Number of employment 

sites (in the thousands) that are made accessible by Job Access and 

Reverse Commute (JARC) transportation services,” DOT could not 

characterize performance since data had not yet been received from 

JARC grantees. The 2002 performance report stated that a new easier 

to use reporting system is being implemented that should improve data 

gathering performance. The report further stated that DOT would meet 

this target in fiscal year 2003.

6. HUD also disagreed with how we presented the performance 

information in its summary report cards (see fig. 22). HUD noted that 

many of the results were explained in the individual indicator write-ups 
Appendix X

Comments from the Department of Housing 

and Urban Development

Page 254 GAO-04-38 Results-Oriented Government

that followed the summary information. Our review of HUD’s reports 

included, among other things, qualitative aspects of how the 

information was presented, such as its usefulness to inform the average 

reader with little to no exposure on the subject matter, and the extent 

to which it presented summarized performance information that was 

complete and user-friendly. Our analysis of HUD’s performance 

information was largely based on a review of the information and terms 

used in the performance report cards. We characterized some of HUD’s 

performance indicators as being “undetermined,” given that HUD did 

not clearly indicate whether or not a goal was achieved. Instead, HUD 

provided footnotes, such as “results too complex to summarize.” We 

also characterized some performance targets as having “no 

explanation,” given that information was missing from the report card 

to determine whether HUD had reached its desired target. To develop 

the graphic summarizing HUD’s performance information, we compiled 

the results of HUD’s performance indicators across all five report cards 

contained in the report.
Page 255 GAO-04-38 Results-Oriented Government

Appendix XI

Comments from the Social Security 

Administration ApendxiXI

Note: GAO comments 

supplementing those in 

the report text appear 

at the end of this 

appendix.
Appendix XI

Comments from the Social Security 

Administration

Page 256 GAO-04-38 Results-Oriented Government

See comment 1.
Appendix XI

Comments from the Social Security 

Administration

Page 257 GAO-04-38 Results-Oriented Government

See comment 2.

See comment 3.

See comment 4.
Appendix XI

Comments from the Social Security 

Administration

Page 258 GAO-04-38 Results-Oriented Government

See comment 5.
Appendix XI

Comments from the Social Security 

Administration

Page 259 GAO-04-38 Results-Oriented Government

Now on pp. 65 and 215.
Appendix XI

Comments from the Social Security 

Administration

Page 260 GAO-04-38 Results-Oriented Government

The following are our comments on SSA’s letter dated January 16, 2004.

GAO Comments In general, SSA agreed with our conclusions. SSA also agreed to 

incorporate the suggestions for improvement in its future planning efforts. 

SSA made several points of clarification and disagreed with our assessment 

in one area.

1. In our draft report, we noted that SSA did not explicitly link external 

factors that may affect its programs to its general goals and state how 

these factors could affect goal attainment. SSA attests that its four 

categories of environmental factors are discussed under each of the 

strategic goals, as appropriate, and the relationship between these 

factors and SSA’s strategic priorities is described. This general 

discussion of the environmental factors is useful in understanding the 

challenges SSA faces in working toward its broad strategic goals. 

However, SSA provides little or no discussion of these challenges in its 

discussion of the agency’s performance goals. Thus, the range of 

challenges facing the agency in meeting each of its performance goals 

is not fully explained.

2. In our draft report, we noted that SSA does not provide timetables or 

schedules for achieving all the results in its strategic plan. SSA noted 

that it expects to achieve its long-term outcomes within the 5-year 

period covered by the strategic plan; in selected instances, shorter time 

frames are specified. SSA noted that more detailed plans and 

timetables are featured in its annual performance plan. GPRA requires 

agencies to furnish a schedule of significant actions in their strategic 

plans; however, SSA does not clearly articulate its timetables and 

schedules for achieving each of its long-term outcomes in its strategic 

plan.

3. We noted that SSA’s strategic plan could be improved by providing 

details on how each performance and accountability challenge will be 

addressed. SSA asserted that the strategic plan addresses some of the 

challenges, but because the challenges are updated every year, they are 

more appropriately addressed in the annual performance plan and 

performance and accountability report. As noted in our discussion of 

the criteria used to analyze agencies’ strategic plans, it is particularly 

important that agencies develop strategies that address management 

challenges that threaten their ability to meet long-term strategic goals, 
Appendix XI

Comments from the Social Security 

Administration

Page 261 GAO-04-38 Results-Oriented Government

as one of the purposes of GPRA is to improve the management of 

federal agencies.

4. In our draft report, we observed that SSA’s discussion of its interactions 

with other agencies, especially those that serve the same beneficiaries, 

was limited. SSA noted that such a discussion would be useful, but is 

not an OMB requirement. While we agree that this is not an OMB 

requirement, we have reported that given scarce resources and 

competing priorities, it would be useful to identify agency efforts to 

maximize its effect through cooperation and coordination across the 

federal government. Better strategic plans not only identify the need to 

coordinate with other agencies, but also discuss how agencies intend to 

coordinate common or complementary goals and strategies with other 

agencies.

5. With regard to SSA’s performance and accountability report, we noted 

that SSA did not clearly state how program evaluations were used to 

answer questions about program performance and results and how 

those results can be improved. SSA disagreed with our observation, 

stating that many of its evaluations rely on surveys, and these surveys 

form the basis for its efforts to deliver high-quality service. SSA also 

noted that it listed other evaluations that are of great importance to its 

ongoing operations. We do not discount the usefulness of SSA’s surveys 

in assessing its day-to-day management of programs. Rather, as we 

noted in the report, it would be helpful for SSA to clearly identify the 

range of evaluations conducted and how each of them contributed to 

improved program performance. For example, we recently 

recommended that SSA evaluate a new initiative to improve the 

integrity of Social Security number issuance to noncitizens; the 

description of such an evaluation would be helpful for SSA to 

determine how it can be best positioned to ensure the integrity of its 

enumeration process.1 Additionally, our September 2000 report on 

program evaluation states that GPRA recognizes the complementary 

nature of program evaluation and performance measurement. Strategic 

plans are to describe the program evaluations that were used in 

establishing and revising goals and to include a schedule for future 

1

U.S. General Accounting Office, Social Security Administration: Actions Taken to 

Strengthen Procedures for Issuing Social Security Numbers to Noncitizens, but Some 

Weaknesses Remain, GAO-04-12 (Washington, D.C.: Oct. 15, 2003).
Appendix XI

Comments from the Social Security 

Administration

Page 262 GAO-04-38 Results-Oriented Government

program evaluations. Agencies are to summarize the findings of 

program evaluations in their annual performance reports.

Additionally, SSA made technical comments that we incorporated into the 

report, as appropriate.
Page 263 GAO-04-38 Results-Oriented Government

Appendix XII

GAO Contact and Staff Acknowledgments ApendxiXI

GAO Contact Patricia Dalton, (202) 512-6806

Acknowledgments In addition to the persons mentioned above, Thomas Beall, Daniel Bertoni, 

Kay Brown, Joyce Corry, Elizabeth Curda, David Dornisch, William Fenzel, 

Kimberly Gianopoulos, Evan Gilman, Katie Harris, Susan Higgins, 

Benjamin Licht, William McKelligott, James Noel, Carol Petersen, Carolyn 

Samuels, Teresa Spisak, Daren Sweeney, Carolyn Taylor, Michael Volpe, 

Lynn Wasielewski, and Steven Westley made key contributions to this 

report.
Page 264 GAO-04-38 Results-Oriented Government

Related GAO Products

GPRA/Managing for 

Results

Results-Oriented Government: Using GPRA to Address 21st Century 

Challenges, GAO-03-1166T (Washington, D.C.: Sept. 18, 2003).

Results-Oriented Management: Agency Crosscutting Actions and Plans 

in Border Control, Flood Mitigation and Insurance, Wetlands, and 

Wildland Fire Management, GAO-03-321 (Washington, D.C.: Dec. 20, 2002).

Results-Oriented Management: Agency Crosscutting Actions and Plans 

in Drug Control, Family Poverty, Financial Institution Regulation, and 

Public Health Systems, GAO-03-320 (Washington, D.C.: Dec. 20, 2002).

Performance and Accountability: Reported Agency Actions and Plans to 

Address 2001 Management Challenges and Program Risks, GAO-03-225

(Washington, D.C.: Oct. 31, 2002).

Managing for Results: Next Steps to Improve the Federal Government’s 

Management and Performance, GAO-02-439T (Washington, D.C.: Feb. 15, 

2002).

Managing for Results: Federal Managers’ Views on Key Management 

Issues Vary Widely Across Agencies, GAO-01-592 (Washington, D.C.: 

May 25, 2001).

Managing for Results: Federal Managers’ Views Show Need for Ensuring 

Top Leadership Skills, GAO-01-127 (Washington, D.C.: Oct. 20, 2000).

Managing for Results: Barriers to Interagency Coordination, 

GAO/GGD-00-106 (Washington, D.C.: Mar. 29, 2000).

Management Reform: Elements of Successful Improvement Initiatives, 

GAO/T-GGD-00-26 (Washington, D.C.: Oct. 15, 1999).

Management Reform: Using the Results Act and Quality Management to 

Improve Federal Performance, GAO/T-GGD-99-151 (Washington, D.C.: 

July 29, 1999).

Managing for Results: Opportunities for Continued Improvements in 

Agencies’ Performance Plans, GAO/GGD/AIMD-99-215 (Washington, D.C.: 

July 20, 1999).
Related GAO Products

Page 265 GAO-04-38 Results-Oriented Government

Agency Performance Plans: Examples of Practices That Can Improve 

Usefulness to Decisionmakers, GAO/GGD/AIMD-99-69 (Washington, D.C.: 

Feb. 26, 1999).

The Results Act: Assessment of the Governmentwide Performance Plan 

for Fiscal Year 1999, GAO/AIMD/GGD-98-159 (Washington, D.C.: Sept. 8, 

1998).

Managing for Results: An Agenda to Improve the Usefulness of Agencies’ 

Annual Performance Plans, GAO/GGD/AIMD-98-228 (Washington, D.C.: 

Sept. 8, 1998).

The Results Act: An Evaluator’s Guide to Assessing Agency Annual 

Performance Plans. GAO/GGD-10.1.20 (Washington, D.C.: April 1998).

Agencies’ Annual Performance Plans Under the Results Act: An 

Assessment Guide to Facilitate Congressional Decisionmaking, 

GAO/GGD/AIMD-10.1.18 (Washington, D.C.: February 1998).

Managing for Results: Agencies’ Annual Performance Plans Can Help 

Address Strategic Planning Challenges, GAO/GGD-98-44 (Washington, 

D.C.: Jan. 30, 1998).

Managing for Results: Building on Agencies’ Strategic Plans to Improve 

Federal Management, GAO/T-GGD/AIMD-98-29 (Washington, D.C.: Oct. 30, 

1997).

Managing for Results: Critical Issues for Improving Federal Agencies’ 

Strategic Plans, GAO/GGD-97-180 (Washington, D.C.: Sept. 16, 1997).

Managing for Results: Using the Results Act to Address Mission 

Fragmentation and Program Overlap, GAO/AIMD-97-146 (Washington, 

D.C.: Aug. 29, 1997).

Managing for Results: The Statutory Framework for Improving Federal 

Management and Effectiveness, GAO/T-GGD/AIMD-97-144 (Washington, 

D.C.: June 24, 1997).

The Government Performance and Results Act: 1997 Governmentwide 

Implementation Will Be Uneven, GAO/GGD-97-109 (Washington, D.C.: 

June 2, 1997).
Related GAO Products

Page 266 GAO-04-38 Results-Oriented Government

Agencies’ Strategic Plans Under GPRA: Key Questions to Facilitate 

Congressional Review, GAO/GGD-10.1.16 (Washington, D.C.: May 1997).

GPRA: Managerial Accountability and Flexibility Pilot Did Not Work as 

Intended, GAO/GGD-97-36 (Washington, D.C.: Apr. 10, 1997).

Managing for Results: Enhancing the Usefulness of GPRA Consultations 

Between the Executive Branch and Congress, GAO/T-GGD-97-56

(Washington, D.C.: Mar. 10, 1997).

Managing for Results: Key Steps and Challenges in Implementing GPRA 

in Science Agencies, GAO/T-GGD/RCED-96-214 (Washington, D.C.: July 10, 

1996).

Executive Guide: Effectively Implementing the Government Performance 

and Results Act, GAO/GGD-96-118 (Washington, D.C.: June 1996).

Strategic Human 

Capital Management

Human Capital: A Guide for Assessing Strategic Training and 

Development Efforts in the Federal Government (Exposure Draft),

GAO-03-893G (Washington, D.C.: July 2003).

Results-Oriented Cultures: Creating a Clear Linkage between Individual 

Performance and Organizational Success, GAO-03-488 (Washington, D.C.: 

Mar. 14, 2003).

Results-Oriented Cultures: Insights for U.S. Agencies from Other 

Countries’ Performance Management Initiatives, GAO-02-862 

(Washington, D.C.: Aug. 2, 2002).

Human Capital: Key Principles From Nine Private Sector 

Organizations, GAO/GGD-00-28 (Washington, D.C.: Jan. 31, 2000).

Linking Resources to 

Results

Performance Budgeting: Observations on the Use of OMB’s Program 

Assessment Rating Tool for the Fiscal Year 2004 Budget, GAO-04-174

(Washington, D.C.: Jan. 30, 2004).

Managing for Results: Efforts to Strengthen the Link Between Resources 

and Results at the Nuclear Regulatory Commission, GAO-03-258

(Washington, D.C.: Dec. 10, 2002).
Related GAO Products

Page 267 GAO-04-38 Results-Oriented Government

Managing for Results: Efforts to Strengthen the Link Between Resources 

and Results at the Administration for Children and Families, GAO-03-9

(Washington, D.C.: Dec. 10, 2002).

Managing for Results: Efforts to Strengthen the Link Between Resources 

and Results at the Veterans Health Administration, GAO-03-10

(Washington, D.C.: Dec. 10, 2002).

Managing for Results: Agency Progress in Linking Performance Plans 

With Budgets and Financial Statements, GAO-02-236 (Washington, D.C.: 

Jan. 4, 2002).

Performance Budgeting: Fiscal Year 2000 Progress in Linking Plans 

With Budgets, GAO/AIMD-99-239R (Washington, D.C.: July 30, 1999).

Performance Budgeting: Initial Experiences Under the Results Act in 

Linking Plans With Budgets, GAO/AIMD/GGD-99-67 (Washington, D.C.: 

Apr. 12, 1999).

Performance Budgeting: Past Initiatives Offer Insights for GPRA 

Implementation, GAO/AIMD-97-46 (Washington, D.C.: Mar. 27, 1997).

Measuring 

Performance

Program Evaluation: An Evaluation Culture and Collaborative 

Partnerships Help Build Agency Capacity, GAO-03-454 (Washington, D.C.:

May 2, 2003).

Managing for Results: Next Steps to Improve the Federal Government’s 

Management and Performance, GAO-02-439T (Washington, D.C.: Feb. 15, 

2002).

Program Evaluation: Studies Helped Agencies Measure or Explain 

Program Performance, GAO/GGD-00-204 (Washington, D.C. Sept. 29, 

2000).

Managing for Results: Strengthening Regulatory Agencies’ Performance 

Management Practices, GAO/GGD-00-10 (Washington, D.C.: Oct. 28, 1999).

Managing for Results: Measuring Program Results That Are Under 

Limited Federal Control, GAO/GGD-99-16 (Washington, D.C.: Dec. 11, 

1998).
Related GAO Products

Page 268 GAO-04-38 Results-Oriented Government

Grant Programs: Design Features Shape Flexibility, Accountability, and 

Performance Information, GAO/GGD-98-137 (Washington, D.C.: June 22, 

1998).

Program Evaluation: Agencies Challenged by New Demand for 

Information on Program Results, GAO/GGD-98-53 (Washington, D.C.: 

Apr. 24, 1998).

Managing for Results: Regulatory Agencies Identified Significant 

Barriers to Focusing on Results, GAO/GGD-97-83 (Washington, D.C.: 

June 24, 1997).

Managing for Results: Analytic Challenges in Measuring Performance, 

GAO/HEHS/GGD-97-138 (Washington, D.C.: May 30, 1997).

Data Credibility Performance Reporting: Few Agencies Reported on the Completeness and 

Reliability of Performance Data, GAO-02-372 (Washington, D.C.: Apr. 26, 

2002).

Managing for Results: Assessing the Quality of Program Performance 

Data, GAO/GGD-00-140R (Washington, D.C.: May 25, 2000).

Managing for Results: Challenges Agencies Face in Producing Credible 

Performance Information, GAO/GGD-00-52 (Washington, D.C.: Feb. 4, 

2000).

Performance Plans: Selected Approaches for Verification and Validation 

of Agency Performance Information, GAO/GGD-99-139 (Washington, D.C.: 

July 30, 1999).

Using Performance 

Information

Managing for Results: Views on Ensuring the Usefulness of Agency 

Performance Information to Congress, GAO/GGD-00-35 (Washington, 

D.C.: Jan. 26, 2000).

Managing for Results: Using GPRA to Assist Congressional and 

Executive Branch Decisionmaking, GAO/T-GGD-97-43 (Washington, D.C.: 

Feb. 12, 1997).
Related GAO Products

Page 269 GAO-04-38 Results-Oriented Government

Managing for Results: Achieving GPRA’s Objectives Requires Strong 

Congressional Role, GAO/T-GGD-96-79 (Washington, D.C.: Mar. 6, 1996).

(450181)
GAO’s Mission The General Accounting Office, the audit, evaluation and investigative arm of 

Congress, exists to support Congress in meeting its constitutional responsibilities 

and to help improve the performance and accountability of the federal government 

for the American people. GAO examines the use of public funds; evaluates federal 

programs and policies; and provides analyses, recommendations, and other 

assistance to help Congress make informed oversight, policy, and funding 

decisions. GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

Obtaining Copies of 

GAO Reports and 

Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older 

products. The Web site features a search engine to help you locate documents 

using key words and phrases. You can print these documents in their entirety, 

including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site 

daily. The list contains links to the full-text document files. To have GAO e-mail this 

list to you every afternoon, go to www.gao.gov and select “Subscribe to 

e-mail alerts” under the “Order GAO Products” heading.

Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. A check 

or money order should be made out to the Superintendent of Documents. GAO 

also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single 

address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Public Affairs Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Service Requested

Presorted Standard

Postage & Fees Paid

GAO

Permit No. GI00