United States General Accounting Office

GAO Report to the Chairman, Committee on

Governmental Affairs, U.S. Senate

February 1999 AGENCY

PERFORMANCE PLANS

Examples of Practices

That Can Improve

Usefulness to

Decisionmakers

GAO/GGD/AIMD-99-69

GAO United States

General Accounting Office

Washington, D.C. 20548

General Government Division

B-279902 

February 26, 1999

The Honorable Fred Thompson

Chairman, Committee on Governmental Affairs

United States Senate

Dear Mr. Chairman:

The Government Performance and Results Act of 1993 seeks to improve

the effectiveness, efficiency, and accountability of federal programs by

establishing a system for agencies to set goals for program performance

and to measure results. Under the Act, agencies are to prepare annual

performance plans to systematically provide decisionmakers with

information on the results to be achieved for a proposed level of

resources. These plans are to reinforce the connection between the

long-term strategic goals outlined in agencies’ strategic plans and the

day-to-day activities of their program managers and staff. In so doing, the

annual performance plans provide a basis for establishing accountability

for results.

Fiscal year 1999 marks the first time agencies across the federal

government were to prepare annual performance plans covering each

program activity in their budgets.1

 As requested by several members of the

congressional leadership, we reviewed the performance plans of the 24

agencies covered by the Chief Financial Officers (CFO) Act. After briefing

congressional staff, we issued separate reports on each of the 24 agencies’

fiscal year 1999 performance plans.2

 In September 1998, we issued a report

that summarized our observations on the agencies’ plans and noted that

substantial further development is needed for these plans to be useful in a

significant way to congressional and other decisionmakers.3

 That report

identified opportunities to improve the usefulness of future performance

plans. These opportunities include (1) better articulating a results

orientation, (2) coordinating crosscutting programs, (3) clearly showing

how strategies will be used to achieve goals, (4) showing performance

consequences of budget decisions, and (5) building capacity within

agencies to gather and use performance information.

1

The term “program activity” refers to the listing of projects and activities in the appendix portion of

the Budget of the United States Government. Program activity structures are intended to provide a

meaningful representation of the operations financed by a specific budget account.

2

See appendix I for the list of reports on fiscal year 1999 performance plans.

3

Managing for Results: An Agenda to Improve the Usefulness of Agencies’ Annual Performance Plans

(GAO/GGD/AIMD-98-228, Sept. 8, 1998).

Page 1 GAO/GGD/AIMD-99-69 Agency Performance Plans
B-279902 

As you requested, this report builds on those improvement opportunities

by (1) identifying and describing practices that were applied in some plans

that might, if consistently applied, improve the usefulness of all agencies’

annual performance plans; and (2) providing examples from agencies’

fiscal year 1999 performance plans that illustrate each practice. A

forthcoming report that you also requested will provide additional

information and examples related to one of those practices—showing the

relationship of budgetary resources to performance goals.

Results Taken as a whole, some of the fiscal year 1999 annual performance plans

contain practices that, if consistently applied, should help improve the

usefulness of subsequent agency performance plans to congressional and

other decisionmakers. These practices are shown in table 1.

Page 2 GAO/GGD/AIMD-99-69 Agency Performance Plans
B-279902 

Table 1: Summary of Opportunities to Improve the Usefulness of Annual Plans and Applied Practices

Opportunities to improve the usefulness

of annual plans Applied practices

Better articulate a results orientation. 1. Create a set of performance goals and measures that addresses important

dimensions of program performance and balances competing priorities.

2. Use intermediate goals and measures to show progress or contribution to intended

results.

3. Include explanatory information on the goals and measures.

4. Develop performance goals to address mission-critical management problems.

5. Show baseline and trend data for past performance.

6. Identify projected target levels of performance for multiyear goals.

7. Link the goals of component organizations to departmental strategic goals.

Coordinate crosscutting programs. 8. Identify programs that contribute to the same or similar results.

9. Set complementary performance goals to show how differing program strategies are

mutually reinforcing and establish common or complementary performance measures,

as appropriate.

10. Describe—briefly or refer to a separate document—planned coordination strategies.

Show how strategies will be used to achieve

goals.

11. Link strategies and programs to specific performance goals and describe how they

will contribute to the achievement of those goals.

12. Describe strategies to leverage or mitigate the effects of external factors on the

accomplishment of performance goals.

13. Discuss strategies to resolve mission-critical management problems.

14. Discuss—briefly or refer to a separate document—plans to ensure that

mission-critical processes and information systems function properly and are secure.

Show performance consequences of budget

and other resource decisions.

15. Show how budgetary resources relate to the achievement of performance goals.

16. Discuss—briefly and refer to the agency capital plan—how proposed capital assets

(specifically information technology investments) will contribute to achieving

performance goals.

17. Discuss—briefly or refer to a separate plan—how the agency will use its human

capital to help achieve performance goals.

Build the capacity to gather and use

performance information.

18. Identify internal and external sources for data.

19. Describe efforts to verify and validate performance data.

20. Identify actions to compensate for unavailable or low-quality data.

21. Discuss implications of data limitations for assessing performance.

Source: GAO analysis of agency performance plans.

Page 3 GAO/GGD/AIMD-99-69 Agency Performance Plans
B-279902 

The applied practices and examples illustrating how these practices were

incorporated into certain agencies’ fiscal year 1999 performance plans are

discussed in the individual sections accompanying this report.

Objectives, Scope,

and Methodology

At your request, the objectives of this review were to (1) identify and

describe practices to improve the usefulness of agencies’ annual

performance plans and (2) provide examples from agencies’ fiscal year

1999 performance plans that illustrate each practice.

To meet both of our objectives, we contacted selected House and Senate

congressional staff—representing appropriation, authorization, budget,

and oversight committees that reviewed the 24 CFO Act agencies’ annual

plans—to solicit their views on the features they found to be particularly

useful in the plans. We also reviewed our individual and summary reports

on agencies’ fiscal year 1999 plans; the Results Act requirements for

agency annual performance plans; guidelines contained in the Office of

Management and Budget (OMB) Circular No. A-11, Part 2; fiscal year 1999

performance plans of the 24 CFO Act agencies; our guides to assist

agencies and Congress to effectively implement the Results Act;4

 and our

work over the last several years examining agencies’ efforts to implement

the Results Act. Appendix II provides references to the Results Act; OMB

Circular No. A-11, Part 2; and our recently issued related products and is

arranged by the sections discussed in this report.

We did our work from March 1998 to January 1999 in Washington, D.C., in

accordance with generally accepted government auditing standards. We

did not seek agency comments on this report because much of the analysis

was drawn from our review of the fiscal year 1999 performance plans on

which the agencies had already commented. However, we asked officials

in each of the agencies profiled to verify the accuracy of the information

presented on their respective agencies’ fiscal year 1999 performance plans,

and we incorporated their comments where applicable. We did not

independently verify the accuracy of the information contained in the

agencies’ fiscal year 1999 performance plans.

4

The Results Act: An Evaluator’s Guide to Assessing Agency Annual Performance Plans, Version 1

(GAO/GGD-10.1.20, Apr. 1998); Agencies’ Annual Performance Plans Under the Results Act: An

Assessment Guide to Facilitate Congressional Decisionmaking, Version 1 (GAO/GGD/AIMD-10.1.18,

Feb. 1998); Agencies’ Strategic Plans Under GPRA: Key Questions to Facilitate Congressional Review,

Version 1 (GAO/GGD-10.1.16, May 1997); and Executive Guide: Effectively Implementing the

Government Performance and Results Act (GAO/GGD-96-118, June 1996). See also GPRA Performance

Reports (GAO/GGD-96-66R, Feb. 14, 1996).

Page 4 GAO/GGD/AIMD-99-69 Agency Performance Plans
B-279902 

We are sending copies of this report to the Ranking Minority Member of

the Senate Committee on Governmental Affairs; the House Majority

Leader; the Chair and Ranking Minority Member of the House Committee

on Government Reform; other interested congressional committees; and to

the Director, Office of Management and Budget. We will also make copies

available to others on request.

The major contributors to this report are listed in appendix III. If you have

any questions concerning this report, please contact me on (202) 512-8676.

Sincerely yours,

J. Christopher Mihm

Associate Director, Federal Management and Workforce Issues

Page 5 GAO/GGD/AIMD-99-69 Agency Performance Plans
Contents

Letter 1

Section 1 

Better Articulate a

Results Orientation

10

Section 2 

Coordinate

Crosscutting

Programs

19

Section 3 

Show How Strategies

Will Be Used to

Achieve Goals

24

Section 4 

Show Performance

Consequences of

Budget and Other

Resource Decisions

29

Section 5 

Build the Capacity to

Gather and Use

Performance

Information

38

Page 6 GAO/GGD/AIMD-99-69 Agency Performance Plans
Contents

Appendix I 

GAO Products on

Fiscal Year 1999

Performance Plans

42

Departments 42

Independent Agencies 43

Appendix II 

Related Guidance on

Annual Performance

Plans

45

Section 1: Better Articulate a Results Orientation 45

Section 2: Coordinate Crosscutting Programs 46

Section 3: Show How Strategies Will Be Used to Achieve Goals 46

Section 4: Show Performance Consequences of Budget and Other

Resource Decisions

48

Section 5: Build the Capacity to Gather and Use Performance

Information

50

Appendix III 

Major Contributors

51

Tables Table 1: Summary of Opportunities to Improve the Usefulness of

Annual Plans and Applied Practices

3

Table 1.1: Applied Practices to Address a Results Orientation 10

Table 2.1: Applied Practices to Address Crosscutting Programs 19

Table 3.1: Applied Practices to Address Goal Achievement

Strategies

24

Table 4.1: Applied Practices to Address Budget and Resource

Decisions

29

Table 5.1: Applied Practices to Address PerformanceInformation

Capacity

38

Figures Figure 1.1: Excerpt From the Department of Commerce’s Fiscal

Year 1999 Annual Performance Plan Featuring the National

Oceanic and Atmospheric Administration

12

Figure 1.2: Excerpt From the Department of the Treasury’s Fiscal

Year 1999 Annual Performance Plan Featuring the U.S. Customs

Service

16

Figure 1.3: Excerpt From the Department of Veterans Affairs’

Fiscal Year 1999 Annual Performance Plan

17

Figure 2.1: Excerpt From the Department of Commerce’s Fiscal

Year 1999 Annual Performance Plan

20

Page 7 GAO/GGD/AIMD-99-69 Agency Performance Plans
Contents

Figure 2.2: Excerpt From the Department of Education’s Fiscal

Year 1999 Annual Performance Plan

23

Figure 4.1: Excerpt From the Department of Health and Human

Services’ Fiscal Year 1999 Annual Performance Plan Featuring

the Indian Health Service

31

Figure 4.2: Excerpt From the Environmental Protection Agency’s

Fiscal Year 1999 Annual Performance Plan

33

Figure 4.3: Excerpt From the Department of the Treasury’s Fiscal

Year 1999 Annual Performance Plan Featuring the Internal

Revenue Service

35

Abbreviations

CFO Act Chief Financial Officers Act

GPRA Government Performance and Results Act

OMB Office of Management and Budget

ONDCP Office of National Drug Control Policy

OPM Office of Personnel Management

Page 8 GAO/GGD/AIMD-99-69 Agency Performance Plans
Page 9 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1 

Better Articulate a Results Orientation

Annual performance plans could be more useful documents in helping to

guide decisions and, subsequently, assess actual performance if the plans

consistently showed how the agency was adopting a results orientation.

Table 1.1 identifies practices that can help agencies articulate a results

orientation.

Table 1.1: Applied Practices to

Address a Results Orientation

A performance plan that better articulates

a results orientation includes the

following practices:

1. Create a set of performance goals and

measures that addresses important

dimensions of program performance and

balances competing priorities.

2. Use intermediate goals and measures to

show progress or contribution to intended

results.

3. Include explanatory information on the

goals and measures.

4. Develop performance goals to address

mission-critical management problems.

5. Show baseline and trend data for past

performance.

6. Identify projected target levels of

performance for multiyear goals.

7. Link the goals of component

organizations to departmental strategic

goals.

1. Create a set of performance goals and measures that addresses

important dimensions of program performance and balances

competing priorities.

Performance goals and measures that successfully address important and

varied aspects of program performance are key aspects of a results

orientation. To illustrate, the Department of Transportation’s general goal

to advance highway safety was included in its performance plan and was

accompanied by various goals and measures to depict the complex

performance they are intended to assess. For example, although the

department’s performance goals and measures on the highway-related

fatality rate and highway-related injury rate are the primary gauges of

safety, they are complemented by other goals and measures that

contribute to those results. Specifically, the plan includes a goal to

Page 10 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1 

Better Articulate a Results Orientation

increase the percentage rate of seat belt usage by front seat occupants, a

key element in reducing overall highway-related fatalities and injuries.

Federal programs are designed and implemented in dynamic environments

where competing program priorities and stakeholders’ needs must be

balanced continuously and new needs must be addressed.1

 As a result,

programs are often forced to strike difficult balances among priorities that

reflect competing demands, such as timeliness, service quality, customer

satisfaction, program cost, and other stakeholder concerns. Sets of

performance goals and measures could provide a balanced perspective of

the intended performance of a program’s multiple priorities. In its

performance plan, for example, the Department of Commerce’s National

Oceanic and Atmospheric Administration defined its expected

performance to improve the timeliness and accuracy of short-range

environmental predictions that have immediate impact on individuals and

many sectors of the economy. The relationship between “timeliness” and

“accuracy” can represent competing demands because the accuracy of

predictions could be improved as the weather event comes closer to a

vicinity. However, the closer the weather event to a vicinity, the shorter

and less timely the warning or lead time to the public in that vicinity. For

fiscal year 1999, the agency set a lead time of 11 minutes for tornado

warnings with 70-percent accuracy. The agency also reported that it set a

lead time of 10 minutes with 59-percent accuracy in fiscal year 1997 and 10

minutes with 65-percent accuracy in fiscal year 1998. (See fig. 1.1.)

1

Performance and Accountability Series: Major Management Challenges and Program Risks: A

Governmentwide Perspective (GAO/OCG-99-1, Jan. 1999).

Page 11 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1 

Better Articulate a Results Orientation

Figure 1.1: Excerpt From the Department of Commerce’s Fiscal Year 1999 Annual Performance Plan Featuring the National

Oceanic and Atmospheric Administration

Program cost is another important factor to be considered. In crafting the

Results Act, Congress expected that agencies, whenever possible, would

develop performance measures that correlate the level of program activity

with program costs, such as costs per unit of result, costs per unit of

Page 12 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1 

Better Articulate a Results Orientation

service, or costs per unit of output. Agencies were expected to assign a

high priority to developing these types of unit cost measures. The

successful implementation of the managerial cost accounting standards

recommended by the Federal Accounting Standards Advisory Board and

issued by OMB and GAO are vital to providing agencies the program cost

information needed to develop such performance measures. Effective for

fiscal year 1998, these standards are to provide decisionmakers with

information on the costs of all resources used and the costs of services

provided by others to support activities or programs, thus allowing

decisionmakers to compare the costs of various programs and activities

with their performance outputs and results.

2. Use intermediate goals and measures to show progress or

contribution to intended results.

Intermediate goals and measures, such as outputs or intermediate

outcomes, can be used to show progress or contribution to intended

results. For instance, when it may take years before an agency sees the

results of its programs, intermediate goals and measures can provide

information on interim results. Also, when program results could be

influenced by external factors, agencies can use intermediate goals and

measures to identify the programs’ discrete contribution to a specific

result.

As an example, the Department of Health and Human Services’ Centers for

Disease Control and Prevention reported in its performance plan that

heart disease is the nation’s number one killer among men and women of

all racial and ethnic groups. However, because of the long latency period

of chronic diseases, such as heart disease, health outcome measures do

not provide a complete picture of the agency’s annual performance. To

address the burden of heart disease, the Centers for Disease Control and

Prevention intends, among other things, to focus on the prevention of risk

factors, such as tobacco use. The plan explained that comprehensive state

programs have been shown to be effective in reducing the prevalence of

tobacco use. To show a nationwide reduction in morbidity and mortality

attributable to behavioral risk factors in tobacco use, the plan presented

an output measure to increase the number of state programs with tobacco

prevention capacities from 17 in 1996 to 30 in 1999. The Centers for

Disease Control and Prevention intends to fund programs in all 50 states.

Page 13 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1 

Better Articulate a Results Orientation

3. Include explanatory information on the goals and measures.

Explanatory information in a performance plan can help to show the

relationship among results-oriented goals, measures, and program outputs

and services. Such information is particularly important when an agency

uses intermediate goals or has several goals that need to be balanced. In

its performance plan, the Centers for Disease Control and Prevention

linked its program objectives to its performance goals and measures and

to the Department of Health and Human Services’ strategic goals. Also

included was a rationale for the use of process and output measures to

inform the user of the plan about limitations of the data and associated

program constraints. As previously illustrated, the section in its

performance plan on chronic disease prevention noted that health

outcome measures for these programs have been difficult to define for a

number of reasons, including the long latency period of chronic diseases

like heart disease.

4. Develop performance goals to address mission-critical

management problems.

Weaknesses in internal management processes and systems undermine the

achievement of program results. Therefore, the value of performance

plans could be increased if they more fully included performance goals to

address mission-critical management problems that may exist. Discussing

the most critical management problems—such as those identified by the

administration as a priority management objective in the governmentwide

performance plan, the agency’s financial audit, and other agency

assessments made by external organizations, including our recent

performance and accountability and high-risk series2

—ensures that those

management problems that would have the greatest impact on results

receive the most attention. A specific reference to those priority

management objectives identified in the governmentwide performance

plan can help show an integrated approach to their resolution.

The Department of Education’s annual performance plan referred to the

priority management objectives identified in the fiscal year 1999

governmentwide performance plan. Specifically, the plan acknowledged

that Education’s strategic objective on postsecondary student aid delivery

and program management relates to the governmentwide performance

plan’s priority management objective on student aid programs. Education

is responsible for the collection of more than $150 billion in outstanding

2

Performance and Accountability Series: Major Management Challenges and Program Risks

(GAO/OCG-99-22SET, Jan. 1999) and High-Risk Series: An Update (GAO/HR-99-1, Jan. 1999).

Page 14 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1 

Better Articulate a Results Orientation

loans, and its data systems track 93 million student loans and 15 million

Pell Grants. One of Education’s measures for the Federal Family

Education Loan and Direct Loan programs was a decline in the percentage

of borrowers leaving school who default on their student loans within 2

years. For the Federal Family Education Loan program, the annual plan

provided baseline and trend data for the default rate and indicated that the

rate declined from 22.4 to 10.4 percent from fiscal years 1990 to 1995. For

fiscal year 1999, Education’s goal for this program is to reduce the default

rate to 10.1 percent of borrowers. According to the plan, future declines

are likely to be steady but smaller because of the large number of

high-default schools that have already been eliminated from the program.

5. Show baseline and trend data for past performance.

With baseline and trend data, the more useful performance plans provided

a context for drawing conclusions about whether performance goals are

reasonable and appropriate. Decisionmakers can use such information to

gauge how a program’s anticipated performance level compares with

improvements or declines in past performance.

In addition to baseline and trend data on past performance, the

Department of the Treasury’s U.S. Customs Service performance plan

showed both actual and planned levels of performance for fiscal year 1997.

For example, in presenting its goal to maximize trade compliance,

Customs reported the extent of actual and proposed compliance with

trade laws in key industries. Specifically, in the bearings industry, Customs

reported that in fiscal year 1997, it achieved an 85.6 percent level of

compliance compared to a planned level of 83 percent; in the automobile

and truck parts industry, it achieved an 82.5 percent compliance level

compared to a planned level of 86 percent. The proposed performance

goals for fiscal year 1999 in both industries are 89 percent. (See fig. 1.2.)

Page 15 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1 

Better Articulate a Results Orientation

Figure 1.2: Excerpt From the Department of the Treasury’s Fiscal Year 1999 Annual Performance Plan Featuring the U.S.

Customs Service

6. Identify projected target levels of performance for multiyear

goals.

Where appropriate, an agency can convey what it expects to achieve in the

long term by including multiyear performance goals in its performance

plan. Such information can provide congressional and other

decisionmakers with an indication of the incremental progress the agency

expects to make in achieving results. As a start, for example, the

Page 16 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1 

Better Articulate a Results Orientation

Department of Veterans Affairs’ performance plan has an objective for

fiscal year 2002 to increase by 20 percent the number of patients (as

opposed to visits) of its healthcare system over its fiscal year 1997

baseline. The plan shows that the department projects achieving a

4.4 percent increase in patients treated in fiscal year 1998 and an

8.6 percent increase in fiscal year 1999. (See fig. 1.3.)

Figure 1.3: Excerpt From the Department of Veterans Affairs’ Fiscal Year 1999 Annual Performance Plan

7. Link the goals of component organizations to departmental

strategic goals.

Linking component performance to departmental goals can provide a

clear, direct understanding of how the achievement of the components’

annual goals will lead to the achievement of the agency’s strategic goals.

The Centers for Disease Control and Prevention, a component of the

Department of Health and Human Services, linked its annual performance

goals to the departmentwide strategic goals. For example, the plan set an

objective to reduce the case rate of tuberculosis—an infectious disease

that was nearly eliminated in the United States in the mid-1980s, only to

reemerge strongly in the 1990s with drug-resistant strains. The plan linked

Page 17 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 1 

Better Articulate a Results Orientation

that objective to Health and Human Services’ departmentwide goals to

reduce major threats to the health and productivity of all Americans.

Specific performance goals and measures were included in the plan to

reduce the tuberculosis case rate, such as increase to 85 percent the

patients who will complete a course of curative treatment within 12

months of initiation of treatment.

Page 18 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 2 

Coordinate Crosscutting Programs

A focus on results implies that agencies will coordinate efforts in

significant program areas where responsibility for achieving results is

shared among agencies. Agencies’ annual performance plans can be useful

tools for identifying such crosscutting programs, setting complementary

performance goals, and describing planned coordination strategies. Table

2.1 identifies applied practices that can help agencies address

coordination of crosscutting programs.

Table 2.1: Applied Practices to

Address Crosscutting Programs

A performance plan that addresses

coordination of crosscutting programs

includes the following practices:

8. Identify programs that contribute to the

same or similar results.

9. Set complementary performance goals

to show how differing program strategies

are mutually reinforcing and establish

common or complementary performance

measures, as appropriate.

10. Describe—briefly or refer to a separate

document—planned coordination

strategies.

8. Identify programs that contribute to the same or similar results.

Annual performance plans that include listings of programs and agencies

that share common results are useful in that they begin to document the

range and degree to which those federal programs have shared

responsibilities for achieving results. These listings can serve as the

prelude to the substantive work and fuller discussion of how agencies’

contributions to the same or similar results are being coordinated. As a

first step, the Department of Commerce included in its performance plan a

table identifying interagency activities within which the department

participates, such as formulating policy proposals. The three themes

drawn from Commerce’s mission statement provide the organizational

framework for the listing that includes crosscutting activities by bureaus

between Commerce and other federal agencies. (See fig. 2.1.)

Page 19 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 2 

Coordinate Crosscutting Programs

Figure 2.1: Excerpt From the Department of Commerce’s Fiscal Year 1999 Annual Performance Plan

9. Set complementary performance goals to show how differing

program strategies are mutually reinforcing and establish common

or complementary performance measures, as appropriate.

After agencies have identified other agencies with which they have a

shared responsibility, the next step is to include in their performance

plans complementary performance goals to show how differing program

strategies are mutually reinforcing and establish common or

complementary performance measures, as appropriate. Taking this

Page 20 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 2 

Coordinate Crosscutting Programs

further, the usefulness of a performance plan to congressional and other

decisionmakers could be enhanced if an agency identifies the

results-oriented performance goals that involve other agencies and sets

intermediate goals that clarify the specific contribution the agency makes

to the common result.

The Department of Health and Human Services’ Indian Health Service

associated its annual performance goals to the governmentwide goals of

Healthy People. Started in 1979, Healthy People is a series of

outcome-based public health objectives and measures developed and

updated each decade by the U.S. Public Health Service in consultation

with other federal agencies, state governments, and national organizations.

The objectives for 2000 succeeded the 1990 health objectives that were set

in 1980. The Public Health Service has started planning for the 2010 health

objectives, which are scheduled to be released in January 2000.

In developing its performance plan, the Indian Health Service reviewed the

appropriateness of the Healthy People objectives that focused on the most

significant health problems affecting American Indians and Alaska Natives

and the essential services that address them. The plan reported that

virtually all of the Service’s performance goals support the Healthy People

objectives for 2000. For example, the Indian Health Service’s fiscal year

1999 goal to ensure that the proportion of the American Indian and Alaska

Native female population over 40 years of age who have had screening

mammography is no lower than the fiscal year 1996 level relates to a

Healthy People objective on reducing the number of women who die from

breast cancer. The plan also described the approach for meeting this goal,

as well as the data sources and baseline data.

The performance plans for both the Department of Transportation and the

Department of Justice referenced the Office of National Drug Control

Policy (ONDCP) goal framework. Established in 1988, the principal purpose

of ONDCP is to establish policies, priorities, and objectives for the nation’s

drug control program. In conjunction with stakeholders from both the

public and private sectors, ONDCP has proposed a system to assess the

effectiveness of the national drug control strategy to reduce the use of

drugs, their availability, and the damaging consequences associated with

drug use and trafficking. The strategic goals and performance measures

form a framework that shows how the efforts of more than 50 federal drug

control agencies’ programs—along with state, local, and foreign

governments and the private sector—can contribute to the drug control

strategy.

Page 21 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 2 

Coordinate Crosscutting Programs

Transportation’s performance plan stated that the Coast Guard is working

with ONDCP to develop a performance goal and indicator measuring the

rate of illegal drug smuggling in noncommercial maritime routes. Justice’s

performance plan stated it will continue to work with ONDCP and other

agencies in developing and implementing an interagency performance

measurement system to help gauge the effectiveness of the national drug

control strategy.

10. Describe—briefly or refer to a separate document—planned

coordination strategies.

The more useful performance plans not only identified crosscutting

efforts, they also described how agencies expected to coordinate efforts

with other agencies that have similar responsibilities. When interagency

coordination is difficult because of disparate or even conflicting missions

across agencies, performance plans offer agencies an opportunity to

identify statutory or regulatory options for better aligning disparate

missions and for identifying initiatives to avoid or mitigate conflicts over

strategic and performance goals.

Because progress in coordinating crosscutting programs is still in its

infancy, discussions in performance plans of what is being done can

contribute to the development of a common base for understanding the

strengths and weaknesses of various approaches to coordination. The

discussion in the annual performance plans can describe opportunities for

interagency coordination that are identified in other agencies’ strategic

and performance plans, describe the status of coordination efforts, and

develop interagency strategies to achieve shared outcome goals in the

most efficient and effective manner. As a next step, annual performance

plans could discuss how agencies can cooperate in data collection and

program evaluation.

For example, the Department of Education’s performance plan discussed

how it plans to coordinate its programs that share common results and its

efforts to cooperate with others to collect data and conduct program

evaluations. For its strategic objective that schools are strong, safe,

disciplined, and drug-free, the performance plan indicated that the agency

will continue to pursue joint prevention activities, such as truancy and

hate crimes initiatives, with the Department of Justice. Education intends

to develop a database for the President’s Report Card on School Violence

with the Department of Justice and make maximum use of data from other

federal agencies. Education also plans to continue to cooperate on

Page 22 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 2 

Coordinate Crosscutting Programs

evaluation projects with Justice and Health and Human Services. (See fig.

2.2.)

Figure 2.2: Excerpt From the Department of Education’s Fiscal Year 1999 Annual Performance Plan

Page 23 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 3 

Show How Strategies Will Be Used to

Achieve Goals

The more useful annual performance plans discussed how specific

strategies and programs will contribute to achieving performance goals.

Agencies can also use their performance plans to articulate their strategies

to leverage or mitigate key external factors that will affect program

performance and strategies to address mission-critical management

problems that threaten the achievement of program results. Table 3.1

identifies applied practices that can help agencies show how strategies

will be used to achieve goals.

Table 3.1: Applied Practices to

Address Goal Achievement Strategies

A performance plan that shows how

strategies will be used to achieve goals

includes the following practices:

11. Link strategies and programs to

specific performance goals and describe

how they will contribute to the achievement

of those goals.

12. Describe strategies to leverage or

mitigate the effects of external factors on

the accomplishment of performance goals.

13. Discuss strategies to resolve

mission-critical management problems.

14. Discuss—briefly or refer to a separate

document—plans to ensure that

mission-critical processes and information

systems function properly and are secure.

11. Link strategies and programs to specific performance goals and

describe how they will contribute to the achievement of those

goals.

The more useful annual performance plans linked the agencies’ strategies

and programs to specific performance goals and described how they will

contribute to the achievement of those goals. The listings of current

programs and initiatives that often were included in agencies’ fiscal year

1999 plans were useful for providing an understanding of what agencies

do. However, presentations that more directly explain how programs and

initiatives achieve goals will be most helpful to congressional and other

decisionmakers in assessing the degree to which strategies are appropriate

and reasonable. For example, an agency might state in its performance

plan that it will administer grants to states. In order to link this strategy to

its intended results, the agency should explain how administering grants to

states will help it achieve specific performance goals. Over time and as

agencies gain experience in linking strategies to results, agencies will be in

Page 24 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 3 

Show How Strategies Will Be Used to

Achieve Goals

a better position to understand and discuss alternative strategies and

program design and the relative contributions these alternatives could

make to results. A subsequent discussion of how different governing tools

(for example, intergovernmental partnerships, performance-based

contracts, financial grants and credits, and direct service provision by the

federal government) can be used in achieving goals could further enhance

the plans.

The Department of Agriculture’s Food Safety and Inspection Service

provided a detailed discussion of its strategy to achieve its performance

goal of reducing pathogens on raw meat and poultry products. The

performance plan proposed to expand in-plant inspections and laboratory

testing of meat and poultry samples taken from slaughtering plants.

Laboratory personnel will conduct expanded testing of pathogens of

samples taken from plants slaughtering cattle, swine, chicken, and turkeys

and use upgraded technology to conduct inspections including

stethoscopes, thermometers, and computer software programs that are

not currently available.

The Federal Emergency Management Agency identified in its performance

plan a number of strategies relating to its Project Impact program, which

is designed to promote predisaster mitigation. One performance goal for

this program is to increase the number of Project Impact communities in

each state by at least one. The strategies identified for achieving this goal

included working with states and federal agencies to identify candidate

communities, providing grants as seed funding, providing technical

information, and monitoring progress.

12. Describe strategies to leverage or mitigate the effects of

external factors on the accomplishment of performance goals.

The extent to which results are achieved is influenced by a number of

external factors, such as economic, social, and technological

developments; and the actions of numerous outside entities, including

state and local governments. The Results Act requires agencies to identify

key external factors in their strategic plans that would affect the degree to

which they achieve their strategic goals. The value of performance plans

can be augmented by including discussions on the strategies agencies plan

to take to leverage or mitigate the influence these factors can have on

achieving results. Such discussions can help congressional and other

decisionmakers determine if the agency has the best mix of program

Page 25 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 3 

Show How Strategies Will Be Used to

Achieve Goals

strategies in place to achieve its goals, or if additional agency or

congressional actions are needed to better meet those goals.

The Department of Agriculture’s Food and Nutrition Service discussed

external factors as well as strategies to mitigate conditions. The annual

performance plan included a performance goal to reduce food stamp

trafficking, which is the illegal sale of food stamps for cash. The plan cited

the agency’s dependence on grocery stores to properly handle food stamps

as an external factor that affects the agency’s ability to achieve this goal.

To mitigate this factor, the plan noted that it has promulgated rules that

participating stores must follow and instituted a system of sanctions that

may be applied to stores that violate the rules.

13. Discuss strategies to resolve mission-critical management

problems.

Performance plans that contain specific strategies to resolve

mission-critical management problems provide congressional and other

decisionmakers with an understanding of how the agency plans to

improve its management. By discussing such activities, plans also provide

a basis for accountability. As mentioned earlier, agencies should set goals

to address these management problems, such as those identified by the

administration as a priority management objective in the governmentwide

performance plan, the agency’s financial audit, and other agency

assessments by external organizations, including our performance and

accountability and high-risk series. Annual performance plans can lay out

concrete actions the agency plans to resolve mission-critical management

problems that threaten the achievement of major program goals.

As a first step, Commerce’s performance plan identified major

management challenges raised by the Inspector General, such as the

department’s financial management system, which is also included in our

performance and accountability series. The plan stated that the system,

the Commerce Administrative Management System, is substantially over

its original estimated cost and continues to experience performance

shortfalls and schedule delays. The plan identified actions the department

will take to reduce expenditures and impose greater control over the

project’s direction, such as implementing the system as a pilot at the

Census Bureau.

Page 26 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 3 

Show How Strategies Will Be Used to

Achieve Goals

14. Discuss—briefly or refer to a separate document—plans to

ensure that mission-critical processes and information systems

function properly and are secure.

By including information technology issues in annual performance plans,

agencies underscore the critical role that technology plays in achieving

results. Key information technology issues include the ability to ensure

that mission-critical information systems will function properly after the

year 1999 and are secure from intrusion and misuse—two of the issues we

have identified to be high risk for the federal government.1

 Year

2000-induced failures of one or more mission-critical systems may have a

severe impact on agencies’ abilities to deliver critical services. Agencies

can use their fiscal year 2000 performance plans, which become effective

in October 1999, to discuss their final business continuity and contingency

plans to reduce the risk of Year 2000 business failures.

As a start, the Social Security Administration reported in its performance

plan that it expects that by January 1999 all of its mission-critical systems

will have the ability to operate properly in 2000. It reported that it is also

preparing a Year 2000 contingency plan that addresses how core business

processes will be supported if planned conversion activities experience

unforeseen disruptions.

Despite the sensitivity and criticality of federal information systems, they

are not being adequately protected from unauthorized access.2

 Although

advances in the use of electronic data promise to streamline federal

operations and improve the delivery of federal services, security

weaknesses can create serious pervasive risks for the federal government,

such as potential disclosure of sensitive data, loss of assets worth billions

of dollars due to fraud, and disruption of critical operations. The annual

performance plan could discuss the agency’s ability to ensure the integrity

and availability of information resources supporting mission-critical

operations and the confidentiality of sensitive information.

As a first step, the Department of Transportation’s fiscal year 1999

performance plan stated that a key component of continued security of its

information system is the education of the workforce. By the end of fiscal

year 1998, the plan stated that the agency will develop information systems

security training for various elements of the workforce. A fiscal year 1999

1

High-Risk Series: An Update (GAO/HR-99-1, Jan. 1999).

2

Information Security: Serious Weaknesses Place Critical Federal Operations and Assets at Risk

(GAO/AIMD-98-92, Sept. 23, 1998).

Page 27 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 3 

Show How Strategies Will Be Used to

Achieve Goals

milestone is that 100 percent of senior management, 75 percent of system

administrators, and 60 percent of end users will be trained.

In its performance plan, the Office of Personnel Management discussed

plans to enhance its information security program by conducting internal

and external evaluations of its systems, such as engaging the assistance of

the National Security Agency to review its security capabilities and

implementing appropriate recommendations to improve its security. In

addition to training staff, the agency will have in place a tested disaster

recovery capability for general support and major financial, benefits, and

workforce information application systems.

Page 28 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4 

Show Performance Consequences of Budget

and Other Resource Decisions

The more useful annual performance plans related the performance

consequences of budget and other resource decisions by indicating how

the funding from agencies’ program activities will be allocated to a

discrete set of performance goals. Table 4.1 identifies applied practices

that can help agencies show performance consequences of budget and

other resource decisions.

Table 4.1: Applied Practices to

Address Budget and Resource

Decisions A performance plan that shows

performance consequences of budget

and other resource decisions includes

the following practices:

15. Show how budgetary resources relate

to the achievement of performance goals.

16. Discuss—briefly and refer to the

agency capital plan—how proposed

capital assets (specifically information

technology investments) will contribute to

achieving performance goals.

17. Discuss—briefly or refer to a separate

plan—how the agency will use its human

capital to help achieve performance goals.

15. Show how budgetary resources relate to the achievement of

performance goals.

A key expectation of the Results Act is that Congress will gain a clearer

understanding of what is being achieved in relation to what is being spent.

Therefore, plans are more useful when they convey the requested funding

level associated with achieving a discrete set of performance goals and

identify where that funding was included in the structure of agencies’

budget requests. With such information, decisionmakers can better

compare planned levels of accomplishments with the resources requested.

Decisionmakers can then consider how significant changes from the

requested amounts for these goals could affect performance. Our

forthcoming report on this practice provides additional information on the

variety of approaches agencies used to connect budget resources with

results.

The Results Act gives agencies the flexibility to aggregate, disaggregate, or

consolidate the budget’s program activities so that they align with the

performance goals. If an agency takes one of these approaches, its

performance plan will be more useful if it includes a crosswalk that shows

how the program activities in its budget request relate to the program

activities in its annual performance plan. In addition to this flexibility, an

Page 29 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4 

Show Performance Consequences of Budget

and Other Resource Decisions

agency may propose a change to its budget account or program activity

structures to facilitate an understanding of performance.

The Department of Health and Human Services’ Indian Health Service has

taken the first step in its performance plan to aggregate program activities

for the purposes of performance planning. The plan includes a crosswalk

that aligns its budget activities with its performance in four areas

identified as Results Act (GPRA) aggregated program activities: Treatment;

Prevention; Capital Programming/Infrastructure; and Partnerships,

Consultation, Core Functions, and Advocacy. The crosswalk indicates that

from two separate budget accounts—Services and Facilities—the Indian

Health Service consolidated program activities (6, 7, 8, and 9 from the

Services budget account and 21 and 22 from the Facilities budget account)

in the area of Prevention. The crosswalk also shows that budget program

activities 21 and 22 support GPRA program activities 2, 3, and 4 (Prevention;

Capital Programming/Infrastructure; and Partnerships, Consultation, Core

Functions, and Advocacy). (See fig. 4.1.) The Indian Health Service’s

intended performance in the Prevention area includes such things as

increasing childhood immunization rates, reducing deaths by unintentional

injuries, and reducing childhood obesity.

Page 30 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4 

Show Performance Consequences of Budget

and Other Resource Decisions

Figure 4.1: Excerpt From the Department of Health and Human Services’ Fiscal Year 1999 Annual Performance Plan

Featuring the Indian Health Service

Page 31 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4 

Show Performance Consequences of Budget

and Other Resource Decisions

The Environmental Protection Agency proposed changes to its program

activity structure to better align its budget resources with performance

goals. The agency proposed a uniform program activity structure across all

of its accounts in which each program activity represents one of the

agency’s strategic goals. Using this realigned program activity structure in

its annual plan, the agency showed, by account, the funding it requested in

order to achieve each strategic objective and the supporting annual

performance goals. For example, it requested almost $22 million in fiscal

year 1999 to meet its strategic objective related to the reduction of acid

rain. This request is associated with performance goals relating to

emissions of sulfur dioxide (SO2) and nitrogen oxides (NOx) that react in

the atmosphere and fall to earth as acid rain. The request for the acid rain

strategic objective was related to three accounts from the “clean air”

program activity: Environmental Program and Management, Science and

Technology, and State and Tribal Assistance Grants. (See fig. 4.2.)

Page 32 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4 

Show Performance Consequences of Budget

and Other Resource Decisions

Figure 4.2: Excerpt From the Environmental Protection Agency’s Fiscal Year 1999 Annual Performance Plan

Page 33 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4 

Show Performance Consequences of Budget

and Other Resource Decisions

In addition to aligning budget resources with performance goals and

including trend data and target levels of performance, the Department of

the Treasury’s Internal Revenue Service displayed in its performance plan

the budget amounts that corresponded with the performance actually

achieved in a given fiscal year. Specifically, the agency showed that for

fiscal year 1997, it received budget authority of almost $795 million in its

Submission Processing Activity to achieve a set of performance goals:

Improve Customer Service, Increase Compliance, and Increased

Productivity. The performance level that it achieved in fiscal year 1997 to

Improve Customer Service included, among other things, a timeliness

refund rate of 14.5 days for electronic filings, an improvement over the

rates in previous years of 15.5 days in fiscal year 1996 and 21 days in fiscal

year 1995. (See fig. 4.3.)

Page 34 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4 

Show Performance Consequences of Budget

and Other Resource Decisions

Figure 4.3: Excerpt From the Department of the Treasury’s Fiscal Year 1999 Annual Performance Plan Featuring the

Internal Revenue Service

Page 35 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4 

Show Performance Consequences of Budget

and Other Resource Decisions

16. Discuss—briefly and refer to the agency capital plan—how

proposed capital assets (specifically information technology

investments) will contribute to achieving performance goals.

The federal government’s spending on major physical capital investment is

projected to total over $68 billion in fiscal year 1999. Capital assets include

information technology; equipment; land; structures; and intellectual

property, such as software. With federal agencies facing increasing

demands to improve performance, the importance of making the most

effective capital acquisition choices—choices that are linked to

results—will also intensify. A number of laws are beginning to propel

agencies toward improving their capital decisionmaking practices. For

example, Title V of the Federal Acquisition Streamlining Act of 1994 was

designed to foster the development of measurable cost, schedule, and

performance goals and incentives for acquisition personnel to reach these

goals. The Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of

1996 put in place statutory requirements for agencies to better link their

information technology plans and decisions to their missions and

programmatic goals.

OMB has taken an active leadership role in seeking to link capital

decisionmaking to program results. OMB developed a Capital Programming

Guide that provides agencies with the key elements for producing effective

plans and investments.1 OMB’s Guide drew on our work on best practices

used by leading private sector and state and local governments, which was

subsequently published.2

 Consistent with these best practices, OMB has

required agencies to submit 5-year capital spending plans and

justifications—thus encouraging a longer-term consideration of agency

capital needs and alternatives for addressing them. OMB’s Guide provides a

basic reference on principles and techniques, including appropriate

strategies for analyzing benefits and costs, preparing budget justifications,

and managing capital assets once they are in place. In addition, OMB has

worked closely with the President’s Commission to Study Capital

Budgeting, which is expected to issue its report and recommendations

soon.

In this regard, a more useful performance plan describes the processes,

technologies, and types of resources, including capital, that are needed to

achieve performance goals. Further, performance plans provide agencies

with the opportunity to show how a proposed capital asset will, for

1

Capital Programming Guide, Version 1, July 1997 (Executive Office of the President, OMB).

2

Executive Guide: Leading Practices in Capital Decision-Making (GAO/AIMD-99-32, Dec. 1998).

Page 36 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 4 

Show Performance Consequences of Budget

and Other Resource Decisions

example, decrease costs, improve service quality, or increase productivity.

For example, the Department of the Treasury’s U.S. Customs Service

performance plan explained that continuing the acquisition and

installation of the Land Border automation equipment is needed to allow

inspectors to perform more careful screening and questioning of vehicle

occupants, which should help to achieve Customs’ goal of improving its

efficiency at targeting arriving vehicles for enforcement purposes.

17. Discuss—briefly or refer to a separate plan—how the agency

will use its human capital to help achieve performance goals.

In an era that demands improved performance at reduced costs, an

agency’s success depends upon its ability to assemble the human capital

with the right blend of talents and skills. Thus, human capital planning

must be an integral part of an organization’s strategic and program

planning; human capital itself should be thought of not as a cost to be

minimized but as an asset to be enhanced. To show that the vital

integration of human capital and program planning is in place, the more

useful annual performance plans could discuss the size and

composition—in terms of knowledge, skills, and abilities—of the human

capital needed to support the achievement of performance goals and

explain the workforce planning methods by which these needs were

determined. Also, the more useful plans could describe how strategies—in

such areas as recruitment and hiring, retention and separation, training

and career development, and employee incentives and accountability

systems—could meet workforce needs and support the achievement of

annual performance goals.

The U.S. Agency for International Development stated in its performance

plan that its direct-hire workforce has been trimmed by hiring freezes,

early retirements, and a reduction-in-force in 1996. In the wake of these

events, the plan recognized that the agency must identify critical skills

needed to achieve its goals. Its Office of Human Resources is examining

issues and changes in workforce planning and is focusing, among other

things, on improving basic personnel operations, such as assignments and

employee evaluations.

Page 37 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 5 

Build the Capacity to Gather and Use

Performance Information

In order to successfully measure progress toward intended results,

agencies need to build the capacity to gather and use performance

information. Under the Results Act, agencies are to discuss in their annual

performance plans how they will verify and validate the performance

information that they plan to use to show whether goals are being met.

The usefulness of agencies’ plans could increase if they identify data

sources, describe efforts to verify and validate data, identify actions to

compensate for unavailable data or low-quality data, and discuss

implications of data limitations for assessing performance. Table 5.1

identifies applied practices that can help agencies describe their capacity

to gather and use performance information.

Table 5.1: Applied Practices to

Address Performanceinformation

Capacity A performance plan that describes an

agency’s capacity to gather and use

performance information includes the

following practices:

18. Identify internal and external sources

for data.

19. Describe efforts to verify and validate

performance data.

20. Identify actions to compensate for

unavailable or low-quality data.

21. Discuss implications of data limitations

for assessing performance.

18. Identify internal and external sources for data.

The usefulness of performance data ultimately depends on the degree of

confidence that users have in that data. The more informative annual

performance plans can provide decisionmakers with confidence in the

agency’s ability to report on its performance goals and measures by

identifying existing internal and external data sources. In those instances

when performance data do not exist, a description in the performance

plan of efforts by the agency to identify relevant data sources could inform

decisionmakers of actions the agency is taking in order to measure

performance.

The Department of Justice identified in its performance plan specific

sources of performance data for many of its performance measures.

According to its plan, data are already collected and reported through

existing statistical series and internal Justice data systems, such as the

department’s Bureau of Justice Statistics—the primary source of justice

Page 38 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 5 

Build the Capacity to Gather and Use

Performance Information

statistics on crime, criminal offenders, victims of crime, and the operation

of the justice system at all levels of government.

As another example, the Small Business Administration recognized in its

performance plan its need for data from external sources, such as the

Federal Procurement Data System. This system provides information

relating to the agency’s goal to help increase the share of federal

procurement dollars awarded to small firms. Specifically, the Federal

Procurement Data System provides the agency with data on the

percentage of federal procurement dollars awarded to small businesses,

women-owned businesses, and small disadvantaged businesses.

19. Describe efforts to verify and validate performance data.

Performance plans are to include descriptions of the procedures that will

be used to verify and validate the measured values of actual performance.

The procedures should be credible and specific to ensure that

performance information is sufficiently complete, accurate, and consistent

to document performance and support decisions on how best to manage

programs. Information in the performance plan about the procedures to be

used to verify and validate the data—such as scope, methodology, and

timing—can reinforce an agency’s ability to generate reliable and timely

performance data.

To ensure the accuracy and reliability of reported data, the Department of

Justice stated in its performance plan that it will (1) examine and analyze

data to identify anomalies or inconsistencies, (2) make comparisons to

other data series measuring the same or similar variables, (3) require the

submitting entities to certify the accuracy and completeness of data and to

identify any data limitations, (4) institute appropriate quality control

checks, (5) conduct periodic data audits, and (6) obtain independent

audits of its financial statements. Justice expects that its component

organizations will independently pursue several of these initiatives,

consistent with their individual data requirements.

As part of its efforts to underscore that program managers are to use

performance data to make decisions, the Department of Education has

decided to hold program managers accountable for the quality of the

performance data related to their programs. In its performance plan, the

Department of Education established accountability for data quality with

its program managers as one of its strategies for ensuring high-quality

information. Specifically, program managers will attest that the data used

Page 39 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 5 

Build the Capacity to Gather and Use

Performance Information

for their program’s performance measurement are reliable and valid, or

they will submit plans for data improvement. To validate program

managers’ assertions to the quality of their data, Inspector General audits

and program evaluations will be used. Education will support this strategy

through guidance to managers on developing and monitoring quality data

systems and using data to manage program performance.

In addition to the procedures used to verify and validate performance

data, the use of program evaluations is important for assessing how well a

program is working, identifying the factors affecting performance, and

pinpointing improvement opportunities. In its performance plan, the

Department of Transportation recognized that program evaluation will

help it set meaningful performance goals by quantifying the relationship

between program efforts and levels of outcomes. The plan further

identified specific program evaluations planned for fiscal years 1998 and

1999. The department also noted that it is developing a method to both

plan and track program evaluations, as well as to develop the skills and

resources sufficient to conduct evaluations throughout its organization.

20. Identify actions to compensate for unavailable or low-quality

data.

When data are unavailable or existing data contain significant limitations,

providing credible data may require new information systems or major

changes to existing systems. We recently reported, for example, that some

agencies developed or adopted standardized data collection systems to

obtain data not readily available to them or to fill information gaps.1

However, an agency’s ability to collect and generate data often needs to be

weighed against several factors, such as cost or the burden on

respondents. The more informative annual performance plans provide a

vehicle for discussing where the agency struck such a balance and the

implications for the quality of the data—including accuracy, reliability, and

timeliness—and the cost of collection.

The Department of Education recognized that to effectively report the

performance of the department in achieving its strategic plan goals and

objectives requires developing some new data systems and fixing old ones.

For example, Education stated in its performance plan that its student aid

delivery system has suffered from significant data quality problems. The

plan outlined several steps that Education plans to take to address these

1

Managing for Results: Measuring Program Results That Are Under Limited Federal Control

(GAO/GGD-99-16, Dec. 11, 1998).

Page 40 GAO/GGD/AIMD-99-69 Agency Performance Plans
Section 5 

Build the Capacity to Gather and Use

Performance Information

problems, including improving data accuracy by establishing industrywide

standards for data exchanges, receiving individual student loan data

directly from lenders, expanding efforts to verify data reported to the

National Student Loan Data System, and preparing a systems architecture

for the delivery of federal student aid.

21. Discuss implications of data limitations for assessing

performance.

Performance plans could be more useful if they recognize and identify

significant data limitations and their implications for assessing

performance. Plans can identify known data limitations from internal and

external sources, such as quality or timeliness of the data; indicate what

actions will be taken to compensate for the limitations; and report when

those actions will be taken. The more informative performance plans also

identify implications for decisionmaking arising from data limitations on

reported results.

The Department of Transportation stated in its performance plan that one

of the most significant limitations of both internal and external data is

timeliness. One way the department will deal with this limitation is to

compile preliminary estimates from the portion of data that is available in

time to report on the performance measures. According to the plan,

fatality data from the first 6 months of the year could be compared with

data from the first 6 months of the previous year for an initial performance

measurement.

Page 41 GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix I 

GAO Products on Fiscal Year 1999

Performance Plans

Managing for Results: An Agenda to Improve the Usefulness of Agencies’

Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

The Results Act: Assessment of the Governmentwide Performance Plan

for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept. 8, 1998).

Managing for Results: Measuring Program Results That Are Under Limited

Federal Control (GAO/GGD-99-16, Dec. 11, 1998).

Departments Results Act: Observations on the U.S. Department of Agriculture’s Annual

Performance Plan for Fiscal Year 1999 (GAO/RCED-98-212R, June 11, 1998).

Results Act: Observations on the Department of Commerce’s Annual

Performance Plan for Fiscal Year 1999 (GAO/GGD-98-135R, June 24, 1998).

Results Act: DOD’s Annual Performance Plan for Fiscal Year 1999

(GAO/NSIAD-98-188R, June 5, 1998).

The Results Act: Observations on the Department of Education’s Fiscal

Year 1999 Annual Performance Plan (GAO/HEHS-98-172R, June 8, 1998).

Results Act: Observations on DOE’s Annual Performance Plan for Fiscal

Year 1999 (GAO/RCED-98-194R, May 28, 1998).

The Results Act: Observations on the Department of Health and Human

Services’ Fiscal Year 1999 Annual Performance Plan (GAO/HEHS-98-180R,

June 17, 1998).

Results Act: Observations on the Department of Housing and Urban

Development’s Fiscal Year 1999 Annual Performance Plan

(GAO/RCED-98-159R, June 5, 1998).

Results Act: Department of the Interior’s Annual Performance Plan for

Fiscal Year 1999 (GAO/RCED-98-206R, May 28, 1998).

Observations on the Department of Justice’s Fiscal Year 1999 Performance

Plan (GAO/GGD-98-134R, May 29, 1998).

Results Act: Observations on Labor’s Fiscal Year 1999 Performance Plan

(GAO/HEHS-98-175R, June 4, 1998).

Page 42 GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix I 

GAO Products on Fiscal Year 1999

Performance Plans

The Results Act: Observations on the Department of State’s Fiscal Year

1999 Annual Performance Plan (GAO/NSIAD-98-210R, June 17, 1998).

Results Act: Observations on the Department of Transportation’s Annual

Performance Plan for Fiscal Year 1999 (GAO/RCED-98-180R, May 12, 1998).

Results Act: Observations on Treasury’s Fiscal Year 1999 Annual

Performance Plan (GAO/GGD-98-149, June 30, 1998).

Results Act: Observations on VA’s Fiscal Year 1999 Performance Plan

(GAO/HEHS-98-181R, June 10, 1998).

Independent Agencies Results Act: EPA’s Annual Performance Plan for Fiscal Year 1999

(GAO/RCED-98-166R, Apr. 28, 1998).

Results Act: Observations on the Federal Emergency Management

Agency’s Fiscal Year 1999 Annual Performance Plan (GAO/RCED-98-207R,

June 1, 1998).

Results Act: Observations on the General Services Administration’s Annual

Performance Plan (GAO/GGD-98-110, May 11, 1998).

Managing for Results: Observations on NASA’s Fiscal Year 1999

Performance Plan (GAO/NSIAD-98-181, June 5, 1998).

Results Act: NSF’s Annual Performance Plan for Fiscal Year 1999

(GAO/RCED-98-192R, May 19, 1998).

Results Act: NRC’s Annual Performance Plan for Fiscal Year 1999

(GAO/RCED-98-195R, May 27, 1998).

Results Act: Observations on the Office of Personnel Management’s

Annual Performance Plan (GAO/GGD-98-130, July 28, 1998).

Results Act: Observations on the Small Business Administration’s Fiscal

Year 1999 Annual Performance Plan (GAO/RCED-98-200R, May 28, 1998).

The Results Act: Observations on the Social Security Administration’s

Fiscal Year 1999 Performance Plan (GAO/HEHS-98-178R, June 9, 1998).

Page 43 GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix I 

GAO Products on Fiscal Year 1999

Performance Plans

The Results Act: Observations on USAID’s Fiscal Year 1999 Annual

Performance Plan (GAO/NSIAD-98-194R, June 25, 1998).

Page 44 GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix II 

Related Guidance on Annual Performance

Plans

This is a compilation of related guidance, including the Results Act, OMB

Circular A-11, and GAO reports and is arranged by the sections discussed in

this report.

Section 1: Better

Articulate a Results

Orientation

Government Performance and Results Act (Results Act), 5 U.S.C. 306(c),

31 U.S.C. 1115(a)(1), 1115(a)(2), 1115(a)(4), and 1115(a)(5).

Senate Committee on Governmental Affairs Report accompanying the

Results Act (Senate Report 103-58, June 16, 1993), pp. 15-16, “Performance

Plans”; pp. 27-28, “Federal Government Performance Plan”; p. 28, “Agency

Performance Plans”; p. 29, “Performance Goals”; and pp. 29-30,

“Performance Indicators.”

OMB Circular A-11(1998), secs. 210.4, 220.1, 220.3, 220.4, 220.5(a), 220.5(b),

220.5(c), 220.5(f), 220.6(a), 220.6(b), 220.6(d), 220.7(a), 220.7(b), 220.7(c),

220.7(d), 220.7(f), 220.9(a), 220.9(b), 220.9(c), 220.9(d), 220.9(e), 220.13,

220.15, 220.16, 221.1(a), 221.1(b), 221.1(c), 221.2(a), 221.2(b), 221.2(c),

221.2(d), 221.2(e), and 221.3.

Managing for Results: Measuring Program Results That Are Under Limited

Federal Control (GAO/GGD-99-16, Dec. 11, 1998).

Managing for Results: An Agenda to Improve the Usefulness of Agencies’

Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

The Results Act: Assessment of the Governmentwide Performance Plan

for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept. 8, 1998).

The Results Act: An Evaluator’s Guide to Assessing Agency Annual

Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr. 1998).

Agencies’ Annual Performance Plans Under the Results Act: An

Assessment Guide to Facilitate Congressional Decisionmaking, Version 1

(GAO/GGD/AIMD-10.1.18, Feb. 1998).

The Government Performance and Results Act: 1997 Governmentwide

Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).

Managing for Results: Analytic Challenges in Measuring Performance

(GAO/HEHS/GGD-97-138, May 30, 1997).

Page 45 GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix II 

Related Guidance on Annual Performance

Plans

Measuring Performance: Strengths and Limitations of Research Indicators

(GAO/RCED-97-91, Mar. 21, 1997).

Executive Guide: Effectively Implementing the Government Performance

and Results Act (GAO/GGD-96-118, June 1996).

GPRA Performance Reports (GAO/GGD-96-66R, Feb. 14, 1996).

Implementation of the Government Performance and Results Act (GPRA), A

Report on the Chief Financial Officer’s Role and Other Issues Critical to

the Governmentwide Success of GPRA, Chief Financial Officers Council,

GPRA Implementation Committee, May 1995.

Section 2: Coordinate

Crosscutting

Programs

OMB Circular A-11(1998), secs. 220.7(e) and 220.9(f).

Managing for Results: An Agenda to Improve the Usefulness of Agencies’

Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

The Results Act: Assessment of the Governmentwide Performance Plan

for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept. 8, 1998).

The Results Act: An Evaluator’s Guide to Assessing Agency Annual

Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr. 1998).

Agencies’ Annual Performance Plans Under the Results Act: An

Assessment Guide to Facilitate Congressional Decisionmaking, Version 1

(GAO/GGD/AIMD-10.1.18, Feb. 1998).

Managing for Results: Using the Results Act to Address Mission

Fragmentation and Program Overlap (GAO/AIMD-97-146, Aug. 29, 1997).

The Government Performance and Results Act: 1997 Governmentwide

Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).

Section 3: Show How

Strategies Will Be

Used to Achieve Goals

Results Act, 31 U.S.C. 9703.

Senate Committee on Governmental Affairs Report accompanying Results

Act (Senate Report 103-58, June 16, 1993), pp. 15-16, “Performance Plans”;

pp. 17-18, “Managerial Flexibility Waivers”; and pp. 34-36, “Section 5.

Managerial Accountability and Flexibility.”

Page 46 GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix II 

Related Guidance on Annual Performance

Plans

OMB Circular A-11 (1998), secs. 220.5(d), 220.5(e), 220.9(b), 220.9(d),

220.11(a), 220.11(b), 220.11(c), 220.11(d), and 220.11(e).

Year 2000 Computing Crisis: A Testing Guide (GAO/AIMD-10.1.21, Nov. 1998).

Managing for Results: An Agenda to Improve the Usefulness of Agencies’

Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

The Results Act: Assessment of the Governmentwide Performance Plan

for Fiscal Year 1999 (GAO/AIMD/GGD-98-159, Sept. 8, 1998).

Year 2000 Computing Crisis: Business Continuity and Contingency

Planning (GAO/AIMD-10.1.19, Aug. 1998).

Executive Guide: Information Security Management (GAO/AIMD-98-68,

May 1998).

The Results Act: An Evaluator’s Guide to Assessing Agency Annual

Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr. 1998).

Executive Guide: Measuring Performance and Demonstrating Results of

Information Technology Investments (GAO/AIMD-98-89, Mar. 1998).

Agencies’ Annual Performance Plans Under the Results Act: An

Assessment Guide to Facilitate Congressional Decisionmaking, Version 1

(GAO/GGD/AIMD-10.1.18, Feb. 1998).

Year 2000 Computing Crisis: An Assessment Guide (GAO/AIMD-10.1.14, Sept.

1997).

The Government Performance and Results Act: 1997 Governmentwide

Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).

Managing for Results: Analytic Challenges in Measuring Performance

(GAO/HEHS/GGD-97-138, May 30, 1997).

Business Process Reengineering Assessment Guide, Version 3

(GAO/AIMD-10.1.15, Apr. 1997).

Privatization: Lessons Learned by State and Local Governments

(GAO/GGD-97-48, Mar. 14, 1997).

Page 47 GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix II 

Related Guidance on Annual Performance

Plans

Executive Guide: Effectively Implementing the Government Performance

and Results Act (GAO/GGD-96-118, June 1996).

Section 4: Show

Performance

Consequences of

Budget and Other

Resource Decisions

Results Act, 31 U.S.C. 1115(a)(3) and 1115(c).

Senate Committee on Governmental Affairs Report accompanying Results

Act (Senate Report 103-58, June 16, 1993), pp. 15-16, “Performance Plans”;

pp. 18-19, “Performance Budgeting”; pp. 29-30, “Performance Indicators”;

and p. 31, “Coverage of Program Activities.”

OMB Circular A-11 (1998), secs. 220.1, 220.8(a), 220.8(b), 220.8(c), 220.8(d),

220.8(e), 220.9(d), 220.10(a), 220.10(b), 220.10(c), 220.11(a), 220.11(d),

221.1(a), 221.1(b), 221.1(c), and Part 3.

OMB Capital Programming Guide, v. 1.0 (July 1997).

Executive Guide: Leading Practices in Capital Decision-Making

(GAO/AIMD-99-32, Dec. 1998).

Managing for Results: An Agenda to Improve the Usefulness of Agencies’

Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

Performance Management: Aligning Employee Performance With Agency

Goals at Six Results Act Pilots (GAO/GGD-98-162, Sept. 4, 1998).

The Results Act: An Evaluator’s Guide to Assessing Agency Annual

Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr. 1998).

Acquisition Reform: Implementation of Key Aspects of the Federal

Acquisition Streamlining Act of 1994 (GAO/NSIAD-98-81, Mar. 9, 1998).

Budget Issues: Budgeting for Capital (GAO/T-AIMD-98-99, Mar. 6, 1998).

Executive Guide: Measuring Performance and Demonstrating Results of

Information Technology Investments (GAO/AIMD-98-89, Mar. 1998).

Agencies’ Annual Performance Plans Under the Results Act: An

Assessment Guide to Facilitate Congressional Decisionmaking, Version 1

(GAO/GGD/AIMD-10.1.18, Feb. 1998).

Page 48 GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix II 

Related Guidance on Annual Performance

Plans

The Government Performance and Results Act: 1997 Governmentwide

Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).

Performance Budgeting: Past Initiatives Offer Insights for GPRA

Implementation (GAO/AIMD-97-46, Mar. 27, 1997).

Assessing Risks and Returns: A Guide for Evaluating Federal Agencies’ IT

Investment Decision-making, Version 1 (GAO/AIMD-10.1.13, Feb. 1997).

Information Technology Investment: Agencies Can Improve Performance,

Reduce Costs, and Minimize Risks (GAO/AIMD-96-64, Sept. 30, 1996).

Executive Guide: Effectively Implementing the Government Performance

and Results Act (GAO/GGD-96-118, June 1996).

GPRA Performance Reports (GAO/GGD-96-66R, Feb. 14, 1996).

Transforming the Civil Service: Building the Workforce of the

Future—Results of a GAO-Sponsored Symposium (GAO/GGD-96-35, Dec. 26,

1995).

Integrating Performance Measurement into the Budget Process, Chief

Financial Officers Council, GPRA Implementation Committee

Subcommittee Project, September 22, 1997.

Federal Accounting Standards Advisory Board (FASAB) Volume 1 Original

Statements: Statements of Federal Financial Accounting Concepts and

Standards, Statement of Federal Financial Accounting Standards No. 1,

Objectives of Federal Financial Reporting (GAO/AIMD-21.1.1, Mar. 1997).

FASAB Volume 1 Original Statements: Statements of Federal Financial

Accounting Concepts and Standards, Statement of Federal Financial

Accounting Standards No. 4, Managerial Cost Accounting Standards

(GAO/AIMD-21.1.1, Mar. 1997).

Budget and Financial Management: Progress and Agenda for the Future

(GAO/T-AIMD-96-80, Apr. 23, 1996).

Page 49 GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix II 

Related Guidance on Annual Performance

Plans

Section 5: Build the

Capacity to Gather

and Use Performance

Information

Results Act 31 U.S.C. 1115 (a)(6).

Senate Committee on Governmental Affairs Report accompanying Results

Act (Senate Report 103-58, June 16, 1993), p. 30, “Verification and

Validation.”

OMB Circular A-11(1998), secs. 220.6(a), 220.12(a), and 220.12(b).

Managing for Results: Measuring Program Results That Are Under Limited

Federal Control (GAO/GGD-99-16, Dec. 11, 1998).

Managing for Results: An Agenda to Improve the Usefulness of Agencies’

Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).

The Results Act: An Evaluator’s Guide to Assessing Agency Annual

Performance Plans, Version 1 (GAO/GGD-10.1.20, Apr. 1998).

Agencies’ Annual Performance Plans Under the Results Act: An

Assessment Guide to Facilitate Congressional Decisionmaking, Version 1

(GAO/GGD/AIMD-10.1.18, Feb. 1998).

Managing for Results: Regulatory Agencies Identified Significant Barriers

to Focusing on Results (GAO/GGD-97-83, June 24, 1997).

The Government Performance and Results Act: 1997 Governmentwide

Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997).

Managing for Results: Analytic Challenges in Measuring Performance

(GAO/HEHS/GGD-97-138, May 30, 1997).

Measuring Performance: Strengths and Limitations of Research Indicators

(GAO/RCED-97-91, Mar. 21, 1997).

Executive Guide: Effectively Implementing the Government Performance

and Results Act (GAO/GGD-96-118, June 1996).

GPRA Performance Reports (GAO/GGD-96-66R, Feb. 14, 1996).

Page 50 GAO/GGD/AIMD-99-69 Agency Performance Plans
Appendix III 

Major Contributors

General Government

Division

Alan M. Stapleton, Assistant Director

Lisa R. Shames, Project Manager

Dorothy L. Self, Senior Evaluator

Acknowledgments The examples used in this report are drawn from the assessments of the

individual agency annual performance plans that were done by staff across

GAO. Thus, in addition to the individuals noted above, the staff who worked

on the individual agency plan assessments and summary assessment also

made important contributions to this report. These individuals are

identified in the separate reports on agency fiscal year 1999 plans.

(410311) Page 51 GAO/GGD/AIMD-99-69 Agency Performance Plans
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100