United States General Accounting Office

GAO Report to Congressional Committees

June 1997 THE GOVERNMENT

PERFORMANCE AND

RESULTS ACT

1997 Governmentwide

Implementation Will Be

Uneven

GAO/GGD-97-109

GAO United States

General Accounting Office

Washington, D.C. 20548

General Government Division

B-277071 

June 2, 1997

The Honorable Fred Thompson

Chairman

The Honorable John Glenn

Ranking Minority Member

Committee on Governmental Affairs

United States Senate

The Honorable Ted Stevens

Chairman

The Honorable Robert C. Byrd

Ranking Minority Member

Committee on Appropriations

United States Senate

The Honorable Pete V. Domenici

Chairman

The Honorable Frank R. Lautenberg

Ranking Minority Member

Committee on Budget

United States Senate

The Honorable Dan Burton

Chairman

The Honorable Henry A. Waxman

Ranking Minority Member

Committee on Government Reform

 and Oversight

House of Representatives

The Honorable Bob Livingston

Chairman

The Honorable David Obey

Ranking Minority Member

Committee on Appropriations

House of Representatives
B-277071 

The Honorable John R. Kasich

Chairman

The Honorable John M. Spratt, Jr.

Ranking Minority Member

Committee on Budget

House of Representatives

This report responds to the requirement in the Government Performance and Results Act of

1993 that we report to Congress on the implementation of the Act, including prospects for

compliance by federal agencies beyond those participating as pilots under the Act. Our specific

objectives for this report were to (1) assess the status of the Act’s implementation efforts;

(2) identify significant challenges confronting executive agencies in their efforts to become

more results-oriented; and (3) describe ongoing efforts to integrate program, cost, and budget

information into a reporting framework that allows for fuller consideration of resource

allocations, operational costs, and performance results.

We are sending copies of this report to other appropriate congressional committees and

executive branch agencies, including the Director of the Office of Management and Budget and

other interested parties. We also will make copies available to others on request.

If you or your staff have any questions concerning this report, I can be reached at

(202) 512-2637. The major contributors to this report are listed in appendix II.

L. Nye Stevens

Director, Federal Management

 and Workforce Issues

Page 2 GAO/GGD-97-109 Results Act
Executive Summary

Purpose A consensus that the nation’s persistent federal deficit must be addressed

and that the effectiveness of federal programs must improve substantially

has spurred widespread efforts in the executive branch and Congress to

dramatically change the way the federal government is managed. During

the 1990s, Congress enacted a broad statutory framework, with the

Government Performance and Results Act of 1993 as its centerpiece, to

improve the management and accountability of federal agencies. In

essence, the Act, which is referred to as “GPRA” or “the Results Act,” seeks

to shift the focus of federal management and decisionmaking away from a

preoccupation with the activities that are undertaken to a focus on the

results of those activities as reflected in citizens’ lives.

Congress understood that the management changes required to effectively

implement the Results Act would not come quickly or easily. The Act

therefore included a phased implementation approach that began in fiscal

year 1994 with pilot projects on the Act’s performance planning and

reporting requirements. Under the Results Act, GAO is to report to

Congress on the implementation of the Act, including the prospects for

compliance by executive agencies beyond those that participated in the

pilot phase. This report culminates GAO’s efforts to meet that mandate.

GAO’s specific objectives for this report were to (1) assess the status of the

Results Act’s implementation efforts; (2) identify significant challenges

confronting executive agencies in their efforts to become more results

oriented; and (3) describe ongoing efforts to integrate program, cost, and

budget information into a reporting framework that allows for fuller

consideration of resource allocations, operational costs, and performance

results.

Background The Results Act is intended to improve the efficiency and effectiveness of

federal programs by establishing a system to set goals for program

performance and to measure results. Specifically, the Act requires

executive agencies to prepare multiyear strategic plans, annual

performance plans, and annual performance reports. As a starting point,

the Act requires virtually every executive agency to develop a strategic

plan, covering a period of at least 5 years forward from the fiscal year in

which it is submitted. These strategic plans are to include an agency’s

mission statement, general goals and objectives, and the strategies that the

agency will use to achieve those goals and objectives. Agencies are to

consult with Congress and solicit the views of other stakeholders in the

development of those plans and to submit the first strategic plans to the

Page 3 GAO/GGD-97-109 Results Act
Executive Summary

Office of Management and Budget (OMB) and Congress by September 30,

1997.

Next, the Results Act requires executive agencies to develop annual

performance plans covering each program activity set forth in the

agencies’ budgets. The first annual performance plans, covering fiscal year

1999, are to be provided to Congress after the President’s budget is

submitted to Congress in 1998. The annual performance plan is to contain

an agency’s annual goals, the measures that the agency will use to gauge

its performance toward meeting those goals, and the resources the agency

will need to meet its goals. Finally, the Results Act requires executive

agencies to prepare annual reports on program performance for the

previous fiscal year. The performance reports are to be issued by March 31

each year, with the first (for fiscal year 1999) to be issued by March 31,

2000. In each report, an agency is to compare its performance against its

goals, summarize the findings of program evaluations completed during

the year, and describe the actions needed to address any unmet goals.

The Results Act established requirements for pilot projects so that

agencies could gain experience in using key provisions of the Act and

provide lessons for pilots and for other agencies. The Act required OMB to

designate at least 10 agencies to participate in the first set of pilot projects

during fiscal years 1994 through 1996, which focused on the preparation of

annual performance plans and reports. A second set of pilot projects in

managerial accountability and flexibility were to propose waivers of

administrative procedural requirements and controls for one or more of

the major functions and operations of the agency to determine the effects

of providing federal managers with increased managerial flexibility in

exchange for greater accountability for performance. OMB was to designate

at least five agencies from the first set of pilot projects to test managerial

accountability and flexibility during fiscal years 1995 and 1996. A third set

of pilot projects, which are scheduled in the Act for fiscal years 1998 and

1999, are to test performance budgeting—i.e., the presentation of the

varying levels of performance that would result from different budget

levels.

To prepare this report, GAO drew on a large body of work it had done in

recent years on the Results Act and on related goal-setting, performance

measurement, and accountability concepts. GAO also surveyed a sample of

civilian managers at the general schedule (GS) and general management

(GM) levels, GS/GM-13, through Senior Executive Service (SES) levels in 24

major agencies. These 24 agencies accounted for over 99 percent of the

Page 4 GAO/GGD-97-109 Results Act
Executive Summary

federal government’s net outlay for fiscal year 1996. The sample was

stratified by whether the manager was SES or non-SES and by whether the

manager was working in a selected agency or agency component that was

designated as a Results Act pilot. Of the approximately 1,300 managers GAO

surveyed, it received usable responses from about 72 percent.

The survey results are statistically generalizable to the 24 agencies

included in the survey. The survey data reported throughout this report

are the estimated percentages of how officials would have responded had

the entire universe of eligible officials been surveyed. In general,

percentages reported for the entire sample have confidence intervals

ranging from + 5 percentage points to + 12 percentage points. In other

words, if all managers and supervisors in the 24 agencies included in GAO’s

population had been surveyed, the chances are 95 out of 100 that the

results obtained would not differ from the sample estimate, in the most

extreme case, by more than + 12 percentage points.

Results in Brief GAO’s work shows that the Results Act’s implementation to this point has

achieved mixed results, which will lead to highly uneven governmentwide

implementation in the fall of 1997. While agencies are likely to meet the

upcoming statutory deadlines for producing initial strategic plans and

annual performance plans, GAO found that those documents will not be of

a consistently high quality or as useful for congressional and agency

decisionmaking as they could be. On a more promising note, OMB selected

over 70 performance planning and reporting pilots that far exceeded the

number required by the Results Act and that should provide a rich body of

experience for agencies to draw on in the future.

GAO also found that the experiences of some of the Results Act pilot

agencies, such as the Social Security Administration (SSA), and related

efforts by nonpilot agencies, such as the Veterans Health Administration,

showed that significant performance improvements were possible—even

in the short term—when an agency adopted a disciplined approach to

setting results-oriented goals, measuring its performance, and using

performance information to improve effectiveness. However, the reported

examples of substantial performance improvements were relatively few,

and many agencies did not appear to be well positioned to provide in 1997

a results-oriented answer to the fundamental Results Act question: What

are we accomplishing?

Page 5 GAO/GGD-97-109 Results Act
Executive Summary

GAO’s review of efforts to date under the Results Act have shown that to

effectively implement the Act, agencies face a variety of significant

challenges, some of which will not be resolved quickly. One set of

challenges arises from the complications of government structure and

from program proliferation. Others involve methodological difficulties in

identifying performance measures or the lack of data needed to establish

goals and assess performance. The following are among the challenges

that GAO observed.

• Overlapping and fragmented crosscutting program efforts, such as student

loan or economic development programs, present the logical need to

coordinate efforts to ensure that goals are consistent and, as appropriate,

that program efforts are mutually reinforcing. GAO found that overlapping

and fragmented program efforts can undermine efforts to establish clear

missions and goals. In addition to the problem of overlapping and

fragmented programs, agencies are challenged in setting goals because

those goals often must reflect a balance of competing policy priorities. For

example, GAO has reported that the effectiveness of the Forest Service had

been compromised because it had not reconciled conflicts concerning its

role in regulating the use of its lands (e.g., promoting timber sales versus

protecting wildlife). Striking the right balance is a continuing and difficult

challenge because forging the political consensus needed to create and

sustain a program often results in that program having competing and/or

broadly stated goals.

• The often limited or indirect influence that the federal government has in

determining whether a desired result is achieved complicates the effort to

identify and measure the discrete contribution of the federal initiative to a

specific program result. GAO work has shown that measuring the federal

contribution is particularly challenging for regulatory programs; scientific

research programs; and programs that deliver services to taxpayers

through third parties, such as state and local governments.

• The lack of results-oriented performance information in many agencies

hampers efforts to identify appropriate goals and confidently assess

performance. Even when data exist, GAO has consistently found that the

quality of agencies’ performance data is often questionable due to several

factors, including the need to rely on third parties to provide data.

• Instilling within agencies an organizational culture that focuses on results

remains a work in progress across the federal government. According to

GAO’s survey, about 57 percent of managers said that their agencies’ top

leadership demonstrated a strong commitment to achieving results to a

great or very great extent. In contrast, however, only about 16 percent said

that program changes by management above their levels were based on

Page 6 GAO/GGD-97-109 Results Act
Executive Summary

results-oriented performance information to a great or very great extent.

This suggests that despite the strength of managers’ perceptions about top

leadership commitment, the extent to which leaders are demonstrating

their commitment by using performance information for decisionmaking is

not as great.

• Linking agencies’ performance plans directly to the budget process,

through the Results Act requirement to base the annual program

performance goals on the budget’s program activity structure, may present

significant difficulties. GAO’s analysis has shown that the extent to which

the budget’s program activity structure can be directly linked to a

results-oriented performance framework varies widely among activities,

and GAO’s work has suggested that adjustments and accommodations in

the program activity structure may be needed. GAO’s work also has

suggested that reaching agreement on such changes between Congress

and the executive branch will be a time-consuming and difficult process

that will take more than one budget cycle to resolve.

The Results Act is the cornerstone of a series of initiatives that are

intended to provide a comprehensive framework for integrating program,

cost, and budget information. GAO notes that improved financial reporting

and auditing required by the Chief Financial Officers Act (CFO) Act should

strengthen the reliability of cost and performance information. GAO also

notes that the Federal Accounting Standards Advisory Board (FASAB) has

developed a new set of reporting concepts and accounting standards that

underpin OMB’s guidance to agencies on the form and content of their

agencywide financial statements. These standards include a new reporting

model for federal agencies geared to providing users with consolidated

performance, cost, and other information. These standards also include

cost accounting standards that became effective beginning with fiscal year

1997 and are the first set of standards that are to account for the full costs

of federal programs. GAO work has shown that developing the necessary

information systems to gather, process, and analyze the needed program

and cost information will be a substantial undertaking for most federal

agencies because of agencies’ long-standing weaknesses in the use of

information technology. To help address problems in agencies’ use of

information technology, Congress enacted the Paperwork Reduction Act

of 1995 and the Clinger-Cohen Act of 1996.

Addressing some of the challenges highlighted by GAO, such as

crosscutting program efforts and balances among competing priorities,

will raise significant policy issues for Congress and the administration to

consider, some of which will likely be very difficult to resolve. In this

Page 7 GAO/GGD-97-109 Results Act
Executive Summary

regard, GAO believes that the Results Act’s success or failure should not be

judged on whether contentious policy issues are fully resolved; rather,

judgment of the success or failure of the Results Act should turn on the

extent to which the information produced through the Results Act’s

goal-setting and performance measurement practices—once those

practices are successfully implemented—helps inform policy decisions

and improve program management.

Principal Findings

Performance

Improvements Have

Been Made, but

Overall Results Act

Implementation

Efforts Are Mixed

Executive agencies, OMB, and Congress are undertaking efforts to

implement the Results Act. Over 70 federal organizations—in some cases,

entire agencies such as SSA and the Internal Revenue Service—participated

in the performance planning and reporting pilot phase. GAO’s work

indicated that in addition to overseeing those pilots, OMB has achieved

some success working with agencies to seek agreement on strategic and

annual goals and performance measures and to increase the prominence

of performance information in the executive branch budget process. For

example, when GAO reviewed fiscal year 1996 OMB budget documents, it

found that more attention had been placed on performance-related issues

than in the previous fiscal year. Congress, too, has shown a growing

interest in and support for the governmentwide implementation of the

Results Act. For example, the House Majority has established teams

consisting of staff from various committees to lead its strategic plan

consultation efforts. These teams have been reaching out to agencies to

review and comment on agencies’ strategic plans.

Most important, GAO found some agencies have reported improved results

due to the Results Act or related efforts. For example, GAO has reported

that the Veterans Health Administration improved services to veterans by

more rigorously assessing the results of the medical care it provides to the

nation’s veterans. In particular, the Veterans Health Administration

reported that it used performance information to target the most

important improvement opportunities and thereby lowered the mortality

rate for cardiac procedures by an average of 13 percent over the last 8

years. In another example, involving SSA’s national toll-free 800 telephone

number to handle citizen inquiries, SSA used customer satisfaction and

other performance information to identify and make program changes,

including providing additional staff to handle phone calls. As a result, the

Page 8 GAO/GGD-97-109 Results Act
Executive Summary

busy rate decreased from 49 to 34 percent, and the percentage of calls

answered within 5 minutes increased from 74 to 83 percent from fiscal

year 1995 to fiscal year 1996. GAO notes that these and other performance

improvements strongly suggest that the basic goal-setting and

performance measurement model used by the Results Act, if successfully

implemented, will be an important tool for improving federal management

and performance.

Although some agencies reported that significant performance

improvements have been made, GAO noted that, in general, few agencies

have shown substantial performance improvements thus far. These limited

reports of improvements may be understandable at this early stage of the

Results Act’s implementation. However, GAO’s work also indicates that

agencies do not appear to have consistently in place the critical

performance measurement processes that would allow them to make

future improvements. For example, GAO reported in January 1997 that the

Department of Housing and Urban Development’s Public Housing

Management Assessment Program did not collect important information

needed to manage and assess its results. The program is to assess the

performance of local housing authorities by measuring factors such as the

numbers of outstanding work orders and uncollected rents. However, the

system does not measure other factors, such as housing quality, that are

essential for assessing the results that housing authorities are achieving, as

well as for determining which housing authorities are performing well or

poorly.

The situation at the Department of Housing and Urban Development

appears to be typical. According to GAO’s survey, fewer than one-third of

managers in the agencies included in its survey reported that

results-oriented performance measures exist for their programs to a great

or very great extent. The existence of other types of performance

measures needed to successfully manage programs also was reported as

low. For example, of the total managers reporting the existence of such

measures to a great or very great extent, 38 percent of the managers

reported the existence of measures of outputs, 32 percent reported the

existence of customer satisfaction measures, 31 percent reported the

existence of measures of product or service quality, and 26 percent

reported the existence of measures of efficiency.

On a more positive note, while still viewed as low, significantly more

managers reported in GAO’s survey that results-oriented and other

performance measures existed to a greater extent currently than 3 years

Page 9 GAO/GGD-97-109 Results Act
Executive Summary

ago. For example, GAO asked managers about the current situation as

compared with what they recollected 3 years ago concerning the existence

of results-oriented performance measures and measures of customer

satisfaction. The percentage of managers who reported the existence of

these measures to a great or very great extent was higher currently by 13

and 21 percentage points, respectively. Although this increase is a very

favorable development, GAO notes that the overall low level still is a matter

for concern.

Obviously, it is not sufficient merely to measure current performance. The

Results Act envisions that performance information will be used to make

decisions and better manage programs. In that regard, about one-fifth of

federal managers from agencies included in GAO’s survey reported that

results-oriented performance information was used to a great or very great

extent to develop their agencies’ budgets. Managers reported even lower

levels in the use of results-oriented performance information to help make

funding decisions, legislative changes, or program changes. However,

managers rated the extent of use of results-oriented performance

information currently as somewhat higher than they rated the extent of

use 3 years ago. The reported limited availability of results-oriented

performance measures, combined with the still infrequent use of such

information to help make key decisions, suggests to GAO that many

agencies may have a difficult time developing meaningful results-oriented

measures in time for fiscal year 1999 performance plans that are due to

OMB this fall.

Key Challenges

Remain to Effective

Implementation of the

Results Act

GAO found that agencies are confronting a variety of difficult challenges

that have limited the implementation of the Results Act. These challenges

include establishing clear agency missions and strategic goals, especially

when program efforts are overlapping or fragmented; measuring

performance, particularly when the federal contribution to a result is

difficult to determine; generating the results-oriented performance

information needed to set goals and assess progress; instilling a

results-oriented organizational culture within agencies; and linking

performance plans to the budget process. The experiences of pilot

agencies and related efforts by nonpilot agencies suggest that these

challenges will not be quickly or easily resolved.

Establishing Clear Agency

Missions and Strategic

Goals Remains a Challenge

Attempting to reach a reasonable degree of consensus on agencies’

missions and establishing clear and precise strategic goals are likely to

raise contentious policy issues that may not be easily resolved. Past GAO

Page 10 GAO/GGD-97-109 Results Act
Executive Summary

work has shown that as Congress and the executive branch have

responded to new national needs and problems, many different agencies

have been given responsibility for addressing the same or similar national

issues. Of the 18 national mission areas displayed in the federal budget, 14

were addressed by more than 1 executive branch department or major

agency in fiscal year 1996. In some cases, this shared responsibility

demonstrates that addressing a national need requires the efforts of more

than one agency.

However, past GAO work has suggested that some important federal

program areas have suffered from overlap and fragmentation. Overlapping

and fragmented program efforts can frustrate program customers, waste

scarce resources, and limit the overall effectiveness of the federal effort.

For example, GAO reported in 1995 on the Department of Education

programs that provided loans and grants to students to help finance their

higher education. GAO found that although the student loan and Pell grant

programs provided the majority of federal financial aid to students for

postsecondary education, another 22 smaller programs were targeted to

specific segments of the postsecondary school population, such as

prospective students from disadvantaged families or women and

minorities who are underrepresented in graduate education. These 22

programs were collectively funded at $1.1 billion for fiscal year 1995. GAO

concluded that these smaller grant programs could be considered

candidates for consolidation—with other larger programs or among

themselves—with no adverse impact on students’ access to postsecondary

education. GAO also found that the federal government could anticipate

administrative savings of 10 percent each year, or a total of $550 million in

budget authority (adjusted for inflation) over 5 years.

In addition, GAO notes that many agencies confront competing policy

objectives, which are a natural by-product of the complex social and

political environment in which programs are created, funded, and

managed. For example, GAO reported in April 1997 that the Forest Service

had increasingly shifted the emphasis of its efforts from producing timber

to sustaining wildlife. This shift was taking place in reaction to

requirements in planning and environmental laws and their judicial

interpretation—reflecting changing public values and concerns—together

with social, ecological, and other factors. However, GAO noted that the

demand for recreation was also expected to grow and may increasingly

conflict with efforts aimed at sustaining wildlife and producing timber. GAO

found that the disagreement both within the Forest Service and among key

external stakeholders, including Congress, on how the Forest Service is to

Page 11 GAO/GGD-97-109 Results Act
Executive Summary

resolve conflicts or make choices among competing uses on its lands had

seriously undermined its efforts to establish the goals and performance

measures needed to ensure accountability. GAO concluded that until

general agreement is reached, the Forest Service’s decisionmaking will

continue to be inefficient and ineffective.

Lack of Federal Control

Over Results Complicates

Performance Measurement

Efforts

GAO’s past work shows that agencies must grapple with some difficult

analytic and technical challenges in developing results-oriented

performance measures. The variety of influences beyond direct federal

control that may determine whether, and the degree to which, a federal

effort is successful is a major challenge to agencies’ efforts to measure

performance. Moreover, the end result of a program may occur long after

the federal intervention, which makes it difficult to measure progress

annually. Because of issues like these, performance measurement efforts,

such as those for scientific research, regulatory, and intergovernmental

programs, are particularly challenging.

For example, GAO found that determining the impact of economic

development programs has been a daunting task because of the numerous

external forces—including broad national economic trends and the

assistance that communities may receive from state and local

governments and the private sector—that may contribute to local

economic development. Separating out the effects of federal program

efforts can be extremely difficult, as GAO observed in a 1996 review on

economic development programs, because it would require, first,

documentation that there had been some improvement in a targeted area;

second, linkage of specific program elements to actual economic changes;

and third, measurement of the growth stemming from other influences on

the economy of the targeted area in order to isolate the impact that could

be attributed to the economic development program.

GAO found that although such performance measurement efforts constitute

a substantial undertaking, they have the potential to provide important

information for decisionmakers. For example, one study of the

effectiveness of the Department of Commerce’s Economic Development

Administration (EDA) programs found that income in the counties that

received EDA funding grew significantly faster than income in the counties

that received no aid. However, when EDA’s programs and factors unrelated

to EDA were considered simultaneously, the study found that EDA’s

programs had a very small effect on income growth rates during the period

that the aid was received and had no significant effect in the 3 years after

Page 12 GAO/GGD-97-109 Results Act
Executive Summary

the aid ceased. The study found that EDA’s programs could explain only a

small part of the difference in these growth rates between the two groups

of counties.

GAO found that agencies are exploring a number of approaches for

addressing difficulties in developing results-oriented performance

measures. These approaches include undertaking program evaluations to

try to isolate program impact, such as in the EDA case already discussed;

using measures of intermediate results; employing a range of measures to

assess whether programs are making progress; and working with

stakeholders to seek agreement on appropriate measures.

Absence of Quality

Results-Oriented

Performance Information

Hampers Efforts to Set

Goals and Assess

Performance

GAO has found that a lack of results-oriented performance information to

use as a baseline complicates agencies’ efforts to set appropriate

improvement targets. Department of Veterans Affairs officials told GAO

that some of their results-oriented measures for a Loan Guaranty program

were new and that baseline data were not available on those measures.

Consequently, they did not have data on past performance to use in setting

some of the program’s fiscal year 1998 goals. In some of these cases, the

Department indicated in its fiscal year 1998 budget submission that those

goals were “to be determined.” In another example, Department of

Agriculture officials told GAO that they eliminated some performance

measures that had been part of their Results Act pilot’s annual

performance plan because they did not have a way to collect data on those

measures. Lacking these data, they did not have an informed basis on

which to set goals.

Past GAO work also has shown that agencies face a variety of challenges to

collecting valid and reliable performance information, including ensuring

that standard definitions are used when measuring performance. For

example, GAO reported in 1996 on management challenges that the

Department of Health and Human Service’s Office of Child Support

Enforcement (OCSE) faced because of a lack of comparable performance

data across state and local jurisdictions. OCSE officials said that

discrepancies resulting from differences in the way the states and local

jurisdictions defined what constitutes a child support enforcement case

had contributed to the difficulty of uniformly measuring state

performance. To address these discrepancies, OCSE worked with state and

local authorities to develop standard data definitions for key child support

enforcement terms, including a definition for what constitutes a child

support enforcement case, and incorporated the use of standardized

Page 13 GAO/GGD-97-109 Results Act
Executive Summary

definitions for measuring state performance. Together, they also

developed measures to assess state performance in obtaining support

orders and agreed that states would use the OCSE-established definition of

a child support enforcement case to report these data.

Making Needed Cultural

Changes Is a Work in

Progress Across Agencies

According to GAO’s survey, federal managers rated the commitments of top

leadership to achieving results as higher currently than they did for 3 years

ago. However, federal managers’ responses to GAO’s survey also suggested

that more progress is needed for agencies to develop and sustain cultures

that focus on results. For example, when GAO asked federal managers

about the extent to which they or supervisors at their levels had the

authority they needed to help their agencies accomplish their strategic

goals, the managers did not perceive that they had more such authority

currently than they recalled having 3 years ago.

Significantly, for managers from the Results Act pilots that GAO was able to

isolate for its sample, GAO’s survey found these managers’ perception of

the extent of their authority currently was much lower than their

perception of the situation 3 years ago. For example, 56 percent of SES

managers from selected Results Act pilots reported that managers at their

level had authority to help the agency accomplish its strategic goals to a

great or very great extent 3 years ago. Fewer SES managers, 40 percent,

reported having authority to a comparable extent currently—a difference

of 16 percentage points.

These survey results suggest to GAO that as agencies implement the Results

Act and strive to become more results oriented, they need to pay special

attention to ensuring that key managers have the authority they need to

achieve intended results. In passing the Results Act, Congress recognized

that if federal managers were to be held accountable for program results,

they would need the authority and flexibility to achieve those results.

Congress also understood the importance of affording federal program

managers the freedom to be innovative and creative and to marshal

resources to achieve results. Indeed, Congress has provided agencies with

additional authority in the key area of procurement that allows for more

flexibility in managing their programs. In addition, the administration’s

National Performance Review has sought the empowerment of employees

to achieve results as one of its major initiatives.

Page 14 GAO/GGD-97-109 Results Act
Executive Summary

Integrating Performance

Information Into the

Budget Will Likely Prove

Difficult

The annual performance plans that agencies are to develop under the

Results Act provide the opportunity, if successfully implemented, to assist

congressional and executive branch resource allocation decisionmaking

by providing direct linkages between the performance information

contained in performance plans and agency budget requests. Under the

Results Act, the annual performance goals are to be based on the program

activities listed in the president’s budget. GAO has observed that although

budgeting is inherently an exercise in political choice, the Results Act is

based on the premise that budget decisions should be more clearly linked

to expectations about program performance.

In a 1997 report on past federal initiatives in performance budgeting, GAO

noted that an agency’s program activities generally result from

negotiations between the agency, OMB, and the relevant congressional

appropriations subcommittees. Program activity structures, which

represent programmatic, process, organizational, or other orientations,

therefore depend on the needs, interests, and experiences of agencies,

OMB, and Congress. Because of this wide variability, the suitability of

current program activity structures for the Results Act performance

planning and measurement may vary.

GAO’s discussions with agency officials and congressional staff as part of

its review of past performance budgeting initiatives highlighted the

differing expectations held by the two branches of government on the use

of the budget’s program activity structure as a basis for planning and

measuring agencies’ performance. Congressional staff were generally

comfortable with existing activity structures and questioned whether

changes would frustrate congressional oversight. The staff generally

viewed these structures as fundamental to congressional oversight of

agency activities, and thus they viewed changes with apprehension and

concern. However, some agency officials saw program activity structures

as secondary to planning. Therefore, where current program activity

structures proved unsuitable for planning purposes, these officials viewed

change in program activity structures as inevitable and appropriate.

However, they noted that negotiating changes with Congress could prove

difficult and time-consuming.

Page 15 GAO/GGD-97-109 Results Act
Executive Summary

An Augmented

Reporting Framework

Holds Promise for

Strengthening Federal

Decisionmaking and

Accountability

GAO’s past work indicates that integrated accountability reports have the

potential of further strengthening congressional and executive branch

decisionmaking and agencies’ accountability. The FASAB accountability

reporting concepts and accounting standards—building on CFO Act

requirements—are to bring together program performance information

with audited financial information to provide congressional and other

decisionmakers with a more complete picture of the results, operational

performance, and the costs of agencies’ operations. For the first time,

decisionmakers are to be provided with annual “report cards” on the costs,

management, and effectiveness of federal agencies. The FASAB cost

accounting standards hold a similar promise to improve decisionmaking, if

successfully implemented. These standards are to provide decisionmakers

with information on the costs of all resources used, including the costs of

services provided by others to support activities or programs. Such

information would allow for comparisons of the costs of various programs

and activities with their performance outputs and results.

Congress, in enacting the Results Act, the CFO Act, and information

technology reform legislation, sought to create a more focused,

results-oriented management and decisionmaking process within both

Congress and the executive branch. GAO has noted that these laws were

intended to respond to a need for accurate, reliable information for

congressional and executive branch decisionmaking, information that had

been badly lacking. GAO observed that implemented together, these laws

provide a powerful framework for developing fully integrated information

about agencies’ missions and strategic priorities, results-oriented

performance goals that flow from those priorities, performance data to

show the achievement (or not) of those goals, the relationship of

information technology investments to the achievement of performance

goals, and accurate and audited financial information about the costs of

achieving mission results.

Recommendations GAO has long supported the goal-setting, performance measurement, and

results-based accountability concepts embodied in the Results Act. As part

of GAO’s statutory mandate under the Results Act, and at the request of

congressional committees, GAO monitored the performance planning and

reporting pilot phase of the Results Act, highlighted the key

implementation issues needing additional attention as those issues were

identified, and made suggestions to Congress and agencies as appropriate.

Moreover, the GAO reports that examined management issues in individual

agencies contained recommendations, as appropriate, to improve the

Page 16 GAO/GGD-97-109 Results Act
Executive Summary

management of those agencies. Overall, GAO found nothing in the work

that it did specifically for this report that would lead it to alter its position

in support of the concepts in the Results Act or the recommendations it

has made to specific agencies. Therefore, GAO is making no new

recommendations in this report.

Agency Comments GAO sent a draft of this report to the Director of OMB for comments. On

May 21 and 22, 1997, senior OMB officials provided technical comments

that were incorporated where appropriate.

Page 17 GAO/GGD-97-109 Results Act
Contents

Executive Summary 3

Chapter 1 

Introduction

22

Congress Has Enacted a Statutory Framework for Improving

Federal Management and Decisionmaking

22

The Results Act Establishes Specific Requirements 25

The Results Act Incorporates Lessons From Past Initiatives to

Improve Public Sector Performance

30

Leading Organizations Show a Way 31

Objectives, Scope, and Methodology 34

Chapter 2 

Progress of the

Results Act

Implementation Has

Been Mixed

40

The Number of Performance Planning and Reporting Pilots

Exceeded the Results Act Requirements

40

Managerial Accountability and Flexibility Pilot Effort Did Not

Work as Intended

41

Performance Budgeting Pilots Likely to Be Delayed 42

OMB Actions to Prepare the Executive Branch for the Results Act

Implementation

42

Congressional Commitment to the Results Act Is Reflected in Its

Increased Involvement and Activities

44

Some Agencies Reported Improved Performance 44

Managers Reported Results-Oriented Performance Measures and

Their Use Was Limited

46

Conclusions 50

Chapter 3 

Unclear Agency

Missions and Goals

Will Challenge Results

Act Implementation

51

Crosscutting Federal Programs Pose Challenge for Results Act

Implementation

51

Balancing Goals Is Complicated by Competing Demands 55

Congressional Consultations Provide Mechanism for Identifying

and Addressing Crosscutting Issues and Seeking to Balance

Priorities

57

Conclusions 59

Page 18 GAO/GGD-97-109 Results Act
Contents

Chapter 4 

Developing Useful

Results-Oriented

Performance

Information Will Be an

Ongoing Difficulty

61

Several Factors Have Complicated Agencies’ Ability to Set Goals

and Identify Results-Oriented Performance Measures

61

Agencies Have Faced Challenges Collecting Reliable and Valid

Results-Oriented Performance Data

67

Approaches to Improve the Usefulness of Results-Oriented

Performance Information Are Beginning to Emerge

69

Conclusions 74

Chapter 5 

Building an

Organizational

Culture to Help Create

and Sustain a Focus

on Results Remains a

Work in Progress

76

Top Leadership Has Not Fully Demonstrated Commitment to

Results-Oriented Management

76

Managers Reported Needing Additional Training 79

Managers Reported Reduced Authority to Achieve Results 82

Less Than Half of Federal Managers Reported Positive

Recognition for Achieving Results

83

Involvement of Federal Managers in Results Act-Related

Activities Is Varied

84

Federal Managers’ Views About GPRA’s Past and Future Effects

on Agency Programs Are Mixed

86

Conclusions 89

Chapter 6 

Efforts to Integrate

Performance

Information With

Budget Decisions and

to Develop a New

Reporting Framework

Are Promising

90

The Results Act Seeks to Better Align Desired Results With

Budget Decisions

91

Integrated Performance, Cost, and Management Reports Provide

Basis for Supporting Key Decisions

93

Conclusions 96

Appendixes Appendix I: Survey on Results-Oriented Management Practices 98

Appendix II: Major Contributors and Acknowledgements 111

Related GAO Products 115

Page 19 GAO/GGD-97-109 Results Act
Contents

Tables Table 1.1: Analysis of Sample Dispositions 37

Table 2.1: Percent of Federal Managers Who Reported Their

Programs Have Various Performance Measures and Recollect

They Had Such Measures 3 Years Ago

48

 Table 2.2: Percent of Federal Managers Reporting Extent to

Which Results-Oriented Performance Information Is Used

Currently and Compared to 3 Years Ago

49

Table 2.3: Percent of Federal Managers Reporting Use of

Performance Information to a Great or Very Great Extent Among

Managers With Performance Measures Demonstrating

Achievement of Results to a Great or Very Great Extent

50

Table 5.1: Percent of Federal Managers Who Reported That Their

Agencies’ Top Leadership Demonstrated a Strong Commitment to

Achieving Results

77

Table 5.2: Percent of Federal Managers Who Reported That

During the Past 3 Years Their Agency Provided, Arranged, or Paid

for Training That Would Help Them to Accomplish Specific Tasks

80

Table 5.3: Percent of Federal Managers Who Reported Needing

Training or Additional Training to Accomplish Specific Tasks

80

Table 5.4: Percent of Federal Managers Reporting That Agency

Managers/Supervisors at Their Levels Had the Decisionmaking

Authority They Needed to Help the Agency Accomplish Its

Strategic Goals

83

Table 5.5: Percent of Federal Managers Reporting That They Had

Been Involved in GPRA-Related Activities During the Past 3 Years

85

Figures Figure 1.1: Key Steps and Critical Practices for Effectively

Implementing GPRA

33

Figure 5.1: Distribution of Federal Managers Expressing an

Opinion on the Extent to Which Implementing GPRA to Date Had

Improved Their Agencies’ Programs

87

Figure 5.2: Distribution of Federal Managers Expressing an

Opinion on the Extent to Which Implementing GPRA Could

Improve Their Agencies’ Programs in the Future

88

Page 20 GAO/GGD-97-109 Results Act
Contents

Abbreviations

ARC Appalachian Regional Commission

ARL Army Research Laboratory

BEP Bureau of Engraving and Printing

CFO Chief Financial Officers

CPDF Civilian Personnel Data File

DLA Defense Logistics Agency

DOD Department of Defense

EDA Economic Development Administration

EPA Environmental Protection Agency

FAA Federal Aviation Administration

FASAB Federal Accounting Standards Advisory Board

FDA Food and Drug Administration

GM General Management

GMRA Government Management Reform Act

GPRA Government Performance and Results Act

GS General Schedule

HHS Department of Health and Human Services

HUD Department of Housing and Urban Development

IRS Internal Revenue Service

JFMIP Joint Financial Management Improvement Program

MBO Management by Objectives

NCS National Cemetery System

NHTSA National Highway and Traffic Safety Administration

NPR National Performance Review

OCSE Office of Child Support Enforcement

OMB Office of Management and Budget

ONDCP Office of National Drug Control Policy

OPM Office of Personnel Management

OSHA Occupational Safety and Health Administration

PPBS Planning-Programming-Budgeting System

R&D Research and Development

SES Senior Executive Service

SSA Social Security Administration

USDA Department of Agriculture

VA Department of Veterans Affairs

ZBB Zero-Based Budgeting

Page 21 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

A consensus that the nation’s persistent federal deficit must be addressed

and that the effectiveness of federal programs must improve substantially

has spurred widespread efforts under way in federal agencies and

Congress to dramatically change the way the federal government is

managed. Although proposals to reduce the deficit are prompting difficult

policy debates over the role of the federal government, better management

of federal programs and activities can provide part of the solution to the

government’s fiscal problems. But the hard work of managing downsized

federal agencies is made more difficult when those agencies lack goals

that focus on results; cost-effective strategies to achieve those goals;

sound measures of performance; and reliable, accurate, and timely

information needed for decisionmaking.

Our work over many years repeatedly has found that agencies lack these

basic underpinnings of well-run organizations. In enacting the Government

Performance and Results Act (GPRA) of 1993, which is referred to as “GPRA”

or “the Results Act,” Congress expressed frustration that congressional

and executive branch decisionmaking was often hampered by the lack of

good information on the results of federal program efforts. However,

Congress also recognized that improving management in the federal sector

will not be easy or come quickly.

Congress Has Enacted

a Statutory

Framework for

Improving Federal

Management and

Decisionmaking

In the 1990s, Congress put in place a statutory framework to address the

long-standing weaknesses in federal operations, improve federal

management practices, and provide greater accountability for achieving

results. This framework included as its essential elements financial

management reform legislation, information technology reform legislation,

and the Results Act.

In enacting this framework, Congress sought to create a more focused,

results-oriented management and decisionmaking process within both

Congress and the executive branch. These laws seek to improve federal

management by responding to a need for accurate, reliable information for

congressional and executive branch decisionmaking, information that has

been badly lacking in the past, as much of our work has demonstrated.

Implemented together, these laws provide a powerful framework for

developing fully integrated information about agencies’ missions and

strategic priorities, the results-oriented performance goals that flow from

those priorities, performance data to show the achievement (or not) of

those goals, the relationship of information technology investments to the

Page 22 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

achievement of performance goals, and accurate and audited financial

information about the costs of achieving mission results.

Financial Management

Reform Legislation

The primary financial management reform legislation Congress enacted is

the Chief Financial Officers Act of 1990 (CFO) Act, as expanded by the

Government Management Reform Act of 1994 (GMRA). This law provides

the basis for identifying and correcting financial management weaknesses

that have cost the federal government billions of dollars and leave it

vulnerable to waste, fraud, and mismanagement. The expanded CFO Act

spelled out a long overdue and ambitious agenda to help the government

resolve its lack of timely, reliable, useful, and consistent financial

information. First, 24 major executive agencies, covering over 99 percent

of the federal government’s net outlay for fiscal year 1996, are required to

prepare and have audited financial statements for their entire operations,

beginning with fiscal year 1996. Second, every year starting with fiscal year

1997, the Secretary of the Treasury is required to produce a

governmentwide financial statement in coordination with the Director of

the Office of Management and Budget (OMB), and the statement is to be

audited by the Comptroller General of the United States. With successful

implementation, the audited financial statements required by the CFO Act,

as expanded by GMRA, will provide congressional and executive branch

decisionmakers with the financial and program cost information that they

have not previously had.

Information Technology

Reform Legislation

Information technology reform legislation, including the Paperwork

Reduction Act of 1995 and the Clinger-Cohen Act of 1996, is based on the

best practices used by leading public and private organizations to more

effectively manage information technology.1

 Even though agencies have

obligated billions of dollars—$145 billion in the last 6 years alone—to

build up and maintain their information technology resources, the benefits

frequently have been disappointing. Federal technology projects often

experience schedule slippages, incur extensive cost overruns, and fail to

provide promised performance improvements.

We have noted that the sound application and management of information

technology to support strategic program goals must be an important part

of any serious attempt to improve agency mission performance, cut costs,

1

Executive Guide: Improving Mission Performance Through Strategic Information Management and

Technology—Learning from Leading Organizations (GAO/AIMD-94-115, May 1994).

Page 23 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

and enhance responsiveness to the public.2 Under the information

technology reform laws, agencies are to better link their technology plans

and information technology use to their programs’ missions and goals. To

do this, agencies are to, among other things, (1) involve senior executives

in information management decisions; (2) establish senior-level Chief

Information Officers who are to, among other things, evaluate information

technology programs on the basis of applicable performance

measurements; (3) impose much-needed discipline on technology

spending; (4) redesign inefficient work processes; and (5) use

performance measures to assess technology’s contribution to achieving

mission results.

Government Performance

and Results Act

The landmark Results Act is the centerpiece of an integrated statutory

framework, which includes reform legislation in the areas of financial

management and information technology. The Act focuses on clarifying

missions, setting program goals, and measuring performance toward

achieving those goals. It emphasizes managing for results and pinpointing

opportunities for improved performance and increased accountability. The

purposes of the Act are to:

“(1) improve the confidence of the American people in the capability of

the Federal Government, by systematically holding Federal agencies

accountable for achieving program results;

“(2) initiate program performance reform with a series of pilot projects in

setting program goals, measuring program performance against those

goals, and reporting publicly on their progress;

“(3) improve Federal program effectiveness and public accountability by

promoting a new focus on results, service quality, and customer

satisfaction;

“(4) help Federal managers improve service delivery, by requiring that they

plan for meeting program objectives and by providing them with

information about program results and service quality;

“(5) improve congressional decisionmaking by providing more objective

information on achieving statutory objectives, and on the relative

effectiveness and efficiency of Federal programs and spending; and

2

High-Risk Areas: Actions Needed to Solve Pressing Management Problems (GAO/T-AIMD/GGD-97-60,

Mar. 5, 1997).

Page 24 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

“(6) improve internal management of the Federal Government.”3

Congress intended for the Act to improve the effectiveness of federal

programs by fundamentally shifting the focus of management and

decisionmaking away from a preoccupation with staffing and activity

levels to a broader focus on the results of federal programs. As measures

of their performance, agencies have traditionally used the amount of

money spent to support their programs, the number of staff employed, or

the number of tasks completed. But at a time when the value of many

federal programs is undergoing intense public scrutiny, an agency that

uses and reports only these measures has not answered the defining

question of whether these programs have produced real results. Under the

Results Act, virtually every executive agency is required to ask itself some

basic questions: What is our mission? What are our goals, and how can we

achieve them? How can we measure our performance? How will we use

performance information to make improvements?

The Results Act

Establishes Specific

Requirements

The Results Act requires executive agencies to set goals, measure

performance, and report on the degree to which goals were met. Because

Congress understood that most agencies would need to make significant

management changes to effectively implement the Act, requirements were

to be phased in over several years. Thus, the implementation of the Act

began with a set of pilot projects to be designated by OMB so that selected

agencies could gain experience in using key provisions of the Act and

provide lessons for other agencies. These pilots were to focus on

• the development of agency performance plans, which provide the direct

linkage between an agency’s long-term goals (commonly referred to as

strategic goals) and what its managers and employees do day-to-day; and

performance reports, which are the feedback to managers, policymakers,

and the public as to what was actually accomplished for the resources

expended;

• managerial accountability and flexibility, which is the granting of waivers

from certain nonstatutory administrative procedural requirements and

controls in return for greater management accountability for performance;

and

• performance budgeting, which is intended to provide Congress with

information on the direct relationship between proposed program

spending and expected program results and the anticipated effects of

varying spending levels on results.

3

P.L. 103-62, sec. 2.

Page 25 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

The Act requires the Director of OMB to report to Congress and the

president on the results of the pilot projects. The first report was to cover

pilot projects in performance planning and reporting and managerial

accountability and flexibility and was due no later than May 1, 1997. This

report was to include, among other things, an assessment of the benefits

and costs of pilot performance plans and reports and any significant

difficulties experienced by the pilot agencies in preparing proposed

waivers.4 OMB is to prepare a second report to Congress and the president

on the performance budgeting pilots. This report is due no later than

March 31, 2001.

Agencies Are to Set

Strategic Direction,

Identify Annual

Performance Goals, and

Report on Performance

The Results Act is designed to improve the efficiency and effectiveness of

federal programs by establishing a system to set goals for program

performance and to measure results. Specifically, the Act requires

executive agencies to prepare multiyear strategic plans, annual

performance plans, and annual performance reports.

Multiyear Strategic Plans The Results Act requires virtually every executive agency to develop

strategic plans covering a period of at least 5 years forward from the fiscal

year in which it is submitted and to update those plans at least every 3

years.5

 Agencies’ first strategic plans are to be submitted to Congress and

the Director of OMB by September 30, 1997. The strategic plans are to

(1) include the agencies’ mission statements; (2) identify long-term general

goals and objectives, also known as strategic goals; (3) describe how the

agencies intend to achieve those goals through their activities and through

their human, capital, information, and other resources; and (4) explain the

key external factors that could significantly affect the achievement of

those goals. Under the Act, strategic plans are the starting point for

agencies to set annual performance goals and to measure program

performance in achieving those goals. Consequently, strategic plans are

also to include a description of how long-term general goals will be related

to annual performance goals as well as a description of the program

evaluations that agencies used to establish their long-term general goals

4

The Government Performance and Results Act: Report to the President and the Congress from the

Director of the Office of Management and Budget, May 19, 1997.

5

The Results Act applies to agencies as defined in 5 U.S.C. 306(f), which generally covers executive

departments, government corporations, and independent establishments.

Page 26 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

and a schedule for subsequent evaluations. As part of the strategic

planning process, agencies are required to consult with Congress and

solicit the views of other stakeholders—those governmental and

nongovernmental entities potentially affected by, or interested in, the

agencies’ activities.

Annual Performance Plans Building on the decisions made as part of the strategic planning process,

the Results Act requires executive agencies to develop annual

performance plans covering each program activity set forth in the

agencies’ budgets.6

 The first annual performance plans, covering fiscal

year 1999, are to be submitted to OMB this fall and to Congress after the

President’s budget in 1998. The plan is to contain an agency’s annual

performance goals, which the agency is to use in order to gauge its

progress toward accomplishing its strategic goals. The plan also is to

include the performance measures that the agency will use to gauge

progress toward the achievement of its annual performance goals and the

resources the agency will need to meet its goals. Finally, the plan is to

discuss how the agency will verify the resulting performance data.

The Act authorizes agencies to apply for managerial flexibility waivers in

their annual performance plans. Agencies’ authority to request waivers of

nonstatutory administrative procedural requirements and controls is

intended to provide federal managers with more flexibility to structure

agency systems to better support performance goals. An example of

increased flexibility would be to allow an organization to recapture

unspent operating funds because of increased efficiencies and then to use

these funds to purchase new equipment or expand employee training.

Another example might involve delegating more authority to line managers

to make procurement decisions.

OMB is to use the performance plans that agencies submit to develop an

overall federal government performance plan. OMB is to submit this

governmentwide plan each year to Congress with the president’s budget.

According to the Senate Committee on Governmental Affairs report on the

Act, the overall federal government performance plan is to present to

Congress a single cohesive picture of the federal government’s annual

performance goals for the fiscal year.7

 The first overall plan is due with the

President’s fiscal year 1999 budget.

6

The term “program activity” refers to the listings of projects and activities in the Appendix portion of

the Budget of the United States Government. Program activity structures are intended to provide a

meaningful representation of the operations financed by a specific budget account.

7

S. Rep. No. 58, 103d Cong. 1st Sess. (1993).

Page 27 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

Annual Performance Reports Finally, the Results Act requires each executive agency to prepare annual

reports on program performance for the previous fiscal year. The first

performance reports for fiscal year 1999 are due to Congress and the

president no later than March 31, 2000, and subsequent reports are due by

March 31 for the years that follow. In each report, an agency is to review

and discuss its performance compared with the performance goals it

established in its annual performance plan. When a goal is not met, the

agency is to explain in the report the reasons the goal was not met; plans

and schedules for meeting the goal; and, if the goal was impractical or not

feasible, the reasons for that and the actions recommended. According to

the Senate committee report on the Act, actions needed to accomplish a

goal could include legislative, regulatory, or other actions. If an agency

finds a goal to be impractical or not feasible, it is to include a discussion of

whether the goal should be modified.

In addition to evaluating the progress made toward achieving its annual

goals, an agency’s program performance report is to evaluate the agency’s

performance plan for the fiscal year in which the performance report was

submitted. Thus, in their fiscal year 1999 performance reports that are due

by March 31, 2000, agencies are required to evaluate their performance

plans for fiscal year 2000 on the basis of their reported performance in

fiscal year 1999. This evaluation is to help show how an agency’s actual

performance is influencing its performance plan. The report also is to

include (1) the summary findings of program evaluations completed

during the fiscal year covered by the report and (2) the use and

effectiveness of any of the Results Act managerial flexibility waivers that

an agency received.

Agencies also are to include baseline and trend data in annual

performance reports to help ensure that their reports are complete and

that performance is viewed in context. Such data can show whether

performance goals are realistic given the past performance of an agency.

Such data can also assist users of reports to draw more informed

conclusions than they would by comparing only a single year’s

performance against an annual goal, because users of reports can see

improvements or declines in an agency’s performance over prior years.8

For fiscal years 2000 and 2001, agencies’ reports are to include data on the

extent to which their performance achieved their goals, beginning with

fiscal year 1999. For each subsequent year, agencies are to include

performance data for the year covered by the report and 3 prior years.

Congress recognized that in some cases not all the performance data will

8

GPRA Performance Reports (GAO/GGD-96-66R, Feb. 14, 1996).

Page 28 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

be available in time for the required reporting date. In such cases, agencies

are to provide whatever data are available with a notation as to their

incomplete status. Subsequent annual performance reports are to include

the complete data as part of the trend information.

The Results Act Used a

Phased Implementation

Approach to Build

Experience and Provide

Lessons

The Results Act required OMB to designate at least 10 agencies to

participate in the first set of pilot projects, which were to focus on the

preparation of program performance plans and reports. The selected

agencies were to reflect a representative range of government functions

and capabilities in measuring and reporting program performance. Also,

the Act required these pilot projects to use strategic plans during 1 or

more years of the pilot period, which ran from fiscal years 1994 through

1996.

Congress recognized that managerial accountability for results is linked to

managers having sufficient flexibility, discretion, and authority to

accomplish desired results. Thus, the Act specified a second set of pilot

projects in managerial accountability and flexibility. These pilot projects

were to propose waivers of administrative procedural requirements and

controls for one or more of the major functions and operations of the

agency to determine the effects of providing federal managers with

increased managerial flexibility in exchange for the greater accountability

for performance. At least five agencies from the first set of pilot projects

were to be designated by OMB to test managerial accountability and

flexibility during fiscal years 1995 and 1996.

Finally, the Results Act called for a third set of pilot projects for fiscal

years 1998 and 1999 to test performance budgeting—the presentation of

varying levels of performance that would result from different budget

levels—before requiring it statutorily. OMB is to select at least five

agencies, at least three of which had experience developing performance

plans during the first pilot phase, to test performance budgeting. By

March 31, 2001, OMB is to report to Congress and the president on the pilot

test of performance budgeting. OMB’s report is to assess the feasibility of

performance budgeting as part of the president’s budget, recommend

whether legislation requiring performance budgets should be proposed,

and identify any other recommended changes to the requirements of the

Act.

Page 29 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

The Results Act

Incorporates Lessons

From Past Initiatives

to Improve Public

Sector Performance

The Results Act is the most recent federal government reform initiative

designed to better align spending decisions with expected results. Through

acquiring better information on the relative efficiency and effectiveness of

federal programs and spending, the Act seeks to help federal managers

improve program performance. We recently reported on the key design

elements and approaches of the Act and compared them with those of past

federal initiatives that sought to link resources with results—broadly

known as performance budgeting.9

 Since 1950, the federal government has

attempted four governmentwide performance budgeting initiatives:

(1) reforms flowing from the first Hoover Commission in its efforts to

downsize the post-World War II government; (2) the

Planning-Programming-Budgeting-System (PPBS) that President Johnson

began in 1965, which introduced a decisionmaking framework to the

executive branch budget formulation process by presenting and analyzing

choices among long-term policy objectives and alternative ways of

achieving them; (3) Management by Objectives (MBO) that President Nixon

initiated in 1973, which sought to link agencies’ stated objectives to their

budget requests and put in place a process to hold agency managers

responsible for achieving agreed-upon outputs and results; and

(4) Zero-Based Budgeting (ZBB) that President Carter initiated in 1977,

which required agencies to set priorities on the basis of the program

results that could be achieved at alternative spending levels, one of which

was to be below current funding. Even though these reforms were

generally perceived as having fallen short of their stated goals, they

contributed to the evolution of performance-based measurement and

budgeting in the federal government.

The Results Act incorporates critical lessons learned from these previous

efforts.

• Where past efforts failed to link executive branch performance planning

and measurement with congressional resource allocation processes, the

Act requires explicit consultation between the executive and legislative

branches on agency strategic plans.

• Where past initiatives devised unique performance information formats

often unconnected to the structures used in congressional budget

presentations, the Act requires agencies to plan and measure performance

using the program activities listed in their budget submissions.

• Where past initiatives were generally unprepared for the difficulties

associated with measuring the results of federal programs and often

9

Performance Budgeting: Past Initiatives Offer Insights for GPRA Implementation (GAO/AIMD-97-46,

Mar. 27, 1997).

Page 30 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

retreated to simple output or workload measures, the Act states a

preference for results-oriented measures while recognizing the need to

develop a range of measures, including output and, in some cases,

nonquantitative measures.

Leading Organizations

Show a Way

The federal government’s efforts to become more results oriented are

consistent with a broad national and international trend among public

organizations. According to the Senate committee report on the Results

Act, use of performance measurement was a growing trend in state

governments. The report further noted that work done by the Organization

for Economic Cooperation and Development suggested that several

foreign governments may have been 5 to 10 years ahead of the United

States in the development of performance measurement, which was a

basic part of their broader efforts to better manage for results. More

specifically, these foreign governments set out broad themes to (1) define

goals clearly, develop measures, and report on progress; and (2) give

managers the flexibility to manage for results by providing them the tools

and incentives to act.

At the request of the Senate Committee on Governmental Affairs and the

House Committee on Government Reform and Oversight, we reported on a

number of state and foreign governments that were successfully pursuing

management reform and becoming more results-oriented.10 We studied the

state governments of Florida, Oregon, Minnesota, North Carolina, Texas,

and Virginia; and the foreign governments of Australia, Canada, New

Zealand, and the United Kingdom. Much like the federal government, a

number of these governments were dissatisfied with their public service.

Many of them were called upon to improve performance while

simultaneously reducing costs. Some faced economic challenges, such as

economies dominated by government spending and high budget deficits.

Each of these organizations set its agenda for management reform

according to its own environment, needs, and capabilities. Yet, despite

their differing approaches to reform, these organizations were seeking to

become more results oriented.

We were asked by the Senate Committee on Governmental Affairs and the

House Committee on Government Reform and Oversight whether the

experiences of these and other public organizations could yield

worthwhile lessons for executive agencies as they attempt to implement

10Managing for Results: State Experiences Provide Insights for Federal Management Reforms

(GAO/GGD-95-22, Dec. 21, 1994); and Managing for Results: Experiences Abroad Suggest Insights for

Federal Management Reforms (GAO/GGD-95-120, May 2, 1995).

Page 31 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

the Results Act. In response to that request, we issued the Executive

Guide: Effectively Implementing the Government Performance and Results

Act.

11 The Executive Guide identifies a set of key steps and associated

practices that leading federal and other public sector organizations have

used to successfully implement reform efforts that are consistent with the

Results Act. Accompanying the discussion of each practice is a case

illustration of a federal agency that has made progress in incorporating

that practice into its operations.

Taken together, the key steps and practices drawn from the organizations

we studied provide a useful framework to assist Congress and the

executive branch as they work to implement the Act. These steps were to

(1) define mission and desired outcomes, or results; (2) measure

performance to gauge progress; and (3) use performance information as a

basis for decisionmaking. In taking these steps, leading organizations also

found that certain leadership practices, such as devolving operational

authority and creating incentives for managers and staff to focus on

results, were central to making the changes needed for the organizations

to become more results oriented. Figure 1.1 illustrates the key steps and

associated practices.

11GAO/GGD-96-118, June 1996.

Page 32 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

Figure 1.1: Key Steps and Critical Practices for Effectively Implementing GPRA

Reinforce GPRA Implementation

Practices:

9. Devolve decisionmaking

with accountability

10. Create incentives

11. Build expertise

12. Integrate management

reforms

Step 1:

Define Mission and

Desired Outcomes

Practices:

1. Involve stakeholders

2. Assess environment

3. Align activities,

core processes,

and resources

Step 3:

Use Performance

Information

Practices:

6. Identify performance

gaps

7. Report information

8. Use information

Step 2:

Measure Performance

Practices:

4. Produce measures at

each organizational

level that

demonstrate results,

are limited to the vital

few,

respond to multiple

priorities, and

link to responsible

programs

5. Collect data

Source: Executive Guide.

Page 33 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

Objectives, Scope,

and Methodology

As required by the Act, our overall objective was to report to Congress on

the implementation of the Act, including the prospects for compliance by

executive agencies beyond those participating as pilot projects. More

specifically, our objectives for this report were to (1) assess the status of

the Results Act implementation efforts; (2) identify significant challenges

confronting executive agencies in their efforts to become more results

oriented; and (3) describe ongoing efforts to integrate program, cost, and

budget information into a reporting framework that allows for fuller

consideration of resource allocations, operational costs, and performance

results. As agreed with the Senate Committee on Government Affairs and

the House Committee on Government Reform and Oversight, our strategy

was to partially fulfill our legislative requirement by reporting regularly on

the Act and related initiatives throughout the pilot phase of performance

planning, measurement, and reporting. This report culminates our efforts

to meet this mandate.

To meet each of our objectives, we relied on a wide range of management

work we have done for the reports, testimonies, and other products we

issued to congressional committees, OMB, and executive agencies on the

progress of agencies’ efforts to become results oriented. These products

include, most recently, the Executive Guide: Effectively Implementing the

Government Performance and Results Act (GAO/GGD-96-118, June 1996); GPRA

Performance Reports (GAO/GGD-96-66R, Feb. 14, 1996); GPRA: Managerial

Accountability and Flexibility Pilot Did Not Work As Intended

(GAO/GGD-97-36, Apr. 10, 1997); Managing for Results: Analytic Challenges in

Measuring Performance (GAO/HEHS/GGD-97-138, May 30, 1997); Performance

Budgeting: Past Initiatives Offer Insights for GPRA Implementation

(GAO/AIMD-97-46, Mar. 27, 1997); and numerous congressional testimonies.

This work also included detailed examinations of related federal

management improvement efforts, in particular the National Performance

Review (NPR). In addition, we used products we have issued in recent

years on the critical management issues confronting specific agencies,

such as the Forest Service and the Department of Energy; program efforts

that cut across several agencies, such as federal land management, food

safety, and early childhood development programs; and, most

prominently, our examination of 25 high-risk areas—those federal areas

most vulnerable to waste, fraud, and mismanagement.12 The Related GAO

Products section at the end of this report lists selected reports and

congressional testimonies that we issued on the Results Act

implementation and related results-oriented management initiatives in

pilot and nonpilot agencies.

12See, for example, High-Risk Series: An Overview (GAO/HR-97-1, Feb. 1997).

Page 34 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

We reviewed OMB circulars, memoranda, and general guidance to

executive agencies on a wide range of the Results Act implementation

activities, such as the preparation and submission of strategic plans and

examples of performance measurement. For selected pilot agencies, we

reviewed performance plans and performance reports. For these and

nonpilot agencies that we selected to reflect a representative range of

federal functions, we also reviewed draft strategic plans and other

applicable management documents. To supplement our existing

information, we reviewed the American Society of Public Administration’s

(ASPA) case studies on performance management in the federal

government and the National Academy of Public Administration’s (NAPA)

assessment of the initial pilot performance plans.13

To further meet our first two objectives, we sent a questionnaire to a

random sample of 1,300 mid-level and upper level civilian managers and

supervisors working in the 24 executive branch agencies covered by the

CFO Act. These agencies represent about 97 percent of the executive

branch full-time workforce and cover over 99 percent of the federal

government’s net outlay for fiscal year 1996. In reporting this

questionnaire data, when we use the term governmentwide, we are

referring to these 24 CFO Act agencies; and when we use the term federal

managers, we are referring to both managers and supervisors. We drew

our sample from the Office of Personnel Management’s (OPM) Civilian

Personnel Data File (CPDF) as of June 30, 1996, using file designators to

identify individuals who perform management and supervisory functions.

We restricted our sample to individuals who were in the United States due

to the difficulty and time associated with forwarding and receiving mail in

some foreign countries.

This sample was stratified by whether the manager was SES or non-SES and

by whether the manager was working in an agency or agency component

that was designated as a Results Act pilot. The management levels covered

general schedule (GS) and general management (GM) levels GS/GM-13

through the career Senior Executive Service (SES). The identification of

managers working in pilots was limited to selection from the following 13

Results Act pilots: Defense Logistics Agency, Army Research Laboratory,

Social Security Administration, Small Business Administration,

Occupational Safety and Health Administration, National Highway Traffic

Safety Administration, Forest Service, National Technical Information

13See the federal case studies developed at the initiative of ASPA on selected agencies’ strategic

planning and performance measurement efforts and Toward Useful Performance Measurement:

Lessons Learned from Initial Pilot Performance Plans Prepared Under the Government Performance

and Results Act, NAPA, Nov. 1994.

Page 35 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

Service, National Oceanographic and Atmospheric Administration,

Internal Revenue Service, U.S. Mint, Bureau of Engraving and Printing, and

the U.S. Customs Service’s Office of Enforcement.

Our selection of pilots from which to sample managers was guided by the

following requirements: (1) unique CPDF agency component identifier

codes that corresponded with the scope or bounds of the Results Act pilot,

(2) the pilot designation was made before fiscal year 1996, and (3) the pilot

covered 350 or more employees. The first restriction was needed to ensure

that any managers who were randomly selected for the pilot strata would

be from an agency or agency component that was encompassed in its

entirety by the pilot. Because many pilots were aligned according to

agency functions or processes and cut across multiple agency

components, the CPDF identifier codes could not be reliably used to ensure

that a randomly selected manager was working within the scope of the

pilot. We also excluded pilots that were designated in fiscal year 1996,

because any significant initiatives would have been fairly recent and may

not have been sufficiently implemented for any effects to be reflected in

questionnaire responses. Small-scale pilots covering fewer than 350

employees were excluded in order to allocate the sample more efficiently,

given the resources and time available to do the survey. In general, our

selected pilots, 12 of which were designated as pilots in fiscal year 1994,

contained a preponderance of pilots that encompassed the whole agency

or a major agency component.

Our questionnaire asked managers about their perceptions on such topics

as (1) agency mission and goals, (2) ways of measuring performance,

(3) the use of performance information now and 3 years ago,

(4) hindrances to measuring and using performance information,

(5) agency climate, (6) Results Act-related training, (7) knowledge of the

Act, (8) involvement in the Act, and (9) improvements seen or expected as

a result of implementation of the Act. Most of the items on the

questionnaire were closed-ended, meaning that depending on the

particular item, respondents could choose one or more response

categories or rate the strength of their perception on a 5-point extent

scale. After designing our questionnaire, we conducted pretests at four

federal agencies: the Department of Defense, the Department of

Commerce, the Department of Agriculture’s Forest Service, and the

Department of Justice. In all, 32 federal managers took part in these

pretests. The questionnaire was revised after each pretest on the basis of

the feedback received.

Page 36 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

The survey was initially sent out between November 27, 1996, and

January 3, 1997. Managers who did not respond to the initial questionnaire

were sent one follow-up questionnaire. Table 1.1 summarizes the

disposition of the 1,300 surveys sent to managers originally sampled.

Table 1.1: Analysis of Sample

Dispositions Number

Total managers sampled 1,300

Selectees deleted from samplea 47

Eligible sample 1,253

Unable to identify or locate sample selectee 22

Refused to participate 23

Questionnaires not returned 299

Questionnaires returned, not usable 4

Usable questionnaires returned 905

aIncludes individuals who were outside the United States, retired, separated, died, or otherwise

left the agency or had some other reason that excluded them from the population of interest.

We received usable responses from 72 percent of the eligible sample. The

response rate across strata ranged from 61 percent to 88 percent. Although

we did not test the validity of the respondents’ answers or the comments

they made, we took several steps to check the quality of our survey data.

We reviewed and edited the completed questionnaires, made internal

consistency checks on selected items, and checked the accuracy of data

entry on a sample of surveys.

We followed up on a sample of the nonrespondents to assess whether the

views of the nonrespondent group differed from the views of those who

returned the survey. We selected a stratified, random sample of 101

nonrespondents to telephone and urge them to return their questionnaires.

If necessary, they were sent new questionnaires. Nine of these

nonresponse sample members returned completed questionnaires before

we contacted them. Of the remaining 92 individuals, we determined that 2

were no longer in the population of interest and that various

circumstances (e.g., maternity leave) precluded participation by 4 others.

Of the 86 individuals we did contact, 51 returned completed, usable

questionnaires; 16 refused; 1 returned the questionnaire too late for the

data to be included in the analysis; and 18 failed to return the

questionnaire. We analyzed the responses of the 51 individuals returning

the questionnaire and compared their responses on selected items to those

received from all other respondents. Our assessment indicated that across

Page 37 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

the selected items and types of analyses performed, the responses of the

follow-up group did not demonstrate a sufficient degree of significant and

consistent differences to merit separate treatment in subsequent analyses

of the questionnaire data. The responses of the 51 individuals were

included with all other responses. Appendix I contains a copy of the

questionnaire with weighted percentage of managers responding to each

item. Percents presented in this report have been rounded to the nearest

whole number.

The overall survey results are generalizable to the CFO Act agencies.

Results presented for the selected pilots are generalizable only to those

pilots, but not to all the other Results Act pilots. All results are subject to

some uncertainty or sampling error as well as nonsampling error. Because

a complex sample design was needed to support a diverse set of planned

analyses and sample size was constrained by the resources and time

available to do the survey, it was not possible to employ a sample size that

would consistently limit overall sampling error to 5 percent or less at the

95 percent confidence level. In general, percentages reported for the entire

sample have confidence intervals ranging from +5 percentage points to

+12 percentage points. In other words, if all CFO Act managers in our

population had been surveyed, the chances are 95 out of 100 that the

results obtained would not differ from our sample estimate, in the most

extreme case, by more than +12 percentage points. The magnitude of

sampling error will also vary in relation to the particular groups or items

being compared. As a result, the magnitude of difference needed to attain

statistical significance between groups or items will also vary. In some

instances, a difference of a certain magnitude may be statistically

significant; in other instances, depending on the nature of the comparison

being made, a difference of equal or even greater magnitude may not be

statistically significant. We note throughout the report when differences

between groups are significant.

The objectives, scopes, and methodologies of the specific products upon

which this report is based are detailed in those products, and, as

appropriate, those products contained comments from the relevant

agencies. That work was done over the last several years. Our work for

this report was conducted between September 1996 and April 1997 in

accordance with generally accepted government auditing standards. We

requested comments from the Director of OMB because of OMB’s

responsibilities for designating pilots, overseeing the pilot process, and

generally leading the executive branch’s Results Act efforts. On May 21

Page 38 GAO/GGD-97-109 Results Act
Chapter 1 

Introduction

and 22, 1997, senior OMB officials provided technical comments. We have

incorporated OMB’s technical commments where appropriate.

Page 39 GAO/GGD-97-109 Results Act
Chapter 2 

Progress of the Results Act Implementation

Has Been Mixed

Progress in implementing the Results Act has been mixed. The first

pilot—on performance planning and reporting—had far more participants

than required, although no participants were designated for the second

pilot on managerial accountability and flexibility, and the third pilot on

performance budgeting likely will be delayed by at least a year. Still, OMB

has taken a number of actions to prepare for the implementation of the

Act, such as expanding the amount and prominence of performance

information that is used during the executive branch budget process.

Congress has also taken actions to demonstrate its commitment to the Act

and reinforce to agencies the importance that it places on the full and

complete implementation of the Act.

Overall, the experiences of the pilot agencies and related efforts in

nonpilot agencies show that substantial improvements in performance are

possible—even in the short term—when organizations adopt a disciplined

approach to setting goals, measuring performance, and using performance

information to improve effectiveness. However, after 3 years of piloting

the Act, the reported examples of substantial performance improvements

still are fairly isolated, and many agencies do not appear to be

well-positioned to provide a results-oriented answer to the fundamental

Results Act question: What are we accomplishing?

Federal managers report that results-oriented performance measures often

do not exist. As a result, it is not surprising that managers also report that

results-oriented performance information frequently is not used to make

important decisions affecting their agencies and programs, even when

performance information is available.

The Number of

Performance Planning

and Reporting Pilots

Exceeded the Results

Act Requirements

The number of the performance planning and reporting pilots exceeded

the Act requirements, thereby giving a large number of organizations a

head start on the Results Act implementation and providing a rich body of

experience for nonpilot organizations to draw upon.1

 For the first set of

pilot projects, the Act required OMB to select at least 10 agencies to pilot

the performance planning and reporting requirements for 1 or more of an

agency’s major functions and operations during fiscal years 1994 through

1996. However, a total of 77 pilot projects, including both entire

agencies—e.g., the Defense Logistics Agency (DLA), the Internal Revenue

Service (IRS), and the Social Security Administration (SSA)—and

programs—e.g., the Department of Veterans Affairs (VA) Loan Guaranty

1

During the pilot phase, GAO and others issued various assessments of pilot agencies. See, for

example, GAO/GGD-96-66R, February 14, 1996, and the federal case studies developed at the initiative

of ASPA on selected agencies’ strategic planning and performance measurement efforts.

Page 40 GAO/GGD-97-109 Results Act
Chapter 2 

Progress of the Results Act Implementation

Has Been Mixed

Program, participated at some point during this first pilot. When this part

of the pilot phase concluded at the end of fiscal year 1996, a total of 68

pilots representing 28 agencies were project participants.

The pilots represented a wide range of government activities and

functions. For example, the first set of pilot projects included military

operations, such as those conducted by the Air Force Air Combat

Command in the Department of Defense (DOD); regulatory programs, such

as those conducted by the Occupational Safety and Health Administration

(OSHA) in the Department of Labor; intergovernmental programs, such as

those conducted by the Office of Child Support Enforcement (OCSE) in the

Department of Health and Human Services (HHS); and businesslike

functions, such as those conducted by the U.S. Mint in the Department of

the Treasury.

Managerial

Accountability and

Flexibility Pilot Effort

Did Not Work as

Intended

The Results Act required OMB to select a second set of pilot projects during

fiscal years 1995 and 1996 in order to determine the effects of providing

managers of federal programs with increased managerial flexibility in

exchange for the greater accountability for performance. The Act required

OMB to designate at least five agencies from the first set of pilot projects on

performance planning and reporting to participate in the second pilot on

managerial accountability and flexibility. However, we found that the

managerial accountability and flexibility pilot did not work as intended.2

OMB did not designate as pilot projects any of the 7 departments and 1

independent agency that submitted a total of 61 waiver proposals.

We found that three major factors contributed to the failure of the

managerial accountability and flexibility pilot to work as intended. First,

changes in federal management practices and laws that occurred after the

Act was enacted affected agencies’ need for the managerial flexibility

waivers. For example, the Federal Workforce Restructuring Act of 1994

established a new personnel ceiling for all of the executive branch, which

effectively limited OMB’s ability to waive agency personnel ceilings

established in the budget. Second, agencies could use other, less rigorous,

means to obtain waivers from administrative requirements, such as the

administration’s NPR reinvention labs, which were designed to test ways

that agencies could improve their performance and customer service by

reengineering work processes and eliminating unnecessary regulations.

Third, unlike its proactive approach to the first set of pilots, OMB did not

2

GPRA: Managerial Accountability and Flexibility Pilot Did Not Work As Intended (GAO/GGD-97-36,

Apr. 10, 1997).

Page 41 GAO/GGD-97-109 Results Act
Chapter 2 

Progress of the Results Act Implementation

Has Been Mixed

work actively with agencies that were seeking to take part in the

managerial accountability and flexibility pilot. For example, OMB’s

feedback to agencies concerning their waiver proposals for consideration

as pilots was very limited. This contrasted directly with the approach that

OMB took with the performance planning and reporting pilots, for which it

issued a summary assessment of the 1994 performance plans, including

the strengths, weaknesses, and additional actions agencies needed to take

to improve these plans.

Performance

Budgeting Pilots

Likely to Be Delayed

OMB is also to select a third set of pilot projects to test performance

budgeting for fiscal years 1998 and 1999. Under the test, pilot agencies are

to prepare performance budgets, which are to provide Congress with

information on the direct relationship between proposed program

spending and expected program results. The Senate committee report on

the Results Act said that the performance budgeting pilots are to begin

“only after agencies had sufficient experience in preparing strategic and

performance plans, and several years of collecting performance data.”3

Because of this and because it recognizes the importance of concentrating

on governmentwide Results Act implementation in fiscal year 1998, OMB

has indicated that these pilots will be delayed for at least a year. As

envisioned under the Act, performance budgets must show the effects on

performance of marginal changes in funding. According to OMB, few

agencies currently have either sufficient baseline performance or financial

information or the ability to use sophisticated analytic techniques to

calculate the effects on performance of marginal changes in funding.

According to OMB, delaying the performance budgeting pilots will provide

agencies time to develop needed information and abilities.

OMB Actions to

Prepare the Executive

Branch for the Results

Act Implementation

In the nearly 4 years since the passage of the Results Act, OMB has sought

to expand the amount and prominence of performance information within

the executive branch budget process. For example, our 1995 review of

budget documents and interviews with OMB staff 1 year after OMB 2000—a

major April 1994 reorganization and process change in OMB—indicated that

there was greater attention paid to agency management issues and the use

of performance information in the fiscal year 1996 budget process than

there had been in the fiscal year 1995 process.4

 Whereas the fiscal year

1995 documents discussed streamlining primarily in terms of the number

3

S. Rep. No. 58, 103d Cong. 1st Sess. p. 38. (1993).

4

Office of Management and Budget: Changes Resulting From the OMB 2000 Reorganization

(GAO/GGD/AIMD-96-50, Dec. 29, 1995).

Page 42 GAO/GGD-97-109 Results Act
Chapter 2 

Progress of the Results Act Implementation

Has Been Mixed

of positions to be eliminated, the fiscal year 1996 documents also included

discussions about how proposed staff reductions could affect the agency’s

performance.

We reported that OMB continued to expand its focus on performance

measures and information during the fiscal year 1997 budget preparation

process. Between March and June 1995, OMB held a “Spring Review” as a

prelude for development of the fiscal year 1997 budget that was intended

to help OMB and agencies work together to identify useful performance

information. OMB conducted a “Summer Review” in July 1996 that

examined the status of agencies’ strategic planning efforts. Guidance from

OMB to executive agencies stated that the review would focus on the

adequacy, relevance, and appropriateness of their mission statements and

their strategic goals.

The 1996 “Fall Review,” according to OMB, was intended to produce

agreement between OMB and the agencies on the performance goals and

measures that agencies would include in their fiscal year 1999 annual

performance plans. OMB’s most recent effort, which was under way in the

spring of 1997, was its strategic assessment initiative. The purpose of this

initiative, which focused on agencies’ goals, commitments, and main

priorities, is to assist agencies in the preparation of their strategic plans

and annual performance plans that are due in September 1997.

To help ensure the success of the Results Act, the CFO Council, chaired by

OMB’s Deputy Director for Management, established a GPRA Implementation

Committee.5

 The Committee provides guidance and information to CFOs

and managers in the 24 agencies covered by the CFO Act. The Committee,

concluding that uncertainty or fear of failure may immobilize an agency’s

efforts to implement the Act and that its implementation is evolutionary in

that proficiency comes with time and experience, has been actively

working to assist federal managers with the implementation of the Act. As

part of that effort, the Committee published a set of guiding principles and

key issues for implementing the Act.6

 More recently, the Committee issued

a user guide to help federal managers meet the Results Act requirements

and, among other things, develop a framework for using performance

5

The CFO Council was created by the CFO Act to provide the leadership foundation necessary to

effectively carry out the responsibilities of executive agency CFOs.

6

Implementation of the Government Performance and Results Act (GPRA), A Report on the Chief

Financial Officer’s Role and Other Issues Critical to the Governmentwide Success of GPRA, Chief

Financial Officers Council, GPRA Implementation Committee, May 1995.

Page 43 GAO/GGD-97-109 Results Act
Chapter 2 

Progress of the Results Act Implementation

Has Been Mixed

information in making budget decisions.7 The Committee also has

undertaken a number of training efforts for federal managers, including

sponsoring a conference on the Act in the fall of 1996 and developing a

training package. The purpose of the training package, which was

provided to executive agencies, was to build awareness and support for

the Act among political appointees and senior career staff.

Congressional

Commitment to the

Results Act Is

Reflected in Its

Increased

Involvement and

Activities

Congressional involvement is critical to sustain the momentum of the

Results Act implementation and to reinforce to agencies the importance

that Congress places on the successful and thorough implementation of

the Act. We testified in March 1996 that officials in some pilot agencies

believed Congress’ interest in the Act was limited since, for example,

Congress seldom asked about the implementation of the Act in their

agencies.8

 However, as the date for governmentwide implementation has

approached, Congress has signaled its strong commitment to the Act

through periodic hearings and other actions. Most prominently, on

February 25, 1997, the Speaker of the House, the House Majority Leader,

the Senate Majority Leader, and key committee chairmen from the House

and the Senate sent a letter to the Director of OMB. The letter underscored

the importance that the congressional Majority places on the

implementation of the Results Act, noted a willingness on the part of

Congress to work cooperatively with the administration, and established

expectations for consultations. Building on its commitment to the Act,

Congress has begun to consult with agencies on their strategic plans. In

the House, the Majority’s consultation effort is being led by teams

consisting of staff from various committees that are focusing on specific

agencies. These teams have been reaching out to agencies to review and

comment on agencies’ strategic plans.

Some Agencies

Reported Improved

Performance

The Results Act pilot phase has shown that short-term, substantial

program improvements are possible when organizations adopt a

disciplined approach to setting goals, measuring performance, and using

performance information to improve effectiveness, although the reported

examples of substantial performance improvements still are limited, as is

to be expected at this early stage of implementation. For example, as

highlighted in our Executive Guide, the Coast Guard reexamined its

7

Integrating Performance Measurement into the Budget Process: A User Guide to Multiple Ideas,

Practices, Formats, Processes, Steps, and One Model Tying Everything Together, Chief Financial

Officers Council, GPRA Implementation Committee, Jan. 21, 1997.

8

Managing for Results: Achieving GPRA’s Objectives Requires Strong Congressional Role

(GAO/T-GGD-96-79, Mar. 6, 1996).

Page 44 GAO/GGD-97-109 Results Act
Chapter 2 

Progress of the Results Act Implementation

Has Been Mixed

mission and shifted the focus of its marine safety program from regulation

to one that includes education.9

 This change contributed to a significant

decline in the towing industry fatality rate—from 91 per 100,000 industry

employees in 1990 to 27 per 100,000 in 1995. Similarly, the Veterans Health

Administration improved services to veterans by more rigorously

assessing the results of the medical care it provides to the nation’s

veterans. In particular, the Veterans Health Administration reported that it

used performance information to target the most important improvement

opportunities and thereby lowered the mortality rate for cardiac

procedures by an average of 13 percent over the last 8 years.

We also have identified other examples of agencies that have reported

improvements to their operations. SSA has had a single, national toll-free

number as an alternative to field office contact for conducting simple

business since 1989. On the basis of customer feedback indicating that this

convenient telephone service was important to the public, yet had

persistently high busy rates, SSA took several steps to increase public

access, including doubling the trained and available workforce to answer

800-number calls. Primarily as a result of the workforce increase, the busy

rate decreased from 49 to 34 percent, and the percentage of calls that were

answered within 5 minutes increased from 74 to 83 percent from fiscal

years 1995 to 1996.10

DLA shows how an agency can make substantial improvements in its

service to its customers and inventory management. DLA’s reengineering

efforts for its delivery and supply processes focused not only on saving

time but also on lowering costs, two results valued by its customers. DLA

focused on improved partnerships between suppliers and its customers,

primarily DOD facilities. According to DLA, this overall focus enabled it to

reduce the delivery time of nearly all medical supplies from 30 days in

1993 to within the current 24 hours; at the same time, it reduced prices by

35 percent. DLA also reported that it is providing Internet ordering of

primarily electronics parts, which took from 35 to 40 days from purchase

to receipt prior to 1996 and now takes from 3 to 5 days, with an estimated

savings of $500 million over 10 years. DLA further reported that it avoided

inventory holding and storage costs of $95.7 million in fiscal year 1995,

with a projection of an additional $353 million over the next 5 years. DLA’s

initiatives have had departmentwide impact. We estimated in March 1997

that between September 1991 and September 1996, DOD reduced its

9

GAO/GGD-96-118, June 1996.

10Social Security Administration: Significant Challenges Await New Commissioner (GAO/HEHS-97-53,

Feb. 20, 1997).

Page 45 GAO/GGD-97-109 Results Act
Chapter 2 

Progress of the Results Act Implementation

Has Been Mixed

pharmaceutical, medical, and surgical inventories and associated

management costs by about $714 million.11

The Food and Drug Administration (FDA) is allowed to collect user fees

from drug companies seeking approval to market drugs under the

Prescription Drug User Fee Act of 1992. The law dedicates the revenues to

expediting FDA’s reviews of human drug applications and incorporated

time-specific performance goals that FDA is to meet by the end of fiscal

year 1997. To satisfy these objectives, FDA consulted with its stakeholders

to determine appropriate performance measures and target levels of

performance and developed output-oriented performance goals. FDA

reported that the program had exceeded its performance goals, improving

the speed and efficiency of the drug review process.12 Specifically, the

median time taken to approve 53 applications for human drugs that have

not been marketed in the United States before was 14.3 months in fiscal

year 1996. In comparison, in 1993, which was the beginning of the

program, FDA approved only 25 applications, and the median approval time

was about 23 months.

The U.S. Mint, as one part of its mission, manufactures and markets

numismatic and commemorative coins and products. Before 1994, the

Mint estimated it took from 8 weeks to about 27 weeks to fill customers’

orders for coins. Because of the number of customer inquiries about the

status of their orders, the Mint learned that the time it took to fill orders

was a problem. Subsequently, the Mint set goals and reduced the time to

fill orders. As of January 1997, 97 percent of the orders were filled within 4

weeks. Also, the time for customers to get replacements has

declined—from 47 to 7 days—from October 1995 to October 1996.

Managers Reported

Results-Oriented

Performance

Measures and Their

Use Was Limited

Although agencies have reported selected instances where they have used

results-oriented performance information to make important

improvements in performance, federal managers reported to us that

overall, results-oriented performance measures often were not available,

and such information was not frequently used to help make important

decisions affecting their programs and agencies. According to our survey,

although 76 percent of federal managers governmentwide reported that

the programs, operations, or projects they were involved in had

11Defense Inventory Management: Problems, Progress, and Additional Actions Needed

(GAO/T-NSIAD-97-109, Mar. 20, 1997).

12Department of Health and Human Services: Management Challenges and Opportunities

(GAO/T-HEHS-97-98, Mar. 18, 1997).

Page 46 GAO/GGD-97-109 Results Act
Chapter 2 

Progress of the Results Act Implementation

Has Been Mixed

performance measures, such measures typically were not results oriented.

That is, the measures failed to help answer an essential question, “Is this

program accomplishing its intended results and contributing to the

achievement of the agency’s strategic goals and mission?”

Managers Reported

Limited Results-Oriented

Performance Measurement

In crafting the Results Act, Congress recognized that congressional and

agency decisionmaking often had been hampered by the absence of sound

information on the results of federal programs. In the past, to the extent

performance information was collected, agencies generally collected

information primarily on such things as the amount of dollars received and

spent, staffing levels, and the number of activities performed. Although

such information is valuable for program management and accountability,

it does not present a complete picture of the results of the federal effort.

Our more recent work has confirmed that results-oriented performance

information still does not exist as often as it should. For example, the

Department of Housing and Urban Development’s (HUD) Public Housing

Management Assessment Program has a system that collects data, such as

outstanding work orders and uncollected rents, to measure the

performance of housing authorities. However, the system does not collect

data on housing quality or the quality of maintenance, which are essential

for assessing the results that housing authorities are achieving and for

determining which housing authorities are performing well or poorly.13

Our survey of federal managers suggests that these are not isolated

occurrences of a lack of performance information in federal agencies.

Table 2.1 shows that, according to our survey, 32 percent of federal

managers said that, to a great or very great extent, they have the types of

performance measures that would demonstrate whether their programs or

operations were achieving their intended result. The table also shows that

38 percent or less of federal managers reported having, to a great or very

great extent, other important performance-related information, such as

efficiency and quality measures.

13Public Housing: HUD Should Improve the Usefulness and Accuracy of its Management Assessment

Program (GAO/RCED-97-27, Jan. 29, 1997).

Page 47 GAO/GGD-97-109 Results Act
Chapter 2 

Progress of the Results Act Implementation

Has Been Mixed

Table 2.1: Percent of Federal Managers

Who Reported Their Programs Have

Various Performance Measures and

Recollect They Had Such Measures 3

Years Ago

Percent of federal managers reporting to a

great or very great extent they had or have

these types of measures

Measures that

3 years

ago Currently Differencea

would demonstrate to someone

outside their agencies whether or not

they are achieving their intended

results

19 32 +13

tell whether or not they are satisfying

their customers

11 32 +21

tell about the quality of the products or

services they provide

19 31 +12

tell how many things they produce or

services they provide

27 38 +11

tell if they are operating efficiently 17 26 + 9

aAll differences are statistically significant.

Source: GAO survey data.

Despite the lower percentage of federal managers reporting that their

programs or operations have performance-related measures to a great or

very great extent, there where statistically significant increases in the

percent of federal managers reporting the existence of these types of

measures over what they recollected the situation to have been 3 years

ago. For example, when asked to recollect what the situation was 3 years

ago, 19 percent of federal managers reported that, to a great or very great

extent, they had results-oriented measures, compared to 32 percent who

reported that they had such measures today. This represents a

13 percentage point change over what federal managers perceived the

situation to have been 3 years ago, suggesting that results-oriented

performance information, which is essential to the success of the Results

Act, is becoming more widely available.

According to federal managers’ responses to our survey, it appears that

agencies are making the greatest change in developing measures for

assessing customer satisfaction. When asked to recollect what the

situation was 3 years ago, 11 percent of federal managers governmentwide

reported that, to a great or very great extent, they had measures for

assessing customer satisfaction. However, 32 percent of federal managers

reported that, to a great or very great extent, they currently had such

measures. This represents a 21 percentage point change. This reported

Page 48 GAO/GGD-97-109 Results Act
Chapter 2 

Progress of the Results Act Implementation

Has Been Mixed

change in agencies’ efforts to measure customer satisfaction may be a

reflection of agencies’ efforts to comply with Executive Order No. 12862,

September 1993, which required agencies to establish and implement

customer service standards to guide their operations.

Managers Reported

Limited Use of

Results-Oriented

Performance Information

Given the limited reported availability of results-oriented information, it is

not surprising that federal managers also reported limited use of such

information to make decisions affecting their programs and agencies, as

shown in table 2.2.

Table 2.2: Percent of Federal Managers

Reporting Extent to Which

Results-Oriented Performance

Information Is Used Currently and

Compared to 3 Years Ago

Percent of federal managers reporting use

to a great or very great extent

Results-oriented performance

information

3 years

ago Currently Difference

used by the agency to help develop its

budget

16 21 +5a

used as the basis for funding decisions 14 20 +6a

used as the basis for legislative changes 9 13 +4b

used as the basis for program changes

by management above respondent

12 16 +4a

aStatistically significant.

bApproaches statistical significance.

Source: GAO survey data.

Table 2.2 shows that federal managers reported some improvements over

what they recollected the situation to have been 3 years ago. Although

these improvements are statistically significant for most of the uses shown

in the table, these improvements have been modest.

The reported use of results-oriented performance information to help

make key decisions was not high even among federal managers who

reported that they had performance measures that demonstrate their

programs are achieving intended results, to a great or very great extent. At

most, 37 percent these federal managers reported that results-oriented

performance information was used to a great or very great extent. Table

2.3 shows these results.

Page 49 GAO/GGD-97-109 Results Act
Chapter 2 

Progress of the Results Act Implementation

Has Been Mixed

Table 2.3: Percent of Federal Managers

Reporting Use of Performance

Information to a Great or Very Great

Extent Among Managers With

Performance Measures Demonstrating

Achievement of Results to a Great or

Very Great Extent

Results-oriented performance information

Percent of federal

managers reporting use to

a great or very great

extenta

used by the agency to help develop its budget 37

used as the basis for funding decisions 34

used as the basis for legislative changes 21

used as the basis for program changes by management

above respondent

31

aAll percentages of use are statistically significant and higher compared to respective

percentages of use for all other managers.

Source: GAO survey data.

Conclusions The Results Act performance planning and reporting pilot phase provided

many agencies with important experience in meeting key requirements of

the Act. These experiences, along with OMB’s efforts to work with agencies

on the Act, suggest that most agencies are likely to meet the upcoming

statutory deadlines for producing initial strategic plans and annual

performance plans. More importantly, the experiences of the pilot

agencies and related efforts by nonpilot agencies show that substantial

improvements in performance are possible when organizations adopt a

disciplined approach to setting goals, measuring performance, and using

performance information to improve effectiveness.

However, the pilot phase also underscored how far organizations still have

to progress in the development and use of results-oriented performance

information. Although there has been progress over the last 3 years,

agencies still have not developed the information necessary to determine

whether their programs are accomplishing their intended results. Even in

those instances where agencies have results-oriented information,

agencies are generally not using the information to a great extent to make

decisions affecting their programs. The subsequent chapters will detail

that agencies face a number of formidable challenges in establishing clear

missions and results-oriented goals, measuring their performance, building

the organizational cultures needed to sustain a focus on results, and

linking performance plans to the budget process.

Page 50 GAO/GGD-97-109 Results Act
Chapter 3 

Unclear Agency Missions and Goals Will

Challenge Results Act Implementation

Reaching a reasonable degree of consensus on coordinating the missions

and goals of individual agencies that are involved in crosscutting program

areas and balancing individual agencies’ multiple strategic goals can raise

contentious policy issues that may not be easily resolved. The Results Act,

with its focus on results and requirement for consultation, can help

Congress and the executive branch clarify over the long term these

management issues confronting federal agencies.

Our work has shown that Congress’ and the executive branch’s approach

to addressing national issues has, in many cases, resulted in overlap and

fragmentation in federal program efforts. Coordinating these program

efforts will take time and may, in some instances, require legislative

action. Moreover, many agencies confront competing demands that are a

natural by-product of the complex social and political environment in

which programs are created, funded, and managed. Balancing these

demands will continue to be an ongoing process for federal agencies.

Efforts under the Results Act will not answer questions about whether the

crosscutting program efforts should be consolidated or reorganized or

about the best balance for competing priorities. Those are policy decisions

that should appropriately consider many factors, including program

performance. However, congressional consultations on agencies’ strategic

plans provide an ongoing opportunity for Congress and the executive

branch to work together to identify the extent and potential consequences

of overlap and fragmentation in federal program efforts and to identify the

performance trade-offs associated with striking different balances of

competing demands.

Crosscutting Federal

Programs Pose

Challenge for Results

Act Implementation

Although federal programs have been designed for different purposes or

targeted for different population groups, coordination among federal

programs with related responsibilities is essential to efficiently and

effectively meet national concerns. Uncoordinated program efforts can

waste scarce funds, confuse and frustrate program customers, and limit

the overall effectiveness of the federal effort. A focus on results, as

envisioned by the Results Act, implies that federal programs contributing

to the same or similar results should be closely coordinated to ensure that

goals are consistent, and, as appropriate, program efforts are mutually

reinforcing. This means that federal agencies are to look beyond their

organizational boundaries and coordinate with other agencies to ensure

that their efforts are aligned. Although some initiatives are under way in

Page 51 GAO/GGD-97-109 Results Act
Chapter 3 

Unclear Agency Missions and Goals Will

Challenge Results Act Implementation

this regard, fully coordinating federal programs will take time and require

sustained attention.

Our work has shown that as the federal government has responded over

time to new needs and problems, many federal agencies have been given

responsibilities for addressing the same or similar national issues.1

 Of the

18 federal missions or areas of national need described as budget

functions, 14 were addressed by more than 1 executive branch department

or major agency in fiscal year 1996, such as health, international affairs,

and justice. Six of the national missions, including education, income

security, and commerce and housing credit were addressed by six or more

executive branch departments and major agencies. Some of this shared

responsibility was intended to recognize that addressing some issues from

a national perspective would necessarily involve more than one federal

agency or more than one approach. For example, we have reported that

multiagency initiatives to address environmental concerns recognize that

many programs can have an impact on the environment.2

However, in many program areas—such as in food safety, employment

training, early childhood development, at-risk and delinquent youth

programs, federal land management, and national laboratories—our work

suggested that significant overlap and fragmentation existed in the federal

response to national needs and problems.3

 For example, in 1995 we

testified on the Department of Education’s programs that provided loans

and grants to students to help finance their higher education.4

 We found

that although the student loan and Pell grant programs provided the

majority of federal financial aid to students for postsecondary education,

another 22 smaller programs were targeted to specific segments of the

postsecondary school population, such as prospective students from

disadvantaged families or women and minorities who are

underrepresented in graduate education. These 22 programs were

1

Budget Issues: Fiscal Year 1996 Agency Spending by Budget Function (GAO/AIMD-97-95, May 13,

1997); and Government Restructuring: Identifying Potential Duplication in Federal Missions and

Approaches (GAO/T-AIMD-95-161, June 7, 1995).

2

Restoring the Everglades: Public Participation in Federal Efforts (GAO/RCED-96-5, Oct. 24, 1995); and

Ecosystem Management: Additional Actions Needed to Adequately Test a Promising Approach

(GAO/RCED-94-111, Aug. 16, 1994).

3

Government Reorganization: Issues and Principles (GAO/T-GGD-AIMD-95-166, May 17, 1995); At-Risk

and Delinquent Youth: Multiple Federal Programs Raise Efficiency Questions (GAO/HEHS-96-34, Mar.

6, 1996); Federal Land Management: Streamlining and Reorganization Issues (GAO/T-RCED-96-209,

June 27, 1996); and Federal R&D Laboratories (GAO/RCED/NSIAD-96-78R, Feb. 29, 1996).

4

Department of Education: Information on Consolidation Opportunities and Student Aid

(GAO/T-HEHS-95-130, Apr. 6, 1995); and Department of Education: Opportunities to Realize Savings

(GAO/T-HEHS-95-56, Jan. 18, 1995).

Page 52 GAO/GGD-97-109 Results Act
Chapter 3 

Unclear Agency Missions and Goals Will

Challenge Results Act Implementation

collectively funded at $1.1 billion for fiscal year 1995. We concluded that

these smaller grant programs may be considered candidates for

consolidation—with other larger programs or among themselves—with no

adverse impact on students’ access to postsecondary education and that

the federal government could anticipate administrative savings of 10

percent each year, or a total of $550 million in budget authority (adjusted

for inflation) over 5 years.5

Similarly, we reported in 1995 on the federal programs that assisted urban

communities and their residents through a complex system involving

multiple federal agencies.6

 At that time, the Catalog of Federal Domestic

Assistance listed 342 economic development-related programs

administered by 13 agencies.7

 These programs were in the areas of

housing, economic development, and social services. We reported that

considered individually, many of these programs may have made sense.

However, the proliferation of federal programs and the lack of

coordination among agencies could impose a burden on local

organizations that attempted to piece together programs to serve their

communities. In particular, the neighborhood organizations we studied

found it burdensome to manage multiple programs with individual funding

streams, application requirements, and reporting expectations.

Efforts Are Under Way to

Coordinate Crosscutting

Programs

In recent years, a number of efforts have been undertaken to coordinate

federal programs to help ensure that national needs are being effectively

targeted. These efforts—some of which were started before the Results

Act—have shown that coordinating crosscutting programs takes time and

requires sustained attention. Moreover, because of the statutory bases of

crosscutting programs, congressional involvement is often needed to

integrate the federal response to national needs.

Within the executive branch, federal agencies have initiated several efforts

to address crosscutting issues. As of March 1997, about 25 interagency

groups had been formed to discuss common concerns in crosscutting

issues, including the Federal Credit Policy Working Group, the Research

5

Addressing The Deficit: Budgetary Implications of Selected GAO Work for Fiscal Year 1998

(GAO/OCG-97-2, Mar. 14, 1997).

6

Economic Development Programs (GAO/RCED-95-251R, July 28, 1995); and Community

Development: Comprehensive Approaches and Local Flexibility Issues (GAO/T-RCED-96-53, Dec. 5,

1995).

7

The Catalog of Federal Domestic Assistance is a governmentwide compendium of federal programs,

projects, services, and activities coordinated by OMB and compiled by the General Services

Administration.

Page 53 GAO/GGD-97-109 Results Act
Chapter 3 

Unclear Agency Missions and Goals Will

Challenge Results Act Implementation

Roundtable, and the Interagency Regulatory Reinvention Forum, among

others.8

 Typically, these groups were sponsored by a lead agency and

provided a forum for agency officials to discuss informally a wide range of

concerns, such as goal-setting and performance measures, but not full

coordination of program efforts. The Federal Credit Policy Working Group

worked on appropriate goals and performance measures for credit

programs. The Research Roundtable produced a paper on the challenges

of developing and reporting performance measures for research programs.

The Interagency Regulatory Forum discussed common concerns among

regulatory agencies, including identifying customers and conducting

customer service surveys. Although these agency forums have provided an

important opportunity for agencies to work together to address common

concerns in goal-setting and performance measurement, they have not

generally attempted to coordinate crosscutting program efforts. Thus,

despite these initiatives, during its review of major portions of agencies’

strategic plans in the summer of 1996, OMB concluded that on the whole,

there was little sign of significant coordination among agencies.

Addressing the overlap and fragmentation in federal program efforts often

requires congressional involvement. In particular, legislative action can

ensure that the goals and measures of existing programs are coordinated

to meet federal needs. For example, Congress incorporated Healthy

People objectives for the year 2000 into national legislation. Started in

1979, Healthy People is a series of outcome-based public health objectives

and measures developed and updated each decade by the U.S. Public

Health Service in consultation with other federal agencies, state

governments, and national organizations. The year 2000 objectives

succeeded the 1990 health objectives that were set in 1980. The Public

Health Service has started planning for the 2010 health objectives. Over

time, Congress has required three federal programs to incorporate Healthy

People objectives for the year 2000 into their goals; and in one of those

programs, it has linked resource allocations to Healthy People objectives.

The first, in 1989, was the requirement that the Maternal and Child Health

Block Grant Program, Title V of the Social Security Act, report on the

progress made by states toward accomplishing Healthy People objectives

on maternal and child health. The Indian Health Care Improvement Act, as

amended in 1992, created an annual reporting requirement for the Indian

Health Service to inform Congress on the health status of American

Indians and Alaska Natives. In addition, the 1992 authorization of the

8

Information provided by Carl J. Metzger, Management Systems International, Inc., Washington, D.C.,

March 1997.

Page 54 GAO/GGD-97-109 Results Act
Chapter 3 

Unclear Agency Missions and Goals Will

Challenge Results Act Implementation

Preventive Health and Health Services Block Grant program linked the

state grant-funded activities to Healthy People objectives.

However, the mere existence of a legislated coordinating mechanism does

not ensure success, and sustained effort is required to build consensus on

common goals and how best to coordinate approaches. For example, the

Office of National Drug Control Policy (ONDCP), established in 1988 by law,

is responsible for producing the national drug control strategy and

coordinating its implementation with other federal agencies. Given the

complexity of drug control efforts and the fragmentation of the approach

to the national drug control strategy among more than 50 federal agencies,

we have endorsed the need for a central planning agency, such as ONDCP,

to coordinate the nation’s efforts and recommended its reauthorization for

a finite period of time.9

 However, our work has shown that despite some

successes, international drug control efforts have not materially reduced

the availability of drugs in the United States.10 One reason was a lack of

ways to tell whether or how well counternarcotics efforts were

contributing to the goals of the national strategy. Consistent with the

intent of the Results Act, we recommended that ONDCP complete a

long-term plan with meaningful performance measures and multiyear

funding needs that were linked to the goals and objectives of the

international drug control strategy. Such a plan would provide managers

and policymakers with the information to make more informed decisions

on prioritizing funding levels based on performance and results. ONDCP

concurred with our recommendation. In February 1997, ONDCP proposed a

10-year strategy with five major goals—including “break foreign and

domestic drug sources of supply”—as the basis for a long-term national

effort and stated that it will continue to work on a 5-year drug control

budget concept. ONDCP is making progress toward developing performance

targets and measures for each of its goals.

Balancing Goals Is

Complicated by

Competing Demands

Federal agencies often face a variety of competing priorities that force

congressional and executive branch decisionmakers to balance public

expectations, cost, quality, and other factors. These multiple priorities,

which in many cases are built into the intent and design of an agency or

program, constitute one of the primary challenges to public sector

9

Drug Control: Reauthorization of the Office of National Drug Control Policy (GAO/T-GGD-97-97, May

1, 1997); and Drug Control: Observations on Elements of the Federal Drug Control Strategy

(GAO/GGD-97-42, Mar. 14, 1997).

10Drug Control: Long-Standing Problems Hinder U.S. International Efforts (GAO/NSIAD-97-75, Feb. 27,

1997).

Page 55 GAO/GGD-97-109 Results Act
Chapter 3 

Unclear Agency Missions and Goals Will

Challenge Results Act Implementation

goal-setting because priorities continually need to be balanced; otherwise,

one or two priorities may be inappropriately overemphasized at the

expense of others.

According to our survey, nearly half of the federal managers reported that

reconciling differing congressional views and the views of other parties on

an agency’s mission and strategic goals has been, or will be, somewhat to

very difficult. A greater proportion of federal managers from the Results

Act pilots that we were able to isolate from our survey reported difficulty,

suggesting that as agencies gain experience with the Results Act they will

come to better appreciate the need for, and difficulty of, reconciling

competing views. Fifty-eight percent of managers in selected Act pilots

reported that reconciling differing congressional views has been or will be

somewhat to very difficult, compared to 46 percent of federal managers in

all other agencies. Similarly, 59 percent of managers in selected Results

Act pilots reported that reconciling the views of other parties has been or

will be somewhat to very difficult, compared to 48 percent of federal

managers in all other agencies. The differences between the managers in

selected Results Act pilots and federal managers in all other agencies for

reconciling differing congressional views and the views of other parties

are statistically significant.

We have identified numerous examples of agencies that can benefit from

using the Results Act to help them better balance competing priorities and

thereby improve their effectiveness. For example, we reported in

April 1997 that the Forest Service had been increasingly shifting the

emphasis of its efforts from producing timber to sustaining wildlife.11 This

shift was taking place in reaction to requirements in planning and to

environmental laws and their judicial interpretation—reflecting changing

public values and concerns—together with social, ecological, and other

factors. However, we noted the demand for recreation is also expected to

grow and may increasingly conflict with sustaining wildlife and producing

timber. We found that the disagreement both within the Forest Service and

among key external stakeholders, including Congress, on how the Forest

Service is to resolve conflicts or make choices among competing uses on

its lands had seriously undermined its ability to establish the goals and

performance measures needed to ensure its accountability. Until general

agreement is reached, we believe the Forest Service’s decisionmaking will

continue to be inefficient and ineffective.

11Forest Service Decision-Making: A Framework for Improving Performance (GAO/RCED-97-71,

Apr. 29, 1997).

Page 56 GAO/GGD-97-109 Results Act
Chapter 3 

Unclear Agency Missions and Goals Will

Challenge Results Act Implementation

Our ongoing work examining federal credit agencies identified multiple

competing priorities as a challenge in setting results-oriented goals for

those programs. Agency officials told us that on the one hand, they are

expected to increase program service while also reducing program costs

and minimizing default rates. However, these programs were designed to

offer credit to a population that the private sector would consider

high-risk. For example, to be eligible for a rural housing loan, the borrower

must be ineligible to get credit from any other source. Therefore, the target

population may be more likely to default on a loan.

The need to identify and balance competing demands can be especially

complicated in cases where an agency lacks an integrated legislative

mission. For example, the Environmental Protection Agency (EPA) was

established in 1970 under a presidential reorganization plan in response to

concerns over protecting public health as well as air, water, and land

resources. Since then, as environmental threats were identified, Congress

gave EPA responsibility for implementing over a dozen environmental laws.

We reported in May 1995 that because it did not have an overarching

legislative mission and its environmental responsibilities had not been

integrated, EPA had not been able to target its resources to reduce the

greatest risks to human health and the environment.12 EPA has been

working with state governments, industry, environmental groups, and

other interested parties since 1992 to select the most appropriate national

environmental goals. Building on these goals, EPA is also ranking the

relative risk of environmental problems to help identify the most

appropriate goals for the agency.

Congressional

Consultations Provide

Mechanism for

Identifying and

Addressing

Crosscutting Issues

and Seeking to

Balance Priorities

The Results Act requirement that agencies consult with Congress in

developing their strategic plans presents an important opportunity for

congressional committees and the executive branch to work together to

identify the extent and the potential consequences of overlap and

fragmentation of federal program efforts.13 As agencies work with

congressional committees on developing, and subsequently updating, their

strategic plans, they can identify legislative changes that may be needed to

clarify congressional intent as well as legislative adjustments to better

ensure an effective, coordinated response to national issues. The

consultation process should also be helpful to agencies in addressing key

management issues, such as identifying multiple priorities, reaching a

12Environmental Protection: Current Environmental Challenges Require New Approaches

(GAO/T-RCED-95-190, May 17, 1995).

13GAO/T-GGD-96-79, March 6, 1996.

Page 57 GAO/GGD-97-109 Results Act
Chapter 3 

Unclear Agency Missions and Goals Will

Challenge Results Act Implementation

workable agreement on priorities, and then establishing results-oriented

goals to reflect the balance struck among those priorities. The requirement

for consultations is also perhaps the Results Act’s most significant

challenge, because consultations will likely raise contentious policy issues

that are inherent in the political process.

We testified in February 1997 that although no single set of best practices

has yet emerged for the Results Act consultation process, making

consultations most useful would require that both Congress and agencies

be actively engaged in the effort.14 On the whole, the House committee

staff we spoke with stressed the very limited nature of the meetings with

agencies that had taken place through February 1997. Rather than

consultations, House staff characterized the meetings as briefings,

preconsultations, or preliminary consultations. As of April 30, 1997,

however, some House Majority committee staff we spoke with had

expressed concern that in their view, not enough agencies had provided

draft strategic plans for consultations. In particular, a number of agencies

have told congressional staff that draft plans will not be available until

June or early July. Congressional staff have said that an important

opportunity for Congress and the administration to clarify and seek

agreement on agencies’ missions, goals, and strategies will be lost if draft

plans are not provided early enough to permit meaningful reviews and

consultations during the consultation process. For example, although EPA

provided a detailed outline of its strategic plan, including a mission

statement, goals, and strategic “principles,” completing a draft plan in

early July may not leave adequate time for EPA to obtain congressional

input and fully consider and incorporate congressional views before the

final plans are due in September 1997.

Although the Results Act’s required consultation can help improve the

management of federal agencies, it will have a less direct role in resolving

difficult policy choices, because such choices should appropriately

consider many factors, including program performance. For example, we

have observed that transferring weapons production from the Department

of Energy to DOD, as was proposed by some, would require careful

consideration of many policy and management issues.15 In a survey of

former Department of Energy executives and experts on energy policy,

some argued that because of the declining strategic role of nuclear

14Managing For Results: Enhancing the Usefulness of GPRA Consultations Between the Executive

Branch and Congress (GAO/T-GGD-97-56, Mar. 10, 1997).

15Department of Energy: Observations on the Future of the Department (GAO/T-RCED-96-224, Sept. 4,

1996).

Page 58 GAO/GGD-97-109 Results Act
Chapter 3 

Unclear Agency Missions and Goals Will

Challenge Results Act Implementation

weapons, DOD might be better able to balance resource allocations among

nuclear and other types of weapons if the weapons complex were

completely under its control. Others argued, however, that the need to

maintain civilian control over nuclear weapons outweighs any other

advantages.

The Results Act can assist decisionmakers in making policy choices by

providing data on the management and performance implications of

different options. For example, data generated as part of the Results Act

can identify potential performance improvements that might result from

consolidating programs or shifting responsibilities among agencies.

Congress and the executive branch can then weigh this information

against policy considerations. Our work suggests that ultimately a

successful strategic planning effort will often reflect hard choices by the

agency, and there may be disagreement between the agency and key

stakeholders about particular goals or the means by which the goals will

be achieved. In this regard, an OMB official has noted that a strategic plan

that has achieved complete agreement among all interested parties is

likely to be at such a high level of generality that the usefulness of the plan

as a decisionmaking tool would be fairly limited.

Conclusions The Results Act, with its focus on results and requirement for

consultation, can help Congress and the executive branch address the

management implications of two of the seemingly intractable issues

confronting federal agencies: overlap and fragmentation in program efforts

that cut across agencies and the need to balance multiple priorities within

individual agencies. However, although the Results Act provides a

potentially effective vehicle for addressing these issues, their existence

also makes the prospects for the effective governmentwide

implementation of the Act uncertain in the near term. Efforts under the

Results Act can help identify the extent and consequences of the overlap

and fragmentation, but addressing such problems will likely be a lengthy

process, because Congress and the executive branch will have to consider

policy trade-offs to reach a reasonable degree of consensus on the

appropriate federal response to national needs. Similarly, balancing

multiple priorities will need to be an ongoing process, because priorities

change as new needs arise.

The strategic planning consultation process is a starting point for Congress

and the executive branch to work through these different policy issues. If

successfully implemented, the Results Act will be most helpful to

Page 59 GAO/GGD-97-109 Results Act
Chapter 3 

Unclear Agency Missions and Goals Will

Challenge Results Act Implementation

decisionmakers in providing performance information that identifies the

management implications of different structures and policy choices among

competing demands.

Page 60 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

Agencies have encountered some difficult analytic and technical

challenges in developing useful results-oriented performance information

to use in planning their efforts and gauging progress toward the

achievement of their goals and missions. There are two primary

considerations for agencies in their development of such information. The

first is to identify those performance measures that will be meaningful for

the agency to use in gauging progress in achieving its goals. The other is to

collect reliable and valid data on those measures so that program

managers and other decisionmakers have the data they need to effectively

manage federal programs.

However, several factors—including a lack of data on results-oriented

performance measures; the influence of external forces, such as emerging

economic, social, and technological trends and the role that third parties,

such as state and local governments, may have in determining whether

program results are achieved; and the long time frames sometimes needed

before the results of agency or program actions can be seen—have

complicated agencies’ ability to set goals and identify results-oriented

performance measures. Agencies also are challenged in their collection of

valid and reliable data. Reasons for this challenge include different parties

using different definitions to measure performance and agencies’ reliance

on outside parties to provide results-oriented performance data. Some

agencies are exploring approaches for addressing the difficulty they have

had in developing useful results-oriented performance information.

Several Factors Have

Complicated

Agencies’ Ability to

Set Goals and Identify

Results-Oriented

Performance

Measures

Several factors have complicated the degree to which agencies have faced

challenges in setting goals and identifying results-oriented performance

measures. These factors include a lack of data on results-oriented

performance measures; the influence of external forces, such as emerging

economic, social, and technological trends and the role that third parties,

such as state and local governments, may have in determining whether

program results are achieved; and the long time frames sometimes needed

before the results of agency or program actions can be seen.

Agencies’ Lack of Data

Complicates Efforts to Set

Goals

Our work has shown that baseline and trend data on past performance can

help agencies set realistic goals for their programs given the past

performance of those programs. We have previously reported that prior

efforts to link resources to results were hampered by the absence of

Page 61 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

systems to collect needed data.1 Under PPBS, agencies that attempted to

gather results-oriented performance data found the process to be far more

difficult than expected. Agency officials reported that developing the

information and collection systems envisioned by PPBS would take several

years.

Because agencies often did not focus on having results-oriented

performance information in the past, they generally have not collected

such data. Therefore, they do not have all of the baseline and trend data

needed to set goals. For example, our reviews of the National Park

Service—whose mission is, in part, to preserve and protect park

resources—showed that baseline information about natural and cultural

resources was frequently incomplete or nonexistent.2

 Consequently, it was

difficult for Park Service officials to determine whether the best

management decisions about those resources were being made. For

example, for 70 years, the Service has stocked nonnative fish in various

lakes and waterways in Yosemite National Park. However, officials said

they knew little about the types or numbers of fish in the lakes and

waterways, as well as other species inhabiting the park, including birds,

badgers, river otters, wolverines, and red foxes. By not collecting data on

those resources, officials realized that some of their decisions about those

resources were not effective in helping the Park Service achieve intended

results. At the time of our review, nonnative fish outnumbered native

rainbow trout by a 4-to-1 margin, and the stocking had reduced the

numbers of at least one federally protected species.

VA officials said that some of the results-oriented measures for their

agency’s Loan Guaranty program were new and baseline data were not

available on those measures. Consequently, VA did not have data on past

performance to use in setting some of the program’s fiscal year 1998 goals.

In some of these cases, VA indicated in its fiscal year 1998 budget

submission to OMB that those goals were “to be determined.” For example,

according to VA, a key result of the program is providing veterans with

timely service in obtaining a VA guaranteed loan. Performance measures

include the percentage of veterans satisfied with the time it took to

process their loans and the average time taken to process loans. Although

VA set fiscal year 1998 goals for customer satisfaction with timeliness, VA

1

GAO/AIMD-97-46, March 27, 1997.

2

National Parks: Park Service Needs Better Information to Preserve and Protect Resources

(GAO/T-RCED-97-76, Feb. 27, 1997); and Park Service: Managing for Results Could Strengthen

Accountability (GAO/RCED-97-125, Apr. 10, 1997).

Page 62 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

reported that goals for average processing time would need to be

determined.

In another example, U.S. Department of Agriculture (USDA) officials said

they dropped some performance measures that had been part of their

Single Family Housing Results Act pilot’s annual performance plan

because they did not have information systems to collect data on those

measures.3

 Such measures included those that would have provided

performance data on the number of loans made in targeted geographic

areas and the number of houses upgraded from substandard to safe and

sanitary in targeted areas. Lacking these data, USDA did not have an

informed basis on which to set goals.

In recent years, Congress has enacted landmark information technology

legislation—the Paperwork Reduction Act of 1995 and the Clinger-Cohen

Act of 1996—that, if successfully implemented, holds a key to ensuring

that agencies put in place the systems needed to collect results-oriented

performance data. These acts establish an investment framework for

having agencies better plan and manage their technology efforts and link

those efforts directly to the achievement of agency program goals and

mission.4

Agencies’ Results Can Be

Influenced by External

Forces

The efforts of federal agencies often are but one factor among many

external forces that may influence whether, and the degree to which, their

programs achieve their intended results. Many agencies have been

challenged to separate out the influence that program activities have had

on the achievement of program results when those results also could have

been influenced by external forces. This challenge has complicated

agencies’ efforts to identify those performance measures that will be

meaningful for the agency to use in gauging progress in achieving its goals.

Identifying meaningful performance measures has been a long-standing

problem for federal agencies. We have reported that agency officials

implementing the PPBS initiative, which was mandated governmentwide by

President Johnson in 1965, found it far more difficult than they expected

3

Information systems are a discrete set of information resources and processes, automated or manual,

organized for the collection, processing, maintenance, use, sharing, or dissemination of information.

See GAO/AIMD-94-115, May 1994.

4

For further information see: Information Technology Investment: Agencies Can Improve

Performance, Reduce Costs, and Minimize Risks (GAO/AIMD-96-64, Sept. 30, 1996); Information

Management Reform: Effective Implementation Is Essential for Improving Federal Performance

(GAO/T-AIMD-96-132, July 17, 1996); and GAO/AIMD-94-115, May 1994.

Page 63 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

to relate program activities to a stated result or to separate out other

influences that might affect intended results.5

 For example, under the PPBS

initiative, the Upward Bound program was designed to increase skills and

motivation for low-income high school students. Agency officials had no

way to isolate the program’s effect from other environmental influences

that might also have contributed to the success or failure of program

participants.

The influence that external forces can have on the achievement of

program results has continued to challenge agencies’ ability to identify

meaningful measures. Our work looking at the efforts of economic

development and our forthcoming report on regulatory programs

illustrates this challenge. For each of these types of programs, isolating the

federal contribution to the achievement of an intended result has been

exceedingly difficult and accordingly has hindered agency efforts to

identify meaningful performance measures. In situations where the federal

program effort is but one factor among many external forces that may

determine the degree to which an intended result is achieved, determining

the level of federal influence on such achievement requires agencies to

understand and measure the nonfederal influence.

For example, in 1995 we reported on the existence of 342 economic

development-related programs.6

 Numerous external forces, including

broad national economic trends, contribute to local economic

development, and communities also may receive assistance from state and

local governments and the private sector. Therefore, determining the

impact of one of these programs on economic development has been a

daunting task. A persuasive study of a program’s impact, as we observed in

our 1996 report,7

 would require three elements. First, it would have to

document that there had been some improvement in a targeted area;

second, it would have to link specific program elements to actual

economic changes; and third, it would have to measure the growth

stemming from other influences on the economy of the targeted area in

order to isolate the impact that could be attributed to the economic

development program.

Although such analysis entails a substantial undertaking, it can potentially

provide important information for decisionmakers. As part of that 1996

5

GAO/AIMD-97-46, March 27, 1997.

6

GAO/RCED-95-251R, July 28, 1995.

7

Economic Development: Limited Information Exists on the Impact of Assistance Provided by Three

Agencies (GAO/RCED-96-103, Apr. 3, 1996).

Page 64 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

review, we reported that one study of the effectiveness of the Department

of Commerce’s Economic Development Administration (EDA) programs

found that income in the counties that received EDA funding grew

significantly faster than income in the counties that received no aid.

However, when the researchers simultaneously considered EDA’s programs

and factors unrelated to EDA, they found that EDA’s programs had a very

small effect on income growth rates during the period that the aid was

received and had no significant effects in the 3 years after the aid ceased.

EDA’s programs could explain only a small part of the difference in the

growth rates between the two groups of counties.

Regulatory agencies also have had difficulty sorting out the interaction

that external forces have had on the results that those agencies were

trying to achieve and accordingly have had difficulty identifying

meaningful performance measures. For example, OSHA officials said they

were having difficulty identifying measures to assess program results

because many other factors, such as business cycles and the development

of safer machines, affect the number of workplace injuries each year more

than OSHA’s actions do. In addition, a senior OSHA official said that worker

safety data are sometimes influenced by catastrophic events, such as

bombings or airplane crashes over which OSHA has no control.

Conversely, the less that program results are affected by external forces,

the greater will be the agency’s influence in accomplishing its intended

results. Such agencies also will tend to have an easier time identifying

results-oriented performance measures that demonstrate the extent to

which the agency achieved its goals. For example, the intended goals of

VA’s National Cemetery System (NCS) include burying eligible veterans and

their family members in national cemeteries and maintaining the graves

and their surroundings as national shrines. Because NCS exerts a relatively

high degree of influence over program results, identifying results-oriented

performance measures is a fairly straightforward effort. Its fiscal year 1996

results-oriented performance measures included the percentage of

veterans who chose a burial option provided by NCS and/or who chose a VA

headstone or marker. Because of NCS’ high degree of influence over

program results, it also can more readily pinpoint opportunities for

managing its efforts and improving its performance. To help it do so, NCS

(1) conducted 14 focus groups at 5 locations throughout the country with

specific customer groups who had recent direct contact with the cemetery

and (2) placed 20,000 visitor comment cards at 76 national cemeteries for

visitors to comment on their satisfaction with various aspects of cemetery

appearance and maintenance and what their priorities were.

Page 65 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

Similarly, the Bureau of Engraving and Printing (BEP) primarily produces

currency and stamps in the volume and with the quality required by two

primary, immediate customers: the Federal Reserve Board and the U.S.

Postal Service. BEP receives no appropriations from Congress and is

self-financed by a revolving fund whereby the Reserve Board and Postal

Service reimburse BEP for its costs. Because of its direct relationship with

its customers, BEP has a relatively high degree of influence over the

achievement of its intended results and was able to more readily identify

results-oriented performance measures. These measures, which BEP has

used for years, focus on customer service and include currency notes

delivered (in billions) and postage stamps delivered (in billions), currency

and postage stamp productivity measures, and customer satisfaction

measures.

Long Time Frames to

Achieve Results Hinder

Annual Performance

Measurement

Often it can take years before agencies see the results of their programs’

activities. This factor has made it difficult for agencies to identify

performance measures that will provide them with information on the

annual progress they are making toward achieving program results. For

example, OSHA officials said it was difficult to track their agency’s yearly

progress in eliminating workplace hazards when the impact of some

regulatory actions could not be seen for years. They said that the latency

periods between exposure to a hazardous substance, such as asbestos, and

a resulting illness can be 20 years or more. Therefore, OSHA officials said,

the results of any actions OSHA takes to reduce exposures to those

substances may be equally long in coming. Officials at EPA also said it was

difficult to track the yearly progress of some of their agency’s regulatory

actions because, for example, a significant lag usually existed between the

elimination of a chemical hazard and any corresponding change in illness

rates.

Our work on research and development (R&D) programs exemplified the

significance of the challenge agencies face in identifying meaningful

performance measures to use in assessing program results when

achievement of those results can take several years and the results also

can be affected by external forces.8

 Over $71 billion was appropriated for

federal R&D efforts in 1996. However, as we and others have reported for

almost 20 years, agencies with such programs have encountered difficulty

in identifying meaningful performance measures, particularly quantitative

measures, that can be used to assess the results that the technological

8

Measuring Performance: Strengths and Limitations of Research Indicators (GAO/RCED-97-91, Mar. 21,

1997).

Page 66 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

advancements of these programs have on economic growth and the

overall standard of living. Because of the difficulties in identifying the

impacts of research, decisionmakers in the public and private sectors

typically have chosen to measure results using a variety of proxies. Three

of the most frequently cited quantitative measures are return on

investment, patents issued, and bibliometrics—i.e., the study of published

data, in particular, counts of citations. These measures imply a degree of

precision. However, they generally were not designed to measure the

long-term results of R&D programs and, therefore, are not easily adaptable

to such a purpose.

For example, return on investment is aimed at assessing the sales and

profits resulting from investments in R&D. However, long time periods and

the multiple research investments involved make the task of calculating

the return on basic research especially difficult. Productivity growth may

lag 20 years behind the first appearance of research in the scientific

community. A more serious impediment, however, is the fact that results

are often not directly traceable to a specific research investment or may

result from a combination of such investments.

Agencies Have Faced

Challenges Collecting

Reliable and Valid

Results-Oriented

Performance Data

In passing the Results Act, Congress emphasized that the usefulness of

agencies’ performance data depends, to a large degree, on the reliability

and validity of those data. However, the reliability and validity of

performance data are often questionable. From previous reviews, we

found that agencies have faced challenges in collecting reliable and valid

performance data. These challenges include different parties using

different definitions to measure performance and agencies’ reliance on

outside parties to provide results-oriented performance data.

Use of Different

Definitions for Measuring

Performance

The lack of standard definitions for performance measurement data can

significantly hinder agencies’ ability to use such data in planning and

reporting. For example, we reported challenges that OCSE faced because of

a lack of comparable performance data across state and local

jurisdictions.9 OCSE officials said that discrepancies resulting from

differences in the way the states and local jurisdictions defined what

constitutes a child support enforcement case had contributed to the

difficulty of uniformly measuring state performance. To address these

discrepancies, OCSE worked with state and local authorities to develop

9

Child Support Enforcement: Reorienting Management Toward Achieving Better Program Results

(GAO/HEHS/GGD-97-14, Oct. 25, 1996).

Page 67 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

standard data definitions for key child support enforcement terms,

including a definition for what constitutes a child support enforcement

case, and incorporated the use of standardized definitions for measuring

state performance. Together, they developed measures to assess state

performance in establishing paternities, obtaining support orders, and

collecting child support payments, using the established definition of a

child support enforcement case.

We also found that three economic development agencies we

reviewed—EDA, the Appalachian Regional Commission, and the Tennessee

Valley Authority’s nonpower programs—computed a key performance

measure differently.10 Specifically, each of these three agencies uses

“performance ratios” to measure the extent to which other federal, state,

local, or private investment in communities is attracted to an economic

development project as a result of the agency’s investment. The ratio is a

comparison of total dollars invested in or planned for an economic

development project—including the funding from other federal

programs—with the dollars contributed by the agency itself. However, we

found that each agency, in computing its ratio, defined “total dollars”

differently and calculated the ratio for only a portion of its programs.

Inconsistent definitions for what will be measured among crosscutting

programs can hamper efforts to compare the relative effectiveness of

individual programs.

Reliance on Outside

Parties for

Results-Oriented

Performance Data

Relying on parties outside of agencies for performance data has created

challenges for agencies to ensure the accuracy of such data. For example,

the Department of Education had faced difficulty in obtaining accurate

data from adult education programs administered by states.11 We reported

that states are required to submit to the Department of Education annual

statistical performance reports giving basic program information, such as

number of students served, student progress over the years, and various

types of student achievement. However, Education and state officials

acknowledged serious problems with the quality of the data contained in

those reports. These problems were based in part on double-counting or

under-counting of students in adult education programs.

As noted earlier, OCSE has had difficulty in obtaining comparable

state-reported data. OCSE also has faced challenges in ensuring that states

10GAO/RCED-96-103, April 3, 1996.

11Adult Education: Measuring Program Results Has Been Challenging (GAO/HEHS-95-153, Sept. 8,

1995).

Page 68 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

provide accurate data. In 1994, we recommended that OCSE implement

additional oversight tools for ensuring that the data states submit are

accurate and comparable.12 In a subsequent report, we found that OCSE has

placed greater emphasis on its reviews of state reporting systems, which

analyze the procedures and systems states use to accumulate, record, and

report data.13 However, in its reviews of 20 state reporting systems, OCSE

found that most of those states did not have reliable systems for reporting

data accurately and that improvements would be needed as it moves to

results-oriented management. In 1996, the Personal Responsibility and

Work Opportunity Reconciliation Act required OCSE to review and validate

the data states report on progress toward achieving program goals.

Approaches to

Improve the

Usefulness of

Results-Oriented

Performance

Information Are

Beginning to Emerge

Some agencies are exploring approaches that begin to address the

difficulty they are having in developing useful results-oriented

performance information. Five approaches we identified include: (1) using

impact evaluations; (2) using intermediate performance measures;

(3) using a range of measures; (4) providing decisionmakers with

information on the reliability and validity of performance data; and

(5) working with stakeholders to identify and reach consensus on the most

meaningful measures for the program, data sources for those measures,

and data collection strategies.

Using Impact Evaluations During our recent review of analytic challenges that agencies have faced in

measuring their performance, agency officials described using a variety of

approaches, including techniques that are employed in program

evaluations, specifically those evaluating program impact, to address the

challenge of isolating the impact of a program on its intended results.14

The Results Act defines program evaluation as “an assessment, through

objective measurement and systematic analysis, of the manner and extent

to which federal programs achieve intended objectives.” Such assessments

are to provide evaluative information about whether, and in what

important respects, a program is working well or poorly and may address

questions of program implementation and impact. Notably, the impact

evaluations described by the agency officials were often employed at state

or local levels where the influence of other variables was either reduced or

easier to observe and control for. For example, because they were well

12Child Support Enforcement: Families Could Benefit From Stronger Enforcement Program

(GAO/HEHS-95-24, Dec. 27, 1994).

13GAO/HEHS/GGD-97-14, October 25, 1996.

14Managing for Results: Analytic Challenges in Measuring Performance (GAO/HEHS/GGD-97-138,

May 30, 1997).

Page 69 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

aware that the economy has a strong effect on a loan program’s

performance, officials from one such program monitored changes in the

economy very closely at the regional level. By breaking down the data to

follow one regional economy at a time, these officials were able to

determine whether an increase in loan defaults in a given region reflected

a faltering economy or indicated some problem in the program that

needed to be followed up.

Although the Results Act does not require agencies to conduct impact

evaluations, it does require them to measure progress toward their goals,

identify how external forces might affect such progress, and explain why a

goal was not met. We previously reported that agencies often need

information on a program’s impact on its results relative to the impact of

external factors to confidently attribute the achievement of intended

results to the program.15 Further, congressional and other decisionmakers

will likely request such information to help them make informed decisions.

We also have reported on how findings of program evaluations can be a

potentially critical source of information on the reasons an agency’s goals

were not met and actions that could be taken to improve performance.16

However, we have reported that because of their complexity, such

evaluation studies may be costly, and many agencies may not have staff

with the skills to conduct program evaluations.17

Using Intermediate

Measures

As noted earlier, an agency’s ability to identify meaningful performance

measures to use in planning its efforts and gauging progress—especially

annual progress—toward achieving its goals is sometimes hindered

because the intended results of the agency’s programs can be affected by

various factors or require many years to come to fruition. Our work has

found that in such cases, agencies may be able to use intermediate

performance measures that provide information on interim results to

gauge their progress in meeting their goals.18

EPA’s Office of Water has identified intermediate results, with

accompanying measures, for local estuary protection programs to use in

15Program Evaluation: Improving the Flow of Information to the Congress (GAO/PEMD-95-1, Jan. 30,

1995).

16GAO/GGD-96-66R, February 14, 1996.

17Program Evaluation Issues (GAO/OCG-93-6TR, Dec. 1992).

18GAO/GGD-95-22, December 21, 1994.

Page 70 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

assessing their progress in meeting intended results. According to EPA’s

Coastal Management Branch Chief, those programs are to develop

partnerships among all government agencies that oversee estuary

resources and the people who depend on those resources for their

livelihood and quality of life. The estuary programs generally try to affect

the behavior of government agencies, businesses, households, and boaters

toward achieving improvements in the quality of estuaries and the health

of organisms living in those estuaries. According to EPA, because it may

take years to see such improvements, intermediate measures can provide

information on the performance of those programs on a more timely basis

and are, therefore, important for managing estuary protection efforts. EPA

has developed procedures for managers of local estuary programs to

monitor and report on program results and the progress in improving and

maintaining the quality of their estuary waters, according to the official.

Further, to the extent that many estuary programs use similar procedures

for assessing their progress, such information also can provide a national

perspective on progress in estuary protection. EPA’s Coastal Management

Branch Chief said that most of the 28 estuary programs have adopted or

are considering this approach.

EPA identified three sequential intermediate results—which are referred to

as first-, second-, and third-order results—that are expected to affect

changes in bay quality and the health of organisms living in

estuaries—which are referred to as the fourth-order, or end, results of the

program. Measures for first-order intermediate results provide information

on the extent to which desirable estuary protection actions have been

adequately implemented by governments, businesses, or households.

Measures for second-order intermediate results provide information on

the extent to which those actions have led to reductions in pollutant

discharges. Measures for third-order intermediate results provide

information on the extent to which water or sediment quality has changed.

Finally, measures for assessing the fourth-order results of the program

provide information on the extent to which the health of fish, shellfish,

other wildlife, habitat and vegetation, and the region’s economy has

changed.

Using a Range of Measures We found that in cases where agencies faced significant challenges

identifying results-oriented performance measures, they sometimes used a

range of measures to provide a more complete picture of agency

performance. For example, the Army Research Laboratory (ARL) found

that it could not predict the results or ultimate value of its research.

Page 71 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

Therefore, it could not objectively and quantitatively measure whether its

research was achieving a specific intended result. Instead, ARL devised a

performance measurement approach that is to make use of “three pillars”:

metrics, customer feedback, and peer review. For example, its metrics

included about 60 quantitative measures that according to ARL provided

useful information for understanding the functional health of the

organization and the management of ARL. However, ARL cautioned that the

resulting performance information would not enable it to determine the

real quality and impact of its programs. Because of its lack of

results-oriented performance measures, ARL is relying more heavily on

customer surveys and peer review for information about the quality of its

efforts.

Our work on R&D programs found that although there are strengths to

using peer reviews, there are also limitations.19 Peer review uses technical

experts to judge R&D results on the basis of the experts’ evaluation of the

quality of research. We reported that although peer review has been used

extensively in the selection of proposed research projects, it is subject to

two serious shortcomings. First, peer review is based on individuals’

perceptions of quality, which depend largely on the expertise of the

selected experts. Thus, the subjective nature of peer reviews makes

performance results vulnerable to bias. Frequently, a numerical rating

scale, such as 1 for poor through 5 for excellent, is used to judge the

quality of research projects and the selection of proposed projects.

However, despite the appearance of precision conferred by a specific

number, the numbers represent the best of sometimes widely differing

judgments. Second, peer review can be expensive. For example, ARL

contracted for a peer review of its activities, which calls for National

Research Council technical assessment board reviews of one-third to

one-half of ARL’s programs each year at a cost of approximately $650,000

per year.

Providing Decisionmakers

Information on the

Reliability and Validity of

Performance Data

As noted earlier, in passing the Results Act, Congress emphasized that the

usefulness of agencies’ performance data depends, to a large degree, on

the reliability and validity of those data. Consequently, the Act requires

that agencies describe in their annual performance plans the means to be

used to verify and validate performance data. We found that including

such information in performance reports could be equally important in

providing assurance to report users of the quality of the data.20 The

19GAO/RCED-97-91, March 21, 1997.

20GAO/GGD-96-66R, February 14, 1996.

Page 72 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

National Highway and Traffic Safety Administration (NHTSA) included an

appendix to its fiscal year 1994 Results Act pilot performance report that

discussed the sources, and in some cases the limitations, of the data it

used to report on performance. For example, NHTSA’s accident data are

based on police reports, but various sources suggested that about half of

the motor vehicle crashes in the country were not reported to police.

NHTSA believed that the majority of these unreported crashes involved only

minor property damage and no significant injury. However, NHTSA’s report

stated that it planned “to assess the unreported injury problem.” By

incorporating a discussion of the limitations of its accident data, NHTSA

provided a context to assist the users of the data in assessing NHTSA’s

performance.

Working With Stakeholders

to Reach Consensus on

Measures and Data

Collection

Stakeholders can help agencies identify results-oriented performance

measures and the ways to collect needed performance data. For example,

in 1994, we reported that OCSE lacked essential management tools to

improve its responsiveness to the child support needs of children and

families and recommended actions OCSE could take in this regard.21 Since

that time, OCSE has made progress in reorienting its management of the

program toward achieving intended program results through greater

stakeholder involvement and other means.22 OCSE’s effort to involve states

and other stakeholders took time and concerted effort.

One of the first steps OCSE took was specifying the goals that it expected

states to achieve in such areas as paternities established and collections

received. However, state program officials strongly objected to this

mandate, because they did not have an opportunity to participate in the

planning process. OCSE then sought to obtain wider participation from

program officials at the federal, state, and local levels of government

through a joint planning process. During the planning process, participants

agreed that intended national results would be based on the collective

suggestions of the states and consensus among participants. OCSE also

established a Performance Measures Work Group to identify statistical

measures for assessing state progress toward achieving national results.

The work group, which consisted of officials from the Department of

Health and Human Services’ (HHS) Administration for Children and

Families, OCSE, and state and local child support enforcement programs,

met several times in 1995 and 1996 to discuss mutually acceptable

performance measures.

21GAO/HEHS-95-24, December 27, 1994.

22GAO/HEHS/GGD-97-14, October 25, 1996.

Page 73 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

Officials in VA’s Loan Guaranty program also discussed the importance of

involving stakeholders in identifying meaningful measures. According to

those officials, they had difficulty linking field office activities to the

intended results of the program as established by headquarters staff. To

address this, VA used input from OMB and a performance measures work

group that included both headquarters and field office staff to reach

consensus on the most meaningful performance measures for the program

and how data would be collected on those measures. Also, key

headquarters and field managers were brought together to reach

agreement on the goals to be included in the program’s fiscal year 1997

business plan.

Perhaps most important, congressional consultations on agencies’

strategic plans also can assist agencies in identifying the most meaningful

measures for the agencies’ programs. A VA official reported that most of

their discussions with congressional staff have included useful exchanges

about the most meaningful performance measures for assessing the results

of VA’s benefit programs. In several cases, congressional staff proposed

additional performance measures for VA programs. For example, the staff

suggested that VA’s Vocational Rehabilitation and Counseling program add

measures to assess the socioeconomic impact of the program on veterans,

such as the extent to which veterans have to rely on other government

assistance programs.

Conclusions The challenges confronting agencies as they seek to develop useful

results-oriented performance information are substantial, long-standing,

and will not be quickly or easily resolved. In cases where results-oriented

baseline and trend data on past performance do not exist, agencies’ will

have difficulty setting realistic goals. Many agencies also are grappling

with how best to assess their intended results in an environment where

federal program efforts are but one factor among many external forces

that may influence whether program results are achieved. Identifying

results-oriented performance measures will often require an agency to

separate out the influence on the achievement of its intended results from

those of forces external to it in order to isolate the agency’s contribution

to the achievement of those results—an analytically daunting challenge.

Moreover, identifying such measures also is challenged by the long lead

times that may occur between a federal agency’s actions and the

achievement of intended results.

Page 74 GAO/GGD-97-109 Results Act
Chapter 4 

Developing Useful Results-Oriented

Performance Information Will Be an

Ongoing Difficulty

Finally, once agencies have identified results-oriented performance

measures, collecting reliable and valid data on those measures poses

challenges for agencies. Reasons for these challenges include different

parties using different definitions to measure performance and agencies

relying on outside parties to provide results-oriented performance data.

Some agencies have begun exploring approaches to address the difficulty

in developing useful results-oriented performance information. However,

the nature of the challenges agencies need to address and the time and

effort needed to overcome those challenges suggest that agencies will be

hard-pressed to consistently identify results-oriented measures in time for

the first annual performance plans to be submitted to OMB this fall. Those

agencies with the most direct influence on their results generally will

make the most progress in identifying such measures, while those

agencies with less influence will continue to struggle to identify the most

meaningful measures for informing decisionmakers in Congress and the

executive branch.

Page 75 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

Successful implementation of the Results Act in federal agencies will

depend on the degree to which those agencies create organizational

cultures that foster and maintain a focus on results at all levels. The

committed support of top agency leadership is critical to building a

results-oriented culture. Yet, according to the opinions of the federal

managers who responded to our survey, top agency leadership is not fully

committed to results-oriented management. In addition, successful

implementation of the Results Act will depend, in part, on managers

having the knowledge, skills, and abilities to set goals, measure

performance, and use performance information to improve effectiveness.

However, managers’ responses to our survey indicated that federal

managers needed additional training on critical results-oriented

management-related tasks. A results-oriented organization also strives to

ensure that its managers at all levels have the authority needed to

accomplish goals and provides those managers with the incentives and

opportunities for doing so. However, federal managers’ responses to our

survey also suggest that more progress is needed in these areas as well.

Top Leadership Has

Not Fully

Demonstrated

Commitment to

Results-Oriented

Management

For the Results Act to become part of an agency’s culture, the agency

needs the sustained, demonstrated commitment of its top leadership to

initiate and insist on the use of results-oriented management practices, to

keep the agency focused on results, and to embed related principles in the

agency’s approach to doing business.1

 By showing managers and staff its

commitment to achieving an agency’s goals, top agency leadership can

encourage a focus on results. When asked about the extent to which their

agencies’ top leadership demonstrated a strong commitment to achieving

results, federal managers in our survey reported a significantly greater

extent of current top leadership commitment than they recalled existing 3

years ago. According to our survey, about 44 percent of managers reported

that their agencies’ top leadership demonstrated a strong commitment to

achieving results to a great or very great extent 3 years ago, and 57 percent

of managers reported such commitment to a great or very great extent

currently, which is a 13 percentage point difference.

The percentage of federal managers reporting that top leadership

demonstrated strong commitment to a great or very great extent both

currently and 3 years ago was significantly higher for SES managers than

for non-SES managers. This suggests that the perception of top leadership

commitment to results is stronger for federal managers at higher levels

1

GAO/GGD-96-118, June 1996.

Page 76 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

and that this perception has not reached managers at all agency levels to

the same extent. (See table 5.1.)

Table 5.1: Percent of Federal Managers

Who Reported That Their Agencies’

Top Leadership Demonstrated a

Strong Commitment to Achieving

Results

Percent of federal managers who reported

that their agencies’ top leadership

demonstrated a strong commitment to

achieving results to a great or very great

extent

Level of managers 3 years ago Currently

SES 60% 76%

Non-SES 42 56

Source: GAO survey data.

Without top leadership making its commitment to results-oriented

management clear, the Results Act risks the danger that all management

reforms face: becoming a hollow, paper-driven exercise. As we have said

in our Executive Guide, leaders who integrate results-oriented

management into the culture and day-to-day activities of their

organizations will help avoid that danger.2

One of the most meaningful demonstrations of leaders’ commitment to

results-oriented management is the use of performance information to

make decisions. However, federal managers’ perceptions about the extent

to which such use is occurring are not as strong as their perceptions about

top leadership’s demonstrated commitment in general. This suggests that

despite the strength of managers’ perceptions about top leadership

commitment, the extent to which leaders are demonstrating their

commitment by using performance information for decisionmaking about

their agencies’ day-to-day activities is not as great.

Compared to the 57 percent of managers who reported strong top

leadership commitment to a great or very great extent, only about

16 percent reported that changes by management above their levels to the

programs for which they were responsible were based on results or

outcome-oriented performance information to a great or very great extent.

When we compared managers who reported a great or very great extent of

strong top leadership commitment to managers who rated leadership

commitment from moderate to none, we found that 25 percent of the

managers who rated leadership commitment as great or very great also

reported that managers above their level made program changes on the

2

GAO/GGD-96-118, June 1996.

Page 77 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

basis of performance information to a great or very great extent. Only

5 percent of managers who rated the extent of leadership commitment

from moderate to none rated the extent of managerial program changes

based on performance information as great or very great. Although the

difference between these two groups is statistically significant, in either

case, only a minority of managers perceived program changes based on

performance information occurring to a great or very great extent even

when top leadership commitment was perceived to be high.

Although still a minority, more federal managers said that the individuals

to whom they reported periodically reviewed with them the results or

outcomes of the programs for which they were responsible than said that

program changes were made based on performance information. Both of

these activities are indicators of the use of performance information for

management decisionmaking. About 42 percent of managers said that the

individuals to whom they reported periodically reviewed with them the

results or outcomes of the programs for which they were responsible to a

great or very great extent. As before, a comparison of managers who

reported a great or very great extent of strong top leadership commitment

to managers who rated leadership commitment from moderate to none

showed a significant difference. We found that 57 percent of the managers

who rated leadership commitment as great or very great also rated the

extent of periodic reviews of their programs’ results or outcomes as great

or very great. Only 21 percent of managers who rated the extent of

leadership commitment from moderate to none rated the extent of

periodic reviews of their programs’ results or outcomes as great or very

great.

Managers’ responses regarding the extent to which changes to programs

were based on results and the extent to which individuals to whom they

reported reviewed program results with them indicate that as the

perception of strong top leadership commitment increased, the perception

of the use of performance information for decisionmaking also increased.

In addition, as their responses indicate, many managers did not perceive

that performance information was being used to make such decisions to a

great or very great extent.

Page 78 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

Managers Reported

Needing Additional

Training

In our work on identifying techniques that are needed to change an

organization’s culture, we found that training was one of the most

important techniques used.3

 From our symposium on transforming the

civil service, we discerned that treating continuous learning as an

investment in success rather than as a cost to be minimized was a key

principle for managing people.4

 Investing in training and professional

development could help federal agencies meet changing customer needs,

keep skills up to date, and develop new personal and organizational

competencies. In work we did to identify insights for Results Act

implementation from past initiatives on performance budgeting, we spoke

with current and former executive branch officials and legislative branch

staff as well as individuals with expertise in the Results Act, budgeting,

and public administration.5

 These experts said that participants in

implementing the Act needed to acquire the skills to develop and use

performance information. One of the critical practices for implementing

the Results Act that we identified in our Executive Guide was the need to

build expertise for staff at all levels of the organization so that they are

skilled in strategic planning, performance measurement, and the use of

performance information in decisionmaking. 6

When asked whether, during the past 3 years, their agencies had provided,

arranged, or paid for training that would help them accomplish several

critical results-oriented management-related tasks, the percentage of SES

managers who responded “yes” was significantly greater than that of

non-SES managers. According to our survey, 43 to 61 percent of SES

managers and 17 to 38 percent of non-SES managers responded yes to

questions on training that would help them accomplish each of several

critical results-oriented management-related tasks. The task for which the

highest percentage in general responded yes was for training on

conducting strategic planning. For this task, 61 percent of SES managers

and 38 percent of non-SES managers responded yes. However, for the

remaining five of the six tasks, fewer than 38 percent of non-SES managers

responded yes. Only 46 percent of SES managers and 17 percent of non-SES

managers responded yes regarding training on implementing the

3

Organizational Culture: Techniques Companies Use To Perpetuate or Change Beliefs and Values

(GAO/NSIAD-92-105, Feb. 27, 1992).

4

Transforming the Civil Service: Building the Workforce of the Future—Results of a GAO-Sponsored

Symposium (GAO/GGD-96-35, Dec. 20, 1995).

5

GAO/AIMD-97-46, March 27, 1997.

6

GAO/GGD-96-118, June 1996.

Page 79 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

requirements of GPRA. The results for training on each task for SES and

non-SES managers appear in table 5.2.

Table 5.2: Percent of Federal Managers

Who Reported That During the Past 3

Years Their Agency Provided,

Arranged, or Paid for Training That

Would Help Them to Accomplish

Specific Tasks

Percent of federal

managers

Tasks SES Non-SES

Conduct strategic planning 61% 38%

Set program performance goals 56 33

Develop program performance measures 52 35

Use program performance information to make decisions 43 30

Link the performance of program(s)/ operation(s)/project(s) to

the achievement of agency strategic goals

46 27

Implement the requirements of GPRA 46 17

Source: GAO survey data.

In addition, we asked federal managers to what extent they thought they

needed training or additional training to help them accomplish each of

these tasks. For each task, the percentage of non-SES managers was

significantly higher than the percentage of SES managers reporting a need

for training to a great or very great extent. Table 5.3 shows the results for

training or additional training needed to a great or very great extent by SES

and non-SES managers for each task.

Table 5.3: Percent of Federal Managers

Who Reported Needing Training or

Additional Training to Accomplish

Specific Tasks

Percent of federal

managers who

reported needing

training or additional

training to a great or

very great extent to

accomplish specific

tasks

Tasks SES Non-SES

Conduct strategic planning 14% 28%

Set program performance goals 15 32

Develop program performance measures 20 40

Use program performance information to make decisions 14 32

Link the performance of program(s)/operation(s)/

project(s) to the achievement of agency strategic goals

17 33

Implement the requirements of GPRA 18 43

Source: GAO survey data.

Page 80 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

Implementing the requirements of GPRA was the only task for which a

significant difference existed between the ratings of managers who

reported training that their agencies provided, arranged, or paid for and

managers who reported no such training. About 29 percent of managers

who reported the training in implementing the requirements of GPRA also

reported a great or very great need for additional training for this task.

However, more than half of managers who reported no such training in

implementing the requirements of the Act reported a great or very great

need for it. A need for training on this task received the highest proportion

of great and very great responses from managers who reported no such

training for any given task.

Although responses to the need for training varied, many managers

believed that they needed training to a great or very great extent for at

least some of the tasks. For example, based on our survey, 43 percent of

managers reported that they needed training to a great or very great extent

in at least two of the tasks listed.

Providing the skills to develop and use performance information may well

be a significant challenge to implementing the Results Act because of

declining resources and serious budgetary constraints. Executive branch

officials we spoke with as part of our review of past initiatives on

performance budgeting tended to think that the Results Act was “the right

thing to do.” Thus, they believed that the resources for needed training

related to implementing the Act would be found.7

 In an April 2, 1997,

OPM-sponsored interactive broadcast on the Results Act, OMB’s Deputy

Director for Management said he recognized the difficult problem that

federal agencies faced, given the training and other capacity-building

activities needed to measure and evaluate program performance during

this time of significant budgetary constraint.8

 But he stressed that federal

agencies do have to devote some resources to these important activities to

increase the effectiveness of their programs, for which they use most of

their resources. Because of this environment, agencies will need to seek

new, creative, and less costly ways to build their capacities to implement

the Act in order to improve their programs. In our past work, we found

that one approach—which has been used at HHS and the Department of the

Interior—was to limit training costs by having in-house coordinators lead

7

GAO/AIMD-97-46, March 27, 1997.

8

Office of Personnel Management Federal Human Resources Forum: Interactive Broadcast on the

Government Performance and Results Act and How It Affects Every Federal Government Agency

Manager and Worker, April 2, 1997.

Page 81 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

training efforts and serve as mentors on the Results Act.9 Managers’

responses to our survey questions on training suggest that federal

agencies’ efforts to train managers on results-oriented

management-related tasks may best be focused on non-SES managers. The

responses also suggest that such training especially include training on

implementing the requirements of the Results Act.

Managers Reported

Reduced Authority to

Achieve Results

In crafting the Results Act, Congress recognized that if federal managers

were to be held accountable for program results, they would need the

authority and flexibility to achieve those results. Congress also understood

the importance of affording federal program managers the freedom to be

innovative and creative and to marshal resources to achieve results.

Congress has provided agencies with additional authority in the key area

of procurement that allows agencies more flexibility in managing their

programs. In addition, one of the major initiatives of NPR has been the

empowerment of federal employees to get results.

Overall, when asked about the extent to which managers or supervisors at

their levels had the authority they needed to help their agencies

accomplish their strategic goals, federal managers did not perceive that

they had a greater extent of such authority than they recalled having 3

years ago. Interestingly, for managers from selected Results Act pilots, the

perception of the extent of this authority currently was significantly lower

than the perception for 3 years ago. In contrast, for managers who were

not in selected pilots, the perception of the extent of authority currently

was essentially unchanged from the perception for both 3 years ago. In

addition, for both 3 years ago and currently, the percentage of SES

managers reporting a great or very great extent of authority was

significantly higher than the percentage of non-SES managers. (See table

5.4.)

9

Managing for Results: Status of the Government Performance and Results Act (GAO/T-GGD-95-193,

June 27, 1995).

Page 82 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

Table 5.4: Percent of Federal Managers

Reporting That Agency

Managers/Supervisors at Their Levels

Had the Decisionmaking Authority

They Needed to Help the Agency

Accomplish Its Strategic Goals

Percent of federal managers reporting that

managers/supervisors had the

decisionmaking authority they needed to a

great or very great extent

Level of managers

3 years

ago Currently Difference

SES

Selected Results Act pilots 56% 40% –16%

All other agencies 53 52 –1

Non-SES

Selected Results Act pilots 36 24 –12

All other agencies 29 31 +2

Source: GAO survey data.

When asked about the extent to which managers or supervisors at their

levels were held accountable for the results of the programs for which

they were responsible, no such differences between managers from

selected Results Act pilots and managers from all other agencies or

between SES and non-SES managers were observed. However, more

managers overall rated the extent of accountability for results as higher

currently than for 3 years ago. For example, about 55 percent of managers

reported that managers or supervisors at their levels were currently being

held accountable to a great or very great extent for the results of the

programs for which they were responsible. About 48 percent of managers

reported having this perception for 3 years ago.

Less Than Half of

Federal Managers

Reported Positive

Recognition for

Achieving Results

In crafting the Results Act, Congress recognized the need to create

incentives for managers to use results-oriented performance information.

The commitment demonstrated by Congress and top leadership when they

use results-oriented performance information to make decisions is one of

the greatest incentives for changing agencies’ cultures and managers’

behavior to focus on achieving intended results. In addition, agencies have

traditionally rewarded their employees formally through pay increases and

other monetary and nonmonetary awards for performance that helped

agencies achieve their goals. However, the current environment of

constrained resources challenges agency leaders to think creatively about

informal incentives, which need not be costly, such as positive

recognition, that they can use to encourage results-oriented management.

Page 83 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

When asked about the extent to which employees in their agencies

currently received positive recognition and received such recognition 3

years ago for helping the agencies accomplish their strategic goals, federal

managers reported essentially no difference. However, statistically

significant differences existed between the perceptions of SES and non-SES

managers, both currently and for 3 years ago. According to our survey,

48 percent of SES managers and 25 percent of non-SES managers reported

that, to a great or very great extent, employees in their agencies currently

received positive recognition for helping the agencies accomplish their

strategic goals. When asked to recollect the situation 3 years ago,

43 percent of SES managers and 27 percent of non-SES managers reported

such recognition.

In work we did on state governments’ experiences in using

results-oriented management, we found that in Oregon and Minnesota,

managers and staff in some state agencies were reluctant to commit to

achieving results that they did not totally control because of concerns that

performance information would be used against them.10 In our survey of

federal managers, we listed 16 factors and asked managers to rate each

factor on the extent to which it hindered measuring performance or using

performance information for the programs with which they were involved.

One of these factors was concern that performance information could be

used against their programs or agencies. When asked about this factor,

only about 9 percent of managers reported it as a hindrance to a great or

very great extent—least often of the 16 factors included in the survey.

Involvement of

Federal Managers in

Results Act-Related

Activities Is Varied

Congress expected that under the Results Act, managers throughout

federal agencies would have active roles in making their organizations

more results oriented. One of the stated purposes of the Act was to help

federal managers improve service delivery by requiring that they plan for

meeting program objectives and by providing them with information about

program results and service quality. More recently, during OPM’s April 1997

interactive broadcast on the Results Act, the OMB Deputy Director for

Management underscored OMB’s commitment to the effective

implementation of the Results Act.11 He told federal managers that if they

had not been involved in strategic planning, they should be asking their

10GAO/GGD-95-22, December 21, 1994.

11Office of Personnel Management Federal Human Resources Forum: Interactive Broadcast on the

Government Performance and Results Act and How It Affects Every Federal Government Agency

Manager and Worker, April 2, 1997.

Page 84 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

agencies why they had not been involved and saying that OMB wanted them

to be involved.

However, when we asked federal managers whether, during the past 3

years, they had been involved in a variety of GPRA-related activities that

incorporated a number of key practices needed to implement the Act, they

reported varied levels of involvement, depending on the activity.

Significant differences were especially evident for each activity between

SES managers’ and non-SES managers’ reported levels of involvement. For

example, according to our survey, 72 percent of SES managers reported

involvement in establishing long-term strategic goals for their

agencies—which is a key step in strategic planning—while only 35 percent

of non-SES managers reported involvement in this activity. Table 5.5 shows

the percentages of both SES and non-SES managers as well as the overall

percentages of federal managers who reported that they had been involved

in these activities during the past 3 years.

Table 5.5: Percent of Federal Managers

Reporting That They Had Been

Involved in GPRA-Related Activities

During the Past 3 Years

Percent of federal managers

reporting that they had been

involved in these activities during

the past 3 years

Activities SES Non-SES Overall

Establishing long-term strategic goals for the

agency

72% 35% 38%

Reconciling differing congressional views on the

agency’s mission and strategic goals

34 8 10

Reconciling the views of other interested parties

on the agency’s mission and strategic goals

53 29 31

Developing ways to measure whether program

performance goals are being achieved

72 47 49

Gathering and analyzing data to measure

whether programs are meeting their specific

performance goals

65 54 55

Using measures for program performance goals

to determine if the agency’s strategic goals are

being achieved

55 34 35

Publicly reporting on how well the agency’s

programs are meeting their specific performance

goals

44 18 20

Source: GAO survey data.

Of activities that showed significantly lower percentages of non-SES

managers reporting involvement as compared to SES managers, some are

Page 85 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

activities where such differences might be expected—for example,

reconciling differing congressional views on the agency’s mission and

strategic goals and reconciling the views of other interested parties on the

agency’s mission and strategic goals. However, such significant differences

between SES and non-SES managers’ reported involvement were also

evident for activities where involvement of managers at all levels could

reasonably be expected—for example, developing ways to measure

whether program performance goals are being achieved and gathering and

analyzing data to measure whether programs are meeting their specific

performance goals. However, gathering and analyzing data to measure

whether programs are meeting their specific performance goals was the

only one of these GPRA-related activities for which at least half of managers

overall, including both SES and non-SES managers, reported being involved.

Federal Managers’

Views About GPRA’s

Past and Future

Effects on Agency

Programs Are Mixed

When we asked federal managers about the extent to which their agencies’

efforts to implement GPRA to date had improved their agencies’ programs,

57 percent overall responded that they had not been sufficiently involved

in the Act in their agencies to have an opinion, comprising 22 percent of

SES managers and 60 percent of non-SES managers. Of the 41 percent of

managers who did express an opinion on the extent to which their

agencies’ efforts to implement GPRA to date had improved their agencies’

programs, 42 percent responded that the Act had improved programs to a

moderate or greater extent. Figure 5.1 shows the percentages of the

federal managers who did express an opinion for each extent level.

Page 86 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

Figure 5.1: Distribution of Federal

Managers Expressing an Opinion on

the Extent to Which Implementing

GPRA to Date Had Improved Their

Agencies’ Programs

0

10

20

30

40

1

9

32

36

22

Very great

extent

Great extent Moderate

extent

Small extent No extent

Percent

Source: GAO survey data.

Governmentwide implementation of the Results Act is just beginning for

many federal agencies and programs and, as our earlier work has shown,

changing to a results-oriented culture can take many years.12 Thus, the

57 percent of managers overall reporting that they had not been

sufficiently involved in GPRA to have an opinion on the extent to which

efforts to implement the Act to date had improved their agencies’

programs and the much greater proportion of non-SES managers

(60 percent) than SES managers (22 percent) reporting that they had not

been sufficiently involved to have an opinion are not surprising. In

addition, given the long-term challenges that a move toward a

results-oriented culture involves, it is reasonable to expect that many

federal managers are not yet seeing their agencies’ results improving as an

effect of the Results Act.

12GAO/T-GGD-95-193, June 27, 1995.

Page 87 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

When asked about the extent to which implementing GPRA could improve

their agencies’ programs in the future, federal managers’ opinions were

more optimistic. About 36 percent of managers overall did not express an

opinion, responding that they had no basis to judge. Of the 62 percent of

managers who did express an opinion, about 71 percent responded that

implementing the Act could improve programs in the future to a moderate

or greater extent. Figure 5.2 shows the percentages of the federal

managers who did express an opinion for each extent level.

Figure 5.2: Distribution of Federal

Managers Expressing an Opinion on

the Extent to Which Implementing

GPRA Could Improve Their Agencies’

Programs in the Future

0

10

20

30

40

11

25

35

23

6

Percent

Very great

extent

Great extent Moderate

extent

Small extent No extent

Source: GAO survey data.

Still, many federal managers remained unconvinced or uncertain about

whether the Results Act could help their agencies’ programs even

moderately.

Page 88 GAO/GGD-97-109 Results Act
Chapter 5 

Building an Organizational Culture to Help

Create and Sustain a Focus on Results

Remains a Work in Progress

Conclusions According to the opinions of federal managers, building the organizational

cultures necessary to create and sustain a results-orientation in the federal

government appears to be a work in progress. In some ways, changes

toward a culture that will allow a focus on results appear to be occurring.

For example, the perceptions of both SES and non-SES managers about the

extent to which top leadership is demonstrating a strong commitment to

achieving results were higher currently than they were for 3 years ago.

However, we would expect to see federal managers’ experiences with and

positive perceptions about results-oriented management practices become

more prevalent if the Results Act is to succeed.

Over the next several years, we would expect to see more federal

managers reporting positively on indicators, such as top management’s use

of results to make decisions, agencies providing training on key

results-oriented management-related tasks, and managers’ involvement in

GPRA-related activities. But just as importantly, we would expect to see the

gap between SES and non-SES managers regarding positive experiences and

perceptions about results-oriented management begin to narrow if the

Results Act is to work as intended. Agencies can especially concentrate

their efforts on areas where managers are not perceiving or experiencing

progress, such as on devolving decisionmaking authority to managers

throughout the agencies.

When both mid-level and upper level managers support and are involved in

the changes that the Results Act entails, the probability of sustaining those

changes will increase substantially. Providing federal managers at all

levels with greater authority and greater opportunities to participate in

results-oriented activities can give them the experiences needed to

increase their confidence in the benefits of managing for results. When

such indicators are more widespread and pervasive among federal

managers, agencies’ commitment to achieving results and their ability to

do so should be enhanced governmentwide.

Page 89 GAO/GGD-97-109 Results Act
Chapter 6 

Efforts to Integrate Performance

Information With Budget Decisions and to

Develop a New Reporting Framework Are

Promising

Addressing some of the challenges discussed earlier in this report, such as

crosscutting program efforts and balances among competing priorities,

raises significant policy issues for Congress and the administration to

consider, and some issues will likely be very difficult to resolve. Our work

has indicated that the program performance information that is to be

generated under the Results Act will become most useful to congressional

and executive branch decisionmakers when that information is

systematically linked with resource allocation decisions and reported

within a framework that allows for fuller assessments of results,

operational performance, and costs.

Although budgeting is inherently an exercise in political choice, the

Results Act is based on the premise that budget decisions should take into

consideration an agency’s expectations about program performance.

Therefore, the Results Act’s success or failure should not be judged on

whether contentious budget and other policy issues are fully resolved;

rather, the success or failure of the Results Act will likely turn on the

extent to which the information produced through the Act’s goal-setting

and performance measurement practices, once those practices are

successfully implemented, helps Congress and the executive branch make

informed policy decisions and improve program management.

Agencies are to provide the first annual performance plans, covering fiscal

year 1999, to OMB in the fall of 1997 and to Congress after the President’s

budget is submitted in 1998. These plans are to directly link executive

branch performance planning with the budget process. Agencies are to

accomplish this link by basing their annual program performance goals on

their program activity structures as listed in the President’s budget.

However, our recent report on previous governmentwide initiatives to

better align spending decisions with expected performance—commonly

referred to as performance budgeting—found that the extent to which the

budget’s program activity structures can be directly linked to a

results-oriented performance framework varies widely among program

activities. For Congress and the executive branch to reach agreement on

changes to these structures will be a time-consuming and difficult process

that will require more than one budget cycle to resolve.1

Over the longer term, improved financial reporting and auditing as

required by the CFO Act is to strengthen the cost basis and reliability of

performance information. In this regard, reporting concepts and standards

developed by the Federal Accounting Standards Advisory Board (FASAB)

1

GAO/AIMD-97-46, March 27, 1997.

Page 90 GAO/GGD-97-109 Results Act
Chapter 6 

Efforts to Integrate Performance

Information With Budget Decisions and to

Develop a New Reporting Framework Are

Promising

are intended to provide congressional and other decisionmakers for the

first time with annual “report cards” on the costs, management, and

effectiveness of federal agencies in order to help strengthen federal

decisionmaking and agencies’ accountability.2

 In addition, FASAB’s new

cost accounting standards are intended to strengthen federal

decisionmaking by requiring agencies to provide relevant and reliable cost

information that can be used, for example, to relate the costs of various

programs and activities with their performance outputs and results.

The Results Act Seeks

to Better Align

Desired Results With

Budget Decisions

Congress intended for the Results Act’s annual performance plans to help

Congress and the executive branch make informed decisions by providing

a simple, straightforward linkage among plans, budgets, and performance

results—a linkage that we have found did not exist in the past. Our report

on performance budgeting initiatives demonstrated the importance of

making this linkage if the Results Act’s goal of better aligning expected

results with budget decisions is to be met. We reported that past

performance budgeting initiatives, such as PPBS, which was initiated

governmentwide in 1965 by President Johnson, generated unique

performance information that was often unconnected to the structures

used in congressional budget presentations. We found that consequently,

congressional interest in these initiatives quickly waned as plans and

performance results could not be directly linked to familiar oversight and

budget structures. In the end, we concluded that structural

incompatibilities meant that resource decisions were not linked to the

information generated by the past performance budgeting efforts.

The Results Act attempts to establish this linkage by requiring agencies to

base the goals in their annual performance plans on the program activity

structures used in their budget submissions and to provide these

performance plans to Congress each year after the president’s budget is

submitted. Program activities are listings of projects and activities

contained in the president’s budget and are intended to provide a

meaningful representation of agency operations. The Results Act allows

agencies to aggregate, disaggregate, or consolidate program activities for

performance planning purposes. We noted in our performance budgeting

report that program activity structures are used to provide a relatively

consistent framework for OMB and congressional budget decisions.

Congress also often uses the program activity structures to form the basic

2

FASAB was created in October 1990 by the Secretary of the Treasury, the Director of OMB, and the

Comptroller General to consider and recommend accounting principles for the federal government.

Page 91 GAO/GGD-97-109 Results Act
Chapter 6 

Efforts to Integrate Performance

Information With Budget Decisions and to

Develop a New Reporting Framework Are

Promising

unit of congressional oversight for determining reprogramming

thresholds.3

We reported that an agency’s program activities generally result from

negotiations between the agency, OMB, and the relevant congressional

appropriations subcommittees. Program activity structures, therefore,

typically are tailored to meet the specific needs of an agency, OMB, and

Congress. Thus, program activity structures represent programmatic,

process, organizational, or other orientations depending on the needs,

interests, and experiences of agencies, OMB, and Congress. Moreover,

program activities differ from agency to agency; and within agencies, they

differ from budget account to budget account. For example, the “space

station” program activity in the National Aeronautics and Space

Administration’s budget is a discrete program. In contrast, the “park

management” program activity under the National Park Service’s budget

represents process and/or functional efforts.

Because of the wide variability across program activities, we noted that

the suitability of current program activity structures for the Results Act’s

performance planning and measurement also will vary. Agency officials

we spoke to as part of our review of previous performance budgeting

initiatives highlighted the varying suitability of their program activity

structures for the Results Act’s purposes. One agency successfully worked

through the performance planning process using its existing program

activities. A second agency had program activities that reflected its

organizational units—a structural approach that is useful for traditional

accountability purposes, such as monitoring outputs and staff levels, but

less useful for results-oriented planning. This agency therefore found it

necessary to devise a separate planning structure and then crosswalk

performance goals back to the budget’s program activities. Still other

agencies separated performance planning from program activity

structures, believing it necessary to first establish appropriate program

goals, objectives, and measures before considering the link to the budget.

Our discussions with agency officials and congressional staff found that

they tended to view the need for and benefits of adjustments to program

activities from very different perspectives. Congressional staff were

generally comfortable with existing activity structures and questioned

3

Reprogramming is the shifting of funds within an appropriation to purposes different from those

contemplated at the time the appropriation was requested and provided. Several appropriations

subcommittees use program activity structures to establish reprogramming thresholds. If an agency

needs to shift funds from one activity to another above the threshold, it is expected to notify the

appropriate subcommittee.

Page 92 GAO/GGD-97-109 Results Act
Chapter 6 

Efforts to Integrate Performance

Information With Budget Decisions and to

Develop a New Reporting Framework Are

Promising

whether changes would frustrate congressional oversight. The staff

generally viewed these structures as fundamental to congressional

oversight of agency activities; thus, they viewed changes with

apprehension and concern. However, some agency officials saw program

activity structures as secondary to planning; thus, where current program

activity structures proved unsuitable for planning purposes, these officials

viewed change in program activity structures as inevitable and

appropriate. However, they noted that negotiating changes with Congress

could prove difficult and time-consuming.

Integrated

Performance, Cost,

and Management

Reports Provide Basis

for Supporting Key

Decisions

Our work has shown that congressional and executive branch

decisionmakers have been handicapped by the absence of reliable and

valid performance, program cost, and management information needed to

make well-informed decisions.4

 If successfully implemented, the

performance measurement and reporting requirements under the Results

Act will provide congressional and executive branch decisionmakers with

a wealth of information on the results of agencies’ efforts. We have

testified that such information will be most useful to congressional and

executive branch decisionmakers if it is consolidated with critical

financial and program cost data in financial statements, which agencies

are to produce and have audited under the CFO Act.5

 The CFO Act also sets

expectations for agencies to deploy modern systems to replace existing

antiquated, often manual, processes; develop better performance and cost

measures; and design results-oriented reports on the government’s

financial condition and operating performance by integrating budget,

accounting, and program information.

The results-oriented reports will be most effective if they support a wide

range of decisions—including budget, policy, program management, and

accountability—that are routinely made by Congress and the executive

branch. The challenge of developing reports that are specifically tailored

to meet decisionmakers’ needs and that are based on appropriate

accounting standards is being met by FASAB. FASAB has put together a new

set of accounting concepts and standards based on a framework that

4

See Information Resources: Summary of Federal Agencies’ Information Resources Management

Programs (GAO/IMTEC-92-13FS, Feb. 13, 1992); Financial Management: CFO Act Is Achieving

Meaningful Progress (GAO/T-AIMD-94-149, June 21, 1994); Government Reform: Using Reengineering

and Technology to Improve Government Performance (GAO/T-OCG-95-2, Feb. 2, 1995); Managing for

Results: Critical Actions for Measuring Performance (GAO/T-GGD/AIMD-95-187, June 20, 1995); and

High-Risk Areas: Actions Need to Solve Pressing Management Problems (GAO/T-AIMD/GGD-97-60,

Mar. 5, 1997).

5

Financial Management: Continued Momentum Essential to Achieve CFO Act Goals

(GAO/T-AIMD-96-10, Dec. 14, 1995).

Page 93 GAO/GGD-97-109 Results Act
Chapter 6 

Efforts to Integrate Performance

Information With Budget Decisions and to

Develop a New Reporting Framework Are

Promising

financial reporting should demonstrate accountability and provide

information useful for planning, controlling, and conducting the federal

government’s functions. These concepts and standards underpin OMB’s

guidance to agencies on the form and content of their agencywide

financial statements.

The new accounting concept statements—the Objectives of Federal

Financial Reporting and Entity and Display—establish a new reporting

model for federal agencies that is geared to providing users with

information about budgetary integrity, operating performance,

stewardship, and systems and controls. Through this new reporting model,

Congress, agency decisionmakers, and the American public will receive

annual audited financial reports that are to

• present a top-level overview discussion of the significance of the financial

information in accounting reports and commentary on performance goals

and results that are consistent with measures an agency uses under the

Results Act;

• report, and thus make auditable, (1) uses of budgetary resources that are

made available by Congress, (2) obligations incurred, (3) the balance of

budgetary resources, and (4) outlays;

• show total cost, earned revenues (if any), and net cost of agency

operations and programs in a manner that enables users to relate costs to

outputs and results;

• report (1) unspent funds; (2) other operating assets available for use in

providing government goods, services, and benefits; and (3) all liabilities

incurred, with separate displays of those that would require future funding

by Congress; and

• reflect the nation’s assets and investments for which the federal

government has a stewardship responsibility.

As authorized by GMRA—which expanded the requirements of the CFO

Act—OMB is piloting accountability reports that are consistent with the

FASAB accountability reporting concept. OMB worked with six agencies to

pilot the development of accountability reports for fiscal year 1995 and

added an additional eight agencies to the pilot test in fiscal year 1996.

According to OMB, additional agencies will produce accountability reports

for fiscal year 1997. By seeking to consolidate and integrate the separate

reporting requirements of the Results Act, the CFO Act, and other specified

Page 94 GAO/GGD-97-109 Results Act
Chapter 6 

Efforts to Integrate Performance

Information With Budget Decisions and to

Develop a New Reporting Framework Are

Promising

acts, the accountability reports are to show the degree to which an agency

met its goals, at what cost, and whether the agency was well run.6

Along with the new reporting model, FASAB developed cost accounting

standards that are to be effective beginning with fiscal year 1997—the first

set of standards that are to account for the costs of federal programs.

These standards require agencies to develop measures of the full costs of

carrying out a mission or producing products or services. Thus,

decisionmakers are to have information on the costs of all resources used,

including the cost of services provided by others to support activities or

programs. Such information would allow for comparisons of the costs of

various programs and activities with their performance outputs and

results. To help agencies implement the cost accounting standards and

related requirements set forth in the CFO Act and the Result Act, the Joint

Financial Management Improvement Program (JFMIP)—which develops the

requirements that agencies’ financial management systems are to

meet—has issued an exposure draft, entitled “Managerial Cost Accounting

System Requirements.”7

 This document is intended to facilitate the

acquisition and development of managerial cost accounting systems by

specifying the minimum information and functional processing

requirements agencies needed to accumulate and assign cost data

consistent with governmentwide guidance.

In addition, fiscal year 1997 will usher in the new requirements of the

Federal Financial Management Improvement Act of 1996, which will help

ensure greater attention to making much needed improvements in

underlying financial management systems. This act requires agencies’

financial management systems to comply with FASAB standards and JFMIP

system requirements.

Our work has shown that developing the necessary information systems to

gather, process, and analyze the needed program and cost information will

be a substantial undertaking for most federal agencies because of

agencies’ persistent weaknesses in the use of information technology.8

 To

help address problems in agencies’ use of information technology,

Congress enacted the Paperwork Reduction Act of 1995 and the

6

Managing For Results: Using GPRA to Assist Congressional and Executive Branch Decisionmaking

(GAO/T-GGD-97-43, Feb. 12, 1997).

7

JFMIP is a joint initiative of OMB, GAO, the Department of the Treasury and the Office of Personnel

Management, which are working in cooperation to improve financial management practices

throughout the federal government.

8

See, most recently, High-Risk Series: Information Management and Technology (GAO/HR-97-9, Feb.

1997).

Page 95 GAO/GGD-97-109 Results Act
Chapter 6 

Efforts to Integrate Performance

Information With Budget Decisions and to

Develop a New Reporting Framework Are

Promising

Clinger-Cohen Act of 1996. This information technology reform legislation

introduced requirements, in large part based on GAO’s research, that are

intended to significantly improve agencies’ management, including how

they select and manage information technology resources. We have

reported that together with GPRA and the CFO Act, this reform legislation

directs agencies to implement a framework of modern technology

management—one based on practices followed by leading public sector

and private sector organizations that have successfully used technology to

dramatically improve performance and meet strategic goals.9

Conclusions The Results Act’s annual performance plans are to establish a direct

linkage between the goals in those plans and the congressional and

executive branch budget processes. This linkage between the Results Act

and budget decisions is key to Congress and the executive branch

ultimately being able to connect resources to results. Our work reviewing

the Results Act and previous performance budgeting efforts suggests that

due to the great variability among agencies’ program activity structures,

Congress and the executive branch will likely be challenged as they

attempt to link performance goals with the budget’s program activity

structures.

Therefore, the successful integration of performance goals developed

under the Results Act into budget decisionmaking may take several annual

budget cycles before Congress, OMB, and executive agencies are able to

make significant progress. We have noted that in the short term, the

challenges in reaching agreements on whether and how to realign program

activities may require agencies to develop crosswalks between their

performance goals and current program activity structures. However, if

long-term progress in this integration is not achieved, the overall

usefulness of the Results Act’s performance plans as a tool for improved

congressional and executive branch decisionmaking may be

compromised.

Reports that present an agency’s financial condition and the results of its

operations in an integrated way also hold promise for enhancing the

usefulness of performance information. We have noted that such reports,

which are independently audited, will help correct the lack of complete

and reliable information that has been a source of concern for

congressional and executive branch decisionmakers for decades. Under

9

GAO/AIMD-94-115, May 1994 and Assessing Risks and Returns: A Guide for Evaluating Federal

Agencies’ IT Investment Decision-making (GAO/AIMD-10.1.13, Feb. 1997).

Page 96 GAO/GGD-97-109 Results Act
Chapter 6 

Efforts to Integrate Performance

Information With Budget Decisions and to

Develop a New Reporting Framework Are

Promising

FASAB’s new cost accounting standards, agencies are also required, for the

first time, to provide decisionmakers with measures of the full costs of

carrying out a mission or providing a service. If fully and effectively

implemented, integrated accountability reports can facilitate

congressional and executive branch decisionmaking by consolidating

performance, cost, and financial data in a single, user-friendly document.

Page 97 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 98 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 99 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 100 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 101 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 102 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 103 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 104 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 105 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 106 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 107 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 108 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 109 GAO/GGD-97-109 Results Act
Appendix I 

Survey on Results-Oriented Management

Practices

Page 110 GAO/GGD-97-109 Results Act
Appendix II 

Major Contributors and Acknowledgements

General Government

Division

Michael Brostek, Associate Director, (202) 512-9039

J. Christopher Mihm, Acting Associate Director, (202) 512-3236

Joseph S. Wholey, Senior Advisor for Evaluation Methodology

Thomas M. Beall, Senior Social Science Analyst

Donna J. Byers, Senior Evaluator

William R. Chatlos, Senior Social Science Analyst

Joyce D. Corry, Senior Evaluator

Victoria M. O’Dea, Senior Evaluator

Kathleen M. Peyman, Senior Evaluator

Dorothy L. Self, Evaluator

Lisa R. Shames, Senior Evaluator

Kiki Theodoropoulos, Senior Evaluator

Acknowledgements In addition to those named above, the following individuals made

important contributions to this report: From the Accounting and

Information Management Division: Linda F. Baker, Senior Evaluator;

Michael J. Curro, Assistant Director; and Carolyn Yocum, Senior

Evaluator. From the General Government Division: James M. Fields,

Senior Social Science Analyst; Tracy J. Harris, Report Production

Assistant; Catherine M. Hurley, Computer Programmer Analyst; Stuart M.

Kaufman, Senior Social Science Analyst; Donna M. Leiss, Communications

Analyst; Susan Ragland, Senior Evaluator; Theodore H. Saks, Senior

Evaluator; Stephanie Shipman, Assistant Director; Gregory H. Wilmoth,

Senior Social Science-Analyst; and Ellen Wineholt, Senior Evaluator. From

the Office of Information Management and Technology: Delaney Branch,

Customer Service Representative; Kenneth L. Cooper, Electronic

Publishing Specialist; Gail M. Traynham, Lead Electronic Publishing

Specialist; and Wanda L. Okoro, Customer Service Representative.

Page 111 GAO/GGD-97-109 Results Act
Appendix II 

Major Contributors and Acknowledgements

Page 112 GAO/GGD-97-109 Results Act
Appendix II 

Major Contributors and Acknowledgements

Page 113 GAO/GGD-97-109 Results Act
Appendix II 

Major Contributors and Acknowledgements

Page 114 GAO/GGD-97-109 Results Act
Related GAO Products

Managing for Results: Analytic Challenges in Measuring Performance

(GAO/HEHS/GGD-97-138, May 30, 1997).

Agencies’ Strategic Plans Under GPRA: Key Questions to Facilitate

Congressional Review (GAO/GGD-10.1.16, May 1997).

GPRA: Managerial Accountability and Flexibility Pilot Did Not Work As

Intended (GAO/GGD-97-36, Apr. 10, 1997).

Performance Budgeting: Past Initiatives Offer Insights for GPRA

Implementation (GAO/AIMD-97-46, Mar. 27, 1997).

Measuring Performance: Strengths and Limitations of Research Indicators

(GAO/RCED-97-91, Mar. 21, 1997).

Managing for Results: Enhancing the Usefulness of GPRA Consultations

Between the Executive Branch and Congress (GAO/T-GGD-97-56, Mar. 10,

1997).

Managing for Results: Using GPRA to Assist Congressional and Executive

Branch Decisionmaking (GAO/T-GGD-97-43, Feb. 12, 1997).

DOD’s GPRA Implementation (GAO/NSIAD/GGD-97-65R, Jan. 31, 1997).

Child Support Enforcement: Reorienting Management Toward Achieving

Better Program Results (GAO/HEHS/GGD-97-14, Oct. 25, 1996).

Information Technology Investment: Agencies Can Improve Performance,

Reduce Costs, and Minimize Risks (GAO/AIMD-96-64, Sept. 30, 1996).

Federal Downsizing: Better Workforce and Strategic Planning Could Have

Made Buyouts More Effective (GAO/GGD-96-62, Aug. 26, 1996).

Information Management Reform: Effective Implementation Is Essential

for Improving Federal Performance (GAO/T-AIMD-96-132, July 17, 1996).

Managing for Results: Key Steps and Challenges in Implementing GPRA in

Science Agencies (GAO/T-GGD/RCED-96-214, July 10, 1996).

Executive Guide: Effectively Implementing the Government Performance

and Results Act (GAO/GGD-96-118, June 1996).

Page 115 GAO/GGD-97-109 Results Act
Related GAO Products

Budget and Financial Management: Progress and Agenda for the Future

(GAO/T-AIMD-96-80, Apr. 23, 1996).

Budget Issues: Deficit Reduction and the Long Term (GAO/T-AIMD-96-66,

Mar. 13, 1996).

Managing for Results: Achieving GPRA’s Objectives Requires Strong

Congressional Role (GAO/T-GGD-96-79, Mar. 6, 1996).

GPRA Performance Reports (GAO/GGD-96-66R, Feb. 14, 1996).

Transforming the Civil Service: Building The Workforce of the Future,

Results of A GAO-Sponsored Symposium (GAO/GGD-96-35, Dec. 20, 1995).

Financial Management: Continued Momentum Essential to Achieve CFO

Act Goals (GAO/T-AIMD-96-10, Dec. 14, 1995).

Financial Management: Momentum Must Be Sustained to Achieve the

Reform Goals of the Chief Financial Officers Act (GAO/T-AIMD-95-204, July 25,

1995).

Managing for Results: Status of the Government Performance and Results

Act (GAO/T-GGD-95-193, June 27, 1995).

Government Restructuring: Identifying Potential Duplication in Federal

Missions and Approaches (GAO/T-AIMD-95-161, June 7, 1995).

Government Reorganization: Issues and Principles (GAO/T-GGD/AIMD-95-166,

May 17, 1995).

Executive Guide: Improving Mission Performance Through Strategic

Information Management and Technology (GAO/AIMD-94-115, May 1994).

(410135) Page 116 GAO/GGD-97-109 Results Act
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 6015

Gaithersburg, MD 20884-6015

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100