United States General Accounting Office

GAO/GGD-98-162

GAO Report to the Chairman, Subcommittee

on Civil Service, Committee on

Government Reform and Oversight,

House of Representatives

PERFORMANCE

MANAGEMENT

Aligning Employee

Performance With

Agency Goals at Six

Results Act Pilots

September 1998

United States

General Accounting Office

Washington, D.C. 20548

General Government Division

B-275942

Page 1 GAO/GGD-98-162 Performance Management

GAO

September 4, 1998

The Honorable John L. Mica

Chairman, Subcommittee on Civil Service

Committee on Government Reform and Oversight

House of Representatives

Dear Mr. Chairman:

Now in its first year of governmentwide implementation, the Government

Performance and Results Act of 1993 was intended by Congress to shift the

focus of federal agencies away from such traditional concerns as staffing

and activity levels and onto a single overriding factor: results. Achieving a

corresponding shift in the focus of federal employees themselves—toward

a greater awareness of their role in their agencies’ missions and toward

greater accountability for accomplishing their agencies’ goals—is widely

considered a key to fulfilling the promise of the act.

Known as the Results Act, the legislation requires federal agencies to set

goals, measure performance, and report on their accomplishments. As

federal agencies have begun to implement the Results Act, some agencies

have increasingly recognized that effectively aligning their employee

performance management approaches with their missions and goals is a

crucial part of becoming performance-based. The first federal entities to

grapple with implementation were the Results Act pilot projects, which

were called for under the act to develop a body of experience in

implementing key provisions and addressing issues and challenges

involved in becoming performance-based.

You have stated that for the Results Act to succeed in improving the

performance of federal agencies, it requires a mechanism to link

employees’ performance with the results expected of their agencies.

Because of your interest in the issue, you asked us to review selected

Results Act pilot projects that, while developing performance-based

management approaches, had taken steps to align employee performance

management measures and practices with organizational missions and

goals. This report, which responds to your request, provides information

on (1) the primary approaches taken by selected pilot projects to align

employee performance management with organizational missions and

goals and (2) the issues or challenges, if any, that these pilot projects

commonly confronted while developing and implementing these

approaches.
B-275942

Page 2 GAO/GGD-98-162 Performance Management

Although all six pilot projects we reviewed included specific efforts to

align their employee performance management systems with

organizational missions and goals, their approaches varied in significant

ways. One way was in the kinds or groups of employees on whom they

focused. Four of the pilot projects focused on managers rather than on

staff at other levels. Officials at the four pilot projects commonly said the

focus was on managers because they were the ones most responsible for

the business of the organization and for implementing change among its

employees. In contrast, the two other pilot projects we reviewed focused

their approaches on essentially all employees, relying on total quality

management (TQM) principles as a guide and organizing employees into

self-directed work teams.

Another way the pilot projects varied was in the extent to which

organizational goals were explicitly reflected in employees’ performance

plans. This explicit “linking” was commonly, although not uniformly, done

for the managers who were the focus of their agencies’ performance

management efforts. It was generally done less consistently for employees

at lower levels of these agencies. At the two pilot projects using selfdirected teams, the “link” to organizational goals was, at least initially, less

explicit than was the expectation that employees would contribute to their

teams’ performance.

The pilot projects also varied in whether they emphasized creating

accountability for results at the individual or team level. At the four pilot

projects at which managers were the focus of performance management, a

common factor was their generally traditional approach: managers’

accountability for results was carried out through a system of expectation

setting and evaluation by their superiors. In contrast, at the two pilot

projects using self-directed teams, officials emphasized TQM principles

and focused on improving work processes and team performance rather

than on conventional supervisory evaluations of employees. This

difference was further reflected in the extent to which feedback by fellow

employees was used as a performance management tool.

None of the six pilot projects we reviewed reported having formally

evaluated its employee performance management approach, but all

reported possible benefits, some involving improved teamwork and

communications and perceptions of greater accountability, and some

involving higher customer satisfaction and improved service delivery. In

addition, all six pilot agencies considered their approaches worth pursuing

further, as demonstrated by the fact that each of them continued to refine

Results in Brief
B-275942

Page 3 GAO/GGD-98-162 Performance Management

or expand upon its efforts to align performance management with

organizational missions and goals after the pilot project phase ended.

Despite the variety of approaches they took, the six pilot projects faced

several common issues or challenges. The first of these involved

identifying the flexibilities available to them—including human resource

management (HRM) waivers and demonstration project authority—to

tailor their HRM systems to their organizational missions and goals and

other organizational circumstances or needs. Officials at five of the six

pilot projects said they had requested HRM waivers, but that, in general,

their requests either did not gain approval or received no response at the

departmental level. Three of the six pilot projects either have become or

are expected to become demonstration projects, although none was a

demonstration project during the period in which it was a Results Act pilot

project (that is, fiscal years 1994 to 1996). However, despite the lack of

HRM waivers or demonstration project authority during the pilot project

period, all six pilot agencies found sufficient flexibilities available to them

to take at least some steps toward aligning their performance management

systems with their missions and goals and to address other organizational

circumstances or needs.

The second issue or challenge we identified involved the six pilot projects’

efforts to include in their performance management approaches

appropriate and meaningful goals and performance measures. Most of the

pilot projects that focused their performance management efforts on

managers took generally similar approaches; in essence, the goals and

measures for the managers’ units or functional areas were the goals and

measures for the managers. These pilot projects varied widely in their

efforts to “cascade” these goals and measures to employees at lower

levels. In contrast, the two pilot projects that used self-directed work

teams took a different approach, establishing performance standards for

employees using goals and measures that varied somewhat from those

used at the organizational or unit level, and which were intended to focus

employees on their contributions to their teams’ performance.

Third, as the pilot projects worked to become more performance-based,

each was confronted with the need to redirect its organizational culture

toward a new understanding of the organization’s mission or way of doing

business and to secure the buy-in of employees. Pilot project officials who

spoke about redirecting organizational culture almost uniformly said that

cultural change at their agencies had not yet been fully accomplished.
B-275942

Page 4 GAO/GGD-98-162 Performance Management

The fourth issue or challenge we identified involved unintended

consequences that pilot project officials said they confronted in

implementing their new approaches, some finding that employees might

try to “game” the system—that is, manipulate the performance measures to

make their performance look better than it might actually have been—or

might perceive a lack of fairness in the approach’s implementation.

Officials at the six pilot agencies generally were aware of their employees’

concerns and reported steps they had taken to keep abreast of the views of

the managers and other employees who were the focus of the agencies’

performance management efforts.

The Results Act has lent new relevance to questions of how to align the

management of federal employees with their agencies’ missions and goals.

The Results Act itself calls specifically for agencies to address HRM issues

in the context of performance-based management, requiring that agencies’

multiyear strategic plans and annual performance plans describe how the

agencies will use their human resources to support the accomplishment of

agency goals and objectives. Similarly, in its guidance for preparing agency

strategic plans, the Office of Management and Budget (OMB) underscored

the importance of managing employees for results, directing federal

agencies to “outline the process for communicating goals and objectives

throughout the agency, and for assigning accountability to managers and

staff for achievement of objectives.”1

In our previous work, we noted that successful performance management

systems depend, in part, on agencies putting in place a framework of

clearly defined organizational missions, goals, and performance measures

that can serve as the basis for setting employees’ performance

expectations and evaluating their performance. But two broad issues are

involved here and both have proven challenging to agencies: first, how to

establish this framework of clearly defined organizational missions, goals,

and performance measures; second, how to develop and implement

employee performance management systems that are aligned with these

organizational missions, goals, and performance measures so that federal

employees can contribute most effectively to their agencies’ success. Part

of the latter challenge involves working within the flexibilities available to

agencies to tailor employee performance management systems to

organizational missions and needs.

 

1

OMB Circular A-11, June 1997, Sec. 210.8.

Background
B-275942

Page 5 GAO/GGD-98-162 Performance Management

In our early work on Results Act implementation, we found that clearly

defining organizational missions, goals, and performance measures were

key steps for agencies in becoming results-oriented.2

 Our later work

revealed some of the challenges involved. For example, when we reviewed

the experiences of five regulatory agencies covered by the President’s

March 4, 1995, memorandum directing them to measure agency and

employee performance in terms of results, officials at these agencies cited

various barriers to becoming results-oriented, most of which involved

reaching agreement on agency goals and developing results-oriented

performance measures. The officials told us that some of these barriers

made it more difficult for them to set strategic goals, identify resultsoriented performance measures, or create results-oriented performance

standards for employees.3

 In our broader, governmentwide review of

Results Act implementation, we identified similar challenges to becoming

results-oriented among other agencies, including the difficulties of

reaching consensus on agency missions and of building organizational

cultures to create and sustain a focus on results at all levels.4

Agencies that have sought ways of developing and implementing employee

performance management systems that are aligned with their

organizational missions and goals—or with other specific circumstances

or needs—have commonly cited as a challenge the constraints of title 5

U.S.C., the statutory framework under which most of the executive branch

workforce is employed. In our 1993 study of agencies’ performance

management systems, we concluded that “One thing clearly needed is

greater flexibility among agencies to tailor their performance management

systems to the way they do their work, serve their customers, and manage

for results.”5

 Recognizing this need for flexibility, in 1995, the Office of

Personnel Management (OPM) issued new regulations under 5 C.F.R. 430,

allowing agencies, with OPM approval, to initiate new performance

management approaches for their employees, including pass/fail

performance ratings and the use of new appraisal elements to manage

group performance. However, whether agencies now have sufficient

flexibility to customize their HRM systems to their missions and needs

 2

Executive Guide: Effectively Implementing the Government Performance and Results Act (GAO/GGD96-118, June 1996).

3

Managing for Results: Regulatory Agencies Identified Significant Barriers to Focusing on Results

(GAO/GGD-97-83, June 1997).

4

The Government Performance and Results Act: 1997 Governmentwide Implementation Will Be Uneven

(GAO/GGD-97-109, June 2, 1997).

5

Federal Performance Management: Agencies Need Greater Flexibility in Designing Their Systems

(GAO/GGD-93-57, Feb. 24, 1993).

Previously Identified

Challenges of Becoming

Results-Oriented

Agencies’ Flexibility Is an

Ongoing Issue
B-275942

Page 6 GAO/GGD-98-162 Performance Management

remains an open question. For example, since the adoption of the new

performance management regulations, agencies such as the Federal

Aviation Administration, the Federal Bureau of Investigation, and the

Internal Revenue Service, have sought, and in some cases, obtained

exemptions from some or all the provisions of title 5.

Agencies have had an additional, although limited, measure of flexibility

available to them through OPM’s demonstration project authority, which

allows OPM to waive certain civil service rules so that federal agencies can

try new HRM approaches. OPM demonstration projects have focused on

such areas as streamlined hiring, classification, compensation systems,

and skill-based pay. The Civil Service Reform Act of 1978 (CSRA), which

established OPM’s demonstration project authority, specified that no more

than 10 demonstration projects may be active at any given time, that each

demonstration project may cover no more than 5,000 employees, and that

each project may take no longer than 5 years to complete.

During the 20 years in which OPM demonstration project authority has

been available, it has been put to only limited use. According to OPM,

since passage of CSRA, five OPM demonstration projects have been

completed and two—Navy’s China Lake facility and the National Institutes

of Standards and Technology (NIST)—have been made permanent by

legislation.6

 One other demonstration project is now active and another

has been formally proposed.7

 When we surveyed officials at 26 agencies

near the end of the demonstration program’s first decade, two reasons for

the agencies’ limited use of the demonstration project authority were most

widely cited: (1) the time and resources required to develop and propose

projects and (2) the difficulty of getting project proposals through

agencies’ approval processes.8

 6

Additionally, while this report was in production, Congress was considering legislation to make

permanent the recently completed demonstration project at the Department of Agriculture.

7

The active demonstration project is at the Department of Commerce. The proposed demonstration

project is at the Department of Veterans Affairs. Five additional demonstration projects are active at

Department of Defense (DOD) facilities. These demonstration projects were authorized by Congress

outside OPM demonstration authority, but were developed with input from OPM.

8

Federal Personnel: Status of Personnel Research and Demonstration Projects (GAO/GGD-87-116BR,

Sept. 1987). OPM has told us that these two reasons remain the most prominent.
B-275942

Page 7 GAO/GGD-98-162 Performance Management

When Congress passed the Results Act, it recognized that agencies would

need to make significant management changes and that substantial issues

in areas such as HRM would emerge during implementation of the act. For

this reason, it specified that the requirements of the Results Act would be

phased in over several years. Implementation of the Results Act would

begin with a series of pilot projects, under which a body of experience

would be developed in implementing the act’s key provisions and

addressing the issues and challenges involved in becoming performancebased. The first of these pilot projects, operating during fiscal years 1994

through 1996, were to focus on performance measurement and the

preparation of annual performance plans and reports by the participating

agencies. When these initial pilot projects concluded at the end of fiscal

year 1996, participants included 68 pilot projects representing 28 federal

agencies.

Some of these pilot projects, while grappling with the Results Act’s

performance planning and reporting requirements, included specific

efforts to align employee performance management with organizational

missions and goals. To inform the ongoing discussion about managing

employees more effectively in a performance-based environment, we

reviewed six of these pilot projects to report on the approaches they took

and to identify some of the specific issues or challenges that confronted

them.

To provide information on (1) the primary approaches taken by selected

Results Act pilot projects to align employee performance management

with organizational missions and (2) the issues or challenges, if any, that

these pilot projects commonly confronted while developing and

implementing these approaches, we judgmentally selected six pilot

projects for review and developed case studies on them. To inform our

selection, we carried out structured telephone interviews with pilot project

coordinators and/or human resource managers at the 68 pilot projects

operating at the end of fiscal year 1996, the conclusion of the pilot project

phase. We used data from the interviews to choose a diverse group of

organizations, based on (1) the varied extents to which they indicated

having taken steps to align employee performance management with

organizational missions and goals; (2) the types of agencies (e.g., DOD

versus non-DOD); (3) the types of missions (e.g., those focused on service

delivery, or on research and development, or on case processing); (4) the

performance management approaches they indicated they had taken; (5)

their geographical locations; and (6) their status or lack of status as

proposed performance-based organizations (PBO) or OPM demonstration

projects. We also applied the following guidelines: (1) exclude pilot

Some Results Act Pilot

Projects Included Employee

Performance Management

Efforts

Scope and

Methodology
B-275942

Page 8 GAO/GGD-98-162 Performance Management

projects covered by our other current or recent work addressing

performance management; (2) consider pilot projects that included

elements in which the requester expressed specific interest, such as

mechanisms to explicitly link employees’ performance to the results

expected of their agencies; and (3) give priority to pilot projects that

included multiple performance management tools.

We reached an agreement to proceed with our review of the six pilot

projects we had selected in a follow-up discussion with your office. The six

pilot projects we agreed to review were (1) the Army Audit Agency (AAA);

(2) the Army Research Laboratory (ARL); (3) the Department of Energy’s

Federal Energy Technology Center (FETC) in Morgantown, WV;9

 (4) the

National Oceanic and Atmospheric Administration (NOAA) of the

Department of Commerce; (5) the Small Business Administration (SBA);

and (6) the Department of Veterans Affairs’ (VA) New York Regional Office

(NYRO) in New York, NY.

Our review of the six pilot projects included site visits to each agency;

reviews of agency strategic plans and other agency documents as well as

pertinent articles, reports, and other materials produced by outside

organizations, including ourselves; structured and open-ended interviews

with responsible agency officials, including pilot project coordinators,

agency heads or highest-ranking available agency officials, and human

resource officials; and group interviews with managers and/or those most

affected by the pilot projects’ performance management approaches. The

participants in the group interviews were suggested by our primary

contacts at the agencies, with an understanding that participants be

chosen for their varied perspectives.

Much of the information we present in this report is the testimony of

project coordinators, human resource managers, or other officials at the

pilot projects. Except for a review of pertinent articles, reports, and other

materials available from other parties, we did not independently verify the

information.

The information presented in the six case studies, as well as our

observations on the approaches taken by the pilot projects we reviewed

 

9

FETC was created in 1997 by the organizational merger of Department of Energy technology research

centers in Morgantown and Pittsburgh, PA. The Morgantown Energy Technology Center (METC) was

the site of the Results Act pilot project from 1994 to 1996. After the merger with the Pittsburgh center,

the employee performance management approaches begun at METC were being incorporated into

FETC, with certain modifications. Most of the information we present relating to FETC is based on the

METC experience, but we will refer to the pilot agency as FETC for the sake of consistency.
B-275942

Page 9 GAO/GGD-98-162 Performance Management

and some of the issues and challenges they faced, is not generalizable

either to the broader universe of Results Acts pilots or to federal agencies

in general. Because our selection methodology emphasized diversity, we

do not intend that the six pilot projects be considered “best practices.”

We conducted our review in Morgantown, WV; New York, NY; and the

Washington, D.C. metropolitan area, from January 1997 through June 1998,

in accordance with generally accepted government auditing standards.

We solicited oral comments on a draft of this report from the heads of

AAA, ARL, FETC, NOAA, SBA, VA, and OPM, or their designees. All of

these organizations responded either with no comments or with comments

of a technical or clarifying nature. We considered the comments and made

changes as appropriate in finalizing this report.

All six pilot projects we reviewed included efforts to align their employee

performance management systems with organizational missions and goals

(see apps. I-VI). In general, pilot project officials said their employee

performance management approaches were intended specifically to

support their missions and goals and to address the organizations’

particular circumstances or needs. The extent to which these approaches

entailed a reworking of existing employee performance management

systems varied considerably. To the extent the pilot projects employed

new approaches, these also varied in significant ways, such as (1) in the

specific kinds or groups of employees on whom they focused, (2) in the

extent to which organizational goals were explicitly reflected in

employees’ performance plans, and (3) in whether these efforts

emphasized creating accountability for results at the individual or team

level.

At the time of our review, none of the six pilot projects reported having

formally evaluated its employee performance management approach, but

all reported possible benefits, some involving improved teamwork and

communications and perceptions of greater accountability, and some

involving higher customer satisfaction and improved service delivery.

Although officials at the pilot projects could rarely state with certainty that

their employee performance management approaches had improved

employee performance or had contributed measurably to the

accomplishment of organizational missions and goals, all six pilot projects

considered their approaches worth pursuing further, as demonstrated by

the fact that each of them continued to refine or expand upon its efforts to

align employee performance management with organizational missions

and goals after the pilot project phase ended.

The Six Pilot Projects

Varied in Their Efforts

to Align Employee

Performance

Management With

Organizational Goals,

but All Found These

Efforts Worth Pursuing
B-275942

Page 10 GAO/GGD-98-162 Performance Management

Four of the pilot projects we reviewed focused their employee

performance management approaches on managers rather than on staff at

other levels. At each of the pilot projects, the leadership chose a particular

group or level of managers as the focus of the organization’s efforts to

align employee performance management with missions and goals or other

organizational circumstances or needs. Pilot project officials said that the

focus of employee performance management was on managers because

they were the ones most responsible for the business of the organization

and for implementing change among the organization’s employees.

Each of these four pilot projects identified a mission-related goal or

circumstance on which it wanted to focus its managers’ efforts and for

which its managers would need to provide leadership to the staff. At AAA,

for example, the Auditor General wanted to instill a new customer-service

orientation among an auditing staff that held a traditional auditing

perspective. To lead the change, he turned to the agency’s GS-15 managers,

most of whom were program directors who headed the functional areas

into which the AAA’s audit work was organized. At ARL, where seven

preexisting “full-spectrum” research and development (R&D)

organizations had recently been merged into one, the Director aligned the

performance plans of the laboratory’s seven technical directorate and

center heads to support the organization’s new research-oriented mission

and to measure performance using its new “Performance Evaluation

Construct.” At NOAA, when the leadership wanted to introduce

agencywide strategic planning, it called upon its senior executive corps for

participation and support. And at SBA, when the administrator wanted to

increase capital lending to what had been identified as underserved

segments of the small business community, he began setting specific

capital lending goals for the directors of SBA’s district offices, which he

called “the primary vehicle for the Agency to achieve its mission.”

In contrast to these four pilot projects, the two others we reviewed

focused their employee performance management efforts on essentially all

of their employees. With the goal of enhancing mission accomplishment at

FETC and of improving service to veterans and their families at VA’s

NYRO, both organizations turned to TQM principles as a guide and

organized their employees into self-directed work teams.

The Pilot Projects Varied in

the Kinds or Specific

Groups of Employees They

Focused Upon
B-275942

Page 11 GAO/GGD-98-162 Performance Management

Although each of the pilot projects we reviewed made some efforts to align

its employee performance management systems with organizational

missions and goals, not every one required that organizational goals be

explicitly reflected in (i.e., “linked to”) employees’ performance plans. This

explicit “linking” was commonly, although not uniformly, done for the

managers who were the focus of their agencies’ employee performance

management efforts. It was generally done less consistently for employees

at lower levels of these same agencies. At the two pilot projects in which

employees worked in self-directed teams, the “link” to organizational goals

was, at least initially, less explicit than the expectation that employees

would contribute to their teams’ performance.

For managers who were the focus of their agencies’ employee

performance management efforts—at AAA, ARL, NOAA, and SBA—the

method of aligning individual performance plans with organizational goals

was fairly straightforward. The approach was generally one of establishing

goals and measures for managers’ organizational units (or functions) and

then holding these managers accountable for their units’ performance. For

example, capital lending goals at SBA were established for district offices,

but it was the district directors, as heads of these offices, who were

responsible for meeting these goals. The exception to this approach was at

AAA, where program directors and other GS-15 managers were designated

as members of “Team 15” and given identical performance expectations

aligned with the agency’s strategic goals, particularly those involving

customer service. However, the Auditor General told us that after 2 years,

he felt that not all of AAA’s program directors were responding to this

uniform approach. He said that beginning with the 1998-1999 ratings year,

he would assign Team 15 members more individualized performance

expectations, still reflecting the agency’s strategic goals but tailored to

their specific functional areas.

Although AAA, ARL, NOAA, and SBA explicitly built organizational goals

and measures into the performance plans of some of their top managers,

they did so with less consistency for employees at lower levels. AAA

reported making incremental, year-by-year progress in cascading strategic

goals down to staff at levels below GS-15. During the 1996-1997 ratings

year—the year after organizational goals were made an explicit part of

Team 15 members’ performance plans—AAA began drafting corresponding

standards for audit managers and staff at lower levels. By the 1997-1998

ratings year, the agency had adapted some of its five new “Corporate Goals

and Measures” into the staff’s performance plans. At ARL, where officials

said that the laboratory’s annual performance goals were reflected in the

performance plans of its senior executives, the same officials reported that

The Extent to Which

Organizational Goals Were

Reflected in Employees’

Performance Plans Varied
B-275942

Page 12 GAO/GGD-98-162 Performance Management

strategic human resource planning had not yet been integrated into the

organization’s overall strategic plan and that deployment down to the

workforce and first-level supervisors was inconsistent (see app. II).

Similarly, at NOAA, a top official told us that NOAA’s leadership wanted to

see how well the practice of including strategic planning goals in

individuals’ performance plans was working at the senior executive level

before extending it to others (see app. IV). The same official noted the

challenge involved in making employees at all levels—for example,

weather forecasters in the National Weather Service—aware of how their

individual efforts contribute to agency goals. A NOAA manager echoed this

concern, saying that while it was implicitly understood that strategic

planning goals should cascade down to the performance plans of lower

level staff, it was not clear to managers how to translate these goals into

individual employees’ performance expectations. At SBA, a human

resources official told us that district directors were required to cascade

the goals they were assigned down to their staff’s individual performance

appraisals whenever possible, but the same official noted that a recent

check by the Office of Human Resources showed that the practice was not

widespread. When we spoke with three SBA district managers, they said

they varied both in their approaches to cascading goals down to the staff

and in the levels of employees they included (see app. V).

At the two pilot projects we reviewed in which employees worked in selfdirected teams, organizational goals were not explicitly reflected in

individual employees’ performance plans. The emphasis at both these

organizations, officials said, was on team performance. Employees, they

said, were expected to contribute to organizational goals by contributing

to the performance of their teams. Individual performance standards were

established accordingly. At FETC, for example, officials said employee

performance standards emphasized the competencies and behaviors

considered necessary to support team efforts to meet FETC’s strategic

goals (see app. III). At VA’s NYRO, team goals were set and team

performance was tracked using the office’s five “balanced scorecard”

measures, but the performance measures for individuals were different

from the balanced scorecard measures and were designed to measure

employees’ contributions to their teams (see app. VI).

A key difference among the six pilot projects we reviewed was in whether

they chose to place the accountability for achieving results with individual

employees or with teams. At AAA, ARL, NOAA, and SBA, a common factor

was the generally traditional approach taken to accountability: managers’

accountability for results was carried out through a system of expectation

setting and evaluation by their superiors. We noted one exception: AAA’s

Individual Versus Team

Accountability for Results
B-275942

Page 13 GAO/GGD-98-162 Performance Management

Team 15 approach, under which program directors and other GS-15

managers were given identical expectations, ratings, and rewards. The

intent, AAA officials explained, was to make them accountable as a team

for improving the agency’s service to its customers. As explained earlier,

experience with the Team 15 approach led the Auditor General to decide

to retain the Team 15 concept but to create more individualized

expectations and to vary ratings and rewards according to individual

performance.

In contrast, accountability for achieving results at FETC and VA’s NYRO

was centered on self-directed work teams. Officials at both pilot projects

emphasized TQM principles and a focus on improving work processes and

team performance rather than on conventional supervisory evaluations of

employees. This fundamental difference was reflected in the extent to

which feedback by fellow employees, as opposed to supervisors alone,

was used as a performance management tool.

At FETC, 360-degree feedback from each employee’s supervisor,

subordinates (where applicable), peers, and internal customers was

gathered through a formal 360-degree survey, the results of which were to

be used for identifying areas in which employees’ performance could be

improved and for drawing distinctions among performers. At VA’s NYRO,

feedback from team members regarding each employee’s contributions to

the team—a process NYRO called “peer assessment”—was a consideration

in annual ratings, but it was also a means of flagging performance issues

needing managers’ attention. The criteria for peer assessment at NYRO

were not identical to the five balanced scorecard measures used to track

the performance of the teams themselves. Instead, NYRO officials told us,

the peer assessment criteria were intended to focus less on individual

productivity than on each employee’s contribution to the team in meeting

its organizational outcome measures, and were to be used to identify the

areas in which employees could improve their performance and thereby

contribute more effectively to the performance of the team.

Officials in FETC’s human resources division said that the 360-degree

feedback approach needed to be refined to make the system more

consistent in its application and to make the scoring less subjective. The

officials said that FETC expected to begin pilot testing the inclusion of

measurable performance goals in the 360-degree survey form and in formal

performance appraisals. One human resource official said this was

intended to emphasize employees’ contributions to their teams as well as

to measure their individual job performance. The official said that

including measurable goals would allow for performance expectations that
B-275942

Page 14 GAO/GGD-98-162 Performance Management

were more explicitly aligned with the agency’s annual performance plans

and that would provide for greater individual accountability.

Both FETC and VA’s NYRO began with five-level rating systems for

performance appraisals but have since adopted two-level (e.g., “pass/fail”)

systems. An FETC human resources official explained that the center used

the two-level approach, among other reasons, to counter the “grade

[ratings] creep” that had led to over 90 percent of employees being rated in

the top two categories. The official added her view that for a two-level

rating scale to work, a separate approach, such as the 360-degree feedback

system, must exist for determining distinctions among performers who

meet or exceed expectations. NYRO, which used a five-level rating scale

during the pilot project period, joined the rest of VA in adopting a two-level

system in fiscal year 1998. Of the other four pilot projects we reviewed,

only NOAA told us of plans to move to a two-level system. NOAA human

resource officials explained that employees’ within-grade increases would

still be determined by their ratings but that other awards would be delinked from ratings and would be determined, instead, by quarterly reviews

of their performance against organizational goals. One of the reasons for

going to two-level ratings, an official said, was simplification. Another

reason was to lessen the focus on individual performance and encourage

team performance. Still another reason was to make it easier to cascade

strategic goals down to levels below senior executives. The official said

the two-level ratings system would be evaluated after a year to see if these

intents had been accomplished.

Although both FETC and VA’s NYRO retained individual performance

award programs, both pilot projects developed new team awards programs

to highlight team accomplishments. Similarly, at NOAA, the human

resources official we spoke with said that more group awards were likely

as the agency did more to encourage team performance.

None of the six pilot projects we reviewed reported having formally

evaluated its approach toward aligning employee performance

management with organizational missions and goals, but all reported

possible benefits, some involving improved teamwork and

communications and perceptions of greater accountability, and some

involving higher customer satisfaction and improved service delivery. For

example, officials at AAA, FETC, and VA’s NYRO said their organizations

had improved communications and teamwork. At ARL, NOAA, and SBA—

pilot projects at which managers were the focus of the organizations’

employee performance management efforts—the managers we spoke with

commonly said they felt more accountable for their performance than they

Evaluative Data Were

Lacking, but Alignment

Efforts Were Perceived as

Beneficial
B-275942

Page 15 GAO/GGD-98-162 Performance Management

had previously. Program directors’ stated perceptions of their

accountability were more mixed at AAA, where, as Team 15 members, they

had been given identical performance expectations, ratings, and rewards.

Regarding customer satisfaction and service delivery, the four agencies

that instituted customer surveys—AAA, ARL, FETC, and NYRO—all

reported receiving higher scores from their customers since their

performance management efforts began. At SBA, where the agency’s

efforts were aimed mainly at increasing capital lending to underserved

segments of the small business community, lending to all these groups

improved the first year in which district directors were given capital

lending goals. At NYRO, where the office’s efforts were aimed at improving

veterans’ benefit claims processing, some of the organizational measures,

such as the number of cases pending, had improved in recent years. An

NYRO official cautioned, however, that the degree to which organizational

improvements could be directly attributed to NYRO’s performance

management efforts had not been formally evaluated, and that other

factors, such as hiring freezes and government shutdowns, had affected

the office’s productivity.

This NYRO official’s observation was not unique. The six pilot projects we

reviewed made their employee performance management efforts in

tandem with other steps they were taking to become performance-based

and while other events, such as downsizing, reorganizations, or leadership

changes, were taking place in their organizations. Therefore, pilot project

officials generally could not state with any certainty that any changes in

organizational performance were directly attributable to the organizations’

employee performance management approaches. At AAA, for example, the

thrust of the agency’s organizational efforts was to improve customer

service. AAA officials told us that customer survey scores had, indeed,

risen since these organizational efforts began, but they said they did not

know the extent to which these rising scores could be attributed to the

inclusion of customer service goals in Team 15 members’ performance

plans.

Despite the general lack of formal evaluative data regarding the results of

their employee performance management approaches, all six of the pilot

projects we reviewed found their approaches worth pursuing further, as

demonstrated by the fact that each of them continued to refine or expand

upon its efforts to align employee performance management with

organizational missions and goals after the pilot project phase ended. With

some changes, AAA continued the Team 15 approach, and began cascading

organizational goals into the performance plans of supervisors and

auditing staff. ARL continued refining its “Performance Evaluation
B-275942

Page 16 GAO/GGD-98-162 Performance Management

Construct” for providing information on the health and productivity of the

laboratory and for use in focusing expectations for its technical director

heads. FETC retained and continued to refine its team approach and 360-

degree feedback system even after an organizational merger with another

center. A NOAA official told us that strategic planning was “an iterative

process” that they were continuing to learn. SBA’s new administrator,

arriving after the conclusion of the pilot project phase, stated that the

agency “will link performance appraisals with performance against the

goals in the annual business plan.” And the balanced scorecard approach

begun in one division of VA’s NYRO was expanded to another division,

while the office prepared for a further evolution of its employee

performance management approach as an incipient OPM demonstration

project.

Despite the variety of approaches taken to align employee performance

management approaches with organizational missions and goals, the six

pilot projects we reviewed faced several common issues or challenges.

First, they sought to identify the flexibilities available to them to tailor

their HRM systems to their organizational missions and goals and other

specific organizational circumstances or needs. Second, to establish

employee performance standards, assign accountability for results, and

evaluate and reward employee performance, they sought to include in their

employee performance management approaches appropriate and

meaningful goals and performance measures. Third, as the pilot projects

worked to become more performance-based, each confronted the need to

redirect the organizational culture toward a new understanding of the

organization’s mission or way of doing business and to secure the buy-in of

employees. Fourth, pilot project officials said they confronted unintended

consequences in implementing their new employee performance

management approaches, some finding that employees might try to game

the system or might perceive a lack of fairness in its implementation.

Officials at the pilot projects we reviewed told us that their employee

performance management approaches were intended to address specific

organizational circumstances or needs—for example, a decision to make

the agency more customer-oriented, to introduce agencywide strategic

planning, or to change the organization’s job processes and have

employees work in teams. In some instances, the agencies expressed

concern with the extent to which current HRM rules and structures

allowed them to tailor their own HRM approaches to these needs. For

example, AAA’s 1997-2002 Strategic Plan stated the assumption that

“Personnel regulations and organizations will continue to be a stumbling

block to achieving Agency goals.” ARL’s demonstration project plan

In Aligning Employee

Performance

Management With

Organizational Goals,

the Pilot Projects

Faced Several

Common Issues or

Challenges

Flexibilities
B-275942

Page 17 GAO/GGD-98-162 Performance Management

proposal stated that “ARL finds the current Federal Personnel system to be

cumbersome, confusing, and unable to provide the flexibility necessary...”

and that the current system “often inhibits rather than supports the goals

of developing, recognizing, and retaining the employees needed to realign

the organization with its changing fiscal and production requirements.”10

VA’s proposal for a demonstration project involving its regional offices in

New York and Detroit stated similar concerns, including a lack of

flexibility for managers to structure teams effectively or encourage

employees to acquire a broader base of skills to meet organizational

demands.11

The pilot project officials who were involved in adapting their agencies’

employee performance management systems to their organizational needs

commonly had to explore, and then work within, the HRM flexibilities

available to them. They commonly reported looking into two areas:

administrative waivers and OPM demonstration project authority.

Several of the officials we spoke with said that the hope of gaining greater

flexibilities to tailor their HRM systems to their organizational needs was

one of the reasons their agencies had become pilot projects. At AAA, for

example, the Auditor General said that by becoming a pilot project, the

agency had expected to find it easier to obtain approval for a broadbanded classification and compensation system. Broadbanding, he said,

would allow AAA to promote auditors whose technical skills were high

without requiring that they become managers. Similarly, ARL’s director

said that, in part, the laboratory became a pilot project to obtain waivers

from administrative requirements promulgated both within and outside

DOD. The director said he was particularly interested in waivers that

would help the laboratory hire top scientists and reward those who were

doing the best work.

Despite their hopes that pilot project status would help them gain new

HRM flexibilities, the pilot projects we reviewed generally did not obtain

the HRM flexibilities they requested. In the case of AAA, for example, the

Auditor General said the Department of the Army (DA) did not support

AAA’s broadbanding proposal on the grounds that other Army agencies

were not interested in pursuing broadbanding and that broadbanding had

been amply demonstrated elsewhere. ARL’s pilot project manager told us

that most of the waivers that were approved for the laboratory were in the

 

10Federal Register, Vol. 62, No. 48, Mar. 12, 1997, p. 11647.

11Federal Register, Vol. 61, No. 224, Nov. 19, 1996, p. 58943.
B-275942

Page 18 GAO/GGD-98-162 Performance Management

areas of logistics and facilities management rather than HRM. He said that

most of ARL’s proposals for HRM waivers did not advance beyond DA.

Similar experiences were typical among most of the pilot projects we

reviewed. Five of the six pilot projects—all but SBA—said they had

requested HRM waivers, but none reported getting any HRM waivers of

significance. Officials generally told us their requests for HRM waivers

either did not gain approval or received no response at the departmental

level.

Another potential method by which the pilot agencies could seek

enhanced HRM flexibilities was through OPM’s personnel demonstration

project authority. In addition to requesting administrative HRM waivers as

part of their Results Act pilot project initiatives, four of the pilot agencies

we reviewed—AAA, ARL, NOAA, and VA’s NYRO—proposed becoming

demonstration projects. The agencies’ approaches to seeking

demonstration project approval varied, as did their success in the attempt.

AAA’s proposals, lacking support at the departmental level, did not

advance very far. In contrast, ARL’s proposal, made in concert with other

demonstration projects involving DOD research facilities and supported by

specific legislation, was implemented June 7, 1998, according to an ARL

official. Proposals for demonstration projects including all or parts of

NOAA and NYRO were both supported by their departments and covered

at least one other departmental entity; NOAA’s demonstration project is

now active and OPM said NYRO’s demonstration project could be

implemented early in calendar year 1999.

Officials at AAA said they packaged their request to develop a

broadbanded classification and compensation system in a proposal to

become a demonstration project. A provision for gaining direct hiring

authority was also included in the proposal. As mentioned earlier,

however, the request for broadbanding, along with the rest of the

demonstration project proposal, was not approved at the DA level. AAA

officials said they were discouraged by what they perceived as lengthy

planning and paperwork requirements and by the necessity for extensive

departmental approvals. Because agency resources were needed

elsewhere, the officials said, they abandoned the effort.

In contrast to AAA, ARL’s proposal to become a demonstration project has

advanced to the point of implementation.12 ARL made its proposal to

become a demonstration project in the framework of a wider DOD

 

12Called the Science and Technology Reinvention Laboratory Demonstration Project at the U.S. Army

Research Laboratory, it was formally proposed in the Federal Register, Mar. 12, 1997.
B-275942

Page 19 GAO/GGD-98-162 Performance Management

initiative to test new HRM approaches at DOD research facilities.13 The

proposal was legislatively authorized outside OPM’s demonstration

authority, but was developed with OPM’s input.14 According to the

proposal, ARL’s demonstration project was modeled on earlier

demonstration projects at Navy’s China Lake research facility—a project

that was made permanent by legislation in 1994—and at NIST, a project

that was made permanent by legislation in 1996. The ARL demonstration

project includes several major features: broadbanding, pay for

performance, automated classification, enhanced hiring flexibilities,

modified reduction-in-force procedures, and expanded developmental

opportunities for staff. An ARL official said that, similar to AAA, the

laboratory also sought direct hiring authority but could not obtain DA

support for it.

At NOAA, parts of three component organizations (the National

Environmental Satellite, Data, and Information Service; the National

Marine Fisheries Service; and the Office of Oceanic and Atmospheric

Research) are participating in an OPM demonstration project that was

implemented in March 1998 and which includes three other Department of

Commerce (DOC) entities.15 NOAA officials stated that the new

demonstration project is largely patterned after the earlier demonstration

project at NIST, another DOC organization. Among other features, the

demonstration project aggregates comparable occupations into career

paths and replaces the 15 General Schedule (GS) pay grades with five pay

bands. Although all career paths will have five pay bands, the salary range

in each band will differ from one career path to the next. Broadbanding,

NOAA said, is intended to give managers the ability to reward and retain

good performers and to provide more flexibility in attracting new hires.

The demonstration project includes other features as well, including a payfor-performance component that includes a direct link between

performance accomplishments and (1) the annual adjustment to basic pay,

 

13According to OPM, four other legislatively authorized DOD demonstration projects—all involving

research facilities, and all centering on pay for performance—have already been implemented; two

others have been formally proposed, and several others are in various stages of development. The

statutory basis for these demonstration projects was contained in the National Defense Authorization

Act for Fiscal Year 1995, which authorized the Secretary of Defense, with OPM’s approval, to conduct

personnel demonstration projects at DOD laboratories designated as Science and Technology

Reinvention Laboratories.

14Because these DOD demonstration projects were authorized separately by Congress, they do not

count against the statutory limit in 5 U.S.C. 4703 on the number of OPM demonstration projects that

may be active at any given time. That number, as explained earlier in the background section of this

report, is 10.

15The DOC demonstration project was proposed by OPM in the Federal Register, May 2, 1997.
B-275942

Page 20 GAO/GGD-98-162 Performance Management

which includes the annual general increase and the locality pay increase;

(2) annual performance pay increases; and (3) bonuses.

As discussed earlier in this report, in November 1996, VA proposed a

demonstration project for its regional offices in New York and Detroit,

expanding upon the HRM practices that NYRO was developing during the

Results Act pilot project phase. According to OPM, the demonstration

project could be implemented early in calendar year 1999.

Although three of the six Results Act pilot projects we reviewed either

have become or are expected to become demonstration projects, none was

a demonstration project during the period in which it was a Results Act

pilot project (that is, fiscal years 1994 to 1996). But despite this fact and

despite the fact that the pilot projects commonly did not obtain the HRM

waivers they requested, all of the pilot projects found sufficient flexibilities

available to them to take at least some steps toward aligning their

employee performance management systems with their missions and goals

and addressing other organizational circumstances or needs. The pilot

projects could, for example, establish organizational missions and goals,

establish strategic planning and performance measurement processes, and

align many HRM activities, such as employee performance evaluations and

rewards, within this framework.16 Further, some pilot project officials—at

ARL and VA’s NYRO, for example—said they came to realize that, in most

cases, waivers outside their departments were not needed to make

appropriate HRM changes. Structural HRM changes, such as in

classification or in creating broad-banded compensation systems, were the

kinds of changes more likely to require waivers or demonstration project

authority.

As discussed in the background section of this report, our previous work

on Results Act implementation showed that federal agencies have faced

certain common issues or challenges in becoming performance-based.

Two of these issues or challenges involved establishing organizational

goals and developing results-oriented performance measures. Both of

these issues have come into play at many levels among organizations

seeking to become more performance-based. Leading organizations we

studied in our previous work tried to establish clear hierarchies of goals

and performance measures, aligning the goals and measures for each

 

16OPM has produced a template of the flexibilities available to agencies. See Template of Personnel

Flexibilities for Use by Agencies Selected for Conversion to Performance-Based Organizations, OPM,

Mar. 27, 1996. For a discussion of how some agencies view and, in some cases, underestimate the

flexibilities available to them, see also, Transforming the Civil Service: Building the Workforce of the

Future—Results of A GAO-Sponsored Symposium (GAO/GGD-96-35, Dec. 20, 1995).

Goals and Measures
B-275942

Page 21 GAO/GGD-98-162 Performance Management

organizational level with those of successive levels and ultimately with the

organization’s strategic goals. Similarly, in trying to develop and implement

employee performance management approaches that were aligned with

their organizational missions and goals, the six pilot projects we reviewed

sought to include goals and performance measures that were appropriate

and meaningful at the employee level and that would be useful in

establishing employee performance standards, assigning accountability for

results, and evaluating and rewarding employee performance. The

approach to doing this generally differed between the pilot projects that

focused their employee performance management approaches on

managers and those that focused on teams. In addition, officials from both

groups commonly mentioned other issues and challenges involved in

establishing goals and measures.

As discussed earlier, the four pilot projects we reviewed that focused their

employee performance management efforts on managers took generally

similar approaches—with the exception of AAA—to establishing goals and

performance measures for these managers. In essence, the goals and

measures for the managers’ units or functional areas were the goals and

measures for the managers. The four pilot projects varied in their efforts to

“cascade” these goals and measures to employees at lower levels. Among

these four pilot organizations, only at AAA were coordinated efforts being

made during the time of our review to develop employee performance

plans in which organizational goals—in the case of AAA, “Corporate Goals

and Measures”—were explicitly reflected.

As Results Act planning and reporting pilot projects, all of the pilot

projects we reviewed took steps to adopt goals and measures appropriate

to their specific organizational missions and other circumstances and

needs. At ARL, for example, where the leadership faced the widely

recognized challenges involved in evaluating the performance of research

organizations,17 it adopted a three-part “Performance Evaluation

Construct” based on customer feedback, peer review, and metrics—three

elements that, used in various combinations, could provide information on

the relevance, productivity, and quality of the organization and its work. At

VA’s NYRO, where the leadership needed to focus its workforce on

improving service to benefit claims applicants, it retired its traditional

performance indicators and established a “balanced scorecard” designed

to set goals and measure performance in five areas: speed, accuracy, cost,

customer satisfaction, and employee development.

 

17See Measuring Performance: Strengths and Limitations of Research Indicators (GAO/RCED-97-91,

Mar. 1997).
B-275942

Page 22 GAO/GGD-98-162 Performance Management

The two pilot projects—FETC and VA’s NYRO—that emphasized

teamwork over individual performance both established performance

standards for employees that were somewhat different from the goals and

measures used at organizational or unit levels. Standards for FETC

employees, for example, although established to support the organization’s

strategic plan, focused largely on the competencies and behaviors

considered necessary to support their teams’ efforts to meet FETC’s

strategic goals. Similarly, the standards for NYRO employees were specific

to the contributions of members in benefit claims teams. The standards for

NYRO employees were not identical with the five balanced scorecard

measures by which their teams’ performance was tracked. NYRO officials

told us it was at the team level that these organizational outcome measures

were brought into the employees’ “line of sight,” but it was through

another set of criteria—output, customer service, reliability, team support,

and accuracy—that employees were evaluated. The employee standards

were developed to focus on employees’ contributions to their team in

meeting its organizational outcome measures. Their expectations, peer

assessments, formal evaluations, and individual awards were based on the

contributions they made to their teams. Additionally, “variable pay”—the

money made available for some awards to team members—was based on

teams’ overall scorecard performance, and was intended to reinforce the

importance of working together and achieving organizational goals.

Pilot project officials mentioned several other issues or challenges related

to goals and measures, including (1) making goals and measures more

outcome-oriented, (2) choosing the right number of goals and measures,

and (3) creating appropriate goals and measures when employees’

assignments were expected to take a long time to complete or could not be

expected to yield short-term results.

Pilot project officials commonly said that developing results-oriented goals

and measures was a continuing challenge. One SBA official, for example,

said that although the agency could measure the level of its capital lending

to the small business community, it was difficult to establish whether

SBA’s lending activities actually result in successful small businesses. A

NOAA official emphasized the fact that agencies must learn through

experience how to do effective strategic planning, adding that for strategic

goals and performance measures to become meaningful to employees,

they need to evolve over time. The official noted, for example, that two of

NOAA’s seven original agencywide strategic goals had been changed

through experience to be more outcome-oriented: “modernize navigation

systems” had become “promote safe navigation,” while “coastal ecosystem

health” had become “sustain healthy coasts.” Nonetheless, another NOAA
B-275942

Page 23 GAO/GGD-98-162 Performance Management

official noted the continuing difficulty of making the achievement of

broader outcomes an intrinsic part of employees’ approach to their work,

telling us that the performance plans of most agency staff are driven by the

performance of specific activities rather than tied to organizational goals.

At AAA, officials also spoke of continuing to refine performance measures

to give employees a better indication of the results of their work. For

example, whereas the audit agency used to simply track the number of

audit recommendations made, AAA officials said it now tracks the

percentage of audit recommendations implemented and the impact of

those recommendations.

Whereas pilot project officials noted the importance of the kinds of goals

and measures used in managing their employees, they said that the number

of goals and measures was important as well. At AAA, officials said the

agency had initially identified more than 40 strategic goals, a number that

was then reduced to 18. But the same officials said that having so many

strategic goals made it difficult to establish an organizational focus or to

create an alignment with the employee performance management system;

the goals were not prioritized and employees did not understand the

direction the agency wanted them to take. Recognizing the need for fewer,

more focused goals, AAA developed the five overarching “Corporate Goals

and Measures” that it began to build into employees’ performance plans

during the 1997-1998 ratings year. The need to identify which goals and

measures were most important was recognized at ARL as well. ARL

identified and continues to monitor approximately 60 metrics—i.e.,

quantifiable inputs and outputs, such as the number of PhDs on the staff or

the number of technical reports produced—but to focus the performance

of the laboratory’s directorate heads, ARL’s director selected only about 15

metrics for inclusion in directorate heads’ performance plans. In contrast,

at SBA, the number of goals and measures for district directors has grown

substantially beyond the original focus on capital lending. An SBA official

told us that the expanding number of goals and measures for which district

directors are responsible has made it hard to maintain a focus.

Some pilot project officials noted the difficulties involved in creating

appropriate goals and measures when employees’ assignments were

expected to take a long time to complete or could not be expected to yield

short-term results. The challenge was particularly pertinent to the

management of employees involved in research. For example, at NOAA’s

Office of Oceanic and Atmospheric Research (OAR), the Acting Assistant

Administrator said the office’s annual operating plans had to recognize the

long-term nature of research projects and stress the achievement of

interim milestones or the delivery of particular products, rather than the
B-275942

Page 24 GAO/GGD-98-162 Performance Management

completion of work that might take years. ARL, in addition to peer review

and the various metrics used to give an indication of the health and

performance of the laboratory, used customer feedback as an indicator of

the laboratory’s productivity. For customers to whom the laboratory

delivered specific, identifiable products, customer surveys were used. But

for strategic guidance and feedback on the laboratory’s long-range

contributions, ARL officials said the laboratory drew upon senior Army

leaders and user group representatives directly, through the formation of a

Stakeholders’ Advisory Board.

In discussing their efforts to align employee performance management

with missions and goals or other organizational circumstances or needs,

officials at the six pilot projects we reviewed spoke of the need to redirect

the organizational culture toward a new understanding of the

organization’s mission or way of doing business. The officials commonly

spoke of the organizational culture that had been in place, the kind of

organizational culture that was now required, and the strategies for making

the transition. The officials took a variety of steps to redirect the culture

and to gain the buy-in of employees. However, the officials who spoke

about redirecting organizational culture almost uniformly said that the task

of doing so was an ongoing challenge and that the transition at their

agency had not been fully accomplished.

At AAA, for example, the Auditor General believed that the agency needed

to move from a traditional auditors’ orientation to one focused more on

customer service. In 1992, he introduced TQM principles to the agency,

establishing a Quality Council and an Office of Productivity to coordinate

the agency’s efforts to improve its service to clients and involving its

employees in several ways, such as through Process Action Teams and

internal surveys. The goal, according to AAA, was to convert “an

introverted, task-oriented group into a entrepreneurial customer-service

force...” After 5 years’ efforts—including the introduction of the Team 15

concept, under which managers were to work together toward this

organizational goal—the Auditor General said his greatest challenge still

lay in accomplishing this change in culture—without which, he said, there

could be no meaningful change in organizational performance. He said that

“you’re pretty limited in what you can use to influence the performance of

managers,” citing what he said was a lack of meaningful performance

incentives and of the ready ability to remove managers from their

positions. However, he said that substantial progress in changing AAA’s

culture was being made—although it was being made, in his words, “battle

by battle.”

Redirecting Organizational

Culture and Securing

Employee Buy-in
B-275942

Page 25 GAO/GGD-98-162 Performance Management

The challenge of redirecting organizational culture was also pointed out at

ARL. The laboratory, created out of seven preexisting research and

development facilities, faced the task of integrating them into one

laboratory and refocusing the staff away from “full-spectrum” R&D and

onto basic and applied research. The laboratory’s first strategic plan

addressed four areas: culture, process, people, and structure. But it stated

that although these four areas are interrelated, culture is “the hardest to

change of all human attributes,” and reorganization alone would not be

sufficient to guarantee improved operational effectiveness or efficiency.

To help gain the buy-in of managers, ARL’s director established crossdirectorate teams for addressing each of the laboratory’s primary mission

areas and for encouraging senior managers to work toward common goals

and to share ownership and responsibility for results. ARL also established

a Staff Members Committee to serve as a channel of communications from

employees to top management during the development of the laboratory’s

new personnel demonstration project. One specific way in which ARL

officials said they tried to steer this cultural change was by benchmarking

the laboratory’s metrics goals against world-class research facilities such

as Bell Labs. ARL’s director said that by emphasizing such metrics as the

number of PhD’s on the staff or the number of research papers published,

he could send a message to the staff of what kind of organization he

wanted ARL to be. An ARL official told us, however, that this kind of

cultural change has not come easily to some staff. The official said, for

example, that when engineers are asked to do things that are more familiar

to scientists, such as publish research papers, they have found themselves

uncomfortable with meeting requirements that lie outside their

professional experience.

At FETC, another research facility, an official told us of the difficulties

encountered in transitioning the staff to a team environment. At a facility

at which the research workload was increasing while staff resources were

not, FETC’s leadership wanted to enhance its productivity through the

implementation of TQM principles. FETC created self-directed product

teams for doing the work of the organization and process action teams for

generating improvements. It emphasized teamwork-based standards for

employees’ performance and rewards, and a 360-degree appraisal system

so that staff members could tell each other how they were doing and

identify areas for improving their contributions to their teams. But the

same human resources official said that FETC continued to face a

challenge in getting its professional staff to work in teams; scientists and

researchers, the official said, tend to be introverted and independent, and

center’s traditional culture of competition rather than collaboration was

hard to change.
B-275942

Page 26 GAO/GGD-98-162 Performance Management

Officials at NOAA told us of somewhat similar challenges when the

leadership introduced agencywide strategic planning. NOAA’s

components, the officials said, had traditionally gone their own ways, but

were now asked to think of their interrelationships and their contributions

to the agency’s overall goals. When the agency’s new leadership decided to

adopt agencywide strategic planning, it took its senior managers off-site

for an extended period to reevaluate the agency and to develop

agencywide goals. NOAA’s strategic plan was the product of seven crossfunctional strategic planning teams, in which representatives of NOAA’s

various components were asked to develop strategic goals in concert with

one another rather than independently and which were intended to help

NOAA’s managers better understand their own and each others’ place in

the agency and their potential contributions to it. Such strategies

notwithstanding, a top NOAA official said that achieving the change in

organizational culture required for successful strategic planning remained

a challenge.

In March 1994, SBA’s administrator sent a memo to its 69 district directors

and other managers, identifying the characteristics, as he saw them, of

agencies that successfully manage for results, and stating that “I envision

the SBA as one of these agencies.” This memo, SBA officials said, was the

means by which the administrator set SBA’s course toward creating a

culture that would support performance-based management, with an

emphasis on accountability for results. In the memo, the administrator

announced that a key step for SBA would be becoming a Results Act pilot

project and that, in keeping with Results Act requirements, it would soon

develop a strategic plan—what he called a “business plan”—with

measurable performance goals. Since that time, although SBA has

proceeded to introduce performance-based practices, including

performance goals for its district directors and a system for tracking the

performance of its district offices against organizational targets, the

agency’s experience has been marked by a key factor: turnover in

leadership. Since 1994, SBA has had three administrators. As reflected in

our review of the pilot project (see app. V), changes in leadership at SBA

have entailed changes in the agency’s employee performance management

approach, including the performance goals of its district managers and the

means by which these goals were determined, as well as more basic

reconsiderations of the agency’s mission and goals.

The effort to improve service delivery to veterans and their families at VA’s

NYRO entailed fundamental changes in the office’s expectations of its

employees—for example, that they take on wider responsibilities for

claims processing and that they work in self-directed teams. Among other
B-275942

Page 27 GAO/GGD-98-162 Performance Management

things, coaches were to help team members appreciate the meaning and

importance of performance data, and the teams themselves met weekly to

track their performance. But in addition to using the “balanced scorecard”

performance measures and other performance management tools, an

NYRO document pointed out, “to change an organization, values and

behaviors play important roles.” An NYRO official said management’s view

was that how people work with each other and act in the office are critical

to an organization’s success, and that it saw job performance and personal

behavior as inseparably linked. Recognizing, for example, that the new

claims process and office environment gave the public open access, the

office adopted a dress code for all employees. NYRO reported that the staff

adapted to the dress code, but that other behavioral changes were more

difficult to achieve. For example, in 1995, NYRO management informed the

staff that the abuse of sick leave and the misuse of telephones by a

relatively small number of employees was affecting teamwork and

damaging the office’s standing versus other VA regional offices. As a result,

by 1996, the office had taken 52 disciplinary and adverse actions because

of leave abuse and telephone misuse. In fiscal year 1996, the office had 22

unfair labor practice charges filed against it, 43 grievances, and 16

arbitrations. Dealing with these actions required an investment of

resources, NYRO reported, but was necessary “if the NYRO was to stay

focused on its vision and the results it hoped to achieve.”

Pilot project officials told us of some unintended consequences their

agencies encountered in implementing their new employee performance

management approaches, some finding that employees might try to game

the system or might perceive a lack of fairness in its implementation.

Some pilot project officials noted that in establishing new goals and

measures or new appraisal systems for their employees, they had

encountered cases in which employees were “gaming” the system—that is,

manipulating the performance measures to make their performance look

better than it might actually have been. For example, FETC’s human

resources director said that when 360-degree appraisals were first tested at

the Morgantown center, some of the staff in one work team scored one

another with 10’s—the highest possible score—in all of the ratings

categories of the 360-degree survey. These employees, the same official

told us, were called into his office and informed that management was

aware of this attempt to game the system and that it would not be

tolerated. He said that word of these sessions spread through the center

and that gaming has not been a problem since then. Nonetheless, the same

official said that, with experience, FETC perceived a need to ensure that

360-degree scoring is done more consistently and equitably than in the

Unintended Consequences
B-275942

Page 28 GAO/GGD-98-162 Performance Management

past, especially since two research facilities—at Morgantown, WV and

Pittsburgh, PA, were merged into one. To make scoring less subjective, the

official said, FETC intended to better define the rating factors used in the

360-degree survey form and, where feasible, include measurable

performance goals for employees.

At AAA, agency officials pointed out the possibility of employees gaming

performance indicators, but also the possibility that the use of the

indicators themselves might skew performance. For example, because

AAA’s performance management efforts centered on improving the

agency’s service to its customers, the members of Team 15 were given

performance goals based on the composite scores received on customer

surveys. However, the Auditor General said, he had found that some

program directors were sending out fewer customer surveys than others,

targeting them toward respondents who were more likely than others to

give them positive ratings. He said the agency needed to give more

attention to the comments that customers wrote on the survey forms and

to the circumstances surrounding the ratings to better understand their

basis. AAA officials said this was necessary to avoid an unintended

consequence: that auditors might try so hard to get high customer survey

scores that they would sacrifice their independence. Their concern was

that auditors might be afraid that their audit findings could jeopardize the

customer survey scores on which their performance was evaluated.

The experiences of the pilot projects we reviewed indicated that

employees’ perceptions of unfairness were often factors in the extent to

which these organizations won support for their new employee

performance management approaches. Some examples are as follows:

• At AAA, the Auditor General said the uniform ratings for Team 15

members met with resentment from some lower-level staff, who felt that

GS-15 managers were being held less accountable for their performance

than they were. Some of the Team 15 members with whom we spoke

raised a similar concern on their own behalf.

• At ARL, the use of metrics as performance indicators remained a point of

contention, according to the pilot project manager, who said that some

directorate heads initially felt they were assigned metrics that were

inappropriate to their areas.

• At SBA, some of the district directors with whom we spoke said that

because district directors were given measurable performance goals while

other senior SBA managers were not, the “performance bar” was higher for

them than for the others. Some of the district directors regretted that their

goals were now assigned by the Office of Field Operations rather than
B-275942

Page 29 GAO/GGD-98-162 Performance Management

negotiated with the administrator, as was done in 1994 and 1995. One

district director said that if a district office meets its goals one year, the

goals go up the next, so that “no good deed goes unpunished.”

• At VA’s NYRO, officials said, some employees who had experienced peer

assessments by team members were not convinced of the competence of

some of their team members to assess others’ performance.

Officials at the six pilot agencies we reviewed were generally aware of the

kinds of fairness concerns raised by employees. Pilot project officials

commonly reported specific steps taken to keep abreast of the views of the

managers or other employees who were the focus of the agencies’

performance management efforts. These steps ranged from the creation of

a District Directors Advisory Council at SBA to periodic “internal climate

surveys” to keep abreast of all employees’ views at NYRO. The attempts to

gather information and learn from experience were in keeping with the

view expressed by several pilot project officials—that their transition to

performance-based management and to employee performance

management systems that were aligned with organizational missions and

goals was continuing, but that this “work in progress” was not yet

complete.

We are sending copies of this report to the Ranking Minority Member of

the Subcommittee; to the Chairman and Ranking Minority Member of the

House Committee on Government Reform and Oversight and the Senate

Committee on Governmental Affairs and its Subcommittee on

International Security, Proliferation, and Federal Services; the heads of the

six agencies we reviewed; the Director, OPM; and other interested parties.

We will also make this report available to others upon request.

Major contributors to this report are listed in appendix VII. Please contact

me at (202) 512-8676 if you have any questions concerning this report.

Sincerely yours,

Michael Brostek

Associate Director, Federal Management

and Workforce Issues
Page 30 GAO/GGD-98-162 Performance Management

Contents

Letter 1

Appendix I

Case Study: The Army

Audit Agency and

“Team 15”

34

Appendix II

Case Study: The Army

Research Laboratory

and the Performance

Evaluation Construct

37

Appendix III

Case Study: The

Federal Energy

Technology Center and

Its Team-Based

Performance

Management Approach

41

Appendix IV

Case Study: The

National Oceanic and

Atmospheric

Administration:

Securing SES

Commitment to

Strategic Planning

46
Contents

Page 31 GAO/GGD-98-162 Performance Management

Appendix V

Case Study: The Small

Business

Administration and

Goal Setting for

District Directors

50

Appendix VI

Case Study: VA’s New

York Regional Office

and Its “Scorecard”

Driven, Team-Based

Approach to

Performance

Management

55

Appendix VII

Major Contributors to

This Report

61
Contents

Page 32 GAO/GGD-98-162 Performance Management

Abbreviations

AAA Army Audit Agency

ARL Army Research Laboratory

DA Department of the Army

DOC Department of Commerce

DOD Department of Defense

FETC Federal Energy Technology Center

GAC Group Awards Committee

HRM human resource management

METC Morgantown Energy Technology Center

NIST National Institute of Standards and Technology

NOAA National Oceanic and Atmospheric Administration

NYRO New York Regional Office

OAR Office of Oceanic and Atmospheric Research

OMB Office of Management and Budget

OPM Office of Personnel Management

PBO performance-based organization

R&D research and development

SBA Small Business Administration

SES Senior Executive Service

TAPES Total Army Performance Evaluation System

TEAMS Technical Energy/Environmental and Management Services

TQM total quality management

VA Department of Veterans Affairs
Page 33 GAO/GGD-98-162 Performance Management
Appendix I

Case Study: The Army Audit Agency and

“Team 15”

Page 34 GAO/GGD-98-162 Performance Management

The Army Audit Agency (AAA), headquartered in Alexandria, Virginia, is

the Department of the Army’s (DA) internal audit organization. In fiscal

year 1997, AAA employed about 650 persons, of whom about 580 were

professional auditors.

According to the Auditor General, by 1992, AAA had come to believe that

budget pressures and downsizing threatened the agency’s survival. To

ensure AAA’s viability, he said, the agency needed to move away from its

traditional approach—one he called the “gotcha” role—to one more

focused on service to its Army clients. To help accomplish this culture

change, he turned to the agency’s GS-15 managers and initiated a

performance management effort called “Team 15.”

Team 15 was composed of the agency’s 21 GS-15 managers, 17 of whom

were program directors. The Auditor General said that as heads of the

functional areas into which AAA’s audit work is organized, the program

directors could play a critical leadership role in creating a customer

service orientation throughout the agency. In 1994, following 2 years of

agencywide total quality management (TQM) efforts, AAA issued its first 5-

year strategic plan. The plan included the long-term goal of “service to the

customer.” Beginning with the 1995-1996 ratings year, Team 15 was

established and “service to the customer” was made an explicit part of its

members’ performance plans.

As in the past, Team 15 members’ performance plans were to conform with

the departmentwide Total Army Performance Evaluation System (TAPES),

which includes a five-level rating scale. However, Team 15 members’

performance plans were now to include performance expectations

mirroring the AAA strategic plan’s customer service goals. For example,

one of the performance expectations for Team 15 members for the 1996-

1997 ratings year was to “Develop a 96-97 audit program that focuses on

the Army’s needs, issues and concerns as demonstrated by at least a 4.5

composite rating by the Secretariat and DA Staff of their satisfaction with

the subjects and timeliness of our scheduled audit services.” The

quantitative measure of performance was to be provided by customer

surveys distributed to audit requesters and audited parties after each audit.

The surveys asked respondents for ratings in such areas as competence of

the auditors, treatment of the client, and usefulness of the audit.

Respondents were asked to provide a rating in each area on an ascending

scale of 1 to 5 and to add their comments on problems or issues they

perceived and on additional services they might desire.
Appendix I

Case Study: The Army Audit Agency and “Team 15”

Page 35 GAO/GGD-98-162 Performance Management

According to AAA officials, another new feature of Team 15 was that the

Auditor General assigned the Team 15 members identical performance

expectations. The Auditor General said his intent was to encourage

program directors to put teamwork ahead of their individual interests in

pursuing agencywide goals. In addition, he said that during the 1995-1996

and 1996-1997 ratings years, he gave all Team 15 members identical

ratings. He said his intent was to convey to the team members the message

that their individual accomplishments were dependent on how the team

performed as a whole.

Team 15 members’ performance bonuses were also identical; each Team

15 member received a performance bonus of $800 at the end of the 1995-

1996 ratings year, and $1,400 at the end of the 1996-1997 ratings year.

At the time of our review, AAA had not formally evaluated the Team 15

approach. The Auditor General said that various measures, such as cycle

time and customer surveys, were used as indicators of team and individual

performance. AAA officials said that customer ratings of AAA’s audit work

were rising, but they did not know whether this was the result of including

customer service goals in Team 15 members’ performance plans. The

officials also said it was their impression that the Team 15 approach had

improved communications and teamwork.

The Auditor General said he planned to retain the Team 15 concept, but

with modifications. He said that, beginning with the 1998-1999 ratings year,

he would assign the team members more individualized performance

expectations, still aligned with the agency’s strategic goals but tailored to

their specific functional areas. Further, he said, beginning with the 1997-

1998 ratings year, he would give team members ratings and performance

bonuses based on their individual performance. The Auditor General gave

two reasons for these modifications. First, he said that although most of

the program directors had subscribed to AAA’s new emphasis on customer

service, teamwork, and the use of TQM, some had not; they were good at

what they had been doing, he said, but they had been doing it their way for

a long time and were reluctant to change. Second, he cited concerns

among some lower-level staff with the fact that all GS-15s received the

same rating regardless of their individual performance. The Auditor

General said that the dissatisfied staff members had asserted that some

GS-15 managers were being held less accountable for their performance

than employees below the GS-15 level.

AAA did not create a team structure comparable to Team 15 for audit

managers and staff below the GS-15 level. But beginning with the
Appendix I

Case Study: The Army Audit Agency and “Team 15”

Page 36 GAO/GGD-98-162 Performance Management

1997-1998 ratings year, performance standards for managers and staff from

the GS-14 level down to the GS-12 level were instituted that were explicitly

aligned with AAA’s five new “Corporate Goals and Measures,” including (1)

value to the Army, (2) customer satisfaction, (3) world class workforce, (4)

cycle time (timeliness), and (5) information technology. Developed by five

AAA employee teams working under the auspices of the agency’s Quality

Council, the performance standards were tailored to the duties of

employees at the GS-14 level (primarily audit managers) down to the GS-12

level (auditors). For auditors at the GS-11 level down to the GS-5 level,

performance standards were not explicitly aligned with the Corporate

Goals and Measures. Instead, each employee’s performance plan included

a statement highlighting the relevance of the Corporate Goals and

Measures and the fact that accomplishing individual performance

objectives would contribute to the agency’s ability to meet its strategic

goals. The Auditor General said that the new performance standards for

employees other than Team 15 members were one step in an ongoing

attempt to cascade the goals of the organization down to individual

employee performance plans.
Appendix II

Case Study: The Army Research Laboratory

and the Performance Evaluation Construct

Page 37 GAO/GGD-98-162 Performance Management

With principal locations in Adelphi and Aberdeen, Maryland, the Army

Research Laboratory (ARL) is the Army Materiel Command’s “corporate”

laboratory. According to its mission statement, ARL does “fundamental

and applied research to provide the Army with key technologies and the

analytical support necessary to assure supremacy in future land warfare.”

In fiscal year 1997, ARL employed over 2,300 people, of whom about 1,400

were scientists and engineers.

ARL was formed in October 1992 through the consolidation of seven

formerly independent Army laboratories. According to ARL’s pilot project

manager, the new laboratory’s emphasis on fundamental and applied

research was a major change from the “full spectrum” research and

development (R&D) mission of its seven predecessor organizations, which,

in addition to research, had carried out a variety of development,

engineering, and initial production tasks. He said the consolidation was

seen as an opportunity to introduce performance-based management

practices—namely, strategic planning and organizational performance

evaluation—to make ARL function in a more businesslike fashion.

As a research organization, a key challenge for ARL lay in developing

meaningful performance measures. As ARL and others have noted, the

results of research are often unpredictable and often require long time

frames in which to become apparent.1

 Further, as ARL’s Director pointed

out, the value of research outcomes is not always quantifiable. He added,

however, that while he believed that measuring the outcomes of research

itself was not feasible, reporting on such performance characteristics as

the relevance, productivity, and quality of the laboratory’s work was

possible and, indeed, vital for managing the organization. ARL’s strategy

for performance measurement was to focus on the performance and

functional health of the laboratory, gathering a range of information that

was both qualitative and quantitative. ARL called its approach the

“Performance Evaluation Construct.”

The Construct included peer review, customer feedback, and metrics.

According to the pilot project manager, these three tools could be used “in

various combinations and to varying degrees” to provide information on

the performance and functional health of the laboratory and its

components. Additionally, for ARL’s senior executives—i.e., the heads of

ARL’s seven technical directorates and centers—the Construct could be

 

1

Applying the Principles of the Government Performance and Results Act to the Research and

Development Function: A Case Study Submitted to the Office of Management and Budget, Dr. Edward

A. Brown, Army Research Laboratory, 1996. See also Measuring Performance: Strengths and

Limitations of Research Indicators (GAO/RCED-97-91, Mar. 1997).
Appendix II

Case Study: The Army Research Laboratory and the Performance Evaluation Construct

Page 38 GAO/GGD-98-162 Performance Management

used by the director to align their performance expectations with

organizational goals and to reinforce accountability through ratings and

performance bonuses.2

ARL officials said that at the beginning of each ratings year, the director

met individually with the laboratory’s senior executives to establish their

performance expectations. If peer reviews of previous or ongoing work

raised performance issues, these were to be addressed in the new

expectations. Further, the director would set a customer service goal

keyed to the directorates’ ratings on customer surveys. The surveys, sent

to customers to whom ARL had delivered specific products, asked

questions about such things as timeliness, technical quality, and value of

the work. The surveys used an ascending rating scale of 1 to 5; the goal for

the directorates’ average scores on these questionnaires was set at 4.8.3

The customer survey scores were one of the metrics for which specific

goals were included in the senior executives’ performance plans.

According to the pilot project manager, ARL’s metrics covered inputs and

outputs but not outcomes. Metrics were considered “countables” that have

only limited value as measures of an R&D organization’s results but which

can provide management with useful information on the organization’s

functional health.

By benchmarking against “world class” research organizations with

functions similar to those of ARL, such as the National Institute of

Standards and Technology in the public sector and Bell Labs in the private

sector, ARL established a set of about 60 metrics. ARL’s metrics

included—among many others—the numbers of technical objectives

accomplished, refereed journal articles written, technical reports

produced, patents received, PhDs on the staff, and others for monitoring

personnel and fiscal resources. All of these metrics were tracked by some

functional office of the laboratory, but of these metrics, only about 15 were

chosen by the director for special emphasis and are used to set

expectations for the directorate heads’ performance. Through his choice of

metrics, the director could focus the efforts of the senior executives in

accordance with the year’s organizational goals and seek continuous

improvement of the laboratory’s research environment. Further, the

 

2

ARL had 11 technical directorates during the time the pilot project was in effect. In fiscal year 1997,

ARL reorganized its business lines into 5 directorates of from 200 to 400 employees each, and 2 centers

of about 100 each.

3

In addition, the Director required that if any score on any survey were lower than 3.0, or if any survey

contained a negative comment from a customer, the cognizant senior executive would personally

contact the customer within 5 working days to resolve the problem.
Appendix II

Case Study: The Army Research Laboratory and the Performance Evaluation Construct

Page 39 GAO/GGD-98-162 Performance Management

director said, by choosing to emphasize certain metrics, he could continue

to influence the evolution of ARL’s culture from “full-spectrum” R&D to a

focus on fundamental and applied research—an ongoing effort that began

with ARL’s creation in 1992. For example, he said, his emphasis on the

numbers of papers written and the percentage of PhDs on the staff

reinforced ARL’s goal of being a world-class research organization.

According to the pilot project manager, the goals assigned to the senior

executives for the various metrics varied by directorate. For example, the

head of a directorate oriented toward basic research would be assigned a

relatively higher goal for the number of refereed journal articles produced

by the staff, whereas the head of a directorate oriented toward applied

research would be expected to produce a higher number of technical

reports for customers.

ARL officials also said that senior executives’ performance plans were to

include their top five or six project-specific goals for the year. These

project-specific goals were to correspond with goals appearing in the

laboratory’s annual performance plan.

At the end of the ratings year, ARL officials said, the director prepared the

ratings of all the senior executives, using the Army’s departmentwide

TAPES evaluation forms, which use a five-level descending rating scale. In

turn, these ratings were to be reviewed by an independent board whose

members were selected by the director for 1-year terms. Besides reviewing

the ratings, the board was to make performance bonus recommendations.

Senior executives’ bonuses for exceptional performance ranged from 5

percent to 20 percent of salary. In a given year, according to the deputy

chief of staff, no more than 45 percent of the senior executives could

receive a bonus.4

In theory, the director said, the laboratory’s annual performance goals,

which were reflected in the performance plans of the senior executives,

cascade to each employee level. However, the director said, this did not

happen consistently throughout ARL. The pilot project manager said that

no formal process had been established to ensure that it did. Speaking

more broadly, the pilot project manager said that strategic human resource

planning had not yet been integrated into the laboratory’s overall strategic

plan, and that deployment down to the workforce and first-level

 4

At the time of our review, some GS-15 managers were acting directorate heads. These managers were

rated by ARL’s director, but their ratings were not reviewed by the board. Further, these managers

were not eligible for senior executive bonuses but were eligible for special act awards of up to $7,500.
Appendix II

Case Study: The Army Research Laboratory and the Performance Evaluation Construct

Page 40 GAO/GGD-98-162 Performance Management

supervisors was inconsistent. Nonetheless, he said, that directorate heads

were expected to—and did—hold their project chiefs accountable for the

specific projects they supervised and that project chiefs were expected to

hold their lower level project employees accountable for the results

expected of them.

Based on directorate heads with whom we spoke, ARL’s planning and

evaluation efforts appear to have made a difference in accountability at the

senior executive level. For example, two of the directorate heads with

whom we spoke came to ARL from other government laboratories; both

said they felt more accountable under a system in which strategic planning

was used. One said that each component of his former laboratory “did its

own thing,” whereas at ARL, all components were working toward the

same organizational goals.

ARL planned to continue applying all three elements of the Performance

Evaluation Construct—peer review, customer feedback, and metrics. The

pilot project manager said that the scientific community recognizes the

importance of subjecting scientific research to the scrutiny of experts and

that ARL’s peer reviews, which have been done under contract by the

National Research Council, have led to improvements in the technical

quality of ARL’s work. Average customer ratings, ARL officials reported,

had risen since the inception of customer surveys in 1993 from 3.7 to 4.3.

The officials said that other metrics had also risen. For example, the

number of journal articles produced by the staff had increased by 155

percent; the percentage of PhDs on the staff had risen from 22 to 27; and

the number of guest researchers into and out of ARL had increased by 48

percent and 293 percent, respectively. Laboratory officials said while

metrics such as these had risen, all of ARL’s metrics are subject to

continuing reevaluation of their value, collectability, amenity to validation,

and the message they send to the workforce. The pilot project manager

pointed out that “what gets measured gets done” and said that “we need to

assure ourselves that we are measuring the right things so that we will do

the right things.”
Appendix III

Case Study: The Federal Energy Technology

Center and Its Team-Based Performance

Management Approach

Page 41 GAO/GGD-98-162 Performance Management

Formed in 1997 by the organizational merger of two Department of Energy

technology research centers, the Federal Energy Technology Center

(FETC) performs research, development, and demonstration projects on

advanced technology for fossil fuel energy and environmental protection.

Most of its funding is spent on cost-shared projects with the private sector.

FETC has research facilities at two locations, Morgantown, WV, and

Pittsburgh, PA. As of September 1997, FETC employed about 550 federal

workers, most of whom were engineers and scientists, and the services of

about 400 site support contractor employees.

FETC’s new approach to its work began at the Morgantown site before the

merger.1

 In the early 1990s, the director of the Morgantown center

concluded that, because the facility’s workload in research and

demonstration projects was increasing even as the size of its staff was not,

its mission accomplishment could be enhanced through the introduction of

TQM principles. As part of the TQM approach, the technical staff was to

work in teams, rather than as individuals working on their own projects,

and with managers working more as the coaches of teams than as

supervisors in the traditional sense. To help achieve this transition, the

Morgantown center introduced a team-based performance management

approach that it hoped would enhance teamwork, improve productivity

and quality, and help the center accomplish its performance goals.

The team-based performance management approach included (1) a 360-

degree feedback system, under which employees’ performance—that of

managers and supervisors alike—would be assigned scores by fellow

employees and customers, and through which distinctions could be drawn

among performers for purposes of performance improvement and

recognition; (2) formal performance appraisals using a two-level rating

scale (i.e., meets or exceeds expectations/fails to meet expectations) that

was adopted to help eliminate barriers to communications between

supervisors and employees; and (3) a team awards approach to support

better organizational performance.

Both 360-degree feedback and formal performance appraisals were means

of aligning employee performance management with organizational

 

1

The Morgantown center used these performance management approaches during the years (1994 to

1996) it was a pilot project under the Government Performance and Results Act of 1993 and through

1997, when FETC was formed. The approaches were being incorporated into FETC with certain

modifications, and we refer to them as FETC initiatives. However, most of the information we present

about the initiatives came from the Morgantown experience.
Appendix III

Case Study: The Federal Energy Technology Center and Its Team-Based Performance

Management Approach

Page 42 GAO/GGD-98-162 Performance Management

goals.2 According to FETC guidelines for its performance management

processes, the performance elements and standards in each employee’s

formal performance appraisal were to support the organization’s strategic

plan. The ratings factors used in 360-degree feedback largely paralleled

these performance elements and standards; this allowed the 360-degree

feedback results to be used in developing formal performance ratings and

for drawing distinctions among performers.

Under the 360-degree feedback system, each employee was to be rated by

the six to eight members of a group consisting of his or her supervisor,

subordinates (where applicable), peers, and internal customers. The

employee and his or her supervisor would agree to the membership of this

group at the beginning of the year. Each member of the group was to

assign scores to the employee’s performance on a 360-degree survey form,

using an ascending scale of 0 to 10. In addition, the scorers could provide

comments on the employee’s performance. The 360-degree survey listed a

series of factors and subfactors on which to score the employee; these

corresponded directly with the performance elements appearing in the

employee’s formal performance appraisal. The scores were to be averaged

and the scorers’ comments were to be consolidated using software

customized for that purpose. The supervisor was to use this information in

developing the employee’s formal rating, discussing with the employee

areas for performance improvement and recommending individual

performance awards. In addition, when giving the formal rating to the

employee, the supervisor was to provide the results of the 360-degree

feedback. Each employee was to receive a written report that included

both the aggregate results and the individual scorers’ inputs, but without

the scorers’ identities.

When FETC introduced the new system, the scoring factors on employees’

360-degree survey forms and the performance elements in their formal

performance appraisals emphasized the competencies and behaviors

considered necessary to support team efforts to meet FETC’s strategic

goals, rather than measurable employee performance objectives. For

example, in 1994, the strategic plan for the Morgantown facility’s Technical

Energy/Environmental and Management Services (TEAMS) business

sector included the goal to “Develop a fully acknowledged, internally and

externally supported TEAMS business... structured to serve a variety of

customers.” TEAMS employees’ 360-degree survey forms included the

 

2

The approach applied to all employees except its five senior executives, who were under the

governmentwide Senior Executive Service (SES) performance management system. However, the five

senior executives were to participate as raters and/or reviewers under FETC’s performance

management approach.
Appendix III

Case Study: The Federal Energy Technology Center and Its Team-Based Performance

Management Approach

Page 43 GAO/GGD-98-162 Performance Management

scoring factor, “Customer Service and Product Delivery,” that aligned with

this strategic plan goal. Further, members of the employees’ feedback

groups were to consider such subfactors as these: “Support to Strategic

Objectives—Understands his/her job in the context of METC’s Strategic

Plan, and performs the job consistent with... those plans” and “Customer

Focus—Identifies customers, understands their needs and focuses own

and team’s work on delighting the customer through on-time delivery of

quality services and products.”

Under the two-level rating approach, employees were to receive a formal

rating of either “meets or exceeds expectations” or “fails to meet

expectations.” According to FETC officials, the two-level ratings scale

unlinked individual performance awards from formal performance

standards and ratings, while still meeting the requirements of Office of

Personnel Management (OPM) performance management regulations.

Rather than using formal ratings either to draw distinctions among

employees who met or exceeded expectations or to determine individual

performance awards, FETC used the scores that employees received on

their 360-degree feedback surveys. The annual performance rating of

“meets or exceeds expectations” was the determining factor in retaining or

separating an employee.

According to an FETC human resources official involved in the process,

FETC used the two-level rating approach because it would counter the

“grade [ratings] creep” that, when FETC had used a five-level scale, had led

to over 90 percent of employees being rated in the top two categories. It

would also help FETC avoid the labeling of employees (e.g., “successful”

or “highly successful”) that had occurred using both the five-level scale

and the three-level scale that FETC had adopted as an alternative. The

official said that FETC expected the two-level ratings to help eliminate

barriers to communications during performance reviews. Supervisors, it

was felt, could spend less time defending the ratings they had assigned,

while employees would find it easier to accept and make use of feedback

that was provided by their peers and customers through the 360-degree

process. The official added her view that for a two-level formal rating scale

to work, a separate approach, such as the 360-degree feedback system,

must exist for determining distinctions among performers who meet or

exceed expectations and for creating the kind of feedback that employees

are likely to use to improve their performance.

In addition to the performance awards available to individual employees,

FETC developed a team awards approach, under which teams could

nominate themselves for awards. A Group Awards Committee (GAC) was
Appendix III

Case Study: The Federal Energy Technology Center and Its Team-Based Performance

Management Approach

Page 44 GAO/GGD-98-162 Performance Management

created to determine criteria for group awards, review nominations, and

recommend to the director which teams should receive them. The GAC

would also recommend the form (cash or noncash) of awards and, in the

case of cash awards, the amount. In 1995, the highest cash awards for

members of teams in the highest award category was $900.

According to FETC officials, the intent of the group awards approach was

to recognize teams and groups who had worked together to achieve

accomplishments consistent with FETC’s strategic plan. The group awards

were another means of reinforcing the team approach to achieving FETC’s

mission. The reason for having teams nominate themselves was to

encourage teams or groups to review their accomplishments at least

yearly, and to apply the awards criteria to their own performance. The

criteria, such as “value of the result” to the center and “customer

satisfaction,” were designed to be supportive of organizational goals.

According to FETC’s chief engineer, the agency has not evaluated its teambased performance management approach. He said that because the

Morgantown facility had not previously measured team performance, the

basis did not exist for comparing current and past productivity. However,

it was his view that 360-degree feedback and team building (including the

use of team awards) led to better performance by improving workplace

communications and idea sharing and by creating a greater awareness

among employees of their personal strengths and weaknesses.

According to officials in FETC’s human resources division, the 360-degree

feedback approach needed refinement, especially to make the scoring of

employees by their peers more consistent and equitable. The division

director said that when the Morgantown and Pittsburgh facilities were

merged organizationally into FETC, the staff at the two sites rated

employees very differently. He attributed the variation to the different

amounts of experience Morgantown and Pittsburgh employees had gained

in using the 360-degree approach. He believed that, to make the system

more consistent in its application and to make the scoring less subjective,

FETC needed to better define the rating factors used in the 360-degree

survey form and, where feasible, include measurable performance goals.

The inclusion of measurable performance goals, another human resources

official said, was something FETC expected to begin pilot testing

beginning in fiscal year 1998.

The same human resources official told us that the inclusion of measurable

performance goals in employees’ 360-degree survey forms and in their

formal performance appraisals was intended to emphasize employees’
Appendix III

Case Study: The Federal Energy Technology Center and Its Team-Based Performance

Management Approach

Page 45 GAO/GGD-98-162 Performance Management

contributions to their teams as well as to measure their individual job

performance. The official said that including measurable goals would

allow for performance expectations that were more explicitly aligned with

the agency’s annual performance plans and that would provide for greater

individual accountability. In addition, measurable performance goals, as

opposed to such scoring factors as organizational values and interpersonal

skills, would allow FETC to better track individual and team performance

against organizational goals and to base individual awards specifically on

job performance.
Appendix IV

Case Study: The National Oceanic and

Atmospheric Administration: Securing SES

Commitment to Strategic Planning

Page 46 GAO/GGD-98-162 Performance Management

The National Oceanic and Atmospheric Administration (NOAA) is a

Department of Commerce agency whose mission is to describe and predict

changes in the earth’s environment as well as to conserve and manage the

nation’s coastal and marine resources to ensure sustainable economic

opportunities. NOAA has five major components (the National Weather

Service; the National Environmental Satellite, Data, and Information

Service; the National Marine Fisheries Service; the National Ocean Service;

and the Office of Oceanic and Atmospheric Research) and an extensive

nationwide field structure. NOAA is headed by an administrator who also

holds the title of Undersecretary of Commerce for Oceans and

Atmosphere. At the time of our review, NOAA had approximately 12,200

employees, of whom about 120 were Senior Executive Service (SES)

members.

In 1993, according to the Counselor to the Administrator, NOAA’s new

leadership concluded that NOAA lacked a systematic means of

establishing its goals and evaluating its budgetary needs and therefore

introduced agencywide strategic planning. She said that agencywide

strategic planning was intended to help create an agencywide framework

for decisionmaking, emphasize long-term rather than short-term planning,

and allow agency policy to drive the agency’s budget rather than vice

versa. She said, further, that agencywide strategic planning was a

significant change for an agency that had traditionally been managed via

“stovepipes” to its component organizations and whose senior managers

were not accustomed to considering their own plans in the context of the

agency’s overall mission and goals. She said that, for agencywide strategic

planning to work, NOAA’s leadership felt it was crucial that these senior

managers buy into the process. One way NOAA tried to gain their

participation was by involving them in the planning; another way was to

make support for agencywide strategic planning a critical element of the

senior executives’ performance plans.

At the time of our review, NOAA’s planning cycle included (1) a 10-year

strategic plan updated periodically; (2) a 5-year implementation plan that

identified the strategies NOAA would follow to accomplish its strategic

goals; (3) 1-year operating plans for the agency and its component

organizations, defining the year’s specific performance goals; and (4) an

evaluation of the agency’s and each of its components’ progress toward

achieving its annual performance goals. The 5-year implementation plan

was to be updated annually and drive the agencywide budget submission

for the following year.
Appendix IV

Case Study: The National Oceanic and Atmospheric Administration: Securing SES

Commitment to Strategic Planning

Page 47 GAO/GGD-98-162 Performance Management

In 1994, NOAA identified a set of seven agencywide strategic goals. For

each goal, NOAA established a cross-functional strategic planning team

whose members were drawn from all the agency’s line offices. The

purpose of the teams was to develop integrated strategies for

accomplishing the agency’s goals. According to the director of human

resources, about 90 percent of the agency’s senior executives were

involved in these teams, either directly or by providing staff from their

organizations. NOAA’s first strategic plan, developed in 1994, was the work

of the strategic planning teams.

NOAA’s seven agencywide goals were not structured along organizational

lines; nor did all of them apply to all of NOAA’s components. But each

NOAA component was to produce an annual operating plan describing its

strategy for accomplishing the agencywide goals for which it did have

responsibility. NOAA made overall “strategic plan support” a critical

element in its senior executives’ performance plans, generally assigning it

a weight of 30 percent in their performance ratings. In addition, NOAA

created an explicit alignment between senior executives’ performance and

their organizational goals by making them directly accountable for the

goals included in their organizations’ annual operating plans. The

Counselor to the Administrator said that, as NOAA evaluated the progress

made on organizational and agencywide goals as part of its annual

planning cycle, this alignment allowed the agency simultaneously to

monitor program performance and individual managers’ performance.

At the end of the ratings year, senior executives were to prepare selfassessments based on their accomplishment of their annual goals. In

addition, the senior executives’ superiors would assign them ratings on a

five-level scale.1

 The self-assessments and the ratings would then be

forwarded to NOAA’s Performance Review Board for review and

disposition. This process did not change with the advent of agencywide

strategic planning. However, the formula for allocating SES performance

bonus funds eventually did change to some extent. SES performance

bonus funds had been calculated at approximately 3 percent of total SES

payroll; the agency decided in 1997 to withhold 10 percent of this 3 percent

for disbursement by the administrator to senior executives who had

contributed the most to NOAA’s strategic goals. In fiscal year 1996,

41 NOAA senior executives received performance bonuses ranging from

5 percent of salary to $11,217, and 11 received executive salary increases.

 

1

Other NOAA employees were to be rated using two-level approaches.
Appendix IV

Case Study: The National Oceanic and Atmospheric Administration: Securing SES

Commitment to Strategic Planning

Page 48 GAO/GGD-98-162 Performance Management

NOAA officials told us that another way in which senior executives were

held accountable for performance was through quarterly reviews. Agency

officials told us these reviews were full-day events at which the heads of

the five NOAA component organizations and other NOAA managers

briefed the administrator and other top agency leaders on the progress

made under their yearly operating plans. NOAA officials said that if goals

or milestones were not being met, the responsible managers were

expected to explain why this occurred and how they intended to meet

them.

At the time of our review, NOAA officials said, there was a clear alignment

between SES, organizational, and agency goals. No requirement had been

imposed to ensure that this alignment existed for staff below the SES level.

According to the Counselor to the Administrator, NOAA’s leadership

wanted to see how well the practice was working at the SES level before

extending it to others. She said that NOAA planned eventually to include

strategic planning goals in the performance plans of GS-15 and GS-14

office directors. She added that it would be a challenge to instill an

awareness at all levels of the organization of how employees’ individual

efforts contribute to the agency’s strategic goals.

One NOAA manager we spoke with echoed this concern. He said that

although he felt it was implicitly understood that senior executives should

cascade strategic planning goals down to the performance plans of lower

level staff, he said it was not clear to managers how to translate the

agency’s strategic goals into individual employees’ performance

expectations.

NOAA officials said that in an effort to make employees more aware of the

relationship between their jobs and the agency’s goals, a segment on

agencywide strategic planning was included in the supervisory training

given new supervisors. Further, the officials said that when senior

managers visit NOAA’s field offices, they hold forums with the staff at

which they discuss the agency’s strategic goals. In addition, as part of their

career development, some NOAA staff members have been detailed to the

strategic planning office for periods of 1 year to 18 months to become

familiar with the strategic planning process.

The SES officials we spoke with at NOAA said that their involvement in

agencywide strategic planning helped them understand how their

individual goals and measures fit into NOAA’s institutional goals and the

goals of other NOAA component organizations. One senior executive said

that the quarterly review process increased accountability; now, he said,
Appendix IV

Case Study: The National Oceanic and Atmospheric Administration: Securing SES

Commitment to Strategic Planning

Page 49 GAO/GGD-98-162 Performance Management

goal setting must be well thought out, and priority-setting had gained new

importance. The Counselor to the Administrator said that NOAA’s

component organizations now have common goals, resulting in a more

cogent understanding of the agency’s mission and integrated strategies for

accomplishing it. Moreover, she said that strategic planning had helped

NOAA articulate and justify why it should stay intact as an agency and that

it had brought more rigor to planning and budgeting. She cautioned,

however, that NOAA was still learning the strategic planning process; that

its goals and measures needed to be periodically reevaluated, particularly

to make them more outcome-oriented; and that achieving the change in

organizational culture required for successful strategic planning remained

a challenge. NOAA’s director of human resources agreed, saying that “the

stovepipes are breaking down,” but that there was still a way to go.
Appendix V

Case Study: The Small Business

Administration and Goal Setting for District

Directors

Page 50 GAO/GGD-98-162 Performance Management

The Small Business Administration (SBA) administers loans and other

programs that annually provide approximately $10 billion in small business

financing. In fiscal year 1997, SBA had about 2,900 regularly funded fulltime employees. SBA is organized into 15 headquarters program offices,

each headed by a program manager; 10 regional offices, each headed by a

Schedule-C regional administrator; and 69 district offices, each headed by

a district director. The majority of district directors are GM-15s; a small

portion are SES members.

In a March 1994 memo to SBA managers and district directors, SBA’s

administrator announced that he had signed a performance agreement

with the President identifying four policy goals that would become the

basis for agencywide performance plans (which he referred to

alternatively as business plans). At the same time, the administrator

announced that SBA had become a Results Act pilot project and that his

intent was to make SBA a results-oriented agency, one run in a manner

similar to that of successful businesses.

The administrator sought to emphasize SBA’s client services, especially

those involving capital lending to what were identified as underserved

segments of the small business community, such as minorities, women,

and veterans. The administrator stated that SBA’s district offices were “the

primary vehicle for the Agency to achieve its mission.” Accordingly, he

made the district offices the focus of agency performance, signing

performance agreements with the district directors through which each of

them would be held accountable for accomplishing a specific share of the

agency’s capital lending goals.1

 In addition, the Administrator established

(1) a system for monitoring the district offices’ performance, (2) peer

review panels so that recommendations on district managers’ ratings and

performance bonuses would be made by their peers, and (3) a

performance bonus pool for the district directors so that performance

could be linked to compensation.

When goal setting for the district offices began in 1994, district office

lending goals were negotiated by SBA’s administrator, deputy

administrator, and the district directors. SBA’s agencywide lending goals

for fiscal years 1994 and 1995 were derived by totaling the lending goals

negotiated between the administrator and the district directors.

 1

An SBA official told us that the administrator signed performance agreements with 12 headquarters

managers as well but that district directors were the only SBA officials for whom programmatic goals

tied to the agency’s business plan were systematically included in performance appraisals.
Appendix V

Case Study: The Small Business Administration and Goal Setting for District Directors

Page 51 GAO/GGD-98-162 Performance Management

Beginning in fiscal year 1996, under SBA’s next administrator, goal setting

became a more top-down process. Yearly goals for the district offices were

to be developed by SBA’s Office of Field Operations, using a model based

on market size, historical lending data, expected market penetration rates,

and other factors. The district directors were to be given these goals on

yearly goal sheets distributed by Office of Field Operations. The goal

sheets reflected another change that took place in fiscal year 1996: an

expansion in the kinds of goals for which targets were assigned. Goals now

fell into three main categories: program goals, management goals, and

internal control. Program goals continued to focus on capital lending

(particularly to underserved segments of the small business community)

and other financing activities, such as portfolio management and surety

bond guarantees. Management goals focused on SBA’s public outreach and

small business technical assistance programs, some of which involved

partnering with other service organizations. An SBA official explained that

the management goals were less directly targeted at measurable

accomplishments than the program goals, and were established to provide

a kind of “health index” of the district offices. Internal control covered

functions such as (1) annually reviewing small businesses participating in

SBA’s Minority Enterprise Program to ensure their compliance with

applicable laws, regulations, and SBA operating procedures or (2)

monitoring the Small Business Development Program to maintain financial

integrity by ensuring that objectives, costs, and program implementation

complied with applicable laws and regulations.

To track the performance of the district offices, SBA developed an on-line

District Office Goal Monitoring System. Using this system, district

directors and their employees could check the current status of their

offices’ capital lending performance against the year’s goals. They could

also check other district offices’ performance. The information was to be

updated each weekend and available to SBA employees each Monday

morning. According to the supervisory field operations analyst who

maintains the Goal Monitoring System, it has not only provided

performance data for SBA management but has proven to be a motivator

for district office staffs. Two of the three district directors we spoke with

concurred, one saying that the performance figures posted on the wall

each Monday morning were his staff’s “biggest motivator.”

Besides the inclusion of quantitative goals in the district directors’

performance expectations, another change, affecting non-SES district

directors only, was the way in which their ratings and bonus

determinations were made. During fiscal year 1995, the administrator

established a peer review board consisting of five non-SES district
Appendix V

Case Study: The Small Business Administration and Goal Setting for District Directors

Page 52 GAO/GGD-98-162 Performance Management

directors and two non-voting members (the Assistant Administrators for

Human Resources and Equal Opportunity and Civil Rights Compliance).2

The board was to review the non-SES district directors’ initial ratings as

prepared by the regional administrators, along with statements submitted

by the non-SES district directors comparing their accomplishments against

their performance expectations. The board would then recommend to the

Associate Administrator for Field Operations the district directors’ final

ratings and performance bonuses.

In fiscal year 1996, under the new SBA Administrator, the peer review

board system was discontinued. Since then, the regional administrators

have submitted district directors’ ratings and performance bonus

recommendations directly to the Associate Administrator for Field

Operations, as they did prior to fiscal year 1995.

In 1994, along with the new emphasis placed on district directors’

performance, the Administrator created a new performance bonus pool for

non-SES district directors. Similar to the pool already in place for SES

members, the bonus pool for non-SES district directors was limited to 2

percent of the non-SES district directors’ payroll as of the end of the fiscal

year. No more than 60 percent of the non-SES district directors could

receive bonuses in the same year. SBA officials told us that the size of

district directors’ bonuses was dependent on their annual performance

ratings, which used an ascending five-level rating scale. A district director

with a summary rating of 5 (outstanding) could receive a performance

bonus of as much as $5,000; one with a summary rating of 4 (exceeds fully

successful) could receive as much as $3,500. SBA’s Assistant Administrator

for Human Resources said that district directors who received a summary

rating of 3 (fully successful) could get a bonus, but that this did not happen

often because not enough money was available to do so. The bonus pool

for non-SES district directors was still being used at SBA during fiscal year

1997.

SBA officials said that district directors are rated and rewarded on how

well their district offices have performed against the year’s goals.3

 The

officials reported that in fiscal year 1995, the first year in which district

directors had capital lending goals for underserved segments of the small

 

2

An alternate board reviewed the performance appraisals of the five district directors who were on the

primary board.

3

In addition, other management-related issues were considered in performance appraisals, such as

budget, personnel/human resources, equal employment opportunity, and information resources

management.
Appendix V

Case Study: The Small Business Administration and Goal Setting for District Directors

Page 53 GAO/GGD-98-162 Performance Management

business community included in their performance plans, lending to all

such categories increased over the previous year. For example, loans to

minority-owned businesses increased by 53.2 percent, to women-owned

businesses by 85.6 percent, and to veteran-owned businesses by 42.9

percent.

The three district directors we spoke with said that having specific goals in

their performance plans made them feel accountable for their

performance. One district director said that with specific goals in mind, the

staff now works harder to generate demand for SBA services and provide

technical assistance to customers.

According to the Assistant Administrator for Human Resources, SBA

requires district directors to cascade the goals they are assigned down to

their staffs’ individual performance appraisals whenever possible. The

assistant administrator said that a recent check by the Office of Human

Resources showed that the practice of cascading district office goals down

to the staff’s performance appraisals was not widespread. The three

district directors we spoke with varied in their approaches to doing so. For

example, one district director structured his office into teams and divided

the district office goals among the teams; another cascaded the district

office goals down to managers only—that is, employees at the GS-13 level

and above. All three district directors we spoke with made use of spot cash

awards to the staff. Because of SBA’s new focus on performance by the

district offices, the formula for distributing funds for spot cash awards

among SBA offices was changed to make more money available in the

field.

SBA officials said they expected the practice of aligning the district

directors’ goals with those of the agency to continue and that the practice

would eventually be expanded to include other senior managers as well. A

new administrator—SBA’s third since 1994—joined the agency in February

1997. In the administrator’s comments on GAO’s review of SBA’s March

1997 draft strategic plan, she stated that she intended “to begin a process

where the managers have a significant hand in refining the goals,

strategies, and performance measures. SBA will link performance

appraisals with performance against the goals in the annual business

plan.”4

SBA officials said the new administrator brought with her a new vision for

the agency, part of which was to transform SBA from direct service

 

4

SBA’s Draft Strategic Plan (GAO/RCED-97-205R, July 11, 1997).
Appendix V

Case Study: The Small Business Administration and Goal Setting for District Directors

Page 54 GAO/GGD-98-162 Performance Management

provider to lender/overseer. The officials said this change would require

that the agency’s goals be refined and that it would be a challenge to

translate the new administrator’s vision for SBA into measurable outcomes

that could be included in SBA managers’ performance plans. However,

SBA’s chief of staff said that the new administrator was committed to

making SBA managers accountable for results, and that output measures

served a useful purpose in that regard.
Appendix VI

Case Study: VA’s New York Regional Office

and Its “Scorecard” Driven, Team-Based

Approach to Performance Management

Page 55 GAO/GGD-98-162 Performance Management

Located in New York City, the New York Regional Office (NYRO) of the

Department of Veterans Affairs (VA) serves veterans and their families in

eastern New York, New York City, and Long Island, by processing their

claims for disability payments and other veterans’ benefits. NYRO receives

about 500,000 telephone calls and 35,000 visits from veterans and their

families each year. NYRO has about 270 employees, of whom about 180 are

in the Veterans Benefits and Services Division—the focus of the Results

Act pilot that ran from 1993 to 1996.

During the early 1990s, VA encouraged its regional offices to do more to

overcome slow claims processing and poor service to veterans and their

families. At that time, NYRO was processing claims in assembly-line

fashion, with each claim undergoing 20 to 30 specific steps involving 12 or

more different clerks, adjudicators, and ratings specialists. NYRO decided

to replace this claims process with one that would incorporate TQM

principles, and that would allow employees to take on wider

responsibilities in claims processing and to work in self-directed work

teams.1

 Under the new process, a group as small as three employees—one

case technician and two case managers—would process an individual

benefit claim from start to finish.

The Veterans Benefits and Services Division was restructured into four

core groups, each comprising four work teams. Through reclassification,

at least 10 previously existing positions—clerical, professional, and

supervisory—were combined to create three new positions: case

technicians (GS-3 through GS-5), case managers (GS-5 through GS-12), and

coaches (GS-11 through 13). The new NYRO work teams generally

consisted of four case technicians, seven case managers, and a coach.

Case technicians performed duties that previously had been assigned to

four different clerks, while case managers adjudicated claims and

interacted with claimants by phone or in person, thereby performing duties

that previously had been assigned to four other positions. Coaches, in

keeping with TQM practice, focused more on coaching the team than on

supervision, but performed some conventional supervisory duties as well,

such as preparing formal performance evaluations.

With the adoption of its new team structure and work processes, NYRO

retired its previous performance measures—timeliness, quality, and

 

1

For discussions of NYRO’s self-directed work teams, see Veterans’ Benefits: Status of Claims

Processing Initiative in VA’s New York Regional Office (GAO/HEHS-94-183BR, June 17, 1994) and

Management Reform: Status of Agency Reinvention Lab Efforts (GAO/GGD-96-69, Mar. 20, 1996). VA’s

claims processing is also discussed in Veterans’ Benefits: Better Assessments Needed to Guide Claims

Processing Improvements (GAO/HEHS-95-25, Jan. 13, 1995).
Appendix VI

Case Study: VA’s New York Regional Office and Its “Scorecard” Driven, Team-Based

Approach to Performance Management

Page 56 GAO/GGD-98-162 Performance Management

productivity—as obsolete. After some study of private sector performance

measures, NYRO implemented the “balanced scorecard,” a system of

performance measurement intended to provide a balanced picture of the

results it was achieving and to balance the needs and expectations of its

stakeholders. NYRO officials said the five balanced scorecard measures—

speed, accuracy, cost, customer satisfaction, and employee development—

have been incorporated into NYRO’s performance management at the

division, core group, and team levels, and are used to focus the staff on

organizational goals.

According to NYRO, the balanced scorecard measures, while not included

in individual employees’ performance expectations, nonetheless drive

NYRO’s employee performance management approach. Although strategic

goals and performance measures are established at the organizational

level, officials said, it is at the team level that organizational performance

measures are brought into each employee’s “line of sight.” Using balanced

scorecard measures, teams are assigned annual performance targets that

are aligned with NYRO’s strategic goals. Each team’s progress toward

these targets is closely tracked. Through peer assessment, each employee

is given feedback on the employee’s contribution to the team’s

performance. At the end of the year, the team’s performance determines

the amount of money that is made available to reward team members.

According to NYRO, each of the five balanced scorecard measures—speed,

accuracy, cost, customer satisfaction, and employee development—can

include submeasures that are established each year. In fiscal year 1996, for

example, “speed” included such submeasures as the average number of

days taken to complete a claim, the average number of days unprocessed

claims were pending, and the number of pending claims that were more

than 6 months old. For a given measure, a team would receive an overall

score based on its performance on all the submeasures. According to

NYRO, the submeasures are aligned with measurable objectives and

broader strategic goals identified in NYRO’s annual Quality Plan. For

example, in fiscal year 1996, the submeasure involving the number of

pending claims that were more than 6 months old was used to support the

organizational objective to “reduce C&P [compensation and pension] cases

[pending] over six months to 1000,” which was itself a supporting objective

under the strategic goal to “improve customer service.”

Each year, according to the NYRO assistant director, NYRO managers

work with the teams to set expectations regarding the five balanced

scorecard measures. A team’s total possible score on the balanced

scorecard is 100; its actual score is calculated by aggregating, on a
Appendix VI

Case Study: VA’s New York Regional Office and Its “Scorecard” Driven, Team-Based

Approach to Performance Management

Page 57 GAO/GGD-98-162 Performance Management

weighted basis, its scores for speed, cost, customer satisfaction, and

employee development. Customer satisfaction carries a possible 28 points;

speed, cost, and employee development each carries a possible 24 points.

Accuracy is considered a threshold issue; a team cannot earn points on the

other balanced scorecard measures unless it conducts accuracy reviews of

its claims processing.

According to NYRO, the progress that teams make toward their annual

performance targets is monitored monthly using balanced scorecard

assessments. The teams’ scores are aggregated to produce semiannual

scores for the core groups and the division. NYRO officials said that the

balanced scorecard is the primary means of helping to ensure that all

employees, from team members through agency managers, are meeting

performance targets critical to NYRO’s strategic goals and objectives. At

the team level, the officials said, employees were to meet weekly to

discuss the team’s progress; they were to review its balanced scorecard

performance each month. At these meetings, employees could use data

gathered in a variety of ways to help assess team performance. For

example, each team would receive information on customer satisfaction

from a monthly, contractor-conducted telephone survey of customers and

would receive data on the speed of its claims processing from NYRO’s

computerized information system.

According to the human resources manager, each team member’s

contribution to the team’s performance is assessed yearly by the other

team members. Since 1994, NYRO has tested two approaches to providing

feedback via peer assessment. One approach was tested in 1994, 1995, and

1997. Under this approach, each team was to hold a year-end meeting at

which the members shared observations—both positive and negative—

about each others’ contributions.

Another, more structured, approach was tested in 1996 and is in use again

in 1998. Under this approach, team members are to score each other on

five criteria, using an ascending scale of 1 to 5, and provide written

comments to support their scores. The five peer assessment criteria are

not identical to the five balanced scorecard measures used at the division,

core group, and team levels. NYRO said the difference reflected a move

away from having individual performance standards that focus on

individual productivity and toward having individual performance

standards that focus on the employee’s contribution to the team in meeting

its organizational outcome measures. Accordingly, the five criteria used in

peer assessment at NYRO are output, customer service, reliability, team

support, and accuracy. A software application is used to consolidate the
Appendix VI

Case Study: VA’s New York Regional Office and Its “Scorecard” Driven, Team-Based

Approach to Performance Management

Page 58 GAO/GGD-98-162 Performance Management

peer assessment scores and the written comments relating to each team

member, and then the coach is to discuss the results with team members

individually.

Officials at NYRO said that the main purpose of peer assessment is to

identify the areas in which employees can improve their individual

performance and thereby contribute more effectively to the performance

of the team. If an employee’s peers indicate, through their scoring and

comments, that the employee is not contributing sufficiently to the

accomplishment of the team’s performance goals, then the coach is to

assess the situation to determine if a problem exists. The coach and NYRO

management are then to decide what further action may be appropriate,

such as counseling or an individual performance improvement plan.

According to NYRO’s assistant director, employees have demonstrated

their willingness to use the peer assessment tool to provide credible

feedback to each other and to provide managers with a team perspective

on individual members’ contributions; during 1996, for example, 34 (18

percent) of the 186 team members who participated in peer assessment

were flagged by their teams as needing improvement.

According to the assistant director, peer assessments, at this time, do not

directly determine the formal performance ratings that coaches prepare

for the team members. Rather, coaches are to prepare formal performance

ratings based on the information gathered, plus their own understanding of

each employee’s performance. Team coaches themselves are scored by

their team members, but the coaches’ formal performance ratings are

prepared by the division leadership, using observations on the coaches’

performance drawn not only from their team members but from other

NYRO associates of theirs who have knowledge of their performance.

During the period covered by the pilot project, employee performance

ratings at NYRO had five levels, ranging from “unacceptable” to

“outstanding.” Beginning in fiscal year 1998, NYRO adopted a two-level

approach (“unacceptable” and “successful”), in conformance with new VA

policy. With the exception of the NYRO director, who is a member of the

SES and therefore subject to a different system, all NYRO employees are to

be rated using the two-level system.

Performance was tied to compensation at NYRO through a team-based

approach that NYRO called “variable pay.” Traditionally, NYRO said, the

organization had rewarded employees for individual performance based on

standards identified for individual positions. In contrast, the new system

used awards at the core group level to reinforce the importance of working
Appendix VI

Case Study: VA’s New York Regional Office and Its “Scorecard” Driven, Team-Based

Approach to Performance Management

Page 59 GAO/GGD-98-162 Performance Management

together and achieving organizational goals. NYRO officials said that the

variable pay system included three methods of distributing annual cash

awards. First, management awarded each of the four core groups an

amount that was based primarily on the group’s overall scorecard

performance. The amount going to each core group was to be divided

among the members of the four teams in the core group. In fiscal year

1996, team members in the highest-rated core group each received $400.

Second, the four coaches in each core group were given $1,000 to allocate

among the core group employees whom they agreed had contributed the

most during the year, either to team performance or to various core group

efforts. Third, each team was given $1,000 to be allocated among the team

members, without input from the coach. NYRO officials said that some

teams divided the money equally among their members, while others

allocated the money among individual members on the basis of their

contributions to the team. Employees who were under a performance

improvement plan, or who had to address fundamental behavioral issues,

or against whom a disciplinary action had been taken during the year, were

not allowed to receive cash awards, except awards that were decided upon

by team members under the last approach.

According to the NYRO assistant director, performance on some of

NYRO’s organizational measures, such as the number of cases pending and

the scores received on customer satisfaction surveys, have improved in

recent years. She cautioned, however, that although these are encouraging

signs, the extent to which they can be directly attributed to NYRO’s

performance management efforts, such as the balanced scorecard, peer

assessment, and variable pay, has not been formally evaluated. She noted

that other factors have influenced productivity, such as hiring freezes (the

NYRO staff was reduced from about 380 to about 270), government

shutdowns, and the fact that many employees must be diverted from their

work in order to be trained in the skills they need to manage the new

claims process. The assistant director added that NYRO continues to refine

its performance management approaches. Peer assessment, for example,

may be adapted to more directly determine formal ratings, and may

eventually be used to remove poor performers. In addition, the assistant

director said, NYRO has been testing a variety of 360-degree feedback for

its upper- and mid-level managers and team coaches; thus far, it has been

used only as a development tool, but managers and coaches are expected

to act upon the areas for improvement identified through the process.
Appendix VI

Case Study: VA’s New York Regional Office and Its “Scorecard” Driven, Team-Based

Approach to Performance Management

Page 60 GAO/GGD-98-162 Performance Management

In November 1996, OPM announced a proposed “VA Pay for Applied Skills

Demonstration Project” covering NYRO and VA’s Detroit Regional Office.2

The proposal expands upon the human resource management (HRM)

practices NYRO was developing during the Results Act pilot project

phase—retaining, for example, the balanced scorecard, peer assessment,

and variable pay—while describing a variety of HRM flexibilities to

support mission accomplishment. Among other features, the proposal

includes a compensation system that would use “skill blocks” for defining

employees’ technical and interpersonal skills. The compensation system

would also include a skills-based approach to base pay for employees now

covered under the GS system in grades GS-3 through GS-13. Employees

would advance through three ranges of base pay corresponding with levels

of skill blocks—fundamental, proficient, and expert—as they acquired and

demonstrated the successful application of these skills. Variable pay would

continue to be based on core group and organizational performance but

could eventually include “goalsharing,” which would fund variable pay

according to a formula based on the savings produced and the goals

achieved by the organization.

 

2

 Federal Register, Vol. 61, No. 223, Nov. 19, 1996, pp. 58942-58956.
Appendix VII

Major Contributors to This Report

Page 61 GAO/GGD-98-162 Performance Management

Stephen E. Altman, Assistant Director,

Federal Management and Workforce Issues

Gary V. Lawson, Evaluator-in-Charge

Anthony Assia, Senior Evaluator

Allen C. Lomax, Senior Evaluator

Frank J. Minore, Senior Evaluator

Michael J. O’Donnell, Advisor

Gregory H. Wilmoth, Senior Social Science Analyst

Lessie M. Burke, Editor

Ernestine Burt, Issue Area Assistant

(410082)

General Government

Division
Page 62 GAO/GGD-98-162 Performance Management
Page 63 GAO/GGD-98-162 Performance Management
Page 64 GAO/GGD-98-162 Performance Management
Ordering Information

The first copy of each GAO report and testimony is free. Additional

copies are $2 each. Orders should be sent to the following address,

accompanied by a check or money order made out to the

Superintendent of Documents, when necessary. VISA and

MasterCard credit cards are accepted, also. Orders for 100 or more

copies to be mailed to a single address are discounted 25 percent.

Order by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 or by using fax

number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any list

from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to

obtain these lists.

For information on how to access GAO reports on the INTERNET,

send e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States

General Accounting Office

Washington, D.C. 20548

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100