Report to Congressional Requesters

United States General Accounting Office

GAO

July 2002 PERFORMANCE

MANAGEMENT

SYSTEMS

IRS’s Systems for

Frontline Employees

and Managers Align

with Strategic Goals

but Improvements

Can Be Made

GAO-02-804
Page i GAO-02-804 Performance Management Systems

Letter 1

Results in Brief 3

Background 5

Critical Job Responsibilities Align with Strategic Goals but Do Not

Always Align with Other Elements of the Frontline Employee

Performance Management System 8

Enforcement Group Managers’ Commitments Align with IRS’s

Strategic Goals but Are Not Always Written So that Raters Can

Hold Managers Accountable 12

Raters Generally Provided Feedback to Enforcement Group

Managers on Meeting Critical Job Responsibilities and

Commitments but Provided More Extensive Feedback on Some

Critical Job Responsibilities Than Others 17

IRS Has Made Progress but Could Further Enhance Its New

Managers’ and Frontline Employees’ Performance Management

Systems 21

Conclusions 25

Recommendations for Executive Action 26

Agency Comments and Our Evaluation 26

Appendix I Objectives, Scope, and Methodology 28

Objectives 28

Scope and Methodology 28

Appendix II GAO Sampling Methodology 31

Sample Designs 31

Sample Disposition 32

Sampling Error 33

Nonsampling Error 33

Appendix III Samples of Manager and Frontline Employee

Evaluation Forms 35

Appendix IV Examples of Evaluations Addressing Group

Managers’ Responsibilities 43

Contents
Page ii GAO-02-804 Performance Management Systems

Appendix V Distribution of Raters’ Comments for Each Supporting

Behavior 44

Appendix VI Examples of Group Managers’ Commitments 46

Commitments that Were Specific, Outcome or Output Oriented, or

Easily Monitored 46

Commitments that Were Not Specific, Outcome or Output

Oriented, or Easily Monitored 47

Appendix VII Comments from the Internal Revenue Service 49

Appendix VIII GAO Contacts and Staff Acknowledgments 51

GAO Contacts 51

Acknowledgments 51

Tables

Table 1: Examples of Enforcement Group Managers’ Commitments

that Align with IRS’s Strategic Goals 13

Table 2: Percentage of Commitments Reflecting Critical Job

Responsibilities for Fiscal Years 2000 and 2001 14

Table 3: Percentage of Commitments Not Specific,

Outcome/Output Oriented, or Easily Monitored for Fiscal

Years 2000 and 2001 15

Table 4: Estimated Percentage of Fiscal Years 2000 and 2001

Written Evaluations in Which Raters Assessed Critical Job

Responsibilities by Number of Responsibilities 18

Table 5: Estimated Percentage of Enforcement Group Managers

Reporting Verbal Feedback by Number of Critical Job

Responsibilities for Fiscal Year 2001 19

Table 6: Estimated Percentage of Fiscal Year 2000 and 2001

Evaluations Assessing Enforcement Group Managers’

Performance in Each Critical Job Responsibility 19

Table 7: Estimated Percentage of Enforcement Group Managers

Reporting Feedback to a Great or Very Great Extent for

the Critical Job Responsibilities for Fiscal Year 2001 20
Page iii GAO-02-804 Performance Management Systems

Table 8: Estimated Percentage of Enforcement Group Managers

Reporting that the Respective Critical Job Responsibilities

Are Very Important or Important to Their Raters for Fiscal

Year 2001 21

Table 9: Disposition of Sample Cases for Our Review of Group

Manager Evaluations 32

Table 10: Disposition of Sample Cases for Our Survey of Group

Managers’ Perceptions of Raters’ Feedback 33

Table 11: Examples of Critical Job Responsibilities, Supporting

Behaviors, and Evaluation Narratives for Group Managers 43

Table 12: Distribution of Raters’ Comments 44

Figures

Figure 1: Alignment of Strategic Goals, Critical Job

Responsibilities, and Supporting Behaviors for

Enforcement Group Managers 9

Figure 2: Misalignment of Critical Job Responsibility, Narrative

Description, and Supporting Behaviors for Revenue

Agents 11

Abbreviations

CQMS Collections Quality Management System

IDPs Individual Development Plans

IRM Internal Revenue Manual

IRS Internal Revenue Service

NTEU National Treasury Employee Union

RRA 98 Restructuring and Reform Act of 1998

TFRP Trust Fund Recovery Penalty
Page 1 GAO-02-804 Performance Management Systems

July 12, 2002

The Honorable Bill Thomas

Chairman, Committee on Ways and Means

House of Representatives

The Honorable Amo Houghton

Chairman, Subcommittee on Oversight

Committee on Ways and Means

House of Representatives

Performance management systems can be powerful tools in helping an

agency achieve its mission and ensuring employees at every level of the

organization are working toward common ends. Performance

management systems should help employees understand their

responsibilities and how their day-to-day work contributes toward meeting

their agency’s strategic goals as well as providing a mechanism for giving

employees candid, specific feedback on how well they are meeting their

rater’s expectations. For agencies like the Internal Revenue Service (IRS)

that are undergoing a cultural change, performance management systems

help reinforce behaviors and actions that support the agency’s mission.1

In February 2000, IRS implemented a new performance management

system for its executives and managers and in October 2001 implemented

a new performance management system for frontline employees. These

systems were built upon IRS’s three strategic organizational goals—”topquality service to each taxpayer in every interaction,” “top-quality service

to all taxpayers through the fair and uniform application of the law,” and

“productivity through a quality work environment”—and the

corresponding balanced performance measures of customer satisfaction,

business results (quality and quantity), and employee satisfaction that are

applied to all organizational units, from IRS-wide down to the group level.

IRS executives and managers are evaluated on five critical job

responsibilities (customer satisfaction, business results, employee

satisfaction, leadership, and equal employment opportunity) and on

 1

 The mission of IRS is to “provide American taxpayers top quality service by helping them

understand and meet their tax responsibilities, and by applying the tax laws with integrity

and fairness to all.”

United States General Accounting Office

Washington, DC 20548
Page 2 GAO-02-804 Performance Management Systems

written commitments they develop on the specific actions they will take to

support IRS’s strategic goals over the course of the evaluation period.2

Frontline employees are evaluated against five critical job responsibilities,

(customer satisfaction—knowledge, customer satisfaction—application,

business results—quality, and business results—efficiency, and employee

satisfaction—employee contribution).

Because of the challenging nature of implementing a new performance

management system, you asked us to assess how well IRS is aligning its

performance management systems with its strategic goals and is using

these systems to hold managers and employees accountable for meeting

those goals. Specifically, as agreed with your office, our objectives were to

determine (1) the extent to which group managers’ and frontline

employees’ critical job responsibilities and other elements of the employee

performance management systems are structurally aligned with IRS’s

strategic goals and organizational unit performance measures, (2) the

extent to which group managers’ commitments align with IRS’s strategic

goals and are written so that raters can hold managers accountable for

meeting their commitments, (3) the extent to which raters provide

feedback to enforcement group managers on their performance in meeting

critical job responsibilities and commitments, and (4) whether IRS has

adequate plans to monitor and assess the implementation of its

performance management systems.

To address these objectives, we reviewed documents related to IRS’s

performance management systems and discussed the systems with

officials responsible for developing and implementing them. We performed

qualitative analyses of the extent to which the critical job responsibilities

and supporting behaviors for enforcement group managers and frontline

employees3 align with IRS’s strategic goals and organizational unit

performance measures. We performed a content analysis of a

representative sample of the written evaluations for 126 enforcement

group managers for fiscal years 2000 and 2001. We also surveyed the

enforcement group managers about their perceptions of the verbal and

 2

 See Appendix IV for examples of evaluations addressing group managers’ critical job

responsibilities.

3

 Enforcement group managers are those managers in IRS’s field offices that supervise

frontline employees, such as revenue agents who directly deal with taxpayers when

auditing tax returns and revenue officers who deal with taxpayers when collecting unpaid

taxes.
Page 3 GAO-02-804 Performance Management Systems

written feedback they received on their performance.4 We used

information contained in IRS documents and guidance given to group

managers and frontline employees as criteria for assessing those systems.

We did our work between April 2001 and July 2002 in accordance with

generally accepted government auditing standards. A more detailed

description of our methodology is provided in appendix I and information

on our sampling methodology is provided in appendix II.

IRS has established critical job responsibilities for group managers and

frontline employees that align with each of the agency’s three strategic

goals. In the group managers’ performance management system, all the

supporting behaviors—those actions and competencies that the

enforcement group manager is expected to demonstrate during the

performance cycle—clearly align with the critical job responsibilities.

However, for frontline employees, the supporting behaviors do not always

align with IRS’s description of the critical job responsibilities.

Misalignments occur when supporting behaviors reflect concerns not

expressed in the description, when supporting behaviors that relate to a

responsibility are located under other responsibilities, or when no

supporting behavior exists to support the description. For example, the

description of the customer satisfaction–application responsibility for

revenue agents states that employees should communicate with taxpayers

in a clear, user-friendly manner. However, the supporting behaviors focus

more on IRS’s procedural requirements, such as the quality of internal

work products, than on whether taxpayers received understandable audit

reports. In addition to the misalignments in evaluating supporting

behaviors, IRS has not fully aligned its frontline employee performance

management system with its organizational unit performance measures.

These misalignments can reduce the new system’s ability to promote

frontline employees behaviors that would help IRS achieve its mission.

 4

 We express the precision of the results of our analysis of group managers’ commitments

and evaluations, and the survey (that is, the sampling errors associated with these

estimates) as 95 percent confidence intervals. All percentage estimates dealing with our

review of evaluations have sampling errors of +/- 11 percentage points or less, unless

otherwise stated. All percentage estimates dealing with the group manager survey have

sampling errors of +/- 12 percentage points or less, unless otherwise stated.

Results in Brief
Page 4 GAO-02-804 Performance Management Systems

During both fiscal years 2000 and 2001, almost all of the enforcement

group managers’ commitments that we reviewed were aligned with IRS’s

strategic goals. However, commitments were not always well written.

About 85 percent of the commitments did not meet IRS’s criteria of being

specific, outcome or output oriented, or easily monitored, although almost

all met IRS’s criteria of being clear and achievable. Thus, even though

these commitments may encourage desired behaviors, raters may have

difficulty holding managers accountable for meeting the majority of their

commitments because of the way they were written.

Our review of enforcement group managers’ evaluations showed that

raters generally provided feedback to managers on their performance in

meeting critical job responsibilities and commitments. For example,

during fiscal year 2000 and 2001, we estimate that more than 80 percent of

the written performance evaluations assessed managers’ performance in

meeting four or more of the five critical job responsibilities. In addition,

about 76 percent of managers’ commitments were addressed in the written

evaluations. In our survey of enforcement group managers, more than an

estimated 60 percent of managers reported receiving verbal feedback in all

five of the critical job responsibilities, and about half of managers said

they received verbal feedback on how well they met most or all of their

commitments.

According to our analysis of written evaluations, raters provided feedback

on the three critical job responsibilities related to IRS’s organizational

goals in 90 percent or more of evaluations and provided feedback on

leadership and equal employment opportunity in about 70 percent of the

evaluations. In our survey, enforcement group managers reported that

raters addressed business results more often than employee satisfaction or

customer satisfaction during both verbal and written feedback. IRS senior

executives said the relative emphasis raters were placing on the critical

job responsibilities was in line with their expectations, and noted that a

heavy emphasis on a subset of critical job responsibilities could lead

managers away from the balanced behaviors IRS seeks to achieve.

IRS senior executives have no firm plans to monitor how well the group

managers’ and frontline employees’ systems are being implemented or to

assess whether changes need to be made, even though IRS’s management

processes call for obtaining data on how well programs are achieving their

goals. As IRS continues to implement its performance management

systems, IRS can ensure the systems continue to meet its needs by

monitoring how well the systems provide useful feedback to managers and

employees about their performance and assessing whether the systems
Page 5 GAO-02-804 Performance Management Systems

encourage activities and behaviors consistent with its goals. Although IRS

senior executives are currently satisfied with the implementation of the

new systems, if the difference in raters’ emphasis on critical job

responsibilities grows over time, IRS could find that enforcement group

managers are receiving a skewed perception of the behaviors that IRS

values. In addition, IRS can begin planning for a more thorough

assessment of how well the systems are achieving their overall objectives,

such as improving communications, encouraging performance that

actually leads to organizational goals, and ensuring fair and consistent

performance evaluations.

We are making recommendations to the Commissioner of Internal

Revenue that IRS improve the alignment between strategic goals and

elements of the frontline employee performance management system and

ensure the new performance management systems are working as

intended. In a July 5, 2002 letter, the Commissioner agreed with the

recommendations contained in this report. (See agency comments and

our evaluation section and app. VII.)

The enactment of the IRS Restructuring and Reform Act of 1998 (RRA 98)

signaled congressional concern that the IRS had been emphasizing

revenue production at the expense of fairness and consideration of

taxpayers. In response, the Commissioner of Internal Revenue sought to

transform the agency’s culture to one that more equally embraced

responding to legitimate taxpayer needs, ensuring compliance with tax

laws, and considering employee needs as core organizational values. IRS

has undertaken a number of long-range initiatives to make this

transformation a reality, including

• establishing three strategic goals of “top-quality service to each taxpayer in

every interaction,” “top-quality service to all taxpayers through the fair and

uniform application of the law,” and “productivity through a quality work

environment;”

• developing a strategic planning and budgeting process that provides a

framework for operating units to develop goals and action-oriented

business plans that support the strategic goals;

• using balanced performance measures corresponding to each of the three

strategic goals—customer satisfaction, business results in terms of quality

and quantity, and employee satisfaction—to measure the performance of

every organizational unit in achieving those strategic goals; and

Background
Page 6 GAO-02-804 Performance Management Systems

• establishing new performance management systems5 designed to promote

employee behaviors and actions that support IRS’s strategic goals and the

operational business plans that lay out each operating division’s strategy

for achieving IRS’s strategic goals.

IRS’s goal in redesigning, or modernizing, its performance management

systems was to make the process more meaningful for all participants. The

new managers’ and frontline employee performance management systems

were required to include goals and objectives at the organizational, group,

or individual level, and a retention standard6

 to ensure that managers and

frontline employees administer the tax laws fairly and equitably, protect

taxpayers’ rights, and treat taxpayers with honesty, integrity, and respect.

The systems were intended to provide a framework for managers and

employees to improve communications, coordinate planning activities,

align individual performance to organizational performance, and ensure

fair and consistent performance evaluations. Further, the performance

management systems were intended to accurately reflect employees’

performance, facilitate their development, and improve and enhance their

work.

As part of the new process, IRS establishes performance expectations at

the beginning of the performance cycle that serve as the basis for the

group managers’ and frontline employees’ annual performance

evaluations. Beginning in fiscal year 2000, IRS began implementing its new

performance management system for executives and managers. The key

components of the new system for group managers are the critical job

responsibilities (behaviors) and commitments (actions). The critical job

responsibilities, which represent the new core values of IRS, replaced the

general standards for group managers that had been in effect since 1984.

They apply to all IRS executives and managers and are further defined by

supporting behaviors–broad actions and competencies that IRS expects its

 5

 Office of Personnel Management regulations define performance management as the

integrated processes agencies use to (1) communicate and clarify organizational goals, (2)

identify accountability for accomplishing organizational goals, (3) identify and address

developmental needs, (4) assess and improve performance, (5) measure performance for

recognizing and rewarding accomplishments, and (6) prepare appraisals.

6

 In July 1999 IRS incorporated into the evaluation process a new retention standard

relating to the fair and equitable treatment of taxpayers that managers and frontline

employees must meet at a passing level to retain their jobs. When assessing managers and

employees, raters are to first determine whether employees met the standard and, if the

employees did, then proceed to prepare a written evaluation of how the employees met

their critical job responsibilities.
Page 7 GAO-02-804 Performance Management Systems

executives and managers to demonstrate during the performance cycle.

The commitments, which are specific to and developed by each manager,

are distinct actions or desired results to be achieved during the

performance period. Each commitment should relate to, and support, one

or more of the critical job responsibilities.

For group managers, the performance management process includes an

annual performance evaluation—an assessment of performance supported

by written narrative commentary. While there is no current IRS

requirement that raters address managers’ performance in each critical job

responsibility or each commitment individually in the written annual

evaluation, raters are expected to provide a summary of managers’

performance that accurately describes how well managers performed in

meeting both their critical job responsibilities and commitments.

IRS began implementing the performance management system for

frontline employees in fiscal year 2001. The frontline employees’ system is

conceptually similar to that for executives and managers but there are

some differences. As with managers, frontline employees are evaluated on

how well they met their critical job responsibilities. However, the

responsibilities are tailored to reflect the activities and behaviors that

employees with direct contact with taxpayers are expected to display.

Each responsibility consists of a brief description of the key behaviors

related to the responsibility and supporting behaviors, which IRS calls

performance aspects, that further define the responsibility and

demonstrate the expected level of performance. In addition, frontline

employees are not required to develop written commitments about the

specific actions they will take to support IRS’s strategic goals.

IRS has not completed the redesign of its performance management

systems. For example, IRS envisions that its performance management

system for frontline employees will eventually include a requirement to

develop commitments, mirroring the system for executives and managers.

In addition, IRS expects to integrate the new performance management

systems with its overall human resource systems. This integration would,

for example, link evaluations to decisions about developmental needs,

rewards and recognition, and compensation. IRS expects the complete

redesign and implementation of both managers’ and frontline employees’

performance management systems to take about five years.
Page 8 GAO-02-804 Performance Management Systems

IRS has established critical job responsibilities for managers and frontline

employees that align with each of the agency’s three strategic goals.

However, supporting behaviors described in the frontline employees’

performance management system do not always align with critical job

responsibilities or with their organizational unit’s performance measures.

IRS has established critical job responsibilities and supporting behaviors

for group managers that align with the agency’s strategic goals. These

responsibilities and supporting behaviors, described on the evaluation

form as shown in appendix III, provide managers with a consistent

message about how their daily activities are to reflect the organization’s

core values. IRS’s performance management system for group managers

assesses performance in five critical job responsibilities: customer

satisfaction, business results, employee satisfaction, leadership, and equal

employment opportunity. Of these five responsibilities, three align directly

to IRS’s strategic goals, as shown in figure 1. For example, by establishing

a critical job responsibility and supporting behaviors in customer

satisfaction, IRS aligns managers’ performance to its strategic goal of “topquality service to each taxpayer in every interaction.”

Critical Job

Responsibilities Align

with Strategic Goals

but Do Not Always

Align with Other

Elements of the

Frontline Employee

Performance

Management System

Enforcement Group

Managers’ Critical Job

Responsibilities and

Supporting Behaviors

Align with Strategic Goals
Page 9 GAO-02-804 Performance Management Systems

Figure 1: Alignment of Strategic Goals, Critical Job Responsibilities, and Supporting Behaviors for Enforcement Group

Managers

Source: GAO analysis of IRS’s group manager performance management system.

The other two critical job responsibilities, leadership and equal

employment opportunity, support the agency’s strategic goals not because

they directly address IRS’s strategic goals but because they reinforce

behaviors that IRS considers necessary for organizational change and an

open and fair work environment.

As with enforcement group managers, the critical job responsibilities for

frontline employees, shown on the evaluation form in appendix III, align

with the agency’s strategic goals. IRS evaluates frontline employees

against five critical job responsibilities: customer satisfaction—

knowledge, customer satisfaction—application, business results—quality,

business results—efficiency, and employee satisfaction—employee

contribution. All five responsibilities directly align with IRS’s strategic

goals. For example, the two critical job responsibilities addressing

customer satisfaction (customer satisfaction—knowledge and customer

Frontline Employees’

Critical Job

Responsibilities Align with

Strategic Goals but Not

Always with Supporting

Behaviors and

Organizational Unit

Performance Measures
Page 10 GAO-02-804 Performance Management Systems

satisfaction—application) align with IRS’s strategic goal of “top-quality

service to each taxpayer in every interaction.” Similarly, the two critical

job responsibilities addressing business results (business results—quality

and business results—efficiency) align with IRS’s strategic goal of “topquality service to all taxpayers through the fair and uniform application of

the law.” Finally, the critical job responsibility addressing employee

satisfaction—employee contribution aligns with IRS’s strategic goal of

“productivity through a quality work environment.”

Although the framework for IRS’s frontline employee performance

management system supports the agency’s strategic goals, the description

of the critical job responsibility given to IRS employees and the

accompanying supporting behaviors do not always align with one another.

This misalignment occurs when supporting behaviors reflect concerns not

expressed in the description, when supporting behaviors that relate to a

responsibility are located under other responsibilities, or when no

supporting behavior exists to support the description.

For example, under the customer satisfaction—application responsibility

for revenue agents, IRS’s description states that employees should

communicate with taxpayers in a clear, user-friendly manner. However,

the related supporting behaviors focus more on IRS’s own procedural

requirements and internal work products, such as the quality of audit

workpapers, than on whether taxpayers received understandable reports

of their audit, as shown in figure 2. These supporting behaviors would

better align with the business results—quality critical job responsibility,

which covers how well employees accomplish tasks within IRS’s

procedures and established guidelines.
Page 11 GAO-02-804 Performance Management Systems

Figure 2: Misalignment of Critical Job Responsibility, Narrative Description, and

Supporting Behaviors for Revenue Agents

Source: GAO analysis of IRS’s frontline employee performance management system.

Also, for the customer satisfaction—knowledge responsibility for both

revenue agents and revenue officers, the description states that “correct

interpretation of laws, rules, and regulations” is a key component of the

critical job responsibility. However, for revenue agents, the supporting

behaviors that address “correct interpretation of laws” can also be found

under two other responsibilities, customer satisfaction—application and

business results—quality, even though correctly interpreting laws is not

explicitly described as a key component of those responsibilities. For

revenue officers, correctly interpreting laws is not directly addressed

under any of the critical job responsibilities.

In addition to assessing frontline employees individually using its

performance management system, IRS uses organizational unit
Page 12 GAO-02-804 Performance Management Systems

performance measures to assess how well frontline employees are

collectively achieving the agency’s strategic goals. Unit performance

measures provide aggregated feedback at different organizational levels on

how well employees are achieving IRS’s strategic goals. For example,

customer satisfaction surveys measure the quality of service employees

provided to taxpayers and quality measurement systems assess whether

employees followed IRS standards and procedures. In many cases, the

activities addressed on these organizational unit performance measures

align with the behaviors assessed in the frontline employee performance

management system. However, in some cases, the unit performance

measures and the frontline performance management system’s supporting

behaviors do not align. We found seven activities relating to informing

taxpayers of their rights, explaining the audit process, and being courteous

that were addressed in organizational unit performance measures but not

in the performance management system. For example, IRS does not

explicitly evaluate revenue agents on whether they explained the audit

process to taxpayers, an activity addressed on its examination customer

satisfaction survey. During discussions with IRS senior executives, they

agreed that the frontline employee performance management system

should better align with organizational unit performance measures.

According to our analysis of a sample of enforcement group managers’

evaluations during both fiscal years 2000 and 2001, almost all of the

commitments managers developed aligned with IRS’s strategic goals but

these commitments were not always written so that managers could be

held accountable for meeting them because they were not specific,

outcome or output oriented, or easily monitored.

Almost all of the commitments written by our sample of enforcement

group managers aligned with their critical job responsibilities and thus

ultimately with IRS’s strategic goals. Table 1 provides examples of

commitments that align with strategic goals.

Enforcement Group

Managers’

Commitments Align

with IRS’s Strategic

Goals but Are Not

Always Written So

that Raters Can Hold

Managers

Accountable

Commitments Align with

Strategic Goals
Page 13 GAO-02-804 Performance Management Systems

Table 1: Examples of Enforcement Group Managers’ Commitments that Align with

IRS’s Strategic Goals

Strategic goals Commitments

Top-quality service to each

taxpayer in every interaction

“I will ensure employees are prompt and professional

in their dealings with taxpayers by being directly

involved in examinations through observations and

visits.” (improves customer satisfaction)

Top-quality service to all

taxpayers through the fair and

uniform application of the law

“I will review quality measurement system results to

identify deficient areas, address deficiencies at group

meetings and take corrective actions as needed.”

(improves business results)

Productivity through a quality

work environment

“I will perform evaluative reviews, such as case

reviews, on-the-job visits and workload reviews, on

each employee and provide immediate feedback on

their performance.” (improves employee satisfaction)

Source: GAO analysis of fiscal year 2000 and 2001 enforcement group managers’ commitments in

the written performance evaluation.

Although commitments align with IRS’s strategic goals, our analysis of

fiscal year 2000 and 2001 written performance evaluations shows that

enforcement group managers’ commitments were not evenly distributed

among the five critical job responsibilities.7

 Employee satisfaction and

business results were each addressed in more than a third of commitments

each year, while a quarter addressed customer satisfaction. Leadership

and equal employment opportunity were each addressed by fewer than 10

percent of the commitments in each year. Although these critical job

responsibilities do not directly align with IRS’s strategic goals, IRS senior

executives believe that effective leadership is an important component of

changing organizational culture and supporting equal employment

opportunity principles is important to establishing a fair work

environment. Table 2 shows the percentage of commitments that focused

on each responsibility.

 7

 In our analysis, we only counted the number of commitments that aligned with each of

the critical job responsibilities. We did not try to assess the significance or relative

importance of each commitment.
Page 14 GAO-02-804 Performance Management Systems

Table 2: Percentage of Commitments Reflecting Critical Job Responsibilities for

Fiscal Years 2000 and 2001

Percentage of commitments

Critical job responsibility FY 2000 FY 2001

Customer satisfaction 24 25

Business results 34 42

Employee satisfaction 37 40

Leadership 7 6

Equal employment opportunity 7 4

Note: Because managers’ commitments might apply to more than critical job responsibility,

percentages do not add to 100.

Source: GAO analysis of fiscal year 2000 and 2001 enforcement group managers’ commitments in

the written performance evaluation.

IRS senior executives were generally pleased with the distribution of

enforcement group managers’ commitments among the critical job

responsibilities. They said that the distribution of commitments among the

three responsibilities aligned most directly with IRS’s strategic goals was

relatively equal and that it was more difficult for managers to develop

commitments for the other two responsibilities.

In an attempt to assist managers in developing their commitments and

hold them accountable for meeting commitments, IRS issued guidance in

November, 2000 regarding criteria for well-constructed commitments.

According to the guidance, well-constructed commitments should be

clear, specific, achievable, outcome or output oriented, and easily

monitored. Appendix I provides an explanation of how we used these

criteria to assess enforcement group managers’ individual commitments.

In keeping with the guidance for well-written commitments, we found that

almost all of the enforcement group managers’ commitments in our

sample of evaluations met two of IRS’s five criteria—being clear and

achievable. For example, we concluded that the following commitments

would be understood by both the manager and the rater, did not include

jargon unfamiliar to IRS employees, and described actions that could

realistically be accomplished within the normal work environment.

“I will provide resources where necessary to ensure an effective filing season.”

Raters May Have Difficulty

Holding Managers

Accountable
Page 15 GAO-02-804 Performance Management Systems

“I will encourage and support the development of employees through the use of individual

development plans, as submitted, and the assignment of appropriate details whenever

possible.”

“I will work with my group members to ensure that our jointly developed FY 2000 action

plan objectives are met or exceeded.”

In contrast, our analysis showed that about 85 percent of the enforcement

group managers’ commitments, for both rating years, did not meet the

remaining three IRS criteria for well-written commitments of being

specific, outcome or output oriented, or easily monitored. We found that

the majority of commitments were broad, vague statements that did not

give a relatively clear indication of the action that would be taken, and did

not include relevant information such as the frequency of the action or

what would be accomplished. Moreover, there was little focus on

expected outcomes, such as the potential impact of actions on employees

or taxpayers, or outputs, such as the number of activities to be

accomplished, for which the manager’s performance in meeting the

commitment could be assessed or measured. In IRS’s performance

management system, raters and managers share responsibility for ensuring

that commitments meet the criteria for well-written commitments—

managers initially develop the commitments and raters review and, if

necessary, revise them. The majority of group managers reported in our

survey that their raters were actively involved in revising their

commitments.

Table 3: Percentage of Commitments Not Specific, Outcome/Output Oriented, or

Easily Monitored for Fiscal Years 2000 and 2001

Percentage of commitments

not meeting criteria

Criteria FY 2000 FY 2001

Specific 84 85

Outcome/output oriented 84 85

Easily monitored 85 86

Source: GAO analysis of fiscal year 2000 and 2001 enforcement group managers’ commitments in

the written performance evaluation.

The commitments that were not specific, outcome or output oriented, or

easily monitored could, nevertheless, be encouraging desired behaviors.

However, because so many commitments did not meet these criteria,

raters may have difficulty holding enforcement group managers

accountable for meeting the majority of their commitments because of the
Page 16 GAO-02-804 Performance Management Systems

way they were written. The following two commitments illustrate a vague

commitment and a specific, outcome/output oriented and easily monitored

commitment:

• “I will provide leadership by increased managerial involvement in my

employees’ casework.”

• “I will conduct quarterly reviews of casework focusing on IRM (Internal

Revenue Manual) requirements and CQMS (Collections Quality

Management System) to ensure timely closing of cases.”

It would be difficult to definitively judge whether a manager met the first

commitment because the statement is vague about what the manager

intends to do. In contrast, the second commitment is explicit about the

type of managerial involvement, the frequency of the reviews, the

standards to be applied during the reviews, and the potential impact of the

manager’s involvement. A rater could monitor and assess whether the

manager conducted these types of reviews as often as promised, and use

timeliness statistics to assess the manager’s performance in actually

closing cases in less time. Although performance management systems are

inherently subjective to some degree because raters have to assess how

well employees carry out their duties, raters also must make subjective

judgments about whether commitments are met when those commitments

are vague. When commitments are more explicit, there is a greater

likelihood that both managers and raters will know and agree whether

managers have met their commitments, and that commitments are

specific, outcome or output oriented, and easily monitored. Appendix VI

provides additional examples of commitments.
Page 17 GAO-02-804 Performance Management Systems

According to our review of fiscal year 2000 and 2001 written evaluations

and our survey of enforcement group managers, raters generally provided

feedback to managers on how well they met their critical job

responsibilities and commitments by addressing them in written or verbal

feedback. However, raters provided more extensive written and verbal

feedback for some critical job responsibilities than for others. In addition,

for each critical job responsibility, raters provided more extensive

feedback on certain supporting behaviors than on others. In our survey,

enforcement group managers reported receiving more extensive written

and verbal feedback on business results than on the other critical job

responsibilities and because of that they believed that business results was

more important to their raters than the other critical job responsibilities.

Additionally, half of the enforcement group managers we surveyed

reported receiving verbal feedback on how well they met most or all of

their commitments.

Our review of enforcement group managers’ performance evaluations

showed that raters provided written and verbal feedback to managers on

their performance in meeting most critical job responsibilities and

commitments. However, raters provided more extensive written and

verbal feedback for some critical job responsibilities and for certain

supporting behaviors than for others. As shown in table 4, our sample of

performance evaluations for IRS employees who were managers in both

fiscal years 2000 and 2001 showed that more than 80 percent of the written

evaluations assessed managers’ performance in meeting four or more of

the five critical job responsibilities.

Raters Generally

Provided Feedback to

Enforcement Group

Managers on Meeting

Critical Job

Responsibilities and

Commitments but

Provided More

Extensive Feedback

on Some Critical Job

Responsibilities Than

Others

Raters Provided Feedback

to Enforcement Group

Managers on Meeting Most

Critical Job

Responsibilities and

Commitments in

Evaluations and Verbal

Feedback
Page 18 GAO-02-804 Performance Management Systems

Table 4: Estimated Percentage of Fiscal Years 2000 and 2001 Written Evaluations in

Which Raters Assessed Critical Job Responsibilities by Number of Responsibilities

Percentage of

written evaluations

Number of critical job

responsibilities assessed FY 2000 FY 2001

5 54 47

4 29 41

3 or fewer 17 12

Total 100 100

Source: GAO analysis of fiscal year 2000 and 2001 enforcement group managers’ written

performance evaluations.

As noted in the previous section, commitments were not always written in

a way that was conducive for raters to hold enforcement group managers

accountable. This fact notwithstanding, raters provided feedback to

managers on how well they met their commitments. Specifically, raters

provided written feedback on managers’ performance in meeting most of

their commitments. Our sample of fiscal year 2000 and 2001 written

performance evaluations indicates that raters addressed 76 percent of

commitments in employee evaluations, discussing each commitment

separately. Raters addressed 6 percent of commitments by providing

summary feedback such as “you met or exceeded your commitments” and

provided no feedback on 18 percent of commitments. According to senior

executives, IRS is considering new guidance that would require raters to

assess managers’ performance in meeting each commitment, with the

intent of holding managers more accountable for achieving the

commitments they make.

In our survey, most enforcement group managers reported receiving

verbal feedback about their performance in meeting critical job

responsibilities and commitments. We estimate that for more than 60

percent of the managers, raters discussed all five responsibilities in the

verbal feedback that occurs with the delivery of the written performance

evaluation, and in ongoing verbal feedback throughout the year. Table 5

below shows the percentage of enforcement group managers who

reported receiving verbal feedback throughout the year in each of the

critical job responsibilities.
Page 19 GAO-02-804 Performance Management Systems

Table 5: Estimated Percentage of Enforcement Group Managers Reporting Verbal

Feedback by Number of Critical Job Responsibilities for Fiscal Year 2001

Number of critical job

responsibilities covered

during feedback

Verbal feedback with

written performance

evaluation

Ongoing verbal

feedback

5 63 73

4 67

3 or fewer 31 20

Total 100 100

Source: GAO survey of IRS enforcement group managers.

In addition, an estimated half of the enforcement group managers in our

survey reported receiving verbal feedback on how well they met most or

all of their commitments.

In our review of enforcement group managers’ written evaluations, we

found that raters were more likely to assess a manager’s performance in

terms of customer satisfaction, business results, and employee

satisfaction, and less likely to assess performance in leadership and equal

employment opportunity. As shown in table 6, about 72 percent of written

performance evaluations in fiscal years 2000 and 2001 assessed leadership

or equal employment opportunity responsibilities. The employee

satisfaction responsibility was the only critical job responsibility that was

assessed in all evaluations and in both fiscal years.

Table 6: Estimated Percentage of Fiscal Year 2000 and 2001 Evaluations Assessing

Enforcement Group Managers’ Performance in Each Critical Job Responsibility

Percentage of

written evaluations

Critical job responsibility FY 2000 FY 2001

Customer satisfaction 92 96

Business results 96 99

Employee satisfaction 100 100

Leadership 72 71

Equal employment opportunity 72 71

Source: GAO analysis of fiscal year 2000 and 2001 enforcement group managers’ written

performance evaluations.

Furthermore, for each critical job responsibility, certain key supporting

behaviors were emphasized more than others in the written performance

evaluations, as shown in appendix V. For example, IRS has identified

Raters Provide More

Extensive Feedback on

Some Critical Job

Responsibilities and

Supporting Behaviors

Than on Others
Page 20 GAO-02-804 Performance Management Systems

certain key behaviors that support the critical job responsibility of

employee satisfaction. Of these behaviors, we found that “effectively uses

ongoing feedback, coaching, and timely evaluations of performance to

promote cooperation, teamwork, knowledge/skill sharing and goal

accomplishment” was a key behavior that raters discussed in about 80

percent of written performance evaluations. In contrast, our sample

indicated that a key supporting behavior that was discussed in less than 40

percent of evaluations was “creates an environment for continuous

learning, pursuing development opportunities for self and others, with the

intent to increase individual and organizational effectiveness.” During our

discussions, IRS’s senior executives said they were comfortable with the

relative emphasis placed on each of the supporting behaviors.

As shown in Table 7, the evidence from our study is that enforcement

group managers believed that their raters emphasized business results the

most and emphasized equal employment opportunity the least in the

written performance evaluation and ongoing verbal discussions. In

addition, we estimate that equal employment opportunity was emphasized

the least in the verbal feedback occurring when the written evaluation was

delivered.

Table 7: Estimated Percentage of Enforcement Group Managers Reporting

Feedback to a Great or Very Great Extent for the Critical Job Responsibilities for

Fiscal Year 2001

Type of feedback

Critical job responsibility

Written

evaluation

Verbal feedback

with written

performance

evaluation

Ongoing

verbal

feedback

Customer satisfaction 51 31 31

Business results 60 36 43

Employee satisfaction 51 33 33

Leadership 49 34 30

Equal employment opportunity 31 23 13

Source: GAO survey of IRS enforcement group managers.

As shown in table 8, we found that managers believed raters conveyed that

business results was the most important to them and equal employment

opportunity the least, when they took all forms of feedback into

consideration.

Group Managers Report

Raters’ Feedback Focused

More on Business Results
Page 21 GAO-02-804 Performance Management Systems

Table 8: Estimated Percentage of Enforcement Group Managers Reporting that the

Respective Critical Job Responsibilities Are Very Important or Important to Their

Raters for Fiscal Year 2001

Critical job responsibility

Perception of which respective critical

job responsibilities are very important

or important to their raters

Customer satisfaction 69

Business results 87

Employee satisfaction 68

Leadership 74

Equal employment opportunity 58

Source: GAO survey of IRS enforcement group managers.

IRS officials explained that while they do not intend that equal emphasis

be always given to all five critical job responsibilities, or to all

commitments, they believe that an over emphasis on one or just a few of

the job responsibilities would be inappropriate. Overall, they were

satisfied with the emphasis given to each of the five critical job

responsibilities in both the evaluations and the commitments. However,

they noted that a heavy emphasis on some critical job responsibilities

could lead managers away from the balanced behaviors that IRS seeks to

achieve through its new performance management system.

IRS has made progress in the challenging task of redesigning its

performance management systems to help reinforce behaviors and actions

that support the agency’s mission. IRS senior executives were generally

satisfied with the implementation of the new performance management

systems but expect that the systems will continue to evolve as the agency

gains more experience in implementing the new culture and processes.

However, IRS has not established mechanisms to monitor how well the

systems provide useful feedback to group managers and frontline

employees about their performance and whether feedback aligns with

IRS’s strategic goals. For example, IRS does not have plans to determine if

raters are over-emphasizing one critical job responsibility during written

and verbal feedback and when managers develop commitments. Further,

IRS does not have plans to assess at an appropriate time whether the new

performance management systems are achieving their objectives. Such an

assessment would be consistent with IRS’s guidance for implementing

strategic initiatives.

IRS Has Made

Progress but Could

Further Enhance Its

New Managers’ and

Frontline Employees’

Performance

Management Systems
Page 22 GAO-02-804 Performance Management Systems

IRS has implemented a number of initiatives to support its performance

management systems, and set the stage for the reform of its entire

performance management system over the coming years. According to

officials, in response to RRA 98 and as part of its overall modernization

efforts, IRS realized it needed to redesign its performance management

process to better communicate the behaviors constituting customer

satisfaction, business results, and employee satisfaction, and to ensure

managers and employees adopt the newly desired behaviors in their dayto-day activities.

In an attempt to make managers more familiar with the new system, IRS

has undertaken several initiatives, with similar actions also planned for its

frontline employees, such as

• providing interim guidance and templates of sample commitments, selfassessments, and summary evaluations;

• distributing computer discs with user-friendly access to information on

performance management and added performance management

information to the IRS intranet website, where it is constantly being

updated;

• conducting an interactive videoconference for all managers on the new

performance management system.

Further, IRS has started its implementation of a pay-for-performance

system, which emphasizes performance instead of longevity in

determining compensation. IRS is also experimenting to see whether the

manager evaluation forms might also be suitable for making decisions

about developmental needs, rewards and recognitions, and compensation.

IRS has already placed senior managers in a senior pay-band, and expects

that other managers, including those at the group manager level, will be

placed in pay-bands by October 2002. Ultimately, IRS plans to rollout the

pay-for-performance system to include all managers as well as frontline

employees.

IRS Has Made Progress in

Making Its Managers’ and

Frontline Employees’

Performance Management

Systems an Effective

Management Tool
Page 23 GAO-02-804 Performance Management Systems

IRS recognizes that revamping its employee performance management

systems is a major effort that presents significant implementation

challenges. IRS senior executives were generally satisfied with the

implementation of the managers’ performance management system given

that it has only been in place for two years. They expect that

implementation of both systems will improve as raters gain more

experience in implementing the new culture and processes. For example,

they believe it will take some time for raters of group managers and

frontline employees to fully interpret the new critical job responsibilities

and become comfortable with how to apply them.

However, senior executives believe that IRS’s plans to expand its pay-forperformance system underscore the need to ensure verbal and written

feedback are consistent with the message IRS wants to deliver to its

managers and employees about the activities and behaviors it values. They

believe the validity of pay and bonus decisions will depend heavily on how

well the performance management systems clearly communicate what

managers and employees are expected to do as well as provide

comprehensive feedback on how well they performed in carrying out their

responsibilities and meeting their commitments. IRS senior executives are

concerned that if misunderstandings about performance expectations

creep into the system, for example because commitments are vaguely

worded, managers and employees will begin to discredit the performance

management system and the pay-for-performance decisions. IRS senior

executives believe that raters have to be clear about their expectations so

that managers know in advance what they need to do to be successful and

receive bonuses or salary increases.

IRS has not established a monitoring mechanism to ascertain whether

raters are implementing the new employee performance management

systems as intended and has no plans to assess whether the new systems

are achieving their objectives. IRS senior executives said they have done a

limited assessment of the evaluations for executives and senior managers

because they are the only group currently under the pay-for-performance

system. IRS senior executives said they have not established a monitoring

mechanism or a plan to assess the managers’ or frontline employees’

systems due to resource constraints.

A monitoring mechanism could provide useful information on whether

IRS’s processes for evaluating employees are being followed. A monitoring

system could include, for example, an employee survey to obtain

information on whether raters are involved in the development of their

IRS Officials Generally

Satisfied with

Implementation of

Managers’ and Frontline

Employees’ Performance

Management Systems but

Realize Implementation

Challenges Remain

Monitoring and

Assessment Mechanisms

Would Help Ensure IRS’s

Employee Performance

Management Systems Are

Implemented as Intended
Page 24 GAO-02-804 Performance Management Systems

subordinates’ commitments, giving appropriate emphasis to each of the

critical job responsibilities and supporting behaviors, and are providing

useful feedback. Also, content analyses of commitments and written

performance evaluations could help senior executives monitor whether

the systems are encouraging the kinds of behaviors and activities that

support IRS’s strategic goals. Active monitoring could give IRS senior

executives a sense of how the systems are working in practice and

whether any modifications are needed to provide more useful feedback to

managers and employees about their performance and better align the

systems with IRS’s strategic goals.

Although we have not estimated the costs to perform such monitoring,

costs likely could be minimal. Relatively small statistical samples of

performance evaluations and of employees to survey likely would be

sufficient, and once a monitoring methodology is developed it could be

used for several years. Further, to the extent such monitoring could

prevent potential problems, IRS would avoid the costs associated with

employee performance that is not well-balanced and aligned with IRS’s

goals.

In addition to active monitoring of the employee performance

management systems, a more thorough future assessment of whether the

systems are achieving their various goals and objectives could provide IRS

assurance that the systems are properly designed and implemented, or

identify whether changes are needed. For example, IRS could assess

whether the systems are providing a framework for improving

communications between raters and their subordinates, coordinating

planning activities, aligning individual performance to organizational

performance, and ensuring fair and consistent performance evaluations.

Although it would be premature to perform an assessment now—since the

systems are not fully implemented—planning now for such an assessment

could better ensure that IRS will have the necessary data to assess the

performance management systems’ performance in the future. Such upfront planning is, for instance, part of IRS’s guidance for planning and

implementing reorganizations. That guidance requires that a plan for

assessing the results of a reorganization be developed at the same time

that the reorganization is proposed.

Monitoring and assessing IRS’s new performance management systems

also would be in line with the expectations IRS has established for the

strategic initiatives being carried out by its operating divisions. In March

2000, IRS implemented a new strategic planning, budgeting, and

performance management process intended to provide a more structured
Page 25 GAO-02-804 Performance Management Systems

format for establishing strategic direction, identifying the resources

needed to accomplish the operational activities supporting the strategy,

and assessing performance results.8

 IRS recognizes that conducting

assessments of operational activities at all levels of the organization is

essential to ensuring its programs are achieving their goals. As IRS’s

strategic plan points out, regular, structured feedback on program

performance is essential if managers are to recognize and react to

successes or failures and ensure programs are adhering to the agency’s

strategic intent.

Although IRS would incur costs to assess whether its employee

performance management systems achieve their objectives, IRS’s strategic

planning, budgeting and performance management guidance implicitly

recognizes that such costs are properly part of a sound management

system. Further, planning ahead for such assessments can help ensure that

costs are minimized. Finally, as with monitoring, to the extent that an

assessment results in further improvements to IRS’s performance

management systems, costs due to unbalanced or misaligned employee

behaviors would be avoided.

IRS is in the midst of a cultural change that requires group managers and

frontline employees to better balance taxpayer needs, business results,

and employee needs in carrying out their day-to-day responsibilities. By

aligning its new performance management systems with its strategic and

operational goals, IRS has made progress in supporting this cultural

change and providing guidance to managers and employees on how to

prioritize their activities and carry out their responsibilities. In addition,

most enforcement group managers received written or verbal feedback on

how well they met their critical job responsibilities and commitments.

However, we identified several opportunities to enhance IRS’s

performance management systems for managers and employees. IRS

could better align employee performance with strategic goals by

establishing greater consistency within the frontline employee

performance management system. When misalignment exists between the

unit performance measures and the frontline employee performance

management system, IRS is emphasizing different behaviors at the unit

 8

 IRS’s strategic guidance calls for periodic evaluations. For the purposes of this report, we

are using the term assessment.

Conclusions
Page 26 GAO-02-804 Performance Management Systems

level than at the employee level. As a result, IRS is not consistently

communicating its strategic goals to employees.

Commitments could be developed to be more specific, outcome or output

oriented, and easily monitored so that group managers can be held

accountable for actions they intend to take to support IRS’ goals. IRS is

considering new guidance that would require raters to assess managers’

performance in meeting each commitment, with the intent of holding

managers more accountable. Such guidance on providing more

comprehensive feedback would further reinforce use of IRS’s new

performance management system for managers to encourage behaviors

and actions that support IRS’s strategic goals.

Further, if unequal emphases in written and verbal feedback grow over

time, the performance management systems might not adequately align

with strategic goals and provide the intended message about how

managers and employees should conduct their daily activities. If so,

managers could receive a skewed perception of the behaviors that IRS

values. Misusing even a well-designed performance management system

can drive dysfunctional behaviors, such as an overemphasis on certain

activities to the detriment of the balanced approach IRS is trying to

achieve and maintain. Accordingly, it is important for IRS to monitor and

assess its progress in fully implementing the systems and position itself to

take corrective action if and when needed.

To better hold managers accountable for meeting strategic goals and

ensure the new performance management systems are working as

intended, we recommend that the Commissioner of Internal Revenue take

steps to:

• improve the linkage between frontline employees’ critical job

responsibilities, the supporting behaviors, and organizational unit

performance measures and

• develop plans for monitoring and assessing whether the new employee

performance management systems are operating as intended and take

the necessary actions to resolve any identified problems.

On July 5, 2002, we received written comments on a draft of this report

from the Commissioner of Internal Revenue (see app. VII). The

commissioner agreed with the recommendations contained in this report

and said they would be included as IRS works to continuously improve its

Recommendations for

Executive Action

Agency Comments

and Our Evaluation
Page 27 GAO-02-804 Performance Management Systems

performance management system. In his comments, the Commissioner

noted that IRS currently uses performance review boards to ensure

executives’ and senior managers’ performance agreements and evaluations

align with and reflect IRS’s strategic goals but this approach is less

practical at lower levels of the organization due to the larger numbers

involved. Therefore, IRS is going to explore other means of ensuring

alignment. In addition, the Commission said IRS will continue to monitor

the overall efficacy of its performance management system.

As agreed with your office, unless you publicly announce its contents

earlier, we plan no further distribution of this report until 30 days from the

date of this letter. At that time, we will send copies to the Ranking

Minority Member of the Committee on Ways and Means; the Ranking

Minority Member of the Subcommittee on Oversight; the Secretary of the

Treasury; and the Commissioner of Internal Revenue. We will also make

copies available to others on request. In addition, the report will be

available at no charge on the GAO Web site at http://www.gao.gov.

Please contact me at (202) 512-9039, Ralph Block at (415) 904-2150, or

Jonda Van Pelt at (415) 904-2186 if you or your staff have any questions.

Key contributors to this report are acknowledged in appendix VIII.

Michael Brostek

Director, Tax Issues
Appendix I: Objectives, Scope, and

Methodology

Page 28 GAO-02-804 Performance Management Systems

Our objectives in this report were to determine (1) the extent to which

group managers’ and frontline employees’ critical job responsibilities and

other elements of the employee performance management systems are

structurally aligned with IRS’s strategic goals and organizational unit

performance measures, (2) the extent to which group managers’

commitments align with IRS’s strategic goals and are written so that raters

can hold managers accountable for meeting their commitments, (3) the

extent to which raters provide feedback to enforcement group managers

on their performance in meeting critical job responsibilities and

commitments, and (4) whether IRS has adequate plans to monitor and

assess the implementation of its performance management systems.

To determine the extent to which the critical job responsibilities for group

managers and frontline employees align with IRS’s strategic goals, we

performed separate qualitative analyses of both performance management

systems to assess whether the critical job responsibilities align with IRS’s

goals and whether supporting behaviors were consistent with the critical

job responsibilities. Our analysis of the frontline employee system also

included reviewing organizational unit performance measures to

determine if these measures were being reflected in the performance

management system’s supporting behaviors. We only included those

components of the unit measures that deal with employee behaviors and

actions, not those measures that address how well IRS’s procedures and

systems function or that set organizational performance goals.

To determine the extent to which group managers’ commitments align

with IRS’s strategic goals and are written so that managers can be held

accountable for meeting those commitments, we analyzed the

commitments of a statistically valid random sample of group managers in

IRS’s Small Business/Self-Employed Division.1

 We selected this IRS

division because it included managers in key enforcement positions who

supervise employees dealing directly with taxpayers when auditing tax

returns or collecting unpaid taxes. To determine whether commitments

aligned with IRS’s strategic goals, we assessed whether the objective of

the commitment was to improve customer satisfaction, business results,

 1

 We express the precision of the results of our analysis of group managers’ commitments

and evaluations (that is, the sampling errors associated with these estimates) as 95 percent

confidence intervals.

Appendix I: Objectives, Scope, and

Methodology

Objectives

Scope and

Methodology
Appendix I: Objectives, Scope, and

Methodology

Page 29 GAO-02-804 Performance Management Systems

employee satisfaction, leadership or equal employment opportunity.2 To

determine the extent to which managers could be held accountable for

meeting their commitments, we applied IRS’s criteria for formulating wellconstructed commitments, which included that they are clear, specific,

achievable, outcome or output oriented, and are easily monitored. As we

applied the criteria to individual commitments, we found

interrelationships between certain criteria. For example, commitments

that are specific also tend to be outcome or output oriented and easily

monitored. Similarly, commitments that include explicit outcomes or

outputs tend to be easier to monitor. As agreed with IRS officials, we

addressed the following questions in determining whether commitments

met the criteria:

• Clear: Is the commitment easy to understand? Does it contain jargon that

might make it unfamiliar to managers or their supervisors?

• Specific: Does the commitment include details, such as the specific

actions to be taken, when the actions are to be taken, whether other IRS

employees are involved, the expected result, or a numeric target? Is the

commitment specific enough so that a third party can understand the

actions and timeframes to which the manager is committing?

• Achievable: Can the manager accomplish the task in the normal work

environment? To what extent does the manager control the resources

necessary to accomplish the commitment?

Outcome or output oriented: Does the commitment include an expected

result, the type and level of activity to be accomplished, or a program

impact?

• Easily monitored: Does the commitment include a deadline, an

expected result, a numeric target, or some other means of measurement?

To determine the extent to which raters provided feedback to group

managers on how well they met their critical job responsibilities and

commitments, we performed a content analysis of the narrative portion of

the evaluation that discussed managers’ performance. To assess the extent

to which managers were given feedback on meeting their critical job

responsibilities, we assessed whether the evaluation narrative described

behaviors that were consistent with the key supporting behaviors for each

 2

 Our assessment was limited to a count of the number of commitments, not the

significance or the relative challenge of accomplishing them.
Appendix I: Objectives, Scope, and

Methodology

Page 30 GAO-02-804 Performance Management Systems

critical job responsibility as laid out on the managers evaluation form (see

app. III for a sample of the evaluation.) We also reviewed the narrative to

determine whether the rater assessed how well the managers met their

commitments. In addition, we surveyed the managers for information they

received on their performance during verbal feedback sessions held with

their raters.3

To determine whether IRS has adequate plans to monitor and assess the

implementation of its employee performance management systems, we

talked to IRS officials about initiatives taken to date and reviewed IRS

guidance for strategic planning.

Our review was subject to some limitations. Our assessment of group

managers’ commitments and our content analysis required us to make

judgments that were, in part, subjective. To maximize the objectivity of

our analyses, we had our reviewers conduct separate and independent

assessments of (1) whether commitments met IRS’s criteria and (2) the

narrative discussions of each sampled evaluation. When differences arose

between the two reviewers, a collaborative approach was used to resolve

them.

 3

 As with our analysis of managers’ commitments and performance evaluations, we express

the precision of the results of our survey (that is, the sampling errors associated with these

estimates) as 95 percent confidence intervals.
Appendix II: GAO Sampling Methodology

Page 31 GAO-02-804 Performance Management Systems

This appendix discusses the sampling methodology we used to determine

(1) to what extent group managers’ commitments aligned with IRS’s

strategic goals and were written so that raters can hold group managers

accountable for meeting their commitments and (2) to what extent raters

provided feedback to group managers on how well they met their

commitments and critical job responsibilities.

We randomly selected a probability sample of 150 from the study

population of 1,374 group managers with enforcement responsibilities

working in IRSs Small Business/Self-Employed Division. These managers

were identified from a database of IRS personnel information current as of

January 2001. With this statistically valid probability sample each member

of the population had a nonzero probability of being included, and that

probability could be computed for any member. Each manager selected in

the sample was subsequently weighted in the analysis to account

statistically for all the members of the population, including those that

were not selected.

For the objective of determining to what extent group managers’

commitments aligned with IRS’s strategic goals and were written so that

raters can hold managers accountable for meeting their commitments, we

reviewed fiscal year 2000 and fiscal year 2001 written annual employee

evaluations. For this objective, we found that 126 group managers in our

sample had enforcement responsibilities in fiscal years 2000 and 2001. We

therefore estimate that about 1,154 group managers in the study

population had enforcement responsibilities in fiscal years 2000 and 2001.

For the objective of determining to what extent raters provided feedback

to group managers on how well they met their commitments and critical

job responsibilities, we reviewed fiscal year 2000 and fiscal year 2001

employee evaluations. For this objective, we found that 120 group

managers in our sample had enforcement responsibilities in fiscal year

2002. We therefore estimate that about 1,099 group managers in the study

population had enforcement responsibilities in fiscal year 2002. We also

administered a web-based survey to obtain information from current,

fiscal year 2002, managers on the verbal feedback that raters provided to

them.

As we received evaluations from IRS we found that some of the managers

in our sample were temporary, acting group managers who were not

assessed using the same evaluation form as permanent group managers.

Appendix II: GAO Sampling Methodology

Sample Designs
Appendix II: GAO Sampling Methodology

Page 32 GAO-02-804 Performance Management Systems

We were unable to use these evaluations in our analysis and deleted

temporary, acting group managers from our sample.

For our review of written annual performance evaluations, we received

112 usable evaluations (those that indicated the employee was an

enforcement group manager for both, complete fiscal years and included

both commitments and summary narrative) out of 126 eligible respondents

for a response rate of approximately 89 percent. We eliminated 24

ineligible group managers, including those for whom we did not have two

complete evaluations as a group manager or who functioned in a special

capacity such as training specialist.

For our survey of enforcement group managers’ perceptions of the amount

and type of feedback they received from raters, we received 84 usable

responses (those who completed the survey and indicated that they were

currently a group manager with enforcement responsibilities) out of 120

eligible respondents for a response rate of 70 percent. We eliminated 30

ineligible employees, including those who were no longer a group manager

due to job reassignment or retirement.

The disposition for the sampled cases for our review of group manager

evaluations and for our group manager survey is shown in table 9 and

table 10, respectively.

Table 9: Disposition of Sample Cases for Our Review of Group Manager

Evaluations

Total number of initially sampled elements 150

Sampled elements outside the study population (i.e., “ineligibles”)

Not a group manager with enforcement responsibilities for FY 2000 and 2001 24

Subtotal of ineligible elements 24

Sampled elements in the study population (i.e., “eligibles”)

Nonrespondents

Provided unusable evaluation(s) 6

No evaluation(s) available 8

Respondents (provided usable evaluations) 112

Subtotal of eligible elements 126

Evaluation response rate (respondents/subtotal of eligible elements) 89%

Source: IRS data and GAO sample.

Sample Disposition
Appendix II: GAO Sampling Methodology

Page 33 GAO-02-804 Performance Management Systems

Table 10: Disposition of Sample Cases for Our Survey of Group Managers’

Perceptions of Raters’ Feedback

Total number of initially sampled elements 150

Sampled elements outside the study population (i.e., “ineligibles”)

Not a current group manager with enforcement responsibilities for FY 2002 30

Subtotal of ineligible elements 30

Sampled elements in the study population (i.e., “eligibles”)

Nonrespondents

Refused 2

Unable to respond 1

No response after contact 26

Unable to contact due to missing or incorrect contact information 2

Returned unusable survey 5

Respondents (returned usable surveys) 84

Subtotal of eligible elements 120

Evaluation response rate (respondents/subtotal of eligible elements) 70%

Source: IRS data and GAO sample.

Because we followed a probability sampling procedure based on random

selections, our sample is only one of a large number of samples that we

might have drawn. Since each sample could have provided different

estimates, we express our confidence in the precision of our particular

sample’s estimates by providing a sampling error (for example, +/- 10

percentage points) with a specified confidence level, for example, the 95-

percent confidence level. The confidence interval (i.e., the estimate plus or

minus its sampling error) would contain the actual population value for 95

percent of the samples we could have drawn. As a result, we are 95-

percent confident that a confidence interval includes the true value in the

study population. In this report, all percentage estimates dealing with our

review of evaluations have sampling errors of +/- 11 percent or less of the

value of those numerical estimates, unless otherwise stated. All

percentage estimates dealing with the group manager survey have

sampling errors of +/- 12 percent or less, unless otherwise stated.

In addition to the reported sampling errors, the practical difficulties of

conducting any survey may introduce other types or “nonsampling” errors.

For example, differences in how a particular question is interpreted or the

types of individuals that do not respond can introduce unwanted

variability into the survey results. To minimize such nonsampling errors,

we pre-tested the survey with four enforcement group managers and

Sampling Error

Nonsampling Error
Appendix II: GAO Sampling Methodology

Page 34 GAO-02-804 Performance Management Systems

incorporated their feedback into the survey design. We did not analyze the

survey non-respondents or employee evaluations non-respondents to

determine whether the non-response was random. We assumed that the

survey and employee evaluations non-respondents had the same

characteristics as group managers who responded to the survey and

provided usable evaluations, respectively.
Appendix III: Samples of Manager and

Frontline Employee Evaluation Forms

Page 35 GAO-02-804 Performance Management Systems

Appendix III: Samples of Manager and

Frontline Employee Evaluation Forms

The following sixpage form is used to

evaluate

performance for

managers.
Appendix III: Samples of Manager and

Frontline Employee Evaluation Forms

Page 36 GAO-02-804 Performance Management Systems
Appendix III: Samples of Manager and

Frontline Employee Evaluation Forms

Page 37 GAO-02-804 Performance Management Systems
Appendix III: Samples of Manager and

Frontline Employee Evaluation Forms

Page 38 GAO-02-804 Performance Management Systems
Appendix III: Samples of Manager and

Frontline Employee Evaluation Forms

Page 39 GAO-02-804 Performance Management Systems
Appendix III: Samples of Manager and

Frontline Employee Evaluation Forms

Page 40 GAO-02-804 Performance Management Systems
Appendix III: Samples of Manager and

Frontline Employee Evaluation Forms

Page 41 GAO-02-804 Performance Management Systems

The following twopage form is used to

evaluate performance

for IRS frontline

employees, including

revenue agents and

revenue officers.
Appendix III: Samples of Manager and

Frontline Employee Evaluation Forms

Page 42 GAO-02-804 Performance Management Systems
Appendix IV: Examples of Evaluations

Addressing Group Managers’ Responsibilities

Page 43 GAO-02-804 Performance Management Systems

This appendix provides examples of critical job responsibilities,

supporting behaviors, and statements from narratives that we found in our

review of the enforcement group managers’ evaluations.

Table 11: Examples of Critical Job Responsibilities, Supporting Behaviors, and Evaluation Narratives for Group Managers

Critical job

responsibility Key supporting behavior Behavior described in evaluation narrative

Leadership Uses sound judgment to make effective and

timely decisions.

“As a leader, you have demonstrated sound judgment in

your decisions organizing the second group offer.”

Develops, prioritizes and aligns strategies,

objectives and goals, taking into account key

influences on organizational performance.

“He led his employees to develop realistic personal and

group goals that easily meshed into the needs of the

service and the general public.”

Customer satisfaction Constantly listens to customers, analyzes their

feedback to identify their needs and

expectations, and acts continuously to improve

products and services.

“You take the time to speak with the taxpayers and their

representative about their concerns and try to

accommodate their needs.”

Ensures that employees do the same (as

immediately above) and that they are prompt,

fair, and responsive to the circumstances of

individual customers to the extent permitted by

law and regulation.

“He insists his employees take the same approach with

their customers and that they address each case with the

rights of the taxpayers up front.”

Employee satisfaction Effectively uses ongoing feedback, coaching,

and timely evaluations of performance to

promote cooperation, teamwork, knowledge/skill

sharing and goal accomplishment.

“Throughout the year you provided employees with

regular, specific performance feedback geared to improve

and enhance performance, in both verbal and written

form.”

Acts on employee concerns, values, ideas and

perspectives of people from diverse

backgrounds.

“He heartily endorsed Survey 2000 participation and was

rewarded by increased levels of employee satisfaction as

reflected in survey results.”

Business results Develops and executes plans to achieve

organizational goals, leveraging available

resources (e.g., human, financial, etc.) to

maximize efficiency and produce high-quality

results.

“In addition to your responsibilities to the offer program,

you have ensured your employees have supported the

walk-in program when needed.”

Learns about current and emerging

issues/developments in own field of expertise

and applies knowledge to make technically

sound operational decisions.

“Because of his recognized technical expertise, he took

an active part in training during the year and his

employees and peers relied upon him to clarify, explain,

and reinforce complicated or newly enacted tax law.”

Equal employment

opportunity

Promptly responds to allegations of

discrimination and/or harassment and initiates

appropriate action to address the situation.

“You are also sensitive to the work environment and bring

any potential problems or issues to the attention of the

appropriate people.”

Supports staff participation in special emphasis

programs.

“He makes sure that his employees have the opportunity

to participate in EEO and diversity programs, and

demonstrates his commitment by attending himself.”

Appendix IV: Examples of Evaluations

Addressing Group Managers’ Responsibilities
Appendix V: Distribution of Raters’

Comments for Each Supporting Behavior

Page 44 GAO-02-804 Performance Management Systems

This appendix provides information on the distribution of raters’

comments among the behaviors supporting each critical job responsibility.

The supporting behaviors listed for each critical job responsibility are

taken directly from IRS’s evaluation form, shown in appendix III.

Table 12: Distribution of Raters’ Comments

Percentage of managers evaluated

on each supporting behavior

Critical job responsibilities and supporting behaviors FY 2000 FY 2001

Customer satisfaction

Communicates to employees the importance of customer focus as a critical component

of IRS’ mission. 25 19

Constantly listens to customers, analyzes their feedback to identify their needs and

expectations, and acts to continuously improve products and services. 46 56

Insures employees do the same (as immediately above) and that they are prompt,

professional, fair, and responsive to the circumstances of individual customers to the

extent permitted by law and regulation. 47 47

Identifies and utilizes policies, and economic, political and social trends in an effort to

improve organizational performance. 1 0

Builds strong alliances, involving stakeholders (for example NTEU, internal customers,

suppliers, etc.) in making decisions, and gaining cooperation to achieve mutually

satisfying solutions. 37 55

Initiates actions and manages risks to develop new products and services within or

outside the organization. 9 7

Shares innovations with others. 0 1

Business results

Pursues business excellence through effective process management and the application

of balanced measures. 78 87

Develops and executes plans to achieve organizational goals, leveraging available

resources (human, financial, etc.) to maximize efficiency and produce high-quality

results. 64 68

Identifies and analyzes problems to resolve individual and organizational issues in

accordance with law, regulation, and IRS policy. 18 15

Continuously seeks to improve business processes, sharing those efforts with other

units to better overall IRS performance. 21 25

Takes steps to prevent waste, fraud and abuse and instill public trust. 3 7

Learns about current and emerging issues/developments in own field of expertise and

applies knowledge to make technically sound operational decisions. 3 2

Understands and uses organizational realities, networks and accepted practices to

achieve desired results. 2 3

Employee satisfaction

Ensures that a safe, healthy work environment is maintained. 8 16

Demonstrates importance of employee satisfaction in accomplishing IRS’ mission. 7 2

Motivates employees to achieve high performance through empathetic, open and honest

communication, by involving them in decision making, and ensuring that they have the

tools and training to perform their jobs. 82 85

Appendix V: Distribution of Raters’

Comments for Each Supporting Behavior
Appendix V: Distribution of Raters’

Comments for Each Supporting Behavior

Page 45 GAO-02-804 Performance Management Systems

Percentage of managers evaluated

on each supporting behavior

Critical job responsibilities and supporting behaviors FY 2000 FY 2001

Effectively uses ongoing feedback, coaching, and timely evaluations of performance to

promote cooperation, teamwork, knowledge/skill sharing and goal accomplishment. 78 81

Develops and recognizes employees so that they realize their full potential.

62 80

Acts on employee concerns, values, ideas and perspectives of people from diverse

backgrounds. 65 63

Ensures all employees are treated in a fair and equitable manner. 11 12

Creates an environment for continuous learning, pursuing development opportunities for

self and others, with the intent to increase individual and organizational effectiveness. 38 38

Leadership

Demonstrates integrity and the highest ethical standards of public service. 11 7

Develops, prioritizes and aligns strategies, objectives and goals, taking into account key

influences on organizational performance. 10 5

Successfully leads organizational change, effectively communicating IRS’ mission, core

values, and strategic goals to employees and other stakeholders and responding

creatively to changing circumstances. 52 57

Creates and sustains a positive workplace that inspires others to support IRS’ mission

and goals. 15 12

Shows a commitment to public service and citizenship 3 3

Uses sound judgment to make effective and timely decisions. 6 3

Equal employment opportunity

Takes steps to implement EEO and affirmative employment goals established by IRS. 32 28

Supports staff participation in special emphasis programs. 31 34

Promptly responds to allegations of discrimination and/or harassment and initiates

appropriate action to address situation. 7 5

Cooperates with EEO counselors, investigators, and other officials responsible for

conducting inquires into EEO complaints. 8 9

Assigns work and makes employment decisions in areas such as hiring, promotion,

training and developmental assignments without regard to sex, race, color, national

origin, religion, age, disability, sexual orientation or prior participation in EEO process. 26 25

Monitors work environment to prevent instances of prohibited discrimination and/or

harassment. 18 14

Note: Percentages are rounded to the nearest percent.

Source: GAO analysis of fiscal year 2000 and 2001 enforcement group managers’ written

performance evaluations.
Appendix VI : Examples of Group Managers’

Commitments

Page 46 GAO-02-804 Performance Management Systems

This appendix provides examples of commitments written by enforcement

group managers in our sample. We found that almost all commitments met

IRS’s criteria of being clear and achievable, but about 85 percent did not

meet the criteria of being specific, outcome or output oriented, or easily

monitored. The examples of commitments are grouped into two

categories, those that were specific, outcome or output oriented, and

easily monitored, and those that were not.

We judged that the following commitments were specific, outcome or

output oriented, or easily monitored.

“Use of Balanced Measures Matrix will be demonstrated on one case or situation each

quarter.”

“I will meet on a quarterly basis with the Office Assistant to discuss issues relevant to his

job performance.”

“I will have monthly group meetings to discuss case issues, Survey 99 updates, RRA

updates, FY 2000 Operating Plan activities, and group concerns.”

“I will review and update IDPs (Individual Development Plans) on a semi-annual basis to

monitor accomplishments and modify the existing development plans.”

“I will give a presentation to Group __ and/or Branch II managers about what I have

learned about organizing.”

“By 03/31/2000 I will conduct refresher training/briefings for my employees in Group __ on

TFRP (Trust Fund Recovery Penalty) procedures.”

“I will continue to conduct monthly meetings with the employees to communicate changes

and any impact.”

“I will conduct quarterly reviews of casework focusing on IRM (Internal Revenue Manual)

requirements and CQMS (Collection Quality Measurement System) measures to ensure

timely closing of cases.”

“To improve employee satisfaction, I will keep employees informed of their performance

by preparing timely mid-year and annual appraisals supported with sufficient recordation.”

“Walk-in customer service will be provided at four outreach sites; Sioux City, IA, Storm

Lake, IA, O’Neill, NE and Columbus, NE. These sites will be staffed for one day every other

week from January through April 2000.”

Appendix VI : Examples of Group Managers’

Commitments

Commitments that

Were Specific,

Outcome or Output

Oriented, or Easily

Monitored
Appendix VI : Examples of Group Managers’

Commitments

Page 47 GAO-02-804 Performance Management Systems

“To improve Employee Satisfaction, I will ensure I provide timely and constructive

feedback to employees. I will do sample case reviews semimonthly and quarterly bundle

reviews and/or other written recordation concerning individual performance and timely

completion of mid-year and annual appraisals.”

The following commitments were determined not to be specific, outcome

or output oriented, or easily monitored.

“Support and develop employees in a manner which encourages participation by all.”

“Provide leadership to my group in achieving organizational goals by the use of effective

group meetings, effective review and evaluation practices, and the Individual Development

process.”

“I will work with the municipal governments within the State. Relationships built here will

continue to homogenize State employment tax administration efforts.”

“I will provide leadership by increased managerial involvement in my employee’s

casework.”

“I will use required group meetings to provide guidance on relevant issues.”

“I will support the partnership with NTEU [National Treasury Employees Union] so that

positive employee relations are maintained.”

“The identification and development of fraud cases will be encouraged in my work group

by creating an environment of fraud awareness.”

“Provide leadership and direction to the field staff to regain appropriate levels of

compliance on a case by case basis by conducting reviews, timely entity queries and follow

ups.”

“I will support the Field Branches as needed.”

“I will ensure sufficient resources and managerial attention to support the walk-in program

in my post of duty.”

“I will continue to have open and honest communications with all employees.”

“I will continue to be supportive of employees’ efforts to advance their careers.”

“I will lead Field Group #__ through the organizational changes demanded by the

modernization of the IRS by providing leadership, planning, and support to effectively

Commitments that

Were Not Specific,

Outcome or Output

Oriented, or Easily

Monitored
Appendix VI : Examples of Group Managers’

Commitments

Page 48 GAO-02-804 Performance Management Systems

transition the work and human resources of the current district structure to the new

operating units.”
Appendix VII: Comments from the Internal

Revenue Service

Page 49 GAO-02-804 Performance Management Systems

Appendix VII: Comments from the Internal

Revenue Service
Appendix VII: Comments from the Internal

Revenue Service

Page 50 GAO-02-804 Performance Management Systems
Appendix VIII: GAO Contacts and Staff

Acknowledgments

Page 51 GAO-02-804 Performance Management Systems

Michael Brostek (202) 512-9039

Ralph Block (415) 904-2150

Jonda Van Pelt (415) 904-2186

In addition to those named above, Michelle Bowsky, Maya Chakko,

Elizabeth Fan, Tre Forlano, Diana Hu, Ann Lee, and Samuel Scrutchins

made key contributions to this report.

Appendix VIII: GAO Contacts and Staff

Acknowledgments

GAO Contacts

Acknowledgments

(440028)

The General Accounting Office, the investigative arm of Congress, exists to

support Congress in meeting its constitutional responsibilities and to help

improve the performance and accountability of the federal government for the

American people. GAO examines the use of public funds; evaluates federal

programs and policies; and provides analyses, recommendations, and other

assistance to help Congress make informed oversight, policy, and funding

decisions. GAO’s commitment to good government is reflected in its core values

of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older

products. The Web site features a search engine to help you locate documents

using key words and phrases. You can print these documents in their entirety,

including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site

daily. The list contains links to the full-text document files. To have GAO e-mail

this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily

E-mail alert for newly released products” under the GAO Reports heading.

The first copy of each printed report is free. Additional copies are $2 each. A

check or money order should be made out to the Superintendent of Documents.

GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a

single address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000

TDD: (202) 512-2537

Fax: (202) 512-6061

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G Street NW, Room 7149

Washington, D.C. 20548

GAO’s Mission

Obtaining Copies of

GAO Reports and

Testimony

Order by Mail or Phone

To Report Fraud,

Waste, and Abuse in

Federal Programs

Public Affairs