February 2001

GAO-01-234

IRS

MODERNIZATION

IRS Should Enhance

Its Performance

Management System

Report to Congressional Requesters

United States General Accounting Office

GAO
Page i GAO-01-234 IRS' Performance Management System

Letter 1

Results in Brief 3

Background 5

IRS Has Developed Goals, Objectives, and Performance Measures 6

IRS Has Made Progress But Could Further Enhance Its

Performance Management System 12

Conclusions 18

Recommendations for Executive Action 19

Agency Comments and Our Evaluation 19

Appendix I Objectives, Scope, and Methodology 22

Appendix II Assessment of the Characteristics of Operating

 Division Strategic Goals 25

Appendix III Examples of Action Items Developed by IRS

 Organizational Units 27

Specific, Measurable, and Outcome- or Output-Oriented Action

Items 27

Unspecific, Unmeasurable, and Not Outcome- or Output-Oriented

Action Items 28

Appendix IV Comments From the Internal Revenue Service 29

Appendix V GAO Contacts and Staff Acknowledgments 32

Figures

Figure 1: Recommended Enhancements to IRS’ Performance

Management System 4

Figure 2: IRS’ Agencywide Strategic Goals, Objectives, and

Performance Measures 8

Figure 3: W&I Strategic Goal and Operating Objectives

Corresponding to the Agencywide Goal of “Service to

Each Taxpayer” 11

Contents
Page 1 GAO-01-234 IRS' Performance Management System

February 23, 2001

The Honorable Bill Thomas

Chairman, Committee on Ways and Means

House of Representatives

The Honorable Amo Houghton

Chairman, Subcommittee on Oversight

Committee on Ways and Means

House of Representatives

In response to congressional and public criticism that it was emphasizing

revenue collection over taxpayer service, the Internal Revenue Service

(IRS) initiated a major modernization effort to make it more responsive to

taxpayers’ needs. A key part of this modernization effort is developing a

new performance management system that lays out the agency’s goals,

objectives, and performance measures and is intended to balance taxpayer

service and enforcement activities while fostering a productive work

environment. The Commissioner of Internal Revenue considers holding

managers in IRS’ operating divisions1

 accountable for achieving these

goals and objectives critical to accomplishing IRS’ mission. To an

important extent, the performance management system is expected to

create the incentives that influence the behavior of IRS employees and

determine whether IRS’ day-to-day activities support the goals of

modernization.

Recognizing that revamping a performance management system is a

difficult, long-term effort, you asked us to provide information on how IRS

measures its performance. Specifically, as agreed with your office, our

objectives were to (1) describe the status of IRS’ efforts to develop

agencywide and operating division goals, objectives, and performance

measures that support IRS’ mission and (2) assess these efforts and

identify actions, if any, IRS could take to enhance its performance

management system.

 1

The operating divisions are (1) Wage and Investment, serving individual taxpayers; (2)

Small Business and Self-Employed, serving fully or partially self-employed individuals and

small businesses with assets under $5 million; (3) Large and Mid-Size Business, serving

businesses with assets over $5 million; and (4) Tax Exempt and Government Entities,

serving pension plans, exempt organizations, and governments.

United States General Accounting Office

Washington, DC 20548
Page 2 GAO-01-234 IRS' Performance Management System

Our assessment is not a review of IRS’ compliance with the requirements

of the Government Performance and Results Act (GPRA) of 1993.2

Although IRS expects to use certain features of its performance

management system to satisfy GPRA requirements, IRS developed the

system primarily to meet its internal management needs. As a result, our

assessment focused on whether the system met IRS’ unique management

needs. Our assessment is based on characteristics that we found

associated with goals, objectives, and performance measures in the

performance management literature, including GPRA.3

 In particular,

• goals, which are broad statements of desired outcomes, should

• reflect the agency’s priorities,

• provide a clear direction for future agency action,

• identify what impact or outcome will result from an agency’s work, and

• form the basis for formulating clear objectives;

• objectives, which are targets that describe the end results a service or

program is expected to accomplish in a given period of time to meet goals,

generally should be

• clearly stated,

• specific,

• measurable,

• outcome or output oriented, and

• consistent with the agency’s mission and goals; and

• performance measures or indicators, which are often selected after goals

and objectives are developed, should be

• logically and directly related to the goals and objectives,

• focus on expected outcomes or outputs, and

• capture information relevant to the goals and objectives.

 2

P.L. 103-62. GPRA requires agencies to set goals, measure performance, and report on

their accomplishments in their annual performance plans and annual program performance

reports.

3

The publications we reviewed included: OMB Circular A-11, Part 2, Preparation and

Submission of Strategic Plans, Annual Performance Plans, and Annual Program

Performance Reports (July 2000); Agencies’ Strategic Plans Under GPRA: Key Questions

to Facilitate Congressional Review (GAO/GGD-10.1.16, May 1997); The Results Act: An

Evaluators Guide to Assessing Agency Annual Performance Plans (GAO/GGD-10.1.20,

April 1998); Center for Accountability and Performance, Performance Management:

Concepts and Techniques, 2nd ed. (1999); Robert S. Kaplan and David P. Norton, The

Balanced Scorecard: Translating Strategy Into Action (Boston, Mass.: Harvard Business

School Press, 1996); and National Partnership for Reinventing Government, Balancing

Measures: Best Practices in Performance Management (August 1999).
Page 3 GAO-01-234 IRS' Performance Management System

Our work was done between August 1999 and December 2000 in

accordance with generally accepted government auditing standards. See

appendix I for a detailed description of our scope and methodology.

IRS has established three agencywide strategic goals and the

corresponding strategic objectives and has identified the strategic

measures it plans to use for measuring progress toward meeting the

strategic objectives. Below the agencywide level, the operating divisions

developed their own strategic goals that were to align with the agencywide

goals. The operating divisions also developed operational objectives to

meet the division-level strategic goals and operating measures to measure

progress toward some of those objectives. In addition, managers

developed action items that detailed the steps they planned to take to

achieve operational objectives.

IRS has made progress in the challenging task of revamping its

performance management system. In addition to developing goals,

objectives, and performance measures, IRS has implemented a new

strategic planning and budgeting process intended to integrate the use of

performance measures into agency management.

However, as shown in figure 1, we identified several issues that IRS could

address at both the agencywide and division levels to enhance its

performance management system.

Results in Brief
Page 4 GAO-01-234 IRS' Performance Management System

Figure 1: Recommended Enhancements to IRS’ Performance Management System

Source: GAO.

At the agencywide level, the intent of the objective of “increase fairness of

compliance” is not clear, and performance measures do not capture key

information needed to gauge progress toward achieving three strategic

objectives. At the division level, three of 15 goals do not (1) clearly

articulate the divisions’ future direction, (2) indicate the expected impact

of achieving the goal, or (3) provide a clear basis for establishing

objectives. Also, operating objectives are not specific, measurable, or in all

but two cases, outcome or output oriented; and a large number of

operating measures and indicators are not directly linked to objectives. In

addition, because IRS was introducing its new performance management

concepts, many of the action items developed by managers to meet the

fiscal year 2000 operating objectives were not specific, measurable, and

outcome or output oriented.

IRS recognizes that revamping its performance management system will

be an ongoing process because the inherent difficulties in implementing

Agencywide

level

Operating division

level

Improve the linkage between

2 strategic measures and

related objectives

Develop strategic measure

for one objective

Clarify intent of "increase 

fairness of compliance"

Strategic goals Strategic objectives Strategic measures

Operating objectives Operating measures

Develop objectives that are

specific, measurable, and

outcome oriented

Link operating measures

more directly to operating

objectives

Enhancement Enhancement

Enhancement

Enhancements

Strategic goals

Revise 3 goals to articulate

agency's future direction,

indicate expected impact

of achieving the goal, and 

provide a clear basis for

developing objectives

Enhancement

A
Page 5 GAO-01-234 IRS' Performance Management System

such a major change will require continuing adjustments. We are making

recommendations to the Commissioner of Internal Revenue on how IRS

can enhance its performance management system at both the agencywide

and division levels. In commenting on a draft of this report, the

Commissioner agreed with our recommendations at the division level.

However, the Commissioner raised some concerns with our

recommendations to clarify one strategic objective and add a measure for

the strategic objectives concerning customer service.

In response to congressional concerns that IRS was overemphasizing

revenue collection at the expense of fairness to taxpayers, the

Commissioner of Internal Revenue is seeking to refocus the agency toward

an approach that promotes voluntary compliance with the tax laws

through better taxpayer service while concentrating enforcement

resources on those taxpayers who are deliberately and willfully

noncompliant.

IRS realized it needed to revamp its performance management system to

reflect this new focus within the constraints of two legislative

requirements on how it sets goals and measures performance. First, the

Government Performance and Results Act of 1993 requires federal

agencies to establish a hierarchy of goals, objectives, and performance

measures applicable to various organizational units within their agencies.

Second, the IRS Restructuring and Reform Act of 19984

 directed IRS to

establish individual, group, and organizational goals and objectives, but

prohibited IRS from using tax enforcement results5

 to evaluate any IRS

employee or to impose or suggest production quotas or goals.

According to the performance management literature we reviewed, the

purpose of organizational goals, objectives, and measures is to enable an

organization to monitor how well it is achieving its mission. Strategic goals

provide direction to an organization’s work, services, programs, and

activities and generally describe a desired outcome for the organization

and its programs. To achieve goals, organizations develop objectives.

Objectives are more specific than goals and are measurable targets that

 4

P.L. 105-206.

5

Tax enforcement results are outcomes produced when IRS employees exercise their

judgment in recommending or determining whether or how IRS should pursue enforcement

of the tax laws. Examples of tax enforcement results are the number of liens, levies, and

seizures issued; the dollar amount assessed; the dollar amount collected; and the number

of fraud referrals.

Background
Page 6 GAO-01-234 IRS' Performance Management System

describe the end results a service or program is expected to accomplish in

a given time period. Finally, organizations develop performance measures

to monitor progress toward their objectives and ultimately their goals.

Performance measures should be quantifiable measures, such as

outcomes, outputs, efficiency, or cost-effectiveness, and should be useful

in determining whether the organization is achieving its goals.

One framework for developing performance measures is the “balanced

measures approach,” which was adopted by IRS. This approach originated

in the private sector when companies concluded that overemphasizing

short-term financial objectives could actually be detrimental to their longterm success. Instead, these companies began focusing on both financial

and nonfinancial aspects of performance, including areas such as product

convenience and quality, that engender the kind of customer satisfaction

and loyalty necessary for success in the long run.

At both the agencywide and operating division levels, IRS has established

goals and the corresponding objectives and has identified the performance

measures it plans to use for measuring progress toward meeting the goals

and objectives.

In 1998, the Commissioner of Internal Revenue replaced IRS’ old mission

statement with a new one: “Provide America’s taxpayers top quality

service by helping them understand and meet their tax responsibilities and

by applying the tax law with integrity and fairness to all.”

Using the balanced measures approach of emphasizing multiple goals, IRS

established three agencywide strategic goals in 1999 to reflect its new

approach to tax administration. The goals are

• “service to each taxpayer,”

• “service to all taxpayers,” and

• “productivity through a quality work environment.”

IRS chose the balanced measures framework for its new performance

management system because senior executives wanted to ensure that

managers and employees considered taxpayer needs in addition to

revenue collection when making decisions. IRS also established two or

more strategic objectives for each goal.

IRS Has Developed

Goals, Objectives, and

Performance

Measures

IRS Established New

Agencywide Strategic

Goals, Objectives, and

Measures to Reflect Its

New Mission Statement
Page 7 GAO-01-234 IRS' Performance Management System

In 1999, IRS developed and began using the results of its employee

satisfaction survey as a strategic measure for the objective of “increase

employee job satisfaction.” In 2000, IRS identified and began developing

other strategic measures, such as taxpayer burden as a measure for

“making filing easier.” These measures are in various stages of

development. For example, IRS expects to have taxpayer burden estimates

for the Wage and Investment Division in fiscal year 2001 and anticipates

developing estimates for the Small Business and Self-Employed Division in

fiscal year 2002. IRS is not sure when data on productivity and workload

will be available. The strategic goals, objectives, and performance

measures are shown in figure 2.
Page 8 GAO-01-234 IRS' Performance Management System

Figure 2: IRS’ Agencywide Strategic Goals, Objectives, and Performance Measures

Objective Objective Objective

Provide prompt, professional,

helpful treatment to

taxpayers in cases where

additional taxes may be due

Make filing easier Provide first quality service

to each taxpayer needing

help with his or her return

or account

Goal

Performance measure

Strategic

goals

Strategic

performance

measures

Strategic

objectives

Service to

each taxpayer

Taxpayer burden (time and

dollar costs)

Performance measure

Customer satisfaction/

dissatisfaction

Performance measure

Customer satisfaction/

dissatisfaction

a
Page 9 GAO-01-234 IRS' Performance Management System

Source: IRS.

Performance measure Performance measure Performance measure

Objective Objective Objective Objective

Increase employee

job satisfaction

Increase fairness of

compliance

Increase overall

compliance

Hold agency employment

stable while economy

grows and service improves

Payment compliance Employee satisfaction

surveys

Productivity/workload

index

Performance measure

Goal Goal

Service to

all taxpayers

Productivity through

a quality work

environment

None identified

Performance measure

Performance measure

Filing compliance

Reporting compliance

a
Page 10 GAO-01-234 IRS' Performance Management System

In 2000, IRS implemented a new strategic planning and budgeting process

that provides the framework for developing goals, objectives, and

measures at the operating division level. Under this process, the operating

divisions developed program plans that identified the division’s strategic

goals, which were to correspond to each of the agencywide strategic goals.

Such division-level strategic goals were to be tailored to the specific

characteristics and needs of individual taxpayer segments served by the

division. The program plans also identified the operating objectives, which

IRS calls operating priorities, intended to accomplish the division’s

strategic goals.

For example, the Wage and Investment Division (W&I) concluded that

most of the taxpayers it deals with

• prepare their own tax returns;

• are highly compliant, with noncompliance often resulting from taxpayer

confusion;

• contact IRS only once a year; and

• receive refunds.

W&I considered these characteristics when developing its strategic goals

and operational objectives. Figure 3 illustrates the linkage between the

agencywide goal of “service to each taxpayer” and one of W&I’s strategic

goals and associated operating objectives, which addresses improving

taxpayer convenience and communication.

The program plans also identify three types of operating measures or

indicators that the operating divisions will use to measure their

performance. They are

• balanced measures, which includes customer satisfaction to align with

“service to each taxpayer,” business results (quality and quantity) to align

with “service to all taxpayers,” and employee satisfaction to align with

“productivity through a quality work environment,” to measure the overall

health of the organizational unit;

• workload indicators, such as the number of returns processed, to project

the expected level of activity for the organizational unit, identify resource

needs, and justify resource requests; and

• performance indicators, such as the percentage of individual refunds

issued within 40 days of the date of the signed tax return, to measure how

well the organizational units are achieving program objectives.

At the time of our review, each division had developed these measures for

at least some of their organizational units.

IRS’ Strategic Planning and

Budgeting Process

Provides the Framework

for Operating Divisions’

Goals, Objectives, and

Measures
Page 11 GAO-01-234 IRS' Performance Management System

Figure 3: W&I Strategic Goal and Operating Objectives Corresponding to the

Agencywide Goal of “Service to Each Taxpayer”

Source: IRS.

As the final step in this process, IRS requires managers to develop action

items that detail the steps they will take to implement the operational

objectives. IRS’ fiscal year 2000 Operations Plan established six

agencywide priorities to promote consistency among action items. The six

priorities were to (1) implement balanced measures, (2) improve customer

satisfaction, (3) improve employee satisfaction, (4) improve the quality of

work, (5) identify and work with taxpayer segments likely to become

noncompliant, and (6) improve compliance by emphasizing nonenforcement activities.

At each organizational level down to the group level, managers and their

work groups developed action items that addressed each of these six

priorities.

Agencywide

level

Operating division

level

Service to each taxpayer

Strategic goal

Strategic goal

Meet taxpayer demands for

better traditional assistance

services (access and quality):

phone, correspondence, and

in-person

Agencywide level

Operating objectives

Upgrade field assistance

Organize, train, and

 specialize customer

 service representatives

Add trained phone assistors

Leverage partnerships

Research customer segment

 needs

Expand alternative language

 services

Division level

Division level

a
Page 12 GAO-01-234 IRS' Performance Management System

IRS has made progress in the challenging task of revamping its

performance management system from one focused on revenue collection

to one that better balances taxpayer satisfaction, tax law compliance, and

employee satisfaction. However, IRS could make enhancements to its

performance management system by revising some goals, objectives, and

measures at both the agencywide and operating division level.

IRS intends to make its new performance management system the routine

way the agency is managed, which IRS believes will put it in a better

position to manage toward its strategic goals and objectives. Specifically,

IRS intends to use its performance measures to monitor and evaluate its

performance as part of its new planning and budgeting process and then

use the results of this review to reassess and revise its objectives and

goals. In addition, IRS considers the action items developed to implement

operational objectives a primary mechanism for getting employees at all

levels of the agency actively engaged in carrying out IRS’ goals and

mission. Managers and their work groups are expected to develop action

items in response to problems they define and the causes and possible

solutions they identify from their analysis of the operating data. In this

way, IRS hopes to establish a “clear line of sight” between employees’ dayto-day activities and IRS’ overall mission and strategic goals.

Because IRS has not yet completed a full cycle of its new planning and

budgeting process, it is too soon to evaluate the impact of this resultsoriented process on performance. Nevertheless, the effort IRS is making to

integrate the use of performance measures into its day-to-day management

represents progress in using performance management to support the new

mission of the agency.

At the agencywide level, we identified three issues that IRS could address

to enhance the usefulness of its performance management system. The

issues include the (1) clarity of one strategic goal, (2) clarity of one

strategic objective, and (3) linkage between certain strategic measures and

strategic objectives.

With respect to the first issue, two of IRS’ three strategic goals—”service

to each taxpayer” and “productivity through a quality work

environment”—give a relatively clear indication of IRS’ priorities,

direction, and the potential impact its policies can be expected to have on

individual taxpayers and agency employees. However, IRS’ goal of “service

IRS Has Made

Progress But Could

Further Enhance Its

Performance

Management System

IRS’ Efforts to Integrate

Performance Measures

Into Management

Represent Progress in

Supporting Its Mission

Issues to Be Addressed at

the Agencywide Level
Page 13 GAO-01-234 IRS' Performance Management System

to all taxpayers” did not clearly convey its intent, which is to apply the tax

law with integrity and fairness so that taxpayers that do not comply with

the tax laws are not allowed to place a burden on those that do. During the

course of our review, we brought this matter to the attention of top IRS

executives, including the Commissioner of Internal Revenue. IRS agreed

that the goal did not clearly convey its intent. In February 2001, just before

we issued this report, IRS released its strategic plan for fiscal years 2000-

05, which showed the goal statement was modified to read “service to all

taxpayers through fair and uniform application of the law.” This

modification of the goal statement makes clearer that this goal covers how

IRS applies the tax laws to all taxpayers.

With respect to the second issue, IRS’ strategic objectives are consistent

with its mission and with one exception—the objective of “increasing

fairness of compliance”—are clearly stated, specific, measurable, and

outcome or output oriented. As IRS recognizes, this fairness objective

could be subject to various interpretations and difficult to measure. For

example, fairness could mean each individual taxpayer is equally likely to

be audited regardless of such characteristics as occupation or income.

Fairness could also mean that IRS would target enforcement activities

toward pockets of noncompliance so that compliant taxpayers are less

likely to be burdened by audit activities. Rewording this objective to more

clearly express the desired result could help managers better focus their

efforts to achieve this objective.

With respect to the third issue, for six of its seven strategic objectives, IRS

has developed performance measures that are directly related to the

objectives, focus on expected outcomes or outputs, and capture relevant

information. However, for two of these six objectives, IRS’ performance

measures do not capture all the information that is relevant to gauging

progress toward achieving the objectives. IRS has relied strictly on

taxpayer perceptions as measured by customer satisfaction surveys or

complaints to measure how well it is meeting its two strategic objectives

of “providing quality service to taxpayers who need help with their returns

or accounts” and “providing quality service to those who may owe

additional taxes.” Although customer satisfaction surveys and complaints

provide relevant information, they do not address an important

component of IRS’ service—whether the taxpayer’s problem was properly

handled within the context of the tax law. Elements of IRS’ existing quality

management systems might provide the basis for developing such a

measure. The quality management systems currently include assessments

such as whether IRS employees properly computed tax liabilities or

selected the most appropriate payment method for collecting past-due

taxes. In addition, IRS did not have a performance measure for its strategic
Page 14 GAO-01-234 IRS' Performance Management System

objective of “increase fairness of compliance.” Without a performance

measure that is logically and directly related to this objective, IRS cannot

determine its progress toward meeting the objective.

At the operating division level, we identified three issues that IRS could

address to enhance the usefulness of its performance management system.

The issues include (1) the clarity with which goals articulate the divisions’

future direction, indicate the expected impact of achieving the goal, and

provide a clear basis for establishing objectives; (2) whether objectives,

which IRS calls operating priorities, are specific, measurable, and outcome

or output oriented; and (3) the linkage between operating measures and

operational objectives.

With respect to the first issue, our review of 15 division-level strategic

goals for three of four divisions6

 found that they reflected IRS’ agencywide

goals and priorities. However, three of the goals did not

• articulate the divisions’ future direction,

• indicate the expected impact of achieving the goal, or

• provide a clear basis for establishing objectives.

These goals are (1) “address underreporting, nonfiling, and abuse of trusts

and pass-throughs,” (2)”build a tax administration to effectively deal with

globalization,” and (3) “implement comprehensive issue management

strategy.” For example, the strategic goal to “address underreporting,

nonfiling, and abuse of trusts and pass-throughs” is set within the broad

context of increasing overall compliance. However, it is vague and does

not articulate the division’s future direction or a desired impact. And since

the expected impact is not indicated, the goal does not provide a good

basis for formulating objectives. In contrast, the strategic goal “achieve

early, accurate resolution of taxpayer accounts” focuses managers’

attention on the impact that is expected and provides a basis for setting

objectives to accomplish the goal. Appendix II provides a more detailed

analysis of the shortcomings of these the three goals.

With respect to the second issue, our review of division-level operational

objectives found that they were clearly stated and consistent with divisionlevel goals and IRS’ overall goals. However, none of the 72 objectives were

 6

We reviewed the strategic goals for three of IRS’ four operating divisions—Wage and

Investment, Small Business and Self-Employed, and Large and Mid-Size Business.

Issues to Be Addressed at

the Division Level
Page 15 GAO-01-234 IRS' Performance Management System

stated in terms that were specific and measurable—that is, they did not

include a time period, a numeric target, or a means to measure the

objective. Also, 70 of 72 were not outcome or output statements because

they did not include an expected result, a program impact, or a time

period.

We recognize that formulating objectives that describe the desired end

result can be a challenging undertaking; however, objectives that are not

clearly articulated are less useful to managers responsible for

accomplishing division objectives. For instance, the objective “consider

targeted notices and guidance” provides no indication of

• which notices or guidance are of interest,

• what is to be done with them,

• how to measure whether the objective has been met, or

• the expected impact of accomplishing this objective.

In contrast, the objective “centralize processing of OICs [offers-incompromise] to improve quality, timeliness, and efficiency” indicates what

is to be done and the expected impact of meeting this objective.

With respect to the third issue, our review of the operating division’s

program plans found that the operating divisions have identified balanced

measures, workload indicators, and performance indicators for many of

their functional programs.7

 However, for 63 of the 72 objectives, the

operating divisions had not linked operating measures and indicators

directly to the objective. Without such direct linkages, operating divisions

will not be able to measure the extent to which they are meeting their

objectives.

For example, the W&I program plan identified expanding Spanish and

second-language services as an operational objective. However, W&I has

not developed measures specifically directed toward measuring the impact

of its actions, such as the customer satisfaction level of Spanish-speaking

taxpayers, or the extent to which they achieved this objective, such as

whether W&I increased the number of Spanish-speaking tax auditors. In

contrast, W&I did identify specific performance measures for its objective

of leveraging partnerships with volunteer and other organizations to

 7

The work of each operating division is performed through functional programs aligned

with one of three general service areas: prefiling services, filing and account services, and

compliance services. W&I’s Taxpayer Assistance program, for example, is a prefiling

service.
Page 16 GAO-01-234 IRS' Performance Management System

deliver return preparation services. As measures of its success in meeting

this objective, W&I will look to (1) reducing the number of compliance

personnel supporting field assistance, (2) increasing the number of hours

volunteers provide walk-in assistance, and (3) increasing return

preparation services provided by partnering organizations.

In fiscal year 2000, IRS began requiring managers to develop plans each

year that identify the actions they intend to take to support their operating

objectives. We evaluated the action items contained in the plans using the

same characteristics we used to evaluate objectives. We found that, while

clearly stated and consistent with IRS’ mission, few action items were

specific, measurable, and outcome or output oriented. In addition, the

large number of action items reduced the likelihood that they would be

tracked and assessed. IRS recently issued guidance for developing

individual performance commitments that could be a model for better

guidance on action plans.

We received and reviewed 268 action plans from 32 of IRS’ 33 district

offices. Although the 6,398 fiscal year 2000 action items we reviewed were

clearly stated and consistent with IRS’ mission, we found that 91 percent

of them (5,847) were not specific, measurable, and outcome or output

oriented. As a result, IRS could find it difficult to hold managers

accountable for achieving their action items. The following examples

illustrate how the quality of the action items varied:

• “Use reports to identify improvement areas and develop actions to address

them.”

• “Branch will establish a quarterly review of 5 cases from each of four pure

general program groups. The five cases will be reviewed by a manager

from another group for adherence to the EQMS [Examination Quality

Management System] standards.”

The second action item is more specific, measurable, and outcome or

output oriented because it includes such information as who is

responsible for taking action, the frequency of the action, the number of

actions, and the anticipated outcome (adherence to quality standards).

Appendix III provides additional examples of action items.

IRS National Office officials were not surprised by our results. In fiscal

year 2000, their guidance steered managers away from developing specific

and measurable action items for at least two reasons. First, IRS lacked

statistically valid performance data, and therefore, IRS’ training course on

balanced performance measures instructed managers not to establish

In the First Year of

Implementation, IRS’

Managers Lacked

Adequate Guidance for

Developing Action Items
Page 17 GAO-01-234 IRS' Performance Management System

quantitative measures for their action items. Second, IRS felt the need to

introduce the balanced measures concept to managers as quickly as

possible, even though operating divisions did not develop specific goals

and objectives until after the action plans were due. As a result, the action

items that divisions provided to their organizational units as a framework

to follow were vague and nonspecific. For example, the Examination

operations plan included the following action items:

• “Ensure the cultural change and philosophy supporting the protection of

taxpayer rights has been embraced by the organization,”

• “Support the transition to the new operating divisions,” and

• “Revise the Quality Review Program to emphasize uniformity and

consistency.”

In addition, although IRS guidance instructed managers to prepare brief,

focused plans with a few areas of emphasis, we found that each manager’s

plan contained an average of 24 action items. This multitude of action

items could reduce the likelihood that managers would be focusing on

those activities that are critical to achieving the operating division’s goals

and objectives. Furthermore, such a large number of action items might

increase the difficulty of holding managers accountable for achieving their

action items.

IRS officials told us they are taking steps to improve the quality of the

action items. These steps include

• developing training on the balanced measures system that is to include a

module covering the skills needed to analyze data to better identify action

items,

• developing a handbook on action planning intended to help managers

analyze customer satisfaction data,

• issuing a four-page brochure intended to help familiarize both IRS

employees and other interested parties with the purpose of the customer

satisfaction survey and how the data should be used to improve IRS

operations, and

• ensuring that the District Offices of Research and Analysis provide the

operating divisions with more research assistance.

Although these initiatives may help managers identify the areas that

should be targeted by action items, they do not cover how to formulate

action items in specific, measurable, and outcome or output oriented

terms or how to limit the number of action items to the critical few. Such

guidance would have to be provided by the operating divisions since,
Page 18 GAO-01-234 IRS' Performance Management System

under the new strategic planning and budgeting process, they will be

responsible for implementing the strategic plans.

Recently issued guidance on developing individual performance

commitments, a component of the employee evaluation process, could

serve as a model for guidance on developing action items. Performance

commitments are intended to better link IRS’ performance appraisal

process to the balanced measures and set forth specific expected goals

and results tied directly to the organization’s strategic objectives. The

guidance identifies the key features of a well-constructed commitment as

• spelling out specifically what is to be accomplished, not general tasks,

duties, or responsibilities of an ongoing job,

• describing realistically the actions to be taken and the expected outcomes,

• maintaining a “line of sight” to the goals of the Service,

• including a clear timeframe for accomplishment,

• being achievable and capable of being monitored and assessed, and

• outlining required organizational resources or support.

The key features of a well-constructed commitment are similar to those

that would be key features of a good action item. Guidance laying out

similar features, along with examples of specific, measurable, and

outcome- or output-oriented action items, could help managers develop

action items that would in turn help employees carrying out IRS’ day-today operations to better link their activities to IRS’ goals and objectives.

IRS has made progress in the challenging task of revamping its

performance management system. The goals, objectives, and measures

that IRS has developed along with its new strategic planning and

budgeting process are intended to integrate results-oriented management

into IRS’ daily decision-making. However, as IRS officials acknowledge,

extensively revamping a performance management system is a complex

task that takes years to complete. We identified several opportunities to

enhance IRS’ performance management system, including clarifying goals

and objectives, improving the linkages between measures, objectives, and

goals, and developing fewer but more specific action items. Such

enhancements could increase managerial accountability and create

stronger incentives for frontline employees to achieve IRS’ goals and

objectives.

Conclusions
Page 19 GAO-01-234 IRS' Performance Management System

As IRS continues to refine its performance management system, we

recommend that the Commissioner of Internal Revenue take steps at both

the agencywide and division levels to enhance the system. At the

agencywide level, these steps are to

• clarify the strategic objective, “increase fairness of compliance,” to more

precisely express the desired result in ways that can be measured, and

• add a measure of quality—that is, whether the taxpayers’ problems are

correctly handled within the context of the law—to the agencywide

performance measures for the strategic objectives of providing quality

service to taxpayers who need help or who may owe additional taxes, and

provide a performance measure for the objective, “increase fairness of

compliance.”

At the division level, these steps are to

• clarify three strategic goals to better articulate IRS’ future direction,

indicate the expected impact of achieving the goal, and provide a clear

basis for establishing objectives;

• revise or develop operating objectives to ensure that they are specific,

measurable, and outcome or output oriented; and

• revise or develop operational performance measures to ensure that the

measures are directly linked to operational objectives.

In addition, we recommend that IRS provide better guidance to unit

managers on how to develop action items that are few enough to focus

employees’ attention and are specific, measurable, and outcome or output

oriented.

We requested comments on a draft of this report from IRS. We obtained

written comments in a February 13, 2001, letter from the Commissioner of

Internal Revenue (see app. IV). In his letter, the Commissioner

characterized our assessment of IRS’ implementation of its revamped

performance management system as fair and balanced. He said that he

believes IRS has made significant progress, but recognizes that IRS still

faces many challenges in establishing a lasting and effective system for

measuring agency performance. The Commissioner agreed with our

recommendations at the division level. However, the Commissioner raised

some concerns with our recommendations to clarify one strategic

objective and add a measure for the strategic objectives concerning

customer service.

Recommendations for

Executive Action

Agency Comments

and Our Evaluation
Page 20 GAO-01-234 IRS' Performance Management System

In commenting on our recommendation that IRS clarify its strategic

objective of “increasing fairness of compliance,” the Commissioner agreed

that it was ideal for objectives to be completely self-explanatory. However,

he said that he believed it was important to communicate to the public

that IRS valued fairness in conjunction with its compliance role. He said

that IRS intends to continue to elaborate on the intent of this objective in

its ongoing communications with taxpayers and IRS employees. Further,

the Commissioner stated that an aspect of IRS’ current work on the

balanced measures system is to better understand the taxpayers’ concept

of fairness and to develop suitable measures of fairness. We agree that

communicating to the public that IRS values fairness is important.

However, since IRS’ new performance management system is intended to

increase accountability, we continue to believe that IRS needs to clarify

this objective statement so that taxpayers know specifically what to

expect from IRS in the area of fairness, IRS managers can better focus

their efforts on achieving this objective, and Congress can gauge IRS’

progress in meeting this objective.

In commenting on our recommendation that IRS use a quality measure in

addition to a customer satisfaction measure for the strategic goal of

providing quality customer service, the Commissioner agreed conceptually

that it is relevant to determine whether taxpayers’ problems are properly

handled within the context of the law. The Commissioner focused his

comments on measuring customer satisfaction and expressed concern that

weighting the customer satisfaction score or otherwise combining it with a

quality score would diminish the customer view and devalue the

importance of taxpayers’ opinions. We agree that a customer satisfaction

score measures one very important component of providing quality service

to taxpayers. However, taxpayers may not know whether their case was

handled properly within the context of the law. Therefore, we continue to

believe that IRS needs to have a separate score to measure the component

of quality service that deals with whether IRS handles taxpayer problems

properly. It is not our intent that IRS attempt to combine these two

measures into one overall score of customer service, and we have

modified our recommendation to make this clearer.

The Commissioner agreed with our recommendations on improving IRS’

performance management system at the division level regarding the clarity

of goals, the specificity and measurability of objectives, and the linkage

between measures and objectives. He said that while the operating

divisions have already taken some steps in line with our report

recommendations, IRS will be addressing these issues on an ongoing basis

and that refinements will occur as IRS develops more experience with the
Page 21 GAO-01-234 IRS' Performance Management System

strategic planning and budgeting process. The Commissioner also agreed

with our recommendation to provide better guidance to unit managers on

how to develop action items and said IRS had already undertaken

corrective action in this area.

As agreed with your office, unless you publicly announce its contents

earlier, we plan no further distribution of this report until 30 days from the

date of this letter. At that time, we will send copies to the Ranking

Minority Member of the Subcommittee on Oversight; the Ranking Minority

Member of the Committee on Ways and Means; the Secretary of the

Treasury; and the Commissioner of Internal Revenue. We will also make

copies available to others on request.

Please contact me at (202) 512-9110 or Ralph Block at (415) 904-2150 if

you or your staff have any questions. Key contributors to this report are

acknowledged in appendix V.

Sincerely yours,

James R. White

Director, Tax Issues
Appendix I: Objectives, Scope, and

Methodology

Page 22 GAO-01-234 IRS' Performance Management System

Our objectives were to (1) describe the status of IRS’ efforts to develop

agencywide and operating division goals, objectives, and performance

measures that support IRS’ mission and (2) assess these efforts and

identify actions, if any, IRS could take to enhance its performance

management system.

To describe IRS’ efforts to develop goals, objectives, and performance

measures, we interviewed officials responsible for developing and

implementing IRS’ new strategic planning and budgeting process. We

reviewed IRS’ draft strategic plan for fiscal years 2000-05. Because IRS’

new performance management system is intended to balance taxpayer

service and enforcement activities, we focused on the goals, objectives

and performance measures affecting enforcement employees. Therefore,

we reviewed the strategic and program plans prepared by the Wage and

Investment, Small Business and Self-Employed, and Large and Mid-Size

Business Divisions because examination and collection employees were

generally reassigned to one of these divisions during IRS’ reorganization.

In addition, we requested the fiscal year 2000 action plans prepared by the

district and each of the examination and collection divisions and branches

from IRS’ 33 district offices. We received 15 district plans, 56 division

plans, and 197 plans from 60 branch and 32 district offices.

To assess IRS’ efforts in developing the components of its performance

management system, we conducted a literature search on performance

measures in general and balanced measures in particular; identified

characteristics associated with goals, objectives, and performance

measures for use in evaluating IRS’ performance management system; and

discussed IRS’ approach and methodology with IRS contractors involved

in developing the performance measures and analyzing the resulting data.

We also interviewed IRS headquarters officials and supervisors in the

Northern California and Kansas-Missouri District Offices to discuss their

use of performance measurement data and the action plans they

developed. We selected these two offices because of their proximity to the

members of the audit team.

The following provides more information on the characteristics associated

with goals, objectives, and performance measures that we used to assess

IRS’ performance management system and how we determined if those

characteristics were present.

Characteristics associated with strategic goals are that they should (1)

reflect the agency’s priorities, (2) provide clear direction for future agency

Appendix I: Objectives, Scope, and

Methodology

Objectives

Scope and

Methodology

Characteristics Used to

Assess IRS’ Performance

Management System

Characteristics Associated

With Goals
Appendix I: Objectives, Scope, and

Methodology

Page 23 GAO-01-234 IRS' Performance Management System

action, (3) identify what impact or outcome will result from an agency’s

work, and (4) form the basis for formulating clear objectives. Basically, we

were looking for statements that indicated how and why the goal should

be achieved. In assessing IRS’ goals, we addressed the following types of

questions:

• Reflect the agency’s priorities: Does the goal contribute toward achieving

IRS’ mission of helping taxpayers understand and meet their tax

responsibilities and applying the tax law with integrity and fairness?

• Provide clear direction for future action: Does the goal establish a trend

for IRS’ programs and activities?

• Identify impact or outcome: Does the goal indicate what effect achieving

the goal will have on taxpayers, overall compliance, or employees?

• Form the basis for formulating clear objectives: Does the goal indicate

the kinds of activities that managers and employees should focus on?

Characteristics associated with objectives and action items are that they

should be (1) clearly stated, (2) specific, (3) measurable, (4) outcome or

output oriented, and (5) consistent with the agency’s mission and goals.

We used these characteristics to assess IRS’ strategic objectives, operating

objectives, and action items. Basically, we were looking for statements

that would allow managers to be held accountable for their actions. In

assessing IRS’ strategic objectives, operating objectives, and action items,

we addressed the following types of questions:

• Clearly stated: Is the objective or action item easy to understand? Does it

contain jargon that would be unfamiliar to IRS managers and employees?

• Specific: Does the objective or action item include details such as who is

responsible for taking action? Does the objective or action item include a

time period, an expected result, or a numeric target?

• Outcome or output oriented: Does the objective or action item include an

expected result, a program impact, or a time period?

• Measurable: Is the objective or action item quantifiable? Does the action

item include a time period, an expected result, a numeric target, or a

means to measure?

• Consistent with IRS’ mission: Is the objective or action item consistent

with IRS’ mission statement and goals?

Characteristics associated with performance measures are that they

should be (1) logically and directly related to the goals and objectives; (2)

focus on expected outcomes or outputs; and (3) capture information

relevant to the goals and objectives. In assessing IRS’ performance

measures, we addressed the following types of questions:

Characteristics Associated

With Objectives and Action

Items

Characteristics Associated

With Performance Measures
Appendix I: Objectives, Scope, and

Methodology

Page 24 GAO-01-234 IRS' Performance Management System

• Logically and directly related: Does the measure address the important

components of the goals or objectives?

• Focus on expected outcomes or outputs: Does the measure address the

impact that IRS’ activities have on taxpayers and employees or the output

of IRS programs?

• Capture relevant information: Is the measure providing information

useful to managers in assessing the extent to which the objective has been

achieved or the impact of IRS activities?
Appendix II: Assessment of the

Characteristics of Operating Division

Strategic Goals

Page 25 GAO-01-234 IRS' Performance Management System

We assessed whether all of the 15 strategic goals established by the Wage

and Investment, Small Business and Self-Employed, and Large and Midsize Business Divisions had the characteristics described in appendix I. We

found that all 15 reflected IRS’ priorities. However, three of the goals did

not provide a clear direction for future action, identify the expected

impact of achieving the goal, or form the basis for establishing clear

objectives. A discussion of how we assessed these three goals follows.

“Address Underreporting, Nonfiling, and Abuse of Trusts and PassThroughs”

This goal, established by the Small Business and Self-Employed Division,

is set within the context of increasing overall compliance. However, the

word “address” is vague and does not provide a clear indication of the

direction IRS will be taking in the future. For example, “address” could

mean increasing taxpayer education, outreach to practitioners, or

compliance monitoring. In addition, the goal statement does not articulate

what the expected impact will be—presumably, to reduce the amount of

unreported income that results from taxpayers’ underreporting income,

failing to file tax returns, or using questionable trusts and pass-throughs.

The goal also does not provide a good basis for formulating objectives for

accomplishing it because it does not indicate how IRS will deal with

underreporting, nonfiling, and abuse of trusts and pass-throughs. As a

result, the goal statement does not communicate the activities IRS will

focus on in both the short term and the long term and the

accomplishments IRS is striving to achieve.

“Build a Tax Administration to Effectively Deal With

Globalization”

This statement, established by the Large and Mid-Size Business Division, is

attempting to articulate a goal for administering the tax laws in a global

economic environment where such factors as transnational operations,

rapidly changing technology, and changing business practices must be

recognized. However, the goal is broad and nonspecific. As with the prior

example, the phrase “build a tax administration” does not articulate the

direction IRS intends to take in the future. In addition, the goal does not

indicate the expected impact on taxpayers or overall compliance. For

example, the phrase “effectively deal with globalization” could refer to

identifying and responding to emerging trends quickly or to developing

technical competence and sophistication in interacting with multinational

corporations. The goal statement also does not indicate what type of

activities will be undertaken or the areas that should be improved and so

Appendix II: Assessment of the

Characteristics of Operating Division

Strategic Goals
Appendix II: Assessment of the

Characteristics of Operating Division

Strategic Goals

Page 26 GAO-01-234 IRS' Performance Management System

does not provide the basis for formulating objectives to accomplish the

goal.

“Implement Comprehensive Issue Management Strategy”

With this goal statement, IRS’ Large and Mid-Size Business Division is

attempting to address the dissatisfaction of taxpayers related to the

examination of large corporate returns. However, as with the other goals,

the phrase “implement a comprehensive issue management strategy” is too

vague to indicate the future direction of IRS’ programs and activities. The

statement also does not provide an indication of how achieving this goal

will impact taxpayers and overall compliance, focusing instead on IRS’

internal processes. In addition, the vagueness and generality of the goal

statement provide little basis for formulating objectives. For example, if

IRS has identified the factors contributing to taxpayer dissatisfaction, the

goal statement should articulate how IRS will deal with those factors.
Appendix III: Examples of Action Items

Developed by IRS Organizational Units

Page 27 GAO-01-234 IRS' Performance Management System

This appendix provides examples of action items developed by IRS

managers in the examination and collection functions. The action items

are grouped in two categories, those that were specific, measurable, and

outcome or output oriented, and those that were not.

The following action items were determined to be specific, measurable, or

outcome or output oriented.

• “Branch will establish a quarterly review of 5 cases from each of four pure

general program groups. The five cases will be reviewed by a manager

from another group for adherence to the EQMS standard.”

• “Specialty Group Managers (E & G [Estate and Gift], SEP [Special

Enforcement Program], Employment Tax) will ensure attorneys and

examiners receive information regarding the recommendations of

Business Measures Task Force Study within 60 days of issuance of report.”

• “Ensure all contact employees receive examination customer service

training by December 31, 1999.”

• “Train CEP [Coordinated Examination Program] and Specialty agents on

recommendations of Business Measures Task Force Study within 60 days

of issuance of report.”

• “Compile 9/30/99 data by 12/31/99 for baseline compilations.”

• “Ensure all employees receive Customer Service Examination Specific

training by 12/31/99.”

• “CEB-3 [Chief of Examination Branch 3] will attend one group meeting

each month to discuss employee feedback and elevate issues when

appropriate.”

• “Group manager will conduct field visits with revenue officers at least

once during the fiscal year.”

• “Complete balanced measures training 8530(A) by 12/31/99.”

• “Improve manager and employee understanding of auditing standards

through consistency reviews and at least one non-evaluative live case

review per examiner by 12/31/99.”

• “Managers and secretaries will not use the VMS [Voice Mail System] but

will answer the phones personally. R/A [revenue agent], T/A [tax auditor],

and ROE [revenue officer examiner] will use VMS system when not in the

office, but will leave dated and current messages for callers to receive.”

Appendix III: Examples of Action Items

Developed by IRS Organizational Units

Specific, Measurable,

and Outcome- or

Output-Oriented

Action Items
Appendix III: Examples of Action Items

Developed by IRS Organizational Units

Page 28 GAO-01-234 IRS' Performance Management System

In contrast to the above action items, the following were determined not

to be specific, measurable, and outcome or output oriented.

• “Support on-going Research Strategy Projects.”

• “Ensure employees have necessary tools to perform duties (on-going).”

• “Implement/follow the district’s market segment plan.”

• “Identify improvement areas and implement actions to address them.”

• “Ensure managers are correctly implementing the balanced measures.”

• “Monitor group plans to ensure actions are being taken to improve

quality.”

• “Minimize neglected taxpayer groups.”

• “Emphasize one stop service in our examinations.”

• “Dialogue, coach, collaborate, and look forward when planning and

making decisions.”

• “Meet District/Division due date or time line.”

• “Identify potential areas for outreach activities.”

• “Strive to understand and solve problems from the taxpayer’s point of

view.”

• “Increase available help for those taxpayers who are calling in or writing

in for assistance.”

• “Use new IRM [Internal Revenue Manual] standards for operational

reviews (when available).”

• “Utilize all forms of communication available to facilitate keeping

employees informed.”

• “Use balanced measures as part of the decision making process.”

• “Identify transition issues.”

• “Create a positive environment that encourages open communication (e.g.

coaching and mentoring).”

• “Monitor and balance between providing quality customer service support

and providing quality customer service to Collection taxpayers across all

branches.”

Unspecific,

Unmeasurable, and

Not Outcome- or

Output-Oriented

Action Items
Appendix IV: Comments From the Internal

Revenue Service

Page 29 GAO-01-234 IRS' Performance Management System

Appendix IV: Comments From the Internal

Revenue Service
Appendix IV: Comments From the Internal

Revenue Service

Page 30 GAO-01-234 IRS' Performance Management System
Appendix IV: Comments From the Internal

Revenue Service

Page 31 GAO-01-234 IRS' Performance Management System
Appendix V: GAO Contacts and Staff

Acknowledgments

Page 32 GAO-01-234 IRS' Performance Management System

James R. White (202) 512-9110

Ralph Block (415) 904-2150

In addition to those named above, Benjamin Douglas, Suzy Foster, Jonda

Van Pelt, and John Zugar made key contributions to this report.

Appendix V: GAO Contacts and Staff

Acknowledgments

GAO Contacts

Acknowledgments

(268898)
The first copy of each GAO report is free. Additional copies of reports are

$2 each. A check or money order should be made out to the

Superintendent of Documents. VISA and MasterCard credit cards are also

accepted.

Orders for 100 or more copies to be mailed to a single address are

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St., NW (corner of 4th and G Sts. NW)

Washington, DC 20013

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and testimony. To

receive facsimile copies of the daily list or any list from the past 30 days,

please call (202) 512-6000 using a touchtone phone. A recorded menu will

provide information on how to obtain these lists.

Orders by Internet

For information on how to access GAO reports on the Internet, send an email message with “info” in the body to:

Info@www.gao.gov

or visit GAO’s World Wide Web home page at:

http://www.gao.gov

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• E-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)

Ordering Information

To Report Fraud,

Waste, and Abuse in

Federal Programs