GENERAL MANAGEMENT:

Better Performance Measures

 Are Needed For Treasury

 Enforcement Programs

OIG-02-013 November 16, 2001

Office of Inspector General

*******

The Department of the Treasury
Contents

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013)

Page 1

Audit Report.........................................................................................................3

Results in Brief....................................................................................................3

Background.........................................................................................................5

Findings and Recommendations............................................................................6

Reporting of Supplemental Information Needs to Be

Clear, Complete, and Consistent......................................................................6

Recommendations ..........................................................................................8

Treasury’s Performance Reporting Needs Further Improvement.......................10

Recommendations ........................................................................................12

Treasury Should Coordinate Common Measures and Balanced Reporting.........14

Recommendations ........................................................................................16

Critical Statistics Need to Be Consistently Reported.......................................19

Recommendation..........................................................................................20

Appendices

Appendix 1: Objectives, Scope, and Methodology..............................................22

Appendix 2: Treasury Law Enforcement Performance Measures .........................24

Appendix 3: Management Comments ................................................................26

Appendix 4: Major Contributors to this Report ...................................................31

Appendix 5: Report Distribution ........................................................................32
Contents

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 2

Abbreviations

ATF Bureau of Alcohol, Tobacco and Firearms

Customs U.S. Customs Service

FTE Full Time Equivalent

FY Fiscal Year

GAO U.S. General Accounting Office

IRS Internal Revenue Service

Justice Department of Justice

OIG Office of Inspector General

OMB Office of Management and Budget

ONDCP Office of National Drug Control Policy

Results Act Government Performance and Results Act of 1993
OIG Audit

Report

The Department of the Treasury

Office of Inspector General

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 3

November 16, 2001

Jimmy Gurulé

Under Secretary for Enforcement

We have completed an audit to determine if the performance 

measures for the Department of the Treasury (Treasury) and its 

enforcement Bureaus cover the key aspects of enforcement 

performance in a clear and consistent manner. Our audit was 

performed in accordance with the Office of Inspector General’s 

(OIG) Fiscal Year (FY) 2001 Annual Plan.

We reviewed FY 1999 and FY 2000 performance reports for 

Treasury; U.S. Customs Service (Customs); Bureau of Alcohol, 

Tobacco and Firearms (ATF); and U.S. Secret Service (Secret 

Service). However, performance measures for Secret Service 

protection activities and federal law enforcement training were 

specifically excluded from our scope. We interviewed Treasury and 

Bureau officials in Washington, D.C.; Chicago; and Houston. 

Further, we discussed enforcement performance measure issues 

with officials from the U.S. General Accounting Office (GAO), 

Office of National Drug Control Policy (ONDCP), and the Office of 

Management and Budget (OMB). Finally, we discussed our 

preliminary findings and recommendations with staff from six 

interested congressional committees. We performed our audit 

fieldwork between September 2000 and April 2001. Appendix 1 

provides a more detailed description of the audit objectives, scope, 

and methodology.

Results in Brief

Overall, we found that Treasury and Bureau performance reports 

provide a limited picture of actual law enforcement performance. 
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 4

In our opinion, current performance measures do not demonstrate 

the full impact of Treasury’s enforcement programs on the 

protection of the public, economy, and revenue. Treasury has 

been developing and reporting better measures and has included 

beneficial supplemental charts and explanations in its performance 

reports. However, there are major government-wide obstacles to 

measuring the performance of Federal law enforcement that have 

yet to be overcome.

In general, the congressional staffs we met with wanted 

performance measures and reports that were clear and consistent 

between Treasury and the Department of Justice (Justice) law 

enforcement Bureaus. They felt that deterrence, prevention, and 

protection of individual rights need to be included in performance 

reporting to provide balance and prevent unintended consequences. 

Also, they want arrest, prosecution, conviction, and seizure 

workload statistics reported in a clear and consistent manner.

Treasury, working with Justice and ONDCP, needs to develop 

better and more consistent performance reports and reliable data 

sources. This should be done in consultation with congressional 

stakeholders to assure new measures and supplemental information 

is clear, concise, complete, and useful.

This report contains 4 findings and 10 recommendations 

concerning the consistency, clarity, and coordination of Treasury’s 

enforcement performance reporting.

The Office of Enforcement generally concurred with our findings 

and outlined a set of planned corrective actions that, when fully 

implemented, will satisfy the intent of our recommendations. 

However, its response to our draft report did not include target 

dates for the planned corrective actions as required by Treasury 

Directive 40-01, Responsibilities of and to the Inspector General. 

This information will need to be developed and provided to our 

office before we can consider the recommendations to have a 

management decision. The Office of Enforcement’s response is 

provided as Appendix 3.
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 5

Background

The Government Performance and Results Act of 1993 (Results 

Act) was enacted to systematically hold Federal agencies 

accountable for achieving program results. Specifically, Federal 

managers are required to set strategic goals, measure performance, 

and report on their progress. In enacting the Results Act, Congress 

and the Administration realized that the transition to resultsoriented management would not be easy. For that reason, the 

Results Act provided for a phased approach to implementation. 

The Results Act also emphasizes the importance of consultation 

with the Congress as strategies are planned, goals are identified, 

and measures are selected.

The purpose of the Results Act is to:

• Improve the confidence of the American people by holding 

Federal agencies accountable.

• Improve program effectiveness and public accountability by 

promoting a new focus on results, service quality, and customer 

satisfaction.

• Help Federal managers improve service delivery by requiring 

them to plan for meeting program objectives.

• Improve congressional decision-making by providing more 

objective information on achieving statutory objectives.

• Improve internal management of the Federal government.

Overall, Treasury established 340 performance targets in FY 2000. 

Of these, 46 measures were either (1) baselined in FY 2000, 

(2) qualitative and not objectively measurable, or (3) had no data 

available for FY 2000. Treasury has four mission areas: Economic, 

Financial, Enforcement, and Management. The net cost of 

Treasury’s business in FY 2000 was $13.5 billion of which 

$3.2 billion (24 percent) was related to the Enforcement Mission. 

The Enforcement Mission is supported by ATF, Customs, Secret 

Service, Financial Crimes Enforcement Network, Internal Revenue 
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 6

Service (IRS), Federal Law Enforcement Training Center, and 

Departmental Offices. The purpose of Treasury enforcement is to 

protect the public, economy, and revenue from money laundering, 

counterfeiting, financial crime, drug smuggling, violent crime, and 

terrorism. Enforcement performance is measured against five 

major goals: 

• Combat Money Laundering and Other Financial Crimes

• Protect Our Nation’s Borders and Major International 

Transportation Terminals from Traffickers and Smugglers of 

Illicit Drugs

• Reduce Violent Crime and the Threat of Terrorism

• Protect Our Nation’s Leaders and Visiting Dignitaries

• Provide High-Quality Training for Law Enforcement Personnel

Findings and Recommendations

Finding 1 Reporting of Supplemental Information Needs to Be Clear, 

Complete, and Consistent

The Treasury and Bureau performance reports provide a limited 

picture of actual law enforcement performance. In our opinion, 

current performance measures do not demonstrate the full impact 

of Treasury enforcement on the protection of the public, economy, 

and revenue. Treasury enforcement operations have a greater 

impact and broader scope than what is currently reported. 

Performance measurement for Federal law enforcement is still an 

evolving process. To date no one has designed a clear and 

complete set of outcome-related performance measures for Federal 

law enforcement. These limitations are substantial and 

long-standing, and will not be quickly or easily resolved. This is 

partially due to the following unanswered questions, which present 

obstacles to measurement:
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 7

• How can there be reliable and meaningful performance 

measures to reduce crime universes that are unknown or can 

only be estimated?

• How can the impact of a single Bureau be measured in 

programs that involve the participation of multiple Federal, 

foreign, state, and local agencies?

• How can performance be measured in areas where external 

factors have significant impact? (i.e., the economy, culture, 

foreign corruption, and technology)

• How can annual performance measures report on long-term 

undercover operations to disrupt and dismantle major 

international criminal organizations?

• How can protection of the public from terrorism, unsafe 

products, hazardous materials, explosives, and child 

pornography be measured quantitatively?

Recognizing the obstacles that need to be overcome, it is 

necessary to provide clear, complete, and consistent supplemental 

information. This will be necessary to make the Treasury and 

Bureau performance reports more usable while research into better 

measures continues. Customs’ Strategic Plan provides an example 

of how this information could be presented. Specifically, Customs 

provides a chart that lists performance measures reported, planned, 

and used internally.1 A similar presentation by the other 

enforcement Bureaus would allow congressional stakeholders to 

see how measurement is being implemented and what information 

is available if needed.

Performance goals are of minimal value for congressional decision 

makers without a connection to the resources requested. In 

FY 2000, Treasury reported a net cost for the Enforcement Mission 

of $3.2 billion. However, Treasury’s current performance report 

does not indicate the resources or personnel devoted to each of the 

five enforcement goals. In its FY 2000 performance report, 

 

1

 U.S. Customs Service Strategic Plan Fiscal Years 2000-2005, pages 40 

through 42.
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 8

Customs reported that $1 billion and 8,040 Full Time Equivalent 

(FTE) positions were devoted to enforcement. But, Customs did 

not breakout the resources devoted to the three performance goals 

of drug interdiction, money laundering, and other Customs 

enforcement activities. An early draft of the FY 2000 Secret 

Service performance report we reviewed displayed budget dollars, 

FTEs, and performance measures by protective services, 

investigations, and support infrastructure. However, this very 

useful information was not included in its final report. ATF 

reported $0.6 billion and 3,702 FTE devoted to reducing violent 

crime. 

As illustrated by the above, there is a misalignment between the 

current budget activities and budgeting by performance goals. 

Reporting by goals is more detailed and more informative. At a 

minimum, we believe the Treasury and Bureau reports should show 

the resources devoted to the five departmental goals for 

enforcement. This will allow users of the performance reports to 

compare the resources devoted to specific goals with the results 

achieved.

Recommendations

The Under Secretary for Enforcement, in coordination and with 

assistance from the Assistant Secretary for Management and Chief 

Financial Officer (CFO), should ensure that:

1. The enforcement Bureaus include a listing of the enforcement 

performance measures reported, planned, and used internally in 

a consistent manner as part of their performance reports.

Management Comment. The Office of Enforcment concurred 

with this recommendation. It will encourage ATF and Secret 

Service to provide this information in a format similar to that 

used in Customs strategic plan.

OIG Comment. The actions planned by management satisfy the 

intent of our recommendation. We will consider this 

recommendation to have a management decision once an 
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 9

appropriate target completion date for these actions is 

established.

2. The resources used (net costs and FTEs) to accomplish each of 

the five goals for the enforcement mission are presented in a 

consistent manner in the Treasury and Bureau performance 

reports, as applicable.

Management Comment. The Office of Enforcement agrees in 

principle with this recommendation. However, implementing 

the recommendation is contingent on the ability of Treasury’s 

budgeting and accounting systems to provide such information. 

Two years ago, the Department requested that the bureaus 

provide a statement of net costs by strategic goal, but their 

accounting systems were unable to provide this information. 

Treasury is continuing its attempt to provide statements of net 

costs by strategic goal. The Office of Enforcement and the 

enforcement bureaus will participate in the Treasury-wide effort 

in this area. This effort may require new cost accounting 

methodologies and amendment of Treasury strategic goals to 

embrace more of the significant work performed by the 

bureaus. In any event, the enforcement bureaus should not be 

asked to provide the requested information in isolation before 

the Treasury-wide effort is complete.

OIG Comment. The planned actions, outlined in general terms 

above, satisfy the intent of our recommendation. However, 

their implementation will require close coordination between the 

Offices of Enforcement and the Assistant Secretary for 

Management and CFO. We will consider this recommendation 

to have a management decision once an appropriate target 

completion date for these actions is established. The Office of 

Enforcement should work with the Assistant Secretary for 

Management and CFO in developing this date.

3. Research is continued and coordinated with the enforcement 

Bureaus and other agencies to develop new quantitative 

performance measures that show the impact of Treasury 

enforcement operations on the protection of the public, 

economy, and revenue. For those goals for which meaningful 
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 10

and complete outcome measures have yet to be reported, 

Treasury and the Bureaus should describe how operations 

directly impact the protection of the public, economy, and 

revenue in the narrative portion of the performance reports.

Management Comment. The Office of Enforcement concurred 

with this recommendation. Efforts are underway within the 

Office and the enforcement Bureaus to further develop 

meaningful outcome measures for Treasury enforcement 

operations. The Office is working closely with the Department 

of Justice and the Office of National Drug Control Policy to 

improve drug interdiction and drug investigation performance 

measures.

OIG Comment. The actions planned by the Office of 

Enforcement satisfy the intent of our recommendation. We will 

consider this recommendation to have a management decision 

once an appropriate target completion date for these actions is 

established.

Finding 2 Treasury’s Performance Reporting Needs Further 

Improvement

A GAO review2 of Treasury’s FY 1999 performance report 

concluded that the report provides a limited picture of the 

Department’s actual performance. In response, Treasury officials 

stated that certain measures would continue to be output-oriented 

until Treasury is better able to determine the cause-and-effect 

relationships between its various programs and the outcomes they 

are intended to influence. A schedule of the current and planned 

performance measures is included in Appendix 2. The Treasury 

Program Performance Report for FY 2000 presents the results of 

enforcement in eight major areas:

 

2

 Treasury’s FY 1999 Performance Report and FY 2001 Performance Plan

(GAO/GGD/AIMD-00-231R; June 30, 2000).
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 11

• Money Laundering

• Financial Crime

• Counterfeiting

• Drug Interdiction

• Violent Crime

• Terrorism

• Protection

• Training

Money laundering is a high priority program that Treasury and the 

Bureaus expend a great deal of effort on. However, the current 

and planned measures for ATF, Customs, and Secret Service list 

only two money laundering-related seizure statistics. Since 

seizures are not performance measures, it is difficult for 

stakeholders to determine Treasury’s progress in this area. 

Treasury’s Financial Crimes Enforcement Network has an effort 

underway to estimate the money laundering universe. If this effort 

is successful then seizures as a percentage of the universe could 

be used as a performance measure.

The performance measures for financial crime have been improving. 

In the FY 1999 Performance Report, the measures for cases closed 

were replaced with actual financial crime loss and financial crime 

loss avoided. New measures planned include percent of financial 

crime with foreign nexus and the results of an industry response 

survey.

About $1.5 billion of the Federal government’s FY 2000 drug 

control budget was spent by Treasury. Yet, current measures for 

drug interdiction focus primarily on seizures. It is not clear from 

these measures whether changes in seizures are the result of 

successful interdiction or a change in the volume or method of 

smuggling. ONDCP officials informed us that its estimated 

universe of cocaine smuggled into the U.S. was now sufficiently 

reliable to use for performance measurement purposes. However, 

Treasury officials disagreed with the methodology used by ONDCP 

to calculate the current cocaine estimate. Also, ONDCP plans to 

publish estimates of other drug universes over the next couple of 

years. Customs has five new performance measures for the air 

and marine interdiction program. These measures were published 
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 12

in the Customs FY 2000 performance report issued in April 2001. 

The current Treasury report includes Customs passenger 

processing performance measures. These passenger processing 

measures, however, only partially deal with drug interdiction.

The problems and limitations of the performance measures for 

violent crime were outlined in the GAO report on Treasury’s 

FY 1999 performance report and in a June 2001 OIG report on 

ATF’s investigative case targeting and productivity.3 ATF’s 

FY 2000 Performance Report states that: “ATF is working to 

develop a model to fine tune and measure our contribution to the 

decrease in violent crime.”

Of the 6 current and planned performance measures for terrorism, 

2 are seizure numbers. Three are qualitative measures that are not 

objectively measurable. Under the Results Act, OMB can authorize 

the use of qualitative measures for performance reporting. 

However, such alternative reporting should sufficiently describe a 

minimally effective and a successful program in a manner that 

would allow for an accurate, independent determination of whether 

the program’s performance met the description. Treasury’s 

qualitative measures do not presently meet this requirement. In 

general, the congressional staff we met with did not find the 

qualitative measures useful in evaluating performance.

Recommendations

The Under Secretary for Enforcement, in coordination and with 

assistance from the Assistant Secretary for Management and CFO, 

should ensure that:

1. Seizures are reported as a percentage of the estimated universe 

as performance measures for money laundering and drug 

interdiction. The methodology and validation process for these 

estimates should be agreed to by participating agencies.

 

3

 PROTECTING THE PUBLIC: Bureau of Alcohol, Tobacco and Firearms’ 

Investigative Case and Time Data Needs Improvement (OIG-01-077; June 27, 

2001).
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 13

Management Comment. The Office of Enforcement concurred, 

in part, with this recommendation. However, the 

recommendation cannot be implemented until the Treasury has 

reasonable assurance that valid estimates of the money 

laundering and drug trafficking universes can be developed. In 

this regard, the Financial Crimes Enforcement Network has had 

an on-going effort underway for several years to estimate the 

magnitude of money laundering and it has proven to be 

exceptionally difficult to estimate economic activity in an area 

where the goal of the activity itself is concealment from the 

Government. Treasury is participating in ONDCP’s effort to 

develop and validate a drug interdiction measure based on drug 

seizures as a percentage of the relevant trafficking universe. 

Presently, only one trafficking estimate, ONDCP’s cocaine flow 

model, has been developed sufficiently to merit review by the 

Federal drug enforcement community. If the validity of this or 

similar estimation methodologies can be established, the Office 

of Enforcment believes that is should be applied to the seizure 

efforts of law enforcement as a whole and not to individual 

agency performance. The degree of cooperation presently 

maintained and required for effective interdiction could be 

jeopardized if agencies were encouraged to compete for 

seizures to meet their individual performance measures. 

OIG Comment. The actions planned by the Office of 

Enforcement satisfy the intent of our recommendation. 

However, once reasonable universes are available, we believe 

the decision to report measures by Bureau, Department, or 

government-wide should be discussed with congressional 

stakeholders. We will consider this recommendation to have a 

management decision once an appropriate target completion 

date for these actions is established.

2. Qualitative measures are replaced with quantitative measures in 

Treasury and Bureau performance reports. Where this is not 

possible, the qualitative measures should describe a minimally 

effective and a successful program in sufficient detail for 

Treasury’s stakeholders to determine if the program is 

succeeding.
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 14

Management Comment. The Office of Enforcement concurred 

with this recommendation. Where qualitative measures are 

used, Bureaus are being asked to provide a description that will 

help stakeholders determine if the program is succeeding. 

However, the Office of Enforcement will work with the 

enforcement Bureaus to develop better quantitative measures, 

wherever possible.

OIG Comment. The actions planned by Treasury satisfy the 

intent of our recommendation. We will consider this 

recommendation to have a management decision once an 

appropriate target completion date for these actions is 

established.

Finding 3 Treasury Should Coordinate Common Measures and 

Balanced Reporting

Treasury Should Coordinate Development of Common Measures

Treasury, IRS, ATF, Customs, and Secret Service may be reporting 

several new performance measures in the future. While the 

development of performance measures and reliable data sources 

can be an expensive and lengthy process, there is no dedicated 

funding for these efforts. Therefore, Treasury and the Bureaus 

must fund these efforts with resources intended for the 

accomplishment of their missions. If each Bureau conducts these 

efforts independently, then some expenses could be almost four 

times what they need to be. Further, if each Bureau reports what 

are essentially similar measures but in a different manner, 

stakeholders may be confused and the task of consolidating 

individual Bureau performance in Treasury’s performance report is 

more difficult.

A highly trained and motivated workforce, that is properly equipped 

and managed, is essential for Federal law enforcement. The work 

performed by Treasury agents is complex, hazardous, and highly 

specialized. Treasury currently plans to have a performance 

measure for employee satisfaction. In order to evaluate 

enforcement performance, we believe the Department should 
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 15

ensure employee satisfaction surveys are consistent for the 

enforcement Bureaus.

Balanced Reporting Needed

Current Treasury and Bureau performance reports do not include a 

balanced set of measures. There are inherent competing priorities 

in Federal law enforcement. For example, the enforcement 

Bureaus need to balance:

• Investigating criminal acts Vs Preventing and deterring crime

• Fighting crime Vs Protecting individual rights

• Arresting violators Vs Conducting “strategic” investigations

• Taking credit for results Vs Cooperating with other agencies

Current Federal law enforcement performance measures generally 

address the priorities on the left of the above list but Treasury has 

found it difficult to measure the competing priorities on the right. 

Congressional staff we met with told us that balanced reporting 

was important. Performance reports that fail to present a balanced 

picture of performance may result in negative unintended 

consequences. For example, if the performance and subsequent 

funding of Federal law enforcement is based on the investigation of 

criminal acts, then crime prevention and deterrence efforts could be 

cut back. The incentives would be to solve crimes rather than to 

prevent them. This would be in spite of the fact that it is clearly in 

the public interest to prevent crime from occurring.

Among the principal customers of enforcement are the U.S. and 

State Attorney’s offices. Cases presented by Treasury agents 

could be compromised if individuals’ rights were violated, if 

evidence was mishandled, or statutory authorities were exceeded. 

In this regard, government prosecutors are a good source that 

could be surveyed to provide useful feedback on the enforcement 

Bureaus’ performance in these areas.

ONDCP officials informed us that they are working on a means to 

measure deterrence for purposes of performance reporting. As 

ONDCP or other organizations develop valid measures of 
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 16

deterrence, Treasury enforcement Bureaus should adopt them in 

their performance plans where appropriate.

ATF, Customs, and Secret Service all rely on cooperation with 

state and local law enforcement to accomplish their missions. The 

Bureaus leverage their limited resources by sharing intelligence, 

participating in joint task groups, providing technical support, 

providing advanced training, and sharing equipment with state and 

local law enforcement. However, this cooperation is not reflected 

in the current performance measures. State and local law 

enforcement could be surveyed to provide useful feedback to 

enforcement Bureaus.

Recommendations

The Under Secretary for Enforcement, in coordination and with 

assistance from the Assistant Secretary for Management and CFO, 

should:

1. Coordinate with Justice and, where appropriate, ONDCP to 

develop a consistent set of performance measures that address 

the protection of individual rights, crime prevention, crime 

deterrence, strategic investigative cases, and interagency 

cooperation.

Management Comment. The Office of Enforcement concurred 

in part with this recommendation. However, thorough 

exploration of relevant issues with the enforcement Bureaus, as 

well as Justice and ONDCP, will be necessary before the Office 

of Enforcement can commit to developing consistent measures 

throughout the Federal law enforcement community. The 

Office of Enforcement and the enforcement bureaus are 

concerned and sensitive to the protection of individual rights. 

However, management believes that this area is more 

appropriate for internal management monitoring rather than as a 

published performance measure. It should be noted that 

strategic investigative cases and interagency cooperation appear 

to be output measures rather than results-oriented outcome 

measures.
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 17

OIG Comment. The actions planned by Treasury satisfy the 

intent of our recommendation. Treasury may report some 

measures in their performance reports and others in internal 

management reports. However, since all will be listed in the 

performance reports, stakeholders will be aware of the 

information available. We will consider this recommendation to 

have a management decision once an appropriate target 

completion date for these actions is established.

2. Coordinate with the enforcement Bureaus and Justice to 

develop a standard survey of the U.S. Attorney’s offices. The 

survey should be used to develop a performance measure on 

the quality of Treasury cases. For example, the survey should 

inquire about areas concerning protection of individuals’ rights, 

handling of evidence, and application of statutory authorities. A 

similar survey should be conducted at selected State Attorney’s 

offices, as appropriate.

Management Comment. The Office of Enforcement concurred 

in part with this recommendation. Management agrees with the 

concept of a survey of the U.S. Attorney’s Offices, and the 

enforcement bureaus are doing some surveys to determine the 

quality of their cases. Treasury will coordinate with Justice to 

reach agreement on the best methodology for conducting such 

surveys. It will also explore whether a performance measure 

should be developed based on survey. However, the Office of 

Enforcement has some reservations about using this type of 

data for a performance measure because, at best, the data 

would be an output rather than an outcome measure. Due to 

the sensitivity and possible subjectivity of the information, the 

survey results might be used for internal management 

assessments rather than as a published performance measure.

OIG Comment. The actions planned by the Office of 

Enforcement satisfy the intent of our recommendation. 

Treasury may report some measures in their performance 

reports and others in internal management reports. However, 

since all will be listed in the performance reports, stakeholders 

will be aware of the information available. We will consider this 

recommendation to have a management decision once an 
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 18

appropriate target completion date for these actions is 

established.

3. Coordinate with ATF, Customs, and Secret Service to develop 

and implement a consistent enforcement employee satisfaction 

survey and related performance measure.

Management Comment. The Office of Enforcement concurred 

in part with the recommendation because of concern about 

using employee satisfaction surveys to drive a performance 

measure rather than as an internal management tool. However, 

Treasury management is currently considering an employee 

satisfaction performance measure for the entire Department. 

The Office of Enforcement’s efforts should proceed in 

coordination with the Department’s decisions on how to 

address the issue more broadly.

OIG Comment. The actions planned by the Office of 

Enforcement satisfy the intent of our recommendation. 

Treasury may report some measures in their performance 

reports and others in internal management reports. However, 

since all will be listed in the performance reports, stakeholders 

will be aware of the information available. We will consider this 

recommendation to have a management decision once an 

appropriate target completion date for these actions is 

established.

4. Coordinate with ATF, Customs, and Secret Service to develop 

and implement a standard survey of state and local law 

enforcement agencies. The survey should provide the basis for 

a performance measure on the satisfaction with the 

cooperation, assistance, technical support, and training the 

Bureaus provide.

Management Comment. The Office of Enforcement concurred 

in part with this recommendation and will explore, in 

coordination with the bureaus, the feasibility of developing the 

recommended standard survey for use as an internal 

management tool. The recommended measure, however, would 

be an output oriented measure instead of an outcome measure, 
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 19

as cooperation with, technical assistance to, and training of 

state and local law enforcement agencies are means to 

achieving desired outcomes. 

OIG Comment. The actions planned by the Office of 

Enforcement satisfy the intent of our recommendation. 

Treasury may report some measures in their performance 

reports and others in internal management reports. However, 

since all will be listed in the performance reports, stakeholders 

will be aware of the information available. We will consider this 

recommendation to have a management decision once an 

appropriate target completion date for these actions is 

established.

Finding 4 Critical Statistics Need to Be Consistently Reported

There are certain basic statistics that are common to ATF, 

Customs, and Secret Service enforcement functions. These 

include arrests, prosecutions, convictions, and seizures. While 

these statistics are not performance measures, they are needed by 

stakeholders to put performance information in perspective and to 

analyze long-term trends. 

Currently, Secret Service includes the number of cases and arrests 

for financial crime and counterfeit currency in the narrative portion 

of its performance reports. Customs uses seizures for 13 of its 

performance measures. ATF does not report arrests, prosecutions, 

convictions, or seizures in its performance reports. The Treasury 

performance report includes additional statistics that are not in the 

Bureau reports that may be useful to stakeholders. These include 

statistical information on: (1) arrests by Customs, (2) convictions 

of members of drug smuggling organizations, (3) the national crime 

rate, (4) violent crimes, (5) crimes committed with firearms, 

(6) fatal bombing incidents, (7) terrorist incidents, and (8) terrorist 

incidents prevented.

While arrests and prosecutions are easy to measure, they can 

result in outcomes at odds with agency goals. For example, 

tougher strategic investigative cases may be given less emphasis in 

favor of quick insignificant cases in order to boost a score. Also, 
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 20

the pursuit of justice and protection of individual rights may be 

subverted. Additionally, numbers of convictions are more under 

the control of the judicial process rather than Treasury Bureaus.

In numerous reports, GAO has held that seizures are not valid 

measures of performance. Specifically, it is difficult to attribute 

changes in numbers or quantities of seizures to program 

effectiveness. If seizures go up, this could be due to better 

interdiction efforts or an increase in smuggling activity. If seizures 

go down, this could be because successful interdiction efforts have 

reduced smuggling or weaker interdiction efforts. Thus, a Bureau 

could describe both an increase or decrease as being an indicator 

of program success.

Staff from six congressional committees we met with expressed 

the belief that arrests, prosecutions, convictions, or seizures should 

not be used as performance measures. However, they wanted this 

information included as workload statistics in the performance 

reports. Further, they wanted this information to be reported 

consistently for both the Treasury and Justice law enforcement 

Bureaus. If possible, they would like to see arrests classified by 

significance. Staff from one committee commented that such 

workload statistics should be preceded by a statement that they 

are not to be used for projections or targets.

Recommendation

The Under Secretary for Enforcement, in coordination and with the 

assistance of the Assistant Secretary for Management and CFO, 

should ensure that:

1. Workload statistics on the numbers of arrests, prosecutions, 

convictions, and seizures is consistently presented in Treasury, 

ATF, Customs, and Secret Service performance reports. In this 

regard, congressional stakeholders should be consulted as to 

the best presentation of these workload statistics for their 

oversight purposes.

Management Comment. The Office of Enforcement concurred 

with this recommendation. It will confer with the enforcement 
Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 21

Bureaus, Justice, and congressional stakeholders to determine 

the feasibility of and best format for presentation of these 

workload statistics.

OIG Comment. The actions planned by the Office of 

Enforcement satisfy the intent of our recommendation. We will 

consider this recommendation to have a management decision 

once an appropriate target completion date for these actions is 

established.

* * * * *

We would like to extend our appreciation to Treasury and Bureau 

officials for the cooperation and courtesies given to our staff during 

this audit. If you wish to discuss this report, you may contact me 

at (202) 927-5400 or George W. Tabb, Regional Inspector General 

for Audit at (713) 706-4611, ext. 235. Major contributors to this 

report are listed in Appendix 4.

Marla A. Freedman

Assistant Inspector General for Audit
Appendix 1

Objectives, Scope and Methodology

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 22

The objective of the audit was to determine if the performance 

measures for Treasury enforcement Bureaus cover the key aspects 

of performance in a clear and consistent manner. This audit was 

included in our FY 2001 Annual Plan. We conducted our audit 

fieldwork between September 2000 and April 2001.

Treasury’s Enforcement Mission consists of five major goals:

• Combat Money Laundering and Other Financial Crimes

• Protect Our Nation’s Borders and Major International 

Transportation Terminals from Traffickers and Smugglers of 

Illicit Drugs

• Reduce Violent Crime and the Threat of Terrorism

• Protect Our Nation’s Leaders and Visiting Dignitaries

• Provide High-Quality Training for Law Enforcement Personnel

The Enforcement Mission is supported by ATF, Customs, Secret

Service, IRS, Financial Crimes Enforcement Network, Federal Law 

Enforcement Training Center, and Treasury’s Departmental Offices. 

We limited the scope of our audit to the first three goals listed 

above. The scope of the audit also excluded IRS, the Financial 

Crimes Enforcement Network, and the Federal Law Enforcement 

Training Center. However, we coordinated our audit work with an 

audit of enforcement performance measurement at IRS that was 

conducted concurrently by auditors from the Treasury Inspector

General for Tax Administration.

To accomplish our objective, we reviewed the current and planned 

performance measures of the selected Bureaus related to the three 

goals and how Treasury reported them. We also reviewed 

FY 1999 and FY 2000 performance reports for Treasury, ATF, 

Customs, and Secret Service.
Appendix 1

Objectives, Scope and Methodology

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 23

We interviewed Treasury and Bureau headquarters officials and 

enforcement officials of Customs, ATF, and Secret Service located 

in the Bureaus’ Chicago and Houston field offices. Further, we 

discussed enforcement performance measure issues with officials 

from GAO, ONDCP, and OMB. 

In formulating our recommendations, we discussed our preliminary 

audit results with staff of the following six congressional 

committees or subcommittees: (1) Senate Committee on 

Appropriations; (2) Senate Committee on Finance; (3) Senate 

Committee on Governmental Affairs; (4) House Committee on 

Appropriations; (5) House Subcommittee on Crime of the Judiciary 

Committee; and (6) House Subcommittee on Government 

Management, Information and Technology of the Committee on 

Government Reform.

We conducted our audit in accordance with generally accepted 

government auditing standards.
Appendix 2

Current and Planned Treasury Law Enforcement Performance Measures

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 24

Current and Planned Treasury Law Enforcement Performance 

Measures4

Dismantle Money Laundering Networks

Customs Value of Property Seized

Monetary Instrument Seizures

Reduce Counterfeiting and Other Financial Crime

Secret Service Actual Financial Crime Loss

Potential Financial Crime Loss Prevention

Counterfeit Passed per $ Million

Counterfeit Passed to the Public - Domestic

Counterfeit Passed to the Public - Foreign

Percent of Counterfeit Passed with Foreign Nexus

Percent of Cases Accepted for Federal Prosecution

Percent of Financial Crime with Foreign Nexus

U.S. Attorneys’ Assessments

Industry Response Survey

Customs Disruption of Fraudulent Trade Activities and

Organizations (Qualitative)

Deny the Smuggling of Illicit Drugs

Customs Narcotics Seized (pounds)

Number of Narcotics Seizures

Seizures from Transit Zone (pounds)

Number of Air Drops

Number of Short Landings

Number of no Launches

Response Rate to [Border Coordination Initiative]

Air & Marine Requests

Cross Border Air Smuggling Events

 

4

 Planned Measures are shown in bold Italics
Appendix 2

Current and Planned Treasury Law Enforcement Performance Measures

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 25

Support National Drug Control Strategy

Treasury Level of Anti-drug Related Targets Met

Deny Criminals Firearms and Reduce Violent Crime

ATF Crime Related Costs Avoided ($)

Number of Future Crimes Avoided

Number of Traces

Average Trace Response Time

Number of Persons Trained/Developed in the

Integrated Violence Reduction Strategy

Safeguard the Public From Arson and Explosive Incidents

ATF National Response Team Customer Satisfaction

Rating

Fight Terrorist Threats

Customs Outbound Licensing Violations

Stolen Vehicles

Disruption of Cybersmuggling Activities and

Organizations (Qualitative)

Disruption of International Trafficking (Qualitative)

Treasury Maximize Compliance with Sanction Programs

(Qualitative)

Number of Terrorist Incidents that Occur and the

Number Thwarted
Appendix 3

Management Comments

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 26
Appendix 3

Management Comments

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 27
Appendix 3

Management Comments

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 28
Appendix 3

Management Comments

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 29
Appendix 3

Management Comments

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 30
Appendix 4

Major Contributors to this Report

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 31

George W. Tabb, Regional Inspector General for Audit (Southern)

Gene Wendt, Audit Manager

Ken Coleman, Audit Manager

Claire Schmidt, Auditor 

Warren Wilson, Referencer
Appendix 5

Report Distribution

Better Performance Measures Are Needed For Treasury Enforcement 

Programs (OIG-02-013) 

Page 32

The Department of the Treasury

Under Secretary for Enforcement

Assistant Secretary for Enforcement

Assistant Secretary for Management and Chief Financial Officer

Office of Strategic Planning and Evaluations

Office of Accounting and Internal Control

United States Customs Service

Commissioner

Office of Planning and Evaluations

Secret Service

Director

Management and Organization Division

Bureau of Alcohol, Tobacco and Firearms

Director

Chief, Strategic Planning

Office of Management and Budget

OIG Budget Examiner