- AUDIT REPORT ON 


U.S. CUSTOMS SERVICE'S 


PERFORMANCE MEASURES 


FOR COMMERCIAL ACTMTY 


016-01-023 December 5,2000 

Office of Inspector General 

mmt 

UnitedStates Depaltrnent of the Treasury 
DEPARTMENT OF THE TREASURY 

WASHINGTON, D.C. 20220 

December 5, 2000 

OFFICE OF 

INSPECTORGENERAL 

MEMORANDUM FOR RAYMOND KELLY, COMMISSIONER 


UNITED STATES CUSTOMS ERVICE 


FROM: 

Assistant 

Dennis S. 

Inspector 

Schindel &lI-//

General for Audit 


* 

SUBJECT: Audit Report on U.S. Customs Service's 

Performance Measures for Commercial 


Activities 


This memorandum transmits the subject final audit report. 


This audit is the second in a series of planned audits 


covering various aspects of the U.S. Customs Service's 


(Customs) implementation of the Government Performance and 


Results Act (Results Act). 


Our audit found that while considerable progress has been 


made, implementation of the Results Act is still a work in 


process. The report discusses improvements needed in the 


performance measures, supplemental information, consistency 


between measures, and presentation of crosscutting issues. 


We have made 10 recommendations to assist Customs in its 


continuing effort to improve Results Act reporting. The 


actions taken and planned by Customs satisfy the intent of 


our recommendations. The complete text of your response 


dated September 8, 2000, and supplemental response dated 


October 5, 2000, are included as Appendix 4 and 5, 


respectively. 


We appreciate the courtesies and cooperation provided to our 


auditors during the audit. If you have any questions 


regarding this report, you may call me at (202) 927-5400, or 


a member of your staff may call Charles Mataya, Regional 


Inspector General for Audit (Houston), at (713) 706-4611. 


Attachment 

TABLE OF CONTENTS 

OVERVIEW .....................................................................................................................1 


BACKGROUND ................................................................................................................ 1 


OBJECTIVE. SCOPE AND METHODOLOGY ................................................................ 2
 . 
 AUDIT RESULTS.................................................................................................................4 


Both Overall and Significant Compliance Rates Are Necessary ............................... 4 


Recommendation 1 .................................................................................................... 5 


Management Response and OIG Comment............................................................... 5 


An Additional Measure for Facilitation of Trade and Travel is Necessary ............... 5 


Recommendation 2 .................................................................................................... 7 


Management Response and OIG Comment............................................................... 7 


. .
 Better Cost Information 1sNeeded.................1.......................................................7 


Recommendation 3 ....................................................................................................8 


Management Response and OIG Comment...........................................................8 


Recommendation 4 ................................................................................................8 


Management Response and OIG Comment 
..............................................................9

Consistency in the Definition and Development of Measures is Needed .................. 9 


Recommendation 5 ................................................................................................... 9 


Management Response and OIG Comment........................................................... 10 


Recommendation 6 .................................................................................................10 


Management Response and OIG Comment........................................................10 


Disclosure of Crosscutting Efforts is Needed ............................................................ 10 


Recommendation 7 ............................................................................................... 11 


Management Response and OIG Comment.............................................................. 11 


Recommendation 8 ................................................................................:...................12 


Management Response and OIG Comment............................................................... 12 


Additional Disclosures are Needed............................................................................12 


Recommendation 9 .................................................................................................... 13 


Management Response and OIG Comment...............................................................13 


OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page i 


FOR COMMERCUL ACTIVITIES 
1 

2 

3 

4 

5 

6 

7 

TABLE OF CONTENTS €

Recommendation 10 ............................................................................................. 13 


Management Response and OIG Comment............................................................. 13 


APPENDIX .Customs Current and Planned Performance Measures ................................ 14 


APPENDIX .Customs Performance Measures Recommended by OIG............................ 15 
 . 


APPENDIX .Program Logic Model for Customs Commercial Activities ........................ 16 


APPENDIX .Management Response................................................................................ 17 


APPENDIX .Supplemental Management Response.........................................................24 


APPENDIX .Major Contributors To This Report ............................................................ 26 


APPENDIX .Report Distribution...................................................................................... 27 


OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page ii €

FOR COMMERCIAL ACTIVITIES €
Overview 

This report presents the results of our audit of the U.S. Customs Service's 


(Customs) performance measures for commercial activities. The objective of the 


audit was to assess the relevance of the performance measures for commercial 


activities. Also, the audit evaluated Customs coordination with other agencies on 


crosscutting issues. Our audit found that the Government Performance and 


Results Act (Results Act) reports need to be more cle'ar, complete, and consistent. 


The Results Act reports do not contain all of the information that the Department 


of the Treasury (Treasury), Office of Management and Budget (OMB), and 


Congress-the principal stakeholders-need to evaluate Customs and to make 


informed budget decisions. 


Customs' three commercial processes have produced inconsistent reporting 


because of varied capabilities, resources, and systems. Essentially, there are six 


key performance measures to report the results of the three commercial 


processes.' However, to date Customs has been unable to produce reportable 


numbers for three of these key measures. (See Appendix 1) Reporting for the 


remaining three measures has also been inconsistent and incomplete. Our report 


recommends the addition of two performance measures and the elimination of one 


performance measure, as well as the consistent definition and reporting of 


measures. (See Appendix 2) We also recommended additional informative 


disclosures for re* clarity. 


This report also discusses the need for both overall and significant compliance 


rates to evaluate Customs performance, as well as a new performance measure 


needed to cover a key aspect of trade facilitation. Better reporting of cost 


information is needed to allow users of the Results Act reports to compare the 


resources devoted to programs with the results achieved. Consistent definitions 


and measures are needed across processes to make the reports more clear and 


usable. Further, more complete disclosure of crosscutting issues is needed. 


Background 

The Results Act was enacted to systematically hold Federal agencies accountable 

for achieving program results. Specifically, Federal managers are required to set 

strategic goals, measure performance, and report on their progress. In enacting the 

' The six key measures are the Compliance Rate, Cycle Tie, Targeting, Accuracy of Key Statistics, Unit 

Cost, and Customer Satisfaction. 

-

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 1 

FOR COMMERCIAL ACTIVITIES 
Results Act, Congress and the administration realized that the transition to resultsoriented management would not be easy. For that reason, the Results Act 

provided for a phased approach to implementation. The Results Act also 

emphasizes the importance of consultation with Congress as strategies are 

planned, goals are identified, and measures are selected. 

The purpose of the Results Act is to: 

Improve the confidence of the American people by holding Federal agencies 

accountable. 

Improve program effectiveness and public accountability by promoting a new 

focus on results, service quality, and customer satisfaction. 

Help Federal managers improve service delivery by requiring them to plan for 

meeting program objectives. 

Improve congressional decision making by providing more objective 

information on achieving statutory objectives. 

Improve internal management of the Federal government. 

Customs' performance for Fiscal Year (FY) 1999under the Results Act was 

presented in two reports: (1) US. Customs Service Accountability Report 

(Accountability Report) and (2) US.Customs Service FY 2001 President's 

Budget Justification Materials (Budget Justification). The Accountability Report 

combined the performance report and the financial statements. The Budget 

Justification combines the performance plan with the FY 2001 budget request. 

The budget is structured into two major activities: (1) commercial and (2) drug 

and other enforcement. The commercial activity is comprised of three core 

processes: (1) trade compliance, which is the commercial importation of . . 

merchandise; (2) passenger processing, which is the processing of passengers 

entering the country; and (3) outbound processing, which is the commercial . . 

export&on of merchandise and the processing of passengers leaving the country. 

During FY 1999, the most recent complete year of statistical data, Customs 

employed 19,000 personnel and had total budget resource obligations of $2.9 

billion. Customs collected $22.1 billion, processed 21.4 million commercial 

entries valued at an estimated $977 billion, and cleared 480 million passengers. 

Objective, Scope and Methodology 

The objective of the audit was to assess the relevance of performance measures 

for commercial activities. Also, the audit evaluated Customs' coordination with 

OIG-01423 CUSTOMS' PERFORMANCE MEASURES Page 2 

FOR COMMERCIAL ACTIVITIES 
other agencies on crosscutting issues. The audit included reviews of 


FY 1998 - 2001 Results Act reports issued as well as policies and initiatives 


through March 2000. Fieldwork was performed ffom July 1999 through March 


2000 at Customs Headquarters and its facilities in Houston, Texas. The following 


four questions were posed to evaluate the relevance of the performance measures 


for commercial activity. 


Do they present an overall picture of the perfbrmance for commercial 

activities? 

Do they sufficiently cover the key aspects of performance? 

Are they consistent between commercial programs? 

Are they clear and usable? 

Data was gathered from information derived from a prior Office of Inspector 

General (OIG) audit of Customs performance data for commercial operations.' 

We interviewed: (1) Customs officials at its Headquarters and in Houston, Texas; 

and (2) U.S. General Accounting Office (GAO) officials. We researched GAO 

reports, Results Act reports fi-omother agencies, news articles, regulations, and 

public research through extensive Internet queries. In order to assure the 

usefulness of our recommendations, we discussed our preliminary findings and 

recommendations with staff representing eight stakeholders: (1) 0MB; (2)Office

of Strategic Planning, Treasury; (3) Senate Committee on Appropriations; 

(4) Senate Committee on Finance; (5) Senate Committee on Governmental 

Affairs; (6)House Committee on Appropriations; (7) House Committee on Ways 

and Means, Subcommitteeon Trade; and (8) House Committee on Government 

Reform, Subcommittee on Government Management, Information and 

Technology. 

Our audit was conducted in accordance with Government Auditing Standards 

issued by the Comptroller General of the United States and included such audit 

tests as were determined necessary. 

Customs Performance Data for Commercial Activities (OIG-00-036; February 3,2000).

- -

OIG-01-023 CUSTOMS' PEWORMANCE MEASURES Page 3 

FOR COMMERCIAL ACTIVITIES 
Audit Results 

Both Overall and Sirrnificant Com~lianceRates Are Necessary 

Customs has identified the rate of compliance with applicable US. laws 

and regulations as the primary outcome measure for the three commercial 

processes. The compliance measure for the ogtbound process is based on 

the percentage of documents filed timely. It is neither an overall or 

significant compliance rate since it is not based on a statistical sample of 

all types of noncompliance. The trade and passenger processes have been 

reporting different types of compliance rates: (1) an "overall" compliance 

rate based on all discrepancies and (2) a "significant" compliance rate 

based only on significant discrepancies. When reporting trade activity, 

significant discrepancies are defined as those that have major transactional 

impact, such as narcotics seizures or additional duty collections. When 

reporting passenger activity, significant discrepancies are referred to as 

Category one3violations. 

Under certain circumstances, valid arguments can be made for using either 

the overall compliance rate, or the significant compliance rate. However, 

reporting both would give decision-makers a more complete picture of 

performance. Staff from the majority of the stakeholders with whom we 

met indicated that both the overall and significant compliance rates were 

needed to provide complete and usell information. Regardlessof the rate 

reported, users of the Results Act reports should be aware that external

factors can effect compliance rates. For example, economic conditions in 

the U.S. and other countries, as well as other factors, can also effect 

compliance. 

The outbound process will not be able to report compliance rates 

consistent with the trade and passenger processes until a statistical 

sampling plan is implemented. (See Recommendation 6) The trade and 

passenger processes have been moving in different directions on this issue. 

The Results Act reports for FY 1998 and prior used a significant 

compliance rate for passenger activity. In FY 1999, the passenger 

3 Category One - Drugs (above Zero Tolerance),Merchandise (over $1,250 Domestic Value), Prohibited 

Weapons and Explosive Devices (except Fireworks), Controlled (21 Code of Federal Regulations 

$1311.27) or Prohibited Medications (over 1,000 units), Undeclared Currency over $10,000, and Arrests 

(excluding Zero Tolerance). 

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 4 

FOR COMMERCIAL ACTIVITIES 
compliance rate was changed to an overall rate. The trade process has 

been reporting an overall compliance rate, but in FY 2000 plans to change 

to a significant compliance rate. 

Recommendation 1 

The Commissioner of Customs should include both the overall and significant 

compliance rates in the Results Act reports for all three of the commercial 

processes. 

Management Response and OIG Comment 

Customs management concurred with this recommendation. However, while the 

Trade and Passenger processes will be based on statistical samples, the Outbound 

projections will be based on enforcement actions. 

The actions planned by Customs satisfy the intent of this recommendation. 

An Additional Measure for Facilitation of Trade and Travel is 

Necessaty 

Customs has the dual mission of facilitating legitimate trade and travel and 

ensuring that all goods and persons entering and exiting the country do so 

lawfully. In implementing facilitation, Customs wants to reduce the 

regulatory burden on trade and travel, while ensuring compliance. Also, 

Customs attempts to reduce the time that people, cargo, and conveyances 

wait to be cleared. The cycle time performance measure is intended to 

measure the wait time. However, there is no measure in the performance 

plan that recognizes the regulatory requirements, or burden that Customs 

places on international trade and travel. The regulatory functions of 

inspections, audits, investigations, rulings, and regulations all impose 

burdens on importers, exporters, carriers, brokers, and travelers. These 

requirements include the filing of forms, posting of bonds, retention of 

records, payments of duties and taxes, and issuance of licenses. Therefore, 

a performance measure is necessary to evaluate the results of activities in 

this key aspect of Customs' facilitation mission. 

The cost of compliance could be used as an outcome measure of the 

regulatory burden imposed on international trade and travel. This would 

measure the cost to the trade community to comply with Customs 

016-01-023 CUSTOMS' PERFORMANCE MEASURES Page 5 

FOR COMMERCIAL ACTIVITIES 
-- -- -- 

requirements. For example, it would estimate the total dollar cost per 

entry incurred by importers to comply with importation requirements. 

There are approximately 445,000importers who file over 20million 

entries a year. The measure could be a barometer reflecting the effect of 

new laws and regulations on the facilitation of trade, and measure the 

impact of new Customs initiatives and systems on the trade community. 

Other Treasury bureaus currently recognize regulatory burden on the 

public. FOTexample, the Bureau of Alcohol, Tobacco and Firearms, and 

the Financial Crimes Enforcement Network both have performance 

measures which reflect regulatory burden. Also, the Internal Revenue 

Servicepublishes time estimates to complete forms in their publications 

used by taxpayers. Stakeholder representatives we consulted with agreed 

that cost of compliance was a useful measure. 

Stakeholders also suggested that the data be collected from private 

industry in a manner that did not create an undue burden. Some Customs 

managers were concerned that it may not be possible to collect reliable and 

verifiable datafrom importers and exporters without placing an undue 

burden on them. Managers were also aware that there is no legal 

requirement for companies to provide this proprietary information. 

Further, since Customs and the Bureau of the Census share responsibilities 

in the outbound area, it may be difficult for some companies to segregate 

the Customs costs. 

The datacollection process would require only two data inputs, the 

number of entries filed and the cost of meeting all associated Customs 

requirements. For most one time and very small importers, the cost would 

simply be their brokers fees. It will be necessary to assure importers that 

the datacollected would be strictly confidential and would not be used for 

any other purposes. There are benefits to companies for providing the cost 

data First, since this will be a key performance measure, Customs will be 

actively working to reduce importer and exporter costs. Second, it will 

allow the trade community to benchmark their costs against industry 

averages. Finally, it will provide stakeholders with important information 

when considering new laws and regulations that may significantly raise 

industry costs. 

There are several approaches that could be used to obtain this information. 

Specifically: 

-

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 6 

FOR COMMERCIAL ACTIVITIES 
Customs could request that importers and exporters volunteer to 

paitner with them to form focus groups to provide a basis for the 

measure. 

Customs could work with trade associations to define the cost 

elements and estimation techniques to assist in the accumulation of 

information. For example, a stratified sampling plan could be 

developed that would require the cost data fxom about 500 

importers annually. Further, the sampling plan could be developed 

so that no individual importer would be requested to provide data 

more than once every 5 years. 

Customs could seek a legislative requirement for importers to 

provide cost data and allow for a method to verify data reliability. 

Recommendation 2 

The Commissioner of Customs should evaluate the alternatives for measuring 

regulatory burden and implement the most practical measure. 

Management Response and OIG Comment 

Customs concurred with this recommendation. As stated in its supplemental 

management response (see Appendix 5), Customs agreed that an assessment of 

regulatory burden can be a usell measure. However, due to legal and operational 

issues that are problematic for Customs, it will meet with external stakeholders to 

discuss how best to proceed with this recommendation. 

Customs' planned actions satisfy the intent of this recommendation. 

Better Cost Information is Needed 

The Results Act reports reflect total budget and personnel full time

equivalents (FTE) for commercial activity. However, these numbers are 

not separately reported by trade, passenger, and outbound processes. 

Reporting the total cost and FTEs up fxont will put the performance 

measures that follow in proper perspective. This will be especially 

informative because Customs reports the data for prior year actuals, 

current year plans, and the following year's proposed plan in a tabular 

OIG-01-Q23 CUSTOMS' PERFORMANCE MEASURES Page 7 

FOR COMMERCIAL ACTIVITIES 
format. The budget should be reported for current and future years and the 

cost for prior years. This will allow users of the Results Act reports to 

compare the resources devoted to programs with the results achieved. 

Performance goals are of minimal value for congressional appropriations 

decisions without a connection to the resources requested. Staff from 6 of 

the 8 stakeholders with whom we met stated that this would be useful 

information. e 

Customs has a unit cost performance measure. However, the Results Act 

reports to date have not included any data for this measure. The unit cost 

was initially designed as an efficiency measure for balanced reporting. It 

was to be a cost per entry or passenger. However, the unit cost was 

changed from a dollar amount to a ratio of current costs to a baseline year. 

The calculation of the ratio will use an index of 1to 10 with 5 being a 

baseline year. The unit cost ratio may be overly complicated and may not 

be useful for decision making. None of the staff representing the eight 

stakeholders with whom we met indicated that this measure would be 

useful to them. 

Recommendation 3 

The Commissioner of Customs should direct the Office of Finance to include the 

cost and FTEs for the trade, passenger and outbound processes in the Results Act 

reports. If future Results Act reports change the presentation of performance 

measures, then Customs should consult with its key stakeholders to determine the 

cost breakout that would be most useful to them. 

Management Response and OIG Comment 

Customs management concurred with this recommendation. Customs will report 

cost data consistent with the current programs. 

The actions planned by Customs satisfy the intent of this recommendation. 

Recommendation 4 

The Commissioner of Customs should eliminate the reporting of the unit cost 

performance measure. 

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 8 

FOR COMMERCIAL ACTIVITIES 
Management Response and OIG Comment 

Customs management concurred with this recommendation and will eliminate the 

unit cost performance measure. 

The actions planned by Customs satisfy the intent of -this recommendation. 

Consistencv in the Definition and Development ofMeasures is 

Needed 

The performance measures for the three programs in commercial activities 

were developed independently. As a result, there is inconsistency between 

the measures reported for each, and in the definitions of measures. There 

is one targeting performance measure for the passenger process, a different 

measure for the outbound process, and none for the trade process. The 

definitions as to what are included in the compliance rates are different for 

all three processes. Consistency in the measures used and in definitions is 

necessary to make the Results Act reports more clear and useful to 

stakeholders. All of the stakeholders with whom we met indicated that 

consistency would make the reports clearer and more useful to them. 

The compliance and targeting measures are different for the outbound 

process because they are not based on national random samples. 

According to Customs, resources are not available to take samples by port 

for the outbound process such as those taken for the trade and passenger 

processes. However, a much smaller sample could be taken for the 

outbound process by taking a random sample of transactions at statistically 

selected ports and time intervals. This sample could be projected 

nationally and used for annual performance measures. 

Recommendation 5 

The Commissioner of Customs should direct commercial programs to consistently 

apply common performance measures and definitions across all three programs 

where possible. Where this is not possible, the reasons for the differences should 

be explained in the performance report. 

OIG-01-023 CUSTOMS' PERFORMANCE MEASUEaES Page 9 

FOR COMMERCIAL ACTIVITIES 
-- 

ManagementResponse and OIG Comment €

As stated in its supplementalmanagement response (see Appendix 5), Customs 

management -€ concurred with this recommendation. Measurements and definitions

will be applied consistently across all three of the Commercial processes. Where 

this is not possible, a footnote explaining the differences will be included in the 

report. -€

The actions planned by Customs satisfy the intent of this recommendation. 

Recommendation 6 

The Commissioner of Customs should direct the design of a sampling plan for the 

outbound process that can be projected nationally. 

ManagementResponse and OIG Comment €

As stated in its supplemental management response (see Appendix 5), Customs 

management concurred with this recommendation. Currently, Customs does not 

have sufficient resources to conduct a national Outbound sampling measurement 

program. Customs will identify the resources required and request them in the FY 

2003 budget process. 

The actions planned by Customs satisfy the intent of this recommendation. 

Disclosure of Crosscuttinz Efforts is Needed 

Interagency coordination is important for ensuring that crosscutting 

program efforts are mutually reinforcing and efficiently implemented. The 

Results Act reports should identify crosscutting programs with other 

agencies. For those identified, complementary performance goals should 

indicate how differing program strategies are mutually reinforcing and 

complementary performance measures should be set. Also, the reports 

should briefly describe planned coordination strategies. 

In an audit report on FY 2000 Federal performance plans, GAO identified 

coordination of crosscutting efforts as a key weakne~s.~Another GAO 

' Manngingfor Results: Opportunities for Continued Improvements in Agencies 'Performance Plans 

(GGDIAIMD-99-215; July 1999). 

OIG-02-023 CUSTOMS' PERFORMANCEMEASURES Page 10 €

FOR COMMERCIAL ACTIVITIES €
-- -- 

report - on barriers to interagency -. coordination identified border

inspections, drug control, investigative authority, drug traflicking, and 

combating terrorism as areas of fragmentation and overlap.' 

Customs' Results Act reports do not contain a listing of crosscutting 

efforts. 0MB CircularA-I 1 requires that at a minimum, the annual plan 

should indicate those crosscutting programs undertaken with other 

agencies. The Circular fixther states that an iindex displaying agencies and 

the relevant programs or activities may be a useful format. The table of 

crosscutting programs and activities grouped by agency from the 

Department of Commerce performance plan is an example of a best 

practice in this area. 

Customs provides import and export data to the Bureau of the Census, 

which is used to prepare the published trade statistics. However, Customs 

Results Act reports do not discuss complementary performance goals and 

measures with those of the Bureau of the Census dealing with the trade 

data. Currently, Customs has a performance measure "Accuracy of Key 

Statistics" for trade, but none for outbound. This is a crosscutting issue 

that should involve complementary performance measures between the 

Bureau of the Census and Customs' trade and outbound processes. 

Recommendation 7 

The Commissioner of Customs should include a listing of all crosscutting efforts 

with other agencies in the performance plan. 

Management Response and OIG Comment 

Customs management concurred with this recommendation and will follow the 

requirements of OMB Circular A-11.

The actions planned by Customs satisfy the intent of this recommendation. 

Managingfor Results: Barriers to Interagency Coordination (GGD-00-106;March 2000). 

- -

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 11 

FOR COMMERCIAL ACTIVITIES 
Recommendation 8 

The Commissioner of Customsshould contact the Bureau of the Censusto discuss €

mutually reinforcing program strategies and to determine if complementary €

performancemeasuresshould be reported. €

Management Response and OIG Comment 

Customs management concurred with thisrecommendation. €

The actions planned by Customssatisfy the intent of this recommendation. €

Additional Disclosures are Needed 

The Results Act is a phased and iterative implementationprocess. 

Ultimately, the Results Act reportsshould show the benefits to the public 

forthe resources expended. Staff from one stakeholderwith whom we 

met felt that the compliance rates were output rather than outcome 

measures. To avoid such confusion and to showwhere the 

implementation process is heading, Customs needs to describe the logic of 

how individualprograms use inputsin program componentsto produce 

outputs and outcomes. A program logic model, such asthe simplified 

example in Appendix 3, could be used to show the progression of how 

inputs are converted into outputs, initial outcomes, intermediate outcomes, 

and end outcomes. This would help external stakeholderssee how 

changes in resources and program componentsmight better impact 

outcomes. Also, this would provide a basis for discussing the long-term 

direction of the Results Act reports. 

A significant challengeto Customsin implementingthe Results Act is 

obtaining-sufficientreliable data for the *

~erformancemeasures. Of the 19 

performance measures currently identified in the Results Act reports, 

reportable data are not available for 9 measures. (See Appendix 1) The 

Results Act reports do not fully discuss the problems in obtaining the data, 

and do not provide estimatesof when it is anticipated that the data for 

these measureswill be reported. 

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 12 

FOR COMMERCIAL ACTIVITIES 
Recommendation 9 

The Commissioner of Customs should include a program logic model and 

discussion of the long-term plans for performance reporting in the Results Act 

Reports. 

Management Response and OIG Comment -

Customs management concurred with this recommendation. Customs has no 

objection to including a program logic model in future reports. 

The actions planned by Customs satisfy the intent of this recommendation. 

Recommendation 10 

The Commissioner of Customs should include a discussion of the problems in 

obtaining- the data and an estimate of when the data will be reported for those 

performance measures for which reliable data are not currently available. 

Management Response and OIG Comment 

Customs management concurred with this recommendation. 

The actions planned by Customs satisfy the intent of this recommendation. 

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 13 

FOR COMMERCIAL ACTIVITIES 
-- 

Appendix 1 €

Customs Current and Planned Performance Measures €

TRADE €

Overall compliance rate 

Primary Focus Industry (PFI) 

compliance rate 

Revenue compliance rate 

North American Free Trade 

Agreement (NAFTA) compliance 

Cycle time 

Accuracy of key statistics 

Unit cost 

Custonwr satisfaction 

PASSENGER €

Overall compliancerate 

Cycle time 

Targeting efficiency 

Unit cost 


Customer sntisfaction 


Advanced Passenger Information 

System (APIS) rate

OUTBOUND €

Overall compliance rate 

Targeting effectiveness 

Unit cost 

Customer satisfaction 

Automated Export System (AES) 

participation 

FY 1999 Performance measuresreported (in black) 

Perfornzance nwasures lisled in pl~n;Iio~~eve); nunzbers are rialye/ uvnilahle (in red ond Nulics) 

Performance measures and numbers Customs plans to include in future years (in blue and bold) 

OIG-01-023 

CUSTOMS' PERFORMANCE MEASURES Page 14 

FOR COMMERCIAL ACTIVITIES 
Appendix 2 

Customs Performance Measures Recommended by the OIG 

-
 TRADE 

Cost - FTEs 

Performance Measures 

Overall compliance rate 

Significant compliance rate 

Revenue compliance rate 

Cost of contpliance 

Cycle time 

Targeting effectiveness 

Accuracy of trade statistics 

Customer satisfaction 

Interim Measures 

Automated Commercial 


Environment 
(ACE)/Mod Act

PFI compliance rate €

NAFTA compliance rate €

PASSENGER 

Cost - FTEs 

Overall compliancerate 

Significant contpliance rate €

Cost of compliance €

Cycle time €

Targeting effectiveness €

Customer satisfaction 

APIS rate 

OUTBOUND 

Cost - FTEs 

Overall compliance rate 

Sign~jkantcompliance rate 

Cost of compliaace 

Cycle time €

Targeting effectiveness €

Accuracy of trade statistics 

Customer satisfaction 

AES participation 

Customs current and planned performance measures (in black) 

Performance measures and itrforntation recontntended by the OIG (in green and Italics) 

Measure should be developed as ACE is funded (in blue and bold) 

CUSTOMS' PERFORMANCE MEASURES Page 15 

FOR COMMERCIAL ACTIVITIES 
Appendix 3 

Program Logic Model for Customs Commercial Activities 

Program Outputs 

Components 

Goods and 4 Inspections, 

people Informed 

entering and compliance, 

leaving the Enforced 

United States compliance, 

clear Customs Audits, 

Investigations 

Initial Intermediate End 

Outcomes 

Deterrent effect + 


reduces 


intentional 


noncompliance 


and better 


communication 


reduces 


unintentional 


noncompliance 


-

Outcome Outcomes 

Compliance American 

rates public is 

improve protected 

from harmful 

products and 

the American 

economy is 

protected 

from unfair 

practices 

OIGO1-023 CUSTOMS' PERFORMANCE MEASURES Page 16 

FOR COMMERCIAL ACTIVITIES 
Appendix 4 


MANAGEMENT RESPONSE 

DATE: September 8. 

FILE: AUDI-OP MD 

MEMORANDUM FOR DENNIS SCHINDEL 

ASSISTANT INSPECTOR GENERAL 

FROM:  Director, 

Office of Planning 

SUBJECT:  Draft Audit Report on the Customs Service 

Performance Measures 

Thank you for providing us with a copy of your draft report entitled 

"Customs Performance Measures for Commercial Activities' and the 

Page 1 of 7 

2000 

opportunity to discuss the issues in this report. In general, we agree with 

the recommendationstated and have taken steps to address issues 

noted. However, we did find some instanceswithin the report, which 

require clarification. 

We have determined that the information in the audit does not warrant 

protectionunder the Freedomof InformationAct.

Attached are our comments regarding the draft report. Ifyou have any 

questions regarding the attached comments, please have a member of 

your staff contact Ms. Brenda Brockman at (202) 927-1507.

Attachment 

TRADITION 

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 17 

FOR COMMERCIAL ACTIVITIES 
Appendix 4 

MANAGEMENT RESPONSE Page 2 of 7 

U. S. Customs Service 

Commenhi on OIG Draft Report "Customs Performance Measures 

for CommercialActivities" 

" 

Recommendation 1 

The Commissioner of Customs shouM include both the overall and 

significant compliance rates in the Results Act reports for all three of the 

commercial processes. 

Comment 

We wncur with this recommendation. Trade and Passenaw Commercial 

processes currently collect statistics for both overall and Ggniftcant 

compliance rates. Outbound suooorts this recommendation to the extent 

that is understoodthat pmjectibnswill be based on enforcement actions 

and not on statistically valid random samples. 

Outbound lacks sufficient resources to measure compliance in the same 

meaningfulway as does Trade Compliance and Passenger programs. 

Enforcementand compliance activity can be reported as overall and 

significant. 

Recommendation 2 

The Commissioner of Customs should evaluate the alternatives for 

measuring regulatory burden and implement the most practical measure. 

Comment 

We wncur that an assessment of regulatory burden can be a useful 

measure in the context of overall customer satisfaction. A stand-alone 

measurement on regulatory bden is not acceptable because collection 

of data to satisfy this measure will carry increased cosk and burden to 

the trade The traoe wmmuniw vlews cost information as sensitive 

proprietary information, the submissionof which is burdensome. 

Customs has no leaal authoritv to make irnoorters ~rovidecost data. 

There is not an automated &em in place'to colle'ct reliable data. 

Our customer satisfaction measurement approach has been fully 

accepted by all parties potentially affectedbythis measure consistent 

wh the intent of the Results Act. We believe this approach permik the 

-

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 18 

FOR COMMERCIAL ACTIVITIES 
-- 

Appendix 4 

MANAGEMENT RESPONSE Page 3 of 7 

customer and Customs the opportunityto service a variety of issues that 

may be affecting the process. We are already required to provide an 

assessment of regulatory burden costs when we initiate new reaulatow 

requirements. W; believe these costs are better idenmed at th; - beginning of a regulatory process instead of afler. 

Recommendation3 

The Commissioner of Customs should direct the Office of Finance to 

include the cost and FTE's for the trade, passenger and outbound 

Drocesses in the ResultsAct reoorts. If future Results Act reDorts chanae 

the presentation of performanci measures, then Customs siould consult 

with its key stakeholders to determine the cost breakout that would be 

most useful to them. 

Comment: 

We concur with thi recommendation. The cost data will be portrayed 

consistent with the current programmatic breakouts. Any proposed 

change with the current presentation cannot be made available until the 

FY2003 budget submission, at the earliest. In addition. we may not be 

able to retroactively provide a crosswalk between the cument 

programmatic presentations and proposed changes. 

Recommendation 4 

The Commissioner of Customs should eliminate the reporting of the unit 

cost perfomlance measure. 

Comment 

We concur with this recommendation. 

Recommendation 5 

The Commissioner of Customs should direct commercial programsto 

consistentlv . a~~lv ... common oerformance measures and definitions across 

all three programs where pdssib~e.Where this is not possible, the 

reasons for the differences should be explained in the performance 

report. 

Comment: 

We do not support this recommendation. Where possible, measurement 

and definitions have been developed consistently across all three 

programs. It should be noted, however, reporting requirements for export 

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 19 

FOR COMMERCIAL ACTIVITIES 
MANAGEMENT RESPONSE €

Appendix 4 €

Page 4 of 7 €

goods are not equivalent to the enby requirements of the Passenger 

Process or the Trade Process. Additionally, within the Processes, the 

regulatory and statutory requirements change with the mode of 

transportation. We have similar perfonance measures for Trade, 

passenger and Outbound pmce&ing: 

Targeting Efficiency Rate (Trade-Passenger-Outbound) 


Verification Rate (Trade) 


Apprehension Rate (Tradeassenger) 


InconvenienceRate (Trade) 


Cycle Time (Passenger) 


The Trade compliance pmcess is more complex than passenger 

processing is. This complexity must be taken into consideration when 

comparingresults such as targeting efficiency rates. To avoid comparing 

apples to oranaes each process has measures that best speak to their 

p&orrnance --this approach best serves the managers oithe Customs 

Service. In addition to the challenge presentedby differences among the 

processes, it must be emphasized that current automation does not 

permit Customs to segregate trade data on targeted shipments from 

shi~mentsexamined or reviewed onhr because of leaal mandates or the 

ne& to verify compliance improve&nts. ~evelopm\ntof an accurate 

targeting efficiency rate is dependent on further automation 

advancements in line w~thtne Automated Commercial Environment 

initiative 

Recommendation 6 

The Commissioner of Customs should direct the design of a sampling 

plan for the outbound process that can be projected nationally. 

Comment: 

We suooort this recommendation in theorv. however. there are currentlv 

insuffiCkntresources dedicated to the 0uibound PI&XS to undertakes 

sampling program large enough to be statistically valid, comparable to the 

Trade Compliance Proaram. Customs has undertaken several attempts 

to obtain s&stically vzid samples for significantviolations 

These sampling programs have been determined to be insufficientbased 

on the lack of resourcesto consistently collect the data on a large enough 

universe of exports. Because of a lack of resources. Outbound has taken 

alternative actions. 

Baselines for compliance issues can be collected for general violations. 

and can be projected nationally. These violation types include SED 

CUSTOMS' PERFORMANCE MEASURES Page 20 

FOR COMMERCIAL ACTMTIES 
- - 

Appendix 4 

MANAGEMENT RESPONSE Page 5 of 7 

submission, timeliness of submission and compliance with manifest 

requirements. The area of compliance that cannot be measured is linked 

to significant enforcement activities, such as currency smuggling. 

automobile theft, etc For requlatorv issues. compliance measurement 

can use a statistically valid method&gy, pr&cribed by the Office of 

Stratesic Trade (OST). ORbo~ndhas worked with OST in developha 

statistkally valid'me~odologyfor vessel, air, and truck in conducting ­ comoliance measurement. Outbound will continue to work with OSTto 

develop the frequency of the compliance measurements submitted by the 

field (i.e. quarterly, semi-annually, or annually). 

Recommendation7 

The Commissioner of Customs should include a listing of all crosscutting 

efforts with other agencies in the performance plan. 

Comment: 

We concur with this recommendation and will follow the requirements of 

OM6 Circular A-1 I. 

Recommendation8 

The Commissioner of Customs should contact the Bureau of the Census 

to.discuss mutually reinforcing program strategies and to determine if 

complementary performance measures should be reported. 

Comment: 

We concur with this recommendation. Customs has worked with Census 

to develop measures and the result is the one-labeled "Accuracy of Key 

Statistics". For the year 2000. the data to be used will be the overall 

Compliance Measurement rate. We will continue to work with Census to 

synchronize our statistical accuracy measures. 

The Commissioner of Customs should include a program logic model and 

discussion of the long-term - plansfor ~erformancerepoltin in the Results

Act Reports. 

Comment: 

We concur with this recommendation. Customs has no objection to the 

inclusion of a program logic model in the performance reports prepared 

by the agency. 

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 21 

FOR COMMERCIAL ACTIVITIES 
Appendix 4 

MANAGEMENT RESPONSE Page 6 of 7 

c-ent 

We wnsur with this nwhmsndntion.

016-01-023 CUSTOMS' PERFORMANCE MEASURES 

FOR COMMERCIAL ACTIVITIES 

Page 22 
Appendix 4 


MANAGEMENT RESPONSE Page 7of 7 

GENERAL COMMENTS 

The OIG draft report states on page 1that The Results Act was enacted 

to systematically hold Federal agencies accountable for achieving 

program results." Its purpose is much wider. The law states the purpose 

of the Act is to: 

-	 improve the confidence of the American people by holding Federal 

agencies accountable; 

lmprove program effectiveness and public accountability by promoting 

a new focus on results, service quality, and customer satisfaction; 

Help Federal managers improve.service delivery by requiring them to 

plan (realistically) for meeting program objectives; 

lmprove congressional decision making by providing more objective 

information on achieving statutory objectives; and 

Improve internal management of the Federal government. 

This expanded statement of purpose for the Results Ad is	important to 

note in the OIG draft report. 

On page 2, the OIG draft report states that Customs had an operating 

budget of 2.9 billion. This is not accurate, due to the way user fee 

revenues are presented in our budget. In actuality, it is more like $2.4 

billio-n. It also states that Customs processed 21.4 million entries. This is 

correct if the word "commercial" is inserted before entries. Customs 

processes several million informal entries as well. 

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 23 

FOR COMMERCIAL ACTIVITIES 
Appendix 5 

SUPPLEMENTAL MANAGEMENT RESPONSE Page 1 of 2 

U.S. Customs Service 

Memorandum 

DATE: Oclober 5,2000 

FILE: AUDl-OP MD 

MEMORANDUM FOR DENNIS SCHINDEL 

ASSISTANT INSPECTOR GENERAL 

FROM:  Director. 

Office of Planning 

SUBJECT: 	 Draft Audit Report on Customs Performance 

Measures for CommercialActivities 

Members of the Offce of Field Operations. Office of Strategic Trade and 

the Office of Planning met with a representativefrom your office to

address issues regarding our comments to your draft report, subject as 

above. As a result of the meeting, Cudoms agreedto modify its 

.response to Recommendation Numbers 2,5 and 6. 

If you have any questions regarding the attached response, please have 

a member of your staff contact Ms. Brenda Brockman at (202)927-1507. 

Attachment 

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 24 

FOR COMMERCIAL ACTIVITIES 
Appendix 5 

SUPPLEMENTAL MANAGEMENT RESPONSE Page 2 of 2 

U.S. Customs Service 

Supplemental Response to OIG Draft Report 

"Customs Performance Measures for Commercial Activities" 

Recommendation#2: Please revise our response to read: 

We concur with this recommendation and agree that an assessment of 

regulatow burden can be a useful measure in the context of overall 

customer satisfaction. However, the legal and operational issues 

surroundmg this recommendation are problematic for Customs. We will 

work with the Office of Congressional Affalrs to arrange a meeting with 

our external stakeholders to discuss how best to proceed with this 

recommendation. 

Recommendation#5: Please revise our response to read: 

We concur wtth this recommendation. Where possible, measurements 

and definitions wtll be applled consistently across all three of the 

Commercial processes. When measures are not cons~stentlyappl~ed

across all three programs, a footnote explaining all differences will be 

included in the measurement reporting documentsRecommendation#6: Please revise our response to read: 

We concur with thrs recommendation. However, there are currently 

msufficient resources in the Outbound process to undertake any kind of 

national sampling measurement program. In the FY 2003 budget 

process. Customs will ident~fythe resources required to conduct such a 

sampling program. When the resources necessary are available. 

Customs will implement a sampling measurement program for Outbound 

programs. 

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 25 

FOR COMMERCIAL ACTIVITIES 
-- 

-- 

Appendix 6 

MAJOR CONTRIBUTORS TO THIS REPORT Page 1 of 1 

Charles Mataya, Regional Inspector General for Audit (Houston) 

Kenneth W. Coleman, Audit Manager -

Gene Wendt, Auditor 

Clara J. Veal, Referencer 

- -

OIG-01-023 CUSTOMS' PERFORMANCE MEASURES Page 26 

FOR COMMERCIAL ACTIVITIES 
Appendix 7 


REPORT DISTRIBUTION Page 1 of 1€

DEPARTMENT OF THE TREASURY 


Under Secretary for Enforcement 


Assistant Secretary (Enforcement) 


Office of Accounting and Internal Control 


Office of Strategic Planning and Evaluations 


U.S. CUSTOMS SERVICE 


Assistant Commissioner, Office of Field Operations 


Assistant Commissioner, Office of Strategic Trade 


Director, Office of Planning and Evaluation 


Director, Office of Management Inspection 


OFFICE OF MANAGEMENT AND BUDGET 


OIG Budget Examiner 


016-01-023 CUSTOMS' PERFORMANCE MEASURES Page 27 

FOR COMMERCIAL ACTIVITIES