DEPARTMENT OF HOMELAND SECURITY 

Office of Inspector General 

Audit of the Automated Commercial 

Environment Secure Data Portal: 

Management Controls Needed 

Improvement
	

Office of Audits 

OIG-04-35 September 2004 

Washington, DC 20528

Clark Kent Ervin 

Inspector General 

Office of Inspector General 

U.S. Department of Homeland Security 

Preface 

The Department of Homeland Security (DHS) Office of Inspector General (OIG) was established 

by the Homeland Security Act of 2002 (Public Law 107-296) by amendment to the Inspector 

General Act of 1978. This is one of a series of audit, inspection, investigative, and special reports 

prepared by the OIG as part of its DHS oversight responsibility to identify and prevent fraud, 

waste, abuse, and mismanagement. 

This report assesses the strengths and weaknesses of the program or operation under review. It 

is based on interviews with employees and officials of relevant agencies and institutions, direct 

observations, and a review of applicable documents. 

The recommendations herein, if any, have been developed to the best knowledge available to the 

OIG, and have been discussed in draft with those responsible for implementation. It is my hope 

that this report will result in more effective, efficient, and economical operations. I express my 

appreciation to all of those who contributed to the preparation of this report. 

Contents
	

Abbreviations...................................................................................................................................2
	

Introduction......................................................................................................................................3
	

Results in Brief ................................................................................................................................4
	

Background......................................................................................................................................4
	

Process For Gathering User Feedback.............................................................................................5
	

Process to Monitor User Feedback ..................................................................................................6
	

Other Matters ...................................................................................................................................7
	

Appendices 

Appendix 1: Purpose, Scope, and Methodology.....................................................................8 

Appendix 2: Recommendation ...............................................................................................9 

Appendix 3: Overview of ACE Releases..............................................................................10 

Appendix 4: Major Contributors To This Report .................................................................11 

Appendix 5: Report Distribution ..........................................................................................12 

Automated Commercial Environment Secure Data Portal 1
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Contents
	

Abbreviations 

ACE Automated Commercial Environment 

AD Anti-Dumping 

ATS Automated Targeting System 

CBP Customs and Border Protection 

CCIE Cargo, Conveyance, Individual and Equipment 

CFO Chief Financial Officer 

CROSS Customs Rulings Online Search System 

CVD Countervailing Duty 

eCP e-Customs Partnership 

EFT Electronic Funds Transfer 

FCC Federal Communications Commission 

FDA Food and Drug Administration 

FMCSA Federal Motor Carrier Safety Administration 

FOIA Freedom of Information Act 

FTZ Foreign Trade Zone 

HTS Harmonized Tariff Schedule 

IASS Importer Activity Summary Statement 

ITC International Trade Commission 

MARAD Maritime Administration 

NTC National Targeting Center 

OIG Office of Inspector General 

PTR Product Trouble Report 

TBD To Be Determined 

2 Automated Commercial Environment Secure Data Portal 
OIG Audit

Report Department of Homeland Security 

Office of Inspector General 

Introduction 

The Automated Commercial Environment (ACE) Secure Data Portal is a system 

that will allow the private sector trade community to submit data that meets 

all federal requirements for cargo entering and leaving the United States. The 

development of ACE is a massive and multifaceted effort directly related to the 

success of the Customs and Border Protection (CBP) mission. According to CBP, 

ACE will enable CBP to: 

• 

• 

• facilitate cargo processing by permitting the transfer of automated entry, 

entry summary, and manifest information in one standard data set for all 

modes of transportation. 

The vision of ACE is to create a single portal1

 for all Federal requirements for 

international cargo. This will provide the trade community a single web-based 

interface to make periodic payments, post transactions, and view statement 

records by account. This will benefit the federal government by creating a 

common knowledge-based risk management system for joint enforcement 

targeting and intelligence development. 

The objective of the audit was to determine CBP was managing and developing 

the ACE Secure Data Portal to meet user expectations in the areas of user 

feedback and the change request process. The audit work was performed from 

April to November 2003. This report and the two prior reports issued, taken 

together, represent the results of the audit of the ACE Secure Data Portal. 

Our first report addressed a critical contract deliverable and technical reporting 

issues. CBP concurred with our two recommendations relating to acceptance of 

the foundational activities in the deliverable and improving technical reporting. 

(Audit Report Number OIG-01-04, titled Audit of the Automated Commercial 

process and monitor import and export shipments and related trade activity 

more efficiently through trade accounts versus individual transactions; 

 release cargo more efficiently by integrating international law 

enforcement intelligence, commercial intelligence; and data mining results 

to focus efforts on high-risk importers and accounts; and 

1 A “portal” is software that provides a controlled gateway to an internet site. 

3
Environment Secure Data Web Portal: Quality of Deliverables Can Be Improved, 

dated November 25, 2003) 

Our second report addressed security concerns and was issued for official 

use only. (Audit Report Number OIG-04-22, titled Audit of the Automated 

Commercial Environment Secure Data Web Portal: Information System Security 

Requirements Need To Be Implemented, dated May 24, 2004) 

Results in Brief 

The process used by e-Customs Partnership (eCP) to gather user feedback was 

adequate. During a 90-day pilot, eCP established a logical process to gather 

user feedback. This process resulted in the initiation of Product Trouble Reports 

(Trouble Reports) and change requests. 

However, the CBP Modernization Office was not tracking all the user feedback 

collected by eCP. A monitoring process would allow the CBP Modernization 

Office to review, evaluate, monitor, and track user feedback to ensure that issues 

important to CBP are properly addressed. 

In addition, change request packages did not always have the required 

documentation identifying how changes would impact the program. As a result, 

the Change Control Board and Project Directors approved work requirement 

changes without knowing the full impact of the changes to the program. 

Background 

CBP enforces the trade laws of the United States while also facilitating legitimate 

trade and travel. In performing its mission, CBP counters the dual threats of 

narcotics smuggling and terrorist infiltration. The ability of CBP to process 

the growing volume of imports, while improving compliance with trade laws, 

depends heavily on successfully improving the trade compliance process 

and modernizing supporting automated systems. In August 2001, eCP, the 

prime contractor for the development of ACE, started work. Currently, eCP is 

scheduling the full deployment of ACE by December 2007 at an estimated cost of 

$2.24 billion. 

The ACE development coalition, eCP, is delivering ACE in a series of seven 

releases. Each release provides added functionalities and capabilities. (See 

4 Automated Commercial Environment Secure Data Portal 
Appendix 3 for an Overview of Ace Releases.) As of May 2004, eCP had 

deployed Release 22

, which established the initial infrastructure upon which all 

subsequent ACE functionality would rest and provided access to CBP account 

managers and 41 companies who import merchandise into the United States. 

CBP accepted delivery of Release 2 in October 2003 and started testing the 

system with CBP account managers and 41 companies to gather feedback of 

system functionality. Release 3 will increase the number of trade accounts and 

provide monthly statements and payments. Release 3 will also deliver the ACE 

infrastructure through which internal and external users will access revenue, 

sensitive law enforcement, and proprietary corporate information. Release 

3, which allows importers and designated brokers to make periodic monthly 

payments for monthly statements of duties and fees, is scheduled for deployment 

in the summer of 2004. 

Process For Gathering User Feedback 

eCP established an adequate process to gather user feedback needed to help 

initiate Trouble Reports and change requests during the Release 2 pilot 

performance period. The feedback gathered by this process also provided 

additional information to improve future releases of the ACE Secure Data Portal. 

The ACE pilot performance period is a 90-day trial period where the system 

operates in a production environment. The purpose of a pilot performance period 

is to ensure that the release meets the business needs of the users. At the end of a 

pilot, CBP decides whether to accept or reject the release. 

User Acceptance Testing is a formal method of measuring user satisfaction that 

was conducted during part of the pilot performance period. The purpose of 

User Acceptance Testing was to test the ACE Secure Data Portal in a real world 

business environment with real users interfacing with their actual data. Based on 

its experience with User Acceptance Testing feedback from evaluating Release 1, 

eCP engaged a specialist in the field of system usability to develop, administer, 

and evaluate metrics for User Acceptance Testing in Release 2. CBP and eCP 

gathered user feedback through three different methods: 

• The ACE Secure Data Portal has a function that enables users to transmit 

comments and problems to eCP online. Users provided feedback during 

the pilot performance period. 

2

 CBP chose not to implement Release 1 due to problems with infrastructure, performance, and usability. 

Automated Commercial Environment Secure Data Portal 5
• eCP operated the ACE help desk throughout the pilot performance period. 

The ACE help desk generated trouble ticket for problems, issues, and 

suggestions or elevated issues to appropriate parties in eCP or CBP as calls 

were received. 

• CBP and eCP had direct contact with users during User Acceptance 

Testing via telephone calls, e-mails, surveys, and on-site visits. Users 

provided feedback on specific problems or concerns. 

Comments made by the users, and considered material to eCP, resulted in a 

Trouble Reports, change requests, or were elevated to the appropriate party in eCP 

or CBP. eCP created Trouble Reports when it was determined that the comment 

represented the incorrect or incomplete implementation of a system requirement. 

Change requests were initiated for comments that represented a requirement 

for added system functionality. The Trouble Reports and the change requests 

were also used to track and address problems identified by users during the pilot 

performance period. 

eCP provided CBP with a report summarizing the User Acceptance Testing 

results, Trouble Report performance measurements, and help desk performance 

measurements. The report also described the methods used to gather ACE user 

feedback. 

Process To Monitor User Feedback 

The CBP Modernization Office was not monitoring user feedback gathered 

by eCP. The CBP Modernization Office is responsible for monitoring eCP’s 

development of ACE. However, since this was the first pilot performance period 

with external users, the CBP Modernization Office did not have a formal tracking 

system to monitor the disposition of the detailed user feedback. A monitoring 

process is necessary to ensure that issues important to CBP are incorporated into 

ACE. 

We recommended a formal user feedback tracking system to the Workforce 

Transformation Director for the CBP Modernization Office during the audit. As 

a result, on March 12, 2004, the Workforce Transformation Director included a 

process to monitor user feedback as a deliverable in Task Order 16. He provided 

the detailed requirements for tracking user feedback to the contractor in a Product 

Description Document that presented the proposed concept and content of the 

deliverable, titled “Process for Gathering and Disseminating Feedback.” This 

6 Automated Commercial Environment Secure Data Portal 
 

 

deliverable will be used to summarize the findings, analysis, conclusions, and 

recommendations of this activity. 

Recommendation 1 

We recommend that the CBP Modernization Office Director develop and issue a 

formal system for tracking user feedback. 

OIG Comment 

The actions taken by CBP during the audit satisfy the intent of our 

recommendation. 

Other Matters 

Change request packages that needed impact statements did not always address 

all of the critical areas required. This occurred because the packages were not 

adequately reviewed to ensure that all information necessary to make informed 

decisions was included. The Configuration Management Team and Change 

Sponsors are responsible for including all of the required documentation in 

the packages. Further, change request procedures did not require including 

alternative solutions in the packages. As a result, this increased the risk of 

approving changes that may have a negative impact on the cost, schedule, and 

quality of the program. 

During the audit, we discussed this issue with CBP Modernization Office 

management officials. In response to our discussion, the Director of Program 

Control issued a memorandum on December 17, 2003, emphasizing the 

importance of ensuring that impact analyses include, as a minimum, sufficient 

documentation on technical issues, staffing, cost, schedule, system interfaces, 

security, and contract issues. This memo, issued to all change request initiators, 

sponsors, reviewers, and configuration management personnel, also requires that 

any alternative solutions to the change be annotated on the change request. 

Automated Commercial Environment Secure Data Portal 7
Appendix A 

Purpose, Scope, and Methodology 

Purpose, Scope, and Methodology 

The objective of this audit was to determine whether the CBP was properly 

managing user feedback and the change control process to ensure that user 

expectations for the ACE Secure Data Portal were being met. To accomplish this 

objective, our audit work included: 

• Interviews conducted with various CBP and eCP officials at CBP 

Headquarters, as well as CBP and eCP facilities in Virginia. 

• Review of pertinent plans and documents related to the design, 

development, deployment, cost, and schedule of the ACE Secure Data 

Portal, including but not limited to: 

• Program Plans 

• Program Management Reviews 

• Requirement Specification Set 

• Requirements Traceability Management System 

• Security Risk Assessment 

• Production and Operational Readiness Reviews 

• Release 1 and 2 Pilot Test Reports 

• Change request Process 

• User Acceptance Test Reports 

• Management Reserve Fund 

• MITRE Weekly and Monthly Reports 

• Review of change request documents to evaluate documentation support 

and impact analysis statements. 

• Attendance at a September 2003 trade support meeting held in 

Reston, Virginia, and at various recurring meetings held by the CBP 

Modernization Office, support contractors, and eCP relating to ACE 

development. 

• Coordination of OIG audit work with the General Accountability Office. 

We conducted the audit from April 2003 through November 2003 according to 

generally accepted government auditing standards and pursuant to the Inspector 

General Act of 1978, as amended. 

8 Automated Commercial Environment Secure Data Portal 
 

Appendix B 

Recommendation 

Recommendation 1 

We recommend that the CBP Modernization Office Director develop and issue a 

formal system for tracking user feedback. 

Automated Commercial Environment Secure Data Portal 9
Appendix C 

Overview of ACE Releases 

10 Automated Commercial Environment Secure Data Portal 
Appendix D 

Major Contributors to This Report 

George Tabb, Houston Field Office Director 

Gene Wendt, Audit Manager 

Carlos Berrios, Auditor in Charge 

David Porter, Auditor 

Christy Staples, Auditor 

Linda Howarton, Referencer 

Automated Commercial Environment Secure Data Portal 11
Appendix E 

Report Distribution 

Department of Homeland Security 

Deputy Secretary 

Deputy Secretary Chief of Staff 

General Counsel 

Chief Information Officer 

DHS OIG Liaison 

Customs and Border Protection 

Commissioner 

Customs and Boarder Protection Modernization Office Director 

Director, Evaluations and Oversight, Office of Planning Director 

Office of Management and Budget 

Homeland Bureau Chief 

DHS OIG Budget Examiner 

Congress 

Congressional Oversight and Appropriation Committees, as appropriate 

12 Automated Commercial Environment Secure Data Portal 

Additional Information and Copies

To obtain additional copies of this report, call the Office of Inspector General (OIG) 

at (202) 254-4100, fax your request to (202) 254-4285, or visit the OIG web site at 

www.dhs.gov. 

OIG Hotline 

To report alleged fraud, waste, abuse or mismanagement, or any other kind of criminal 

or noncriminal misconduct relative to department programs or operations, call the OIG 

Hotline at 1-800-323-8603; write to Department of Homeland, Washington, DC 20528, 

Attn: Office of Inspector General, Investigations Division – Hotline. The OIG seeks to 

protect the identity of each writer and caller.