DEPARTMENT OF HOMELAND SECURITY

Office of Inspector General

Office of Audits

Audit of the

Automated Commercial Environment

Secure Data Web Portal:

Quality of Deliverables Can Be Improved

OIG-04-01 November 2003

Preface

The Department of Homeland Security (DHS) Offi ce of Inspector General (OIG) 

was established by the Homeland Security Act of 2002 (Public Law 107-296) by 

amendment to the Inspector General Act of 1978. This is one of a series of audit, 

inspection, investigative, and special reports prepared by the OIG periodically 

as part of its oversight responsibility with respect to DHS to identify and prevent 

fraud, waste, abuse, and mismanagement.

This report is the result of an assessment of the strengths and weaknesses of the 

program, operation, or function under review. It is based on interviews with 

employees and offi cials of relevant agencies and institutions, direct observations, 

and a review of applicable documents.

The recommendations herein, if any, have been developed on the basis of the 

best knowledge available to the OIG, and have been discussed in draft with those 

responsible for implementation. It is my hope that this report will result in more 

effective, effi cient, and/or economical operations. I express my appreciation to all 

of those who contributed to the preparation of this report.

Acting Inspector General

Clark Kent Ervin

Audit of the Automated Commercial Environment (ACE) Secure Web Portal Page 1

OIG

Department of Homeland Security

Office of Inspector General

INTERIM REPORT 

AUDIT OF THE AUTOMATED COMMERCIAL ENVIRONMENT 

SECURE DATA WEB PORTAL

On April 4, 2003, the Office of Inspector General (OIG) began an audit of the Automated Commercial 

Environment (ACE) secure data web portal1. During the ongoing audit, we identified matters that 

required immediate management attention. Therefore, we are issuing this interim report, number 

OIG-01-04, titled, Audit of the Automated Commercial Environment Secure Data Web Portal: Quality of 

Deliverables Can Be Improved. The objective of the audit is to determine whether the ACE web portal 

is being properly managed and developed to meet user expectations. 

The purpose of this interim report is to recommend improvements to the Customs and Border Protection 

(CBP) management of the ACE program. During the first year and a half of the ACE contract, there 

were continued concerns about the quality of deliverables from the prime contractor. These concerns 

were highlighted in the Department of Treasury Office of Inspector General (OIG) reports2

 and in U.S. 

Customs Service’s (Customs) correspondence with the contractor. Currently, the ACE contract is at a 

critical stage of development. ACE will be developed in a series of releases. Exhibit 1 illustrates the 

functionalities that will be delivered to users in each release. In Release 2 the contractor will deliver the 

portal through which the international trade community and the government will access ACE. Releases 

1 and 2 are the foundational activities upon which all subsequent ACE development will occur. If the 

portal is not complete, or of sufficient quality, the overall cost and the time to complete ACE could 

increase.

In the current review, CBP, the successor to Customs, concurred with our two recommendations relating 

to acceptance of the Release 2 deliverable and improving technical reporting. Their response was dated 

September 11, 2003 and outlined planned corrective actions that, when fully implemented, will satisfy 

the intent of our recommendations. Target dates were not included in the response; however, products 

1

 A web portal is software that provides a controlled gateway to an internet site.

2 Treasury OIG issued three audit reports on ACE development to Customs. These reports highlighted the continued risk of 

emphasizing schedule over quality. These reports are:

• OIG-02-058, March 4, 2002, Titled: Customs Needs to Adequately Staff the Modernization Office

• OIG-02-102, July 9, 2002, Titled: ACE Development at Risk from Incomplete Management Systems 

• OIG-03-058, February 13, 2003, Titled: Improved Management of Reengineering Needed 
Page 2 Audit of the Automated Commercial Environment (ACE) Secure Web Portal 

and decision points associated with the recommendations are specified in the ACE program schedules. 

The complete CBP response is provided as Exhibit 2.

By way of background, in August 2001 the ACE contract work was started by e-Customs Partnership (eCP), a coalition of contractors headed by IBM Global Services. As of June 2003, Congress appropriated 

and released $735 million for ACE. The CBP plans for ACE to be a customer-oriented, account-centric 

process that provides real-time access to internal and external information through a secure global 

channel for travel and trade. ACE is scheduled to be fully implemented by December 2007. 

 

Quality of Deliverables Can Be Improved

In release 1, CBP accepted deliverables that did not meet expectations. CBP transferred these 

developmental issues to release 2. These issues contributed to a four-month delay in the ACE contract 

schedule for release 2. A 90-day pilot test for Release 2 was started in June 2003. At the end of the 

pilot, CBP will evaluate the acceptability of the deliverable. It is critical that the foundational activity 

in Release 2 is complete and satisfactorily meets quality, security, and user expectations prior to 

acceptance. 

According to the Program Management Plan, contract tasks will include key milestones as “gates” into 

and out of each phase of the project’s life cycle. During each release there are specific deliverables that 

are key milestones. Examples of key milestones in Releases 1 and 2 include the Critical Design Review, 

Test Readiness Review, Production Readiness Review, and Operational Readiness Review. Each 

deliverable should be complete prior to continuing with system development. CBP’s Modernization 

Office reviews each deliverable to determine whether it is acceptable. 

ACE Release 1 was to establish an integrated infrastructure and web portal framework upon which all 

subsequent ACE functionality will be built. In addition, the release was to lay the foundation for a user 

account structure for selected importers at selected locations. ACE Release 2 includes initial Account, 

Trade, and Portal functionality to internal CBP account managers and 41 companies. It will allow CBP 

account managers to establish trade accounts for a number of major importers and track major activities 

related to each account, such as enforcement and compliance activities. Members of the international 

trade community will also be able to access certain information pertaining to their accounts. The Secure 

Data Portal will provide access for both the trade and CBP users to interact with the ACE system.

During Release 1, CBP identified problems in the areas of system infrastructure, performance, and 

usability. However, the Modernization Office and the contractor proceeded to develop Release 2 with 

plans to fix the problems during development. For example, 

• The Modernization Office accepted with conditions the Critical Design Review and Test 

Readiness Review, which allowed the contract to continue to the next key milestone. 
Audit of the Automated Commercial Environment (ACE) Secure Web Portal Page 3

• The solutions to problems identified with infrastructure, stability, and performance were 

postponed to Release 2.

• The User Acceptance Test users deemed the Release 1 Pilot “unacceptable.” A decision was 

made to make the corrections in Release 2.

• During the Release 1 Project Readiness Review, a decision was made to move the correction of 

15 product trouble reports to Release 2. 

• A decision was made prior to the pilot period that Release 1 would not move into an operational 

status, since the pilot would not be completely successful. Therefore, rather than making a 

decision to reject or accept Release 1 at the Operational Readiness Review, the Modernization 

Office simply accepted the Pilot Evaluation Report. 

In addition to infrastructure stability and performance problems discovered during Release 1, 

unexpected infrastructure problems also occurred during Release 2 with the integration of commercial 

off the shelf software packages. However, in order to mitigate problems associated with schedule 

delays, staff was brought in from other project areas to help resolve the problems and some Release 2 

requirements have already been moved to future releases.

Unlike prior deliverables, which were written products, Releases 1 and 2 are to provide a working 

system. This system must be functional for CBP and international trade companies. Some of the prior 

deliverables were accepted with conditions to allow the contract to continue as scheduled. Since Release 

2 will be the infrastructure upon which subsequent ACE functionalities will be built, it must be complete 

and meet higher quality standards prior to acceptance. 

Recommendation 1. We recommend that the contracting officer not accept the delivery of 

Release 2 until it is complete and meets the quality expected for all contract requirements.

Management Comments: CBP concurred and the contracting officer will only accept the 

delivery of ACE Release 2 when it meets the clearly defined acceptance criteria at the Operations 

Readiness Review. 

OIG Comment: The actions taken and planned by CBP satisfy the intent of our 

recommendation. However, CBP expressed concerns that the draft report implies that CBP’s 

emphasis on schedule caused the resultant loss of quality. The OIG did not intend to imply this 

and the report has been revised accordingly. While CBP took several positive direct measures 

with the eCP to deal with problems encountered, our report makes recommendations to further 

improve government oversight of the contract. It is the responsibility of the contracting officer to 

ensure that the government gets what it has contracted for in a timely manner. 
Page 4 Audit of the Automated Commercial Environment (ACE) Secure Web Portal 

Technical Reporting Could Be Improved

MITRE is an independent not-for-profit corporation chartered to work in the public interest. MITRE 

supports the CBP Modernization Office with strategic guidance and provides assessments of e-CP work 

products. It also provides technical expertise in acquisition, engineering, and architecture. 

The current MITRE reports are not designed to provide Modernization Office management with the 

information they need to make the most informed decision. MITRE provides the Modernization Office 

with weekly activity reports and monthly progress reports. Weekly reports form part of the monthly 

progress report. The progress reports describe the work accomplished by MITRE during the reporting 

period and planned activities for the upcoming weeks. Progress reports address MITRE’s activities in 

five areas - Project Management, Strategic Planning, Program Management, Enterprise Architecture 

and Engineering, and Projects. However, the reports provided by MITRE do not provide sufficient 

detail to identify and highlight important issues, nor do they provide alternative actions, risks, and 

recommendations needed by the Modernization Office to understand MITRE’s position on issues clearly. 

For example, the reports are generally limited to a listing of meetings attended or documents reviewed, 

without further commentary. Consequently, the Modernization Office does not have a clear written 

record of MITRE’s assessment results.

Recommendation 2. We recommend that the CBP chief information officer require MITRE 

to provide a report to the Modernization Office at the end of each of the key milestones of the 

project’s life cycle, which, based on MITRE’s technical evaluations, provide management with 

alternative actions, risks, and recommendations, as appropriate.

Management Comments: CBP concurred and has directed MITRE to provide a report as 

described in the recommendation.

OIG Comment: The actions taken and planned by CBP satisfy the intent of our 

recommendation.
Audit of the Automated Commercial Environment (ACE) Secure Web Portal Page 5

Exhibit 1
Page 6 Audit of the Automated Commercial Environment (ACE) Secure Web Portal 

Exhibit 2
Audit of the Automated Commercial Environment (ACE) Secure Web Portal Page 7
Page 8 Audit of the Automated Commercial Environment (ACE) Secure Web Portal 
Audit of the Automated Commercial Environment (ACE) Secure Web Portal Page 9
Page 10 Audit of the Automated Commercial Environment (ACE) Secure Web Portal

Additional Information and Copies

To obtain additional copies of this report, call the Office of Inspector General 

(OIG) at (202) 254-4100, fax your request to (202) 254-4285, or visit the OIG 

web site at www.dhs.gov.

OIG Hotline

To report alleged fraud, waste, abuse or mismanagement, or any other kind 

of criminal or noncriminal misconduct relative to department programs or 

operations, call the OIG Hotline at 1-800-323-8603; write to Department of 

Homeland Security, Washington, DC 20528, Attn: Office of Inspector General, 

Investigations Division – Hotline. The OIG seeks to protect the identity of each 

writer and caller.