Information Technology: Customs Has Made Progress on Automated Commercial Environment System, but It Faces Long-Standing Management Challenges and New Risks
Released: 2006-05-31T00:00:00

Letter

Compliance with Legislative Conditions

The legislative conditions that the Congress placed on the use of fiscal year 2006 ACE appropriated funds have been either partially or fully satisfied by the latest expenditure plan and related program documentation and activities. However, more can be done to better address several aspects of these conditions. For example: One legislative condition states that the plan should meet OMB’s capital planning and investment control review requirements, which include addressing security and privacy issues. However, a privacy impact assessment for ACE has been in draft for several months and is not yet approved. Another capital planning and investment control review requirement is that performance goals and measures be provided in the business case for ACE. Although CBP describes selected performance goals and measures, the goals (i.e., targets) are not always realistic (we provide further discussion of this issue later in this report).

According to another legislative condition, the expenditure plan must comply with DHS’s enterprise architecture. However, DHS does not have a documented methodology for evaluating programs for compliance with its enterprise architecture, other than relying on the professional expertise of its staff.

According to a third legislative condition, the DHS Chief Information Officer is to certify that an independent verification and validation (IV&V) agent is under contract. Although DHS satisfied this condition, the scope of the IV&V contractor’s activities is not consistent with the operative industry standard, which states that IV&V should extend to key system products and development processes.

Status of Our Open Recommendations

CBP has addressed some recommendations, while progress has been slow on others. Each recommendation, along with the status of actions to address it, is summarized below.

Ensure that future expenditure plans are based on cost estimates that are reconciled with independent cost estimates.

Complete. In October 2005, CBP, with contractor support, compared the program plan cost estimate with the independent cost estimate. According to the analysis performed, the two estimates are consistent.

Develop and implement a rigorous and analytically verifiable cost estimating program that embodies the tenets of effective estimating, as defined in the institutional and project-specific estimating models developed by the Software Engineering Institute (SEI).

In progress. CBP has taken steps such as (1) hiring a contractor to develop cost estimates (including contract task order cost estimates) that are independent of CBP’s estimates, and (2) tasking a support contractor with evaluating both the independent and CBP estimates against the criteria defined by SEI. According to the results of the support contractor’s evaluation, the independent estimates satisfied the SEI criteria; CBP’s estimates largely satisfied the criteria.

However, according to the support contractor, CBP’s cost estimating had limitations. First, the CBP estimate did not adequately consider past projects in its cost and schedule estimates. In addition, the CBP estimate was an aggregation of estimates developed separately for three ACE components, each according to a different cost estimating methodology; the support contractor advised against this approach, recommending that component estimates be based on the same methodology.

Immediately develop and implement a human capital management strategy that provides both near- and long-term solutions to program office human capital capacity limitations, and report quarterly to the Appropriations Committees on the progress of efforts to do so.

In progress. CBP has expanded its contractor and government workforce dedicated to the ACE program by merging staff assigned to trade-related legacy systems with the ACE program staff. In addition, it is beginning to use subject matter experts from existing field operations advisory boards to help program officials define requirements for future releases. However, it does not have a documented human capital strategy covering its ACE program.

Have future ACE expenditure plans specifically address any proposals or plans, whether tentative or approved, for extending and using ACE infrastructure to support other homeland security applications, including any impact on ACE of such proposals and plans.

In progress. The expenditure plan describes steps both planned and under way to ensure that ACE infrastructure supports both ACE and other homeland security applications. For example, it states that both ACE and the United States Visitor and Immigrant Status Indicator Technology (US- VISIT) program should conform to the DHS enterprise architecture, which is to define standard shared services that the two systems can request. Such a services oriented architecture is intended to promote reuse, as well as reducing overlap and duplication.

Define measures, and collect and use associated metrics, for determining whether prior and future program management improvements are successful.

In progress. CBP continues to make changes that are intended to improve overall program management, but it has not consistently defined measures to determine whether the changes are successful. For example, CBP has reorganized its Office of Information Technology; this reorganization is intended to improve program management by providing (1) enhanced government oversight of ACE development, (2) better definition of requirements for future ACE releases, and (3) faster and cheaper delivery of ACE capabilities. However, program officials told us that they have not established measures or targets for determining whether the reorganization is providing these benefits.

Define and implement an ACE accountability framework that fulfills 1. The framework should cover all program commitment areas, including key expected or estimated system (a) capabilities, use, and quality; (b) benefits and mission value; (c) costs; and (d) milestones and schedules.

In progress. CBP has prepared an initial version of an accountability framework that it intends to improve as it proceeds. The framework is built around measuring progress against costs, milestones, schedules, and risks for select releases; however, the benefit measurement has not been well defined, and the performance targets are not always realistic. 2. The framework should ensure currency, relevance, and completeness of all program commitments made to the Congress in expenditure plans.

In progress. The fiscal year 2006 expenditure plan includes inaccurate, dated, and incomplete information and omits other relevant information. For example, the plan did not include information regarding CBP’s decision to eliminate the dependencies among the screening and targeting releases and the cargo releases, and to take advantage of the capabilities of its existing Automated Targeting System.3. The framework should ensure reliable data that are relevant to measuring progress against commitments.

In progress. The data that CBP uses to measure progress against commitments are not consistently reliable. For example, data in the defect tracking system show that defects in Release 4 (which is now operational) have not been closed; however, program officials told us that many of these defects have been resolved. 4. The framework should ensure that future expenditure plans report progress against commitments contained in prior expenditure plans.

In progress. The current expenditure plan does not adequately describe progress against commitments made in previous plans. For example, the plan provides a summary of the funding requested in each of the previous six expenditure plans, but it does not provide information on whether these funding amounts were actually expended or obligated as planned. 5. The framework should ensure that criteria for exiting key readiness milestones adequately consider indicators of system maturity, such as the severity of open defects.

In progress. ACE milestone exit criteria provide for addressing the risk associated with severe defects that are unresolved. Using these criteria, CBP passed several release milestones with severe defects still open. However, CBP officials were unable to provide us with any documentation on how they assessed the inherent risks of passing these milestones with open severe defects. 6. The framework should ensure clear and unambiguous delineation of the respective roles and responsibilities of the government and the prime contractor.

Complete. The current ACE program plan describes general roles and responsibilities for the government and the prime contractor. More detailed roles have been documented in a roles and responsibilities matrix that assigns primary responsibility for each activity.

Report quarterly to the House and Senate Appropriations Committees on efforts to address our open recommendations.

In progress. CBP submitted quarterly reports to both Committees on its efforts to address our open recommendations; however, progress in addressing our recommendations was not always reported accurately.

Observations about ACE Management

We have several observations about the development of ACE releases, as well as several more concerning the performance of ACE releases that are deployed and operating.

ACE development: Steps have been taken to address a past pattern of ACE release shortfalls, but new release management weaknesses are emerging.

As we have previously observed, CBP established a pattern of borrowing resources from future releases to address problems with the quality of earlier releases; this led to schedule delays and cost overruns. This pattern has continued with the most recently deployed cargo release, which developed problems that caused delays with a subsequent screening and targeting release. CBP took steps to mitigate this problem by eliminating the dependencies between the cargo releases and the screening and targeting releases.

However, CBP’s planned schedule for developing additional releases includes a significant level of concurrence, because of CBP’s interest in delivering ACE functionality sooner. Such concurrence between ACE release activities has led to cost overruns and schedule delays in the past. Thus, the revised ACE plans and actions are potentially reintroducing the same problems that produced past shortfalls.

We made several specific observations related to these weaknesses, including the following: On two recent releases, key milestones were passed despite unresolved severe defects. Officials told us that the risk of proceeding did not outweigh the need to get the releases to users, and thereby gaining user acceptance and feedback. However, the risks were not documented and formally managed.

Concurrence in developing early ACE releases caused schedule slips and cost overruns. Despite these experiences, CBP has established a risky plan that involves considerable overlap across the development schedules for three future releases.

Although the use of earned value management (EVM) is an OMB requirement, it was not being used to manage the development of two recent releases. For example, CBP discontinued use of EVM on one release because this method was not familiar to staff who were transferred to work on the program.

ACE operations: Operational performance has been mixed, and mission impact is unclear.

ACE releases one through four are in operation. To date, these releases’ operational performance has been uneven. For example, ACE has largely been meeting its goals for being available and responsive in processing virtually all daily transactions, and has decreased truck processing times at some ports. However, ACE is not being used by as many CBP and trade personnel as was expected, and truck processing times at other ports have increased. Moreover, overall user satisfaction has been low.

In addition, ACE goals, expected mission benefits, and performance measures are not fully defined and adequately aligned with each other. For example, not every goal has defined benefits, every benefit is defined only in terms of efficiency gains, not every benefit has an associated business result, and not every benefit and business result has associated performance measures.

Further, where performance measures have been defined, the associated targets are not always realistic. For example, the performance target in fiscal year 2005 for ACE usage was that 11 percent of all CBP employees would use ACE. However, that many CBP employees will never need to use the system. This performance target does not reflect that.

Because performance measures are not always realistic or aligned with program goals and benefits, it is unclear whether ACE has realized—or will realize—the mission value that it was intended to bring to CBP’s and other agencies’ trade- and border security-related operations.

Conclusions

The legislative conditions that the Congress placed on the use of fiscal year 2006 ACE appropriated funds have been either partially or fully satisfied by the latest expenditure plan and related program documentation and activities. Nevertheless, more can be done to better address several aspects of these conditions, such as ensuring that the program’s privacy impact assessment is approved, measuring ACE performance and results, ensuring architectural alignment, and employing effective IV&V practices. Given that the legislative conditions are collectively intended to promote accountability and increase the chances of program success, it is important that each receives DHS’s full attention.

Also important to ACE’s success is the swift and complete implementation of the recommendations that we have previously made to complement the legislative conditions and improve program management, performance, and accountability. In this regard, some recommendations have been addressed, while progress has been slow on others, such as accurately reporting to the Appropriations Committees on CBP’s progress in implementing our prior recommendations; developing and implementing a strategic approach to meeting the program’s human capital needs; using criteria for exiting key milestones that adequately consider indicators of system maturity, such as severity of open defects and the associated risks; and developing and implementing a performance and accountability framework for ensuring that promised capabilities and benefits are delivered on time and within budget.

To its credit, CBP has taken several steps to stem the pattern of cost, schedule, and performance shortfalls that it experienced on early ACE releases. However, future releases are unlikely to realize the impact of these steps because revised ACE plans and actions are reintroducing the same pattern that led to early release shortfalls. This pattern includes not formally and transparently considering, and proactively addressing, the risks associated with passing key release milestones with known severe defects; building considerable overlap and concurrence in the development schedules of releases that will contend for the same resources; and not performing EVM on all releases. If this pattern continues, the prospects for a successful program will be diminished.

Although availability and responsiveness targets are largely being met and long-standing help desk limitations are being addressed, the prospects for a successful program nevertheless remain unclear. The true measure of ACE’s success is arguably the mission value that it brings to CBP’s and other agencies’ trade- and border security-related operations and users. Such value depends both on the operational performance of ACE and on CBP’s ability to demonstrate that this performance is achieving program goals, delivering expected benefits, and producing desired business results. At this juncture, however, neither the system’s performance nor its value is clear because of several factors: the operational performance of deployed releases has been mixed; users’ satisfaction has been low; the relationships among goals, benefits, and desired business outcomes are not evident; and the range of measures needed to create a complete and realistic picture of ACE’s performance is missing.

In summary, a number of ACE activities have been and are being done well; these have contributed to the program’s progress to date and will go a long way in determining the program’s ultimate success. However, it will be important for CBP to effectively address long-standing ACE management challenges along with emerging problems. Until it does so, ACE will remain a risky program.

Recommendations for Executive Action

To assist CBP in managing ACE—and increasing the chances that it will deliver required capabilities on time and within budget, demonstrating promised mission benefits and results—we recommend that the Secretary of Homeland Security direct the appropriate departmental officials to fully address those legislative conditions associated with having an approved privacy impact assessment and ensuring architectural alignment.

We also recommend that the Secretary, through CBP’s Acting Commissioner, direct the Assistant Commissioner for Information and Technology to fully address those legislative conditions associated with measuring ACE performance and results and employing effective IV&V practices; accurately report to the appropriations committees on CBP’s progress in implementing our prior recommendations; include in the June 30, 2006, quarterly update report to the appropriations committees a strategy for managing ACE human capital needs and the ACE framework for managing performance and ensuring accountability; document key milestone decisions in a way that reflects the risks associated with proceeding with unresolved severe defects and provides for mitigating these risks; minimize the degree of overlap and concurrence across ongoing and future ACE releases, and capture and mitigate the associated risks of any residual concurrence; use EVM in the development of all existing and future releases; develop the range of realistic ACE performance measures and targets needed to support an outcome-based, results-oriented accountability framework, including user satisfaction with ACE; and explicitly align ACE program goals, benefits, desired business outcomes, and performance measures.

Agency Comments

In written comments on a draft of this report signed by the Director, Departmental GAO/OIG Liaison, DHS agreed with our findings concerning progress in addressing our prior recommendations, and it agreed with the recommendations in this report. DHS also described actions that it has under way and planned to address the recommendations. The department’s comments are reprinted in appendix II.

We are sending copies of this report to the Chairmen and Ranking Minority Members of other Senate and House committees and subcommittees that have authorization and oversight responsibilities for homeland security. We are also sending copies to the DHS Secretary, the CBP Commissioner and, upon their request, to other interested parties. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

Should you or your offices have any questions on matters discussed in this report, please contact me at (202) 512-3459 or at hiter@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Other contacts and key contributors to this report are listed in appendix III.

Appendix I: Briefing to Subcommittees on Homeland Security, House and Senate Committees on Appropriations

Information Technology: Customs Has Made Progress on Automated Commercial Environment System, but it Faces Long-Standing Management Challenges and New Risks

March 10, 2006

Attachment 1: Scope and Methodology

The Department of Homeland Security’s (DHS) Customs and Border Protection (CBP)1 is about 5 years into its trade processing modernization program, known as the Automated Commercial Environment (ACE). The goals of ACE are as follows:

Support border security by enhancing analysis and information sharing with other government agencies to deal with increasing new security threats to our nation. Provide CBP personnel with the technology and information needed to decide, before a shipment reaches the border, what should be targeted because it is a security threat, and what should be expedited because it complies with U.S. laws. Provide an integrated, fully automated information system to enable the efficient collection, processing, and analysis of commercial import and export data. Streamline time-consuming and labor-intensive tasks for CBP personnel and the trade community, through a single, Web-based interface, reducing costs for the government and the trade community. Enable users to process, view, and manage their accounts nationally, and obtain historical information on cargo, conveyances, and crew, based on screening and targeting rules. Enable CBP to comply with legislative mandates to improve efficiency/effectiveness and provide better customer service to U.S. citizens.

CBP was formed from the former U.S. Customs Service and other entities with border protection responsibility.

OMB Circular A-11 establishes policy for planning, budgeting, acquisition, and management of federal capital assets.

5. is reviewed and approved by the DHS Investment Review Board (IRB),1 the

Secretary of Homeland Security, and OMB; and 6. is reviewed by GAO.

On February 2, 2006, DHS submitted its fiscal year 2006 expenditure plan for $316.8 million to the House and Senate Appropriations Subcommittees on Homeland Security.

DHS currently plans to acquire and deploy ACE in 11 increments, referred to as releases. The first three releases are fully deployed and operating, and the fourth release is being deployed. Other releases are in various stages of definition and development.

The purpose of the Investment Review Board is to integrate capital planning and investment control, budgeting, acquisition, and management of investments. It is also to ensure that spending on investments directly supports and furthers the mission and that this spending provides optimal benefits and capabilities to stakeholders and customers.

Objectives

As agreed, our objectives were to

determine whether the ACE fiscal year 2006 expenditure plan satisfies the determine the status of our open recommendations on ACE, and provide any other observations about the expenditure plan and DHS’s management of the ACE program.

We conducted our work at CBP headquarters and contractor facilities in the Washington, D.C., metropolitan area, as well as at the port in Blaine, Washington, from July 2005 through March 2006, in accordance with generally accepted government auditing standards. Details of our scope and methodology are provided in attachment 1.

The legislative conditions that the Congress placed on the use of fiscal year 2006 ACE appropriated funds have been either partially or fully satisfied by the latest expenditure plan and related program documentation and activities. Nevertheless, more can be done to better address several aspects of these conditions, such as having an approved privacy impact assessment, measuring ACE performance and results, ensuring architectural alignment, and employing effective independent verification and validation (IV&V) practices. The following table summarizes the status of each of the legislative conditions.

1. Meets the capital planning and investment control review requirements

established by OMB, including OMB Circular A-11, part 7. 2. Complies with DHS’s enterprise architecture. 3. Complies with the acquisition rules, requirements, guidelines, and systems acquisition management practices of the federal government. 4. Includes a certification by the Chief Information Officer of DHS that an independent verification and validation agent is currently under contract. 5. Is reviewed and approved by the DHS Investment Review Board, Secretary of Homeland Security, and OMB. 6. Is reviewed by GAO.

Some recommendations have been addressed, while progress has been slow on others, such as

accurately reporting to the Appropriations Committees on CBP’s progress in implementing our prior recommendations; developing and implementing a strategic approach to meeting the program’s using criteria for exiting key milestones that adequately consider indicators of system maturity, such as severity of open defects and the associated risks; and developing and implementing a performance and accountability framework for ensuring that promised capabilities and benefits are delivered on time and within budget.

The following table summarizes the status of each of the open recommendations.

1. Ensure that future expenditure plans are based on cost

estimates that are reconciled with independent cost estimates. 2. Develop and implement a rigorous and analytically verifiable cost estimating program. 3. Immediately develop and implement a human capital management strategy that provides both near- and long-term solutions; develop and implement missing human capital practices.

Number of months open Status 4. Have future ACE expenditure plans specifically address any proposals or plans for extending and using ACE infrastructure to support other homeland security applications. 5. Define measures, and collect and use associated metrics, for determining whether prior and future program management improvements are successful. 6. Define and implement an ACE accountability framework that a. coverage of all program commitment areas, including key expected or estimated system (1) capabilities, use, and quality; (2) benefits and mission value; (3) costs; and (4) milestones and schedules. b. currency, relevance, and completeness of all such commitments made to the Congress in expenditure plans.

11 c. reliable data relevant to measuring progress against commitments. d. reporting in future expenditure plans progress against commitments contained in prior expenditure plans. e. use of criteria for exiting key readiness milestones that adequately consider indicators of system maturity, such as severity of open defects. f. clear and unambiguous delineation of the respective roles and responsibilities of the government and the prime contractor. 7. Report quarterly to the House and Senate Appropriations Committees on efforts to address open GAO recommendations.

Objective 3: Observations

Steps have been taken to address past pattern of ACE release shortfalls, but new release management weaknesses are emerging.

Release 4 pilot revealed performance problems that caused the pilot period to be extended and the pilot scope to be reduced.

Release 4 operational readiness review was passed despite unresolved severe defects, and Release 4 is now being deployed.

Release 4 quality problems and enhancement needs have led to changes in how ACE release requirements are defined.

Release 4 problems delayed Screening 1 and led to a revised strategy for delivering all screening and targeting releases.

Screening 1 key milestones were passed despite unresolved severe defects.

Past pattern of cost, schedule, and performance shortfalls with Releases 2, 3,

and 4 makes new strategy of concurrently developing Releases 5, 6, and 7 risky.

Earned value management (EVM), a technique for measuring progress toward meeting deliverables, is not being used to manage Screening 1 and Release 5.

ACE’s operational performance has been mixed, and mission impact is unclear.

Availability and responsiveness performance targets are largely being met.

Processing times for trucks crossing the border at key ports vary.

Long-standing help desk limitations are being addressed.

Usage by CBP and the trade is lower than expected.

User satisfaction was reported as low.

Performance targets are not always realistic.

Goals, expected mission benefits, and performance measures are not adequately aligned.

To assist CBP in managing ACE and increasing the chances that it will deliver required capabilities on time and within budget and demonstrate promised mission benefits and results, we are recommending that DHS

minimize the degree of overlap and concurrency across ongoing and future ACE releases, and capture and mitigate the associated risks of any residual concurrency; use EVM in the development of all existing and future releases; develop the range of realistic ACE performance measures and targets needed to support an outcome-based, results-oriented accountability framework, including user satisfaction with ACE; and explicitly align ACE program goals, benefits, desired business outcomes, and performance measures.

In their oral comments on a draft of this briefing, DHS and CBP officials, including the Executive Director of Cargo Management Systems, CBP, generally agreed with our findings, conclusions, and recommendations and stated that the presentation was fair and balanced. They also provided clarifying information that we incorporated as appropriate in this briefing.

ACE is to support eight major CBP business areas. 1. Release1 Processing: Processing of cargo for import or export; tracking of conveyances, cargo, and crew; and processing of in-bond, warehouse, Foreign Trade Zone, and special import and export entries. 2. Entry2 Processing: Liquidation and closeout of entries and entry summaries related to imports, and processing of protests and decisions. 3. Finance: Recording of revenue, performance of fund accounting, and maintenance of the general ledger. 4. Account Relationships: Maintenance of trade accounts, their bonds and CBP-issued licenses, and their activity. 5. Legal and Policy: Management of import and export legal, regulatory, policies and procedures, and rulings issues.

A release is the act of CBP permitting imported merchandise to enter the United States.

An entry is the documentation required to be submitted to CBP in order for it to permit imported merchandise to enter the United States.

6. Enforcement: Enforcement of laws, regulations, policies and procedures,

Screening is the method of determining high-risk people or shipments before their arrival at a port.

Targeting is the risk-based determination of whether a shipment should undergo additional documentary review or physical inspection.

The ACE technical architecture is to consist of layers or tiers of computer technology:

The Client Tier includes user workstations and external system interfaces.

The Presentation Tier provides the mechanisms for the user workstations and external systems to access ACE.

The Integration Services Tier provides the middleware for integrating and routing information between ACE software applications and legacy systems.

The Applications Tier includes the ACE software applications comprising commercial products (e.g., SAP1) and custom-developed software that provide the functionality supporting CBP business processes.

The Data Tier provides the data management and warehousing services for ACE, including database backup, restore, recovery, and space management.

Security and data privacy are to be embedded in all five layers.

SAP is a commercial enterprise resource planning software product that has multiple modules, each performing separate but integrated business functions. ACE will use SAP to support many of its business processes and functions. CBP’s Modernization Office is also using SAP as part of a joint project with its Office of Finance to support financial management, procurement, property management, cost accounting, and general ledger processes.

Simplified View of ACE Technical Architecture

CBP currently plans to acquire the 11 ACE releases in about 8.5 years. Screening 2 is to be acquired in two “drops,” or subreleases; Release 5 is to be acquired in two drops; Release 6 is to be acquired in three drops; and Release 7 is to be acquired in two drops.

The following presents the functionality provided by the 11 ACE releases, their status, and associated plans.

Release 1 (ACE Foundation): Provide information technology (IT) infrastructure— computer hardware and system software—to support subsequent system releases. This release was deployed in October 2003 and is operating.

Release 2 (Account Creation): Give initial group of CBP national account managers1 and importers access to account information, such as trade activity. This release was deployed in October 2003 and is operating.

Release 3 (Periodic Payment): Provide additional account managers and importers, as well as brokers and carriers,2 access to account information; provide initial financial transaction processing and CBP revenue collection capability, allowing importers and their brokers to make monthly payments of duties and fees. This release was deployed in July 2004 and is operating.

CBP national account managers work with the largest importers.

Brokers obtain licenses from CBP to conduct business on behalf of the importers by filling out paperwork and obtaining a bond; carriers are individuals or organizations engaged in transporting goods for hire.

Background Summary of ACE Releases Release 4 (e-Manifest: Trucks): Provide electronic truck manifest1 processing and interfacing to legacy enforcement systems and databases. As discussed later, this release is operating at 39 truck border crossings as of March 8, 2006. Additional enhancement releases for Release 4 have been deployed since May 2005.

Screening 1 (Screening Foundation): Establish the foundation for screening cargo and conveyances by centralizing criteria and results into a single standard database; allow users to define and maintain data sources and business rules. This release is scheduled for deployment beginning in March 2006.

Screening 2 (Targeting Foundation): Establish the foundation for advanced targeting capabilities by enabling CBP’s National Targeting Center to search multiple databases for relevant facts and actionable intelligence. This release is scheduled for deployment in two drops: Screening 2 Targeting Platform (TP): Provide a platform to collect and search relevant data and other information from multiple databases. This drop is scheduled for deployment beginning in June 2006.

Manifests are lists of passengers or invoices of cargo for a vehicle, such as a truck, ship, or plane.

Screening 2 Targeting Foundation (TF): Build on the targeting platform to

add new data sources, enhance screening business rules, and provide reporting capabilities. This drop is scheduled for deployment beginning in October 2006; however, CBP deployed a prototype to the National Targeting Center as part of an effort to gather detailed requirements.

Release 5 (Entry Summary, Accounts, and Revenue): Leverage SAP technologies to enhance and expand accounts management, financial management, and entry summary functionality. This release is being developed in two drops: Master Data and Enhanced Accounts (Drop A1): Use SAP to deliver enhanced account creation and maintenance functionality and expand the types of accounts managed in ACE. This drop is scheduled for deployment beginning in May 2007.

Entry Summary and Revenue (Drop A2): Expand ACE to encompass entry summary, interfaces with participating government agencies, calculation of duties and fees, reconciliation processing, and refunds. This drop is scheduled for deployment beginning in July 2008.

Background Summary of ACE Releases Screening 3 (Advanced Targeting Capabilities): Provide enhanced screening for reconciliation, intermodal manifest, Food and Drug Administration data, and in- bond, warehouse, and Foreign Trade Zone authorized movements; integrate additional data sources into targeting capability; and provide risk management capability. This release is scheduled for deployment beginning in February 2007.

Screening 4 (Full Screening and Targeting): Provide full screening and targeting functionality supporting all modes of transportation and all transactions within the cargo management life cycle, including enhanced screening and targeting capability with additional technologies. This release is scheduled for deployment beginning in December 2008.

The multimodal manifest involves the processing and tracking of cargo as it transfers between different modes of transportation, such as cargo that arrives by ship, is transferred to a truck, and then is loaded onto an airplane.

E-Manifest: Rail and Sea (Drop M1): Extend the electronic manifest

functionality to rail and sea shipments; convert rail, sea, and truck electronic manifests into the multimodal manifest. Drop M1 is scheduled for deployment beginning in July 2008.

E-Manifest: Air (Drop M2): Provide the electronic manifest capability to air shipments, and bring all modes of transportation into the multimodal manifest. Drop M2 is scheduled for deployment beginning in October 2007.

E-Manifest: Enhanced Tracking (Drop M3): Provide the capability to track cargo, conveyances, individuals, and equipment, providing more timely and accurate shipment status information. Drop M3 is scheduled for deployment beginning in June 2009.

Background Summary of ACE Releases Release 7 (Exports and Cargo Control): Implement the remaining accounts management, revenue, manifest, and release and export functionality. This release is planned for development in two drops: ESAR: Drawback, Protest, and IASS (Drop A3): Provide the import activity summary statement (IASS),1 drawback functionality, and enhanced protest; provide on-line processing for trade account applications. Drop A3 is scheduled for deployment beginning in December 2009.

E-Manifest: Final Exports and Manifest (Drop M4): Extend the electronic manifest for mail, pipeline, and hand carry; provide for electronic export processing. Drop M4 is scheduled for deployment beginning in December 2009.

An import activity summary statement is a summary of an importer’s shipment activities over a specific period of time that is transmitted electronically to CBP on a periodic basis by importers and brokers.

As of March 8, 2006, ACE, Releases 1 through 4 capabilities have been deployed to 39 of the 99 truck ports (see table).

ACE Satisfaction of Modernization Act Requirements ACE is intended to support CBP satisfaction of the provisions of Title VI of the North American Free Trade Agreement, commonly known as the Modernization Act. Subtitle B of the Modernization Act contains the various automation provisions that were intended to enable the government to modernize international trade processes and permit CBP to adopt an informed compliance approach with industry. The following table illustrates how each ACE release is to fulfill the requirements of Subtitle B.

Status and Description of Completed Task Orders

No. Name

Initial program and project management; continued by task 009.

Initial enterprise architecture and system engineering; continued by task 010.

Initial requirements development and program planning effort; continued by tasks for specific increments/releases.

Design, development, testing, and deployment of Releases 1 and 2 (initially intended to build Increment 1, which was subsequently divided into four releases)

March 2004 Development of Release 5 project plan, documentation of ACE business processes, and development of an ACE implementation strategy.

Design, development, and testing of Releases 3 and 4, and deployment of Release 3.

Follow-on to task 001 to continue program and project management activities.

Follow-on to task 002 to continue enterprise architecture and system engineering activities; continued by task 017.

March 2003 Acquisition and setup of the necessary infrastructure and facilities for the contractor to design, develop, and test releases.

Establishment of the infrastructure to operate and maintain releases.

Conversion of scripts for interfacing desktop applications (MS Word and Excel) and mainframe computer applications.

March 2004 Development, demonstration, and delivery of a prototype to provide CBP insight into whether knowledge-based risk management should be used in ACE.

Status and Description of Active Task Orders

No. Name

Enterprise process improvement integration.

International Trade Data System (ITDS)

Assistance for participating government agencies to define requirements for an integrated ACE/ITDS system.

Development and demonstration of technology prototypes to provide CBP insight into whether the technologies should be used in ACE.

Program management and support to organizational change management through activities such as impact assessments, end user training, communication, and outreach.

Coordination of program activities and alignment of enterprise objectives and technical plans through architecture and engineering activities.

Application of the CBP Enterprise Life Cycle Methodology to integrate multiple projects and other ongoing Customs operations into CBPMO.

Follow-on to task 012 includes establishment, integration, configuration, and maintenance of the infrastructure to support Releases 2, 3, and 4.

Design, develop, test, and deploy the Screening and Targeting (S&T) operational capability.

Project definition and initial design for Release 5; initial project authorization and definition for Release 6.

Chronology of Seven ACE Expenditure Plans Since March 2001, seven ACE expenditure plans have been submitted.1 Collectively, the seven plans have identified a total of $1,698.1 million in funding.

On March 26, 2001, CBP submitted to its appropriations committees the first expenditure plan seeking $45 million for the modernization contract to sustain Customs and Border Protection Modernization Office (CBPMO) operations, including contractor support. The appropriations committees subsequently approved the use of $45 million, bringing the total ACE funding to $50 million.

On February 1, 2002, the second expenditure plan sought $206.9 million to sustain CBPMO operations; define, design, develop, and deploy Increment 1, Release 1 (now Releases 1 and 2); and identify requirements for Increment 2 (now part of Releases 5, 6, and 7 and Screenings 1 and 2). The appropriations committees subsequently approved the use of $188.6 million, bringing total ACE funding to $238.6 million.

In March 2001, appropriations committees approved the use of $5 million in stopgap funding to fund program management office operations.

Chronology of Seven ACE Expenditure Plans On May 24, 2002, the third expenditure plan sought $190.2 million to define, design, develop, and implement Increment 1, Release 2 (now Releases 3 and 4). The appropriations committees subsequently approved the use of $190.2 million, bringing the total ACE funding to $428.8 million.

On November 22, 2002, the fourth expenditure plan sought $314 million to operate and maintain Increment 1 (now Releases 1, 2, 3, and 4); to design and develop Increment 2, Release 1 (now part of Releases 5, 6, and 7 and Screening 1); and to define requirements and plan Increment 3 (now part of Releases 5, 6, and 7 and Screenings 2, 3, and 4). The appropriations committees subsequently approved the use of $314 million, bringing total ACE funding to $742.8 million.

Chronology of Seven ACE Expenditure Plans On January 21, 2004, the fifth expenditure plan sought $318.7 million to implement ACE infrastructure; to support, operate, and maintain ACE; and to define and design Release 6 (now part of Releases 5, 6, and 7) and Selectivity 2 (now Screening 2 and 3). The appropriations committees subsequently approved the use of $316.8 million, bringing total ACE funding to $1,059.6 million.

On November 8, 2004, the sixth expenditure plan sought $321.7 million for design and development of Release 5 and Screening 2, definition of Screening 3, ACE program management, architecture and engineering, and operations and maintenance. The appropriations committees subsequently approved the use of $321.7 million, bringing total ACE funding to $1,381.3 million.

Chronology of Seven ACE Expenditure Plans On February 02, 2006, CBP submitted its seventh expenditure plan, seeking $316.8 million for detailed design and development of Release 5, development of Release 6, deployment of Screening 2, development and deployment of Screening 3, program management and operations, and ACE operations, maintenance, and infrastructure implementation.

Summary of the ACE Fiscal Year 2006 Expenditure Plan

Plan activity Entry Summary, Accounts, and Revenue (Release 5), Design and Development e-Manifest: All Modes and Cargo Security (Release 6), Design and Development Screening and Targeting, Development International Trade Data System Foundation Architecture and Engineering Operations and Maintenance Implementation Infrastructure and Support Communications, Training, and Outcomes ACE Foundation Program Management Program Office Operations Total

The plan does not include management reserve funding. As of December 15, 2005, the program had about $33.8 million in unused management reserve funding from prior years.

Background ACE Testing and Related Milestones Development of each ACE release includes system integration and system acceptance testing, followed by a pilot period that includes user acceptance testing. Generally, the purpose of these tests is to ensure that the system meets defined system requirements or satisfies user needs. The associated readiness reviews are to ensure that the system is ready to proceed to the next stage of testing or operation.

Tests and their related milestones are described in the following table.

Verify that related system, subsystem, or module components are capable of integrating and interfacing with each other.

Verify that the developed system, subsystem, or module operates in accordance with requirements.

Verify that the functional scope of the release meets the business functions for the users.

Defects identified during testing and operation of the system are classified into one of four severity categories, as described below.

Category

Critical (Severity 1)

Defect prevents or precludes the performance of an operational or mission- essential capability, jeopardizes safety or security, or causes the system, application, process, or function to fail to respond or to end abnormally.

Severe (Severity 2)

Defect prevents or precludes system from working as specified and/or produces an error that degrades or impacts the system or user functionality.

Moderate (Severity 3)

Defect prevents or precludes system from working as specified and/or produces an error that degrades or impacts the system or user functionality. An acceptable (reasonable and effective) work-around is in place that rectifies the defect until a permanent fix can be made.

Minor (Severity 4)

Defect is inconsequential, cosmetic, or inconvenient and does not prevent users from using the system to accomplish their tasks.

Appendix II: Comments from the Department of Homeland Security

Appendix III: Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the person named above, Justin Booth, Barbara Collier, William Cook, Neil Doherty, Michael Marshlick, Shannin O’Neill, Tomas Ramirez, and Jennifer Vitalbo made key contributions to this report.