Oceangoing cargo containers have an important role in the movement of cargo between global trading partners. Approximately 90 percent of the world’s trade is transported in cargo containers. In the United States almost half of incoming trade (by value) arrives by containers aboard ships. If terrorists smuggled a weapon of mass destruction into the nation using a cargo container and detonated such a weapon at a seaport, the incident could cause widespread death and damage to the immediate area, perhaps shut down seaports nationwide, cost the U.S. economy billions of dollars, and seriously hamper international trade. The Department of Homeland Security and CBP are responsible for addressing the threat posed by terrorist smuggling of weapons in oceangoing containers. To carry out this responsibility, CBP uses a layered security strategy. One key element of this strategy is ATS. CBP uses ATS to review documentation, including electronic manifest information submitted by the ocean carriers on all arriving shipments, to help identify containers for additional inspection. CBP requires the carriers to submit manifest information 24 hours prior to a United States-bound sea container being loaded onto a vessel in a foreign port. ATS is a complex mathematical model that uses weighted rules that assign a risk score to each arriving shipment in a container based on manifest information. As previously discussed, CBP officers use these scores to help them make decisions on the extent of documentary review or physical inspection to be conducted. ATS is an important part of other layers in the security strategy. Under its CSI program, CBP places staff at designated foreign seaports to work with foreign counterparts to identify and inspect high-risk containers for weapons of mass destruction before they are shipped to the United States. At these foreign seaports, CBP officials use ATS to help target shipments for inspection by foreign customs officials prior to departing for the United States. Approximately 73 percent of cargo containers destined for the United States originate in or go through CSI ports. ATS is also an important factor in the Customs-Trade Partnership Against Terrorism (C-TPAT) program. C-TPAT is a cooperative program linking CBP and members of the international trade community in which private companies agree to improve the security of their supply chains in return for a reduced likelihood that their containers will be inspected. Specifically, C-TPAT members receive a range of benefits, some of which could change the ATS risk characterization of their shipments, thereby reducing the probability of extensive documentary and physical inspection. CBP does not yet have key controls in place to provide reasonable assurance that ATS is effective at targeting oceangoing cargo containers with the highest risk of containing smuggled weapons of mass destruction. To address this shortcoming, CBP is (1) developing and implementing performance metrics to measure the effectives of ATS, (2) planning to compare the results of randomly conducted inspections with the results of its ATS inspections, (3) developing and implementing a simulation and testing environment, and (4) addressing recommendations contained in a 2005 peer review. To date, none of these control activities have been fully completed or implemented. Thus, CBP does not yet have key internal controls in place to be reasonably certain that ATS is providing the best available information to allocate resources for targeting and inspecting containers that are the highest risk and thus not overlook inspecting containers that pose a high threat to the nation. CBP does not yet have performance measures in place to help it determine the effectiveness of ATS at targeting oceangoing cargo containers with the highest risk of smuggled weapons of mass destruction. The Comptroller General’s internal control standards include the establishment and review of performance measures as one example of a control activity to help an entity ensure it is achieving effective results. In July 2005, CBP contracted with a consulting firm to develop such performance metrics. CBP officials and personnel from this consulting firm told us that the firm’s personnel analyzed shipment information in ATS over a 2-year period to obtain additional insights into ATS’s performance and to determine whether ATS is more effective at targeting cargo containers for terrorism related risk than a random sampling inspection approach. CBP officials told us that the consulting firm’s personnel prepared a draft of the results of their analyses and that, as of March 21, 2006, CBP officials are reviewing these analyses. They also said that the consulting firm’s personnel are documenting the methodology for their analyses and related performance measures that CBP can use in the future. CBP officials expect to receive this methodology and the performance measures in April 2006, and told us that they expect to begin using the measures in June 2006. CBP officials also told us that they initially planned to have performance measures developed by August 31, 2005, but that this process has taken longer than expected because of delays in (1) obtaining security clearances for the consulting firm’s personnel, (2) obtaining workspace for the firm’s staff, and (3) arranging for the appropriate levels of access to CBP’s information systems. Currently, CBP is not using the results of its random sampling program to assess the effectiveness of ATS. As part of its Compliance Measurement Program, CBP plans to randomly select 30,000 shipments based on entry information submitted by the trade community and examine those shipments to ensure compliance with supply chain security during fiscal year 2006. At this time, CBP is unable to compare the examination results from its random sampling program with its ATS inspection results, as we recommended in our 2004, report because CBP does not yet have an integrated, comprehensive system in place to compare multiple sets of data—like results of random inspections with results of routine ATS inspections that were triggered by ATS scores and other operational circumstances. Such a comparison would allow examination of if and why the outcomes of ATS’s weighted rule sets are not consistent with the expected outcomes possible in the universe of cargo containers, based on sample projections. Furthermore, the Comptroller General’s standards for internal control state that information should be recorded and communicated to management and others within the entity who need it in a form that enables them to carry out their responsibilities. Currently, CBP does not conduct simulated events (e.g., covert tests and computer-generated simulations)—a key control activity—to test and validate the effectiveness of ATS in targeting oceangoing cargo containers with the highest risk of containing smuggled weapons of mass destruction and has not yet implemented a dedicated simulation and testing environment. Without testing and validation, CBP lacks a vital mechanism for evaluating ATS’s ability to identify high-risk containers. In July 2005, CBP contracted with a consulting firm to obtain assistance in the development of a computer-generated simulation and testing environment. CBP officials report that they have the simulation environment infrastructure in place and have processed mock manifest data to simulate cargo linked to terrorism in the new environment. CBP is currently reviewing the results of this test. Further, CBP officials told us that the consulting firm is continuing to work with CBP to develop system requirements so that officers can effectively use the simulation environment. CBP expects to receive the consulting firm’s final input for the simulation and testing environment by June 2006. CBP officials said that they cannot estimate when this simulation and testing environment will be fully operational until CBP receives the consulting firm’s final product. As with the development of performance measures, CBP officials also told us that this process has taken longer than expected because of delays in (1) obtaining security clearances for the consulting firm’s personnel, (2) obtaining workspace for the firm’s staff, and (3) arranging for the appropriate levels of access to CBP’s information systems. As we reported in 2004, terrorism experts suggested that testing ATS by covertly simulating a realistic event using probable methods of attack would give CBP an opportunity to examine how ATS would perform in an actual terrorist situation. CBP officials told us that although they are considering implementing this kind of practice, they do not currently have a program in place to conduct such tests. The Director of CBP’s Management Inspections and Integrity Assurance office told us that in mid-April 2006, his office will be presenting a proposal to the Acting Commissioner and other senior management to request initiation of a program to conduct testing of the CSI program that will include testing ATS to help ensure that it is appropriately targeting the highest-risk cargo in the CSI program. In response to our 2004 recommendation that CBP initiate an external peer review of ATS, CBP contracted with a consulting firm to evaluate CBP’s targeting methodology and recommend improvements. Specifically, the contractor identified strengths of the CBP targeting methodology and compared ATS with other targeting methodologies. However, the peer review did not evaluate the overall effectiveness of ATS because CBP did not have the systems in place to allow the contractor to do so. The contractor’s final report, issued in April 2005, identified many strengths in the ATS targeting methodology, such as a very capable and highly dedicated team and the application of a layered approach to targeting. It also made several recommendations to improve the targeting methodology that included control activities, such as (1) the development of performance measures, (2) the development of a simulation and testing environment, (3) the development and implementation of a structured plan for continual rules enhancement, and (4) an evaluation and determination of the effectiveness of the ATS targeting rules, several of which reinforced the recommendations we made in our 2004 report. CBP issued a detailed plan, which projected delivery dates, for responding to the recommendations made in the contractor’s final report. However, about half of these dates have not been met. For example, CBP projected that it would have its testing and simulation environment in place by September 30, 2005. Although CBP has been working on this effort, the environment has not yet been implemented.