Supply Chain Security: Examinations of High-Risk Cargo at Foreign Seaports Have Increased, but Improved Data Collection and Performance Measures Are Needed
Published: 2008-01-25T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Customs and Border Protection's (CBP) Container Security Initiative (CSI) aims to identify and examine high-risk U.S.-bound cargo at foreign seaports. GAO reported in 2003 and 2005 that CSI helped to enhance homeland security, and recommended actions to strengthen the program. This report updates information and assesses how CBP has (1) contributed to strategic planning for supply chain security, (2) strengthened CSI operations, and (3) evaluated CSI operations. To address these issues, GAO interviewed CBP officials and reviewed CSI evaluations and performance measures. GAO also visited selected U.S. and CSI seaports, and met with U.S. and foreign government officials.
section_title: What GAO Found
paragraphs: By collaborating on the development of the Department of Homeland Security's Strategy to Enhance International Supply Chain Security, and by revising the CSI strategic plan as GAO recommended, CBP has contributed to the overall U.S. strategic planning efforts related to enhancing the security for the overseas supply chain. Also, CBP reached its targets of operating CSI in 58 foreign seaports, and thereby having 86 percent of all U.S.-bound cargo containers pass through CSI seaports in fiscal year 2007--representing a steady increase in these measures of CSI performance. To strengthen CSI operations, CBP has sought to address human capital challenges and previous GAO recommendations by increasing CSI staffing levels closer to those called for in its staffing model and revising its human capital plan. However, challenges remain because CBP continues to rely, in part, on a temporary workforce; has not determined how to optimize its staffing resources; and reports difficulties in identifying sufficient numbers of qualified staff. In addition, CBP has enhanced relationships with host governments participating in CSI. However, hurdles to cooperation remain at some seaports, such as restrictions on CSI teams witnessing examinations. CBP improved its evaluation of CSI team performance at seaports, but limitations remain in the evaluation process that affect the accuracy and completeness of data collected. CBP has not set minimum technical criteria for equipment or systematically collected information on the equipment, people, and processes involved in CSI host government examinations of high-risk, U.S-bound container cargo. Also, CBP has not developed general guidelines to use in assessing the reliability of these examinations. Thus, CBP potentially lacks information to ensure that host government examinations can detect and identify weapons of mass destruction, which is important because containers are typically not reexamined in the United States if already examined at a CSI seaport. CBP refined overall CSI performance measures, but has not fully developed performance measures and annual targets for core CSI functions, such as the examination of high-risk containers before they are placed on vessels bound for the United States. These weaknesses in CBP's data collection and performance measures potentially limit the information available on overall CSI effectiveness.

section_title: Letter
paragraphs: Oceangoing cargo containers play a vital role in the movement of cargo  Oceangoing cargo containers play a vital role in the movement of cargo  between global trading partners. In fiscal year 2007, more than 10 million  between global trading partners. In fiscal year 2007, more than 10 million  oceangoing cargo containers arrived at U.S. seaports—meaning roughly  oceangoing cargo containers arrived at U.S. seaports—meaning roughly  28,000 oceangoing containers arrived each day that year. The terrorist  28,000 oceangoing containers arrived each day that year. The terrorist  attacks of 2001 heightened concerns about the ability of both the federal  attacks of 2001 heightened concerns about the ability of both the federal  government and companies participating in international maritime  government and companies participating in international maritime  commerce to identify and prevent weapons of mass destruction from  commerce to identify and prevent weapons of mass destruction from  being smuggled inside cargo containers bound for the United States.  being smuggled inside cargo containers bound for the United States.  Balancing security concerns with the need to facilitate the free flow of  Balancing security concerns with the need to facilitate the free flow of  commerce remains an ongoing challenge for the public and private sectors  commerce remains an ongoing challenge for the public and private sectors  alike.  alike.
In the federal government, U.S. Customs and Border Protection (CBP),  In the federal government, U.S. Customs and Border Protection (CBP),  part of the Department of Homeland Security (DHS), is charged with  part of the Department of Homeland Security (DHS), is charged with  managing, securing, and controlling the nation’s border and in its capacity  as the frontline border security agency, plays a lead role in facing maritime  threats.  CBP launched the Container Security Initiative (CSI) in January  2002, which through partnerships with its foreign counterparts, is designed  to help protect global trade lanes by targeting and examining container  cargo that poses a threat as early as possible in the global supply chain. As  part of the program, foreign governments allow CBP officers to be  stationed at foreign seaports.  These officers use intelligence and  automated risk assessment information to target shipments to identify  those at risk of containing weapons of mass destruction (WMD) or other  terrorist contraband.  CBP and host government officials share the role of  assessing the risk of U.S.-bound container cargo leaving the seaports of  countries participating in CSI.  CBP officers at the CSI seaports are  responsible for targeting high-risk cargo shipped in containers and other  tasks, whereas host government customs officials examine the high-risk  cargo—when requested by CBP—by scanning containers using various  types of nonintrusive inspection equipment, such as large-scale X-ray  machines, or by physically searching the container’s contents before it  travels to the United States.
As part of its strategic plan, CBP is partnering with international trade and  security groups to develop supply chain security standards that can be  implemented by the world community.  By engaging international  organizations, CBP is contributing to the development of global security  standards.  Recent legislative actions intended to further enhance  maritime security also updated requirements that affect CSI. In October  2006, Congress passed and the President signed legislation—the Security  and Accountability for Every Port Act (SAFE Port Act) — establishing a  statutory framework for CSI, which previously had been an agency  initiative not specifically required by law.  The act imposed various  mandates, such as requiring CBP to take risk factors including cargo  volume into account when designating seaports as CSI participants. In  August 2007, the Implementing Recommendations of the 9/11 Commission  Act of  2007 (9/11 Act) was enacted, which requires, among other things,  100 percent scanning of U.S.-bound cargo containers by foreign seaports  by 2012, with possible extensions for some ports—replacing a similar  provision in the SAFE Port Act that did not have a deadline.
We have previously reported on CSI’s progress in meeting its strategic  goals and objectives. Our July 2003 and April 2005 reports on CSI  acknowledged the program’s important role in helping to enhance  homeland security, but we also recommended actions to enhance the  strategic planning for the program, such as better defining its goals,  objectives, and performance measures. In addition, we recommended  actions to strengthen the program’s management and operations, such as  conducting human capital planning (which affects CBP staffing levels at  seaports) and establishing minimum technical capability requirements for  equipment used to examine high-risk containers.
Recognizing the importance of the CSI program, you asked us to conduct  another review. For this report, we assessed the following issues:    How has CBP contributed to strategic planning for supply chain  security efforts and the CSI program in particular, and what progress  has been made in achieving CSI performance goals?
How has CBP strengthened CSI operations in response to our 2005  review and what challenges, if any, remain?
How does CBP evaluate CSI seaport operations and assess program  performance overall, and how has this process changed over time?
To address these objectives, we met with CBP officials who have program  responsibilities for CSI, and reviewed available program data and  documentation. Specifically, to review CBP’s strategic planning initiatives,  we reviewed national-level strategic planning documents and those  created for DHS, CBP, and CSI.  To determine CBP’s progress in achieving  its CSI program goals, we reviewed CBP’s statistical data on container  cargo and CSI program activities. To learn about how CBP has  strengthened its operations, we reviewed our previous assessments of the  CSI program, and examined CBP’s efforts to implement our three prior  recommendations. To assess CBP’s progress, we met with CBP officials at  the U.S. National Targeting Center - Cargo (NTCC) in Virginia and three  domestic seaports in different geographical locations and representing  varying volumes of container traffic. We also visited six CSI seaports  located overseas that were selected based on several factors, including  geographic and strategic significance, volume of container traffic, and  when CSI operations began at the seaport. The results from our visits to  seaports provided examples of CBP and host government operations but  cannot be generalized beyond the seaports visited because we did not use  statistical sampling techniques in selecting the seaports.
To determine what progress CBP has made in strengthening its tools for  monitoring and measuring the progress of the CSI program, we reviewed  the performance measures presented in the CSI strategic plan against  criteria developed by the Office of Management and Budget and GAO.  We  also reviewed a nonrepresentative sample of CSI team evaluations.  While  these documents provided examples about program evaluation methods  and CSI program operations, and generally corroborated our seaport site  visit observations, our findings cannot be generalized to the program as a  whole. We conducted this performance audit from May 2006 through  January 2008 in accordance with generally accepted government auditing  standards.  Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives.  We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.  A detailed discussion of  our scope and methodology is contained in appendix I.
subsections: section_title: Background
paragraphs: States.  We disagree.  As discussed in this report, a core element of the CSI  program, specifically the extent to which U.S.-bound containers carrying  high-risk cargo are examined at CSI seaports, is not addressed through  CBP’s performance measures.
subsections: section_title: Vulnerabilities of Containers in the International Supply Chain
paragraphs: Seaports are critical gateways for the movement of commerce through the  international supply chain. The facilities, vessels, and infrastructure within  seaports, and the cargo passing through them, all have vulnerabilities that  terrorists could exploit. The containers carrying goods that are shipped in  oceangoing vessels are of particular concern because they can be filled  overseas at many different locations and are transported through complex  logistics networks before reaching U.S. seaports.
In addition, transporting such a shipping container from its international  point of origin to its final destination involves many different participants  and many points of transfer. The materials in a container can be affected  not only by the manufacturer or supplier of the material being shipped, but  also by carriers who are responsible for getting the material to a port and  by personnel who load containers onto the ships. Others who interact with  the cargo or have access to the records of the goods being shipped  include, among others, exporters who make arrangements for shipping  and loading, freight consolidators who package disparate cargo into  containers, and forwarders who manage and process the information  about what is being loaded onto the ship. Figure 1 illustrates many of the  key participants and points of transfer involved from the time that a  container is loaded for shipping to its arrival at the destination seaport and  ultimately the importer.
Several studies on maritime security conducted by federal, academic,  nonprofit, and business organizations have concluded that the movement  of oceangoing cargo in containers is vulnerable to some form of terrorist  action, largely because of the movement of cargo throughout the supply  chain. Every time responsibility for cargo in containers changes hands  along the supply chain there is the potential for a security breach, and  thus, vulnerabilities exist that terrorists could take advantage of by placing  a WMD into a container for shipment to the United States. While there  have been no known incidents of containers being used to transport  WMDs, criminals have exploited containers for other illegal purposes,  such as smuggling weapons, people, and illicit substances, according to  CBP officials. Finally, while CBP has noted that the likelihood of terrorists  smuggling WMD into the United States in cargo containers is low, the  nation’s vulnerability to this activity and the consequences of such an  attack are potentially high. In 2002, Booz Allen Hamilton sponsored a  simulated scenario in which the detonation of weapons hidden in cargo  containers shut down all U.S. seaports over a period of 12 days. The  results of the simulation estimated that the port closure could result in a  loss of $58 billion in revenue to the U. S. economy, along with significant  disruptions to the movement of trade.
section_title: Efforts to Secure Containers in the International Supply Chain
paragraphs: The federal government has taken many steps to secure the supply chain,  including the cargo in containers destined for the United States. While  CBP officials at domestic seaports continue efforts to identify and  examine high-risk imports arriving in containers, CBP’s post-September 11  strategy also involves focusing security efforts beyond U.S. borders to  target and examine high-risk cargo before it enters U.S. seaports. CBP’s  strategy is based on a layered approach of related initiatives that attempt  to focus resources on potentially risky cargo shipped in containers while  allowing other containers carrying cargo to proceed without unduly  disrupting commerce into the United States. CBP has initiated most of  these efforts, shown in table 1. However, the Department of Energy (DOE)  has led U.S. efforts to detect radiation in cargo containers originating at  foreign seaports.
CBP uses this computerized decision support tool to review documentation, including electronic  manifest information submitted by ocean carriers on all cargo destined for the United States to  help identify shipments requiring additional scrutiny. ATS utilizes complex mathematical models  with weighted rules that assign a risk score to each shipment based on manifested information.  CBP officers review the rule firings that support the ATS score to help them make decisions on  the extent of documentary review or examination to be conducted.
CBP generally requires ocean carriers to electronically transmit cargo manifests to  CBP’s Automated Manifest System 24 hours before the U.S.-bound cargo is loaded  onto a vessel at a foreign seaport. Carriers and importers are to provide information to  CBP that is used to strengthen how ATS assigns risk scores. The cargo manifest  information is submitted by ocean carriers on all arriving cargo shipments, and entry  data (more detailed information about the cargo) are submitted by brokers.
CSI places staff at participating foreign seaports to work with host country customs  officials to target and examine high-risk cargo to be shipped in containers for weapons  of mass destruction before they are shipped to the United States. CBP officials identify  the high-risk containers and request that their foreign counterparts examine the contents  of the containers.
CBP develops voluntary partnerships with members of the international trade  community comprised of importers; customs brokers; forwarders; air, sea, and land  carriers; and contract logistics providers. Private companies agree to improve the  security of their supply chains in return for various benefits, such as a reduced likelihood  that their containers will be examined.
DOE installs radiation detection equipment at key foreign seaports, enabling foreign  government personnel to use radiation detection equipment to screen shipping  containers entering and leaving these seaports, regardless of the containers’  destination, for nuclear and other radioactive material that could be used against the  United States and its allies.
Pilot program at selected CSI seaports to scan 100 percent of U.S.-bound cargo  containers for nuclear and radiological materials overseas using integrated examination  systems that couple nonintrusive inspection equipment and radiation detection  equipment.
In January 2002, CBP began CSI to target container cargo at overseas  seaports so that high-risk cargo could be examined prior to departure for  the United States. More recently, Congress passed legislation affecting the  CSI program, including (1) the SAFE Port Act enacted in October 2006 that  established a statutory framework for CSI and, among other things,  required a pilot program, now known as the Secure Freight Initiative, to  determine the feasibility of 100 percent scanning of U.S.-bound cargo  containers at foreign seaports; and (2) the 9/11 Act enacted in August 2007,  that, among other things, requires by 2012, the scanning of  all U.S.-bound  containers at foreign seaports with potential exceptions if a seaport  cannot meet that deadline.
For the CSI program, CBP officials stated that DHS expended about $138  million and $143 million, respectively for fiscal years 2006 and 2007. The  President’s budget for fiscal year 2008 requested $156 million for CSI. CSI  is now operating at 58 seaports in 33 foreign countries, as shown in figure  2. Appendix III lists the specific CSI seaports.
section_title: Core Elements and Security Activities of CSI Program
paragraphs: According to CBP, the three core elements of CSI include (1) CBP  identifying high-risk containers; (2) CBP requesting, where necessary, that  host governments examine high-risk containers before they are shipped;  and (3) host governments conducting examinations of high-risk  containers. To integrate these elements into CSI operations, CBP  negotiated and entered into bilateral, nonbinding arrangements with  foreign governments, specifying the placement of CBP officials at foreign  seaports and the exchange of information between CBP and foreign  customs administrations. To participate in CSI, a host nation must meet  several criteria developed by CBP. The host nation must utilize (a) a  seaport that has regular, direct, and substantial container traffic to  seaports in the United States; (b) customs staff with the capability of  examining cargo originating in or transiting through its country; and (c)  nonintrusive inspection equipment with gamma or X-ray capabilities and  radiation detection equipment. Additionally, each potential CSI port must  indicate a commitment to (d) establish an automated risk management  system for identifying potentially high-risk container cargo; (e) share  critical data, intelligence, and risk management information with CBP  officials; (f) conduct a seaport assessment to ascertain vulnerable links in  a port’s infrastructure and commit to resolving those vulnerabilities; and  (g) maintain a program to prevent, identify, and combat breaches in  employee integrity.
As part of the arrangements with foreign governments participating in CSI,  CBP most often stations teams of CBP officers at each foreign seaport to  conduct CSI activities in collaboration with host government customs  officials. While the number of CBP officers stationed at CSI seaports varies  by location, typically a CSI team consists of (1) a CSI team leader, who  manages the team and monitors the relationship with the host country; (2)  CBP officers, who target high-risk cargo and observe (where possible) the  host government’s examination of containers carrying the cargo; (3) an  intelligence research specialist, who assimilates data to support timely and  accurate targeting of containers; and (4) a special agent responsible for  CSI-related investigations at the seaport. According to CBP, it is ideal for  the CSI team to be located in close physical proximity with host  government customs counterparts to facilitate collaboration and  information sharing. However, CBP officials also stated that the agency  uses CBP officers stationed at the NTCC as needed to support the CBP  officers located at the CSI seaports. The CBP officials at NTCC assist the  CSI teams at high-volume seaports to ensure all containers that pass  through CSI seaports are targeted to identify high-risk container cargo;  carry out CSI targeting responsibilities for CSI seaports that do not have  CBP officials stationed there; and, according to CBP officials, conduct  targeting for U.S.-bound container cargo that does not pass through CSI  seaports using national sweeps to identify high-risk container cargo.
At CSI seaports, CBP officers share responsibilities with host  governments’ customs officials to target and examine high-risk container  cargo. Figure 3 describes the activities carried out by CBP officers and  host government customs officials, respectively, to target and examine  high-risk container cargo at CSI seaports.
section_title: CBP Collaborated on the DHS Strategy to Enhance International Supply Chain Security, and Met Goals for CSI Expansion and Increased Container Examination
paragraphs: CBP has undertaken strategic planning to guide efforts to secure the  international supply chain and, more specifically, to manage the CSI  program. CBP contributed to an international supply chain security  strategy DHS recently issued that builds on DHS’s existing strategic  framework for maritime security. In 2006 CBP enhanced its strategic plan  for CSI by including three key elements missing from the plan’s previous  iteration, and has achieved two performance goals by expanding CSI  locations and increasing the percentage of total U.S.-bound containers that  pass through CSI seaports. Concurrently, CBP reported an increase in the  number of high-risk containers examined by host governments  participating in CSI.
subsections: section_title: Recently Issued International Supply Chain Security Strategy Builds on DHS’s Existing Strategic Framework for Maritime Security
paragraphs: When it published the Strategy to Enhance International Supply Chain  Security in July 2007, DHS filled a gap that had existed between broad  national strategies and program-specific plans in the federal government’s  strategic planning framework for maritime security. Over the last 5 years,  DHS has made progress in developing a multilayered strategic framework  for securing the maritime domain, including the international supply chain.  This framework consists of high-level national strategies, such as the  National Strategy for Maritime Security and the Maritime Commerce  Security Plan, which describe the federal government’s broad approach to  maritime security. These plans are supplemented by a related hierarchy of  documents that includes the DHS strategic plan, the CBP strategic plan,  and the CSI program’s own strategic plan.
Prior to July 2007, the federal government’s maritime security framework  touched on many specific aspects of maritime trade and commerce, such  as how the CSI program contributes to securing containers bound for U.S.  seaports. However, it did not provide a detailed description of how  federal, state, and local authorities were to collaborate on supply chain  security specifically. In addition, Congress included a provision in the  SAFE Port Act of 2006 requiring DHS to develop a strategic plan to  enhance the security of the international supply chain. Moreover, the DHS  fiscal year 2007 appropriation act withheld $5 million from DHS until a  comprehensive strategic plan for port, cargo, and container security,  which included specific elements, had been submitted to specified  congressional committees.  In response, CBP contributed to the Strategy  to Enhance the International Supply Chain Security, which DHS  developed and issued in July 2007. According to DHS, the supply chain  security strategy is not meant to replace other strategic planning  documents, but seeks to harmonize the goals of the various plans and  programs into a multilayered, unified approach that can be further  developed by DHS components, including CBP.
This new strategic planning document for supply chain security delineates  the supply chain security roles, responsibilities, and authorities of federal,  state, local, and private sector entities. The strategy seeks to build on the  current multilayered strategic framework for maritime security by  establishing an overarching framework for the secure flow of cargo  through the supply chain—from point of origin to final destination. The  strategy describes how CBP’s portfolio of supply chain security  initiatives—including CSI, C-TPAT, cargo screening using ATS, the 24-hour  rule, and the use of nonintrusive inspection equipment to examine  containers—addresses the various stages in the supply chain. In addition,  the strategy provides details on how other organizations’ programs or  efforts—such as DOE’s Megaports initiative, which places radiation  detection equipment at foreign seaports—contribute to different aspects  of supply chain security. Figure 4 describes the major components of the  supply chain and the CBP initiatives that operate to secure them.
section_title: CBP Added Key Elements to the CSI Strategic Plan in Response to our 2005 Recommendation
paragraphs: At the program level, CBP has revised its CSI strategic plan, an important  component of the DHS strategic framework described above,  incorporating three critical elements that were absent from the plan’s  previous iteration. In our April 2005 report on CSI, we reported that the  CSI strategic plan lacked three of the six key elements identified by the  Government Performance and Results Act of 1993 for an agency strategic  plan, including descriptions of  1.  how performance goals and measures are related to program  objectives,  2.  the external factors beyond the control of CBP that could affect  the achievement of program objectives, and  3.  the evaluations that CBP conducts to monitor CSI.
We noted that, given the importance of having an effective strategic plan  for the program, we would continue to monitor CBP’s progress in refining  the plan. CBP has subsequently taken steps to address our concerns. In the  most recent version of the plan, released in August 2006, CBP included  information in three areas, as we had previously recommended. First, the  CSI strategic plan links each performance measure to the strategic goal it  supports. In addition, the plan describes how some performance measures  were designed to act as proxies for program objectives that can be difficult  to measure. Second, the CSI strategic plan also lists a variety of external  factors that have the potential to influence CSI operations, including  regional conflicts, organized crime, and changes in the political  administration of a foreign government participating in CSI. Finally, the  revised plan provides an explanation of the CSI team evaluation process,  thus addressing the third issue identified in our April 2005 report. We  discuss performance measure outcomes, other external factors, and CBP’s  evaluation process in greater detail later in this report.
section_title: CBP Met Performance Goals to Expand Number of CSI Seaports and to Increase Proportion of Total U.S.-bound Containers Passing Through CSI Seaports
paragraphs: The August 2006 CSI strategic plan set specific goals for expanding the  number of seaports participating in CSI, and set targets for related  increases in the percentage of total U.S.-bound containers that pass  through CSI seaports. As of September 2007, CBP reported meeting its  goals in both of these areas. Specifically, the plan called for CBP to expand  CSI program operations from 40 to 50 seaports by the end of fiscal year  2006, and to 58 seaports by the end of fiscal year 2007 (see appendix III for  a complete list of participating seaports). Having reached its goal of 58 CSI  seaports, CBP officials reported it currently does not have plans to add  other CSI seaports, as the costs associated with expanding the program  further would outweigh the potential benefits. In addition, the plan set a  performance target that by 2010, 86 percent of all U.S.-bound container  cargo was to pass through CSI seaports. According to CBP, when U.S.- bound containers pass through CSI seaports there is an opportunity for  high-risk cargo to be examined at the foreign seaport by the host  governments participating in CSI, rather than upon arriving at a U.S.  seaport. CBP reported that about 73 percent and about 80 percent of total  U.S.-bound container cargo passed through CSI seaports in fiscal years  2005 and 2006, respectively, and that it reached its 2010 goal early by  reaching approximately 86 percent by the end of fiscal year 2007. Figure 5  shows that as the number of operational CSI seaports expanded from 2002  to 2007, the proportion of total U.S.-bound container cargo passing  through CSI seaports also continued to increase.
In implementing the CSI program and reaching its goal of 58 operational  CSI seaports, CBP selected foreign seaports to participate in the program  in three phases. CBP officials reported using the following general  selection criteria for each phase as follows:  Most of the 23 phase I seaports were selected because they shipped the  highest volume of U.S.-bound container cargo.
The 19 phase II seaports were selected based on factors such as cargo  volume, strategic threat factors and the foreign government’s level of  interest in CSI.
The 16 phase III seaports were selected using the phase II criteria as  well as diplomatic or political considerations, such as the requests of  foreign governments already participating in CSI.
As CBP expanded the number of CSI seaports and increased the  proportion of total U.S.-bound container cargo passing through CSI  seaports, the agency also achieved increases in security activities that  occur at CSI seaports—targeting (CBP screens container cargo with ATS  to produce risk scores and conducts additional review or research to  ascertain risk levels) and examining high-risk container cargo (host  government officials examine high-risk containers by scanning with  nonintrusive inspection equipment or by physically searching the  container). As of September 2007 CBP reported fully targeting 100 percent  of all U.S.-bound container cargo to identify high-risk cargo as required by  the SAFE Port Act. In addition, foreign governments participating in CSI  have examined an increasing amount of high-risk container cargo as a  growing proportion of total U.S.-bound containers pass through CSI  seaports. In keeping with the CSI program’s risk-based approach, CBP  currently does not request that the host governments examine all U.S.- bound containers passing through the CSI seaports, just those that CBP  officers have determined to be high-risk. In fiscal year 2006, the number of  high-risk containers examined by host government officials at CSI  seaports increased by 77 percent from the previous year to almost 71,000  containers. In fiscal year 2007, examinations continued to increase,  reaching almost 137,000 containers. Moreover, in fiscal year 2007 CBP  reported that host government officials examined approximately 96  percent of the container cargo referred for examination. CBP reported that  about 4 percent of the referrals did not lead to examinations (about 5,600  requests) because (1) logistical difficulties arose, such as the container  had already been loaded on the shipping vessel (about 5,200 requests),or  (2) the host government denied the request (fewer than 400 requests).
section_title: To Strengthen CSI Operations, CBP Has Taken Steps to Address Human Capital Challenges and Enhance Host Government Relations, but Operational Challenges Remain
paragraphs: CBP has made various operational improvements to CSI, though  challenges remain. First, CBP has revised its human capital plan and  added permanent staff at CSI seaports, though it reports difficulties in  hiring and deploying qualified staff. Second, CBP’s relations with CSI host  governments we spoke to that conduct cargo examinations have improved  over time, though access to key examination-related information and  processes is limited by host governments at some CSI seaports. And  finally, CBP’s ability to conduct CSI program activities involves logistical  challenges that are inherent to many seaport environments, such as those  that are densely packed with equipment and personnel.
subsections: section_title: CBP Has Increased Permanent Staffing Levels at CSI Seaports, but Has Yet to Determine Optimum Distribution of Staff to Ensure All Critical Operations Are Performed
paragraphs: The ability of the CSI program to operate in accordance with its mission  and objectives depends, in part, on the success of its human capital  strategy—and CBP’s ability to manage and deploy staff in a way that  ensures that critical security functions are performed. Our April 2005  report on CSI noted that although CBP’s goal is to target all U.S.-bound  cargo shipped in containers at CSI seaports before they depart for the  United States, the agency had not been able to place enough officers at  some CSI seaports to do so. Specifically, CBP had developed a CSI staffing  allocation model to determine the staff needed to target container cargo.  However, at some CSI seaports CBP had been unable to staff the CSI  teams at the levels called for in the CSI staffing model. We noted that  CBP’s staffing model had not, at the time, considered whether some of the  targeting functions could be performed in the United States. We  recommended that CBP revise its staffing model to consider what  functions need to be performed at CSI seaports and what functions can be  performed in the United States, optimum levels of staff at CSI ports, and  the cost of locating CBP targeters overseas at CSI seaports instead of the  United States.
CBP has subsequently taken several steps to increase the number of CSI  officers and to implement our 2005 recommendations. For example, in  response to our concerns about staffing imbalances across seaports and  shortages at the highest-volume seaports, CBP has increased staffing  levels, bringing them closer to those called for in its staffing model— resulting in a parallel increase in the volume of container cargo that is  targeted.  Also, CBP has added 15 staff to CSI targeting duty at the NTCC  since 2005, composed of temporary and permanent officers. In addition, in  fiscal year 2007 CBP deployed an additional 125 permanent and 68  temporary officers to CSI seaports. Considering the officers at both CSI  seaports and the NTCC, as of November 2007, CBP had deployed 209 CSI  officers, which exceeds the 203 called for in the CSI staffing model. As a  result of these efforts, CBP officials told us that they had increased their  targeting of U.S.-bound container cargo from 65 percent in April 2005 to  100 percent in September 2007.
The agency also developed cost estimates for placing a mix of permanent  and temporary staff at CSI seaports (with permanent staff costing about  $330,000 per year and temporary staff about $275,000 per year) in response  to our recommendation. CBP reported that the advantages of placing  officers at CSI seaports on a permanent rather than a temporary basis  include greater opportunities for enhanced communication and  coordination with host governments, and less disruption due to fewer  rotations into and out of the country. At one CSI port that we visited, host  government customs officials told us that the presence of permanent staff  facilitated increased information sharing, which over time could lead to a  decrease in unnecessary examinations.
Despite the progress it has made, CBP continues to face staffing  challenges. CBP officials told us, for example, they continue to face  challenges in obtaining sufficient numbers of qualified officers to be  permanently deployed at CSI seaports. For example, CBP officials  reported that only 9 qualified applicants applied for 40 permanent  positions at CSI seaports. Officials told us that CSI must compete for staff  with targeting or seaport experience with other CBP programs or  positions, such as C-TPAT or other programs that operate at the NTCC. To  fill open positions at CSI seaports, CBP officials reported that in some  instances officers have been deployed who have not received all of the  required training. In addition, CBP evaluation data we reviewed showed  examples of CBP officers at CSI seaports lacking key skills, such as the  ability to target proficiently or communicate in the local language.
In addition, CBP has taken action to enhance its human capital planning  process for CSI, but has not yet included important factors in its staffing  allocation model. As we reported in 2005, one of the features of the CSI  staffing model that may contribute to staffing imbalances was its reliance  on placing officers overseas at CSI seaports.  It did not consider what  functions could be done in the United States. In May 2006, in response to  our recommendations, CBP issued a human capital plan that did not  specify that CSI targeting positions be located at CSI seaports, thus  recognizing that officers could support CSI seaports from the NTCC in the  United States. CBP officers assigned to the NTCC perform many of the  same roles as officers at CSI seaports, including reviewing bills of lading.  CBP officers at the NTCC review bills of lading for high-volume seaports  where the placement of the number of CSI officers required to review all  bills of lading is unfeasible. In addition, according to CBP officials, CBP  officers at the NTCC review bills of lading for U.S.-bound cargo from CSI  seaports where no CBP officers are stationed. Though CBP’s 2006 human  capital plan generally recognizes that some CSI functions can be  performed at either a CSI seaport or at the NTCC, the staffing allocation  model used to calculate the number of targeters necessary to review bills  of lading for each CSI port does not include factors that specify where  these positions should be located.
In addition, CBP’s staffing allocation model does not take into account  activities other than targeting—such as witnessing host government  examinations—that CSI officers perform at CSI seaports. According to  CBP, the agency stations as many of the total officers needed as possible  at the CSI seaports, but if the number of officers needed is higher than the  number of officers allowed by the host government or available to be  stationed in the seaport, then the remainder of the officers target from the  NTCC. However, we found that CBP has still not systematically  determined the optimal number of officers that need to be physically  located on-site at CSI seaports to carry out duties that require an overseas  presence (such as coordinating with host government officials or  witnessing the examinations they conduct) as opposed to other duties that  could be performed off-site in the United States (such as reviewing bills of  lading and databases). Also, CBP’s revised CSI human capital plan does  not include costs related to placing temporary staff at the NTCC and thus  does not have the data needed to conduct a cost-benefit analysis for  determining the optimal location for its CSI officers. As we noted in our  2002 report on a staffing framework for use at U.S. embassies, federal  agencies should consider factors such as cost and physical security of  foreign operations and consider options such as relocating staff to the  United States, as part of their framework for determining the right number  of staff to be placed overseas. Determining optimal staffing levels is  particularly important in light of ongoing challenges CBP reports facing to  identify sufficient numbers of qualified individuals to hire for the program,  and in light of the program’s recent expansion to additional seaports  around the world.
While CBP has taken steps to implement the recommendations from our  April 2005 report, further action is needed regarding the staffing allocation  model.  Specifically, as we recommended in 2005, the model should be  revised to consider (1) what functions need to be performed at CSI  seaports and what functions can be performed in the United States, (2) the  optimum levels of staff needed at CSI seaports to maximize the benefits of  targeting and examination activities in conjunction with host nation  customs officials, and (3) the cost of locating targeting positions overseas  at CSI seaports instead of in the United States.
section_title: Level of Collaboration between U.S. and Host Customs Officials Has Improved, but Challenges Remain at Some CSI Seaports
paragraphs: CSI’s strategic plan emphasizes the importance of CBP’s continued efforts  to foster partnerships with foreign customs officials at CSI seaports to  improve CSI operations. Specifically, according to CBP headquarters  officials, when CSI teams stationed at foreign seaports develop strong  interpersonal relations with foreign government officials, it leads to  increased trust and information sharing and thus improved targeting and  examination of high-risk cargo. While the extent of cooperation across all  of the 58 CSI seaports now operating is difficult to quantify, our  observations at 6 CSI seaports and our review of select CSI team  evaluations provide examples of how collaboration can benefit the CSI  program, and conversely, how the lack thereof can hinder progress.
At all 6 CSI seaports we visited, CBP officers or host government officials  told us that the relationship between the CSI team and the host  government has been positive or has improved over time. CBP and host  government officials we spoke with at all of the seaports we visited  reported that establishing trust and collegiality has led to increased  information sharing, resulting in more effective targeting and examination  of high-risk container cargo. For example, CBP officers noted instances in  which host customs officials would occasionally notify them of container  cargo they thought could be high-risk, so that CBP could take a closer look  at the information available in ATS related to the container cargo. In  addition, a few CBP officers or host government officials stated that the  presence of CSI teams at foreign seaports has in many instances helped to  prevent unnecessary examinations because information provided by host  government customs officials has led to lower risk profiles for certain  container cargo.
Moreover, CBP officials reported that strengthened relationships with host  government officials and the trade community have led host governments  to bolster their customs and port security practices. CBP officials we  spoke to emphasized that, like the United States, most foreign customs  administrations have traditionally focused on revenue collection and the  seizure of contraband, rather than security concerns. During our visits to  CSI seaports, the CBP and host government officials we spoke with  reported several examples of how the presence of CSI teams at seaports  has helped to expand the focus of the efforts of these foreign customs  administrations and the trade community to include enhanced security  practices. For example, one country developed databases with trade  information to achieve its customs goals and to assist CSI after seeing how  gathering historical data benefited CBP. Furthermore, at a couple of the  CSI seaports we visited, the CSI team or host government officials  arranged outreach meetings with the trade community to raise companies’  awareness of security practices and the benefits of providing correct and  complete data about their cargo.
During our visits to CSI seaports and our review of data CBP collected  during its evaluations of CSI teams, we also identified instances where  cooperation between CSI teams and their counterparts in the host  government could be improved—though, as CBP officials noted, some of  the factors involved are beyond CBP’s ability to control directly. For  example, in some locations, CBP officials reported that a country may  have laws that hinder the collaboration of host government officials with  CSI teams. We identified the following issues during our observations at 6  CSI seaports as well as from our review of CBP data collected in fiscal  year 2007 at an additional 12 CSI seaports (for a total of 18 CSI seaports):  At 9 CSI seaports, the CSI teams there reported that they only  interacted infrequently with their host government counterparts or the  host government officials did not readily share information that would  benefit CSI, such as knowledge about potentially suspicious container  cargo.  In one instance the lack of interaction was attributed to the host  government’s competing priorities.
At 6 CSI seaports, host governments restricted CSI teams from viewing  nonintrusive inspection equipment examinations conducted by host  customs authorities or the resulting images of the container’s contents,  which is one of the key purposes for staffing CBP officers at CSI  seaports.
At 4 CSI seaports, host governments prohibited the use of hand-held  radiation detection devices by CBP officials, which is considered by  CBP to be an important way to identify a potential anomaly in a high- risk container. According to CBP officials, a few of the countries  prohibit the equipment due to safety and health concerns about the use  of the equipment.
At 3 of the CSI seaports, host customs officials lacked access to  technical equipment, such as computers or nonintrusive inspection  equipment that worked properly, which CBP believes could limit their  ability to share customs-related information with CSI team members or  efficiently conduct examinations.  According to CBP officials,  sometimes host governments lack resources to meet these  technological needs.
At 6 CSI seaports in 2 countries, CBP officers at the seaport reported  that host customs administrations did not provide a sufficient number  of staff to assist CSI teams or the host government officials were often  unavailable, which, according to CSI teams, can sometimes lead to  delays in examining high-risk containers.
At 3 CSI seaports, there was evidence of challenges to effective  communication, such as some CSI teams having limited proficiency in  the local language.
These examples are not intended to represent the CSI program as a whole,  but are included to illustrate the types of challenges that CSI teams at the  seaports and CBP program managers face. CBP officials responsible for  managing the CSI program have reported that overall there has been a  high level of cooperation at CSI seaports, though they acknowledged that  the degree of involvement and participation that CBP officers have with  foreign customs officials during the examination of high-risk cargo varies  by country. It is also important to note that while CBP negotiates a  written, nonbinding arrangement stating expectations for inclusion in the  CSI program with the participating foreign governments, the agency  cannot compel foreign governments to offer information for the purposes  of CSI or to examine high-risk containers. Later in this report, we describe  the processes CBP has in place to address difficulties that may be  identified at the CSI seaports as part of its program oversight and  monitoring efforts.
section_title: Seaport Environment and Logistics Present Challenges to CSI Operations
paragraphs: Another factor that can affect CBP’s ability to conduct CSI program  operations involves logistical challenges that are inherent to many seaport  environments. For example, as illustrated in figure 6, foreign government  officials we spoke with at CSI seaports reported that many seaports are  densely packed with equipment and personnel, which can make it difficult  for host government customs officials to examine container cargo.
According to CBP, open space to place scanning equipment or to conduct  physical searches of containers can be scarce at some CSI seaports. For  example, in two of the CSI locations we visited, scanning equipment and  examination sites were placed several miles from where container cargo is  unloaded, loaded, or stored. According to the CBP officials we spoke with,  this adds to the costs and time required for examination and may result in  logistical difficulties in having high-risk U.S.-bound containers examined  before being loaded onto the shipping vessel. In addition, at one port we  visited, the host government limited the number of containers it would  examine, in part to limit the cost of examination and the amount of delay  caused by moving these containers, according to the CSI team we spoke  with. CBP officials reported that despite this limit to examine no more  than 250 containers (out of the over 115,000 container cargo shipments to  the United States from this seaport in fiscal year 2007), the country has not  denied many examination requests—only two in fiscal year 2007.  However, this ceiling was not based on risk factors, and an increase in  denied requests could lead to additional containers with high-risk cargo  departing for the United States without being examined.
Finally, CBP officials stated that containers at seaports are generally  stored in a container yard before they are loaded onto the shipping vessel.  These container yards may be very large, and containers in these yards are  often stacked to minimize the time required to load container vessels. As  shown in figure 7, containers on a vessel may be stacked several layers  deep.  Accordingly, CBP and host government officials we spoke to at a  few CSI seaports reported it can sometimes be challenging to access a  container for examination. CBP officials noted that any examinations  requested but not conducted in the CSI seaport would occur at a U.S.  seaport upon arrival.
section_title: CBP Has Enhanced Its CSI Evaluations at CSI Seaports and Performance Measures but Still Does Not Capture Critical Information about Host Government Examination Systems
paragraphs: CBP has enhanced how it collects CSI data by strengthening its approach  to conducting periodic evaluations of CSI officers at CSI seaports through  on-site evaluations of performance. However, weaknesses remain in how  CBP conducts evaluations, the information collected regarding host  government examination systems, and performance measurement of the  program as a whole. For example, CBP does not systematically collect  information on the equipment, people, and processes that are part of the  host government’s overall examination system. Also, while CBP has  refined and updated its performance measures, we identified remaining  limitations, such as the omission of measures for all core program  elements and several performance targets.
subsections: section_title: CBP Significantly Improved Its CSI Evaluations to Assess Program Operations at CSI Seaports, but Weaknesses Remain
paragraphs: CBP conducts evaluations at CSI seaports to determine the effectiveness  of the program. Specifically, CBP uses these on-site evaluations to assess  CSI team operations and capabilities, such as how well CSI team members  use ATS to determine the risk levels associated with U.S.-bound  containers passing through CSI seaports. CBP’s CSI strategic plan states  that these periodic reviews are intended both to ensure that deployed CSI  teams are adhering to standard operating procedures as well as to evaluate  the relationships between the teams and the host customs administrations.  In fiscal years 2006 and 2007, CBP reported conducting 42 and 45  evaluations, respectively. Since the program’s inception in 2002, the  agency reported conducting a total of 202 evaluations.
In November 2006, CBP significantly changed the way it conducts CSI  team evaluations. Prior to that time, CBP officials reported that its  evaluators relied on self-reported information from CSI team members on  how proficiently they performed CSI program activities. CBP’s current  approach to conducting CSI team evaluations seeks to provide a more  thorough review of CSI team performance.  According to CBP officials, the  agency now requires the CSI team members under review to demonstrate  their targeting competence to an evaluator, such as by physically showing  the evaluator how they review information about container cargo to  determine its risk level. To better assess the deployed CSI team’s  performance, CBP augmented its evaluation teams with officers who have  expertise in areas such as targeting and intelligence gathering.
Also, CBP has developed a new software tool that enables evaluators to  record evaluation data electronically, using laptop computers to conduct  the on-site evaluations. This tool, CSI Team Evaluation (CSITE), consists  of a series of yes or no questions that cover the various areas of CSI team  performance, including whether all of the container cargo that the CSI  team designated high-risk were examined and whether these actions were  properly documented. The CSITE tool also provides guidance on each  question and prompts evaluators as they conduct their review by, for  example, directing them to ensure that the CSI team is using the correct  settings in ATS. In addition, employing CSITE, CBP reported it can now  aggregate the results of some or all of its evaluations, a capability it  previously lacked, and can conduct statistical analyses of the results of the  evaluations. The agency can determine, for example, what percentage of  CSI team members successfully demonstrated proficiency in targeting  high-risk containers. According to CBP officials, CSITE will eventually  allow the agency to make comparisons of CSI performance across  seaports. Moreover, CBP now retains the information it collects at CSI  seaports and the resultant evaluation reports in a more systematic fashion.  CBP officials acknowledged that the agency did not always store this data  effectively prior to the implementation of the new evaluation system and  could not provide us with documentation of all of the evaluations it had  conducted since the program’s inception.
While these efforts should help to strengthen the CSI team evaluation  process, CBP is still not consistently collecting all available data to aid in  its analysis of CSI team performance, and we identified instances in which  the agency did not reconcile contradictory information it had collected.  Based on our review of CBP’s documentation associated with 34  evaluations to assess the information the agency collected and its methods  for doing so, we found that evaluators do not always answer all of the  questions contained in CSITE.  For example, the software tool instructs  the CBP evaluation team to collect information on whether  recommendations made in prior evaluations have been implemented. This  information could allow CBP to determine whether past problems have  been addressed, but it is not always provided by the evaluation team. We  also identified discrepancies between (a) the CSITE checklist of questions  that the evaluation team completes during the onsite evaluation, and (b)  the resulting evaluation report produced by CBP headquarters officials for  2 of the 14 locations for which we had both documents to compare. At one  seaport, for example, the CBP evaluation team indicated in the CSITE  checklist that the CSI team did not have all of the data systems it needed  to effectively target outbound shipments, whereas the evaluation report  stated the team had access to all of the appropriate targeting tools and  databases. With more complete information, collected in a consistent  manner, CBP may be better able to determine how well CSI teams are  performing, what corrective actions may be needed to improve the  program, or whether the CSI program is achieving its security goals.
section_title: CBP Lacks a Process for Systematically Gathering Information on Host Government Examination Systems, Which Include Equipment, People, and Processes
subsections: section_title: Host Government Examination Systems— Equipment
paragraphs: In April 2005, we recommended that CBP establish minimum technical  criteria required for the capabilities of nonintrusive inspection equipment  at CSI seaports, while considering sovereignty issues with participating  countries. CBP agreed to evaluate the feasibility of establishing such  criteria. In 2006, section 205(e) of the SAFE Port Act required DHS to  establish minimum technical capability criteria for the use of nonintrusive  inspection equipment and nuclear and radiological detection systems in  conjunction with CSI, but noted that these criteria should not be designed  to conflict with the sovereignty of host countries. In 2007, the 9/11 Act also  required the Secretary of DHS to develop technological standards for  scanning systems that will be used to conduct 100 percent scanning at  foreign seaports in the future and to ensure that these and other actions  implementing the act’s 100 percent scanning provisions do not violate  international trade obligations and are consistent with the World Customs  Organization framework or other international obligations of the United  States. CSI host governments, which are responsible for conducting  examinations of container cargo, purchase and operate nonintrusive  inspection equipment, though as of November 2007, 13 CSI seaports use  equipment on loan from the United States. The capabilities of this  inspection equipment vary by manufacturer and model. The equipment  may differ, for example, in its ability to penetrate steel shielding in order  to generate an image of container contents, or may scan containers at  different rates. Appendix IV describes the capabilities of this equipment in  greater detail.  As of November 2007, CBP had not yet implemented our  prior recommendation or taken actions to meet the SAFE Port and 9/11  Acts requirements for setting minimum technical criteria. CBP officials  stated that the reason for this is that they do not consider the agency to be  a standard-setting organization. While CBP refers host governments to the  World Customs Organization’ SAFE Framework regarding the  procurement of inspection equipment, this document does not include  specific technical criteria or standards. Moreover, they added that it is  important to acknowledge the inherent challenges involved in efforts to  ascertain the capabilities of nonintrusive inspection equipment that is  owned and operated by CSI host governments.
In May 2005, however, CBP put forth minimum technical criteria to  evaluate the quality and performance of nonintrusive imaging inspection  equipment being considered for use at U.S. seaports. These domestic  standards set baseline performance requirements for penetration, contrast  sensitivity, throughput, image quality, and scan size. To determine whether  certain types of nonintrusive inspection equipment were acceptable for  use at domestic seaports—and could meet the criteria that had been set— CBP conducted tests comparing the capabilities of nonintrusive imaging  inspection equipment provided by seven manufacturers with its technical  operating standards. On the basis of the test results, CBP recommended  the inspection equipment from five of the seven manufacturers for use at  domestic seaports, while equipment from two manufacturers was not  recommended. CBP officials stated that there are no plans to  systematically compare the capabilities of inspection equipment at CSI  seaports against these criteria for domestic equipment due to sovereignty  concerns.
CBP collects limited information on certain characteristics of the  inspection equipment installed at CSI seaports, such as manufacturer;  however, information related to capabilities and performance is not  generally obtained. Officials in CBP’s Office of Technology stated that they  have information on the capabilities of equipment that the United States  loans to other countries for 16 CSI seaports, and that only this equipment  can be assured of meeting the CBP domestic requirements.  However,  these CBP officials said that they had neither determined which other CSI  seaports use the inspection equipment that was assessed as part of CBP’s  test and recommended for use at domestic seaports, nor systematically  determined the specific capabilities of the equipment used at those CSI  seaports. Host government officials in the countries we visited stated that  they followed their country’s acquisition procedures, which included  reviewing equipment capabilities and performance, among other things,  for the purchase of nonintrusive imaging inspection equipment. However,  CBP does not have documentation on the testing used by the host  countries or the manufacturers to determine the basis for the equipment’s  stated performance or whether this stated performance is less than, meets,  or exceeds the criteria CBP established for equipment used at domestic  seaports.
According to CBP officials, the capabilities of nonintrusive inspection  equipment are vetted during an assessment phase of the CSI program,  when CBP is determining whether a seaport is prepared to operate within  CSI. While, as part of the assessment phase, CSI officials stated that they  collect descriptive technical information about the type of nonintrusive  inspection equipment to be used at seaports, we did not find—in our  review of CBP’s checklist used to guide its assessment teams as they  examine prospective CSI seaports—questions covering inspection  equipment other than general direction to ascertain whether some type of  this equipment was in place. Also, through our review of CBP’s  assessments of 10 CSI seaports—through which approximately 55 percent  of all U.S.-bound containers passed in fiscal year 2007—we did not find  any assessments that described the performance capabilities of the  equipment or judgments about the proficiency of host government officials  in operating these systems. CBP officials stated that the agency has never  prohibited a seaport from participating in CSI on the basis of its inspection  equipment, and CBP documents show that participation in the program  requires only that some type of nonintrusive inspection equipment be  available at or near the potential CSI port.
section_title: Host Government Examination Systems—People and Processes
paragraphs: The SAFE Port Act also directed DHS to (1) establish standard operating  procedures for the use of nonintrusive inspection equipment at CSI  seaports and (2) require CSI seaports to operate the equipment in  accordance with the criteria and operating procedures established by  DHS. Also, the 9/11 Act required DHS to develop operational standards  for scanning systems that will be used to conduct 100 percent scanning at  foreign seaports in the future. CBP officials stated that they recognize that  the capabilities of nonintrusive inspection equipment are only one element  for determining the effectiveness of examinations that take place at CSI  seaports. It is better, in their view, to make assessments of the whole  examination system, which includes nonintrusive inspection equipment,  personnel, and processes. However, CBP acknowledged it does not  systematically collect information on host governments’ use of  examination systems and has not developed general guidelines or criteria  that could provide CBP with the means to determine the quality of  examinations of high-risk container cargo bound for the United States.  CBP officials stated that they rely on CSI teams to notify headquarters if  they have concerns about the host government customs or examination  practices. Specifically, each CSI team leader is to meet weekly—usually  via teleconference—with a CSI manager located at CBP headquarters to  discuss ongoing CSI operations. However, CBP officials acknowledged  that equipment, capabilities, and examinations practices of host  government customs personnel are not routinely discussed.
CBP officials also reported that CSI team members witness most  examinations of high-risk U.S.-bound containers, and their presence at the  examinations would allow them to make judgments about aspects of the  host government’s examination system. However some host governments  specifically prohibit CSI team members from witnessing examinations.  Also we found that CBP officials did not routinely observe inspections at  one CSI seaport we visited, and were not always able to be present for  inspections at two other CSI seaports because those inspections were  scheduled and conducted when CBP officials were not available.
CBP officials told us that their CSI team evaluations are also a means of  capturing some information on various aspects of the host government’s  examination system. In order to participate in CSI, CBP requires that,  among other things, host governments have customs staff capable of  examining cargo originating in or transiting through its country and  maintain a program to prevent breaches in employee integrity. However,  the 15 CSI team evaluations we reviewed, which CBP had conducted since  the agency revised its evaluation process in November 2006, showed  limited coverage of whether host government customs personnel have  been trained to use nonintrusive inspection equipment or are using it  properly, the sufficiency of host staffing levels, and host government  efforts to ensure the integrity of their customs administration. Specifically,  6 of the 15 CSI team evaluations discussed whether equipment was used  properly, 1 discussed host staffing levels, and none discussed host  integrity programs.
section_title: CBP’s Lack of Information on Host Government Examination Systems Potentially Limits Assurance That Examinations of High- Risk Container Cargo are Effective
paragraphs: CBP’s lack of a systematic way to collect information on host  governments’ examination systems—including their equipment, people,  and processes—potentially limits CBP’s ability to ensure that  examinations of high-risk container cargo at CSI seaports can detect and  identify WMD. Without information on host governments’ examination  systems, CBP management may not be able to determine the reliability of  the host government’s inspections of high-risk U.S.-bound container cargo.  This is of particular concern since, according to CBP officials, most high- risk cargo that has already been examined at a CSI seaport, is generally  not reexamined once it arrives at a U.S. seaport.  CBP officials stated that  if problems are found in the examination process at a CSI seaport, then  high-risk container cargo would be reexamined upon arrival in the United  States.
As already noted, CBP must respect participating countries’ sovereignty.  CBP cannot require that a country use specific equipment.  However, if a  high-risk container was examined using an examination system found by  CBP to be less capable than established criteria, the agency could require  that the container be reexamined upon arrival at a U.S. seaport. CBP  officials stated that they believe that in general the equipment used by  participating governments meets or exceeds the capabilities of the  nonintrusive inspection equipment used at U.S. seaports. However,  because CBP has not set minimum technical criteria for nonintrusive  inspection equipment at CSI seaports, and the agency does not  systematically review the operations of the host government examination  systems at CSI seaports, CBP potentially has limited assurance that their  inspection equipment is capable of detecting and identifying potential  WMDs. In light of the new 9/11 Act requirement that 100 percent of U.S.- bound container cargo be scanned in the future with nonintrusive  inspection equipment at foreign seaports before leaving for the United  States, it is important that CBP have processes in place to gather the  information necessary to ensure that cargo container examinations—and  the equipment used as part of the examination process—are reliable,  regardless of the point of origin.
section_title: CBP Made Efforts to Refine CSI Performance Measures, but Did Not Fully Address our Previous Recommendation
paragraphs: While CBP has taken steps to strengthen performance measures for the  CSI program, we identified areas that did not fully address our April 2005  recommendation to develop outcome-based performance measures or  proxy measures of program functions—if program outcomes could not be  captured—and performance targets to track the program’s progress in  meeting its objectives. Whereas CBP’s CSI team evaluations and program  monitoring activities help to evaluate CSI operations at the seaport level,  CBP uses performance measures to gauge the effectiveness of the overall  program in meeting its broader strategic objectives for CSI across  seaports. By definition, performance measures are a particular value or  characteristic used to quantify a program’s outputs—which describe the  products and services delivered over a period of time—or outcomes— which describe the intended result of carrying out the program. A  performance target is a quantifiable characteristic that establishes a goal  for each measure; agencies can determine the program’s progress, in part,  by comparing the program’s measures against the targets. For example,  the target of one of CBP’s performance measures—the “number of  operational CSI seaports”—was to have 58 CSI seaports operating in fiscal  year 2007, which the agency achieved as described previously in this  report. The Government Performance and Results Act of 1993  incorporated performance measurement as one of its most important  features, and the establishment and review of performance measures are a  key element of the standards for internal control within the federal  government. As discussed in the Government Performance and Results  Act of 1993 and as we reported in 1996, measuring performance allows  organizations to track progress being made toward specific goals and  provides managers crucial information upon which to base their  organizational and management decisions. In addition, leading  organizations recognize that performance measures can create powerful  incentives to influence organizational and individual behavior.
In the past 2 years, CBP has made efforts to refine and modify its  performance measures as the CSI program has matured. Since 2005, for  example, CBP has eliminated five performance measures that it had used  to track the implementation of seaports participating in CSI, measures that  CBP determined were no longer needed because CSI operations were  under way at the majority of planned CSI seaports. Also, in our April 2005  review of CSI, we identified a CSI performance measure that was  calculated inappropriately, and in response, CBP modified how the  measure was calculated to address our concerns. Specifically, for the CSI  measure that tracks the number of container examinations waived  because they are determined to be unnecessary, CBP began excluding  inappropriate data that made the results of the performance measure  misleading. This was an important modification because, as we reported  in November 2002, measures that are defined inconsistently with how they  are calculated can be confusing and create the impression that  performance is better or worse than it actually is.
CBP has made efforts to enhance CSI performance measures, but we  identified limitations in the information available for CSI program  managers to assess the program. In the past, we and the Office of  Management and Budget have encouraged federal departments and  agencies to measure whether programs are achieving their intended  outcomes, such as CSI’s purpose of protecting global trade from being  exploited by international terrorists. However, we and the Office of  Management and Budget have acknowledged the difficulty in developing  outcome measures for programs that aim to deter or prevent specific  behaviors. In such an instance, we have reported that proxy measures  should be designed to assess the effectiveness of program functions. CBP  officials reported the agency has not been able to develop a way to  measure the deterrence effect of the program, as CSI is designed to  support the CBP mission to prevent and deter terrorists and terrorist  weapons from entering the United States. Examples of CSI program  functions include targeting and examining high-risk container shipments  before they are loaded on vessels bound for the United States, and in our  2005 review of CSI we provided guidance on an alternative method of  developing proxy measures to evaluate program performance. Further,  according to the Office of Management and Budget, proxy measures  should be closely tied to the intended program outcome, and it may be  necessary to have a number of proxy measures to help ensure sufficient  safeguards are in place to account for performance results. According to  CBP officials the following three of its existing performance measures  were proxies for program outcomes.(1) The percentage of worldwide U.S.-bound containers passing  through CSI seaports—since these containers are to be targeted  and, if determined high-risk, may be examined by host government  officials, this is a measure of the program goal to detect and  prevent WMDs headed to U.S. seaports from leaving foreign  seaports.  (2) The number of foreign mitigated examinations (that is,  examinations determined to be unnecessary due to information  provided by host government officials and thus waived) by  category—developed to quantify whether collocating CBP officials  at CSI seaports increases information sharing and collaboration.   (3) The number of intelligence reports based on CSI foreign sources— intended to measure whether having CBP officials located at  foreign seaports leads to increased collaboration with foreign  customs officials.
The Office of Management and Budget has stated that performance  measures should capture the most important aspects of a program’s  mission and priorities. However, CBP does not have a measure that tracks  the extent to which U.S.-bound containers carrying high-risk cargo are  examined at CSI seaports, despite the fact that this activity is a core  element of the CSI program.
CBP has taken other actions to address our April 2005 recommendation  that includes ways to improve CSI performance measures, but we found  additional weaknesses as well. The strategic plan demonstrated how each  performance measure corresponds to the three strategic goals of CSI,  which include (1) securing U.S. borders, (2) building a robust CSI cargo  security system, and (3) protecting and facilitating trade. This marked an  improvement, as this linkage had not been made previously. In addition,  CBP addressed an additional aspect of our prior recommendation by  establishing performance targets for four of the six CSI performance  measures currently used.  However, only one measure had a target for  multiple years.  In addition, since issuing the CSI strategic plan the agency  has not updated its performance targets for fiscal year 2008 or beyond for  any of its measures. Without this information about the performance  targets, it may be difficult for CBP to determine whether the results were  more positive or negative than expected.
Also, we identified a weakness in how some CSI performance measures  are calculated. As we noted earlier in this report, as the number of CSI  seaports has increased in recent years, program activities have increased  as well. However, CBP does not appropriately control for this program  growth in how it calculates three of its six performance measures. For  example, since the “number of foreign mitigated examinations by  category”—the number of container examinations determined to be  unnecessary due to information provided by host government officials—is  not calculated on a per-container basis (i.e., per 10,000 containers), it may  be difficult to determine whether fluctuation in the numbers across years  is due to (1) increased collaboration with foreign government officials or  (2) simply an increase in the number of containers reviewed and  considered for examination at the increasing number of CSI seaports.  Similarly, the number of intelligence reports and the number of  investigative cases initiated may be due to an increase in the number of  operational CSI seaports, not increased collaboration with host  government officials. Without controlling for program growth, CBP’s  calculation of results for its performance measures may be misleading or  confusing to CBP and DHS program managers or the Congress, who  provide program oversight.
section_title: Conclusions
paragraphs: Since we began reporting on the CSI program in 2003, CBP has made  significant progress in expanding and developing the program. However,  CBP continues to face several management and operational challenges,  which may limit CBP’s ability to ensure that the CSI program provides the  intended level of security for U.S.-bound container cargo moving through  the international supply chain. Also, balancing security concerns with the  need to facilitate the free flow of commerce remains an ongoing challenge  for CBP.
Recognizing that program evaluation data are important for program  managers to understand why results occur and what value a program  adds, CBP has taken actions to enhance its evaluation of CSI team  activities. The revised evaluation program has increased the information  available to make policy and programmatic decisions regarding the  operations at the CSI seaports. However, limitations that remain in CBP’s  evaluation process affect the accuracy and completeness of the program  information available for making sound management decisions about the  CSI program as a whole. Specifically, when CBP’s evaluation teams do not  complete the evaluation tools or resolve contradictory information,  program managers may receive limited or inaccurate information. Further,  when the data collected using the CSI evaluation tool during the  evaluations are not reliable and readily available for assessment, CBP’s  planned programwide trend analyses of the CSI program may be  misleading.
In assessing CSI performance, CBP lacks information about a very  important aspect of the program—the overall examination systems used  by the host governments to examine high-risk cargo shipped in containers  as requested by CBP. CBP’s efforts have led to the successful participation  of a wide array of foreign governments in the CSI program, and CBP has  established many cooperative relationships with its foreign partners. While  we acknowledge the agency cannot force security requirements upon  foreign governments, the lack of information systematically gathered  about the examination systems used by participating governments is  problematic. Data about the equipment, people and processes involved in  the examination system are vital for determining whether high-risk U.S.- bound containers have been properly examined or should be examined or  reexamined upon arrival at a U.S. seaport. CBP lacks guidelines and  criteria for most of the equipment and the people and processes used by  host government examination systems—as required, in some instances, by  the SAFE Port and 9/11 Acts—for evaluating CSI seaport operations and  determining overall program effectiveness. In light of the new 9/11 Act  requirement that 100 percent of U.S.-bound container cargo be scanned in  the future at foreign seaports before leaving for the United States, it is  important that that CBP have programs in place to gather the information  necessary to ensure that cargo container examinations—and the  equipment used as part of the examination process—are reliable,  regardless of the point of origin.
Program evaluations are just one source of information that managers  need to make decisions, and evaluation data must often be coupled with  performance measurement to assess overall program results. Measuring  the overall impact of the CSI program remains difficult due to the  challenges involved in creating effective performance measures, and  because of great difficulty in measuring the deterrent effect of the  program. As we and the Office of Management and Budget have reported,  performance measurement can be very valuable to program managers, as  the process can indicate what a program is accomplishing and whether  intended results are being achieved. Measuring program performance  encourages managers to focus on the key goals of a program and helps  them by providing information on how resources and efforts are best  allocated to ensure effectiveness.  Though CBP identified performance  measures it considers proxies for program outcomes (given the difficulty  in assessing the deterrent effect of CSI), these measures do not cover a  key core program function, for example a performance measure for the  number of high-risk U.S.-bound containers examined at CSI seaports.  Finally, without clearly developed performance targets for each of its  measures, program managers, Congress, and the public lack information  needed to determine the extent to which the CSI program is performing as  intended. Taken as a whole, the lack of clearly articulated performance  measures and accurate and reliable evaluative data may hinder CBP’s  ability to ensure that the resources it expends for CSI effectively achieve  its goal of helping to secure U.S. borders against terrorists and terrorist  weapons.
section_title: Recommendations for Executive Action
paragraphs: To help ensure that CBP has the information needed to assess its  achievement of CSI program goals to help enhance supply chain  security—while at the same time balancing security concerns with the  need to facilitate the free flow of commerce—we recommend that the  Secretary of Homeland Security direct the Commissioner of U. S. Customs  and Border Protection to take the following actions in three areas:  Strengthen CBP’s process for evaluating CSI teams at overseas ports  by (a) systematically capturing and maintaining all relevant evaluation  data and documentation so that it can be used by CBP management to  guide operating decisions, monitor program performance, and inform  resource allocation decisions; (b) ensuring that CSI evaluation teams  follow established evaluation procedures; and (c) monitoring the  completion, within established time frames, of recommendations made  in previous evaluations.
In collaboration with host government officials, improve the  information gathered about the host governments’ examination  systems—which includes people, processes, and equipment—at each  CSI port by (a) establishing general guidelines and technical criteria  regarding the minimal capability and operating procedures for an  examination system that can provide CBP with  a basis for determining  the reliability of examinations and related CSI activities; (b)  systematically collecting data for that purpose; and (c) analyzing the  data against the guidelines and technical criteria to determine what, if  any, mitigating actions or incentives CBP should take to help ensure  the desired level of security.
Enhance CSI performance measures to better assess CSI performance  overall by (a) developing measures for all core CSI program functions  designed to have a deterrent effect, (b) establishing annual  performance targets—based on explicit assumptions—for all  performance measures, and (c) revising how performance measures  are calculated to take into account CSI program growth.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Department of State and the  Department of Homeland Security for their review and comment.  The  Department of State did not provide written comments but provided  technical comments, which have been incorporated into the report as  appropriate. DHS provided written comments—incorporating comments  from CBP—on December 20, 2007, which are presented in Appendix II.  In  commenting on a draft of this report, DHS noted that it concurred with  one recommendation and partially concurred with the remaining two  recommendations.
In its written comments, DHS and CBP concurred with our  recommendation on strengthening its process for evaluating CSI teams at  overseas locations.  Specifically, CBP noted that by June 2008, it planned  to establish a database that would contain all recommendations and action  plans as a result of CSI port evaluations as well as due dates for  implementing recommendations and actions taken.  To ensure that CSI  evaluation teams follow procedures, CBP indicated that it would make it  mandatory that the teams complete all database fields.  Furthermore, CBP  reported that it would assign values to questions in its evaluation tool on  the basis of the criticality of the activity evaluated in each question to  CSI’s mission as a whole.
DHS commented that CBP partially concurred with our second  recommendation to improve information gathered about host  governments’ examination systems by (a) establishing general guidelines  and technical criteria regarding the minimal capability and operating  procedures; (b) systematically collecting data for that purpose; and (c)  analyzing the data against the guidelines and technical criteria.  CBP  agreed on the importance of an accepted examination process and noted it  continues to take steps in addressing improvements in the information  gathered about host government’s examination systems at CSI seaports by  working directly with host government counterparts, through the World  Customs Organization, and providing capacity building training and  technical assistance.  While CBP does engage in capacity building, it does  so with only 5 of the 33 countries with CSI ports.  CBP also stated that it  will continue to use the WCO through its SAFE Framework of Standards  to address a uniform customs process and technical standards for  equipment.  However, the SAFE Framework mentions no specific  technical capability criteria for inspection equipment.  Additionally, CBP  does not systematically collect or assess information on the people,  processes, or technology used by these host governments to examine high- risk U.S.-bound containers.  CBP also noted in its comments to this report,  that equipment used for inspection of containers in foreign countries is  equal to or better than the equipment used by CBP at its domestic ports.   While CBP has performance information for the 16 seaports that have  inspection equipment on loan from CBP, it is not in a position to assess the  performance of equipment used at the remaining 42 CSI seaports.   Although we repeatedly requested systematic information regarding the  equipment technical capabilities in these other ports, CBP officials were  unable to provide it to us.  In response to our 2005 report, CBP stated that  it would evaluate the feasibility of technical requirements for nonintrusive  inspection equipment, but a legal issue may exist regarding CBP’s ability to  impose such requirements.  While we understand CBP’s position, it could  still gather information on such equipment’s technical capabilities.   Because the CSI inspection might be the only inspection of a container  before it enters the United States, it is important that information on the  people, processes, and equipment used as part of CSI be obtained and  assessed to provide some level of assurance of the likelihood that the  examination system could detect the presence of WMD.  If a port’s  examination system were determined to be insufficient, CBP could take  mitigating actions, such as re-examining container cargo upon its arrival at  a domestic seaport.
Finally, DHS commented that CBP partially concurred with our third  recommendation to enhance CSI performance measures to better assess  CSI performance overall.  CBP stated that it believes its current measures  address core program functions of targeting and collaboration with host  governments to mitigate or substantiate the risk of a maritime container  destined for the United States.  We disagree.  As discussed earlier in this  report, a core element of the CSI program, specifically the extent to which  U.S.-bound containers carrying high-risk cargo are examined at CSI  seaports, is not addressed through CBP’s performance measures.  In its  comments, CBP stated that its outcome performance indicator captures  the number of foreign mitigated examinations by category, however CBP  did not respond to our requests for more information regarding these  categories, including whether risk was a category.  Although it considers  action on this recommendation completed, CBP noted its intention to  continue to refine, evaluate, and implement measures to track progress  toward meeting CSI objectives. As previously stated, since issuing the CSI  strategic plan, CBP has not updated its performance targets for fiscal year  2008 or beyond for any of its measures.  Thus, we believe additional action  is warranted.  Establishing annual targets for performance measures is  important, as agencies can determine the program’s progress, in part, by  comparing the performance measures against the targets.  In addition,  CBP did not address whether it plans to reconsider how it calculates some  of its performance measures to control for CSI program growth. Without  doing so, CBP’s calculation of results for its performance measures may be  misleading or confusing to CBP and DHS program managers, or the  Congress, who provide program oversight.
DHS and CBP also provided technical comments, which have been  incorporated into the report as appropriate.
If you or your staff have any questions about this report, please contact me  at (202) 512-9610 or at caldwells@gao.gov. Key contributors to this report  are listed in appendix VI. This report will also be available at no charge on  the GAO Web site at http://www.gao.gov.
section_title: Appendix I: Objectives, Scope, and Methodology
subsections: section_title: Objectives
paragraphs: We addressed the following issues regarding the U.S. Customs and Border  Protection’s (CBP) Container Security Initiative (CSI):    How has CBP contributed to strategic planning for supply chain  security efforts and the CSI program in particular, and what progress  has been made in achieving CSI performance goals?
How has CBP strengthened CSI operations in response to our 2005  review, and what challenges, if any, remain?
How does CBP evaluate CSI port operations and assess program  performance overall, and how has this process changed over time?
section_title: Scope and Methodology
paragraphs: To address our first objective, we reviewed the strategic plans of the  Department of Homeland Security (DHS), CBP, and CSI as well as national  strategies like the National Maritime Security Strategy and the Strategy  to Enhance International Supply Chain Security. We also analyzed the  CSI strategic plan to determine whether it includes all of the key elements  included in the Government Performance and Results Act. In addition, to  measure CSI’s progress in meeting its performance goals, we reviewed and  analyzed CBP data related to the number of CSI seaports, the cargo CBP  targeted and referred to the host government to examine, and the number  of cargo containers that were (and were not) examined by host  government officials at the CSI seaports. We also met with CBP officials  responsible for managing the CSI program, from the CSI Strategic  Planning and Evaluation Branch, and from CBP’s Office of Field  Operations and Office of International Affairs and Trade Relations, not  only to gather information about CSI strategic planning and performance  goals, but to discuss all of the issues within the scope of this review.
To examine CBP’s efforts to enhance CSI operations and the operational  challenges that remain at CSI seaports, we reviewed GAO’s previous  assessments of the CSI program and examined CBP’s efforts to implement  our three prior recommendations. We also reviewed the CSI human capital  plan and spoke to CBP officials about actions the agency has taken to  ensure that CSI human resources are appropriately allocated. As part of  that process, we met with officials at CBP headquarters and at the  National Targeting Center - Cargo (NTCC) in Virginia to discuss the  agency’s decision to conduct some targeting of high-risk containers from  the NTCC rather than at CSI seaports. In addition, we spoke to CBP  officials at three domestic seaports, selected according to geographical  location and container volume. We also visited six CSI seaports located  overseas, and selected the locations based on geographic and strategic  significance, container volume to the United States from the seaports,  when the seaports began conducting CSI operations, and whether the  seaport was involved in CBP’s Secure Freight Initiative. At the CSI  seaports, we also interviewed host government officials and CSI teams to  discuss the frequency and level of collaboration involved in their  interactions with each other, circumstances at seaport facilities that affect  CSI operations, and financial cost issues associated with examinations.  The results from our visits to seaports provided examples of CBP and host  government operations but cannot be generalized beyond the seaports  visited because we did not use statistical sampling techniques in selecting  the seaports.
To determine what progress CBP has made in strengthening its tools for  monitoring and measuring the progress of the CSI program, we reviewed  the performance measures presented in the CSI strategic plan against  criteria developed by the Office of Management and Budget and GAO. In  addition, to appraise CBP’s efforts to strengthen its methods to evaluate  CSI teams and to learn about operations at CSI seaports, we analyzed a  sample of evaluation documents. Our nonrepresentative sample consisted  of evaluations for all 40 seaports for which we had documentation at the  time of our review, including (1) the 15 evaluations conducted between  November 2006 (when CBP revised its evaluation process and began using  the Container Security Initiative Team Evaluation software tool) and May  2007 (when we conducted our analysis), (2) the 7 available evaluations  that directly preceded them chronologically and were conducted using  CBP’s previous evaluation methodology (for the purpose of comparison),  and (3) the most recent evaluations conducted at each of the additional  locations for which documentation had been provided by CBP. Thus, we  reviewed a total of 34 evaluations (covering 40 CSI seaports) out of the 114  evaluations that GAO had obtained from CBP as of May 2007. For each of  the evaluations reviewed, we assessed any available materials, which  could include a narrative report and/or a checklist of yes or no responses.  While our sample covered various aspects of CBP’s evaluations, our  sample was not selected using statistical sampling techniques.  Thus, the  results from our review of CBP evaluation data provide illustrative  examples about CSI team evaluation methods and program operations at  CSI seaports—and generally corroborated our seaport site visit  observations—but cannot be generalized to the all 58 seaports conducting  CSI operations.
We also met with CBP officials managing the CSI program to assess the  agency’s efforts to collect information about the equipment, people, and  processes involved in the host governments’ examinations of U.S.-bound  container cargo, including the capabilities of examination equipment  operating at CSI seaports and the proficiency of host customs  administrations using the equipment. In addition, we selected and  analyzed a nonrepresentative sample of 10 port assessments among those  that CBP conducted at each port prior to its admission into the CSI  program—the sample was composed of the 6 seaports we visited plus the  4 highest-volume locations as of January 2007. As of that date,  approximately 55 percent of containers bound for the United States  passed through these 10 seaports. Thus, our findings from our review of  the assessments provide examples about the type of information collected  as part of the process, but cannot be generalized to all 58 seaports in the  program.
We conducted this performance audit from May 2006 through January  2008 in accordance with generally accepted government auditing  standards.  Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives.  We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Data Reliability
paragraphs: We met with CBP officials to discuss the agency’s efforts to ensure CSI  data on the number of cargo shipments and containers subject to targeting  and examination are reliable. In our 2005 review of the program, we found  the data to be sufficiently reliable to support our findings. Since that time,  CBP has further enhanced the way in which it collects and aggregates  information about CSI program activities at foreign seaports, including the  targeting and examination of high-risk container cargo. Specifically, CSI  teams now utilize improved technology, eliminating the need for  transmitting data to CBP headquarters via e-mail and thereby reducing the  opportunity for human error in manually entering and aggregating data for  the program. CBP officials at headquarters can now directly access the  data entered at each CSI port as soon as they are entered into the shared  system and can monitor the data on a daily basis to identify errors in or  mischaracterization of the data. While we did not directly test the  reliability of 2006 data, the recent CBP initiatives to improve reliability,  combined with GAO's previous assessment of the 2005 data, gave us  confidence in using CSI targeting and examination data to provide  descriptive, background information regarding the extent to which high- risk container cargo is targeted by CBP and examined by foreign  governments participating in CSI.
section_title: Appendix II: Comments from the Department of Homeland Security
section_title: Appendix III: Container Security Initiative Seaports
paragraphs: This appendix provides information on the 58 foreign seaports  participating in CBP’s Container Security Initiative (CSI). According to  CBP, CSI was operating in 58 foreign seaports by the end of September  2007. Table 2 lists the CSI seaports according to the date when the  seaports began conducting CSI operations, shows the phase (I, II, or III) in  which specific seaports were selected for participation in CSI, the volume  of U.S.-bound shipments passing through the seaport in fiscal year 2007,  and specifies which seaports are participating in the Department of  Energy’s (DOE) Megaports Initiative and in CBP’s Secure Freight  Initiative.
section_title: Appendix IV: CSI Activities and Equipment
subsections: section_title: Targeting and Examining High-risk Containers
paragraphs: This appendix provides a detailed description of activities and equipment  used at CSI seaports to target and examine container cargo. CBP targets  all of the U.S.-bound containers that pass through CSI seaports to identify  and, where feasible, examine high-risk container cargo. The container  targeting and examination activities conducted at the foreign seaports for  U.S.-bound cargo (exports) are very similar to activities CBP conducts at  domestic seaports for arriving containers (imports). Figure 8 illustrates the  various steps and decision points involved in targeting and examining  high-risk U.S.-bound containers at CSI seaports.
Under CSI, the targeting of cargo can include the targeters’ review of the  Automated Targeting System (ATS) score and the information on which it  is based, the bills of lading—which include data about the cargo—and  additional information provided by host government officials. CBP  targeters at CSI seaports are to access bills of lading through ATS, a  system that automatically uses its hundreds of rules to check available  data for every container arriving in the United States and assigns a risk  score to each cargo shipment. Targeters review the bill of lading, making  a cursory check for discrepancies and anomalies in the name and address  of the importer, the commodity, the cargo description and other data  elements.  On the basis of the initial review of the bill of lading, CBP  officials are to either (1) categorize the cargo as low risk, in which case,  the container holding the cargo is loaded onto the departing vessel without  being examined, or (2) conduct further research in order to properly  characterize the risk level of the cargo.  Further research entails targeters  using automated resources, such as the Treasury Enforcement  Communication System or AutoTrack, as well as nonautomated resources,  such as information provided by host government officials, to obtain  applicable information to determine the validity of the shipment. Further  research may also be conducted by the team’s intelligence research  specialist.  After further research is completed, CBP officials are to  characterize the cargo as either (1) low risk, in which case it is loaded  onto the departing vessel without being examined, or (2) high-risk, in  which case it is referred to host government officials for concurrence to  examine. Since CBP officials do not have the legal authority to examine  U.S.-bound containers in foreign seaports, the host government customs  officials conduct the examinations.
Host government officials can respond to the referrals for examination in  one of three ways—cargo is examined or the request is either waived or  denied. After receiving a referral from CSI teams, host customs officials  are to review the bill of lading and the reasons for the referrals to  determine whether or not to examine the container cargo. Some host  governments collect intelligence information on U.S.-bound cargo  independent of CSI, which host officials also consider in decisions of  whether to examine the referred cargo. If host government officials agree  that the cargo is high-risk, they will proceed with an examination.  According to CBP, in general, CSI team members are to observe the  examinations and review and document the results. On the basis of the  results of a nonintrusive examination, such as if an anomaly is apparent in  the image of the container, the host government and CBP officials must  decide whether the host government will conduct a physical examination  of the a container.
Alternatively, the CSI team may waive an examination referral if (1) host  government officials provide the CSI team with additional information that  lowers the risk level of the cargo or (2) logistics prohibit an examination,  such as if the cargo container were already loaded on the departing vessel.  Finally, if the host government officials determine, on the basis of their  review, that the cargo is not high-risk, they will deny examination of the  cargo. For any high-risk cargo for which an examination is waived or  denied, CSI teams are to place a domestic hold on the cargo, so that an  examination will be conducted upon arriving in the United States.  However, if CSI team members are adamant that a cargo container poses  an imminent risk to the carrier or U.S. seaport of arrival but cannot  otherwise convince the host officials to examine the container, CSI team  members are to contact and coordinate with the NTCC to issue a do-not- load order for national security. According to CBP officials, this order  advises the carrier that the specified container will not be permitted to be  unloaded in the United States until a time when any associated imminent  risk to the cargo container is neutralized. Once the risk is neutralized, the  container is to be loaded back onto the carrier and placed on hold for a  domestic examination. According to CBP officials, this type of do-not-load  order for national security has been implemented six times since the  inception of CSI.
section_title: Equipment Used to Conduct Examinations of Cargo Containers
paragraphs: There are generally two types of CSI cargo container examinations— scanning with nonintrusive inspection equipment and physical searches.  For scanning cargo containers, there are two basic types of nonintrusive  inspection equipment currently used at CSI seaports: (1) radiation  detection equipment and (2) imaging inspection equipment, which may  use X-rays or gamma rays. Radiation detection equipment, such as a  radiation portal monitor (RPM) and radiation isotope identifier devices  (RIID) detects the presence of radioactive material that may originate  from a container.  However, only the RIID can determine whether the type  of radiation emitted by the material actually poses a threat or is a normal  emission of radiation, such as that found in ceramic tile.  We observed at a  domestic and a foreign seaport that generally if radioactive emissions are  detected from a cargo container, customs officials will use a RIID (shown  in fig. 9), to determine whether the radiation being emitted poses a threat.
The second type of equipment, referred to as imaging equipment, uses X- ray or gamma ray technology to scan a container and create images of the  container’s contents without opening the container.  CBP officials, along  with host government officials, may review the images produced with the  X-ray or gamma ray equipment to detect anomalies that may indicate the  presence of WMD.  Figure 10 shows a sample image produced by this type  of equipment.
The capabilities of nonintrusive imaging inspection equipment vary by  manufacturer and model. In May 2005, CBP defined minimum  performance capabilities to evaluate the quality and performance of the  nonintrusive imaging inspection equipment being considered for use at  domestic seaports. The domestic standards set baseline performance  requirements for such things as the ability of nonintrusive inspection  equipment to identify images through steel shielding (referred to as  penetration) or the ability to scan an amount of containers in a given time  (referred to as throughput).
section_title: Appendix V: CSI Performance Measures
paragraphs: This appendix provides information on the CSI performance measures  used by CBP. Table 3 describes the performance measures CBP is  currently using to report the overall performance of the CSI program, the  linkage between performance measures and CSI goals, the performance  targets established, and the recent results collected for each measure. In  addition, since our 2005 report, CBP has identified performance measures  one, two and three below as its proxy measures used in place of a measure  for program outcomes, given the difficulty in measuring the deterrent  effect of the program.
section_title: Appendix VI: GAO Contact and Staff Acknowledgments
paragraphs: Stephen L. Caldwell, Director, Homeland Security and Justice Issues,   (202) 512-9610, caldwells@gao.gov.
subsections: section_title: Staff Acknowledgments
paragraphs: This report was prepared under the direction of Christine A. Fossett,  Assistant Director, Homeland Security and Justice Issues. Key  contributions to this report also included Amy Bernstein, Fredrick Berry,  Yecenia Camarillo, Frances Cook, Christopher Conrad, Wendy Dye,  Kathryn Godfrey, Valerie Kasindi, Stanley Kostyla, Matthew Lee, Frederick  Lyles, Robert Rivas, and Leslie Sarapu.
section_title: Related GAO Products
paragraphs: Maritime Security: The SAFE Port Act: Status and Implementation One  Year Later. GAO-08-126T. Washington, D.C.: October 30, 2007.
Maritime Security: One Year Later: A Progress Report on the SAFE Port  Act. GAO-08-171T. Washington, D.C.: October 16, 2007.
Maritime Security: The SAFE Port Act and Efforts to Secure Our  Nation’s Seaports. GAO-08-86T. Washington, D.C.: October 4, 2007.
Combating Nuclear Smuggling: Additional Actions Needed to Ensure  Adequate Testing of Next Generation Radiation Detection Equipment.  GAO-07-1247T. Washington, D.C.: September 18, 2007.
Maritime Security: Observations on Selected Aspects of the SAFE Port  Act. GAO-07-754T. April 26, 2007.
Customs Revenue: Customs and Border Protection Needs to Improve  Workforce Planning and Accountability. GAO-07-529. Washington, D.C.:  April 12, 2007.
Cargo Container Inspections: Preliminary Observations on the Status of  Efforts to Improve the Automated Targeting System. GAO-06-591T.  Washington, D.C.: March 30, 2006.
Combating Nuclear Smuggling: Efforts to Deploy Radiation Detection  Equipment in the United States and in Other Countries. GAO-05-840T.  Washington, D.C.: June 21, 2005.
Container Security: A Flexible Staffing Model and Minimum Equipment  Requirements Would Improve Overseas Targeting and Inspection  Efforts. GAO-05-557. Washington, D.C.: April 26, 2005.
Homeland Security: Key Cargo Security Programs Can Be Improved.  GAO-05-466T. Washington, D.C.: May 26, 2005.
Maritime Security: Enhancements Made, but Implementation and  Sustainability Remain Key Challenges. GAO-05-448T. Washington, D.C.:  May 17, 2005.
Cargo Security: Partnership Program Grants Importers Reduced  Scrutiny with Limited Assurance of Improved Security. GAO-05-404.  Washington, D.C.: March 11, 2005.
Preventing Nuclear Smuggling: DOE Has Made Limited Progress in  Installing Radiation Detection Equipment at Highest Priority Foreign  Seaports. GAO-05-375. Washington, D.C.: March 31, 2005.
Homeland Security: Process for Reporting Lessons Learned from Seaport  Exercises Needs Further Attention. GAO-05-170. Washington, D.C.:  January 14, 2005.
Port Security: Better Planning Needed to Develop and Operate Maritime  Worker Identification Card Program. GAO-05-106. Washington, D.C.:  December 10, 2004.
Maritime Security: Substantial Work Remains to Translate New  Planning Requirements into Effective Port Security. GAO-04-838.  Washington, D.C.: June 30, 2004.
Homeland Security: Summary of Challenges Faced in Targeting  Oceangoing Cargo Containers for Inspection. GAO-04-557T. Washington,  D.C.: March 31, 2004.
Container Security: Expansion of Key Customs Programs Will Require  Greater Attention to Critical Success Factors. GAO-03-770. Washington,  D.C.: July 25, 2003.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.