Secure Border Initiative: Observations on the Importance of Applying Lessons Learned to Future Projects
Released: 2008-02-27T00:00:00

Letter

Background

CBP’s SBI program is responsible for identifying and deploying an appropriate mix of technology, known as SBInet (e.g., sensors, cameras, radars, communications systems, and mounted laptop computers for agent vehicles); tactical infrastructure (e.g., pedestrian and vehicle fencing, roads, and lighting); and personnel (e.g., program staff and Border Patrol agents) that are intended to enable CBP agents and officers to gain effective control of U.S. borders. SBInet technology is also intended to include the development and deployment of a common operating picture (COP) that provides uniform data through a command center environment to Border Patrol agents in the field and all DHS agencies and to be interoperable with stakeholders external to DHS, such as local law enforcement. The current focus of SBI is on the southwest border areas between the ports of entry that CBP has designated as having the highest need for enhanced border security because of serious vulnerabilities. The SBI program office and its offices of tactical infrastructure and SBInet are responsible for overall program implementation and oversight. Figure 1 is a map of the southwest border and the Border Patrol sectors.

In September 2006, CBP awarded a prime contract to the Boeing Company for 3 years, with three additional 1-year options. As the prime contractor, Boeing is responsible for acquiring, deploying, and sustaining selected SBI technology and tactical infrastructure projects. In this way, Boeing has extensive involvement in the SBI program requirements development, design, production, integration, testing, and maintenance and support of SBI projects. Moreover, Boeing is responsible for selecting and managing a team of subcontractors that provide individual components for Boeing to integrate into the SBInet system. The SBInet contract is largely performance-based—that is, CBP has set requirements for the project and Boeing and CBP coordinate and collaborate to develop solutions to meet these requirements—and designed to maximize the use of commercial off- the-shelf technology. CBP’s SBI program office oversees the Boeing-led SBI contractor team.

CBP is executing part of SBI’s activities through a series of task orders to Boeing for individual projects. As of February 15, 2008, CBP had awarded eight task orders to Boeing. Table 1 is a summary of the task orders awarded to Boeing for SBI projects.

In addition to deploying technology across the southwest border, the SBI program office plans to deploy 370 miles of single-layer pedestrian fencing and 300 miles of vehicle fencing by December 31, 2008. Pedestrian fencing is designed to prevent people on foot from crossing the border and vehicle fencing is physical barriers meant to stop the entry of vehicles. The SBI program office, through the tactical infrastructure program, is using the U.S. Army Corps of Engineers (USACE) to contract for fencing and supporting infrastructure (such as lights and roads), complete required environmental assessments, and acquire necessary real estate. In addition, in January 2008, CBP issued Boeing a supply and supply chain management task order for the purchase of construction items, such as steel.

In December 2006, DHS estimated that the total cost for completing the deployment along the southwest border will be $7.6 billion from fiscal years 2007 through 2011. DHS has not yet reported the estimated life cycle cost for the SBI program, which is the total cost to the government for a program over its full life, consisting of research and development, operations, maintenance, and disposal costs. Since fiscal year 2007, Congress has appropriated about $2.5 billion for SBI. DHS has requested an additional $775 million for SBI for fiscal year 2009.

First SBInet Technology Deployment Is Complete, but Lessons Have Been Learned

DHS announced its final acceptance of Project 28 from Boeing on February 22, 2008, completing its first efforts at implementing SBInet, and is now gathering lessons learned from the project that it plans to use for future technology development. The scope of the project, as described in the task order between Boeing and DHS, was to provide a system with the detection, identification, and classification capabilities required to control the border, at a minimum, along 28 miles within the Tucson sector. To do so, Boeing was to provide, among other things, mobile towers equipped with radar, cameras, and other features, a COP that communicates comprehensive situational awareness, and secure-mounted laptop computers retrofitted in vehicles to provide agents in the field with COP information. As we previously reported, Boeing delivered and deployed the individual technology components of Project 28—such as the towers, cameras and radars—on schedule. See figures 2 and 3 below for photographs of SBInet technology along the southwest border.

However, Boeing’s inability to integrate these components with the COP software delayed the implementation of Project 28 over 5 months after the planned June 13, 2007, milestone when Border Patrol agents were to begin using Project 28 technology to support their activities. Specifically, SBI program office officials said that the software that Boeing selected for the COP was intended to be used as a law enforcement dispatch system and was not designed to process and distribute the type of information being collected by the cameras, radars, and sensors. However, SBI officials told us that Boeing selected the system based on initial conversations with Border Patrol officials, but when deployed to the field, Boeing found limitations with the system. As we reported in October 2007, among other technical problems reported were that it was taking too long for radar information to display in command centers and newly deployed radars were being activated by rain or other environmental factors, making the system unusable. According to officials from the SBI program office, Boeing worked to correct these problems from July through November 2007. As one example of improvement, Border Patrol officials reported that Boeing added an auto focus mechanism on the cameras located on the nine towers. However, SBInet and Border Patrol identified issues that remain unresolved. For example, the Border Patrol reported that as of February 2008 problems remained with the resolution of the camera image at distances over 5 kilometers, while expectations were that the cameras would work at about twice that distance.

From June 26 through November 19, 2007, Boeing submitted three corrective action plans, documents that defined Boeing’s technical approach for correcting the problems associated with Project 28 and the steps that needed to occur for DHS to conditionally accept the system. As we reported in October, DHS officially notified Boeing in August 2007 that it would not accept Project 28 until certain problems were corrected. DHS conditionally accepted Project 28 on December 7, 2007, but included a requirement for Boeing to analyze the quality of the project’s video signals, radar data, and the timing of all components by January 11, 2008. Upon conditional acceptance, the government began operating Project 28, and SBI program office and Border Patrol officials told us that plans were under way to conduct additional testing of the system capabilities— including operational testing, which is used to determine that the system performs in the environment in which it is to operate. This testing was not scheduled to take place until after final acceptance of Project 28. According to SBI program office and Border Patrol officials, the results of this testing will not be used to make changes to Project 28, but will instead be used to guide future SBInet development. In addition, DHS announced its final acceptance of Project 28 on February 22, 2008 noting that Boeing met its contractual requirements. However, according to SBI program officials, the outcomes of future SBInet development will define the equipment that will replace most of Project 28 system components. Both SBI program office and Border Patrol officials stated that although Project 28 did not fully meet their expectations, they are gathering lessons learned and are ready to move forward with developing SBInet technologies that will better meet their needs. Table 2 summarizes key events for Project 28.

The SBI program office reported that it is moving forward with SBInet development beyond Project 28; however, it has revised its approach and timeline for doing so. As noted earlier in this statement, in addition to the $20.6 million task order awarded for Project 28, Boeing has also received other task orders as part of its overall contract with CBP. For example, in August 2007 DHS awarded a $69 million task order to Boeing to design the technical, engineering, and management services it would perform to plan and deploy SBInet system components within the Border Patrol’s Tucson, Yuma, and El Paso sectors. In addition, the SBI program office reported that on December 7, 2007, DHS awarded a 14-month task order worth approximately $64.5 million to Boeing to design, develop, and test, among other things, an upgraded COP software system for CBP command centers and agent vehicles, known as COP version 0.5. According to the SBI program office, planned SBInet development, such as the work being conducted by Boeing under these task orders, will eventually replace and improve upon Project 28. These officials stated that in light of the difficulties that DHS encountered during Boeing’s deployment of Project 28, the Secretary requested and CBP has proposed a revised strategy that is more deliberative. As two SBInet program managers put it, they want to develop SBInet “right, not fast”. We reported in October 2007 that SBI program office officials expected to complete all of the first phase of technology projects by the end of calendar year 2008. However, in February 2008, the SBI program office estimated that the first planned deployment of technology—including components linked to the updated COP—will occur in two geographic areas within the Tucson sector by the end of calendar year 2008, with the remainder of the deployments to the Tucson, Yuma, and El Paso sectors completed by the end of calendar year 2011. Officials from the SBI program office said that the Project 28 location is one of the two areas where the planned first deployments will occur. An official from the SBI program office noted that this schedule reflects DHS’s revised approach to developing SBInet technology and that meeting this timeline depends, in part, on the availability of funding. At this time, the SBI program office is still in the process of defining life cycle costs for SBInet development.

SBI program office and Border Patrol officials told us they have learned lessons during the development of Project 28 that will influence future SBInet development, including the technology that is planned to be deployed along the southwest border. For example, testing to ensure the components—such as radar and cameras—were integrated correctly before being deployed to the field at the Tucson sector did not occur given the constraints of the original 8-month timeline of the firm-fixed-price task order with Boeing, according to officials from the SBI program office. As a result, incompatibilities between individual components were not discovered in time to be corrected by the planned Project 28 deployment deadline. To address this issue moving forward with SBInet development, Boeing has established a network of laboratories to test how well the integration of the system works, and according to the SBI program office, deployment will not occur until the technology meets specific performance specifications.

Another lesson learned involved how the Project 28 system requirements were developed by Boeing. SBI program office and Border Patrol officials told us that the requirements for how the Project 28 system was to operate were designed and developed by Boeing with minimal input from the intended operators of the system, including Border Patrol agents. Instead, Boeing based the requirements for how Project 28 was to be designed and developed on information in the contract task order. The lack of user involvement resulted in a system that does not fully address or satisfy user needs. In February 2008, SBI program officials reported that Project 28 was designed to be a demonstration project, rather than a fully operating system, and there was not enough time built into the contract to obtain feedback from all of the intended users of the system during its design and development. While Border Patrol agents in the Tucson sector agreed with Boeing’s conceptual design of Project 28, they said the final system might have been more useful if they and others had been given an opportunity to provide feedback throughout the process. For example, Border Patrol agents told us they would have found the laptops mounted into agent vehicles safer and easier to use if they were larger and manipulated by a touch screen rather than with a pencil-shaped stylus, as using a stylus to manipulate the screen while driving is impractical. In addition, the laptops were not mounted securely enough to prevent significant rattling when driving on rough terrain, making the laptops difficult to use and prone to needing repair.

While user feedback was limited for Project 28, SBI program office officials have recognized the need to involve the intended operators when defining requirements and have efforts underway to do so for future SBInet development. For example, officials from the Border Patrol, CBP Air and Marine, and the CBP Office of Field Operations reported that representatives from their offices were involved in the development of requirements for SBInet technology as early as October 2006 and on an ongoing basis since then. Specifically, SBI program officials stated that Border Patrol users participated in requirements workshops with Boeing held in October 2006 at CBP headquarters and then at various field locations from December 2006 through June 2007, from which the SBInet operational requirements were derived (a process separate from Project 28). According to the SBI program office, users from other CBP offices such as the Office of Field Operations and Air and Marine have been involved in meetings as the SBI program office updates these requirements in preparation for the next development efforts. Additionally, SBI program officials stated that Boeing held meetings in January and February 2008 specifically designed to integrate user input to the development of the COP version 0.5.

Local Border Patrol Users Report that Project 28 is Not an Optimal System, but Those Trained on the System Will Operate it Until It Is Replaced

Since DHS conditionally accepted the task order from Boeing on December 7, 2007, those Border Patrol agents in the Tucson sector that have received updated training on Project 28 have been using the technologies as they conduct their border security activities. Border Patrol agents reported that they would have liked to have been involved sooner with the design and development of Project 28, since they are the ones who operate the system. Border Patrol officials stated that it is not an optimal system. Border Patrol agents from the Tucson sector provided examples of Project 28 capabilities that do not adequately support Border Patrol operations because of their design. As noted earlier in this statement, Border Patrol agents have had difficulties using the laptops mounted into agent vehicles to provide them with COP information. However, according to Border Patrol agents, Project 28 has provided them with improved capabilities over their previous equipment, which included items such as cameras and unattended ground sensors that were only linked to nearby Border Patrol units, not into a centralized command and control center. In addition, Border Patrol officials we spoke with at the Tucson sector noted that Project 28 has helped its agents become more familiar with the types of technological capabilities they are integrating into their operations now and in the future. As we reported in October 2007, the Border Patrol’s Tucson sector was developing a plan to integrate SBInet into its operating procedures. However, in February 2008 a senior official from the Border Patrol’s Tucson sector told us that the plan is still in draft form because of the delays in the deployment of Project 28.

In October 2007 we reported that the 22 trainers and 333 operators who were initially trained on the Project 28 system were to be retrained with revised curriculum because of deployment delays and changes to the COP software. As of January 2008, 312 Border Patrol operators and 18 trainers had been retrained on Project 28. According to Border Patrol agents we spoke with at the Tucson sector, a group of Border Patrol agents provided significant input into the revisions that the Boeing subcontractor made to the Project 28 training curriculum. Officials from the SBInet Training Division and Border Patrol agents reported that originally there were plans to train 728 Border Patrol operators located in the Project 28 area by January 2008. However, now no additional training will be conducted on Project 28, as they are expecting that future SBInet development will eventually replace Project 28. For example, according to the SBInet Training Division, the COP version 0.5 currently under development by Boeing will replace the Project 28 COP, and this will require new training.

Tactical Infrastructure Deployment on Schedule, but Further Deployment Will Be Challenging and Total Costs Are Not Yet Known

Deployment of tactical infrastructure projects along the southwest border is on schedule, but meeting the SBI program office’s goal to have 370 miles of pedestrian fence and 300 miles of vehicle fencing in place by December 31, 2008, will be challenging and total costs are not yet known. As of February 21, 2008, the SBI program office reported that it had constructed 168 miles of pedestrian fence and 135 miles of vehicle fence (see table 3).

According to SBI program office officials, the deployment of tactical infrastructure projects is on schedule, but these officials reported that keeping on schedule will be challenging because of various factors, including difficulties in acquiring rights to border lands. Unlike prior fencing projects that were primarily located on federal land, approximately 54 percent of planned projects are scheduled to be constructed on private property. We previously reported that as of July 2007, CBP anticipated community resistance to deployment for 130 of its 370 miles of pedestrian fencing miles. CBP officials told us that, of 480 owners of private property along the relevant segments of the border, all but 148 gave CBP access to survey their land prior to December 2007. In December, CBP, working in conjunction with the Department of Justice (DOJ), sent letters to most of the 148 remaining land owners reiterating the request for access and notifying them of the government's intent to pursue court-ordered access if necessary. As of February 16, 2008, approximately 50 percent of the land owners who received these letters had given CBP access to their land to do surveys. In some cases where access has not been granted, DOJ has begun the legal process known as “eminent domain” to obtain court-ordered access to the property. SBI program office officials state that they are working to acquire rights to border lands; however, until the land access issues are resolved, this factor will continue to pose a risk to meeting the deployment targets.

SBI program office officials are unable to estimate the total cost of pedestrian and vehicle fencing because they do not yet know the type of terrain where the fencing is to be constructed, the materials to be used, or the cost to acquire the land. In addition, in October 2007, we reported that to minimize one of the many factors that add to the cost, CBP has previously drawn upon its Border Patrol agents and Department of Defense military personnel to assist in such efforts. However, SBI program office officials reported that they plan to use more costly commercial labor for future infrastructure projects to meet their deadlines. In February 2008, SBI program office officials told us that they estimate construction costs for pedestrian fencing will be about $4 million per mile and vehicle fencing costs will be about $2 million per mile. However, total costs will be higher because this estimate does not include other expenses, such as contract management, contract incentives to meet an expedited schedule, higher-than-expected property acquisition costs, and unforeseen costs associated with working in remote areas.

As the SBI program office moves forward with tactical infrastructure construction, it is making modifications based on lessons learned from previous fencing efforts. For example, for future fencing projects, the SBI program office plans to buy construction items, such as steel, in bulk; use approved fence designs; and contract out the maintenance and repair of the tactical infrastructure. SBI program office officials estimate that buying essential items in bulk will make fencing deployment more economical and will reduce the likelihood of shortages and delays of critical equipment. SBI program office officials also believe that using pre- approved and tested fence designs (see fig. 4) will expedite preconstruction planning and will allow for more efficient maintenance and repair. In addition, the SBI program office plans to award a contract to maintain and service all initial, current, and future tactical infrastructure deployed through SBI because it believes that it will be more efficient than relying on Border Patrol agents and military personnel who also have other duties.

Progress Made to Meet Staffing Goals and a Human Capital Plan Published, but Implementation Is in the Early Stages

The SBI program office established a staffing goal of 470 employees for fiscal year 2008, made progress toward meeting this goal and published a human capital plan in December 2007; however, the SBI program office is in the early stages of implementing this plan. As of February 1, 2008, the SBI program office reported having 142 government staff and 163 contractor support staff for a total of 305 employees, up from 247 staff on September 30, 2007. In addition, SBI program office officials reported that they had selected an additional 39 staff that the program office is in the process of bringing onboard. These officials also told us that they believe they will be able to meet their staffing goal by the end of September 2008 and will have 261 government staff and 209 contractor support staff on board (see table 4). In addition, according to SBI program office officials, they would like to bring the ratio of government employees to contractor staff closer to 1:1 because their office has determined that that ratio provides the right mix of personnel with the skills necessary to ensure appropriate government oversight. The targeted ratio, based on the staffing goal for fiscal year 2008, would result in a better than 1:1 ratio of government to contract support staff.

In December 2007, the SBI program office published the first version of its Strategic Human Capital Management Plan and is now in the early implementation phase. As we have previously reported, a strategic human capital plan is a key component used to define the critical skills and competencies that will be needed to achieve programmatic goals and outline ways an organization can fill gaps in knowledge, skills, and abilities. The SBI program office’s plan outlines seven main goals for the office and includes planned activities to accomplish those goals, which align with federal government best practices. However, the activities are in the early stages of implementation. We have previously reported that a properly designed and implemented human capital program can contribute to achieving an agency’s mission and strategic goals. Until the SBI program office fully implements its plan, it will lack a baseline and metrics by which to judge the program. Table 5 summarizes the seven human capital goals, the SBI program office’s planned activities and steps taken to accomplish these activities, as of February 20, 2008.

Concluding Observations

Securing the nation’s borders is a daunting task. Project 28, an early technology project, resulted in a product that did not fully meet user needs and the project’s design will not be used as the basis for future SBInet development. To ensure that future SBInet development efforts deliver an operational capability that meets user needs and delivers technology that can be used in additional projects, it is important that the lessons learned on Project 28 continue to be applied and that user input continues to be sought so that future technology projects are successful. In the tactical infrastructure area, although fencing projects are currently on schedule, meeting future deadlines will be challenging because of various factors, including difficulties in acquiring rights to border land. Furthermore, future tactical infrastructure costs are not yet known because issues regarding land acquisition have not been resolved and other decisions, such as the materials to be used, have not been made. These issues underscore Congress’ need to stay closely attuned to DHS’s progress in the SBI program to make sure that performance, schedule, and cost estimates are achieved and the nation’s border security needs are fully addressed.

This concludes my prepared testimony. I would be happy to respond to any questions that members of the subcommittees may have.

Contacts and Acknowledgments

For questions regarding this testimony, please call Richard M. Stana at (202) 512-8777 or stanar@gao.gov. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Other key contributors to this statement were Susan Quinlan, Assistant Director; Deborah Davis, Assistant Director; Jeanette Espínola; Karen Febey; Michael Parr; Jamelyn Payan; David Perkins; Jeremy Rothgerber; and Leslie Sarapu.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.