Secure Border Initiative: DHS Needs to Follow Through on Plans to Reassess and Better Manage Key Technology Program
Released: 2010-06-17T00:00:00

Letter

Background

Managed by DHS’s Customs and Border Protection (CBP), SBInet is intended to strengthen CBP’s ability to detect, identify, classify, track, and respond to illegal breaches at and between ports of entry. The SBI Program Executive Office, which is organizationally within CBP, is responsible for managing key acquisition functions associated with SBInet, such as requirements management and risk management. Within the Executive Office, the SBInet System Program Office (SPO) is responsible for managing the day-to-day development and deployment of SBInet.

In September 2006, CBP awarded a 3-year contract to the Boeing Company for SBI, with three additional 1-year options. As the prime contractor, Boeing is responsible for designing, producing, testing, deploying, and sustaining the system. In September 2009, CBP extended its contract with Boeing for the first option year. CBP is acquiring SBInet incrementally in a series of discrete units of capabilities, referred to as “blocks.” Each block is to deliver one or more system capabilities from a subset of the total system requirements. The first block, known as Block 1, is to include a mix of surveillance technologies (e.g., cameras, radars, and sensors) and C3I technologies that are to produce a common operating picture—a uniform presentation of activities within specific areas along the border. Block 1 is to be initially deployed within the Tucson Sector to the Tucson Border Patrol Station (TUS-1) and to the Ajo Border Patrol Station (AJO-1). As of May 2010, the TUS-1 system is scheduled for government acceptance in September 2010, with AJO-1 acceptance in November 2010.

In January 2010, the DHS Secretary ordered a departmentwide reassessment of the program to include a comprehensive assessment of alternatives to SBInet to ensure that the department utilizes the most efficient and effective technological and operational solutions to secure the border. Pending the results of the assessment, the Secretary also froze all Block 1 expenditures beyond those needed to complete the implementation of the initial SBInet deployments to TUS-1 and AJO-1. Further, in March 2010, the department announced its plans to redeploy $50 million from its American Recovery and Reinvestment Act of 2009 funding to purchase currently available, stand-alone technology, such as remote-controlled camera systems called Remote Video Surveillance Systems, and truck-mounted systems with cameras and radar, called Mobile Surveillance Systems, to meet near-term operational needs.

Block 1 Capabilities, Geographic Coverage, and Performance Standards Have Continued to Decrease

In order to measure system acquisition progress and promote accountability for results, organizations need to establish clear commitments around what system capabilities will be delivered, and when and where they will be delivered. In September 2008, we reported that the scope of SBInet was becoming more limited without becoming more specific, thus making it unclear and uncertain what system capabilities would be delivered when and to what locations. Accordingly, we recommended that DHS establish and baseline the specific program commitments, including the specific system functional and performance capabilities that are to be deployed to the Tucson, Yuma, and El Paso Sectors, and establish when these capabilities are to be deployed and are to be operational.

To its credit, the SPO subsequently defined the scope of the first incremental block of SBInet capabilities that it intended to deploy and make operational; however, these capabilities and the number of geographic locations to which they are to be deployed have continued to shrink. For example, the number of component-level requirements to be deployed to the TUS-1 and AJO-1 locations has decreased by about 32 percent since October 2008 (see fig. 1).

In addition, the number of sectors that the system is to be deployed to was reduced from three border sectors spanning about 655 miles to two sectors spanning about 387 miles. Further, the stringency of the performance measures was relaxed, to the point that system performance is now deemed acceptable if it identifies less than 50 percent of items of interest that cross the border. According to program officials, the decreases are due to poorly defined requirements and limitations in the capabilities of commercially available system components. The result will be a deployed and operational system that does not live up to user expectations and provides less mission support than was envisioned.

A Reliable Schedule for Completing Block 1 Has Not Been Developed

The success of a large-scale system acquisition program, like SBInet, depends in part on having a reliable schedule of when the program’s set of work activities and milestone events will occur, how long they will take, and how they are related to one another. Among other things, a reliable schedule provides a road map for systematic execution of a program and the means by which to gauge progress, identify and address potential problems, and promote accountability. In September 2008, we reported that the program did not have an approved master schedule that could be used to guide the development of SBInet. Accordingly, we recommended that the SPO finalize and approve an integrated master schedule that reflects the timing and sequencing of SBInet tasks.

However, DHS has yet to develop a reliable integrated master schedule for delivering the first block of SBInet. Specifically, the August 2009 SBInet integrated master schedule, which was the most current version available at the time of our review, did not sufficiently comply with seven of nine schedule estimating practices that relevant guidance states are important to having a reliable schedule. For example, the schedule did not adequately capture all necessary activities to be performed, including those to be performed by the government, such as obtaining environmental permits in order to construct towers. Further, the schedule did not include a valid critical path, which represents the chain of dependent activities with the longest total duration in the schedule, and it does not reflect a schedule risk analysis, which would allow the program to better understand the schedule’s vulnerability to slippages in the completion of tasks.

These limitations are due, in part, to the program’s use of the prime contractor to develop and maintain the integrated master schedule, whose processes and tools do not allow it to include in the schedule work that it does not have under contract to perform, as well as the constantly changing nature of the work to be performed. Without having a reliable schedule, it is unclear when the first block will be completed, and schedule delays are likely to continue.

Cost-Effectiveness of Block 1 Has Not Been Demonstrated

The decision to invest in any system, or major system increment, should be based on reliable estimates of costs and meaningful forecasts of quantifiable and qualitative benefits over the system’s useful life. However, DHS has not demonstrated the cost-effectiveness of Block 1. In particular, it has not reliably estimated the costs of this block over its entire life cycle. To do so requires DHS to ensure that the estimate meets key practices that relevant guidance states are important to having an estimate that is comprehensive, well-documented, accurate, and credible. However, DHS’s cost estimate for Block 1, which is about $1.3 billion, does not sufficiently possess any of these characteristics.

Further, DHS has yet to identify expected quantifiable or qualitative benefits from this block and analyze them relative to costs. According to program officials, it is premature to project such benefits given the uncertainties surrounding the role that Block 1 will ultimately play in overall border control operations, and that operational experience with Block 1 is first needed in order to estimate such benefits. While we recognize the value of operationally evaluating an early, prototypical version of a system in order to better inform investment decisions, we question the basis for spending in excess of a billion dollars to gain this operational experience. Without a meaningful understanding of SBInet costs and benefits, DHS lacks an adequate basis for knowing whether the initial system solution is cost-effective.

Block 1 Has Not Been Managed in Accordance with Key Life Cycle Management Processes

Successful management of large information technology programs, like SBInet, depends in large part on having clearly defined and consistently applied life cycle management processes. In September 2008, we reported that the SBInet life cycle management approach had not been clearly defined. Accordingly, we recommended that the SPO revise, approve, and implement its life cycle management approach, including implementing key requirements development and management practices, to reflect relevant federal guidance and leading practices. To the SPO’s credit, it has defined key life cycle management processes that are largely consistent with relevant guidance and associated best practices. However, it has not effectively implemented these processes. In particular: The SPO revised its Systems Engineering Plan, which documents its life cycle management approach for SBInet definition, development, testing, deployment, and sustainment, in November 2008, and this plan is largely consistent with DHS and other relevant guidance. For example, it defines a number of key life cycle milestone or “gate” reviews that are important in managing the program, such as initial planning reviews, requirements reviews, system design reviews, and test reviews. The plan also requires most key artifacts and program documents that DHS guidance identified as important to each gate review, such as a risk management plan and requirements documentation. However, the SPO has not consistently implemented these life cycle management activities for Block 1. For example, the SPO did not review or consider key artifacts, including plans for testing and evaluating the performance of the system, as well as assessing the robustness of the system’s security capabilities, during its Critical Design Review, which is the point when, according to the plan, verification and testing plans are to be in place.

The SBInet Requirements Development and Management Plan states that (1) a baseline set of requirements should be established by the time of the Critical Design Review; (2) requirements should be achievable, verifiable, unambiguous, and complete; and (3) requirements should be bi- directionally traceable from high-level operational requirements through detailed low-level requirements to test plans. Further, the plan states that ensuring traceability of requirements from lower-level requirements to higher-level requirements is an integral part of ensuring that testing is properly planned and conducted. However, not all Block 1 component requirements were sufficiently defined at the time that they were baselined at the Critical Design Review. Further, operational requirements continue to be unclear and unverifiable, which has contributed to testing challenges, including the need to extemporaneously rewrite test cases during test execution. In addition, while requirements are now largely traceable backwards to operational requirements and forward to design requirements and verification methods, this traceability has not been used until recently to verify that higher-level requirements have been satisfied.

In 2008, the SPO documented a risk management approach that largely complies with relevant guidance. However, it has not effectively implemented this approach for all risks. Moreover, available documentation does not demonstrate that significant risks were disclosed to DHS and congressional decision makers in a timely fashion as we previously recommended, and, while risk disclosure to DHS leadership has recently improved, not all risks have been formally captured and thus shared. For example, some of the risks that have not been formally captured include the lack of well-defined acquisition management processes, staff with the appropriate acquisition expertise, and agreement on key system performance parameters. However, the SPO recently established a risk management process for capturing SBI enterprisewide risks, including the lack of well-defined acquisition management processes and staff expertise.

Reasons cited by program officials for not implementing these processes include their decision to rely on task order requirements that were developed prior to the Systems Engineering Plan and competing SPO priorities, including meeting an aggressive deployment schedule. Until the SPO consistently implements these processes, it will remain challenged in its ability to successfully deliver SBInet.

DHS Has Agreed to Implement GAO Recommendations Aimed at Addressing SBInet Longstanding Uncertainties and Risks

To address the program’s risks, uncertainties, and acquisition management weaknesses, our report being released today provides 12 recommendations.

In summary, we recommended that DHS limit future investment in SBInet to work that is either already under contract and supports the completion of Block 1 activities for deployment to TUS-1 and AJO-1 and/or provides a basis for a departmental decision on what, if any, expanded investment in SBInet is justifiable as a prudent use of DHS’s resources for carrying out its border security and immigration management mission. As part of this recommendation, we reiterated prior recommendations pertaining to program management challenges and recommended that DHS address weaknesses identified in our report by, for example, ensuring that the SBInet integrated master schedule, Block 1 requirements, and the Systems Engineering Plan, among other program elements, are consistent with best practices.

We also recommended that the program undertake a detailed cost-benefit analysis of any incremental block of SBInet capabilities beyond Block 1 and report the results of such analyses to CBP and DHS leadership. Further, we recommended that DHS decide whether proceeding with expanded investment in SBInet represents a prudent use of the department’s resources, and report the decision, and the basis for it, to the department’s authorization and appropriations committees.

To DHS’s credit, it has initiated actions to address our recommendations. In particular, and as previously mentioned, the department froze all funding beyond the initial TUS-1 and AJO-1 deployments until it completes a comprehensive reassessment of the program that includes an analysis of the cost and projected benefits of additional SBInet deployments, as well as the cost and mission effectiveness of alternative technologies.

Further, in written comments on a draft of our report, DHS described steps it is taking to fully incorporate best practices into its management of the program. For example, DHS stated that, in response to our previous recommendations, it has instituted more rigorous oversight of SBInet, requiring the program to report to the department’s Acquisition Review Board at specified milestones and receive approval before proceeding with the next deployment increment. With respect to our new recommendations, DHS stated that it is, among other things, taking steps to bring the Block 1 schedule into alignment with best practices, verifying requirements and validating performance parameters, updating its Systems Engineering Plan, and improving its risk management process.

In closing, let me emphasize our long held position that SBInet is a risky program. To minimize the program’s exposure to risk, it is imperative for DHS to follow through on its stated commitment to ensure that SBInet, as proposed, is the right course of action for meeting its stated border security and immigration management goals and outcomes, and once this is established, for it to ensure that the program is executed in accordance with proven acquisition management best practices. To do less will perpetuate a program that has for too long been oversold and under delivered.

This concludes my prepared statement. I would be pleased to respond to any questions that you or other Members of the Subcommittees may have.

Contact and Staff Acknowledgments

For questions about this statement, please contact Randolph C. Hite at (202) 512-3439 or hiter@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this testimony. Individuals making key contributions to this testimony include Deborah Davis, Assistant Director; David Alexander; Rebecca Alvarez; Carl Barden; Sylvia Bascopé; Tisha Derricotte; Neil Doherty; Nancy Glover; Dan Gordon; Cheryl Dottermusch; Thomas J. Johnson; Kaelin P. Kuhn; Jason T. Lee; Jeremy Manion; Taylor Matheson; Lee McCracken; Jamelyn Payan; Karen Richey; Karl W.D. Seifert; Matt Snyder; Sushmita Srikanth; Jennifer Stavros-Turner; Stacey L. Steele; Karen Talley; and Juan Tapia-Videla.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.