Arizona Border Surveillance Technology Plan: Additional Actions Needed to Strengthen Management and Assess Effectiveness
Published: 2014-03-12T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: This testimony summarizes the information contained in GAO's March 3, 2014 report, entitled Arizona Border Surveillance Technology Plan: Additional Actions Needed to Strengthen Management and Assess Effectiveness , GAO-14-368 .
section_title: What GAO Found
paragraphs: The Department of Homeland Security's (DHS) U.S. Customs and Border Protection's (CBP) schedules and Life-cycle Cost Estimates for the Arizona Border Surveillance Technology Plan (the Plan) reflect some, but not all, best practices. Scheduling best practices are summarized into four characteristics of reliable schedules—comprehensive, well constructed, credible, and controlled (i.e., schedules are periodically updated and progress is monitored). GAO assessed CBP's schedules as of March 2013 for the three highest-cost programs that represent 97 percent of the Plan's estimated cost. GAO found that schedules for two of the programs at least partially met each characteristic (i.e., satisfied about half of the criterion), and the schedule for the other program at least minimally met each characteristic (i.e., satisfied a small portion of the criterion), as shown in the table below. For example, the schedule for one of the Plan's programs partially met the characteristic of being credible in that CBP had performed a schedule risk analysis for the program, but the risk analysis was not based on any connection between risks and specific activities. For another program, the schedule minimally met the characteristic of being controlled in that it did not have valid baseline dates for activities or milestones by which CBP could track progress.
Source: GAO analysis of CBP data.
Note: Not met—CBP provided no evidence that satisfies any of the criterion. Minimally met—CBP provided evidence that satisfies a small portion of the criterion. Partially met—CBP provided evidence that satisfies about half of the criterion. Substantially met—CBP provided evidence that satisfies a large portion of the criterion. Met—CBP provided complete evidence that satisfies the entire criterion.
Further, CBP has not developed an Integrated Master Schedule for the Plan in accordance with best practices. Rather, CBP has used the separate schedules for each program to manage implementation of the Plan, as CBP officials stated that the Plan contains individual acquisition programs rather than integrated programs. However, collectively these programs are intended to provide CBP with a combination of surveillance capabilities to be used along the Arizona border with Mexico, and resources are shared among the programs. According to scheduling best practices, an Integrated Master Schedule is a critical management tool for complex systems that involve a number of different projects, such as the Plan, to allow managers to monitor all work activities, how long activities will take, and how the activities are related to one another. Developing and maintaining an Integrated Master Schedule for the Plan could help provide CBP a comprehensive view of the Plan and help CBP better understand how schedule changes in each individual program could affect implementation of the overall Plan.
Moreover, cost-estimating best practices are summarized into four characteristics—well documented, comprehensive, accurate, and credible. GAO's analysis of CBP's estimate for the Plan and estimates completed at the time of GAO's review for the two highest-cost programs showed that these estimates at least partially met three of these characteristics: well documented, comprehensive, and accurate. In terms of being credible, these estimates had not been verified with independent cost estimates in accordance with best practices. Ensuring that scheduling best practices are applied to the three programs' schedules and verifying Life-cycle Cost Estimates with independent estimates could help better ensure the reliability of the schedules and estimates.
CBP did not fully follow key aspects of DHS's acquisition management guidance for the Plan's three highest-cost programs. For example, CBP plans to conduct limited testing of the highest-cost program—the Integrated Fixed Tower (IFT: towers with cameras and radars)—to determine its mission contributions, but not its effectiveness and suitability for the various environmental conditions, such as weather, in which it will be deployed. This testing, as outlined in CBP's test plan, is not consistent with DHS's guidance, which states that testing should occur to determine effectiveness and suitability in the environmental conditions in which a system will be used. Revising the test plan to more fully test the program in the conditions in which it will be used could help provide CBP with more complete information on how the towers will operate once they are fully deployed.
CBP has identified mission benefits for technologies under the Plan, but has not yet developed performance metrics. CBP has identified such mission benefits as improved situational awareness and agent safety. Further, a DHS database enables CBP to collect data on asset assists, defined as instances in which a technology, such as a camera, or other asset, such as a canine team, contributed to an apprehension or seizure, that in combination with other relevant performance metrics or indicators, could be used to better determine the contributions of CBP's surveillance technologies and inform resource allocation decisions. However, CBP is not capturing complete data on asset assists, as Border Patrol agents are not required to record and track such data. For example, from fiscal year 2010 through June 2013, Border Patrol did not record whether an asset assist contributed to an apprehension event for 69 percent of such events in the Tucson sector. Requiring the reporting and tracking of asset assist data could help CBP determine the extent to which its surveillance technologies are contributing to CBP's border security efforts.
This is a public version of a For Official Use Only—Law Enforcement Sensitive report that GAO issued in February 2014. Information DHS deemed as For Official Use Only—Law Enforcement Sensitive has been redacted.

section_title: Letter
paragraphs: I am pleased to be here today to discuss the findings from our March  2014 report, being released today, in which we assessed the Department  of Homeland Security’s (DHS) U.S. Customs and Border Protection’s  (CBP) efforts to develop and implement the Arizona Border Surveillance  Technology Plan (the Plan). In recent years, nearly half of all annual  apprehensions of illegal entrants along the southwest border with Mexico  have occurred along the Arizona border, according to DHS data. A top  priority for CBP is preventing, detecting, and apprehending illegal  entrants. In November 2005, DHS announced the launch of the Secure  Border Initiative (SBI), a multiyear, multibillion-dollar program aimed at  securing U.S. borders and reducing illegal immigration. CBP intended for  the SBI Network (SBInet) to include technologies such as fixed sensor  towers, a common operating picture, and tactical infrastructure to create a  “virtual fence” along the southwest border to enhance CBP’s capability to  detect, identify, classify, track, and respond to illegal breaches at and  between land ports of entry. At a cost of about $1 billion, in 2010, CBP  deployed SBInet systems, referred to as Block 1 systems, along the 53  miles of Arizona’s 387-mile border with Mexico that represent one of the  highest-risk areas for illegal entry attempts. However, in January 2011, in  response to internal and external assessments that identified concerns  regarding the performance, cost, and schedule for implementing the  systems, the Secretary of Homeland Security announced the cancellation  of further procurements of SBInet systems.
After the cancellation of SBInet in January 2011, CBP developed the  Plan, which includes a mix of radars, sensors, and cameras to help  provide security for the remainder of the Arizona border. Under the Plan,  CBP identified seven programs to be implemented ranging in estimated  costs from $3 million to about $961 million. The three highest-cost  programs under the Plan are the Integrated Fixed Tower (IFT), Remote  Video Surveillance System (RVSS), and Mobile Surveillance Capability  (MSC), accounting for 97 percent of the Plan’s estimated cost. In  November 2011, we reported on CBP’s development of, and estimated  life-cycle costs for, implementing the Plan. Specifically, we reported that  CBP needed more information for the Plan and its costs before  proceeding with implementation, and we recommended that CBP, among  other things, determine the mission benefits to be derived from the  implementation of the Plan and develop and apply key attributes for  metrics to assess program implementation, conduct a post  implementation review and operational assessment of SBInet, and update  the cost estimate for the Plan using best practices. DHS concurred with  these recommendations and has actions under way to address some of  them.
My testimony today is based on and summarizes the key findings of our  report on the status of the Plan, which was publicly released today. Like the report, my statement will address CBP’s efforts to (1) develop  schedules and Life-cycle Cost Estimates for the Plan in accordance with  best practices, (2) follow key aspects of DHS’s acquisition management  framework in managing the Plan’s three highest-cost programs, and (3)  assess the performance of technologies deployed under SBInet and  identify mission benefits and develop performance metrics for  surveillance technologies to be deployed under the Plan. To conduct work  for the March 2014 report, we analyzed DHS and CBP program  schedules and Life-cycle Cost Estimates and interviewed DHS and CBP  officials responsible for developing and overseeing schedules and cost  estimates, including officials from CBP’s Office of Technology Innovation  and Acquisition (OTIA), which manages implementation of the Plan. We  also analyzed DHS and CBP documents, including DHS Acquisition  Management Directive 102-01 and its associated DHS Instruction Manual  102-01-001, program briefing slides, budget documents, Acquisition  Decision Memorandums, and program risk sheets. Finally, we analyzed  performance assessment documentation and metrics used by CBP to  determine the effectiveness of technologies deployed under SBInet and  interviewed CBP officials responsible for performance measurement  activities, and analyzed CBP data on apprehensions, seizures, and asset  assists from fiscal year 2010 through June 2013 to determine the extent  to which the data could be used to measure the contributions of SBInet  technologies in enhancing border security. We conducted this work in  accordance with generally accepted government auditing standards.  More detailed information on the scope and methodology of our published  report can be found therein.
subsections: section_title: CBP’s Program Schedules and Life- cycle Cost Estimates Reflect Some but Not All Best Practices
paragraphs: In our March 2014 report, we assessed OTIA’s schedules as of March  2013 for the IFT, RVSS, and MSC programs and found that these  program schedules addressed some, but not all, best practices for  scheduling. The Schedule Assessment Guide identifies 10 best practices  associated with effective scheduling, which are summarized into four  characteristics of a reliable schedule—comprehensive, well constructed,  credible, and controlled. According to our overall analysis, OTIA at least  partially met the four characteristics of reliable schedules for the IFT and  RVSS schedules (i.e., satisfied about half of the criterion), and partially or  minimally met the four characteristics for the MSC schedule, as shown in  table 1. For example, we reported that the schedule for the IFT program  partially met the characteristic of being credible in that CBP had  performed a schedule risk analysis for the program, but the risk analysis  was not based on any connection between risks and specific activities.
We recommended that CBP ensure that scheduling best practices are  applied to the IFT, RVSS, and MSC schedules. DHS concurred with the  recommendation and stated that OTIA plans to ensure that scheduling  best practices are applied as far as practical when updating the three  programs’ schedules.
Further, in March 2014 we reported that CBP has not developed an  Integrated Master Schedule for the Plan in accordance with best  practices. Rather, OTIA has used the separate schedules for each  individual program (or “project”) to manage implementation of the Plan.  OTIA officials stated that an Integrated Master Schedule for the  overarching Plan is not needed because the Plan contains individual  acquisition programs as opposed to a plan consisting of seven integrated  programs. However, collectively these programs are intended to provide  CBP with a combination of surveillance capabilities to be used along the  Arizona border with Mexico. Moreover, while the programs themselves  may be independent of one another, the Plan’s resources are being  shared among the programs.
OTIA officials stated that when schedules were developed for the Plan’s  programs, they assumed that personnel would be dedicated to work on  individual programs and not be shared between programs. However, as  OTIA has initiated and continued work on the Plan’s programs, it has  shared resources such as personnel among the programs, contributing, in  part, to delays experienced by the programs. According to schedule best  practices, an Integrated Master Schedule that allows managers to monitor  all work activities, how long the activities will take, and how the activities  are related to one another is a critical management tool for complex  systems that involve the incorporation of a number of different projects,  such as the Plan. Thus, we recommended that CBP develop an  Integrated Master Schedule for the Plan.
DHS did not concur with this recommendation. In particular, DHS stated  that maintaining an Integrated Master Schedule for the Plan undermines  the DHS-approved implementation strategy for the individual programs  making up the Plan and that a key element of the Plan has been the  disaggregation of technology procurements. However, we continue to  believe that developing an Integrated Master Schedule for the Plan is  needed. As we reported in March 2014, this recommendation is not  intended to imply that DHS needs to re-aggregate the Plan’s seven  programs into a “system of systems” or change its procurement strategy  in any form. The intent of the recommendation is for DHS to insert the  individual schedules for each of the Plan’s programs into a single  electronic Integrated Master Schedule file in order to identify any resource  allocation issues among the programs’ schedules. Developing and  maintaining an Integrated Master Schedule for the Plan could allow OTIA  insight into current or programmed allocation of resources for all  programs as opposed to attempting to resolve any resource constraints  for each program individually.
In addition in March 2014, we reported that OTIA’s rough order of  magnitude estimate for the Plan and individual Life-cycle Cost Estimates  for the IFT and RVSS programs met some but not all best practices for  such estimates. Cost-estimating best practices are summarized into four  characteristics—well documented, comprehensive, accurate, and  credible.completed at the time of our review for the IFT and RVSS programs  showed that these estimates at least partially met three of these  characteristics—well documented, comprehensive, and accurate. In  terms of being credible, these estimates had not been verified with  independent cost estimates in accordance with best practices. We  recommended that CBP verify the Life-cycle Cost Estimates for the IFT   Our analysis of CBP’s estimate for the Plan and estimates  and RVSS programs with independent cost estimates and reconcile any  differences.
DHS said it concurred with this recommendation, although we reported  that DHS’s planned actions will not fully address the intent of the  recommendation unless assumptions underlying the cost estimates  change. In particular, DHS stated that at this point it does not believe that  there would be a benefit in expending funds to obtain independent cost  estimates and that if the costs realized to date continue to hold, there may  be no requirement or value added in conducting full-blown updates with  independent cost estimates. DHS noted, though, that if this assumption  changes, OTIA will complete updates and consider preparing  independent cost estimates, as appropriate. We recognize the need to  balance the cost and time to verify the Life-cycle Cost Estimates with the  benefits to be gained from verification with independent cost estimates.  However, we continue to believe that independently verifying the Life- cycle Cost Estimates for the IFT and RVSS programs and reconciling any  differences, consistent with best practices, could help CBP better ensure  the reliability of the estimates.
section_title: CBP Did Not Fully Complete Documents for Acquisition Decisions Consistent with the Guidance
paragraphs: In March 2014, we reported for the Plan’s three highest-cost programs— IFT, RVSS, and MSC—DHS and CBP did not consistently approve key  acquisition documents before or at the Acquisition Decision Events, in  accordance with DHS’s acquisition guidance. An important aspect of an  Acquisition Decision Event is the review and approval of key acquisition  documents critical to establishing the need for a program, its operational  requirements, an acquisition baseline, and test and support plans,  according to DHS guidance. On the basis of our analysis for IFT, RVSS,  and MSC programs under the Plan, we reported that the DHS Acquisition  Decision Authority approved the IFT program and the CBP Acquisition  Decision Authority approved the RVSS and MSC programs to proceed to  subsequent phases in the Acquisition Life-cycle Framework without  approving all six required acquisition documents for each program.  Furthermore, we reported that one document for the IFT program, five  documents for the RVSS program, and two documents for the MSC  program were subsequently approved after the programs received  authority to proceed to the next phase. DHS plans to complete and  approve those documents for the IFT, RVSS, and MSC programs that  have not yet been completed and approved.
With regard to one of the required documents—the Test and Evaluation  Master Plan—we reported in March 2014 that this document for the IFT  program, which was approved by DHS in November 2013, does not  describe testing to evaluate the operational effectiveness and suitability of  the system. Rather, the Test and Evaluation Master Plan describes CBP’s  plans to conduct a limited user test of the IFT. According to the Test and  Evaluation Master Plan, the limited user test will be designed to  determine the IFT’s mission contribution. According to OTIA and the Test  and Evaluation Master Plan, this testing is planned to occur during 30  days in environmental conditions present at one site—the Nogales  station. CBP plans to conduct limited user testing for the IFT under the  same process that is typically performed in any operational test and  evaluation, according to the Test and Evaluation Master Plan. The  November 2013 IFT Test and Evaluation Master Plan notes that, because  the IFT acquisition strategy is to acquire non-developmental IFT systems  from the marketplace (sometimes referred to as a commercial off-the- shelf system), a limited user test will provide Border Patrol with the  information it needs to determine the mission contributions from the IFTs,  and thus CBP does not plan to conduct more robust testing. However,  this approach is not consistent with DHS’s acquisition guidance, which  states that even for commercial off-the-shelf systems, operational test  and evaluation should occur in the environmental conditions in which a  system will be used before a full production decision for the system is  made and the system is subsequently deployed.
As we reported, we recognize the need to balance the cost and time to  conduct testing to determine the IFT’s operational effectiveness and  suitability with the benefits to be gained from such testing. Although the  limited user test should help provide CBP with information on the IFTs’  mission contribution and how Border Patrol can use the system in its  operations, the limited user test does not position CBP to obtain  information on how the IFTs may perform under the various  environmental conditions the system could face once deployed.  Conducting limited user testing in one area in Arizona—the Nogales  station—for 30 days could limit the information available to CBP on how  the IFT may perform in other conditions and locations along the Arizona  border with Mexico. As of November 2013, CBP intended to deploy IFTs  to 50 locations in southern Arizona, which can include differences in  terrain and climate throughout the year.
We recommended that CBP revise the IFT Test and Evaluation Master  Plan to more fully test the IFT program, before beginning full production,  in the various environmental conditions in which IFTs will be used to  determine operational effectiveness and suitability. DHS did not concur  with this recommendation and stated that the Test and Evaluation Master  Plan includes tailored testing and user assessments that will provide  much, if not all, of the insight contemplated by the intent of the  recommendation. However, as we reported in March 2014, we continue  to believe that revising the Test and Evaluation Master Plan to include  more robust testing to determine operational effectiveness and suitability  could better position CBP to evaluate IFT capabilities before moving to  full production for the system, help provide CBP with information on the  extent to which the towers satisfy Border Patrol’s user requirements, and  help reduce potential program risks.
section_title: CBP Has Identified Mission Benefits, but Does Not Capture Complete Data on the Contributions of Its Surveillance Technologies
paragraphs: We reported in March 2014 that CBP has identified the mission benefits  of its surveillance technologies, but does not capture complete data on  the contributions of these technologies, which in combination with other  relevant performance metrics or indicators, could be used to better  determine the contributions of CBP’s surveillance technologies and inform  resource allocation decisions. CBP has identified mission benefits of  surveillance technologies to be deployed under the Plan, such as  improved situational awareness and agent safety.
While CBP has defined these mission benefits, the agency has not  developed key attributes for performance metrics for all surveillance  technologies to be deployed as part of the Plan, as we recommended in  November 2011. In our April 2013 update on the progress made by the  agencies to address our findings on duplication and cost savings across  the federal government, CBP officials stated that operations of its two  SBInet surveillance systems identified examples of key attributes for  metrics that can be useful in assessing the Plan’s implementation for  technologies. For example, according to CBP officials, to help measure  whether illegal activity has decreased, examples of key attributes include  decreases in the amount of arrests, complaints by ranchers and other  citizens, and destruction of public and private lands and property. While  the development of key attributes for metrics for the two SBInet  surveillance systems is a positive step, CBP has not identified attributes  for metrics for all technologies to be acquired and deployed as part of the  Plan. Thus, to fully address the intent of our recommendation, CBP would  need to develop and apply key attributes for performance metrics for each  of the technologies to be deployed under the Plan to assess its progress  in implementing the Plan and determine when mission benefits have been  fully realized.
Furthermore, we reported in March 2014 that CBP is not capturing  complete asset assist data on the contributions of its surveillance  technologies to apprehensions and seizures, and these data are not  being consistently recorded by Border Patrol agents and across locations.  Although CBP has a field within its Enforcement Integrated Database  (EID) for maintaining data on whether technological assets, such as  SBInet surveillance towers, and non-technological assets, such as canine  teams, assisted or contributed to the apprehension of illegal entrants, and  seizure of drugs and other contraband, according to CBP officials, Border  Patrol agents are not required to record these data. This limits CBP’s  ability to collect, track, and analyze available data on asset assists to help  monitor the contribution of surveillance technologies, including its SBInet  system, to Border Patrol apprehensions and seizures and inform resource  allocation decisions.
In our March 2014 report, we defined an “apprehension or seizure event” as the  occasion on which Border Patrol agents make an apprehension of an illegal entrant or a  seizure of drugs or other contraband. The event is recorded in the EID and a date and  unique identifying number are assigned. An event can involve the apprehension of one or  multiple illegal entrants or types of items, and each individual illegal entrant apprehended  or type of item seized in the event is associated with the assigned unique identifying  number. Our analysis of apprehension events included instances in which an event had at  least one deportable individual. apprehension events. Since data on asset assists are not required to be  reported, it is unclear whether the data were not reported because an  asset was not a contributing factor in the apprehension or seizure or  whether an asset was a contributing factor but was not recorded by  agents.
As a result, CBP is not positioned to determine the contribution of  surveillance technologies in the apprehension of illegal entrants and  seizure of drugs and other contraband during the specified time frame.  We reported that an Associate Chief at Border Patrol told us that while  data on asset assists are not systematically recorded and tracked, Border  Patrol recognizes the benefits of assessments of asset assists data,  including those from surveillance technologies, such as the SBInet  system. The Associate Chief further noted that these data in combination  with other data, such as numbers of apprehensions and seizures, are  used on a limited basis to help the agency make adjustments to its  acquisition plans prior to deploying resources, thereby enabling the  agency to make more informed deployment decisions.
We recommended that CBP require data on asset assists to be recorded  and tracked within EID and that once these data are required to recorded  and tracked, analyze available data on apprehensions and technological  assists, in combination with other relevant performance metrics or  indicators, as appropriate, to determine the contribution of surveillance  technologies to CBP’s border security efforts. CBP concurred with our  recommendations and stated that Border Patrol is changing its data  collection process to allow for improved reporting on asset assists for  apprehensions and seizures and intends to make it mandatory to record  whether an asset assisted in an apprehension or seizure. DHS plans to  change its process by December 31, 2014.
Chairwoman Miller, Ranking Member Jackson Lee, and members of the  subcommittee, this concludes my prepared statement. I would be pleased  to answer any questions that you may have.
section_title: GAO Contact and Staff Acknowledgments
paragraphs: For questions about this statement, please contact Rebecca Gambler at  (202) 512-6912 or gamblerr@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this statement. Individuals making key contributions to this testimony  and our related report are Jeanette Espinola, Assistant Director, and  Michelle Woods, Analyst-in-Charge. Additional contributors include David  Alexander, Frances Cook, Joseph E. Dewechter, Jennifer Echard, Yvette  Gutierrez, Richard Hung, Jason Lee, Grant Mallie, Karen Richey, Doug  Sloane, Nate Tranquilli, Katherine Trimble, and Jim Ungvarsky.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.