Border Patrol has reported that its primary mission is to prevent terrorists and weapons of terrorism from entering the United States and also to detect, interdict, and apprehend those who attempt to illegally enter or smuggle any person or contraband across the nation’s borders. Geographic responsibility for the southwest border is divided among nine Border Patrol sectors, two of which are in Arizona—Tucson and Yuma. Each sector has a varying number of stations, with agents responsible for patrolling within defined geographic areas. Border Patrol collects and analyzes various data on its enforcement efforts and the number and types of entrants who illegally cross the southwest border between the land ports of entry. These data include apprehensions and seizures of drugs and other contraband. The Border Patrol collects and maintains data on apprehensions and seizures in DHS’s Enforcement Integrated Database (EID). This database also includes an asset assists field in which agents can specify whether an asset, such as SBInet surveillance towers, contributed to apprehensions or seizures. CBP’s OTIA was created to help ensure CBP’s technology efforts are properly focused on the mission and are well integrated, and to strengthen CBP’s expertise and effectiveness in program management and acquisition. OTIA’s mission is to conduct and facilitate effective identification, acquisition, and life-cycle support of products and services while driving innovation to improve CBP’s performance in securing U.S. borders and facilitating lawful movement of goods and people. OTIA manages the implementation of the Plan and is acquiring seven technology programs in the Plan for use by Border Patrol in Arizona. The goal of the Plan is to achieve situational awareness along the Arizona border where the Plan’s technologies are deployed. For fiscal year 2013, OTIA budgeted $297 million in development and deployment funds for the Plan’s seven technology programs. Table 1 describes the Plan’s programs, and appendix II provides a photograph of each technology program. The overall policy and structure for acquisition management outlined in DHS Acquisition Management Directive 102-01 and its associated Instruction Manual 102-01-001 includes an Acquisition Life-cycle Framework to plan and execute the department’s acquisition programs. According to the directive, DHS adopted the Acquisition Life-cycle Framework to ensure consistent and efficient acquisition management, support, review, and approval throughout the department. As shown in figure 1, DHS’s Acquisition Life-cycle Framework includes four acquisition phases through which DHS develops, deploys, and operates new capabilities. During the first three phases, the DHS component pursuing the acquisition is required to produce key documents to justify, plan, and execute the acquisition. These phases each culminate in an Acquisition Decision Event where the Acquisition Review Board—a board of senior DHS officials—determines whether a proposed acquisition has met the requirements of the relevant acquisition framework phase and should proceed. The Acquisition Review Board is chaired by the Acquisition Decision Authority—the official responsible for ensuring compliance with Acquisition Management Directive 102-01. DHS classifies acquisitions into three levels that determine whether the Acquisition Decision Authority can be a Component Acquisition Executive or should be DHS’s Deputy Secretary or Under Secretary for Management.program is a Level 2 acquisition, which is overseen by the department, and the DHS Under Secretary for Management serves as the Acquisition Decision Authority. The other six programs in the Plan are Level 3 acquisitions, which are overseen by CBP’s Acquisition Review Board, and the Acquisition Decision Authority is a CBP official who serves as both the Assistant Commissioner for OTIA and Component Acquisition Executive. As of January 2014, CBP has awarded contracts for four of the Plan’s seven programs and has initiated or completed deployment of technology to Arizona for three of the four programs under contract, as shown in table 2. OTIA has developed a schedule for each of the Plan’s seven programs, and four programs will not meet their originally planned completion dates. OTIA established schedules for each program, serving as the original program plans with the required sequence of events, resource assignments, and dates for deliverables. However, as of March 2013, five of the Plan’s programs—IFT, RVSS, MSC, APSS, and UGS/IS—have experienced delays relative to their baseline schedules, as shown in figure 2. OTIA officials attributed program delays to various factors, including higher than expected numbers of proposals from vendors for some of the programs, system performance problems, and limited resources. In particular, OTIA officials stated that they initiated acquisitions for a number of the Plan’s programs around the same time, but OTIA did not have a sufficient number of acquisition staff with sufficient experience and skills to review contract proposals or manage the programs, a fact that contributed to program delays.for both the IFT and RVSS programs, the source selection process to decide which vendor would be awarded the contract was extended because of a higher than expected number of proposals received from vendors and a limited acquisition workforce to review and process the proposals, including not having a dedicated contracting officer for each of the programs. In addition, for the MSC program, OTIA officials attributed delays to problems both vendors who were awarded contracts experienced with their systems, as previously discussed. OTIA took various actions in response to these delays, such as extending the scheduled contract award date for the IFT and RVSS programs and extending scheduled activities for the MSC program from July 2014 to September 2015. For example, OTIA officials stated that According to best practices, in acquisition programs, agencies may make modifications to program schedules to reflect changes to programs; CBP has consistently updated each program’s schedule in response to program delays. However, we assessed OTIA’s schedules as of March 2013 for the three highest-cost technology programs—IFT, RVSS, and MSC—and found that these program schedules addressed some, but not all, best practices for scheduling. The Schedule Assessment Guide identifies 10 best practices associated with effective scheduling, which are summarized into four characteristics of a reliable schedule— comprehensive, well constructed, credible, and controlled. summarizes our assessment of the IFT, RVSS, and MSC schedules. Appendix III provides more detailed information on the description of each best practice and on the results of our assessment. GAO-12-120G. A schedule risk analysis is performed to calculate the amount of contingency time that is needed to complete the program on time. According to our overall analysis, OTIA at least partially met the four characteristics of reliable schedules for the IFT and RVSS schedules and partially or minimally met the four characteristics for the MSC schedule. For example: Comprehensive: OTIA’s schedule for the IFT, RVSS, and MSC programs partially met best practices in terms of being comprehensive. For example, our analysis found that all three program schedules reflected the work that needed to be accomplished for the schedules, and each schedule had duration estimates that at least substantially met best practices. The IFT schedule contained a clear start and a finish milestone, and the RVSS schedule contained at least a clear start milestone. However, the schedules for these programs did not meet other best practices in terms of being comprehensive. For example, the MSC schedule did not contain fields that map activities to a program work breakdown structure; and the schedules for the IFT and RVSS programs did not fully map all schedule activities to each program’s work breakdown structure in accordance with best practices. Moreover, the IFT and RVSS schedules did not include the level of detail expected to provide oversight of ongoing construction work, as activities associated with the construction work were reflected in the schedules as milestones, limiting OTIA’s ability to monitor the progress of these efforts. Specifically, these activities were reflected in the schedules as a milestone that was a point in time, rather than a range of time, as called for by best practices. In addition, resources were not assigned to some activities in all three schedules. According to best practices, a schedule without resources implies an unlimited number of resources and their unlimited availability. Best practices note that assigning resources to activities across programs can help prevent any future overallocation of resources. Well constructed: OTIA’s schedule for the IFT program substantially met the characteristic of being well constructed; the schedules for RVSS and the MSC programs partially met this characteristic. For example, our analysis found the IFT program schedule had few missing or incorrect logic links and the critical path—the chain of dependent activities with the longest total duration—was found to be a straightforward, continuous path of activities that depicted the effort driving the key milestones. Our analysis of the RVSS and MSC program schedules found that these schedules had no missing or incorrect logic links. However, we could not verify a reliable critical path that was continuous from the status date to contract award for these schedules. In addition, our analysis shows that each of the three programs’ schedules exhibited unreasonable amounts of total float—that is, the amount of time by which an activity can slip before the delay affects the program’s estimated finish date appeared to be overestimating true schedule flexibility. For example, 25 percent of the activities in the IFT schedule appeared to be able to slip at least 10 working months before affecting the final milestone of the program. Credible: OTIA’s schedules for the IFT and RVSS programs partially met the characteristic of being credible; the MSC program schedule minimally met this characteristic. For example, our analysis found that the IFT and RVSS schedules responded when significant delays were introduced into the planned activities in the schedules; that is, when we tested the robustness of the schedules by extending activity durations, forecasted dates recalculated appropriately. However, the MSC schedule responded to schedule delays in some instances but not in others, and some forecasted dates did not recalculate to account for changes we made in the duration of activities when testing the MSC schedule. Additionally, OTIA performed a risk analysis for the IFT and RVSS programs; however, the IFT and RVSS analyses did not include the risks most likely to delay the project or how much contingency reserve (that is, time held in reserve for potential delays) was needed for each schedule. For the MSC schedule, OTIA did not conduct a schedule risk analysis because, according to program officials, OTIA did not have a tool for conducting schedule risk assessment at the time the MSC schedule was developed. According to best practices, without this analysis, the program office may not sufficiently understand the level of confidence in meeting the program’s completion date and identify any potential reserves for contingencies. Controlled: OTIA’s schedules for the IFT and the RVSS programs partially met the characteristic of being controlled; the MSC program schedule minimally met this characteristic. For example, our analyses determined all three schedules were well maintained, updated periodically by a trained scheduler, and contained no out-of-sequence activities. We also found that the IFT and the RVSS schedules contained no date anomalies, but the MSC schedule did have anomalies. For example, the MSC schedule contained 13 activities in the past with no actual start or finish dates. Further, our analysis showed that none of the schedules had valid baseline dates for activities or milestones by which management could track current performance. The IFT baseline schedule was originally approved in July 2011, and the baseline for the RVSS was approved in September 2012; however, both of these programs have been delayed. Rebaselining resets the estimated schedule that is used to determine how the program will be held accountable. Once a program is rebaselined, OTIA officials stated that the office plans to report on the performance of the program based on the revised schedule. However, none of the schedules we assessed contained valid baseline dates that could be used to track on-time, delayed, or accelerated effort. For example, a baseline schedule was not established for the MSC program and both the IFT and RVSS schedules were missing some baseline dates for activities and milestones. In addition, according to our analyses, none of the three schedules were supported by a schedule baseline document, which is a single document that defines the organization of a schedule, describes the logic of the network, describes the basic approach to managing resources, and provides a basis for all parameters used to calculate dates. OTIA officials stated that the Acquisition Program Baseline for both the IFT and RVSS serves as the baseline schedule document, which defines the cost, schedule, and performance baselines; however, the Acquisition Program Baseline and related guidance present an overview of OTIA schedule policy rather than assumptions specific to individual program schedules. OTIA officials stated that they believe the schedules for the IFT, RVSS, and MSC programs are generally reliable, but also stated that these schedules may not fully meet all best practices. OTIA officials stated that they plan to rebaseline the IFT and RVSS program schedules after contract award and the MSC program schedule after contract negotiations. Rebaselining these schedules would help OTIA better address some of the best practices, such as to help ensure a more full and consistent allocation of resources, to address gaps in the critical path to program completion, and to address schedule risk assessments. However, OTIA’s plans to rebaseline the schedules would not position OTIA to meet all best practices, which are designed to ensure reliable schedules. According to best practices, to be considered reliable, a schedule must substantially or fully meet all four schedule characteristics. As our analysis indicates, OTIA does not have the information it needs in the schedules to effectively use them in managing and overseeing the IFT, RVSS, and MSC programs. While OTIA’s plans to rebaseline the schedules are positive steps, ensuring that all schedule best practices are applied to the IFT, RVSS, and MSC schedules when updating them could help OTIA better ensure the reliability of the three programs’ schedules and could help better position OTIA to identify and address any potential further delays in the programs’ commitment dates. OTIA has not developed an Integrated Master Schedule for scheduling, executing, and tracking the work to implement the Plan and its seven programs. Rather, OTIA has used the separate schedules for each individual program (or “project”) to manage implementation of the Plan. The use of an Integrated Master Schedule is a well-established practice in program and project management and is a necessary tool for coordination of independently managed projects that have dependencies—including resource dependencies—on one another.According to schedule best practices, an Integrated Master Schedule shows the effect of delayed or accelerated government activities on contractor activities, as well as the opposite effect for multiple programs. In addition, an Integrated Master Schedule that allows managers to monitor all work activities, how long the activities will take, and how the activities are related to one another is a critical management tool for complex systems that involve the incorporation of a number of different projects, such as the Plan. OTIA officials stated that an Integrated Master Schedule for the overarching Plan is not needed because the Plan contains individual acquisition programs as opposed to a plan consisting of seven integrated programs. However, collectively, these programs are intended to provide Border Patrol with a combination of surveillance capabilities to assist in achieving situational awareness along the Arizona border with Mexico, as referenced in CBP’s planning documents. As a document that integrates the planned work, the resources necessary to accomplish that work, and the associated budget, an Integrated Master Schedule provides information and oversight regarding the schedule. According to best practices, an Integrated Master Schedule also helps agencies monitor progress against overall completion dates. However, OTIA has not established a target completion date for an Integrated Master Schedule for the overall Plan. Department of Homeland Security, Multi-Year Investment and Management Plan for Border Security Fencing, Infrastructure, and Technology (BSFIT) for Fiscal Years 2014- 2017 (Washington, D.C.: Apr. 17, 2013). dedicated contracting officers to plan and execute the programs’ source selection and environmental activities. Developing and maintaining an Integrated Master Schedule for the Plan could allow OTIA insight into current or programmed allocation of resources for all programs as opposed to attempting to resolve any resource constraints for each program individually. Because OTIA does not have an Integrated Master Schedule for the Plan, it is not well positioned to understand how schedule changes in each individual program could affect implementation of the overall Plan. An Integrated Master Schedule could also help provide CBP a comprehensive view of the Plan and help CBP to reliably commit to when the Plan will be fully implemented, as well as help CBP to better predict whether estimated completion dates are realistic to manage programs’ performance. OTIA has developed a rough order of magnitude estimate for the Plan and individual Life-cycle Cost Estimates for the IFT and RVSS programs that meet some but not all best practices for such estimates. Best practices for cost estimating and Office of Management and Budget guidance emphasize that reliable cost estimates are important for program approval and continued receipt of annual funding. DHS policy similarly provides that Life-cycle Cost Estimates are essential to an effective budget process and form the basis for annual budget decisions. Reliable Life-cycle Cost Estimates reflect four characteristics—they are (1) well documented, (2) comprehensive, (3) accurate, and (4) credible— which encompass 12 best practices. For example, a best practice for a credible cost estimate is independently verifying a program’s Life-cycle Cost Estimate with an independent cost estimate and reconciling any differences. In August 2010, OTIA developed a rough order of magnitude cost estimate for the Plan—a high-level estimate without much detail—which was about $1.54 billion, including approximately $750 million in acquisition costs and approximately $800 million in operations and maintenance costs. In June 2013, OTIA revised this cost estimate for the Plan, estimating the cost at $1.39 billion, including about $480 million in acquisition costs and about $910 million in operations and maintenance costs. According to OTIA officials, some of the differences in costs between the August 2010 and June 2013 estimates are attributable to using more current information for the June 2013 estimate. Table 4 provides the June 2013 estimated cost and number of units to be procured and deployed for each of the Plan’s seven programs. In November 2011, we reported on the results of our analysis of the Plan’s August 2010 estimate. Specifically, we found that the August 2010 estimate substantially met best practices in terms of being comprehensive and accurate, and partially met best practices in terms of being well documented. For example, we reported that, in terms of being comprehensive, the estimate included documented technical data. In terms of accuracy, we reported that the cost estimate was continually updated and refined as more information became known. However, we also found that the August 2010 estimate minimally met best practices for being credible. For example, CBP officials had not conducted a sensitivity analysis and a cost-risk and uncertainty analysis to determine a level of confidence in the estimate, nor did CBP compare it with an independent estimate. At that time, OTIA officials stated that CBP’s approach was to develop and report an initial rough order of magnitude cost estimate for the programs in the Plan, not necessarily a Life-cycle Cost Estimate that met all best practices. In our November 2011 report, we recommended that CBP update its August 2010 cost estimate for the Plan using best practices, so that the estimate would be comprehensive, accurate, well CBP concurred with the recommendation. documented, and credible. In November 2012, OTIA officials told us that CBP no longer intends to develop a Life-cycle Cost Estimate for the Plan that meets all best practices. OTIA officials also stated that they used a risk-based approach to improve cost-estimating certainty and confidence by focusing on the Life-cycle Cost Estimates for the IFT and RVSS programs, which compose 90 percent of the Plan’s estimated cost. According to the officials, developing a Life-cycle Cost Estimate for the Plan that followed all best practices at this point in the acquisition cycle would not contribute much cost management benefit because a number of programs are under contract and units were being deployed to the field. However, as we recommended in November 2011, we continue to believe that a Life-cycle Cost Estimate for the Plan, developed using best practices, is needed to ensure that the estimate is comprehensive, accurate, well documented, and credible to help the agency and Congress fully understand the impacts of the Plan’s various programs. Moreover, CBP’s June 2013 revised cost estimate for the Plan does not address the concerns we identified in November 2011 with CBP’s original cost estimate. For example, the IFT and RVSS programs compose 90 percent of the Plan’s cost in the June 2013 Life-cycle Cost Estimate; however, OTIA has not independently verified its Life-cycle Cost Estimates for the IFT and RVSS programs with independent cost estimates and reconciled any differences with each program’s respective Life-cycle Cost Estimate, consistent with best practices. Furthermore, the remainder of the June 2013 Life-cycle Cost Estimate is not fully documented. The costs for programs other than the IFT and RVSS are provided as a summary program cost without a detailed description provided. In contrast, the IFT and RVSS Life-cycle Cost Estimates provided backup documentation, including labor hours and methodology. After CBP developed the initial cost estimate for the Plan in August 2010, CBP developed separate Life-cycle Cost Estimates for the IFT and RVSS programs in January and March 2012, respectively. The estimates for the IFT and RVSS programs met some but not all best practices for cost estimates. Specifically, our analysis shows that, in developing these estimates, CBP partially documented the data used in the cost model for the IFT’s Life-cycle Cost Estimate and fully documented the cost model for the RVSS’s Life-cycle Cost Estimate. CBP also conducted a sensitivity analysis and risk and uncertainty analysis to determine the level of confidence in both Life-cycle Cost Estimates so that contingency funding could be established relative to quantified risk. However, our analysis showed that CBP did not independently verify its draft Life-cycle Cost Estimates for the IFT and RVSS programs with independent cost estimates and reconcile any differences with each program’s respective Life-cycle Cost Estimate, consistent with best practices. According to OTIA officials, the IFT program’s Life-cycle Cost Estimate will be updated after the contract is awarded, the cost model for the updated Life-cycle Cost Estimate will be fully documented in accordance with best practices for cost estimating, and DHS’s Office of Program Accountability and Risk Management is expected to review the updated IFT Life-cycle Cost Estimate. Also, OTIA officials stated that they expect to update the RVSS Life-cycle Cost Estimate and receive approval for it in February 2014. However, OTIA is uncertain as to whether the updated IFT and RVSS Life-cycle Cost Estimates will be verified with independent cost estimates and any differences reconciled with the respective updated Life-cycle Cost Estimates. Specifically, OTIA officials stated that the IFT contract award will drive changes to the scope, schedule, and cost/budget baseline for the IFT program; CBP plans to update the Life-cycle Cost Estimate with programming and cost assumptions; and CBP plans to provide the updated cost estimate to the department as part of a revised submission of the Acquisition Program Baseline document. For the RVSS program, OTIA officials stated that the contract award resulted in changes that required updates to and reconciliation between the Cost Estimating Baseline Document and the Life-cycle Cost Estimate for the program’s scope, schedule, and cost/budget baseline. CBP intends to update the RVSS program’s Life-cycle Cost Estimate with programming and cost assumptions during the second quarter of fiscal year 2014 and provide the updated cost estimate to DHS for review. However, according to OTIA officials, as of November 2013, the agency had not yet determined whether to independently verify or validate the IFT and RVSS Life-cycle Cost Estimates. As CBP no longer intends to develop a Life-cycle Cost Estimate for the entire Plan, when updating the IFT and RVSS Life-cycle Cost Estimates, independently verifying the cost estimates and reconciling any differences, in accordance with cost-estimating best practices, could help better ensure the reliability of each estimate. Consistent with DHS acquisition guidance, CBP tailored the DHS Acquisition Life-cycle Framework for the IFT, RVSS, and MSC programs, primarily because the agency’s strategy for the three programs includes acquiring nondevelopmental technologies, preferably commercial-off-the- shelf systems, as opposed to developing technologies. As a result, rather than entering the DHS acquisition framework at Acquisition Decision Event 1, when a system includes technology development, the IFT program entered at combined Acquisition Decision Events 2B/3, and the RVSS and MSC programs entered at Acquisition Decision Event 2B. In pursuing its strategy to acquire nondevelopmental systems for the Plan’s three highest-cost programs, OTIA identified requirements and capabilities for each program, consistent with DHS acquisition guidance. Specifically, OTIA identified requirements for the IFT and RVSS programs that were approved in 2012, and capabilities for the MSC program that were developed in 2009. As part of the strategy to acquire commercial- off-the-shelf systems, CBP traded off, that is, reduced, some requirements for the RVSS and expects to trade off some requirements for the IFT for cost-effectiveness or schedule reasons. For example, with regard to the RVSS, OTIA traded off two requirements because, according to OTIA officials, they were not offered with the selected RVSS, which presented the best value to the government while providing as many requirements as possible. According to DHS Acquisition Management Directive 102-01 guidance, as part of the acquisition process, a program office may make trade-offs among performance, life- cycle cost, schedule, and risk. For example, the guidance states that a small reduction in performance that does not impair the mission might result in a large cost reduction. For the Plan’s three highest-cost programs, DHS and CBP did not consistently approve key acquisition documents before or at the Acquisition Decision Events, in accordance with DHS’s acquisition guidance. An important aspect of an Acquisition Decision Event is the review and approval of key acquisition documents critical to establishing the need for a program, its operational requirements, an acquisition baseline, and test and support plans, according to DHS guidance. DHS Acquisition Management Directive 102-01—and the associated DHS Instruction Manual 102-01-001 and appendixes—requires program offices to develop documents demonstrating critical knowledge that would help leaders make better-informed investment decisions when managing individual programs. The DHS guidance provides information for preparing acquisition documents, which require department- or component-level approval before a program moves to the next acquisition phase. In a September 2012 report, we found that while DHS had initiated efforts to validate required acquisition documents in a timely manner at major milestones, DHS leadership had authorized and continued to invest in major acquisition programs even though the vast majority of those programs lacked foundational documents demonstrating the knowledge needed to help manage risks and measure performance.in September 2012 that this limited DHS’s ability to proactively identify and address the challenges facing individual programs. We recommended, among other things, that DHS ensure all major acquisition programs fully comply with DHS acquisition policy by obtaining department-level approval for key acquisition documents before approving their movement through the acquisition life cycle. DHS concurred and since the time of our September 2012 report has approved We concluded additional acquisition documents. However, DHS has not yet demonstrated progress in obtaining department-level approval for most of its major acquisition programs’ key acquisition documents. On the basis of our analysis for IFT, RVSS, and MSC programs under the Plan, the DHS Acquisition Decision Authority approved the IFT program and the CBP Acquisition Decision Authority approved the RVSS and MSC programs to proceed to subsequent phases in the Acquisition Life-cycle Framework without approving all six required acquisition documents for each program. We also found that one document for the IFT program, five documents for the RVSS program, and two documents for the MSC program were subsequently approved after the programs received authority to proceed to the next phase. Table 5 provides a comparison of when key acquisition documents were required to be approved and when they were approved for the IFT, RVSS, and MSC programs. We discuss the status of key acquisition documents for the three highest- cost programs below. IFT program. Our analyses found that the DHS Acquisition Decision Authority approved four of the six documents required at Acquisition Decision Event 2B/3—the Acquisition Plan, Acquisition Program Baseline, Integrated Logistics Support Plan, and Operational Requirements Document—but did not approve two others—the Life-cycle Cost Estimate and Test and Evaluation Master Plan. At the time of the Acquisition Decision Event, CBP had a Life-cycle Cost Estimate for the IFT, but the cost estimate had not yet been approved by DHS. According to OTIA officials, the Life-cycle Cost Estimate for the IFT was discussed at the Acquisition Decision Event 3 meeting and approved by the DHS Under Secretary for Management and DHS’s Office of Program Accountability and Risk Management. However, CBP did not provide documentation showing that the estimate was approved by DHS. The DHS Director of Operational Test and Evaluation approved the revised IFT Test and Evaluation Master Plan on November 27, 2013, over DHS and CBP officials 18 months after it was required to be approved.attributed the delay in approving the Test and Evaluation Master Plan, in part, to discussions within CBP about the type and level of testing to be conducted on the IFTs. Specifically, CBP officials stated that a June 2012 version of the draft Test and Evaluation Master Plan did not include robust operational test and evaluation because of the IFT program’s strategy to acquire a nondevelopmental system (sometimes referred to as a commercial-off-the-shelf system). As a result, Border Patrol requested that rigorous, disciplined testing be included in the Test and Evaluation Master Plan to obtain familiarization with, and confidence in, the system and establish baseline performance information. According to DHS’s acquisition guidance, the Test and Evaluation Master Plan is important because it describes the strategy for conducting developmental and operational testing to evaluate a system’s technical performance, including its operational effectiveness and suitability. However, the IFT Test and Evaluation Master Plan approved by DHS in November 2013 does not describe testing to evaluate the operational effectiveness and suitability of the system. Rather, the Test and Evaluation Master Plan describes CBP’s plans to conduct a limited user test of the IFT. According to the Test and Evaluation Master Plan, the limited user test will be designed to determine the IFT’s mission contribution. According to OTIA and the Test and Evaluation Master Plan, this testing is planned to occur during 30 days in environmental conditions present at one site—the Nogales station. CBP plans to conduct limited user testing for the IFT under the same process that is typically performed in any operational test and evaluation, according to the Test and Evaluation Master Plan. The November 2013 IFT Test and Evaluation Master Plan notes that, because the IFT acquisition strategy is to acquire nondevelopmental IFT systems from the marketplace, a limited user test will provide Border Patrol with the information it needs to determine the mission contributions from the IFTs, and thus CBP does not plan to conduct more robust testing. However, this approach is not consistent with DHS’s acquisition guidance, which states that even for commercial-off-the-shelf systems, operational test and evaluation should occur in the environmental conditions in which a system will be used before a full production decision for the system is made and the system is subsequently deployed. This guidance also states that for commercial-off-the-shelf systems, operational tests should be conducted to ensure that the systems satisfy user-defined requirements. In addition, DHS guidance states that the primary purpose of test and evaluation is to provide timely and accurate information to managers, decision makers, and other stakeholders to support research, development, and acquisition, in a manner that reduces programmatic financial, schedule, and performance risk. We recognize the need to balance the cost and time to conduct testing to determine the IFT’s operational effectiveness and suitability with the benefits to be gained from such testing. However, revising the Test and Evaluation Master Plan to include more robust testing to determine operational effectiveness and suitability that more fully accounts for the various environmental conditions under which the IFTs will operate could better position CBP to evaluate IFT capabilities before moving to full production for the systems, help provide CBP with information on the extent to which the towers satisfy the Border Patrol’s user requirements, and help reduce potential program risks. In particular, although the limited user test should help provide CBP with information on the IFTs’ mission contribution and how Border Patrol can use the system in its operations, the limited user test does not position CBP to obtain information on how the IFTs may perform under the various environmental conditions the system could face once deployed. For example, in November 2013, the DHS Director of Test and Evaluation stated that testing the IFT at only one location during a clear, warm day without much wind would not produce representative results for days when it would be, for example, rainy, windy, freezing, or snowy, or when there was lightning. Likewise, he said testing in one location, such as Nogales, would not necessarily produce the same results as testing in Tucson because of the different terrains for the two locations. Conducting limited user testing in one area in Arizona—the Nogales station—for 30 days could limit the information available to CBP on how the IFT may perform in other conditions and locations along the Arizona border with Mexico. As of November 2013, CBP intends to deploy IFTs to 50 locations in southern Arizona, which can include different terrain and differences in climate throughout the year. Although the IFT program is not the same as SBInet, according to the Plan, the IFTs are to be deployed to locations with similar environmental and terrain conditions as SBInet towers, and IFT and SBInet systems may have similar types of technologies, such as cameras and radar. CBP previously encountered testing issues with SBInet. For example, in a January 2010 report, we found that while DHS’s approach to SBInet testing appropriately consisted of a series of progressively expansive developmental and operational events, the test plans and procedures for some test events were not defined in accordance with guidance. In January 2010, we concluded that effective testing was integral to successfully acquiring and deploying a large-scale, complex system, like SBInet. We further concluded that to do less unnecessarily increased the risk of problems going undetected until late in the system’s life cycle, such as when it was being accepted for use. In addition, in a November 2011 report, we found that the U.S. Army Test and Evaluation Command (ATEC) operationally tested SBInet at Tucson and that testing revealed challenges regarding the effectiveness and suitability of the technology for border surveillance. Among other things, this testing found that the rugged, restrictive terrain and weather conditions prevalent where SBInet is deployed affected the performance of the system’s radar, which affected success in detecting, identifying, and classifying items of interest. Revising the Test and Evaluation Master Plan to more fully test the IFT in the various environmental conditions in which it will be used to determine operational effectiveness and suitability before IFTs move to full production, in accordance with DHS acquisition guidance, could help provide CBP with more complete information on how the IFTs will operate under a variety of conditions before beginning full production. It could also help better position CBP to understand how the IFTs will meet Border Patrol’s operational requirements for the towers in contributing to Border Patrol’s border security mission. Without conducting operational testing in accordance with DHS guidance, the IFT program may be at risk of not meeting Border Patrol operational needs. RVSS program. The CBP Acquisition Decision Authority approved the program at Acquisition Decision Event 2B; however, the official had not approved any of the six required documents as required by DHS acquisition guidance at the time of that event. According to OTIA officials, the Acquisition Decision Authority approved the program for this Acquisition Decision Event because all of the necessary programmatic information was sufficiently developed and coordinated to support this decision. However, the Acquisition Decision Authority did not approve five of the documents until months after this event, and a sixth document, a Life-cycle Cost Estimate, was in draft form in November 2013—2 years after its required approval date. According to OTIA officials, the RVSS Life-cycle Cost Estimate is expected to be completed and approved in the second quarter of fiscal year 2014 and provided to DHS for review. MSC program. The CBP Acquisition Decision Authority approved two of the required six documents by Acquisition Decision Event 2B—the Acquisition Plan and Operational Requirements Document. However, the Integrated Logistics Support Plan was not approved until about 21 months after Acquisition Decision Event 2B. Also, the Acquisition Program Baseline was not expected to be approved until the second quarter of fiscal year 2014, more than 3 years after it was required to be approved for Acquisition Decision Event 2B and at least 16 months after it was required to be approved for Acquisition Decision Event 3. Furthermore, a Life-cycle Cost Estimate for the MSC’s operations and maintenance costs was expected to be completed in late 2013, more than 3 years after it was required to be approved for Acquisition Decision Event 2B. Since we last reported on CBP’s efforts to assess the performance of its SBInet surveillance systems in November 2011, CBP has taken steps to assess the performance of these technologies. In November 2011, we found that CBP had not conducted a post-implementation review and developed a plan to address SBInet operational test outcomes. Specifically, we found that CBP had not addressed the findings of ATEC’s March 2011 operational test results for the SBInet system at Tucson, which revealed challenges regarding the effectiveness and suitability of the technology for border surveillance and made nine recommendations At that time, CBP officials stated that to address performance issues.the agency did not conduct a post-implementation review or develop a plan to address the ATEC test results because the Secretary of Homeland Security canceled SBInet in January 2011. In November 2011, we recommended that CBP, in accordance with DHS guidance, conduct a post-implementation review and operational assessment of its SBInet system, and assess costs and benefits of taking action on the results of ATEC’s operational test.that conducting such a review, and weighing the costs and benefits of taking action on recommendations resulting from ATEC’s test of the SBInet system, could inform CBP’s decisions about future deployments of similar technologies, such as the IFTs. In making this recommendation, we concluded In response to our November 2011 recommendation, OTIA tasked the Johns Hopkins University Applied Physics Laboratory with conducting an independent post-implementation review of its SBInet Block 1 system. In January 2013, CBP released the results of the SBInet Block 1 Post Implementation Review (PIR), an assessment of the performance of its The PIR two SBInet surveillance system locations at Tucson and Ajo. concluded that CBP’s SBInet surveillance system has enhanced overall situational awareness within system viewsheds, improved agent safety, and been operationally available and effective with costs consistent with those anticipated for the system. For instance, the PIR concluded that the system broadened the agents’ situational awareness beyond the tactical, agent-on-the-ground sphere of awareness, and increased their ability to monitor incursions. The PIR also made five recommendations for CBP to improve future operational assessments of its SBInet surveillance system and to plan for new acquisition sensor deployments, such as for CBP to conduct a more detailed assessment of the impacts of Block 1 systems and develop more on-the-job agent training.as of May 2013, CBP is in the process of documenting and reviewing each recommendation outlined in the PIR, and intends to document its plans to address those recommendations that OTIA and the Office of Border Patrol determine need corrective action. However, these officials stated that some of the findings and recommendations outlined in the PIR will not be explicitly addressed or applied to future deployment efforts. For instance, according to officials, because the technologies planned for deployment under the Plan are commercial-off-the-shelf products, the PIR finding about recording the documentation of environmental factors, such as weather and terrain, that impede the system performance will not apply to the technologies to be deployed under the Plan, as those technologies include requirements on documentation of environmental factors. Border Patrol officials further stated that the contractor and Border Patrol will have a process to enable them to determine where the best deployment locations, given the variable terrain, will be for the technologies to be deployed under the Plan. Moreover, Border Patrol officials stated that Tucson sector officials have been assigned responsibility to determine the extent to which corrective actions are needed to address each recommendation outlined in the PIR because these sector officials have a better understanding of the environment in which the SBInet system is operating. According to OTIA officials, the agency plans to conduct annual operational assessments of its SBInet system. As additional surveillance technologies are deployed, we will continue to monitor Border Patrol’s efforts to address issues identified by the PIR as part of our recommendation follow-up process. According to OTIA and Border Patrol officials, In addition, the PIR concluded that as of January 2013, six of the nine recommendations outlined in ATEC’s operational test have either been addressed or are in the process of being addressed. The ATEC recommendations that remain to be addressed include, for example, addressing software reliability, improving sustainability cost, and reducing maintenance issues. OTIA officials stated that the agency plans to take actions to address the remaining three recommendations by, for example, pursuing alternative technical solutions to extend the life-cycle of the SBInet system and improving sustainability costs by reducing the contractor’s responsibility for field maintenance and other functions by transitioning to government support in 2014. CBP is not capturing complete asset assist data on the contributions of its surveillance technologies to apprehensions and seizures, and these data are not being consistently recorded by Border Patrol agents and across locations. Although CBP has a field within the EID for maintaining data on whether technological assets, such as SBInet surveillance towers, and nontechnological assets, such as canine teams, assisted or contributed to the apprehension of illegal entrants, and seizure of drugs and other contraband, according to CBP officials, Border Patrol agents are not required to record these data. This limits CBP’s ability to collect, track, and analyze available data on asset assists to help monitor the contribution of surveillance technologies, including its SBInet system, to Border Patrol apprehensions and seizures and inform resource allocation decisions. Our analysis of EID asset assist data for apprehensions and seizures in the Tucson and Yuma sectors from fiscal year 2010 through June 2013 shows that information on asset assists was generally not recorded for all apprehension and seizure events. For instance, for the 166,976 apprehension events reported by the Border Patrol across the Tucson sector during fiscal year 2010 through June 2013, an asset assist was not recorded for 115,517 (or about 69 percent) of these apprehension events. In the Yuma sector, of the 8,237 apprehension events reported by Border Patrol agents during the specified time period, an asset assist was not recorded for 7,150 (or about 87 percent) of these apprehension events. Similarly, data on seizure events reported across the Tucson and Yuma sectors show that for some seizure events, asset assists were not reported from fiscal year 2010 through June 2013 (about 32 percent and about 67 percent, respectively). According to Border Patrol officials, in the absence of requirements for Border Patrol agents to record data on asset assists, differences in the reporting of these data at the station level are likely attributable to the emphasis placed on the recording of these data by supervisory agents. Appendix IV contains summary statistics on the extent to which data on asset assists are recorded for apprehensions and seizures across the Tucson and Yuma sectors from fiscal year 2010 through June 2013. Since data on asset assists are not required to be reported, it is unclear whether the data were not reported because an asset was not a contributing factor in the apprehension or seizure or whether an asset was a contributing factor but was not recorded by agents. As a result, CBP is not positioned to determine the contribution of surveillance technologies in the apprehension of illegal entrants and seizure of drugs and other contraband during the specified time frame. As shown in figures 3 and 4, while the recording of asset assists increased from fiscal year 2010 through June 2013 from about 18 percent to about 47 percent in the Tucson sector and from about 8 percent to about 21 percent in the Yuma sector, for more than one-half of the apprehension event records for the Tucson sector and four-fifths for the Yuma sector, asset assists were not reported for the first three quarters of fiscal year 2013. Border Patrol officials did not specify why the agency does not require the recording and tracking of data on asset assists. However, Border Patrol officials stated that agents are encouraged to select the appropriate asset assist code when assets contributed to an apprehension or seizure. Border Patrol officials also stated that although they do not regularly track and analyze data on asset assists, including those from surveillance technologies, these data are tracked and analyzed on an ad hoc basis to help determine Border Patrol’s resource allocation and operational needs, and more specifically, what resources are available at the strategic level to help mitigate the threat of illegal entrants, drugs, and other contraband. Moreover, an Associate Chief at Border Patrol told us that while data on asset assists are not systematically recorded and tracked, Border Patrol recognizes the benefits of assessments of asset assists data, including those from surveillance technologies, such as the SBInet system, as these data in combination with other data, such as numbers of apprehensions and seizures, are used on a limited basis to help the agency make adjustments to its acquisition plans prior to deploying resources, thereby enabling the agency to make more informed deployment decisions. Border Patrol also uses these other data, such as numbers of apprehensions and seizures, to help inform assessment of its efforts to secure the border. Border Patrol officials cautioned that while asset assists data are the only available data directly linking apprehensions and seizures to the agency’s surveillance technologies, these data do not enable direct attributions of the SBInet system’s contribution to border security strategic goals because of several factors, such as changes in the flows of illegal entrants across sectors or in economic conditions in the United States and Mexico. Moreover, the officials said that surveillance technologies such as SBInet and RVSS towers enable the detection of apprehensions and seizures and accordingly, it is the agents who identify and track the illegal activity and ultimately apprehend illegal entrants and seize contraband. Despite the absence of complete data on the contribution of CBP’s surveillance technologies to apprehensions and seizures, our analysis of Border Patrol’s data on the location of apprehensions and seizures provides some insights into where Border Patrol apprehensions and seizures occurred in relation to the locations of its two highest-cost surveillance technologies—SBInet towers and RVSS. For example, our analysis of apprehensions events data, as determined by Geographic Information System data entered by Border Patrol agents when recording apprehensions and seizures, shows that across the Tucson sector from fiscal year 2010 through June 2013, of the 166,976 apprehension events, 71,397 (or about 43 percent) occurred within the camera and radar range of SBInet and RVSS towers. As shown in figure 5, the percentage of apprehension events occurring within the range of both SBInet and RVSS surveillance technologies has changed little, if at all, over time. Apprehension events occurring within the radar and camera range of SBInet towers have remained relatively unchanged, while apprehension events occurring within the range of RVSS towers increased by about 1 percent during our specified time frame. Moreover, of those 115,517 apprehension events in the Tucson sector that do not have data on asset assists, 8,751 (or about 8 percent) occurred within the camera range, and 9,818 (or about 9 percent) occurred within the radar range of SBInet towers. Moreover, data on asset assists were not recorded in 35,147 (or about 30 percent) of apprehension events within the range of RVSS towers. Table 6 shows the reporting of asset assists for apprehension and seizure events occurring across the Tucson sector within the range of SBInet and RVSS towers during our specified time period. Border Patrol officials stated that while analyzing data on the contributions of Border Patrol’s surveillance technologies is a relevant measure of the agency’s ability to meet its border security goals, conclusions regarding the contributions and impacts of its surveillance technologies on Border Patrol’s enforcement efforts cannot be formed solely on the basis of the proximity of apprehension or seizure events to the locations of its surveillance technologies. These officials stated that there are instances in which illegal entrants were detected by some combination of cameras or radar closer to the border; however, to gain a better tactical advantage, Border Patrol agents made the apprehensions farther from the border. As we reported in December 2012, Border Patrol officials stated that apprehensions occur in areas farther from the border because several factors preclude greater border presence, including terrain that is inaccessible or creates a tactical disadvantage, the distance from Border Patrol stations to the border, and access to ranches and lands that are federally protected and environmentally sensitive. Standards for Internal Control in the Federal Government calls for agencies to ensure that ongoing monitoring occurs during the course of normal operations to help evaluate program effectiveness. standards also state that agencies should promptly and accurately record transactions to maintain their relevance and value for management decision making and that this information should be readily available for use by agency management and others so that they can carry out their duties with the goal of achieving all of their objectives, including making operating decisions and allocating resources. These standards further state that to be effective, agencies need to clearly document all transactions in a timely manner to ensure that they are making appropriately informed decisions. Moreover, the standards call for clear documentation of and procedures that are readily available for examination. In addition, these standards call for comparisons and assessments relating different sets of data to one another so that analyses of the relationships can be made and appropriate actions taken. Because DHS’s EID database already includes the asset assists data field and these data are used by Border Patrol on a limited basis to make decisions about resources, requiring agents to record and track asset assists data could help ensure that these data are complete and, if analyzed, could help better inform CBP’s resource allocation decisions. Moreover, we acknowledge that conclusions regarding the contributions of surveillance technologies based on location and proximity data alone may not be sufficient to examine the contribution of CBP’s surveillance technologies in achieving their strategic goals. However, analyzing data on apprehensions, seizures, and asset assists in combination with other relevant performance metrics or indicators as appropriate could provide more robust analysis of the contributions of surveillance technologies, and accordingly could better position CBP to be able determine the extent to which its technology investments have contributed to the agency’s border security efforts. GAO/AIMD-00-21.3.1. In response to our November 2011 recommendation regarding the identification of mission benefits and development of key attributes for performance metrics for the surveillance technologies to be deployed as part of the Plan, CBP has identified mission benefits expected from the implementation of the surveillance technologies to be acquired or deployed as part of the Plan, but has not fully developed key attributes for In November 2011, we performance metrics for these technologies.reported that agency officials had not yet defined the mission benefits expected or quantified metrics to assess the contribution of the selected approaches in achieving their goal of situational awareness and detection of border activity using surveillance technology. We recommended that CBP determine the mission benefits to be derived from implementation of the Plan and develop and apply key attributes for metrics to assess program implementation. CBP concurred with our recommendation. In April 2013, CBP issued its Multi-Year Investment and Management Plan for Border Security Fencing, Infrastructure, and Technology for Fiscal Years 2014-2017, which identifies specific mission benefits to be achieved by the deployment of each of the seven technologies under the Plan.technologies will provide the mission benefits of improved situational awareness and agent safety. Furthermore, CBP officials stated that each of the seven technologies deployed or planned for deployment as part of the Plan will help enhance the ability of Border Patrol agents to detect, identify, deter, and respond to threats along the border. A summary of the mission benefits of each surveillance technology deployed or planned for deployment under the Plan is presented in appendix V. According to CBP officials, the majority of these surveillance While CBP has defined mission benefits for the technology programs under the Plan, the agency has not yet developed key attributes for performance metrics for all surveillance technologies to be deployed as part of the Plan. The Clinger-Cohen Act of 1996 and Office of Management and Budget (OMB) guidance emphasize the need to ensure that information technology investments, such as IFT systems, produce tangible, observable improvements in mission performance. In our April 2013 update on the progress made by the agencies to address our findings on duplication and cost savings across the federal government, CBP officials stated that operations of its two SBInet surveillance systems identified examples of key attributes for metrics that can be useful in assessing the Plan’s implementation for technologies. For example, according to CBP officials, to help measure whether illegal activity has decreased, examples of key attributes include decreases in the amount of arrests, complaints by ranchers and other citizens, and destruction of public and private lands and property. While the development of key attributes for metrics for the two SBInet surveillance systems is a positive step, as of April 2013, CBP has not yet identified attributes for metrics for all technologies to be acquired and deployed as part of the Plan. In addition to these efforts, CBP officials stated that in response to our prior recommendations regarding the establishment of a performance goal or goals and associated performance metrics that define how border security is to be measured, Border Patrol, as of December 2013, was in the process of developing and implementing performance goals and measures to assess Border Patrol’s efforts to secure the border. However, CBP officials stated that none of the current measures directly address the operational impact of technology. The officials further stated that the Tucson sector has submitted an issue paper that identifies potential data that can attribute a certain level of effectiveness to its SBInet system, but it is still under review by CBP. While these are positive steps, to fully address the intent of our recommendation, CBP would need to develop and apply key attributes for performance metrics for each of the technologies to be deployed under the Plan to assess its progress in implementing the Plan and determine when mission benefits have been fully realized. CBP has established schedules for the Plan and the IFT, RVSS, and MSC programs that meet some but not all best practices for scheduling, hindering CBP’s ability to reliably commit to when it will deliver all of the Plan’s technologies to Arizona. Ensuring that all schedule best practices are applied to the IFT, RVSS, and MSC schedules when updating them could help OTIA better ensure the schedules’ reliability and could help better position OTIA to identify and address any potential further delays in the program’s milestone commitment dates. Further, developing and maintaining an Integrated Master Schedule for the Plan, in accordance with best practices, could allow insight into current or programmed allocation of resources for the Plan and help CBP to reliably commit to when the Plan will be fully implemented. Also, CBP has developed Life- cycle Cost Estimates for the IFT and RVSS programs. Although OTIA officials stated that DHS’s Office of Program Accountability and Risk Management conducted an assessment of the IFT Life-cycle Cost Estimate, an assessment is not equivalent to verifying the estimate with an independent cost estimate. When updating the Life-cycle Cost Estimates for the IFT and RVSS programs, verifying the estimates with independent cost estimates and reconciling any differences, consistent with best practices, could help to better ensure the credibility of CBP’s cost estimates for these programs. DHS and CBP have approved some key acquisition documents as directed by DHS and CBP Acquisition Review Boards, but work remains to approve all key acquisition documents in accordance with DHS acquisition guidance. Specifically, revising the Test and Evaluation Master Plan to more fully test the IFTs in the various environmental conditions in which they will be used to determine operational effectiveness and suitability, in accordance with DHS acquisition guidance, could help provide CBP with more complete information on how the IFTs will operate under a variety of conditions before beginning full production. Requiring the collection of data on the extent to which technology assets assisted in apprehensions and seizures could better position Border Patrol to assess the contribution of surveillance technologies to its enforcement efforts and its goals of achieving and maintaining operational control and situational awareness along the southwest border. Conducting analysis of such data, once collected, in combination with other relevant performance metrics or indicators as appropriate, could help better position CBP to be able to determine the extent to which its technology investments have contributed to border security efforts. To improve the acquisition management of the Plan and the reliability of its cost estimates and schedules, assess the effectiveness of deployed technologies, and better inform CBP’s deployment decisions, we recommend that the Commissioner of CBP take the following six actions: When updating the schedules for the IFT, RVSS, and MSC programs, ensure that scheduling best practices, as outlined in our schedule assessment guide, are applied to the three programs’ schedules. Develop and maintain an Integrated Master Schedule for the Plan that is consistent with scheduling best practices. When updating Life-cycle Cost Estimates for the IFT and RVSS programs, verify the Life-cycle Cost Estimates with independent cost estimates and reconcile any differences. Revise the IFT Test and Evaluation Master Plan to more fully test the IFT program, before beginning full production, in the various environmental conditions in which IFTs will be used to determine operational effectiveness and suitability, in accordance with DHS acquisition guidance. Require data on asset assists to be recorded and tracked within the Enforcement Integrated Database, which contains data on apprehensions and seizures. Once data on asset assists are required to be recorded and tracked, analyze available data on apprehensions and seizures and technological assists, in combination with other relevant performance metrics or indicators, as appropriate, to determine the contribution of surveillance technologies to CBP’s border security efforts. We provided a draft of this report to DHS for review and comment. DHS provided written comments, which are summarized below and reproduced in full in appendix VI, and technical comments, which we incorporated as appropriate. DHS concurred with four of the recommendations in the report. DHS did not concur with the other two recommendations in the report. With regard to the first recommendation, that CBP ensure that scheduling best practices are applied to the IFT, RVSS, and MSC programs’ schedules when they are updated, DHS concurred and stated that OTIA plans to ensure that scheduling best practices are applied as far as practical when updating the three programs’ schedules. DHS plans to update the programs’ schedules by July 2015. With regard to the second recommendation, that CBP develop and maintain an Integrated Master Schedule for the Plan, DHS did not concur with this recommendation. DHS stated that maintaining an Integrated Master Schedule for the Plan undermines the DHS-approved implementation strategy for the individual programs making up the Plan and that a key element of the Plan has been the disaggregation of technology procurements. According to DHS, the implementation of this recommendation would essentially create a large, aggregated program, similar to SBInet, and effectively create an aggregate “system of systems.” DHS stated that CBP believes its strategy of disaggregation has been effective and has reduced overall risk and cost. DHS also stated that each program within the Plan has its own schedule and that forcing linkages among the Plan’s programs into a single Integrated Master Schedule contradicts lessons learned and the approved implementation strategy for the Plan. We continue to believe that developing and maintaining an Integrated Master Schedule for the Plan, consistent with best practices for scheduling, is needed. As noted in the report, the use of an Integrated Master Schedule is a well-established practice in program and project management and is a necessary tool to coordinate independently managed projects that have dependencies—including resource dependencies—on one another. The programs under the Plan are intended to provide Border Patrol with a combination of surveillance capabilities to assist in achieving situational awareness along the Arizona border with Mexico; and while the programs themselves may be independent of one another, the Plan’s resources are being shared among the programs. Furthermore, this recommendation is not intended to imply that DHS needs to re-aggregate the Plan’s seven programs into a "system of systems" or change its procurement strategy in any form. Rather, the intent of our recommendation is for DHS to insert the individual schedules for each of the Plan’s programs into a single electronic Integrated Master Schedule file in order to identify any resource allocation issues among the programs’ schedules. Developing and maintaining an Integrated Master Schedule for the Plan could allow OTIA insight into current or programmed allocation of resources for all programs as opposed to attempting to resolve any resource constraints for each program individually. In addition to helping identify resource constraints, an Integrated Master Schedule can be a useful tool for consolidating multiple projects or program files into a single master file, even if those projects or programs have no direct links among activities. For example, aggregating individual files into a master schedule is useful for reporting purposes, particularly if the projects or programs are under the purview of a single management organization or a single customer. In this case, the master schedule would allow for a concise view of all projects or programs for which the stakeholder is responsible or has an interest. A master schedule of this nature is often referred to as a consolidated schedule, although the terms “consolidated schedule” and “Integrated Master Schedule” are often synonymous. We continue to believe that developing and maintaining an Integrated Master Schedule for the Plan could help provide CBP a comprehensive view of the Plan and help CBP to reliably commit to when the Plan will be fully implemented and better predict whether estimated completion dates are realistic to manage programs’ performance, as noted in the report. With regard to the third recommendation, that CBP verify the Life-cycle Cost Estimates for the IFT and RVSS programs with independent cost estimates and reconcile any differences, DHS concurred, although its planned actions will not fully address the intent of the recommendation unless assumptions underlying the cost estimates change. DHS stated that while OTIA did not obtain a traditional independent cost estimate for the programs, the Life-cycle Cost Estimates were meant to be conservative in managing program risk and that the estimated life-cycle costs to date are less than originally projected. DHS further stated that at this point it does not believe that there is a benefit in expending funds to obtain independent cost estimates and that if the costs realized to date continue to hold, there may be no requirement or value added in conducting full-blown updates with independent cost estimates. DHS noted, though, that if this assumption changes, OTIA will complete updates and consider preparing independent cost estimates, as appropriate. We recognize the need to balance the cost and time to verify the Life-cycle Cost Estimates with the benefits to be gained from verification with independent cost estimates. However, as noted in this report, independently verifying the cost estimates is consistent with best practices and could help provide CBP with more insights into program costs. An independent cost estimate provides an independent view of expected program costs that tests the program office’s estimate for reasonableness. Independent cost estimates frequently use different methods and are less burdened with organizational bias than a program office’s estimate, helping to provide decision makers with insight into a program’s potential costs. Thus, we continue to believe that independently verifying the Life-cycle Cost Estimates for the IFT and RVSS programs and reconciling any differences, consistent with best practices, could help CBP better ensure the reliability of the estimates. With regard to the fourth recommendation, that CBP revise the IFT Test and Evaluation Master Plan to more fully test the IFT program in the various environmental conditions in which IFTs will be used to determine operational effectiveness and suitability, DHS did not concur with the recommendation. Specifically, DHS stated that the Test and Evaluation Master Plan includes tailored testing and user assessments that will provide much, if not all, of the insight contemplated by the intent of the recommendation. According to DHS, the approved non-developmental item acquisition strategy for the IFT program was based on market surveys and observations during field use by other customers and the incorporation of system demonstrations conducted during source selection. DHS also stated that there is no requirement for expansive, formal operational test and evaluation and to re-write the Test and Evaluation Master Plan to incorporate operational testing undermines and removes the benefits of the non-developmental item strategy. Moreover, DHS stated that the user test currently outlined in the Test and Evaluation Master Plan will provide the operational user the information needed to validate system requirements and operational characteristics. DHS also noted that Acquisition Decision Event 3 has been approved for IFT production, and after the initial IFT system undergoes testing in accordance with the Test and Evaluation Master Plan, the Office of Border Patrol will make the determination regarding operational readiness prior to deploying additional systems. We continue to believe that DHS should revise the Test and Evaluation Master Plan to more fully test the IFT program, before beginning full production, in the various environmental conditions in which the IFT will be used to determine operational effectiveness and suitability. DHS’s acquisition guidance states that the Test and Evaluation Master Plan is important because it describes the strategy for conducting developmental and operational testing to evaluate a system’s technical performance, including its operational effectiveness and suitability. The guidance states that, even for commercial-off-the-shelf systems, such as the IFT program, operational test and evaluation should occur in the environmental conditions in which a system will be used before a full production decision for the system is made and the system is subsequently deployed. In addition, DHS guidance states that the primary purpose of test and evaluation is to provide timely and accurate information to managers, decision makers, and other stakeholders to support research, development, and acquisition in a manner that reduces programmatic financial, schedule, and performance risks. The current Test and Evaluation Master Plan describes CBP’s plans to conduct a limited user test of the IFT, which will be designed to determine the IFT’s mission contribution. However, determining mission contribution is not equivalent to determining operational effectiveness and suitability, which specifically identifies how effective and reliable a system is in meeting its operational requirements in its intended environment. DHS plans to conduct limited user testing during a 30-day period in environmental conditions present at one site—the Nogales station. However, as of November 2013, CBP intended to deploy IFTs to 50 locations in southern Arizona, which can include different terrain and differences in climate throughout the year. As we noted in the report, conducting limited user testing in one area in Arizona for 30 days could limit the information available to CBP on how the IFTs may perform in other conditions and locations along the Arizona border. Therefore, CBPs approach to use limited user testing will not specifically identify how effective and reliable a system is in meeting its operational requirements in its intended environment. Moreover, while DHS has approved the IFT program for production at Acquisition Decision Event 3, testing for the IFTs has not yet begun. As noted in the report, revising the Test and Evaluation Master Plan to include more robust testing to determine operational effectiveness and suitability could better position CBP to evaluate IFT capabilities before moving to full production for the system, help provide CBP with information on the extent to which the towers satisfy the Border Patrol’s user requirements, and help reduce potential program risks. Furthermore, although the IFT program is not the same as SBInet, according to the Plan, the IFTs are to be deployed to locations with similar environmental and terrain conditions as SBInet towers, and IFT and SBInet systems may have similar types of technologies, such as cameras and radar. As noted in the report, we previously identified testing issues CBP encountered with SBInet, such as DHS’s test plans and procedures for some SBInet test events not being defined in accordance with guidance and that operational tests of SBInet at Tucson revealed challenges regarding the effectiveness and suitability of the technology for border surveillance. Thus, we continue to believe that revising the Test and Evaluation Master Plan to more fully test the IFT in the various environmental conditions in which it will be used to determine operational effectiveness and suitability, before beginning full production, could help provide CBP with more complete information on how the IFTs will operate under a variety of conditions. Without conducting operational testing in accordance with DHS guidance, the IFT program may be at increased risk of not meeting Border Patrol operational needs. With regard to the fifth recommendation, that CBP require data on asset assists to be recorded and tracked within the Enforcement Integrated Database, DHS concurred and stated that Border Patrol is changing its data collection process to allow for improved reporting on asset assists for apprehensions and seizures and intends to make it mandatory to record whether an asset assisted in an apprehension or seizure. DHS plans to change its process by December 31, 2014. With regard to the sixth recommendation, that CBP analyze available data on apprehensions and seizures and technology assists to determine the contribution of surveillance technologies to its border security efforts, DHS concurred and stated that Border Patrol intends to create a plan of action with milestones to explore and develop a process to answer how different classes of technology, within a certain environment, contribute to Border Patrol’s mission. DHS stated that Border Patrol plans to develop an initial set of quantitative and qualitative technology-related measures by September 30, 2014, as an interim milestone; gather baseline data for the measures in fiscal year 2015 and begin to use these data to evaluate the contributions of specific technology assets by the end of that fiscal year; and by the end of fiscal year 2016, use measures associated with technology to assist in determining levels of situational awareness in different areas of the border. These planned actions, if implemented effectively, should address the intent of the recommendations. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretary of Homeland Security, appropriate congressional committees, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-8777 or gamblerr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix VII. Our objectives were to determine the extent to which U.S. Customs and Border Protection (CBP) has (1) developed schedules and Life-cycle Cost Estimates for the Arizona Border Surveillance Technologies Plan (the Plan) in accordance with best practices; (2) followed key aspects of the Department of Homeland Security’s (DHS) acquisition management framework in managing the Plan’s three highest cost programs; and (3) assessed the performance of technologies deployed under the Secure Border Initiative Network (SBInet), identified mission benefits, and developed performance metrics for surveillance technologies to be deployed under the Plan. To determine the extent to which CBP has followed best practices in developing schedules and Life-cycle Cost Estimates for the Plan’s three highest-cost programs—the Integrated Fixed Tower (IFT), Remote Video Surveillance System (RVSS), and Mobile Surveillance Capability (MSC)—we obtained CBP’s Office of Technology Innovation and Acquisition’s (OTIA) program schedules as of March 2013, which were current at the time of our review, for the these programs and compared them against best practices for developing schedules. Specifically, we assessed the extent to which the schedules for these three programs met each of the 10 best practices identified in the schedule assessment guide. We characterized whether the schedules met each of the 10 best practices based on the following scale: Not met—the program provided no evidence that satisfies any of the criterion. Minimally met—the program provided evidence that satisfies a small portion of the criterion. Partially met—the program provided evidence that satisfies about half of the criterion. Substantially met—the program provided evidence that satisfies a large portion of the criterion. Met—the program provided complete evidence that satisfies the entire criterion. In conducting our analysis, we focused, for example, on whether the schedules reflect best practices for a reliable schedule, such as whether the schedules define the work necessary to accomplish a program’s objectives. More details on our assessment and methodology are presented in appendix III. By assessing the schedules against best practices, we also identified schedule challenges that CBP was experiencing in testing, procuring, deploying, and operating technologies in the Plan and interviewed CBP officials to determine the reasons for the schedule challenges and steps that CBP had taken or was taking to address them. In addition, we obtained and analyzed the August 2010 and June 2013 Life-cycle Cost Estimates for the Plan. We also analyzed the IFT and RVSS January 2012 and March 2012 Life-cycle Cost Estimates, respectively, which were current at the time of our review, and compared them against best practices for cost estimating. We analyzed DHS and CBP documents and interviewed officials regarding their efforts to implement our November 2011 recommendations to update the Life- To cycle Cost Estimate for the Plan in accordance with best practices.assess the reliability of cost estimate data that we used, we reviewed relevant program documentation, such as cost estimation spreadsheets, as available, to substantiate evidence obtained from interviews with knowledgeable agency officials. We found the data to be sufficiently reliable for the purposes of our report. To determine the extent to which CBP followed key aspects of DHS’s acquisition management framework in managing the Plan’s three highest- cost programs, we analyzed DHS and CBP documents, including DHS Acquisition Management Directive 102-01 and its associated DHS Instruction Manual 102-01-001, program briefing slides, budget documents, Acquisition Decision Memorandums, schedules, and program We focused on the IFT, RVSS, and MSC programs for more risk sheets.in-depth analyses because they are the Plan’s three highest-cost programs and represent 97 percent of the estimated cost of the Plan. Specifically, to assess the acquisition strategy for the Plan, we focused on the IFT, RVSS, and MSC programs and analyzed their acquisition plans and discussed the approaches with CBP officials. To assess system requirements and capabilities for the IFT, RVSS, and MSC programs, we obtained and analyzed requirements and capabilities documents and worked with CBP officials to identify any changes to requirements and capabilities since they were initially approved and whether any requirements or capabilities had been traded off for cost, schedule, or other purposes. To assess the extent to which CBP followed DHS acquisition guidance, we selected aspects of Acquisition Management Directive 102-01 that were relevant to where these programs were in the acquisition process during fiscal year 2013. Specifically, we determined whether acquisition documents had been approved by the time of the applicable Acquisition Decision Events as required by DHS acquisition guidance. To determine the extent to which CBP has assessed the performance of technologies deployed under SBInet and developed performance metrics to assess the performance of surveillance technologies planned for deployment under the Plan, we analyzed performance assessment documentation and interviewed CBP officials responsible for performance measurement activities regarding the establishment of performance metrics used by CBP to determine the effectiveness and the contributions of its surveillance technologies toward the agency’s stated border security goals. With respect to CBP’s assessment of the performance of technologies deployed under SBInet, we analyzed the results of the January 2013 post implementation review, which was conducted by the Johns Hopkins University Applied Physics Laboratory to determine the effectiveness of SBInet technologies in achieving their intended results. We reviewed our November 2011 report to determine the extent to which CBP’s post implementation review aligned with DHS guidance and the Office of Management and Budget’s (OMB) Capital Programming Guide, a supplement to OMB Circular A-11, which identifies a post implementation review as a tool to evaluate an investment’s efficiency and effectiveness. We also analyzed CBP and DHS documents, such as CBP’s July 2013 SBInet Block 1 After Action Report, and interviewed officials to assess corrective actions taken to improve SBInet performance issues. Specifically, we analyzed CBP documentation and interviewed agency officials within OTIA and the Office of Border Patrol to determine the progress the agency has made in addressing findings and recommendations outlined in CBP’s post implementation review and prior performance assessments, including the Army Test and Evaluation Command’s assessment of its SBInet technologies. On the basis of interviews with agency officials regarding the methodology and implementation of the review, we found the review to be sufficiently reliable for our report. In addition, we analyzed CBP data on apprehensions of illegal entrants and seizures of drugs and other contraband for the Tucson and Yuma sectors maintained in the Enforcement Integrated Database (EID), a DHS-shared common database repository for several DHS law enforcement and homeland security applications, as well as policy, planning, and budget documents provided by Border Patrol to determine whether such data could be used to determine the contributions of the SBInet technologies to apprehensions and seizures. We analyzed apprehension and seizure data for the Tucson and Yuma sectors within Arizona, because these are the Border Patrol sectors contained within Arizona and covered by the Plan. For the purposes of this report, we analyzed apprehension and seizure events recorded in the EID for fiscal years 2010 through June of fiscal year 2013. An apprehension or seizure event is defined as an occasion on which Border Patrol agents apprehend an illegal entrant or seize drugs and other contraband. Each reported apprehension or seizure event is assigned a unique identifier in the EID, and Border Patrol agents assign an additional identifier to each individual illegal entrant or type of seized item associated with the event. As a result, a single apprehension event may involve the apprehension of multiple illegal entrants, and a single seizure event may result in the seizure of multiple items. Appendix IV contains the results of our analysis of all recorded apprehensions and seizures occurring across the Tucson and Yuma sectors during the specified time frame. For our analysis, we also obtained data on asset assists recorded in the EID for apprehensions and seizures. According to Border Patrol officials, the asset assist data field was added to the EID in May of 2009. Agents may select from a drop-down menu to identify whether a technological or nontechnological asset assisted in the apprehension or seizure. Multiple assets can be selected for a single event, if relevant. For the purposes of this report, technological assets identified within the Border Patrol’s asset assists data field drop-down menu are those assets for which Border Patrol continues to make significant funding investments and are included as part of the Plan, and include Cameras, Mobile Surveillance Systems, Scope Trucks, and Unattended Ground Sensors. According to Border Patrol headquarters officials, agents identifying “Cameras” are most likely attributing the asset assist to either SBInet towers or Remote Video Surveillance Systems. In addition, for our analysis, we obtained Geographic Information Systems data for apprehensions, seizures, and Border Patrol’s two highest-cost surveillance systems—SBInet and RVSS towers—to show the latitude and longitude coordinates of apprehensions and seizures in relation to the location of SBInet towers and RVSS towers. We used Geographic Information Systems data to determine the percentage of apprehensions and seizures that occurred within the proximity of the radar and camera range of SBInet and camera range of RVSS towers, and the extent to which asset assists were reported for apprehensions and seizures occurring within the proximity of the surveillance systems. For the purposes of this report, the ranges of the SBInet and RVSS towers are “buffer ranges” that, according to Border Patrol headquarters officials, do not account for obstructions due to terrain, land features, and vegetation. To perform these analyses, we compared Border Patrol data on the longitude and latitude of apprehensions, seizures, SBInet towers, and RVSS towers with agency mapping data, which allowed us to determine the extent to which apprehensions and seizures occurred within the proximity of SBInet and RVSS towers. We interviewed Border Patrol headquarters officials regarding data collection and analysis procedures, and performance assessment activities. We analyzed apprehensions and seizures data from fiscal year 2010 through June 2013 because fiscal year 2010 was the first fiscal year for which data on asset assists were available following Border Patrol’s deployment of its SBInet technologies, and the collection of data on Geographic Information Systems coordinates for apprehensions and seizures was required. To assess the reliability of apprehensions and seizures data, including the asset assist and Geographic Information Systems data, we interviewed Border Patrol headquarters officials who oversee the maintenance and analyses of the data about agency guidance and processes for collecting and reporting the data. We determined that the apprehensions, seizures, asset assists, and Geographic Information Systems data were sufficiently reliable for the purposes of this report. However, as we reported in December 2012, because of potential inconsistencies in how the data are collected, these data cannot be compared across sectors but can be compared within a sector over time.were sufficiently reliable for the purposes of this report, but found limitations with the consistency in which these data are recorded for all apprehensions and seizures, a fact that we discuss in the report. Although we determined that the latitude and longitude coordinates for some apprehensions and seizures were invalid—e.g., they were identified as occurring outside U.S. national boundaries—the numbers were not significant, and we determined that the Geographic Information Systems data were sufficiently reliable for the purposes of this report. Location data that were determined to be invalid were not included in our analysis. We compared CBP’s reporting requirements and use of asset assists data against criteria in Standards for Internal Control in the Federal Government, which, among other things, call for ensuring effectiveness We determined that the recorded data on asset assists and efficiency of management operations, including the use of the entity’s resources. In addition, we visited Border Patrol’s Tucson sector in Arizona to observe Border Patrol agents operating SBInet technologies and other selected technologies, such as RVSS towers, and discussed agents’ experiences in using these technologies. While visiting the Tucson sector, we interviewed officials regarding the deployment and contributions of surveillance technologies within the sector. We visited the Tucson sector because of the presence of surveillance technologies, such as SBInet and RVSS towers, in that sector and because, under the Plan, the Tucson sector has locations for which additional technology deployments, such as IFTs, are planned. While the information we obtained from our visit cannot be generalized to all Border Patrol sectors, it did provide us with insights about the use of the deployed surveillance technologies. Finally, we analyzed documents, including CBP’s Multi-Year Investment and Management Plan for Border Security Fencing, Infrastructure, and Technology for Fiscal Years 2014-2017, and interviewed CBP officials responsible for overseeing the progress CBP and DHS have made in implementing our November 2011 recommendations to identify the mission benefits to be derived from technologies in the Plan and metrics to measure the extent to which border security is expected to improve by using these technologies. We conducted this performance audit from September 2012 through March 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Best practices for cost estimating and scheduling identify 10 practices associated with effective scheduling. These are (1) capturing all activities, (2) sequencing all activities, (3) assigning resources to all activities, (4) establishing the duration of all activities, (5) verifying that the schedule is traceable horizontally and vertically, (6) confirming that the critical path is valid, (7) ensuring reasonable total float, (8) conducting a schedule risk analysis, (9) updating the schedule with actual progress and logic, and (10) maintaining a baseline schedule. These practices are summarized into four characteristics of a reliable schedule— comprehensive, well constructed, credible, and controlled. We assessed the extent to which the March 2013 schedules for CBP’s three highest-cost technology programs under the Arizona Border Surveillance Technology Plan—IFT, RVSS, and MSC—met each of the 10 best practices. the 10 best practices as follows: We characterized whether the schedules met each of Not met—the program provided no evidence that satisfies any of the criterion. Minimally met—the program provided evidence that satisfies a small portion of the criterion. Partially met—the program provided evidence that satisfies about half of the criterion. Substantially met—the program provided evidence that satisfies a large portion of the criterion. We determined the overall assessment rating by assigning each individual rating a number: Not met = 1, minimally met = 2, partially met = 3, substantially met = 4, and met = 5. Then, we took the average of the individual assessment ratings to determine the overall rating for each of the four characteristics. The resulting average becomes the Overall Assessment as follows: Not met = 1.0 to 1.4, minimally met = 1.5 to 2.4, partially met = 2.5 to 3.4, substantially met = 3.5 to 4.4, and met = 4.5 to 5.0. We developed this rating scale in consultation with cost-estimating experts who helped develop the Cost Estimating and Assessment Guide. Met—the program provided complete evidence that satisfies the entire criterion. Table 7 provides the results of our analysis of the IFT, RVSS, and MSC schedules as of March 2013. DHS’s Enforcement Integrated Database includes a field that enables Border Patrol agents to identify whether a technological or nontechnological asset assisted in the apprehension of illegal entrants or the seizure of drugs or other contraband. This appendix provides summary statistics on the reporting of asset assists by Border Patrol agents in the apprehension of illegal entrants and seizure of drugs and other contraband across the Tucson and Yuma sectors from fiscal year 2010 through June 2013. For the purposes of this report, “unreported asset assists” could include instances for which an asset was not a contributing factor in apprehensions and seizures. sectors, conclusions about the differences in reported asset assists across sectors and differences within sectors over time cannot be made. As shown in tables 8 and 9, the 166,976 apprehension events that occurred in the Tucson sector from fiscal year 2010 through June 2013 resulted in the apprehension of 549,357 illegal entrants, and 20,322 seizure events that occurred in the Tucson sector over the same period resulted in the seizure of 21,973 items. and 6,828 seizure events occurring in the Yuma sector during the same time period resulted in the apprehension of 17,580 illegal entrants and 7,892 seized items. The two tables also show that the percentages of reported asset assists for apprehension events and seizure events and the resulting apprehensions and seizures differed across the Tucson and Yuma sectors and in both sectors and over time. Figure 13 shows that the percentages of apprehension events and apprehensions in the Tucson sector for which asset assists were not reported decreased from fiscal year 2010 through June 2013. Figure 14 shows that in the Yuma sector, the percentages of apprehension events and apprehensions for which asset assists were not reported were higher than in the Tucson sector, but similarly decreased over that time period. For the first three quarters of fiscal year 2013, asset assist information was not reported for more than one-half of the apprehension events in the Tucson sector and nearly four-fifths of the apprehension events in the Yuma sector. Figures 13 and 14 also show that the percentages of apprehension events and apprehensions for which technological asset assists and other asset assists were reported increased during that period in both sectors. Because it is difficult to determine whether unreported asset assists do not involve asset assists, or do involve asset assists that were not recorded, it is difficult to determine whether the increases in the percentages of apprehension events and apprehensions involving technology asset assets and other assists involve real increases, or increases resulting from fewer asset assists going unreported. The higher percentage of apprehension events and apprehensions involving technology asset assists in the Tucson sector relative to the Yuma sector may also be partly due to differences in the two sectors in unreported asset assists, and also to differences in the number of technology assets in the two sectors. Figure 15 shows that for seizure events and seizures in the Tucson sector from fiscal year 2010 through June 2013, as was the case for apprehension events and apprehensions, the percentages for which asset assists were unreported declined, while the percentages for which technology asset assists and other asset assists were reported increased. Unreported asset assists were lower for seizures than for apprehensions in the Tucson sector, and the changes with respect to the percentages of seizures involving unreported asset assists, technology asset assists, and other asset assists in the Tucson sector were not as pronounced as the changes with respect to apprehensions. Figure 16 shows that in the Yuma sector over the same period, the percentages for which asset assists were unreported declined, and the percentage for which other (nontechnology) assets were reported increased. In the Yuma sector, the percentage of technology asset assists was small (less than 1 percent) in each of the fiscal years and there was no discernible trend in the percentage of technology asset assists. As with apprehensions, it is difficult to determine how much of the increase in technology asset assists in the Tucson sector and other asset assists in both sectors involves the increased use of technology and other assets, or changes in reporting of asset assists. Table 10 summarizes the mission benefits to be derived from each of the technologies to be deployed as part of the Arizona Border Surveillance Technology Plan as outlined in CBP’s Multi-Year Investment and Management Plan for Border Security Fencing, Infrastructure, and Technology for Fiscal Years 2014-2017. According to CBP officials, each of the seven technologies deployed or planned for deployment as part of the Plan will increase situational awareness and enhance the ability of Border Patrol agents to detect, identify, deter, and respond to threats along the border. In addition to the contact named above, Jeanette Espinola (Assistant Director), David Alexander, Charles Bausell, Frances Cook, Katherine Davis, Joseph E. Dewechter, Jennifer Echard, Shannon Grabich, Yvette Gutierrez, Eric Hauswirth, Richard Hung, Jason Lee, Grant Mallie, Linda Miller, John Mingus, Anna Maria Ortiz, Karen Richey, Doug Sloane, Karl Seifert, Nate Tranquilli, Katherine Trimble, Jim Ungvarsky, and Michelle Woods made key contributions to this report.

Summary: The Department of Homeland Security's (DHS) U.S. Customs and Border Protection's (CBP) schedules and Life-cycle Cost Estimates for the Arizona Border Surveillance Technology Plan (the Plan) reflect some, but not all, best practices. Scheduling best practices are summarized into four characteristics of reliable schedules—comprehensive, well constructed, credible, and controlled (i.e., schedules are periodically updated and progress is monitored). GAO assessed CBP's schedules as of March 2013 for the three highest-cost programs that represent 97 percent of the Plan's estimated cost. GAO found that schedules for two of the programs at least partially met each characteristic (i.e., satisfied about half of the criterion), and the schedule for the other program at least minimally met each characteristic (i.e., satisfied a small portion of the criterion), as shown in the table below. For example, the schedule for one of the Plan's programs partially met the characteristic of being credible in that CBP had performed a schedule risk analysis for the program, but the risk analysis was not based on any connection between risks and specific activities. For another program, the schedule minimally met the characteristic of being controlled in that it did not have valid baseline dates for activities or milestones by which CBP could track progress. Source: GAO analysis of CBP data. Note: Not met—CBP provided no evidence that satisfies any of the criterion. Minimally met—CBP provided evidence that satisfies a small portion of the criterion. Partially met—CBP provided evidence that satisfies about half of the criterion. Substantially met—CBP provided evidence that satisfies a large portion of the criterion. Met—CBP provided complete evidence that satisfies the entire criterion. Further, CBP has not developed an Integrated Master Schedule for the Plan in accordance with best practices. Rather, CBP has used the separate schedules for each program to manage implementation of the Plan, as CBP officials stated that the Plan contains individual acquisition programs rather than integrated programs. However, collectively these programs are intended to provide CBP with a combination of surveillance capabilities to be used along the Arizona border with Mexico, and resources are shared among the programs. According to scheduling best practices, an Integrated Master Schedule is a critical management tool for complex systems that involve a number of different projects, such as the Plan, to allow managers to monitor all work activities, how long activities will take, and how the activities are related to one another. Developing and maintaining an Integrated Master Schedule for the Plan could help provide CBP a comprehensive view of the Plan and help CBP better understand how schedule changes in each individual program could affect implementation of the overall Plan. Moreover, cost-estimating best practices are summarized into four characteristics—well documented, comprehensive, accurate, and credible. GAO's analysis of CBP's estimate for the Plan and estimates completed at the time of GAO's review for the two highest-cost programs showed that these estimates at least partially met three of these characteristics: well documented, comprehensive, and accurate. In terms of being credible, these estimates had not been verified with independent cost estimates in accordance with best practices. Ensuring that scheduling best practices are applied to the three programs' schedules and verifying Life-cycle Cost Estimates with independent estimates could help better ensure the reliability of the schedules and estimates. CBP did not fully follow key aspects of DHS's acquisition management guidance for the Plan's three highest-cost programs. For example, CBP plans to conduct limited testing of the highest-cost program—the Integrated Fixed Tower (IFT: towers with cameras and radars)—to determine its mission contributions, but not its effectiveness and suitability for the various environmental conditions, such as weather, in which it will be deployed. This testing, as outlined in CBP's test plan, is not consistent with DHS's guidance, which states that testing should occur to determine effectiveness and suitability in the environmental conditions in which a system will be used. Revising the test plan to more fully test the program in the conditions in which it will be used could help provide CBP with more complete information on how the towers will operate once they are fully deployed. CBP has identified mission benefits for technologies under the Plan, but has not yet developed performance metrics. CBP has identified such mission benefits as improved situational awareness and agent safety. Further, a DHS database enables CBP to collect data on asset assists, defined as instances in which a technology, such as a camera, or other asset, such as a canine team, contributed to an apprehension or seizure, that in combination with other relevant performance metrics or indicators, could be used to better determine the contributions of CBP's surveillance technologies and inform resource allocation decisions. However, CBP is not capturing complete data on asset assists, as Border Patrol agents are not required to record and track such data. For example, from fiscal year 2010 through June 2013, Border Patrol did not record whether an asset assist contributed to an apprehension event for 69 percent of such events in the Tucson sector. Requiring the reporting and tracking of asset assist data could help CBP determine the extent to which its surveillance technologies are contributing to CBP's border security efforts. This is a public version of a For Official Use Only—Law Enforcement Sensitive report that GAO issued in February 2014. Information DHS deemed as For Official Use Only—Law Enforcement Sensitive has been redacted. GAO recommends that CBP, among other things, apply scheduling best practices, develop an integrated schedule, verify Life-cycle Cost Estimates, revise the IFT test plan, and require tracking of asset assist data. DHS concurred with four of six GAO recommendations. It did not concur with the need for an integrated schedule or a revised IFT test plan. As discussed in this report, GAO continues to believe in the need for a schedule and a revised test plan.