Homeland Security Acquisitions: Outcomes Have Improved but Actions Needed to Enhance Oversight of Schedule Goals
Released: 2019-12-19T00:00:00

Letter

Background

To help manage its multi-billion dollar acquisition investments, DHS has established policies and processes for acquisition management, requirements development, test and evaluation, and resource allocation.

The department uses these policies and processes to deliver systems that are intended to close critical capability gaps, helping enable DHS to execute its missions and achieve its goals.

Acquisition Management Policy

DHS policies and processes for managing its major acquisition programs are primarily set forth in its Acquisition Management Directive 102-01 and Acquisition Management Instruction 102-01-001. DHS issued the initial version of this directive in November 2008 in an effort to establish an acquisition management system that effectively provides required capability to operators in support of the department’s missions. DHS has issued multiple updates to its acquisition management directive and instruction, in part to be responsive to GAO’s recommendations. DHS issued the current version of the directive in February 2019 and the current version of the instruction in May 2019; however, we did not assess programs against these updates because the programs in our review established initial baselines prior to the approval of the directive and instruction. DHS’s Under Secretary for Management is currently designated as the department’s Chief Acquisition Officer and, as such, is responsible for managing the implementation of the department’s acquisition policies.

DHS’s Under Secretary for Management serves as the acquisition decision authority for the department’s largest acquisition programs, those with LCCEs of $1 billion or greater. Component Acquisition Executives—the most senior acquisition management officials within each of DHS’s components—may be delegated acquisition decision authority for programs with cost estimates between $300 million and less than $1 billion. Table 1 identifies how DHS has categorized the 29 major acquisition programs we reviewed in this report, and table 8 in appendix II specifically identifies the programs within each level.

DHS acquisition management policy establishes that a major acquisition program’s decision authority shall review the program at a series of predetermined acquisition decision events (ADE) to assess whether the major program is ready to proceed through the acquisition life cycle phases. Depending on the program, these events can occur within months of each other or be spread over several years. Figure 1 depicts the acquisition life cycle in the March 2016 version of DHS acquisition management policy.

An important aspect of an ADE is the decision authority’s review and approval of key acquisition documents. See table 2 for a description of the type of key acquisition documents identified in the March 2016 acquisition management directive and instruction that required department-level approval before a program moves to the next acquisition phase.

DHS acquisition management policy establishes that the APB is the agreement between program, component, and department-level officials establishing how systems being acquired will perform, when they will be delivered, and what they will cost. Specifically, the APB establishes a program’s schedule, costs, and key performance parameters. DHS defines key performance parameters as a program’s most important and non-negotiable requirements that a system must meet to fulfill its fundamental purpose. For example, a key performance parameter for an aircraft may be airspeed and a key performance parameter for a surveillance system may be detection range.

The APB establishes objective (target) and threshold (maximum acceptable for cost, latest acceptable for schedule, and minimum acceptable for performance) baselines. According to DHS policy, if a program fails to meet any schedule, cost, or performance threshold approved in the APB, it is considered to be in breach. Programs in breach are required to notify their acquisition decision authority and develop a remediation plan that outlines a timeframe for the program to return to its APB parameters, re-baseline—that is, establish new schedule, cost, or performance goals—or have a DHS-led program review that results in recommendations for a revised baseline.

In addition to the acquisition decision authority, other bodies and senior officials support DHS’s acquisition management function:

The Acquisition Review Board reviews major acquisition programs for proper management, oversight, accountability, and alignment with the department’s strategic functions at ADEs and other meetings as needed. The board is chaired by the acquisition decision authority or a designee and consists of individuals who manage DHS’s mission objectives, resources, and contracts.

The Office of Program Accountability and Risk Management (PARM) is responsible for DHS’s overall acquisition governance process, supports the Acquisition Review Board, and reports directly to the Under Secretary for Management. PARM develops and updates program management policies and practices, reviews major programs, provides guidance for workforce planning activities, provides support to program managers, and collects program performance data.

Components, such as U.S. Customs and Border Protection, the Transportation Security Administration, and the U.S. Coast Guard sponsor specific acquisition programs. The head of each component is responsible for oversight of major acquisition programs once the programs complete delivery of all planned capabilities to end users.

Component Acquisition Executives within the components are responsible for overseeing the execution of their respective portfolios.

Program management offices, also within the components, are responsible for planning and executing DHS’s individual programs. They are expected to do so within the cost, schedule, and performance parameters established in their APBs. If they cannot do so, programs are considered to be in breach and must take specific steps, as noted above.

Figure 2 depicts the relationship between acquisition managers at the department, component, and program level.

Requirements Development Process

In 2016, we found that DHS had not effectively implemented or adhered to its review process for major acquisitions and recommended that DHS reinstate the Joint Requirements Council (JRC) to review and approve acquisition requirements and assess potential duplication of effort across the department. DHS established a JRC to develop and lead a component-driven joint requirements process for the department. In March 2016, DHS revised its policy instruction to reflect the addition of the JRC as an acquisition oversight body. Among other responsibilities, the JRC is to provide requirements-related advice and validate key acquisition documentation to prioritize requirements and inform DHS investment decisions among its components. The JRC chair is a member of the Acquisition Review Board and advises the board on capability gaps, needs, and requirements at key milestones in the acquisition life cycle. In March 2019, we reported that the JRC could better fulfill its mission by identifying overlapping or common requirements, and by making recommendations to senior leadership to inform budget decisions and help ensure that DHS uses its finite investment resources wisely. We will continue to monitor the JRC’s efforts through GAO’s high risk work.

Test and Evaluation Policy

In May 2009, DHS established policies that describe processes for testing the capabilities delivered by the department’s major acquisition programs. The primary purpose of test and evaluation is to provide timely, accurate information to managers, decision makers, and other stakeholders to reduce programmatic, financial, schedule, and performance risks. We provide an overview of each of the 29 programs’ test activities in the individual program assessments presented in appendix I.

DHS testing policy assigns specific responsibilities to particular individuals and entities throughout the department:

Program managers have overall responsibility for planning and executing their programs’ testing strategies, including scheduling and funding test activities and delivering systems for testing. They are also responsible for controlling developmental testing, which is used to assist in the development and maturation of products, manufacturing, or support processes. Developmental testing includes engineering- type tests used to verify that design risks are minimized, substantiate achievement of contract technical performance, and certify readiness for operational testing.

Operational test agents are responsible for planning, conducting, and reporting on operational test and evaluation to identify whether a system can meet its key performance parameters and provide an evaluation of the operational effectiveness, suitability, and cybersecurity of a system in a realistic environment. Operational effectiveness refers to the overall ability of a system to provide a desired capability when used by representative personnel. Operational suitability refers to the degree to which a system can be placed into field use and sustained satisfactorily. Operational cybersecurity refers to the degree to which a system is able to accomplish its mission in a cyber-contested environment. The operational test agents may be organic to the component, another government agency, or a contractor, but must be independent of the developer to present credible, objective, and unbiased conclusions.

The Director, Office of Test and Evaluation is responsible for approving major acquisition programs’ operational test agent and test and evaluation master plans, among other things. A program’s test and evaluation master plan must describe the developmental and operational testing needed to determine technical performance and operational effectiveness, suitability, and cybersecurity. As appropriate, the Director is also responsible for observing operational tests, reviewing operational test agents’ reports, and assessing the reports. Prior to a program’s ADE 3, the Director provides the program’s acquisition decision authority a letter of assessment that includes an appraisal of the program’s operational test, a concurrence or non-concurrence with the operational test agent’s evaluation, and any further independent analysis.

As an acquisition program proceeds through its life cycle, the testing emphasis moves gradually from developmental testing to operational testing. See figure 3.

Resource Allocation Process

DHS has established a planning, programming, budgeting, and execution process to allocate resources to acquisition programs and other entities throughout the department. DHS uses this process to produce the department’s annual budget request and multi-year funding plans presented in the FYHSP report, a database that contains, among other things, 5-year funding plans for DHS’s major acquisition programs. According to DHS guidance, the 5-year plans should allow the department to achieve its goals more efficiently than an incremental approach based on 1-year plans. DHS guidance also states that the FYHSP articulates how the department will achieve its strategic goals within fiscal constraints.

At the outset of the annual resource allocation process, the department’s Office of Strategy, Policy, and Plans and Office of the Chief Financial Officer provide planning and fiscal guidance, respectively, to the department’s components. In accordance with this guidance, the components should submit 5-year funding plans to the Chief Financial Officer. These plans are subsequently reviewed by DHS’s senior leaders, including the DHS Secretary and Deputy Secretary. DHS’s senior leaders are expected to modify the plans in accordance with their priorities and assessments, and they document their decisions in formal resource allocation decision memorandums. DHS submits the revised funding plans to the Office of Management and Budget, which uses them to inform the President’s annual budget request—a document sent to Congress requesting new budget authority for federal programs, among other things. In some cases, the funding appropriated to certain accounts in a given fiscal year remains available for obligation and can be carried over to subsequent fiscal years. Figure 4 depicts DHS’s annual resource allocation process.

Federal law requires DHS to submit an annual FYHSP report to Congress at or about the same time as the President’s budget request.

Two offices within DHS’s Office of the Chief Financial Officer support the annual resource allocation process:

The Office of Program Analysis and Evaluation (PA&E) is responsible for establishing policies for the annual resource allocation process and overseeing the development of the FYHSP. In this role, PA&E develops the Chief Financial Officer’s planning and fiscal guidance, reviews the components’ 5-year funding plans, advises DHS’s senior leaders on resource allocation issues, maintains the FYHSP database, and submits the annual FYHSP report to Congress.

The Cost Analysis Division is responsible for reviewing, analyzing, and evaluating acquisition programs’ LCCEs to ensure the cost of DHS programs are presented accurately and completely, in support of resource requests. This division also supports affordability assessments of the department’s budget, in coordination with PA&E, and develops independent cost analyses for major acquisition programs and independent cost estimates upon request by DHS’s Under Secretary for Management or Chief Financial Officer.

Reflecting Improvements Since 2018, 25 of 27 Programs Are on Track to Meet Current Schedule and Cost Goals, with Two Programs Breaching Goals

Of the 27 programs we assessed with approved APBs, 25 are on track to meet their current schedule and cost goals as of August 2019. Of these 25 programs, 11 programs revised their schedule and cost goals in response to a prior breach of their APBs or to incorporate program changes.

Of the 27 programs, two programs breached their schedule or cost goals between January 2018 and August 2019, and as of August 2019 had not yet re-baselined. This shows improvement from our prior review where seven programs were in breach. In addition, some programs, although currently on track to meet their goals, are nonetheless facing risks of breaching schedule or cost goals, or have plans to revise their baseline in the future. Further, as a result of the fiscal year 2019 partial government shutdown, five programs received approval for schedule adjustments, and other programs reported difficulty obligating funds before the end of the fiscal year. Finally, our analysis showed that seven programs are projected to experience an acquisition funding gap in fiscal year 2020, but, according to program officials, these gaps will be mitigated.

We also reviewed two programs that were early in the acquisition process and planned to establish department-approved schedule and cost goals during our review. However, these programs were delayed in getting department approval for their initial APBs for various reasons; therefore, we excluded them from our assessment of whether programs were on track to meet schedule and cost goals. We plan to assess these programs in our future reviews; however, we provide more details on these two programs in the individual assessments in appendix I.

Table 3 summarizes our findings regarding the status of major acquisition programs meeting their schedule and cost goals, and we present more detailed information after the table.

Twenty-five of 27 Programs on Track to Meet Schedule and Cost Goals as of August 2019

We found that 25 of 27 programs we reviewed with department-approved APBs were on track to meet their current baseline schedule and cost goals as of August 2019. Of these, 11 programs met schedule and cost goals established prior to December 2017. Six of these programs are in the process of revising their baselines or plan to revise their baselines in the near future to account for program changes or to add capabilities. For example, the U.S. Coast Guard’s Fast Response Cutter and National Security Cutter programs plan to revise their baselines because they received additional funding to procure more cutters than reflected in their current baselines. Program officials said these programs are planning to update their APBs in fiscal year 2020 to reflect these changes.

In addition, as shown in table 3, five of the 25 programs that met schedule and cost goals had only recently established initial APBs (between January 2018 and August 2019). Three of these five—Customs and Border Protection’s Biometric Entry-Exit program and Border Wall System Program, and the U.S. Coast Guard’s Polar Security Cutter—are new Level 1 major acquisition programs and as of August 2019 their combined life cycle costs were approximately $15 billion. In addition, DHS recently approved baselines for two Transportation Security Administration programs—Advanced Technology and Credential Authentication Technology. These programs were previously projects under the Passenger Screening Program, but according to Transportation Security Administration officials, transitioned into standalone programs to better align program office staffing to capabilities and focus on mitigating capability gaps, among other things.

Eleven of the 25 Programs on Track Had Revised Their Schedule and Cost Goals

Eleven of the 25 programs that we found to be on track to meet current schedule and cost goals revised schedule and cost goals between January 2018 and August 2019. DHS leadership approved revised baselines for these programs for two primary reasons: to remove the program from breach status or to account for program changes, or both.

Five of the 11 programs that revised their baselines did so in response to a breach of their cost or schedule goals and were subsequently removed from breach status. See table 4.

DHS leadership approved revised baselines for these five programs following various actions by the program offices such as:

Customs and Border Protection’s Automated Commercial Environment breached its cost and schedule goals in April 2017, which Customs and Border Protection officials attribute to an underestimation of the level of effort needed to complete development. The program revised its approach to developing remaining functionality by removing some capability from the program’s baseline and delaying development until funding is provided. As shown in table 4, the full operational capability date was delayed. The program’s total life-cycle cost increase is primarily attributed to a change in how threshold cost goals were calculated.

Customs and Border Protection’s Medium Lift Helicopter re- baselined following a schedule breach of its ADE 3, among other things. As part of the re-baselining efforts, the program revised its cost goals to remove personnel costs and update the aircraft operational hours, among other things, which then resulted in a cost decrease of $515 million. Officials reported that the effect of the breach on the program’s schedule was minimal because the program was able to make adjustments to its testing schedule to assess multiple aircraft concurrently.

DHS Management Directorate’s Homeland Advanced Recognition Technology re-baselined following multiple delays in awarding contracts and issues stemming from a subsequent bid protest. The re-baseline included a cost goal decrease resulting from an enhanced solution for biometric data storage.

U.S. Coast Guard’s H-65 Conversion - Sustainment Program re- baselined to address delays which USCG officials primarily attributed to underestimating the technical effort necessary to meet requirements. As part of the re-baseline, the program also added a service life extension program to extend aircraft service life by replacing obsolete components. The program’s total life-cycle cost threshold decreased by approximately $200 million from its prior APB. Coast Guard officials attribute the decrease to the program’s ability to reduce labor costs, among other things, by synchronizing the service life extension program with other aircraft upgrades.

U.S. Citizenship and Immigration Services’ Transformation program re-baselined in June 2018—lifting a strategic pause that limited new program development for 18 months. The program’s revised APB reflects a re-organization of the Transformation program as well as a new development strategy. The program breached its schedule in September 2016 when it failed to upgrade U.S. Citizenship and Immigration Services’ application processing information system to include applications for naturalization.

In addition, between January 2018 and August 2019, DHS leadership approved revisions to six programs’ baselines that were not prompted by a breach. These programs either planned to revise their baselines to incorporate changes in technology, among other things, or to make changes to their scope.

Customs and Border Protection’s Biometric Entry-Exit program revised its schedule goals in March 2019—after establishing an initial baseline in May 2018—to remove ADE 2C, the decision event when low-rate initial production is typically approved.

Customs and Border Protection’s Border Wall System Program revised its baseline in August 2018 to replace sections of the border wall system in the San Diego sector. In addition, in May 2019 the program received approval for an additional baseline to extend the border wall system in the Rio Grande Valley sector.

Customs and Border Protection’s Multi-role Enforcement Aircraft revised its baseline to increase the program’s quantity from 16 to 29 aircraft. The 16 aircraft from the prior APB provided maritime interdiction capabilities. The additional 13 aircraft are for air interdiction capabilities.

Cybersecurity and Infrastructure Security Agency’s National Cybersecurity Protection System Program revised its baseline in January 2018 to inform ADEs for the program’s information sharing and intrusion-prevention capabilities and to account for schedule and cost changes after bid protests. However, the program updated its APB again in October 2018 to address an error found in the LCCE. Specifically, the LCCE that provided the basis for the program’s APB cost goals did not accurately account for the program’s sunk costs. In addition, the program added an additional 2 years of costs to its LCCE and revised its approach to estimating threshold costs. Once revised, the program’s total life-cycle cost threshold increased by more than $1.7 billion (41 percent) from the program’s January 2018 APB. The program’s full operational capability date was extended by two years to March 2021.

Cybersecurity and Infrastructure Security Agency’s Next Generation Networks Priority Services revised its baseline in April 2018 to add capability to provide priority access for landline telephone calls to select government officials during emergencies. As a result, the program’s full operational capability date was extended by 3 years—to December 2025—and total acquisition costs increased by $68 million (10 percent).

Transportation Security Administration’s Technology Infrastructure Modernization program revised its baseline in July 2019 to de-scope the program and narrow the definition of full operational capability. DHS leadership reported that by the time the program had delivered functions needed to meet the needs of end users, the Transportation Security Administration had updated and improved its legacy systems. As a result, costs decreased by $15 million (1 percent) and the program achieved full operational capability 3 years earlier than previously planned.

Two Programs Breached Schedule or Cost Goals and Some Programs Are at Risk of Breaching Goals in the Future

Between January 2018 and August 2019, two programs breached their schedule or cost goals—down from seven programs in our previous assessment. As of August 2019, neither of these programs had revised their baselines.

Customs and Border Protection’s Integrated Fixed Towers program declared a schedule breach of the program’s baseline in February 2019 as a result of delays in negotiations with the Tohono O’odham Nation—a sovereign Native American Nation—regarding access to tribal lands to construct towers and deploy systems. Customs and Border Protection subsequently reached an agreement with the Nation in March 2019. As of September 2019, the program was in the process of revising its APB to adjust deployments within the Nation’s land. Program officials anticipate the program’s full operational capability date will slip from September 2020 to March 2021 as a result of these actions.

Transportation Security Administration’s Electronic Baggage Screening Program updated its LCCE in August 2019 which exceeds its baseline operations and maintenance (O&M) cost threshold. Transportation Security Administration officials attribute the program’s cost breach to an increase in maintenance costs related to sustaining screening technologies longer than initially planned. As of September 2019, the program’s revised APB, which TSA officials said will address the O&M cost increase, had not yet been approved.

In addition, some of the programs on track as of August 2019 are facing risks that might lead to schedule slips or cost growth in the future. For example,

U.S. Coast Guard’s Offshore Patrol Cutter may experience cost increases and schedule slips in the future. Specifically, the program’s shipbuilder reported damages from Hurricane Michael in October 2018 that have resulted in a long-term degradation of its ability to produce the Offshore Patrol Cutters at the previously estimated cost and schedule. As of August 2019, the Coast Guard was still assessing the shipbuilder’s report on the damage sustained and the potential effect on the Offshore Patrol Cutter program.

U.S. Coast Guard’s Polar Security Cutter met established cost and schedule milestones between January 2018 and August 2019, but program officials stated that they anticipate a schedule slip because delivery of the lead ship in the awarded contract is two months after the program’s APB threshold date. We previously found that the program is at risk of experiencing future schedule delays and cost growth. The program’s schedule is driven by the need to address a potential gap in icebreaking capabilities once the Coast Guard’s only operational heavy polar icebreaker reaches the end of its service life as early as 2023. As a result, planned delivery dates are not informed by a realistic assessment of shipbuilding activities. We also found that the program is at risk of costing more than estimated because its LCCE—while adhering to most cost estimating best practices—is not fully reliable as it did not quantify the range of possible costs over the entire life of the program.

Customs and Border Protection’s Biometric Entry-Exit program plans to re-baseline and achieve ADE 3—which will authorize full-rate production—in September 2019. However, program officials stated that not all testing will be completed to inform the ADE 3. As a result, DHS leadership will not have data related to the Biometric Entry-Exit system’s resiliency to cyberattacks before making this decision.

We provide more information in the individual program assessments in appendix I, and we will continue to monitor these programs in future assessments.

Effects from 2019 Partial Government Shutdown Include Schedule Milestone Adjustments and Difficulty Obligating Funds

Due to a lapse in appropriations for fiscal year 2019, the federal government partially shut down from December 22, 2018, to January 25, 2019. Most Level 1 and Level 2 acquisition program staff were furloughed during the partial government shutdown, which affected the execution of these programs. As a result, in March 2019, DHS’s Under Secretary for Management, in coordination with PARM, authorized Component Acquisition Executives to request up to a 3-month extension for any program schedule milestone date, and inform PARM of any proposed changes in writing. PARM officials stated that they developed this process to mitigate program schedule risks since the government shutdown was beyond the control of program officials.

Five programs requested and received approval from DHS leadership to extend schedule milestones by 3 months. Of these, three programs reported that the 3month extension will allow the programs to stay on track to meet their adjusted milestones—Federal Emergency Management Agency’s Logistics Supply Chain Management System, Customs and Border Protection’s Biometric Entry-Exit, and U.S. Coast Guard’s Medium Range Surveillance Aircraft programs. However, Coast Guard officials stated that the Offshore Patrol Cutter program requested approval to extend the program’s ADE 2C milestone to enable Coast Guard officials time to assess the shipbuilder’s report on damage caused by Hurricane Michael before determining the next steps for the program. The Cybersecurity and Infrastructure Security Agency’s Continuous Diagnostics and Mitigation program received approval to extend two schedule milestones—initial operational capability for two segments of the program—because the program experienced delays as a result of the partial government shutdown. In addition, DHS leadership previously directed the program to conduct an ADE 2B for a new segment by March 2019. The ADE 2B has been delayed 9 months to December 2019 to allow the program additional time to complete required acquisition documentation to inform the ADE.

Programs also reported experiencing other effects of the partial government shutdown. Specifically, officials from several programs identified challenges in obligating funds by the end of the fiscal year due to the truncated timeframe. For example, Transportation Security Administration’s Electronic Baggage Screening Program officials reported that as a result of the partial government shutdown, contract awards had been delayed. These officials explained that contracting obligation activities from the component were compressed into the last two quarters of fiscal year 2019 and the program had to compete for contracting officer resources within the limited timeframe.

Affordability Gaps Reported in DHS’s 2020- 2024 Funding Plan Are Generally Mitigated by Funding from Other Sources

Based on the information presented in the 2020-2024 FYHSP report to Congress, DHS’s acquisition portfolio is not affordable over the next 5 years, meaning that the anticipated funding will not be adequate to support the programs. But our analysis found the reported acquisition funding gaps may be overstated when additional information is taken into account. For example, the fiscal year 2020-2024 FYHSP report contained acquisition affordability tables for 21 of the 27 programs we assessed that have approved APBs. Of the 21 programs included in the FYHSP report, 11 were projected to have an acquisition affordability gap in fiscal year 2020. However, some of the cost information used to develop these projections was outdated since the FYHSP report—which was issued in August 2019—relied on cost estimates developed in April 2018. Therefore, we updated the analysis using the programs’ current LCCEs based on the approved scope of the program, as of August 2019 (as presented in the individual assessments in appendix I). In addition, we discussed funding gaps with program officials to determine additional funding sources, such as fees collected, funding from previous fiscal years that remained available for obligation—known as carryover funding, funds provided by components, or funding received above what was originally requested.

Based on our analysis, we found that seven programs may have acquisition funding gaps in fiscal year 2020 rather than the 11 identified in the FYHSP report. However, the affordability gap for all seven programs we identified may be overstated because program officials reported that these programs either had carryover funding, received funding above what was requested, or anticipate receiving funding from the component to mitigate the affordability gap, as shown in table 5.

Further, officials from several programs in our review told us that the programs were projected to experience a funding gap that could cause future program execution challenges, such as cost growth, or that programs were taking steps to mitigate funding gaps. For example, Customs and Border Protection’s Biometric Entry-Exit program—which is primarily fee-funded—conducted an affordability analysis that showed projected fees had declined. To mitigate risks of a potential affordability gap, program officials stated the number of officers to conduct enforcement activities at airport departure gates was reduced and the program is working with the component to identify other sources of funding. In addition, DHS Management Directorate’s Homeland Advanced Recognition Technology program reported that the program will use carryover funding to address the program’s affordability gap in fiscal year 2020. However, the program will also need to defer development of some additional capabilities to 2021 to remain affordable. In addition, officials from Customs and Border Protection’s Border Wall System Program stated the program is mitigating future acquisition funding gaps, in part by not developing its baseline until after funding amounts are determined. According to officials, this was necessary to mitigate program risks due to uncertainty in funding; however, through DHS’s resource allocation process, the program has requested $5 billion each year from fiscal year 2020 to fiscal year 2024.

We elaborate on programs’ affordability over the next 5 years in the individual program assessments in appendix I.

Cost and Performance Goals Generally Trace to Required Documents, but Schedule Goals Do Not

Traceability, which DHS policy and acquisition best practices call for, helps ensure that program goals are aligned with program execution plans, and that a program’s various stakeholders have an accurate and consistent understanding of those plans and goals. We found that the cost and performance goals in the acquisition programs’ approved APBs generally traced to the estimated costs identified in LCCEs and key performance parameters identified in operational requirements documents. That is, information in the APB matched the document required to be used as the basis for the baselines. In contrast, the schedule goals in the approved APBs generally did not trace to the Integrated Master Schedule (IMS), as required by the DHS acquisition management instruction and as a best practice identified in GAO’s Schedule Assessment Guide. Similarly, we found the required basis for the cost and performance goals is consistently identified in DHS acquisition management policy and guidance, whereas the basis for the schedule goals is not.

Acquisition Program Baselines Generally Trace to Required Cost and Performance Documents, but Not to Schedule Documents

We found that cost and performance goals in approved APBs generally traced to estimated costs in LCCEs and key performance parameters in operational requirements documents. However, schedule goals were generally not traceable to the IMSs, as required by DHS acquisition management instruction and as identified as a best practice in GAO’s Schedule Assessment Guide. Of the 27 programs we assessed with established baselines, 21 established or revised their APBs after DHS updated its acquisition management instruction in March 2016, which was the most current version of the guidance when we initiated our review.

Table 6 shows the results of our analysis for the traceability of baselines to cost, schedule, and performance documents for those 21 programs.

As shown in table 6, the APB goals traced to the key performance parameters in the operational requirements documents for all 21 programs that we reviewed. Generally, the APB goals traced to the costs in the LCCEs, though we found that three programs were not traceable. For example:

The APB total life-cycle cost goals for Custom and Border Protection’s Tactical Communications Modernization program traced to the program’s LCCE, but the separate acquisitions and O&M costs were not traceable.

The Transportation Security Administration’s Electronic Baggage Screening Program did not include sunk costs in the LCCE, and as a result the APB cost goals did not trace.

In contrast, we could trace all schedule events and dates in the approved APBs to the programs’ IMS for only six of 21 programs. There was variation in how the programs’ APBs lacked traceability to the IMS. For example:

The IMS for the Customs and Border Protection’s Border Wall System Program estimates the full operational capability dates to be between October 2021 and December 2021, whereas the approved APB includes an objective date of October 2022 and a threshold date of December 2022.

The APB for the U.S. Citizenship and Immigration Services’ Transformation program does not identify a source for the schedule baseline. Program officials told us that they do not have an IMS and instead they use the schedule in the program’s release roadmap, a document that information technology programs use to communicate how they will iteratively deliver features. However, schedule events identified in the APB, such as full operational capability, were not identified in the release roadmap.

Similarly, we found programs that developed an IMS but did not include all future APB milestones, such as Cybersecurity and Infrastructure Security Agency’s Continuous Diagnostics Mitigation and Transportation Security Administration’s Credential Authentication Technology.

According to GAO’s Schedule Assessment Guide, schedules should be verified to ensure that they are vertically traceable—that is, verified to ensure the consistency of dates, status, and scope requirements between different levels of the schedule and management documents. Further, this guide states that a schedule baseline signifies a consensus of stakeholders on the required sequence of events, resources, and key dates. The IMS is more accurate when stakeholders agree on the underlying assumptions. These stakeholders would include, for example, program offices, end users, and component and DHS leadership.

Further, DHS acquisition policy requires programs to obtain review and approval of LCCEs and operational requirements documents from various stakeholders within components and DHS headquarters. However, DHS acquisition policy states that approval of IMSs is based on DHS guidance and component policy and that program managers will provide the IMS to DHS in support of the acquisition review process. Officials from PARM and the Office of the Chief Financial Officer told us that the components vary in their capacity to develop schedules and assess schedule risks and there is a lack of expertise within the department to review program schedules. The lack of traceability between IMSs and schedule goals in the APB indicates that DHS does not have an appropriate oversight process in place to ensure APBs trace to schedule goals in the IMSs, in accordance with DHS policy and GAO’s best practices. Without this traceability, DHS cannot ensure that the understanding of program schedules among different stakeholders is consistent and accurate. As a result, DHS leadership may be approving program schedule goals that do not align with program execution plans.

DHS Acquisition Policy and Guidance Consistently Identifies the Source of Cost and Performance Goals but Not of Schedule Goals

We found that LCCEs and operational requirements documents are consistently identified as the basis of cost and performance goals in DHS’s acquisition management policy and guidance. However, we also found that the documents do not consistently require that an IMS be used as the basis of schedule goals. Specifically, DHS’s acquisition management instruction and DHS’s Systems Engineering Life Cycle Guidebook—which outlines the technical framework for DHS’s acquisition management system—differ regarding the source of APB schedule milestone dates. Table 7 summarizes our findings on DHS’s acquisition policy and guidance related to developing APB cost, schedule, and performance goals.

DHS’s acquisition management instruction states that the APB should trace to the IMS, which is consistent with GAO’s Schedule Assessment Guide. This instruction differs from the guidance in the Systems Engineering Life Cycle Guidebook, which in contrast, directs programs to use the APB as an input when developing the IMS. PARM officials said they were unaware of the inconsistency and confirmed that the IMS should provide the basis of APB schedule goals, as identified in DHS’s acquisition management instruction.

PARM officials also acknowledged that the information related to schedule development should be consistent across all of DHS’s policies, instructions, and guidebooks. Conflicting agency-wide policy and guidance can lead to a lack of clarity and consistency on how programs develop their schedules. In addition, the lack of a well-developed schedule can contribute to poor acquisition outcomes, such as increased costs and delayed delivery of capabilities needed by end users. As previously noted, DHS’s 2019 update to its acquisition management directive and associated instruction addressed a GAO recommendation related to better defining requirements before establishing acquisition program baselines. PARM officials told us they plan to update the Systems Engineering Life Cycle Guidebook by the end of calendar year 2019 to account for the revisions in the acquisition management directive and associated instruction. At that time, they also plan to correct the inconsistency related to the documents used to develop APB schedule goals.

Conclusions

Since we began reviewing DHS’s portfolio of major acquisitions in 2015, the department has strengthened implementation of its policies to improve acquisition oversight. These efforts have begun to yield better results as the performance of DHS’s major acquisition portfolio has improved compared to our last review. As DHS major acquisition policy has evolved over time, the department has put in place oversight and approval processes that help ensure cost and performance goals are clear, consistent, and trace to key acquisition documents serving as the basis for those goals. However, opportunities remain for DHS to provide better oversight of major acquisition programs’ schedule goals, as we found that these goals generally did not trace to the integrated master schedules per DHS policy. When schedule goals are not traceable, DHS decision makers cannot be sure that the schedule presented is consistent and accurate. Until DHS develops an oversight process to ensure schedules are developed and updated appropriately, the department cannot ensure that its most expensive acquisition programs are able to deliver capabilities needed by end users when promised. In addition, we found inconsistencies within DHS’s major acquisition policy and system engineering guidance in identifying the basis of schedule goals. Without consistent schedule development guidance, DHS has no way of knowing that programs establish schedules in a consistent manner and in accordance with GAO’s scheduling best practices.

Recommendations for Executive Action

We are making the following two recommendations to DHS.

The Secretary of Homeland Security should ensure that the Undersecretary for Management develops an oversight process to confirm that programs’ schedule goals are developed and updated in accordance with GAO’s Schedule Assessment Guide, to include ensuring traceability between APB schedule goals and IMSs. (Recommendation 1)

The Secretary of Homeland Security should ensure that the Undersecretary for Management revises the schedule development guidance in the Systems Engineering Life Cycle Guidebook to state clearly that an IMS should be used as the basis for APB schedule goals. (Recommendation 2)

Agency Comments

We provided a draft of this report to DHS for review and comment. DHS’s comments are reproduced in appendix III. DHS also provided technical comments which we incorporated as appropriate. In its comments, DHS concurred with both of our recommendations and identified actions it planned to take to address them.

We are sending copies of this report to the appropriate congressional committees and the Acting Secretary of Homeland Security. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-4841 or makm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Program Assessments

This appendix presents individual assessments for each of the 29 programs we reviewed. Each assessment presents information current as of August 2019. They include standard elements, such as an image, a program description, and summaries of the program’s progress in meeting cost and schedule goals, performance and testing activities, and program management-related issues, such as staffing. The information presented in these assessments was obtained from the Department of Homeland Security (DHS) documentation, answers to our questionnaire by DHS officials, interviews with program officials, and includes our analysis of program information. Each assessment also includes the following figures:

Fiscal Years 2020–2024 Affordability. This figure compares the funding plan presented in the Future Years Homeland Security Program report to Congress for fiscal years 2020-2024 to the program’s current cost estimate. We use this funding plan because the data are approved by DHS and Office of Management and Budget, and was submitted to Congress to inform the fiscal year 2020 budget process. The data do not account for other potential funding sources, such as carryover funding.

Acquisition Program Baseline (APB) Thresholds vs. Current Estimate.

This figure compares the program’s cost thresholds from the initial APB approved after DHS’s acquisition management policy went into effect in November 2008 and the program’s current DHS-approved APB to the program’s expected costs as of August 2019. The source for the current estimate is the most recent cost data we obtained (i.e., a department-approved life-cycle cost estimate, updated life-cycle cost estimates submitted during the resource allocation process to inform the fiscal year 2020 budget request, or a fiscal year 2019 annual life-cycle cost estimate update).

Schedule Changes. This figure consists of two timelines that identify key milestones for the program. The first timeline is based on the initial APB DHS leadership approved after the department’s current acquisition management policy went into effect. The second timeline identifies when the program expected to reach its major milestones as of August 2019 and includes milestones introduced after the program’s initial APB. Dates shown are based on the program’s APB threshold dates or updates provided by the program office.

Test Status. This table identifies key recent and upcoming test events.

It also includes DHS’s Director, Office of Test and Evaluation’s assessment of programs’ test results, if an assessment was conducted.

Staffing Profile. This figure identifies the total number of staff a program needs (measured in full time equivalents) including how many are considered critical and how many staff the program actually has.

Lastly, each program assessment summarizes comments provided by the program office and identifies whether the program provided technical comments.

Page left blank intentionally.

AUTOMATED COMMERCIAL ENVIRONMENT (ACE)

CUSTOMS AND BORDER PROTECTION (CBP)

The ACE program is developing software that will electronically collect and process information submitted by the international trade community. ACE is intended to provide private and public sector stakeholders access to information, enhance the government’s ability to determine whether cargo should be admitted into the United States, increase the efficiency of operations at U.S. ports by eliminating manual and duplicative trade processes, and enable faster decision making.

Program completed operational testing in June 2018, but cybersecurity was not tested.

Collections functionality will remain in the legacy system until additional funding is provided for development.

GAO last reported on this program in May 2018 and March 2018 (GAO-18-339SP, GAO-18- 271).

Not included

Following a cost and schedule breach in April 2017, CBP separated the ACE program’s Collections functionality—which collects and processes duties owed on imported goods—from its Core functionality to permit deployment of the other post-release capabilities, such as Liquidations and Reconciliation. CBP previously reported that officials were not versed in the complexities of collection in the legacy system and underestimated the level of effort required to integrate Collections capabilities into ACE. In August 2018, the program received Department of Homeland Security (DHS) approval to defer Collections functionality as an unfunded requirement. CBP officials said the Collections functionality will remain in the legacy system until funding for development is provided.

ACE continued deployment of the Core functionality and updated acquisition documents including the program’s acquisition program baseline (APB) and life-cycle cost estimate (LCCE) to reflect the program changes. DHS leadership approved the program’s updated APB in November 2018—removing the program from breach status. The program achieved full operational capability (FOC) for Core functionality and received acquisition decision event (ADE) 3 approval in November 2018— approximately 2 years later than initially planned.

Although the program removed costs associated with Collections functionality, the program’s total APB cost threshold increased by more than $500 million from its prior APB. This cost increase is primarily the result of a change in the way the program’s threshold costs were calculated. CBP officials estimated the total cost of decoupling Collections from ACE’s remaining functionality to be $30 million.

In March 2019, the program received funding and approval for ADE 2B for the first of four planned releases of Collections functionality, but did not receive funding for the remaining releases. CBP officials applied for Technology Modernization Funds (TMF). However, in September 2019, CBP officials stated that a decision on TMF funding had not yet been made. CBP officials estimated that it would take 18 months to move Collections into ACE. In June 2019, the program updated its LCCE to inform the budget process—the LCCE includes some costs for Collections functionality, but the total cost is not yet known.

Customs and Border Protection (CBP) AUTOMATED COMMERCIAL ENVIRONMENT (ACE)

• ACE Core functionality met all four of its key performance parameters. • ACE Core functionality is operationally suitable and operationally effective with limitations, primarily because the lack of a mature mass system update function for ESAR decreased the day-to-day operational efficiency. • Cybersecurity was not evaluated.

DOT&E recommended that the program continue the development of the ESAR capabilities to improve operational effectiveness and conduct follow-on OT&E to ensure the issues are corrected. DOT&E also recommended that the program should conduct cybersecurity testing after submitting the test plan for DOT&E approval. In June 2019, CBP officials told GAO that the program plans to conduct follow-on OT&E by March 2020 and to begin cybersecurity testing in late fiscal year 2020, following the migration to cloud services.

When DHS leadership re-baselined the ACE program in 2013, the program adopted an agile software development methodology to accelerate software creation and increase flexibility in the development process. The ACE program office oversees agile teams that conduct development and O&M activities. Staffing needs for ACE have decreased in the last year, which CBP officials attribute to the program completing most development efforts. These officials explained that staff from prior agile development teams were shifted to sustainment teams.

In June 2019, CBP officials told GAO that, while ACE has some critical staffing gaps, these gaps have not affected program execution. CBP officials also stated that they plan to use existing contracts to address staffing needs for the Collections functionality, once funding for development is received.

CBP officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

CUSTOMS AND BORDER PROTECTION (CBP)

The BEE program is intended to verify the identities of travelers leaving the United States at air, land, and sea ports of entries using biometric data, such as facial recognition. The program has developed a capability to match photos of departing travelers to their passport photos or photos obtained upon a traveler’s arrival into the United States to identify foreign nationals that stay in the United States beyond their authorized periods of admission. CBP is currently focused on the air segment.

Program deploying capabilities beyond approved quantity without approval from leadership.

CBP pursuing public/private partnerships to reduce costs.

GAO last reported on this program in May 2018 and February 2017 (GAO-18-339SP, GAO-17-170).

In May 2018, the Department of Homeland Security (DHS) leadership approved BEE’s initial acquisition program baseline (APB) which established the cost, schedule, and performance parameters for the air segment. DHS leadership subsequently granted the BEE program acquisition decision event (ADE) 2A approval for this segment and directed the program to return for a combined ADE 2B/C.

DHS leadership delayed the program’s ADE 2B decision—which will authorize the program to initiate development of the air segment—from October 2018 to December 2018 to allow for the completion of the test and evaluation master plan (TEMP). However, in October 2018, CBP officials told GAO that the facial matching service was ready to support nationwide deployment, and the program was on track to reach its initial operational capability (IOC) of supporting 30 international flights per day by December 2018. DHS leadership approved the program’s request to remove ADE 2C—which would authorize low-rate production—from its APB and granted the program ADE 2B in December 2018. In March 2019, DHS leadership approved the program’s updated APB, which reflected schedule changes related to the TEMP, schedule slips related to the fiscal year 2019 partial government shutdown, and removal of ADE 2C. The program’s APB costs goals remained the same. CBP officials said the program plans to re-baseline and achieve ADE 3—which will authorize full-rate production—in September 2019. However, in June 2019, CBP officials told GAO the program has continued to deploy capabilities to airports and airlines—beyond those needed to achieve IOC.

The BEE program is primarily funded by fees. Congress provided that half the amount collected from fee increases for certain visa applications from fiscal years 2016 through 2025—up to $1 billion—would be available to DHS until expended for the development and implementation of the BEE system. In February 2018, Congress extended this period through fiscal year 2027. CBP officials said the current funding structure poses challenges because fees fluctuate based on immigration rates. The program conducted an affordability analysis in 2018 that showed projected fees had fallen from $115 million per year to $56 million per year. To address the funding gap, the program reduced the number of officers conducting enforcement activities at airport departure gates and is working with CBP to identify other sources of funding. .

Customs and Border Protection (CBP)

Prior to initial OT&E, CBP had conducted a number of tests. For example, from 2013 to 2015, CBP completed a pilot of the air segment solution, among other technologies, to inform the acquisition of a BEE system. In March 2018, CBP completed developmental testing on the cloud-based facial matching service for the air segment, which demonstrated that functional requirements were met.

Since 1996, several federal statutes have required development of an entry and exit system for foreign nationals. DHS has been exploring biometric exit capabilities since 2009 and an Executive Order issued in March 2017 directed DHS to expedite the implementation of the BEE system.

CBP is pursuing public/private partnerships in which airlines and airports invest in the equipment to collect biometric data to reduce program costs and improve the passenger boarding process. In September 2019, CBP officials told GAO they have received commitment letters from 28 airports and airlines since March 2018 and officials expect to operate within the airports with the highest volume of international flights by October 2021. CBP officials also told GAO that the program works independently with airlines and airports and does not seek any component or department approvals before proceeding to deploy technologies. These officials stated they proceed in this manner because program stakeholders have been highly engaged since the program’s ADE 1, internal testing results have been positive, and the congressional mandate necessitates expediency.

CBP officials said the program’s current staffing level is manageable, but they will need more staff in the future to help manage planned partnerships with airlines and airports.

CBP provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

CUSTOMS AND BORDER PROTECTION (CBP)

The border wall system is intended to prevent the illegal entry of people, drugs, and other contraband by enhancing and adding to the 654 miles of existing barriers along the U.S. southern border. CBP plans to create a border enforcement zone between a primary barrier—such as a fence—and a secondary barrier. To establish the enforcement zone, the wall system may also include detection technology, surveillance cameras, lighting, and roads for maintenance and patrolling.

Program establishes baselines as funding is received, but does not have a cost estimate to support funding plan.

Current baselines do not account for all DHS and DOD border wall system construction efforts.

GAO last reported on this program in July 2018 and May 2018 (GAO-18-614, GAO-18- 339SP).

The Department of Homeland Security (DHS) plans to establish cost, schedule, and performance goals for each individual segment of the border wall system in separate acquisition program baselines (APB) as funding becomes available. The program’s current APBs were approved in May 2019 and account for segments funded in fiscal years 2018 and 2019, totaling nearly 123 miles of border wall system.

DHS leadership approved a revised APB for the two segments funded in fiscal year 2018. This included cost and schedule goals for the replacement of an existing 14 miles of primary and secondary barriers in San Diego. It also refined the cost goals for an initial 60 mile segment in the Rio Grande Valley (RGV), because in the 2018 and 2019 Consolidated Appropriations Acts, Congress prohibited use of funds for construction in areas constituting about 4 miles. The program’s total cost for these efforts is nearly $2.2 billion.

DHS leadership approved an initial APB for a second segment of nearly 53 miles in RGV in response to funding received in fiscal year 2019. The program’s total cost for this segment is approximately $2.6 billion. However, the design for this segment has not yet been approved, which could affect APB costs or schedule or both.

In June 2019, to inform the budget process, the program developed a cost estimate that appears much greater than its APB goals because it reflects DHS’s funding request to Congress—not the current plans of the program. DHS officials reported that they did not have a cost estimate to support the requested amounts because the program develops acquisition documentation after funding becomes available.

The current APBs do not account for related construction efforts that may limit oversight of the development of the entire border wall system. For example, in November 2018, CBP leadership was granted approval to oversee a segment replacing about 48 miles of primary pedestrian wall. Further, in February 2019, DHS requested that the Department of Defense (DOD) assist with the construction of infrastructure along the southern border. DOD agreed to provide support and is using $2.5 billion of DOD’s fiscal year 2019 funds to support these efforts. In September 2019 DOD officials identified an additional $3.6 billion, if needed. CBP officials told GAO that they provided a prioritized list of segments and construction standards to DOD, but said that they have limited insight into DOD’s planned efforts.

05/19 FY 2018 APB revised/ FY19 initial APB approved 03/23 FY 2018 segments full operational capability (FOC)

Customs and Border Protection (CBP)

In November 2017, the Science and Technology Directorate’s Office of Systems Engineering completed a technical assessment on the program and identified risks related to the integration and operation of enforcement zone technologies—such as cameras and sensors—which had not been clearly defined or planned for within the wall system. It made several recommendations, including that the program coordinate with an ongoing CBP study of land domain awareness capabilities, which DHS leadership directed CBP to conduct in October 2016 to inform a comprehensive border plan.

CBP previously completed testing of eight barrier prototypes to help refine the requirements and identify new design standards for barriers. However, use of CBP funding appropriated for construction of fencing in the RGV for fiscal year 2018 and 2019 is restricted to operationally effective designs deployed as of May 5, 2017.

The Border Wall System Program was initiated in response to an Executive Order issued in January 2017 stating that the executive branch is to secure the southern border through the immediate construction of a physical wall on the southern border of the United States. To expedite the acquisition planning process, CBP officials said they leveraged expertise from staff that worked on previous border fencing programs and were familiar with implementation challenges, such as land access. CBP intends to prioritize segments based on threat levels, land ownership, and geography, among other things.

CBP plans to continue coordinating with the U.S. Army Corps of Engineers for engineering support and for awarding and overseeing the construction contracts. CBP officials stated that land access and acquisition issues are significant challenges and could affect the program’s ability to meet its schedule goals.

CBP officials reported that the program has sufficient staff to manage the program’s work based on the funding received to date. The program’s unfilled staffing gaps are not yet funded positions. CBP officials stated that they will hire additional staff to fill the vacant positions once funding becomes available.

CBP officials reviewed a draft of this assessment and provided no comments.

CROSS BORDER TUNNEL THREAT (CBTT)

CUSTOMS AND BORDER PROTECTION (CBP)

The CBTT program is intended to help CBP identify, acquire, and implement operational services and technologies necessary to obtain subterranean domain awareness along the United States land border. These technologies will help CBP address existing gaps in the prediction, detection, confirmation, investigation, and remediation of cross border tunnels.

CBP’s analysis of alternatives for detection capabilities identified a solution and CBP will conduct future analysis.

Program performed two technology demonstrations, and CBP officials determined technologies were sufficient.

GAO last reported on the program in August 2018 and May 2017 (GAO-18-550, GAO-17-474).

Not included

In August 2015, the Department of Homeland Security’s (DHS) Under Secretary for Management (USM) granted the CBTT program acquisition decision event (ADE) 1 approval. The program initiated work on an analysis of alternatives (AoA) in March 2016, which considered technologies to detect four CBP classifications of illicit tunnels—rudimentary, sophisticated, mechanically bored, and interconnecting tunnels—but yielded no results. Program leadership and stakeholders subsequently determined that the AoA should be refocused to address tunnel detection threats in seven high-risk operational areas and broadened to incorporate newer tunnel detection technologies, among other things. In May 2018, the AoA was completed and, based on its results, CBP identified a preferred system—a variation of a legacy tunnel detection system used by the Department of Defense (DOD).

In June 2018, DHS leadership directed the program to continue technology demonstrations of upgrades to the legacy tunnel detection system in order to mitigate technical and operational risks and refine program requirements, including identification of the areas where the capability will be deployed. At that time, DHS leadership directed the program to return to the acquisition review board for a combined ADE 2A and 2B to establish an initial acquisition program baseline (APB) for tunnel detection capability. CBP officials said the program now plans to pursue only ADE 2A when it returns to the acquisition review board, per DHS’s revised acquisition policy.

As of September 2019, the program had not yet completed key acquisition documents that will support the program’s APB. CBP officials told GAO that the program experienced delays in updating the acquisition documents—including the operational requirements document—for the detection capability as a result of continued work with stakeholders. The program continues to work with stakeholders to refine end- user requirements, determine testing needs, and complete a technical assessment.

CBP officials told GAO that the program plans to use an incremental acquisition approach to address the other capability gaps. They added that the incremental approach is necessary because the capability gaps the program intends to address are broader than one system can cover.

Customs and Border Protection (CBP)

CROSS BORDER TUNNEL THREAT (CBTT)

PERFORMANCE AND TESTING OPERATIONAL TEST AGENT (OTA): NOT APPLICABLE The AoA results indicated the preferred detection system solution outperformed alternative systems in detection of key tunnel types and activities at operationally significant depths in high-risk areas. The preferred detection system solution supports the program’s priorities of persistent surveillance and actionable information. The AoA scope focused on the capability to detect the presence of tunneling activities and project the trajectory of discovered tunnels. Other capabilities, like predicting tunnel location, will be addressed in future AoAs and technology demonstrations.

In June 2019, CBP officials told GAO that, in response to direction from DHS leadership, the program successfully performed two limited technology demonstrations in high-risk operational areas. The first limited technology demonstration evaluated how the preferred tunnel detection system used by DOD operated in CBP’s border enforcement zone. The second limited technology demonstration, conducted by a contractor, evaluated a different system and software. Based on these technology demonstrations, CBP officials told GAO they determined the technologies were sufficient. CBP officials also told GAO the program plans to continue evaluating technologies in coordination with Border Patrol’s Requirements Division.

In 2008, CBP began collaborating with the DHS Science and Technology Directorate, other federal partners, and private industry to develop and acquire tunnel detection technology. In September 2012, the DHS Inspector General found that CBP did not have the technological capability to detect illicit cross-border tunnels routinely and accurately. DHS leadership subsequently approved the CBTT Mission Needs Statement, which identified six capabilities—predict the location of illicit tunnels; detect the presence of suspected tunnels and tunneling activity and project the trajectory of a discovered tunnel; confirm a tunnel’s existence and map its location and measurements; investigate and exploit tunnels and tunnel activity; remediate discovered tunnels; and coordinate information sharing on tunnel threats.

CBP officials stated that the CBTT Concept of Operations (CONOPS) was approved in June 2019. CBP officials also stated that the development of the CONOPS was informed by market research and AoA activities.

CBP officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

INTEGRATED FIXED TOWERS (IFT)

CUSTOMS AND BORDER PROTECTION (CBP)

The IFT program helps the Border Patrol detect, track, identify, and classify illegal entries in remote areas. IFT consists of fixed surveillance tower systems equipped with ground surveillance radar, daylight and infrared cameras, and communications systems linking the towers to command and control centers. CBP plans to deliver or upgrade approximately 48 IFT systems across six areas of responsibility (AoR) in Arizona: Nogales, Douglas, Sonoita, Ajo, Tucson, and Casa Grande.

System acceptance test completed in Sonoita AoR; all systems accepted by program.

Border Patrol requested CBP add camera suites to address tower reductions in the Ajo and Casa Grande AoRs.

GAO last reported on this program in May 2018 and November 2017 (GAO-18-339SP, GAO-18-119).

The program declared a potential schedule breach in December 2017 because the program did not receive funding from the Department of Homeland Security (DHS) to address new IFT requirements, including camera upgrades and replacement of existing tower systems deployed in Tuscon and Ajo under a legacy program. In January 2018, CBP officials updated the program’s affordability analysis to reflect a reduction of IFT tower deployments—which mitigated the potential schedule breach. Specifically, a resolution passed within the Tohono O’odham Nation—a sovereign Native American Nation—that reduced the number of IFT tower systems CBP can deploy on the Nation’s land from 15 to 10. This reduction mitigated the funding shortfall that had put the program at risk of not achieving full operational capability (FOC) in September 2020.

In February 2019, CBP declared a schedule breach of the program’s current acquisition program baseline (APB) as a result of delays in the negotiations with the Tohono O’odham Nation regarding access to tribal lands to construct towers and deploy IFT systems in the Ajo and Casa Grande AoRs. CBP subsequently reached an agreement with the Nation in March 2019. DHS leadership directed the program to revise its APB to reflect changes in tower deployments. CBP officials told GAO they submitted a revised APB to DHS leadership in June 2019, but as of September 2019 it had not yet been approved. CBP officials anticipate the program’s FOC date will slip to March 2021 as a result of these actions.

In June 2019, the program updated its life-cycle cost estimate (LCCE) to inform the budget process. The updated LCCE includes estimated costs for camera upgrades and accounts for the reduction in IFT systems.

CBP completed deployments in the Sonoita AoR in October 2017 and replaced legacy systems in the Tucson and Ajo AoRs in September 2018 and December 2018, respectively. In January 2015, Border Patrol requested the program prioritize replacing these legacy systems because the technology was obsolete and more expensive to maintain than IFT technology planned for deployment in other AoRs.

10/15 Initial operational capability (Nogales)

Customs and Border Protection (CBP)

INTEGRATED FIXED TOWERS (IFT)

Previously, the OTA found that the program met only 2 of its 3 KPPs and experienced five operational deficiencies during a limited user test conducted in the Nogales AoR in November 2015. However, program and Border Patrol officials did not concur with several of the test results and reported deficiencies with the testing. DHS’s Director, Office of Test and Evaluation did not conduct a formal assessment of the test results because full deployment of the IFT program had already been authorized. Program officials do not plan to conduct additional testing at this time because the program does not have any new requirements. Program officials also stated that if requirements were added, the program would need to conduct additional testing.

When CBP initiated the IFT program, it decided to procure a non-developmental system, and it required that prospective contractors demonstrate their systems prior to CBP awarding the contract. The program awarded the contract to EFW, Inc. in February 2014, but the award was protested. GAO sustained the protest and CBP reevaluated the offerors’ proposals before it decided to re-award the contract to EFW, Inc. As a result, EFW, Inc. could not initiate work at the deployment sites until fiscal year 2015.

According to CBP officials, the number of IFT towers deployed to a single AoR is subject to change based on Border Patrol assessments. Border Patrol was briefed and approved the reduction of towers within tribal lands. To mitigate capability gaps resulting from the tower reduction, Border Patrol requested the program deploy two additional IFT camera suites in Ajo.

DHS leadership directed CBP to develop a border technology plan that includes IFT capabilities. According to CBP officials, the plan calls for an additional 11 AoRs and 35 IFTs. Although the program has not yet received funding for expansion to the 11 AoRs, in September 2018, CBP officials stated they began updating acquisition documents. CBP officials also stated the program does not have a staffing gap, but will require additional staff if funding for the expansion to the 11 AoRs is received.

CBP officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

MEDIUM LIFT HELICOPTER (UH-60)

CUSTOMS AND BORDER PROTECTION (CBP)

UH-60 is a medium-lift helicopter that CBP uses for law enforcement and border security operations, air and mobility support and transport, search and rescue, and other missions. CBP’s UH-60 fleet consists of 20 aircraft acquired from the U.S. Army in three different models. CBP previously acquired 4 modern UH-60M aircraft and converted 6 of its older 16 UH-60A aircraft into more capable UH-60L models. CBP is replacing the remaining 10 UH-60A with reconfigured Army HH-60L aircraft.

Flight acceptance testing for the first reconfigured aircraft completed in February 2018.

Program is assessing additional medium lift capability requirements.

GAO last reported on this program in May 2018 (GAO-18-339SP).

In July 2018, Department of Homeland Security (DHS) leadership granted the program acquisition decision event (ADE) 3 approval and approved the replacement of CBP’s remaining UH-60A aircraft for reconfigured Army HH-60L aircraft. CBP will begin replacing its UH-60A model aircraft on a one-to-one basis as the reconfigured Army HH-60Ls are delivered. DHS leadership previously approved the transfer of three reconfigured HH-60Ls. According to CBP officials, the ADE 3 approval to replace the remaining seven aircraft was based on the evaluation of an initial reconfigured prototype, which was delivered in 2018. CBP officials anticipate that the second and third reconfigured HH-60Ls will be delivered in fiscal year 2020.

The program re-baselined as part of the ADE 3 approval process, removing it from breach status. The program previously experienced cost increases after accommodating a change in DHS’s appropriations structure and schedule slips because of a directive from DHS to develop a comprehensive border plan, which contributed to delays in getting approvals for some of the documents required for ADE 3. The program also anticipated delays in delivery for the second reconfigured HH-60L because of a redesign to be compliant with federal aviation regulations. DHS leadership and CBP officials determined that the effect of the schedule breach was minimal because the program was able to adjust its schedule so that the second and third reconfigured HH-60Ls can be accepted concurrently. The program still plans to achieve full operational capability (FOC) in September 2022 once all 10 of the reconfigured HH-60L aircraft are accepted and deployed.

The program updated its life-cycle cost estimate (LCCE) to inform the program’s revised acquisition program baseline (APB). The program’s acquisition cost thresholds increased by nearly $100 million, and the operations and maintenance (O&M) cost thresholds decreased by approximately $15 million. These changes reflect updates to aircraft operational hours and the results of the Army’s annual obsolescence study, among other things. The updated LCCE also removes personnel costs included in the program’s initial APB, which CBP officials previously told GAO are funded through a separate, central funding account for all of CBP’s air and marine assets.

Customs and Border Protection (CBP)

MEDIUM LIFT HELICOPTER (UH-60)

CBP determined that the converted UH-60L and UH-60M aircraft met all five of the program’s key performance parameters (KPP) through operational test and evaluation (OT&E) conducted in fiscal years 2012 and 2014. However, DHS’s Director, Office of Test and Evaluation (DOT&E) did not validate these results because UH-60 was not considered a major acquisition when the tests were conducted.

In January 2016, DHS leadership directed the program to conduct acceptance functional flight checks on a reconfigured HH-60L prototype prior to receiving approval to proceed with the remaining replacements. This testing concluded in February 2018. Testers rated the aircraft’s performance, handling, and systems integration as excellent, but found a deficiency in the intercom system. The Army designed a fix that is being incorporated into the second and third reconfigured HH-60L aircraft and will be retrofitted into the prototype.

CBP does not plan to conduct formal OT&E on the reconfigured HH-60L because, according to CBP officials, the aircraft has minimal differences from the converted UH-60L aircraft that was previously tested. CBP officials also stated that the program has been able to leverage Army test data, which reduced the risk and testing costs associated with the program. These officials noted that CBP plans to conduct additional testing on the second reconfigured HH-60L to verify design changes and that CBP pilots will perform additional inspections prior to accepting all future aircraft.

In July 2018, DHS leadership directed CBP to address requirements for additional medium-lift capability, including coordinating with Department of Defense and DHS stakeholders, such as the U.S. Coast Guard, that also maintain a fleet of H-60 aircraft. CBP officials stated a desire to replace its other medium lift helicopters as they are retired from the fleet with additional reconfigured HH-60L aircraft. This would not increase the overall number of medium lift helicopters, but would increase the number of UH-60 aircraft. If the number of UH-60 aircraft increases, the program will need to seek approval from DHS and extend its FOC date. In April 2019, CBP updated its interagency agreement with the Army to support completing the program’s currently approved quantity. According to CBP officials, this agreement could support acquiring additional reconfigured HH-60Ls if approved by DHS.

CBP previously acquired UH-60 as a part of its Strategic Air and Marine Program (StAMP). In July 2016, DHS leadership designated UH-60 as a separate and distinct major acquisition program. In October 2018, CBP officials told GAO they continue to maintain a consolidated program office where the same staff from StAMP support all remaining acquisitions, including UH-60. CBP officials said they have refined the program’s staffing profile and taken steps to mitigate the gap. For example, in June 2019, CBP officials said they had hired four new employees and established a memorandum of agreement with CBP’s Office of Acquisition for matrixed support to assist with developing acquisition documents, as needed.

CBP officials stated that as of August 2019, DHS’s Joint Requirements Council validated a requirement for 35 total Medium Lift Helicopters, and the program office is working on a strategy to achieve that inventory target. CBP officials also provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

MULTI-ROLE ENFORCEMENT AIRCRAFT (MEA)

CUSTOMS AND BORDER PROTECTION (CBP)

MEA are fixed-wing, multi-engine aircraft that can be configured to perform multiple missions including maritime, air, and land interdiction, as well as signals detection to support law enforcement. The maritime and air interdiction MEA are equipped with search radar and an electro-optical/infrared sensor to support maritime surveillance and airborne tracking missions. MEA will replace CBP’s fleet of aging C-12, PA-42, and BE-20 aircraft.

Air interdiction configuration is operationally effective and suitable with limitations; cyber testing is not complete.

Program developing requirements for next configuration; pursuing total of 38 MEA.

GAO last reported on this program in May 2018 (GAO-18- 339SP).

In February 2019, Department of Homeland Security (DHS) leadership approved a revised acquisition program baseline (APB), which increased the program’s quantity to 29 MEA: 16 previously approved maritime interdiction MEA and 13 additional air interdiction MEA. CBP officials told GAO they also requested approval to acquire all remaining air interdiction MEA. However, in April 2019, DHS leadership directed CBP to complete follow-on operational test and evaluation (OT&E) of the air interdiction configuration and undergo an acquisition decision event (ADE) 3 review before the program could receive full-rate production approval.

DHS leadership previously approved CBP’s request to procure additional aircraft in the air interdiction configuration that exceeded the program’s initial baseline of 16 MEA. Specifically, DHS leadership approved procurement of MEA 17 in September 2017 after congressional conferees agreed to an additional aircraft beyond DHS’s budget request. In addition, DHS leadership approved MEA 18-20 in August 2018. CBP officials told GAO it was necessary to procure additional MEA to maintain the production schedule for already ordered aircraft.

CBP officials accepted delivery of MEA 16 in February 2019—completing delivery of all maritime interdiction configured MEA. CBP officials said the program experienced a few months delay in delivery of MEA 13-16 because the contractor began laying off staff prior to the program receiving DHS leadership approval to acquire MEA 18-20. According to CBP officials, the program will need to receive ADE 3 approval to procure the remaining air interdiction MEA before the end of September 2019 to avoid future production issues. The program’s revised APB extends the program’s full operational capability (FOC) date by nearly 7 years, to account for the production and delivery of the air interdiction aircraft.

The program updated its life-cycle cost estimate (LCCE) in September 2018 to inform its revised baseline. This estimate decreased by approximately $1.4 billion from the program’s previous LCCE due to a reduction in the number of total aircraft—from the program’s proposed end state of 38 MEA to the 29 included in its revised APB—and planned flight hours.

Customs and Border Protection (CBP)

MULTI-ROLE ENFORCEMENT AIRCRAFT (MEA)

The program previously met all five of its key performance parameters (KPP) for the maritime interdiction configuration. The program established two additional KPPs for the air interdiction configuration related to radar detection. According to CBP officials, the only difference between the maritime and air interdiction configurations is the radar software. The MEA’s new mission system processor was tested in July 2015 on the maritime interdiction configuration.

During the first phase of follow-on OT&E, the program met the two air interdiction KPPs. In August 2019, DHS’s Director, Office of Test and Evaluation (DOT&E) assessed the results and found the air interdiction radar software to be operationally effective but operationally suitable with limitations primarily because of a lack of spare parts, which affects the mission readiness of the MEA fleet. DOT&E recommended that the program develop a maintenance program to better track failure rates and project spare requirements, purchase spares at the level necessary to support the fleet, and complete OT&E of cyber resilience, among other things.

In April 2016, CBP identified capability needs in three additional mission areas and proposed increasing the program’s total to 38 MEA by adding 13 air (reflected in the February 2019 APB), six land interdiction MEA, and three signals detection MEA. The Joint Requirements Council endorsed CBP’s findings, but recommended CBP develop a number of requirements documents—including an operational requirements document (ORD)—to fully validate the findings. In June 2019, CBP officials said they had begun developing requirements for the land interdiction MEA—the next configuration the program plans to pursue.

CBP previously acquired MEA as a part of its Strategic Air and Marine Program (StAMP). In July 2016, DHS leadership designated MEA as a separate and distinct major acquisition program. In October 2018, CBP officials told GAO they continue to maintain a consolidated program office where the same staff from StAMP support all remaining acquisitions, including MEA. CBP officials said they have refined the program’s staffing profile and taken steps to mitigate the gap. For example, in June 2019, CBP officials said they had hired four new employees and established a memorandum of agreement with CBP’s Office of Acquisition for matrixed support to assist with developing acquisition documents, as needed. CBP officials previously told GAO that the staffing gap contributed to delays in developing acquisition documentation for the air interdiction MEA.

CBP officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

CUSTOMS AND BORDER PROTECTION (CBP)

The NII Systems Program supports CBP’s interdiction of weapons of mass destruction, contraband such as narcotics, and illegal aliens being smuggled into the United States, while facilitating the flow of legitimate commerce. CBP officers use large- and small-scale NII systems at air, sea, and land ports of entry; border checkpoints; and international mail facilities to examine the contents of containers, railcars, vehicles, baggage, and mail.

CBP is evaluating technologies to increase efficiencies and address capability gaps.

Staffing challenges pose risk to current program execution and planning for follow-on to NII program.

GAO last reported on this program in May 2018 (GAO-18-339SP).

The NII Systems program is on track to meet its approved cost and schedule goals. The Consolidated Appropriations Act of 2019 included $570 million of acquisition funding for the NII program—$520 million above the President’s budget level. CBP officials told GAO they plan to use the additional acquisition funding primarily to increase scanning capability at land points of entry along the southwest border by recapitalizing some large-scale capabilities and deploying additional small-scale capabilities.

The program updated its life-cycle cost estimate (LCCE) in June 2018. The program’s acquisition costs remain within its acquisition program baseline (APB) cost thresholds and continue to decrease. Compared to the prior year’s estimate, the program’s acquisition costs decreased by $81 million and operations and maintenance increased by $33 million. However, the LCCE update only estimated costs through fiscal year 2026—9 years short of the program’s final year. In June 2019, CBP officials told GAO that they were in the process of updating the program’s LCCE. These officials stated that they plan to extend the LCCE through the program’s final year and adjust program costs based on program changes made in response to the additional funding received.

CBP plans to deploy full operational capability (FOC) quantities of 342 large- and 5,455 small-scale NII systems in fiscal year 2020—4 years earlier than the program’s current APB threshold date. In November 2018, Department of Homeland Security (DHS) leadership decided that once FOC quantities for large and small-scale systems are deployed, CBP will initiate a transfer of the NII program to the operational activity for sustainment efforts. In addition, once FOC quantities are deployed, DHS leadership determined that CBP may adjust large- and small-scale NII deployment quantities in excess of FOC with similarly capable systems to address changing capacity needs and emerging threats. CBP is assessing requirements to address capability gaps, such as increased throughput. In June 2019, CBP officials reported that some technologies being assessed can be procured through the current NII program because CBP considers them to be similarly capable systems. However, these officials also told GAO that CBP is developing acquisition documents to inform a follow-on NII program for other technologies.

Customs and Border Protection (CBP)

CBP officials are coordinating with DHS’s Science and Technology Directorate to evaluate technologies and concepts of operation to increase efficiencies and address capability gaps. CBP officials said that they will incorporate these solutions in a new acquisition program as a follow-on to NII. The NII Systems program is developing a technology demonstration plan to detail how pilot project demonstrations will inform decisions regarding future acquisitions of NII systems technology.

CBP is in the process of assessing requirements to inform the follow-on NII program. In March 2017, the Joint Requirements Council (JRC) validated a capability analysis report (CAR) that assessed capability gaps in NII operations to assist with identifying potential upgrades to existing systems and developing requirements for future systems. DHS leadership approved a new NII Mission Needs Statement (MNS) in August 2018, which updated the capability gaps identified in the CAR and described mission needs and capabilities to address the gaps. The JRC endorsed the MNS, but recommended that CBP address cybersecurity threats and vulnerabilities as requirements and solutions evolve, and also include the Transportation Security Administration—which leverages some of the same equipment to perform their mission—in defining requirements, among other things. CBP officials told GAO that they are developing acquisition documentation to inform acquisition decision event 1 for the follow-on NII program planned for September 2019, including a concept of operations and an initial cost estimate.

CBP’s ability to successfully execute the existing NII Systems program and plan for future efforts may be at risk because of understaffing. As of September 2019, the program continued to face a staffing gap of approximately 21 percent. CBP officials said that they plan to mitigate the gap with government personnel from other offices within the component and with contractor support.

CBP officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

REMOTE VIDEO SURVEILLANCE SYSTEM (RVSS)

CUSTOMS AND BORDER PROTECTION (CBP)

RVSS helps the Border Patrol detect, track, identify, and classify illegal entries across U.S. borders. RVSS consists of daylight and infrared video cameras mounted on towers and buildings with communications systems that link to command and control centers. From 1995 to 2005, CBP deployed approximately 310 RVSS towers along the U.S. northern and southern borders, and initiated efforts to upgrade legacy RVSS towers in Arizona in 2011.

Diesel generators that power relocatable towers cause vibrations that could impact mission operations.

Once funded, program plans to award a contract for additional deployments along the southwest border.

GAO last reported on this program in May 2018 and November 2017 (GAO-18-339SP, GAO-18-119).

In April 2016, Department of Homeland Security (DHS) leadership elevated RVSS from a level 3 program—which focused on upgrading legacy RVSS in Arizona—to a level 1 program after approving CBP’s plan to expand deployments to the Rio Grande Valley (RGV) sector and adding an additional 6 sectors along the southwest border—Laredo, Del Rio, Big Bend, El Paso, El Centro, and San Diego. DHS leadership approved the program to move forward with deployments at two Border Patrol stations within the RGV, which can be completed as options under the program’s existing contract, if exercised. However, DHS leadership also directed the program to re-baseline to account for its expanded scope and conduct an acquisition decision event (ADE) 2A to obtain approval for additional deployments.

CBP officials previously told GAO the program anticipated conducting its ADE 2A and obtaining DHS leadership approval for an acquisition program baseline (APB) establishing cost, schedule and performance goals for the expanded program by December 2018. As of September 2019, the program had not yet received approval for key acquisition documents to conduct ADE 2A, including the APB, but CBP officials anticipate approval of these documents by March 2020. CBP officials primarily attribute these delays to a lack of funding for the additional deployments. CBP officials said the upcoming APB will include only deployments to Arizona and the RGV sector to align with funding received. Future deployments will require additional APB updates, which CBP officials said would be developed as funding becomes available.

In June 2019, the program updated its life-cycle cost estimate (LCCE) to inform the budget process. The updated LCCE included the expansion to the 6 sectors along the southwest border, relocatable RVSS towers, and operations and maintenance costs for previously fielded systems. However, CBP officials told GAO the LCCE is in the process of another update, which will inform the upcoming APB and include the expansion across additional sectors across southwest border and upgrades to legacy RVSS towers.

Customs and Border Protection (CBP)

REMOTE VIDEO SURVEILLANCE SYSTEM (RVSS)

CBP completed a pilot of five relocatable RVSS towers in June 2018, which included a comparison of vibration data measured on camera mounts for relocatable towers and fixed towers. The assessment showed that diesel generators used to recharge batteries in the relocatable towers caused significant vibrations, which caused cameras to shake and can affect operators’ ability to execute the mission. To address the issues stemming from the vibrations, CBP officials said they have connected the five relocatable towers to grid power when they are in use and plan to require solar power sources for future relocatable towers.

In July 2013, CBP awarded a firm fixed-price contract for a commercially available, non-developmental system. This contract covered the program’s initial scope to deploy upgraded RVSS in Arizona and included options for some initial work within the RGV sector. According to CBP officials, the program will need to award a new contract to cover expansion to the remaining six sectors along the southwest border. CBP officials drafted the request for proposals for the new contract, but it cannot be released until funding is received.

CBP officials said the program is experiencing challenges in the RGV sector related to land acquisition. The U.S. Army Corps of Engineers is leading efforts to acquire land for RVSS and other border security programs, including the Border Wall System Program (BWSP). CBP officials told GAO that the RVSS program is coordinating with BWSP on its planned deployments within the RGV sector. Program officials anticipate that some RVSS towers will be co-located within the border wall. In the interim, CBP officials said the program is using short-term agreements with landowners to place relocatable towers in areas where border wall construction is planned. These officials reported that the short-term agreements provide flexibility for the placement of towers and can be completed more quickly than permanent agreements.

CBP officials stated that the program’s current staffing plan was based on receiving funding for the expansion to RGV. Program officials said they will address the staffing needs once additional funding is received, but current operations have not been affected by the staffing gap.

CBP officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

CUSTOMS AND BORDER PROTECTION (CBP)

The TACCOM program is intended to upgrade land mobile radio infrastructure and equipment to support approximately 95,000 users at CBP and other federal agencies. It is replacing obsolete radio systems with modern digital systems across various sectors located in 19 different service areas, linking these service areas to one another through a nationwide network, and building new communications sites to expand coverage in five of the 19 service areas.

CBP officials reported that prior software issues have been addressed.

Program continues to face staffing challenges due to competition from the private sector, among other things.

GAO last reported on this program in May 2018 (GAO-18-339SP).

In September 2018, the TACCOM program achieved full operational capability (FOC)—nine months later than initially planned. However, in July 2018, the program’s operational test authority (OTA) conducted a survey of end users and concluded that there were still large gaps in coverage the TACCOM capabilities were intended to address. CBP officials stated that limited funding has affected the program’s ability to address the remaining gaps in coverage.

Department of Homeland Security (DHS) leadership previously approved a re-baseline of the TACCOM program in November 2017 after it experienced a schedule slip and cost growth. In July 2017, CBP officials notified DHS leadership that the program would not achieve FOC as planned due to issues related to federal information security requirements. In addition, the program experienced cost growth as a result of increased contractor labor costs and support for facilities and infrastructure.

In November 2017, DHS’s Chief Financial Officer (CFO) approved the program’s revised life-cycle cost estimate (LCCE). At that time, DHS‘s CFO noted that the program’s estimate exceeded its available funding and requested that the program address the affordability gap before it was re-baselined. Nevertheless, DHS leadership approved the program’s revised acquisition program baseline (APB). CBP officials subsequently identified errors in the approved APB cost threshold tables and provided revised amounts, which are presented here.

In September 2018, program officials told GAO that they completed an affordability analysis and submitted it to CBP and DHS leadership. CBP officials reported that the funding the program received in 2018 and carryover funds from prior years decreased the program’s affordability gap. However, CBP reported that in future years, funding gaps will require the program to reduce operations and maintenance requirements to match the appropriated funding and will continue to limit the program’s ability to address coverage gaps.

Customs and Border Protection (CBP)

In May 2014, DHS’s Director, Office of Test and Evaluation determined that the TACCOM systems were operationally effective, but test data were insufficient to determine operational suitability. The program’s OTA subsequently found that the TACCOM systems were operationally effective and suitable based on the results of an operational assessment (OA) completed in June 2016.

CBP officials told GAO that in January 2018, the program moved from a mission support office to a joint program office under Border Patrol as part of CBP’s reorganization. The goal of this move was to make CBP land mobile radio capabilities seamless by combining the mission critical voice functions within Air and Marine Operations, the Border Patrol, and the Office of Field Operations—the TACCOM program’s primary customers—under one organizational leader: the Border Patrol Chief.

In September 2018, CBP officials told GAO that the program reorganized staff within the program as it transitioned to an office under Border Patrol. CBP officials reported that hiring and retaining qualified land mobile radio engineers and information technology technical staff is a challenge because of lengthy hiring timeframes and competition with the private sector.

CBP officials stated that the TACCOM upgrades improved interoperability, coverage, capacity, reliability and encryption to provide critical communications support to the agents and officers who secure the Nation’s borders. The program continues to provide LMR System Maintenance to include operation, sustainment and performance monitoring to ensure reliable and consistent border protection communications. CBP officials also provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

CUSTOMS AND BORDER PROTECTION (CBP)

TECS (not an acronym) is a law-enforcement information system that has been in place since the 1980s and that helps CBP officials determine the admissibility of persons entering the United States at border crossings, ports of entry, and prescreening sites located abroad. CBP initiated efforts to modernize TECS to provide users with enhanced capabilities for accessing and managing data.

Costs increased by $400 million in revised cost estimate due to extended sustainment timeframe.

CBP working to address and prevent major system outages.

GAO last reported on this program in May 2018 (GAO-18-339SP).

Not included

Department of Homeland Security (DHS) leadership approved the fourth version of the program’s acquisition program baseline (APB) in July 2016. In this APB, CBP split full operational capability (FOC) into two separate operational capability milestones to better reflect the program’s activities at its primary and secondary data centers. CBP delivered operational capability at the primary data center and transitioned all remaining TECS users to the modernized system in December 2016. CBP delivered operational capability at the secondary data center in June 2017—as scheduled. This data center provides redundant TECS access to minimize downtime during system maintenance or unscheduled outages. However, not all test results were available in time for the program’s acquisition decision event (ADE) 3 decision. In August 2017, DHS leadership directed CBP to conduct follow-on operational test and evaluation (OT&E) activities to address known issues and conduct cybersecurity OT&E. The program completed follow-on OT&E in October 2018. DHS’s Director, Office of Test and Evaluation (DOT&E) completed an assessment of the test results in June 2019—which is intended to inform acquisition decisions.

In June 2019, the program’s annual life-cycle cost estimate (LCCE) was updated in accordance with DHS’s guidance to include operations and maintenance (O&M) costs for 10 years past the program’s planned FOC date. The updated LCCE includes program costs through fiscal year 2028—7 years longer than the prior LCCE and the program’s current APB cost goals. However, the LCCE update does not include estimated costs for all program plans, such as migrating the data centers to a cloud infrastructure. CBP officials plan to incorporate these costs into future LCCE updates when requirements are better defined. The program’s O&M costs increased and exceeded the program’s APB O&M cost threshold by approximately $400 million. DHS officials stated that the additional O&M costs do not constitute a cost breach because the program is considered to be in O&M phase of the acquisition life cycle.

Customs and Border Protection (CBP)

DOT&E found similar results for operational effectiveness and operational suitability during OT&E in July 2017, but tests were not adequate to assess operational cybersecurity. The test results validated that the program had met all eight of its key performance parameters (KPP), but the test team identified several deficiencies related to mission support. In response, DOT&E recommended that CBP conduct a threat assessment, threat-based cybersecurity operational testing, and follow-on OT&E. DHS leadership directed the program to complete these actions by February 2018, but this testing was not completed until October 2018. CBP officials attributed the delays to a lack of understanding of the level of effort required to draft the OT&E plan and supporting documents.

Since the program has completed development, CBP is focused on ensuring that the modernized TECS system works as intended by addressing operational issues as they are identified. For example, in January 2017, TECS Modernization experienced a major outage that resulted in airport delays. CBP officials previously said that they continually monitor system health through a 24/7 operations center and have established a group dedicated to address system issues.

In November 2017, DHS’s Office of Inspector General (OIG) found that CBP took sufficient steps to resolve the January 2017 outage, but underlying issues could result in future outages, including inadequate software capacity testing and deficient software maintenance. The OIG made five recommendations for CBP to implement improvements. CBP concurred with four of the recommendations but did not concur with a recommendation regarding CBP’s need to ensure staff make timely notifications of critical vulnerabilities to operating systems. CBP reported that the program’s notification activities were within DHS’s vulnerability management policy windows for testing and deploying software patches that were not deemed critical.

Further, in September 2017, the DHS OIG found that nearly 100 outages, periods of latency, or instances of degraded service, were reported for TECS Modernization applications between June 2016 and March 2017, and recommended that CBP develop a plan to address factors that contributed to these challenges. CBP concurred with the recommendations.

CBP officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

CONTINUOUS DIAGNOSTICS AND MITIGATION (CDM)

CYBERSECURITY AND INFRASTRUCTURE SECURITY AGENCY (CISA)

The CDM program aims to strengthen cybersecurity of the federal government’s networks by continually monitoring and reporting vulnerabilities at more than 65 civilian agencies. CDM provides four capabilities: Asset Management reports vulnerabilities in hardware and software; Identity and Access Management focuses on user access controls; Network Security Management will report on efforts to prevent attacks; and Data Protection Management will provide encryption to protect network data.

Program revised its key performance parameters to better align with cybersecurity standards.

The program began using a new contract vehicle and is hiring additional staff to support new capabilities.

GAO last reported on this program in May 2018 (GAO-18-339SP).

According to CISA officials, as a result of the 2019 partial government shutdown, the program experienced delays that impacted the program’s ability to achieve initial operational capability (IOC) for Identity and Access Management and Network Security Management capabilities as planned. In response, Department of Homeland Security (DHS) leadership approved a 3-month extension to both milestones. As a result, the IOC threshold date for Identity and Access Management capabilities was extended to and later achieved in June 2019. The IOC threshold date for Network Security Management was extended to December 2019.

The program updated its life-cycle cost estimate (LCCE) in April 2019 to inform the budget process. This estimate exceeds the program’s current operations and maintenance (O&M) and total life-cycle cost thresholds by approximately $300 million and $100 million, respectively. The program’s cost increase is primarily attributed to evolving requirements described in the explanatory statements accompanying recent Appropriations Acts and the Office of Management and Budget (OMB). Specifically, CISA officials said the program received $110 million above the Presidential Budget Request and noted this was to accelerate procurement of CDM capabilities for additional agencies not in the original program scope and accelerate mobile and cloud computing visibility across the .gov domain, among other things. In addition, the program received funding in 2018 and 2019 after OMB directed that the CDM program cover certain costs of sustaining licenses for supported agencies, which CISA officials estimate will cost the program an additional $62 million. The program also estimates that O&M costs for these additional requirements will require a total of an additional $79 million in future years.

In May 2019, CISA officials said the program is updating key acquisition documentation, such as its acquisition program baseline (APB) and LCCE, to inform acquisition decision event (ADE) 2B for Data Management Protection capabilities. They noted that the updated acquisition documents will account for the increased demand for CDM services. The program previously planned to achieve this ADE 2B by March 2019. However, due in part to the partial government shutdown, the program now plans to achieve the ADE 2B in 2020.

Cybersecurity and Infrastructure Security Agency (CISA) CONTINUOUS DIAGNOSTICS AND MITIGATION (CDM)

The CDM program is only authorized to conduct testing on DHS networks, which means the other departments and agencies are responsible for testing the CDM tools on their own networks. CISA officials reported that four other agencies have either conducted or plan to conduct operational studies, which provided the program with informal observations on implementation and was used to support IOC for the Identity and Access Management capability. Under the program’s revised test and evaluation master plan, the OTA plans to perform operational assessments (OA) on DHS’s network to incrementally demonstrate each capability as it is deployed and to reduce risk prior to conducting formal program-level operational test and evaluation. Specifically, the program completed an OA for the Identity and Access Management capability and expected the letter of assessment from DOT&E by June 2019. In addition, the program expects to begin a technology assessment for the Data Protection Management capability by September 2019.

The CDM program updated its acquisition plan to reflect a change in strategy for procuring CDM tools and services. Previously, the program used blanket purchase agreements established by the General Services Administration (GSA) Federal Supply Schedule. CISA officials told GAO that in February 2018 the program began using an existing GSA government-wide acquisition contract and as of August 2019, the program has awarded 5 of 6 planned task orders to obtain CDM tools and services on behalf of participating agencies. According to CISA officials, the new acquisition strategy is intended to provide greater flexibility in contracting for current capabilities and to support future capabilities. Participating agencies will also be able to order additional CDM-approved products or services from GSA’s schedule for information technology equipment, software, and services.

The program previously used the term “phases” and renamed the phases in the fall of 2018 to align with the associated capabilities it deploys. CISA officials explained that a phased deployment implied a serial implementation; however, CDM capabilities can be deployed in parallel.

The program is not currently experiencing workforce challenges. The program received approval for 29 new positions to address staffing needs for the Network Security Management and Data Protection Management capabilities. Officials plan to fill those positions in fiscal years 2019 and 2020.

CISA officials stated that in addition to efforts identified in this assessment, the program continues to manage its budget to ensure program costs match available funding and is leveraging the collective buying power of federal agencies and strategic sourcing to continue achieving government cost savings on CDM products. CISA officials also provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

NATIONAL CYBERSECURITY PROTECTION SYSTEM (NCPS)

CYBERSECURITY AND INFRASTRUCTURE SECURITY AGENCY (CISA)

NCPS is intended to defend the federal civilian government from cyber threats. NCPS develops and delivers capabilities through a series of “blocks.” Blocks 1.0, 2.0, and 2.1 are fully deployed and provide intrusion-detection and analytic capabilities across the government. The NCPS program is currently deploying EINSTEIN 3 Accelerated (E3A) to provide intrusion-prevention capabilities and plans to deliver block 2.2 to improve information sharing across agencies.

Program capabilities determined to be operationally suitable, effective, and cyber resilient with limitations.

Staffing challenges may impact program execution.

GAO last reported on this program in May 2018 (GAO-18-339SP).

In February 2018, the Department of Homeland Security’s (DHS) Under Secretary for Management (USM) granted NCPS acquisition decision event (ADE) 3 approval for E3A to transition to sustainment and ADE 2C approval for block 2.2 to deploy additional capabilities. DHS’s USM also directed NCPS to address several issues identified during test events that informed the ADEs, including the following:

For EA—Conduct follow-on operational test and evaluation (OT&E) by March

For block 2.2—Review the operational requirements document (ORD) and concept 2019 to assess cybersecurity, among other things. of operations (CONOPS) to ensure they accurately reflect the mission environment and processes, review current and planned capabilities to ensure they will adequately address the ORD and CONOPS, and conduct another operational assessment (OA) prior to initial OT&E.

The program revised its acquisition program baseline (APB) in January 2018 in preparation for the ADEs. However, the program updated its APB again in October 2018 to address an error found in the life-cycle cost estimate (LCCE), to add an additional 2 years of program costs, and to revise the approach to estimating threshold costs. Specifically, the LCCE that provided the basis for the program’s APB cost goals did not accurately account for the program’s sunk costs. Once corrected, the program’s total life-cycle cost threshold was $5.9 billion—more than $1.7 billion more than in the program’s January 2018 APB. CISA officials reported that while correcting the sunk costs increased the APB cost goals, the change did not affect estimating future costs and, therefore, will not impact program affordability. In March 2019, to inform the budget process, the program updated its corrected LCCE—which is within its current APB cost goals.

In the program’s January 2018 APB, the ADE 3 date for block 2.2 slipped by 2 years— from March 2019 to March 2021—compared to its prior APB. According to CISA officials, this milestone was revised due to bid-protest-related delays involving the award of the program’s development, operations, and maintenance contract. CISA officials said that due to several protests, the award was delayed until June 2018— nearly 3 years later than planned.

Cybersecurity and Infrastructure Security Agency (CISA)

NATIONAL CYBERSECURITY PROTECTION SYSTEM (NCPS)

A, which included an assessment of cyber resilience for only one of the program’s three internet service providers. In June 2019, DOT&E determined EA effectiveness by integrating automated information sharing solutions and data analysis tools, among other things. In June 2019, CISA officials stated they were working on enhancements to address E

In January 2018, DOT&E determined that it was too soon to assess block 2.2 based on the OA results from October 2017, but noted block 2.2 was at risk of not meeting user needs and made a number of recommendations, including reviewing the ORD and CONOPS and repeating the OA before conducting initial OT&E. CISA officials told GAO that the operator’s processes had changed since the initial ORD and CONOPS were approved. These officials said they plan to revise these documents before conducting another OA in fiscal year 2020.

A intrusion-prevention capabilities have been primarily provided through sole source contracts with internet service providers and a contract to provide basic intrusion-prevention services. In December 2015, Congress required DHS to make available for use by federal civilian agencies, certain capabilities, such as those provided by NCPS’s EA at approximately 93 percent of federal civilian agencies and departments and, in October 2018, CISA officials reported that NCPS was up to 95 percent, with mainly small and micro organizations remaining.

CISA officials said they are working with the various agencies to migrate agency email to a cloud environment, but each department and agency requires a unique solution and coordination can be a challenge.

In April 2019, CISA officials reported that if the program’s staffing gap is not addressed, the program may experience a delay in meeting mission requirements. CISA officials told GAO that the federal hiring process and DHS’s lengthy suitability screening process have made recruitment efforts challenging because qualified candidates often find other employment while waiting for these processes to be completed. In addition, CISA officials anticipate workforce challenges if, in the future, they are not able to use compensation flexibility for cybersecurity specialists.

CISA officials reviewed a draft of this assessment and provided no comments.

NEXT GENERATION NETWORKS PRIORITY SERVICES (NGN-PS)

CYBERSECURITY AND INFRASTRUCTURE SECURITY AGENCY (CISA)

NGN-PS is intended to address an emerging capability gap in the government’s emergency telecommunications service, which prioritizes phone calls for select officials when networks are overwhelmed. CISA executes NGN-PS through commercial telecommunications service providers, which addresses the government’s requirements, as they modernize their own networks.

Full operational capability for wireless capabilities delayed by 3 years to incorporate design changes in network.

New program for acquisition of data and video capabilities to begin in fiscal year 2020.

GAO last reported on this program in May 2018 (GAO-18-339SP).

The NGN-PS program is developing and delivering prioritized voice capability in three increments: increment 1 maintains current priority service on long distance calls as commercial service providers update their networks; increment 2 delivers wireless capabilities; and increment 3 is intended to address landline capabilities.

In October 2018, Department of Homeland Security (DHS) leadership granted the NGN-PS program acquisition decision event (ADE) 3 for increment 1. At that time, the program also declared full operational capability (FOC) for increment 1. Once operational, capabilities acquired by NGN-PS are transferred to CISA’s Priority Telecommunications Service program.

In April 2018, DHS leadership approved a revised acquisition program baseline (APB) for NGN-PS and subsequently authorized the program to initiate development of increment 3. The previous APB included only costs and schedule milestones associated with increments 1 and 2. The revised APB modified the program’s cost and schedule goals to include goals for increment 3 and updates to cost goals previously established for increments 1 and 2. Specifically, the program’s total acquisition cost threshold increased by $68 million. This change reflects $144 million in additional costs to develop landline capabilities and a cost savings of approximately $100 million on previous increments, among other things. Program officials primarily attributed the cost savings on increment 1 to design changes implemented by a commercial service provider within its network. In addition, according to program officials, the increment 2 FOC goal was revised in the updated APB to allow additional time for a commercial service provider to incorporate design changes into its network. As a result, the FOC date for increment 2 slipped 3 years to December 2022. The program plans to achieve FOC for increment 3 in December 2025.

The program updated its life-cycle cost estimate (LCCE) in February 2019. The updated LCCE includes operations and maintenance (O&M) costs, although the APB does not. Officials said this is not considered a breach because the O&M costs include staffing outside of O&M phase activities.

Cybersecurity and Infrastructure Security Agency (CISA)

NEXT GENERATION NETWORKS PRIORITY SERVICES (NGN-PS)

NGN-PS capabilities are evaluated through developmental testing and operational assessments conducted by service providers on their own networks. CISA officials review the service providers’ test plans, oversee tests to verify testing procedures are followed, and approve test results to determine when testing is complete. The OTA then leverages the service providers’ test and actual operational data to assess program performance. In addition, CISA officials said that they continuously review actual NGN-PS performance and service providers undergo annual network service verification testing under the Priority Telecommunications Service program.

In October 2018, DHS leadership approved the separation of the development of capabilities for data and video priority services into a new acquisition program. DHS leadership approved the decision because data and video capabilities are different than landline priority, and the addition of these capabilities would significantly extend the expected end date of the NGN-PS program. CISA officials anticipate establishing a preliminary baseline for the data and video capabilities in early fiscal year 2020.

NGN-PS was established in response to an Executive Order requiring the federal government to have the ability to communicate at all times during all circumstances to address national security issues and manage emergencies. A Presidential Policy Directive issued in July 2016 superseded previous directives requiring continuous communication services for select government officials. According to CISA officials, the new directive validates requirements for the voice phase and was used to develop requirements for the data and video phase.

In May 2019, the program reported four critical staffing vacancies, including two new positions. The program reported that it continues to have difficulty filling a systems engineer billet, which program officials attribute to the lengthy federal hiring process, DHS’s suitability screening process, and the fiscal year 2019 partial government shutdown. To mitigate the impact of the staffing gap on program execution, the program leverages contract support and staff from the Priority Telecommunications Service program.

In addition to activities identified in this assessment, CISA officials stated that the program will continue planning for data and video priority in future budget years. CISA officials also said that service providers undergo annual network service verification testing and that the program is currently making progress in hiring for numerous positions. CISA officials also provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

HOMELAND ADVANCED RECOGNITION TECHNOLOGY (HART)

HART will replace and modernize DHS’s legacy biometric identification system—known as IDENT—which shares information on foreign nationals with U.S. government and foreign partners to facilitate legitimate travel, trade, and immigration. The program plans to develop capabilities in four increments: increments 1 and 2 will replace and enhance IDENT functionality; increments 3 and 4 will provide additional biometric services, as well as a web portal and new tools for analysis and reporting.

Program updated operational requirements document and revised its key performance parameters.

Program is taking steps to address challenges as a result of a shortfall in staff with technical skillsets.

GAO last reported on this program in May 2018 (GAO-18-339SP).

In May 2019, DHS leadership approved a revised acquisition program baseline (APB) for the HART program, removing it from breach status, after the program experienced a schedule slip in June 2017. Specifically, the HART program declared a schedule breach when officials determined the program would not be able to meet its initial APB milestones. HART officials attributed the schedule slip to multiple delays in awarding the contract for increments 1 and 2 and a subsequent bid protest—which GAO denied.

The program initiated work with the contractor in March 2018 and revised key acquisition documents, including its acquisition program baseline (APB) and life-cycle cost estimate (LCCE), to reflect program changes. For example, officials revised these documents to account for schedule delays and the contractor’s solution for enhanced biometric data storage. Specifically, the contractor plans to deliver services using a cloud-based solution rather than through DHS’s data centers. The HART performance work statement shows delivering services through the cloud provides greater flexibility to scale infrastructure supporting services at a lower cost.

The program’s initial operational capability (IOC) date—when all customers will transition from using IDENT to HART—slipped 2 years to December 2020. This is a significant challenge because IDENT is at risk of failure and additional investments are necessary to keep the system operational. HART’s full operational capability (FOC) date—when the program plans to deploy enhancements of biometric services and new tools for analysis and reporting—slipped nearly 3 years to June 2024.

HART’s total APB cost thresholds decreased by approximately $2 billion, which officials primarily attribute to the less expensive cloud-based solution and removal of IDENT upgrade costs, among other things. However, officials identified a risk that costs associated with the cloud-based solution could increase because technical requirements were not fully developed when the LCCE informing the revised APB was developed. As a result, HART is at risk for a future cost breach once these technical requirements are better defined. The affordability surplus from fiscal years 2020 through 2024 may be overstated because, according to officials, projected funding covers both IDENT and HART.

HOMELAND ADVANCED RECOGNITION TECHNOLOGY (HART)

The program updated its operational requirements document in May 2019 to support the program’s re-baseline and revised its eight key performance parameters (KPP) to address evolving DHS biometric requirements. Specifically, the KPPs for increment 1 establish requirements for system availability and a fingerprint biometric identification service. The program added a KPP for increment 1 to address fingerprint search accuracy. Increment 2 KPPs establish requirements for multimodal biometric verification services and interoperability with a Department of Justice system. The program adjusted a KPP for multimodal biometric verification to address iris search accuracy. Increments 3 and 4 KPPs establish requirements for web portal response time and reporting capabilities.

DHS’s Science and Technology Directorate’s (S&T) Office of Systems Engineering completed a technical assessment on HART in February 2016 and concluded that the program had a moderate overall level of technical risk. In October 2016, DHS leadership directed HART to work with S&T to conduct further analysis. In March 2019, S&T updated risks identified in the technical assessment and evaluated the program’s scalability, availability, cybersecurity, and performance modeling risks for the HART system. S&T made several recommendations for the program to consider as it addresses identified risks. S&T will continue to work with the program to address technical and operational challenges.

In April 2019, following the passage of the Cybersecurity and Infrastructure Security Agency (CISA) Act of 2018, the transfer of CISA’s Office of Biometric Identity Management (OBIM)—which includes the HART program—to DHS’s Management Directorate was implemented. The transfer was informed by a working group including OBIM, DHS’s MGMT, and CISA subject matter experts.

In June 2019, HART officials told GAO they are currently planning for increments 3 and 4, which will provide new and enhanced capabilities, analytics, and reporting, and additional biometric modalities and services, among other things. In June 2019, HART officials released a request for information for increments 3 and 4, which will inform the program’s acquisition plan and statement of work for a request for proposal.

At the direction of DHS leadership, HART program officials coordinated with DHS’s Chief Technology Officer to assess the skills and functions of staff necessary to execute the program. In its August 2019 staffing plan, the program reported workforce risks, including a potential shortfall in staff with technical skillsets; however, officials stated that they are mitigating the shortfall, in part, by providing training activities for current staff. In June 2019, HART officials noted that the federal hiring process and DHS’s lengthy security clearance process have made recruitment efforts challenging.

HART officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

LOGISTICS SUPPLY CHAIN MANAGEMENT SYSTEM (LSCMS)

FEDERAL EMERGENCY MANAGEMENT AGENCY (FEMA)

LSCMS is a computer-based tracking system that FEMA officials use to track shipments during disaster-response efforts. It is largely based on commercial-off- the-shelf software. FEMA initially deployed LSCMS in 2005, and initiated efforts to enhance the system in 2009. According to FEMA officials, LSCMS can identify when a shipment leaves a warehouse and the location of a shipment after it reaches a FEMA staging area near a disaster location.

LSCMS found operationally effective and operationally suitable with limitations, but not cyber secure.

Program transitioned to cloud data storage and plans to conduct annual cybersecurity testing.

GAO last reported on this program in May 2018 (GAO-18- 339SP).

In September 2019, Department of Homeland Security (DHS) leadership granted the program approval of acquisition decision event (ADE) 3 and acknowledged the program’s achievement of full operational capability (FOC). DHS leadership previously denied the program’s request for acquisition decision event ADE 3 and FOC approval until issues with the system’s backup server were resolved. Program officials reported that the program addressed these issues in August 2019.

In November 2017, DHS leadership approved a revised acquisition program baseline (APB) after the LSCMS program experienced a schedule slip because of the 2017 hurricane season. FEMA officials said the need to deploy LSCMS personnel in support of response and recovery efforts during multiple hurricanes—Harvey, Irma, and Maria—jeopardized the program’s ability to complete all required activities as planned. Specifically, the program was unable to complete follow-on operational test and evaluation (OT&E) to achieve ADE 3 and FOC by its initially planned APB dates of September 2018 and December 2018, respectively. The program was able to retain most of its initial schedule by working with its operational test agent (OTA) to adjust the follow-on OT&E plan, which significantly reduced the scope of dedicated testing needed to complete follow-on OT&E. Specifically, the OTA collected operational data during the 2017 hurricane response efforts, which allowed them to assess approximately two-thirds of the performance measures required for follow-on OT&E.

In December 2018, the program updated its life-cycle cost estimate (LCCE), which is within the program’s APB cost thresholds. The program’s operations and maintenance (O&M) costs decreased in part because the program plans to transition LSCMS data storage from a physical facility to a cloud environment. The updated LCCE also estimates costs for conducting technology refreshes annually instead of every 5 years, which FEMA officials said will make the program’s future funding needs more stable as the program moves into sustainment.

Federal Emergency Management Agency (FEMA) LOGISTICS SUPPLY CHAIN MANAGEMENT SYSTEM (LSCMS)

Officials reported that in August 2019 the program migrated to the cloud—resolving a majority of the program’s cybersecurity issues. Officials reported that remaining system and enterprise issues will be resolved in September 2020, when the program plans to conduct annual cybersecurity testing.

The LSCMS program previously experienced significant execution challenges because of prior poor governance. FEMA initially deployed the enhanced LSCMS in 2013 without DHS leadership approval, a DOT&E letter of assessment, or a DHS-approved APB documenting the program’s costs, schedule, and performance parameters, as required by DHS’s acquisition policy. DHS’s Office of Inspector General also found that neither DHS nor FEMA leadership ensured the program office identified all mission needs before selecting a solution. In response, DHS leadership paused all LSCMS development efforts in April 2014 until the program addressed these issues, among others. FEMA subsequently completed an analysis of alternatives and developed an APB based on this assessment. DHS leadership approved the program’s initial APB in December 2015 and authorized FEMA to resume all LSCMS development and acquisition efforts in March 2016.

In July 2019, FEMA reported that the program had initiated the hiring process for its vacant positions. In July 2019, FEMA officials told GAO one of the positions had already been filled. According to FEMA officials, the program revised its methodology for completing its most recent staffing profile to reflect the current and future staffing needs of the program. FEMA officials said that the current staffing levels will not change significantly after the program achieves FOC, as there will be a continued need for regular updates to the system.

FEMA officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

NATIONAL BIO AND AGRO-DEFENSE FACILITY (NBAF)

SCIENCE AND TECHNOLOGY DIRECTORATE (S&T)

The NBAF program is constructing a state-of-the-art laboratory in Manhattan, Kansas to replace the Plum Island Animal Disease Center. The facility will enable the Department of Homeland Security (DHS) and the Department of Agriculture (USDA) to conduct research, develop vaccines, and provide enhanced diagnostic capabilities to protect against foreign animal, emerging, and zoonotic diseases that threaten the nation’s food supply, agricultural economy, and public health.

Program is on track to meet May 2021 initial operational capability date.

DHS and USDA have developed a transition plan and are coordinating on commissioning efforts.

GAO last reported on this program in May 2018 (GAO-18-339SP).

Not included

The NBAF program was originally planned be a joint operation between DHS and USDA, with DHS taking the lead on construction and operation of the facility. However, the President’s budget request for fiscal year 2019 proposed transferring operational responsibility for NBAF, which includes operational planning and future facility operations, to USDA. In the Joint Explanatory Statement for the Consolidated Appropriations Act of 2018, congressional conferees specified that DHS would retain responsibility for completing construction of NBAF. As a result, DHS will continue to oversee and manage activities required to complete construction and achieve initial operational capability (IOC), which is facility commissioning. USDA will then be responsible for achieving full operational capability (FOC), including operational stand- up of the facility and all subsequent operations. The program’s acquisition program baseline (APB) has not yet been updated to reflect the change in responsibility for achieving FOC and to remove operational costs, which will now be budgeted for by USDA. NBAF officials said the transition introduces cost and schedule risks to the program because highly integrated activities—such as commissioning and operational stand-up—are now being managed by two different agencies, but DHS and USDA will continue to coordinate through the transition process.

NBAF officials told GAO that construction activities thus far—such as pouring concrete for the main laboratory—have proceeded as anticipated and the program is on track to meet its APB cost and schedule goals through IOC, planned for May 2021.

According to NBAF officials, the program has already received full acquisition funding for the facility construction efforts through federal appropriations and gift funds from the state of Kansas. The program previously planned to use operations and maintenance funding to support operational stand-up activities and awarded a contract for operational planning. However, beginning in fiscal year 2019, DHS will no longer request operations and maintenance funding for NBAF, as all such funding and activities will be the responsibility of USDA. Congressional conferees noted that $42 million in funding to USDA is to address operational stand-up activities and other initial costs to operate and maintain the facility. The Consolidated Appropriations Act of 2019 also authorized DHS to transfer personnel and up to $15 million in certain funds to USDA for contracts and associated support of the operations of NBAF.

Science and Technology Directorate (S&T) NATIONAL BIO AND AGRO-DEFENSE FACILITY (NBAF)

According to NBAF officials, the program has implemented a commissioning process for the facility to determine whether it can meet its sole key performance parameter (KPP) for laboratory spaces that meet various biosafety standards. NBAF officials said that DHS and USDA have been in coordination throughout the commissioning process. A third-party commissioning agent has been retained as a subcontractor to the prime construction management contractor, and NBAF officials said that the commissioning plan has been in place since 2012. According to NBAF officials, the commissioning agent worked with the facility design and construction teams to develop the commissioning plan, and detailed procedures are in place to install and commission equipment in the facility. The commissioning agent will monitor and test the facility’s equipment and building systems while construction is ongoing to ensure they are properly installed and functioning according to appropriate biosafety specifications.

NBAF officials reported that they are coordinating with USDA officials, the commissioning agent, and federal regulators responsible for awarding the registrations needed for NBAF to conduct laboratory operations to determine how the final commissioning report will be structured to support FOC and federal certification to begin laboratory operations.

In June 2019, DHS and USDA signed a memorandum of agreement that established plans to transfer NBAF operational responsibility from DHS to USDA. The memorandum establishes responsibilities related to costs and funding, requirements for establishing NBAF, and considerations for interagency coordination once NBAF is operational, among other things. For example, some USDA staff will participate in the NBAF commissioning process, but they will be integrated with DHS’s onsite construction oversight team to maintain the integrity of DHS’s existing oversight approach for the NBAF construction/ commissioning contract. The memorandum of agreement also states that DHS, in consultation with USDA, will plan for the appropriate timing and necessary mechanism to transfer identified DHS employees to USDA for NBAF activities. According to NBAF officials, DHS plans to transfer staff from both the Plum Island Animal Disease Center and the program’s on-site construction oversight team to USDA to preserve institutional knowledge. USDA was appropriated $3 million in the Consolidated Appropriations Act of 2018 to begin hiring NBAF operational staff and the memorandum of agreement notes that USDA will work with DHS to increase staffing in fiscal year 2019 as required by the construction commissioning schedule.

In April 2019, the program’s staffing assessment was updated to reflect program needs from fiscal year 2019 through IOC. At that time, the NBAF officials reported that the program is fully staffed.

NBAF officials reviewed a draft of this assessment and provided no comments.

ADVANCED TECHNOLOGY (AT)

TRANSPORTATION SECURITY ADMINISTRATION (TSA)

The AT Program supports the checkpoint screening capability by providing capability to detect threats in the passenger’s carry-on baggage, including explosives, weapons, and other prohibited items. The AT-1 and AT-2 X-ray systems screen carry-on baggage providing threat detection capabilities for a wide range of threats. AT-2 Tier I and Tier II systems provide enhanced detection capabilities and improved image resolution. Computed technology (CT)—which offers enhanced three-dimensional imaging and detection capabilities over the currently deployed AT system—is also being procured through AT program.

Both AT and CT units have experienced challenges achieving performance goals.

Procurement and deployment of CT units will transfer to Checkpoint Property Screening System program.

GAO last reported on AT as a part of the Passenger Screening Program in May 2018 (GAO-18- 339SP).

In February 2018, Department of Homeland Security (DHS) leadership approved transitioning existing Passenger Screening Program (PSP) projects—including AT—into stand-alone programs to better align program office staffing to capabilities and focus on mitigating capability gaps, among other things. In fiscal year 2018, TSA determined that CT is the best technology available to address rapidly evolving threats in the transportation sector. As a result, TSA determined it would leverage the AT program to initiate the acquisition of CT systems.

In December 2018, DHS leadership approved an acquisition program baseline (APB) for AT as a standalone program, which included cost and schedule goals for AT and CT that were presented separately. For AT, fiscal year 2018 and prior year costs were not included in the APB cost goals because those costs are considered sunk costs for PSP. AT does not have any acquisition costs because full operational capability for AT was achieved in 2016 under PSP. AT’s operations and maintenance (O&M) costs— which total $590 million—are related to maintaining AT-1 and AT-2 X-ray systems and incorporating upgrades to enhance detection capability and increase passenger volume through AT-2 Tier I and Tier II systems. When DHS leadership approved the APB, they also approved the acquisition decision event (ADE) 3—authorizing the procurement of CT units in fiscal year 2019 only. The APB includes acquisition costs for the fiscal year 2019 procurements but it does not identify any O&M costs for CT.

In March 2019, DHS leadership acknowledged the AT program’s ADE 3 for AT-2 Tier II. The program previously achieved full operational capability (FOC) for AT-2, but ADE 3 was not achieved primarily because one the program’s key performance parameters (KPP) needed to be refined.

The AT program’s surplus from fiscal years 2020-2024 may be overstated in DHS’s funding plan to Congress because costs associated with CT were not previously included in the AT cost estimate. However, the AT and CT costs in the affordability assessment are combined here. The purchase of CT units will become a separate acquisition for the fiscal year 2021 programming and budget cycle with an updated cost estimate.

Transportation Security Administration (TSA) ADVANCED TECHNOLOGY (AT)

In September 2018, the OTA completed certification, qualification and operational test and evaluation (OT&E) on CT systems from four different vendors. DHS’s Director, Office of Test and Evaluation (DOT&E) assessed the results in November 2018 and found that the systems from all four vendors did not meet the KPP related to throughput and the systems from two vendors also did not meet the KPP related to availability. Further, DOT&E rated the systems from the 4 vendors as operationally effective and operationally suitable with limitations. Cyber resiliency was not assessed. DOT&E recommended that TSA validate requirements, refine KPPs specific to the CT systems, and develop a plan to address cyber resilience issues prior to future deployment of networked systems, among other things.

In August 2019, TSA officials said AT systems meet all four of the program’s KPPs. In September 2018, DOT&E reassessed the August 2016 follow-on operational test and evaluation (OT&E) results from AT-2 Tier II based on the program’s revised KPP for throughput—which contributed to DOT&E’s prior effectiveness rating. DOT&E confirmed that the system now meets the revised requirement based on a re- assessment of the test data against the new definition, but did not change the rating.

TSA intends to transition the procurement and deployment of CT units, among other things, to the Checkpoint Property Screening System (CPSS), which, as of August 2019, had not yet been established. CPSS is a separate acquisition program that is intended to address capability gaps in passenger screening technologies. Through CPSS, TSA plans to eventually deploy CT to all checkpoints and replace AT X-ray technology.

According to TSA officials, Automated Screening Lane (ASL) technologies have been managed by the AT program since March 2019. TSA is not incurring acquisition costs for ASLs, but the source of funding for O&M costs is unclear. DHS leadership directed TSA to begin tracking ASL maintenance and repairs to inform future budget requests, among other things.

TSA officials stated that one of the program’s vacant positions has not yet been funded. To mitigate the staffing gap, TSA officials stated they are disbursing tasks among existing staff until the position is filled.

TSA officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

CREDENTIAL AUTHENTICATION TECHNOLOGY (CAT)

TRANSPORTATION SECURITY ADMINISTRATION (TSA)

The CAT system is used to verify and validate passenger travel and identification documents prior to entering secure areas in airports. CAT reads data and security features embedded in identification documentation (ID), verifies security features are correct, and displays authentication results to the operator. The CAT system also verifies the passenger has the appropriate flight reservation to progress through security screening and enter the secure area, among other things.

Program met its key performance parameters, but needs to address cyber resiliency and other issues.

CAT system will require regular updates to address changes to state identification documentation.

GAO last reported on CAT as part of the Passenger Screening Program in May 2018. (GAO-18-339SP).

Not included

In February 2018, the Department of Homeland Security (DHS) approved transitioning existing Passenger Screening Program (PSP) projects, including CAT, into stand-alone programs to better align program office staffing to capabilities and focus on mitigating capability gaps, among other things. In December 2018, DHS leadership approved an acquisition program baseline (APB) for CAT as a stand-alone program. The APB reflected a revised testing and deployment strategy. Specifically, TSA no longer intends to pursue separate deployments of CAT for TSA Pre® and standard lanes. TSA concluded that the separate approach would extend the overall schedule to deploy CAT units to the field and was an inefficient use of resources.

In February 2019, DHS leadership granted the program acquisition decision event (ADE) 3 for procurement and deployment of CAT units and acknowledged the program’s initial operational capability (IOC) based on the fielded units. TSA now plans to achieve full operational capability (FOC) in September 2022—more than 1 year earlier than previously planned for standard lanes, but 8 years later than initially planned under PSP. According to TSA officials, the program recently accelerated its deployment schedule to meet existing and emerging threats.

The program developed an initial life-cycle cost estimate (LCCE) to inform the APB and ADE 3 and updated the estimate in June 2019 to inform the budget process. The program’s June 2019 LCCE reflects an O&M cost decrease of over $80 million, which TSA officials attribute to a reduction in enhancements needed to accelerate deployments.

The program was not included in DHS’s funding plan to Congress for fiscal years 2020-2024 because the program is no longer expected to receive acquisition funding. TSA officials stated that they are working with TSA’s Chief Financial Officer and the CAT vendor to identify and mitigate any funding issues that may arise as the program moves into production.

Transportation Security Administration (TSA) CREDENTIAL AUTHENTICATION TECHNOLOGY (CAT)

DOT&E recommended that the program work with the vendor to improve the authentication rate of IDs, revise its KPP related to availability, conduct a study to understand passenger throughput and update throughput requirements accordingly, and conduct follow-on OT&E, among other things. In July 2019, TSA officials told GAO the program plans to conduct additional cyber resiliency testing and follow-on OT&E once requirements are refined.

TSA officials stated that CAT is expected to be TSA’s primary identification verification method by the end of fiscal year 2019. However, TSA officials said the CAT system will require regular updates to address changes to state IDs. In November 2018, TSA officials reported that states are in the process of adopting new requirements identified in the REAL ID Act of 2005. Among other things, the Act establishes minimum security standards for ID issuance and production, and prohibits federal agencies from accepting IDs from states not meeting these standards unless the Secretary of Homeland Security has granted the issuing state an extension of time to meet the requirements. TSA officials said that the current manual process of verifying a passenger’s ID against their boarding pass will be used if CAT units are unavailable and between system updates.

In May 2019, the program reported two critical staffing vacancies. TSA officials reported that these positions have been filled.

TSA officials reviewed a draft of this assessment and provided no comments.

ELECTRONIC BAGGAGE SCREENING PROGRAM (EBSP)

TRANSPORTATION SECURITY ADMINISTRATION (TSA)

Established in response to the terrorist attacks of September 11, 2001, EBSP tests, procures, and deploys transportation security equipment, such as explosives trace detectors and explosives detection systems, across approximately 440 U.S. airports to ensure 100 percent of checked baggage is screened for explosives. EBSP is primarily focused on delivering new systems with enhanced screening capabilities and developing software upgrades for existing systems.

Follow-on testing completed in January 2019; initial results show improvement in effectiveness.

EBSP is pursuing a new procurement strategy for two types of detection systems.

GAO last reported on this program in May 2018 (GAO-18-339SP).

In August 2019, TSA declared a cost breach of EBSP’s current acquisition program baseline (APB) due to increased maintenance costs. The program previously revised its APB in May 2016 to account for budget reductions and to implement the program’s strategy to prioritize funding to extend the life of screening technologies, among other things. TSA has implemented these changes through ongoing maintenance and system upgrades, to include detection algorithm updates. DHS officials reported that this strategy has improved security effectiveness and operational efficiencies at a lower cost than replacing legacy systems with new systems. However, this approach increased the number of systems that are out-of-warranty and increased the maintenance needed to sustain these systems. This new strategy, coupled with increased maintenance activities, resulted in an operations and maintenance (O&M) cost increase exceeding the program’s APB O&M cost threshold. As of September 2019, the program’s revised APB, which TSA officials said will address the O&M cost increase, had not yet been approved.

In January 2018, DHS leadership approved the program’s request to deploy an explosives detection system with an advanced threat detection algorithm. TSA officials reported that they achieved initial operational capability (IOC) of these systems in February 2018; this is the program’s final APB milestone. TSA leadership subsequently approved the program to deploy detection algorithm updates to fielded systems.

Based on the program’s July 2019 life-cycle cost estimate (LCCE), the program is projected to face an acquisition funding gap of $29 million over the 5-year period. However, the program’s total projected funding gap, including O&M, is expected to be approximately $223 million. TSA officials told GAO that one of their primary challenges is funding, and that to mitigate anticipated funding gaps, the program may shift other projects from one fiscal year to another or cancel them altogether.

Transportation Security Administration (TSA) ELECTRONIC BAGGAGE SCREENING PROGRAM (EBSP)

Since March 2011, DHS’s Director, Office of Test and Evaluation (DOT&E) has assessed the operational test and evaluation results of 11 EBSP systems and determined that six are effective and suitable. Most recently, DOT&E found that a medium speed explosives detection system with an advanced threat detection algorithm tested in May 2017 was effective with limitations and not suitable, primarily because of the increase in manpower needed to operate the system on a long-term, continuous basis. TSA officials reported that they have taken steps to mitigate the increase in manpower needed to operate these systems, such as enabling the use of different algorithms as appropriate.

DOT&E previously found that a reduced-size stand-alone explosives detection system tested in March 2017 was suitable with limitations, but not effective because of multiple factors resulting in the inability of operators to maintain control of baggage. The program’s OTA completed follow-on OT&E on these systems in January 2019 and initial test results showed improvement in the system’s effectiveness rating.

As of July 2019, EBSP has 1,678 explosives detection systems and 2,477 explosives trace detectors deployed nationwide.

In February 2018, DHS leadership approved the program’s updated acquisition plan, which reflects a new procurement strategy. Under the new procurement strategy, the program will transition from procuring systems with different sizes and speeds to two types: (1) inline systems that integrate with a baggage handling system and are linked through a network, and (2) stand-alone systems that may be integrated with a baggage handling system, but not linked to a network. In addition, TSA officials reported that the new strategy reflects updates to EBSP’s vendor qualification process, which is intended to improve collaboration with vendors so they can develop more technically mature systems.

In March 2018, DHS leadership approved a pilot effort in which TSA’s Chief Acquisition Executive (CAE) provides oversight of changes to deployed systems, including algorithm updates. According to TSA officials, this process is intended to limit some steps in the formal oversight process so capabilities can be deployed more rapidly. DHS leadership plans to assess this pilot process to determine its effectiveness.

In May 2019, the program reported that the five vacant positions impact the program’s performance and execution schedules at times. To mitigate the staffing gap, program officials said that current staff are temporarily assuming additional duties.

TSA officials stated that issues identified in DOT&E assessments were corrected, and that follow-on test activities were conducted and resulted in favorable evaluations and capability deployment. TSA officials also provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

TECHNOLOGY INFRASTRUCTURE MODERNIZATION (TIM)

TRANSPORTATION SECURITY ADMINISTRATION (TSA)

The TIM program was initiated to address shortfalls in TSA’s threat assessment screening and vetting functions by providing a modern end-to-end credentialing system. The TIM system will manage credential applications and the review process for millions of transportation workers and travelers by supporting screening and vetting for Transportation Worker Identification Credential (TWIC) and TSA Pre®.

Program achieved full operational capability for TWIC and TSA Pre® capabilities.

Program met its four key performance parameters.

GAO last reported on this program in May 2018 and October 2017 (GAO-18-339SP, GAO-18-46).

In November 2018, Department of Homeland Security (DHS) leadership approved the TIM program’s request to descope and change its definition of full operational capability (FOC) to include only the TWIC and TSA Pre® capabilities. By the time TIM had fully delivered capabilities for TWIC and TSA Pre®, TSA had made ongoing updates and improvements to the remaining legacy vetting and credentialing systems to meet security and mission demands, which had also sufficiently met end user needs. According to TSA officials, any additional system development would produce redundant functionality. Going forward, the program plans to continue to modernize the legacy systems and to achieve additional efficiencies.

The program updated its key acquisition documents, including its acquisition program baseline (APB) and life-cycle cost estimate (LCCE) to reflect the change in scope. In July 2019, DHS leadership approved program’s revised APB. DHS leadership granted the program acquisition decision event (ADE) 3 and acknowledged the program’s achievement of FOC—fulfilling TSA Pre® and TWIC mission needs for vetting and credentialing—in August 2019. DHS leadership previously approved a revised APB for the TIM program in September 2016. Prior to the approval of the program’s 2016 APB, DHS leadership paused new development for 22 months after the program breached its APB goals for various reasons including technical challenges.

In July 2019, DHS headquarters conducted an independent cost assessment to inform ADE 3, which TSA adopted as the program’s LCCE. The revised LCCE reflected the program’s reduced scope. The program’s APB acquisition cost goal decreased by nearly $220 million from the program’s 2016 APB. The reduction in costs is primarily attributed to the reduction in the program’s scope. However, the program’s operations and maintenance APB cost goals increased by $205 million primarily due to maintenance of legacy systems to address user needs.

Transportation Security Administration (TSA)

TECHNOLOGY INFRASTRUCTURE MODERNIZATION (TIM)

DOT&E recommended that the program address issues related to system usability by assessing the need for training materials and job aids to assist users. In addition, DOT&E recommended that the program update its cybersecurity threat assessment and continue to conduct periodic cyber resilience testing.

In October 2017, GAO found that TSA had not fully implemented several leading practices to ensure successful agile adoption. GAO also found that TSA and DHS needed to conduct more effective oversight of the TIM program to reduce the risk of repeating past mistakes. DHS concurred with all 14 GAO recommendations to improve program execution and oversight, and identified actions DHS and TSA can take to address them. As of September 2019, TSA addressed all but one recommendation— to ensure DHS leadership reached consensus on, documented, and implemented oversight and governance changes for agile program reviews.

TSA reported a critical staffing gap of four FTEs in 2019, including a manager position to adapt initiatives to agile business and development processes. TSA officials stated that the staffing gap has had minimal impact on program execution. To mitigate the gap, the program is leveraging support from contractors and matrixed staff.

TSA officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

FAST RESPONSE CUTTER (FRC)

UNITED STATES COAST GUARD (USCG)

The USCG uses the FRC to conduct search and rescue, migrant and drug interdiction, and other law enforcement missions. The FRC carries one cutter boat on board and is able to conduct operations in moderate sea conditions. The FRC replaces the USCG’s Island Class patrol boat and provides improved fuel capacity, surveillance, and communications interoperability with other Department of Homeland Security (DHS) and Department of Defense assets.

FRC found operationally effective and suitable, and all key performance parameters validated.

Defect in ship structure found, requiring changes in production and retrofits to cutters already delivered.

GAO last reported on this program in May 2018 and March 2017 (GAO-18-339SP, GAO-17-218).

The FRC program is on track to meet its current cost and schedule goals. USCG officials told GAO the program is revising its acquisition program baseline (APB) in 2019 to reflect an increase in FRCs. The USCG previously planned to acquire 58 FRCs and, as of August 2019, 35 had been delivered and another 21 were on contract. However, in fiscal years 2018 and 2019, congressional conferees supported funds for the acquisition of 4 additional FRCs to begin replacing 6 cutters currently operating in the Middle East. To account for the increase of up to 6 additional FRCs, USCG officials stated that they are revising the program’s acquisition documents and anticipate completing these updates by the end of calendar year 2019. To inform the budget process, the program updated its life-cycle cost estimate in June 2019 to reflect the additional 4 cutters that have been funded. The updated estimate remains within the program’s current APB cost thresholds.

USCG officials stated that the contractor—Bollinger Shipyards LLC—is meeting the program’s current delivery schedule and the program is on track to achieve full operational capability (FOC) for the original 58 cutters by March 2027, as planned. However, the program’s FOC date will likely be extended to account for the delivery of the additional cutters in the revised APB.

The program’s initial operational capability (IOC) date previously slipped due to a bid protest related to the program’s initial contract award—now known as the phase 1 contract—and the need for structural modifications. USCG officials attributed a subsequent 5-year slip in the program’s FOC date to a decrease in annual procurement quantities under the phase 1 contract. In May 2014, the USCG determined that it would procure only 32 of the 58 FRCs through this contract and initiated efforts to conduct full and open competition for the remaining 26 vessels— known as phase 2. In May 2016, the USCG awarded the phase 2 contract to Bollinger Shipyards LLC for the remaining 26 FRCs. Under the phase 2 contract, the USCG can procure 4 to 6 FRCs per option period. For fiscal year 2019, the USCG reported that it exercised an option for 6 FRCs. According to USCG officials, the phase 2 contract will need to be modified to increase the total quantity allowed under the current contract and account for the additional FRCs, but as of July 2019 the modifications had not been made.

United States Coast Guard (USCG) FAST RESPONSE CUTTER (FRC)

USCG officials stated that they are on track to resolve the remaining deficiencies by the end of fiscal year 2020. They added that these deficiencies will be resolved either through corrective action or a determination that the deficiency is not a hindrance to operations, requiring no further action. For example, the USCG officials reported taking corrective action in response to the FRC’s periodic inability to send communications due to antenna placement. USCG officials stated this was resolved by adding a second antenna.

The USCG continues to work with Bollinger Shipyards LLC to address issues covered by the warranty and acceptance clauses for each ship. For example, in the fall of 2017, USCG officials reported identifying a latent defect that would affect the FRC’s ability to achieve its intended 25-year structural fatigue life. USCG officials said cracks were found in the interior steel structure of two FRCs, prompting a class-wide inspection. Upon further analysis, the USCG determined that the fatigue issues were due to faulty design assumptions and identified 12 areas of structural weakness that will require reinforcements to the ship’s interior steel structure. In response, USCG officials stated that the contractor developed corrective actions—ranging in complexity from adding bracket supports to removing and replacing large sections of steel—that have been approved by the USCG. USCG officials further stated that corrections are being incorporated during production, but FRCs that have already been delivered will need to be retrofitted during regular maintenance periods, scheduled through 2025. These officials added that these defects do not affect current operations. In addition, the contractor is undertaking retrofits for nine of the 10 engine issues covered by the warranty that are affecting the fleet—such as leaking exhaust pipes—and a prototype solution for the remaining issue is being assessed. As of June 2019, USCG officials reported the FRC’s warranty has resulted in $123 million in cost avoidance.

In July 2019, USCG officials stated they had filled the one critical staffing gap and were in the process of hiring staff to address the remaining staffing gaps.

USCG officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

H-65 CONVERSION/SUSTAINMENT PROGRAM (H-65)

UNITED STATES COAST GUARD (USCG)

The H-65 aircraft is a short-range helicopter that the USCG uses to fulfill its missions, including search and rescue, ports and waterways security, marine safety, and defense readiness. The H-65 acquisition program consists of eight discrete segments that incrementally modernize the H-65 aircraft fleet. The program is currently focused on the service life extension program (SLEP) and upgrades to the automatic flight control system (AFCS) and avionics.

H-65 aircraft failed to meet two key performance parameters in testing; has not yet tested cyber resiliency.

Program to synchronize upgrades into scheduled maintenance periods.

GAO last reported on this program in May 2018 (GAO-18-339SP).

In March 2018, Department of Homeland Security (DHS) leadership approved the program’s revised acquisition program baseline (APB), removing it from breach status, which USCG officials primarily attributed to underestimating the technical effort necessary to meet requirements. DHS leadership also granted the program approval for ADE 2C for low-rate initial production of the avionics and AFCS upgrades and ADE 2B for the addition of a SLEP. The SLEP is expected to extend the flight hour service life of each aircraft from 20,000 flight hours to 30,000 flight hours by replacing obsolete aircraft components. USCG officials stated the USCG plans to operate the H-65 aircraft until 2039 so that the USCG can prioritize funding for the Offshore Patrol Cutter. The USCG also plans to align its next helicopter acquisition effort with the Department of Defense’s future vertical lift acquisition plans.

The program’s current APB reflects the restructured program schedule which synchronizes the SLEP with the avionics and AFCS upgrades. Specifically, the new program structure calls for completing the SLEP and upgrades to AFCS and avionics during the same scheduled maintenance period. This structure allows the USCG to leverage accessibility of components the program intends to replace as part of the SLEP while the aircraft is being assembled to accommodate the avionics and AFCS upgrades. As a result, USCG officials reported that the program will avoid some labor costs and will reduce the risk of damaging AFCS and avionics components which would need to be removed during the SLEP. In its current APB the program’s full operational capability (FOC) date was extended by nearly 2 years to September 2024, primarily to incorporate the SLEP. The program’s total life-cycle cost threshold decreased by approximately $200 million from its March 2014 APB, which USCG officials attributed to decreased labor costs, among other things.

USCG officials told GAO they were in the process of updating the program’s key acquisition documents to inform the program’s ADE 3 decisions for full rate production of the avionics and AFCS upgrades and the SLEP. In July 2019, USCG officials said they do not plan to update the program’s APB for the upcoming ADEs because the program is on track and does not require changes to its cost, schedule, or performance goals.

United States Coast Guard (USCG)

H-65 CONVERSION/SUSTAINMENT PROGRAM (H-65)

The USCG conducted a cybersecurity threat assessment for the H-65 in September 2016, but USCG officials stated cyber resilience was not included in initial OT&E because it was not a consideration at the time the testing was planned and the OTA needed more time to adequately plan for the testing. In May 2019, the program completed a cyber tabletop exercise to inform potential testing. However, it is unclear if this testing will be completed in time to inform ADE 3.

The USCG awarded contracts to Rockwell Collins—the original equipment manufacturer of the legacy AFCS and avionics—for continued development of the AFCS and avionics upgrades in July 2016 and March 2017, respectively. USCG officials said they expect delivery of the upgrades to the fleet in May 2020.

USCG officials said there is risk involved with extending the aircrafts’ service life beyond 20,000 flight hours since it has never been done by other agencies that operate the H-65. USCG officials stated that the aircraft manufacturer, Airbus, assisted the USCG’s chief aeronautical engineer in identifying parts that need replacement. As part of the program’s revised acquisition strategy, the USCG plans to synchronize the SLEP with the avionics and AFCS upgrades and conduct this work during the programmed depot maintenance cycles in fiscal years 2020 through 2024. USCG officials reported that this strategy allows the program to leverage the engineering and program management contractors already in place and ensures SLEP component availability before production support from Airbus ends in 2018.

In April 2019, the USCG reported the program had one critical staffing gap—a deputy program manager. USCG officials reported the program filled the position in August 2019.

USCG officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

LONG RANGE SURVEILLANCE AIRCRAFT (HC-130H/J)

UNITED STATES COAST GUARD (USCG)

The USCG uses HC-130H and HC-130J aircraft to conduct search and rescue missions, transport cargo and personnel, support law enforcement, and execute other operations. Both aircraft are quad-engine propeller-driven platforms. The HC-130J is a modernized version of the HC-130H, which has advanced engines, propellers, and equipment that provide enhanced speed, altitude, range, and surveillance capabilities.

Design of new mission system processor is complete; USCG officials reported all key performance parameters met.

Transfer of surplus HC-130H aircraft to other agencies delayed.

GAO last reported on this program in May 2018 (GAO-18-339SP).

As of July 2019, the USCG has yet to complete a more than 4-year effort to revise the acquisition program baseline (APB)—to account for significant program changes. Specifically, the USCG decided to pursue an all HC-130J fleet and, in fiscal year 2014, Congress directed the transfer of 7 HC-130H aircraft to the U.S. Air Force. The USCG was in the process of upgrading these aircraft but canceled further HC-130H upgrades. In September 2017, Department of Homeland Security (DHS) leadership directed the USCG to submit the revised APB by January 2018. As of July 2019, USCG officials had revised key acquisition documents such as the program’s life-cycle cost estimate (LCCE) and operational requirements document (ORD)—which will inform the program’s revised APB—but USCG officials told GAO the APB is not expected to be approved until August 2019.

USCG officials said the re-baseline has been delayed, in part, because Congress directed the USCG to conduct a multi-phased analysis of its mission needs. In November 2016, the USCG submitted the results of its analysis for fixed-wing aircraft, which confirmed the planned total quantity of 22 HC-130J aircraft and an annual flight-hour goal of 800 hours per aircraft. The results of the analysis are reflected in the program’s revised LCCE, which DHS approved in June 2019. However, the USGC plans to decommission the HC-130H fleet by the end of fiscal year 2022, which may result in a capability gap since the program’s revised LCCE indicates that the fleet will consist of only 14 HC-130J aircraft in fiscal year 2022. In addition, the program’s revised ORD includes a full operational capability (FOC) date—when all 22 aircraft are operational and assigned to USCG air stations—of September 2033. The revised FOC date is more than 6 years beyond the program’s current threshold date of March 2027. GAO previously reported that the program was at risk of not meeting its previously planned FOC date because the USCG had not requested adequate funding.

The program’s revised LCCE acquisition costs decreased in part because costs associated with the initially planned HC-130H improvements were removed. However, the program’s operations and maintenance costs increased by over $800 million over the program’s previous estimate, which is primarily attributed to a 13-year increase in the life expectancy of the HC-130J aircraft.

United States Coast Guard (USCG)

LONG RANGE SURVEILLANCE AIRCRAFT (HC-130H/J)

According to USCG officials, the HC-130J has now met all seven of its key performance parameters (KPP). Previously, the program was unable to meet its KPPs related to the detection of targets and the aircraft’s ability to communicate with other assets. However, the USCG is replacing the mission system processor on its fixed-wing aircraft—including the HC-130J—with a system used by the U.S. Navy and DHS’s Customs and Border Protection. The new mission system processor is intended to enhance operator interface and sensor management and replace obsolete equipment. USCG officials said the design of the new mission system processor was approved in March 2018.

The USCG does not plan to operationally test the new processor on the HC-130J, in part because the aircraft has already been tested. In 2009, DHS’s Director, Office of Test and Evaluation and the USCG determined the HC-130J airframe did not need to be operationally tested because the U.S. Air Force conducted operational testing on the base C-130J airframe in 2005. Instead, the USCG plans to operationally test the new mission system processor in fiscal year 2021 during operational testing on the C-27J, which is new to the USCG’s fixed-wing fleet. In addition, the USCG officials stated systems acceptance and delivery testing are conducted on each aircraft. In July 2019, USCG told GAO that all HC-130Js in the fleet are being outfitted with the new mission system processor.

In December 2013, Congress directed the transfer of seven HC-130H aircraft to the U.S. Air Force for modifications—which consist of upgrades and installing a fire retardant delivery system—and subsequent transfer to the U.S. Forest Service. This direction factored into the USCG’s decision to pursue an all HC-130J fleet. However in August 2018, Congress directed that the U.S. Air Force transfer the modified aircraft to the state of California, Natural Resources Agency, for use by the Department of Forestry and Fire Protection. USCG officials reported seven aircraft will be transferred to the state of California, Natural Resources Agency, and the USCG does not plan to retain the surplus aircraft. As of July 2019, no HC-130H aircraft have been transferred.

The USCG plans to procure a total of 22 HC-130Js. In July 2019, USCG officials reported 13 HC-130J aircraft had been delivered and USCG had awarded contracts for three more. At that time, the USCG also had 14 HC-130Hs in its inventory. The USCG planned to remove four of the HC-130Hs from service in 2019 as HC-130Js and C-27Js are delivered.

USCG officials said the program is not experiencing any workforce issues as a result of its staffing gap. The program filled the one critical vacancy in August 2019 and is in the process of hiring staff to fill an additional vacancy.

USCG officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

MEDIUM RANGE SURVEILLANCE AIRCRAFT (HC-144A/ C-27J)

UNITED STATES COAST GUARD (USCG)

The USCG uses HC-144A and C-27J aircraft to conduct all types of missions, including search and rescue and disaster response. All 32 aircraft—18 HC-144A aircraft and 14 C-27J aircraft—are twin-engine propeller driven platforms. The interior of both aircraft are able to be reconfigured to accommodate cargo, personnel, or medical transports.

New mission system processor installed on five HC-144A aircraft.

Program challenges related to purchasing spare parts and accessing technical data are improving.

GAO last reported on this program in May 2018 (GAO-18- 339SP).

In April 2019, Department of Homeland Security (DHS) leadership approved a change to the program’s current acquisition program baseline (APB) to adjust the program’s schedule milestones as a result of the fiscal year 2019 partial government shutdown. USCG officials told GAO that delays in funding limited contracted work for the program during the shutdown. USCG officials stated that the program could not recover from the lost time and, in response, DHS leadership authorized the program’s request for a 3-month extension on the program’s future APB milestones. The current APB was approved in August 2016 to reflect the restructuring of the HC-144A acquisition program. The USCG initially planned to procure a total of 36 HC-144A aircraft, but reduced that number to the 18 it had already procured after Congress directed the transfer of 14 C-27J aircraft from the U.S. Air Force to the USCG in fiscal year 2014.

The program’s APB divides the program into two phases. Phase 1 includes acceptance of the 18 HC-144A aircraft and upgrades to the aircraft’s mission and flight management systems. Phase 2 includes acceptance of and modifications to the C-27J aircraft to meet the USCG’s mission needs. In July 2019, USCG officials said that the program had completed upgrades on five HC-144A aircraft and plans to complete upgrades on all HC-144As by September 2021. For phase 2, the USCG has accepted all 14 C-27Js from the U.S. Air Force and plans to complete the modification of these aircraft by June 2025 to achieve full operational capability (FOC).

To inform the budget process, in June 2019 the program updated its life-cycle cost estimate (LCCE), which is within its current APB cost thresholds. The program’s total life-cycle cost decreased by approximately $115 million. USCG officials attribute the decrease to refinement of the cost estimate based on actual costs, changes to the schedule for the mission system upgrades, and a delay in operating missionized C-27Js—which reduces the total estimated aircraft flight hours—among other things. USCG officials said that they plan to delay operation of missionized C-27Js to ensure adequate logistics support is available for the aircraft. In addition, congressional conferees supported $18 million in fiscal year 2018 for the USCG to purchase a flight simulator for training purposes. According to USCG officials, prioritizing the procurement of the flight simulator in fiscal year 2018 addressed C-27J training needs and provided over $15 million in cost savings for the program.

United States Coast Guard (USCG)

MEDIUM RANGE SURVEILLANCE AIRCRAFT (HC-144A/C-27J)

Neither the HC-144A nor the C-27J will be able to meet two of their seven key performance parameters (KPP) until the USCG installs a new mission system processor on the aircraft. These two KPPs are related to the detection of targets and the aircraft’s ability to communicate with other assets. The USCG is replacing the mission system processor on its fixed-wing aircraft—including the HC-144A and C-27J— with a system used by the U.S. Navy and DHS’s Customs and Border Protection. The new mission system processor is intended to enhance operator interface and sensor management and replace obsolete equipment.

The program plans to conduct developmental testing on the C-27J in fiscal year 2020, once the prototype is complete. In addition, the USCG plans to operationally assess the new mission system processor during operational testing of the C-27J, which is scheduled to begin in fiscal year 2021.

GAO previously found that the program faced challenges purchasing spare parts and accessing technical data for the C-27J, which was affecting the USCG’s ability to transition the aircraft into the fleet. USCG officials told GAO that these issues are improving. Specifically, they stated that program awarded two contracts for spare parts to third-party suppliers in early 2018 and purchased spare parts in bulk in 2017 to maintain the fleet. In July 2019, USCG officials said the program has been able to stock sites well enough to keep assets available for use, and will continue to work with the contractors to address the issue.

USCG officials said that a contract was awarded to the original equipment manufacturer in April 2017 that allows the USCG appropriate rights to the technical data. Also, in August 2019, USCG officials told GAO they received all C-27J technical data in the Air Force’s possession, including operations and maintenance manuals, as part of the transfer of 14 C-27J aircraft from the Air Force to the Coast Guard.

USCG officials told us that the program updated its acquisition plan in February 2018 to incorporate the procurement of a new full-motion flight simulator training device for the C-27J aircraft. The USCG received funding to purchase a flight simulator in fiscal year 2018 and plans to begin instructor training on the device in August 2019.

In July 2019, USCG officials told GAO that the program’s staffing is not negatively impacting program execution. USCG officials explained that they have filled four of the program’s reported staffing vacancies and plan to fill the remaining position soon.

USCG officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

NATIONAL SECURITY CUTTER (NSC)

UNITED STATES COAST GUARD (USCG)

The USCG uses the NSC to conduct search and rescue, migrant and drug interdiction, environmental protection, and other missions. The NSC replaces and provides improved capabilities over the USCG’s High Endurance Cutters. The NSC carries helicopters and cutter boats, provides an extended on-scene presence at forward deployed locations, and operates worldwide.

Follow-on operational testing was completed in 2018, but unmanned aerial surveillance aircraft testing was delayed.

The USCG continues to address issues identified with the NSC propulsion system.

GAO last reported on this program in May 2018 and April 2017 (GAO-18-339SP, GAO-17- 218).

In November 2017, Department of Homeland Security (DHS) leadership approved a revised acquisition program baseline (APB), which accounted for the addition of a ninth NSC to the program of record. The USCG originally planned to acquire eight NSCs; however, in fiscal year 2016 Congress appropriated funds specifically for the production of a ninth NSC. Congressional conferees subsequently included in fiscal year 2018 $540 million and $635 million to be immediately available and allotted to contract for production of a 10th NSC and purchase of long lead time materials and production of an 11th NSC, respectively. According to USCG officials, the USCG awarded a contract to produce the ninth NSC in December 2016 and awarded a production contract for the 10th and 11th NSCs in December 2018. As of August 2019, eight NSCs have been delivered and the remaining three NSCs are under contract for production.

USCG officials reported that the program is currently on track to meet its current APB schedule and anticipate delivery of the ninth NSC in September 2020. However, the program’s full operational capability (FOC) date is expected to be extended until 2024 as a result of the anticipated delivery of the 11th NSC in January 2024.

According to USCG officials, the program’s acquisition documentation, including the APB, is being revised to reflect the additional NSCs and these updates are expected to be complete by July 2020. To inform the budget process, the program updated its LCCE to include the 10th and 11th NSCs. As a result, the program’s life-cycle costs exceed the current APB thresholds. Despite this cost growth, the program’s total life-cycle cost is still less than the program’s initial estimate for eight ships. USCG officials attribute the overall decrease to more accurate estimates and reduced operations and maintenance (O&M) costs. The program’s current APB cost thresholds already reflect cost growth that occurred earlier in the program, when the program implemented several design changes to address equipment issues. As of September 2017, 12 equipment systems had design changes, which USCG estimated cost over $260 million. This work includes structural enhancements on the first two NSCs and the replacement of the gantry crane, which aids in the deployment of cutter boats.

United States Coast Guard (USCG)

NATIONAL SECURITY CUTTER (NSC)

USCG officials said the USCG completed a study directed by DHS’s USM to identify the root cause of engine issues with the NSC’s propulsion systems. GAO previously reported on these issues—including high engine temperatures and cracked cylinder heads—in January 2016. USCG officials reported that the study resulted in nine corrective measures, eight of which are in various stages of implementation. According to USCG officials, they will assess the need to implement the remaining corrective measure following completion of the others.

According to program officials, the USCG relies on the Navy to request funding for and provide certain systems on the NSC such as the Close In Weapon System, which includes a radar-guided gun used to protect against anti-ship cruise missiles. USCG officials reported that some of these Navy systems may not be available in time to support the production of the ninth, 10th and 11th NSCs, since these cutters were unplanned additions to the NSC program and the Navy had not included funding for some of these systems in its budget requests. According to program officials, they are working with the Navy to identify options to mitigate this issue. Officials stated that an option being considered is constructing the NSCs with space available for the Navy equipment to be installed after delivery.

USCG officials said the program’s staffing vacancies had not negatively affected program execution and, as of September 2019, all three vacancies had been filled. The program’s staffing profile represents staffing requirements through NSC 11, and USCG officials reported that the program office would need to reassess future staffing requirements if the USCG acquires additional NSCs.

USCG officials stated that with the exception of small unmanned aerial surveillance aircraft, follow-on OT&E testing is completed. Additional testing are planned in fiscal year 2020. A comprehensive update of the program’s LCCE is being drafted to reflect costs of the 10th and 11th NSC. The program will base the cost goals of the next revision to the APB on this update. The next revision of the APB will include a revised FOC date based on delivery of the 11th NSC in January 2024. USCG officials also provided technical comments on a draft assessment, which GAO incorporated as appropriate.

OFFSHORE PATROL CUTTER (OPC)

UNITED STATES COAST GUARD (USCG)

The USCG plans to use the OPC to conduct patrols for homeland security, law enforcement, and search and rescue operations. The OPC is being designed for long-distance transit, extended on-scene presence, and operations with deployable aircraft and small boats. It is intended to replace the USCG’s aging Medium Endurance Cutters (MEC) and bridge the operational capabilities provided by the Fast Response Cutters and National Security Cutters.

Shipyard sustained damage in Hurricane Michael, expected to result in program cost and schedule changes.

USCG assessing the effects from hurricane and plans to identify a path forward in early fiscal year 2020.

GAO last reported on this program in May and July 2018 (GAO-18-339SP, GAO-18-629T).

In May 2018, the Department of Homeland Security (DHS) approved a revised life-cycle cost estimate (LCCE) for the OPC program, which officials said reflects a refinement of the OPC design and planned systems—including a weight increase of 27 percent—and the incorporation of actual contract data, among other things. The USCG is not reporting a cost increase because the amount of OPC acquisition costs that the program plans to fund, approximately $10.3 billion, remains within the program’s acquisition program baseline (APB) cost thresholds. However, the revised LCCE included a shift of some costs that were previously planned to be funded by the program to other sources, such as other parts of the USCG or the U.S. Navy. This government-furnished equipment, which is now estimated to cost nearly $2 billion, will largely be funded by the U.S. Navy, according to USCG officials. Overall, the total program acquisition costs increased by approximately $1.7 billion from the previous estimate.

In October 2018, the shipbuilder, Eastern Shipbuilding Group, suffered damage as a result of Hurricane Michael. The shipbuilder reported to the USCG in May 2019 that it can no longer afford the estimated costs associated with the OPC contract without assistance from the government. In January 2019, the shipbuilder resumed construction of the lead ship, but the damages sustained have resulted in a long-term degradation of their ability to produce the OPCs at the previously estimated cost and schedule. The shipbuilder has projected hundreds of millions of dollars in increased contract costs—which it attributes to anticipated skilled labor shortages and a loss of production efficiencies—and a 9- to 12-month delivery delay for each of the first nine ships.

Despite these anticipated cost increases and schedule delays, as of July 2019, USCG officials said they had not formally notified DHS leadership of a potential cost or schedule breach because they are continuing to assess how to move forward. DHS leadership granted the program a 3-month extension to achieve its acquisition decision event (ADE) 2C in December 2019 to mitigate impacts from the fiscal year 2019 partial government shutdown. USCG officials said they are preparing for the ADE 2C, but also are using the additional time to assess the shipbuilder’s report, analyze estimates, and determine a path forward by early fiscal year 2020.

United States Coast Guard (USCG)

OFFSHORE PATROL CUTTER (OPC)

The USCG currently plans to conduct initial operational test and evaluation (OT&E) on the first OPC in fiscal year 2023. However, the test results from initial OT&E will not be available to inform key decisions. For example, they will not be available to inform the decision to build two OPCs per year, which USCG officials said is currently scheduled for fiscal year 2021. Without test results to inform these key decisions, the USCG may need to make substantial commitments prior to knowing how well the ship will meet its requirements.

According to USCG program officials, they have established a team with representatives from DHS, USCG, and the U.S. Navy to assess the impact of Hurricane Michael and determine a way forward. As part of its assessment, these officials said they are evaluating a number of options, including modifications to the original contract. Regardless of the path forward, USCG officials stated the program will likely need congressional approval of the contracting strategy and financial resources necessary to execute the new plan.

USCG officials stated that DHS leadership will review the program’s status and determine whether to authorize the construction of OPC 2 and the purchase of initial materials needed for OPC 3 at the program’s ADE 2C. USCG officials stated that they anticipate the exercise of a contract option for the construction of OPC 2 and the materials for OPC 3 will be delayed as the program and shipbuilder continue to assess the impact of the hurricane on OPC production.

The OPC program is continuing to increase staffing as the program matures and production activities increase. In July 2019, USCG officials said the program has a staffing gap of five FTEs, none of which are critical. Officials said they were in the process of hiring staff to fill these positions.

USCG officials provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.

POLAR SECURITY CUTTER (PSC)

UNITED STATES COAST GUARD (USCG)

The PSC program—formerly designated as the Heavy Polar Icebreaker—is intended to assist the USCG in maintaining access to Arctic and Antarctic polar regions. The USCG requires its icebreaking fleet to conduct multiple missions, including defense readiness; marine environmental protection; ports, waterway, and coastal security; and search and rescue. The USCG plans to acquire three PSCs to recapitalize its heavy polar icebreaker fleet, which currently consists of one operational ship.

DHS identified three critical technologies in its June 2019 technology readiness assessment of the program.

Program awarded a $750 million detail design and construction contract to VT Halter Marine in April 2019.

GAO last reported on this program in May and September 2018 (GAO-18-339SP, GAO-18- 600).

In January 2018, Department of Homeland Security (DHS) leadership approved the program’s initial acquisition program baseline (APB), establishing cost, schedule, and performance goals. The program achieved a combined acquisition decision event (ADE) 2A/2B in February 2018, which authorized the initiation of development efforts.

However, in September 2018, GAO found that the program’s schedule and cost estimates are optimistic. Specifically, GAO found that the program’s planned delivery dates are not informed by a realistic assessment of shipbuilding activities. Instead, the schedule is driven by the potential gap in icebreaking capabilities once the USCG’s only operational heavy polar icebreaker reaches the end of its service life. As a result, the program is at risk of experiencing schedule delays. Similarly, GAO found that the program’s life-cycle cost estimate (LCCE) adheres to most cost estimating best practices but is not fully reliable. This was due, in part, to the cost estimate not quantifying the range of possible costs over the entire life of the program. As a result, the program is at risk of costing more than estimated.

In April 2019, the program awarded a $746 million contract to VT Halter Marine for the detail design and construction of the lead PSC. According to USCG officials, the program is revising both the program schedule and cost estimate with information from the shipbuilder. For example, delivery of the lead ship in the awarded contract is anticipated in May 2024—2 months after the program’s APB threshold date. In addition, the program updated its LCCE in June 2019 to inform the budget process, but this estimate does not reflect cost changes as a result of the contract award. USCG officials acknowledged the schedule and cost risks identified by GAO and plan to address these risks as part of the acquisition documentation updates.

From 2013 through 2019, the program received $1.035 billion in funding—$735 million in USCG appropriations and $300 million in Navy appropriations. USCG officials stated that the lead ship is fully funded but any funding gaps in the future may result in delays to delivery of the two follow-on ships.

United States Coast Guard (USCG)

POLAR SECURITY CUTTER (PSC)

DHS leadership approved four key performance parameters related to the ship’s ability to independently break through ice, the ship’s operating duration, and communications. From May to August 2017, the USCG conducted model testing of potential hull designs and propulsion configurations. USCG officials stated that maneuverability was identified as a challenge during model testing and that azimuthing propulsors—propellers that sit below the ship and can rotate up to 360 degrees—offered better maneuverability for the PSC than traditional propulsion systems. According to USCG officials, the PSC program began additional model testing related to ice models and seakeeping in August 2019.

In November 2017, DHS’s Director, Office of Test and Evaluation approved the program’s test and evaluation master plan, which calls for initial operational testing of performance to begin in fiscal year 2024, after delivery of the first PSC. In response to a September 2018 GAO recommendation, DHS’s Science and Technology Directorate completed a technology readiness assessment of the program in June 2019. DHS determined that the PSC has three critical technologies that are mature or approaching maturity: azimuthing propulsors, the integrated electric propulsion system, and the hull form. For the hull form—the only critical technology designated as not yet mature—the Coast Guard plans to use ice model and seakeeping testing to reduce risks. USCG officials stated that they are planning to reassess the critical technologies using information from VT Halter Marine by the preliminary design review scheduled for January 2020.

The USCG established an integrated program office and ship design team with the Navy and, in 2017, DHS, the USCG, and the Navy entered into several agreements that outline major roles and responsibilities, including the Navy’s role in contracting on behalf of the Coast Guard. The ship design team provided technical oversight for the development of the PSC’s concept designs, which the USCG used to inform the ship’s specifications and program’s life-cycle cost estimate.

According to USCG officials, as of July 2019, the USCG and the Navy established a project residence office of three staff at the shipbuilder’s facility in Pascagoula, Mississippi to provide oversight of shipbuilding efforts. In April 2019, USCG reported that it is increasing the required staffing level for the program as it matures, with 5 FTEs added in fiscal year 2019. According to program officials, as of July 2019, three of these five vacancies—including the commanding officer and executive officer of the project resident office—have been filled. USCG officials said the remaining positions were being addressed by active duty USCG staff and through the civilian hiring process.

In September 2018, GAO made six recommendations to DHS, the USCG, and the Navy to address risks GAO identified with the PSC program. As of August 2019, three of the six recommendations remain open.

USCG officials stated that the PSC program awarded a contract for the detail design and construction of up to three cutters to VT Halter Marine in April 2019—ahead of schedule. USCG officials added that the program has either addressed or is in the process of addressing all of GAO’s recommendations contained in GAO-18-600, including an update to the schedule and cost estimate to reflect the award to VT Halter Marine. USCG officials also provided technical comments on a draft assessment, which GAO incorporated as appropriate.

UNITED STATES CITIZENSHIP AND IMMIGRATION SERVICES (USCIS)

The Transformation program was established in 2006 to transition USCIS from a fragmented, paper-based filing environment to a consolidated, paperless environment for electronically processing immigration and citizenship applications. The program is delivering system capability through releases that either deploy electronic, web-based application forms or improve system functionality.

Program revised key performance parameters to reflect the program’s new baseline.

Program reorganized to leverage USCIS expertise and focus on system functionality.

GAO last reported on this program in May 2018 and July 2016 (GAO-18-339SP, GAO-16- 467).

In June 2018, Department of Homeland Security (DHS) leadership approved Transformation’s revised acquisition program baseline (APB) and subsequently removed the program from breach status—lifting a strategic pause that had limited new program development for 18 months. The program experienced a schedule breach in September 2016 when it failed to upgrade to USCIS’s application processing information system to include applications for naturalization.

The new baseline modified the program’s cost, schedule, and performance parameters and reflects changes to the way the program delivers capabilities and a new acquisition strategy. Specifically, the new APB revised the scope of the Transformation program to focus on improving functionality—such as application processing time. Under the prior strategy, the program was focused on adding new applications or forms—from four separate lines of business—to the upgraded processing system.

The program plans to complete major development work in September 2019 and achieve full operational capability (FOC) in March 2020. Despite the 18-month pause in development, the program’s FOC dates slipped only 1 year from its previously revised APB. In August 2019, USCIS officials reported that the program is on track to meet its revised schedule goals.

In its revised APB, the program’s acquisition cost threshold decreased from its previous APB by approximately $200 million primarily because the program shifted costs to operations and maintenance (O&M) to align with DHS’s new common appropriations structure. As a result of this shift in costs and because the new APB extended the program’s life cycle by 2 years, O&M costs increased by nearly $800 million from the program’s previous APB. In June 2019, the program updated its LCCE again to inform the budget process, which is within its APB cost thresholds.

As part of its re-baselining efforts, the Transformation program updated its operational requirements document. The program removed six of its eight key performance parameters (KPP) that were specific to prior Transformation releases, revised two KPPs related to system reliability and availability, and added two new KPPs related to system lead time and cybersecurity. USCIS officials noted that these changes were made to make the KPPs more measurable and testable throughout development and delivery of the capability. The program also updated its test and evaluation master plan (TEMP) to adjust operational assessments to focus on the program’s revised goals under the updated baseline, among other things. The revised TEMP includes plans for three operational assessments that cover (1) development efforts initiated prior to the Transformation program’s June 2018 re-baseline, (2) new development, and (3) cybersecurity.

In March 2019, the program’s OTA completed an operational assessment (OA) of capability developed and released since the program re-baselined in June 2018. The OTA found that the program is meeting all four of its revised KPPs. The OTA recommended the program take steps to plan for cyber resilience testing and evaluation. The OTA plans to conduct a separate OA to assess cybersecurity by September 2019 and plans to complete initial operational test and evaluation of the entire system by December 2019.

In September 2016, the Transformation program breached its schedule baseline when persistent system deficiencies forced the program to revert 84,000 monthly applications for naturalization forms from an upgraded application information system to a legacy platform. USCIS officials said the program had previously prioritized an ambitious release schedule over needed functionality. In response, USCIS dismantled the program office and repositioned Transformation under the USCIS Office of Information Technology so the program could leverage additional engineering expertise. According to officials, the program has also focused on activities like prototyping and beta testing forms, and is deploying updates as targeted changes to specific forms or functionality rather than major system upgrades.

The program previously made significant changes after it experienced a 5-month delay in 2012. DHS attributed this delay to weak contractor performance and pursuing an unnecessarily complex system, among other things. To address these issues, the Office of Management and Budget, DHS, and USCIS determined the program should implement an agile software development methodology and increase competition for development work. These changes were reflected in the program’s April 2015 revised baseline.

In July 2019, the program office reported that it is working to fill staffing vacancies, but the gap has not had a negative impact on program execution. In the meantime, the program is mitigating the gap with existing staff and contractors. However, officials noted that if positions remain unfilled, the program could experience schedule delays, among other things.

USCIS officials reviewed a draft of this assessment and provided no comments.

Appendix II: Objectives, Scope, and Methodology

The objectives of this audit were designed to provide congressional committees insight into the Department of Homeland Security’s (DHS) major acquisition programs. We assessed the extent to which (1) DHS’s major acquisition programs are on track to meet their schedule and cost goals and (2) current program baselines trace to key acquisition documents. To address these questions, we selected 29 of DHS’s 80 major acquisition programs. We selected all 17 of DHS’s Level 1 acquisition programs—those with life-cycle cost estimates (LCCE) of $1 billion or more—that had at least one project, increment, or segment in the Obtain phase—the stage in the acquisition life cycle when programs develop, test, and evaluate systems—at the initiation of our audit. Additionally, we reviewed 12 other major acquisition programs—including 6 Level 1 programs that either had not yet entered or were beyond the Obtain phase, and 6 Level 2 programs that have LCCEs between $300 million and less than $1 billion—that we identified were at risk of not meeting their cost estimates, schedules, or capability requirements based on our past work and discussions with DHS officials. Specifically, we met with representatives from DHS’s Office of Program Accountability and Risk Management (PARM)—DHS’s main body for acquisition oversight— as a part of our scoping effort to determine which programs (if any) were facing difficulties in meeting their cost estimates, schedules, or capability requirements. The 29 selected programs were sponsored by eight different components, and they are identified in table 8, along with our rationale for selecting them.

To determine the extent to which DHS’s major acquisition programs are on track to meet their schedule and cost goals, we collected key acquisition documentation for each of the 29 programs, such as all LCCEs and acquisition program baselines (APB) approved at the department level since DHS’s current acquisition management policy went into effect in November 2008. DHS policy establishes that all major acquisition programs should have a department-approved APB, which establishes a program’s critical cost, schedule, and performance parameters, before they initiate efforts to obtain new capabilities. Twenty- seven of the 29 programs had one or more department-approved LCCEs and APBs between November 2008 and August 31, 2019. We used these APBs to establish the initial and current cost and schedule goals for the programs. We then developed a data collection instrument to help validate the information from the APBs and collect similar information from programs without department-approved APBs. Specifically, for each program, we pre-populated data collection instruments to the extent possible with the schedule and cost information we had obtained from the APBs and our prior assessments (if applicable) to identify schedule and cost goal changes, if any, since (a) the program’s initial baseline was approved and (b) December 2017—the data cut-off date of our 2018 assessment. We shared our data collection instruments with officials from the program offices to confirm or correct our initial analysis and to collect additional information to enhance the timeliness and comprehensiveness of our data sets. We then met with program officials to identify causes and effects associated with any identified schedule and cost goal changes, including changes as a result of the fiscal year 2019 partial government shutdown. Subsequently, we drafted preliminary assessments for each of the 29 programs, shared them with program and component officials, and gave these officials an opportunity to submit comments to help us correct any inaccuracies, which we accounted for as appropriate (such as when new information was available).

Additionally, in July 2018 and July 2019, we obtained copies of the detailed data on affordability that programs submitted to inform the fiscal year 2019 and 2020 resource allocation processes. We also obtained copies of any annual LCCE updates programs submitted in fiscal years 2018 and 2019. For each of the 27 programs with a department-approved APB, we compared (a) the most recent cost data we collected (i.e., a department-approved LCCE, the detailed LCCE information submitted during the resource allocation process, an annual LCCE update, or an update provided by the program office) to (b) DHS’s funding plan presented in the Future Years Homeland Security Program (FYHSP) report to Congress for fiscal years 2020-2024, which presents 5-year funding plans for DHS’s major acquisition programs, to assess the extent to which a program was projected to have an acquisition funding gap. Through this process, we determined that our data elements were sufficiently reliable for the purpose of this engagement.

The FYHSP reports information by the department’s new common appropriation structure, which created standard appropriation fund types including (1) procurement, construction, and improvements and (2) operations and support. We refer to these types of funding as (1) acquisition and (2) operations and maintenance throughout this report. current version of the guidance when we initiated our review. We reviewed each program’s most recent APB to determine whether the APB referenced the documents that were used as the basis of its cost, schedule, and performance parameters. We asked program officials to provide the underlying documentation if the APB did not reference a document. We then compared the APB cost, schedule, and performance parameters to the information in the underlying documents. Specifically, we compared the approved LCCE to the APB objective and threshold cost values, the operational requirements document to the APB key performance parameters, and the integrated master schedule to the APB schedule goals. We determined that the cost and performance goals for a program were traceable if the information from the underlying documentation was the same as the cost and performance parameters in the APB. We determined that program schedule goals were traceable to the integrated master schedule, if all future baseline milestones identified in the APB were identified in the integrated master schedule. In addition, the milestone date from the integrated master schedule was within the range of the objective and threshold schedule goals identified in the APB. We did not include programs in our analysis with APBs approved before DHS updated its acquisition policy in March 2016 because they were developed under previous guidance when the requirements for developing APBs were different. We also did not include the APBs approved after DHS updated its acquisition policy in February 2019 because the update was not in place when we initiated this review.

In addition, we interviewed officials from headquarters organizations, including PARM, to discuss how policies related to developing APBs are being implemented and clarify requirements for establishing APB parameters. We interviewed component and program officials to identify causes of inconsistencies between the approved APB and documents that provided the basis for approved cost, schedule, and performance parameters.

We conducted this performance audit from April 2018 through December 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: Comments from the Department of Homeland Security

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact listed above, Rick Cederholm (Assistant Director), Alexis Olson (Analyst-in-Charge), Whitney Allen, Leigh Ann Haydon, Khaki LaRiviere, Sarah Martin, and Kelsey Wilson made key contributions to this report. Other contributors included Mathew Bader, Andrew Burton, Erin Butkowski, John Crawford, Aryn Ehlow, Lorraine Ettaro, Laurier R. Fish, Alexandra Gebhard, Elizabeth Hosler-Gregory, Stephanie Gustafson, Jason Lee, Claire Li, Ashley Rawson, Jillian Schofield, Roxanna Sun, Anne Louise Taylor, and Lindsay Taylor.

Related GAO Products

Homeland Security Acquisitions: Opportunities Exist to Further Improve DHS’s Oversight of Test and Evaluation Activities. GAO-20-20. Washington, D.C.: October 24, 2019 High-Risk Series: Substantial Efforts Needed to Achieve Greater Progress on High-Risk Areas. GAO-19-157SP. Washington, D.C.: March 6, 2019.

Coast Guard Acquisitions: Polar Icebreaker Program Needs to Address Risks before Committing Resources. GAO-18-600. Washington, D.C.: September 4, 2018.

DHS Acquisitions: Additional Practices Could Help Components Better Develop Operational Requirements. GAO-18-550. Washington, D.C.: August 8, 2018.

Southwest Border Security: CBP Is Evaluating Designs and Locations for Border Barriers but Is Proceeding Without Key Information. GAO-18-614. Washington, D.C.: July 30, 2018.

Coast Guard Acquisitions: Actions Needed to Address Longstanding Portfolio Management Challenges. GAO-18-454. Washington, D.C.: July 24, 2018.

Homeland Security Acquisitions: Leveraging Programs’ Results Could Further DHS’s Progress to Improve Portfolio Management. GAO-18-339SP. Washington, D.C.: May 17, 2018.

DHS Program Costs: Reporting Program-Level Operations and Support Costs to Congress Would Improve Oversight. GAO-18-344. Washington, D.C.: April 25, 2018.

Border Security: Additional Actions Could Strengthen DHS Efforts to Address Subterranean, Aerial, and Maritime Smuggling. GAO-17-474. Washington, D.C.: May 1, 2017.

Homeland Security Acquisitions: Identifying All Non-Major Acquisitions Would Advance Ongoing Efforts to Improve Management, GAO-17-396. Washington, D.C.: April 13, 2017.

Homeland Security Acquisitions: Earlier Requirements Definition and Clear Documentation of Key Decisions Could Facilitate Ongoing Progress. GAO-17-346SP. Washington, D.C.: April 6, 2017.

Homeland Security Acquisitions: Joint Requirements Council’s Initial Approach Is Generally Sound and It Is Developing a Process to Inform Investment Priorities. GAO-17-171. Washington, D.C.: October 24, 2016.

Homeland Security Acquisitions: DHS Has Strengthened Management, but Execution and Affordability Concerns Endure. GAO-16-338SP. Washington, D.C.: March 31, 2016.

Homeland Security Acquisitions: Major Program Assessments Reveal Actions Needed to Improve Accountability. GAO-15-171SP. Washington, D.C.: April 22, 2015.

Homeland Security Acquisitions: DHS Should Better Define Oversight Roles and Improve Program Reporting to Congress. GAO-15-292. Washington, D.C.: March 12, 2015.

Homeland Security Acquisitions: DHS Could Better Manage Its Portfolio to Address Funding Gaps and Improve Communications with Congress. GAO-14-332. Washington, D.C.: April 17, 2014.

Homeland Security: DHS Requires More Disciplined Investment Management to Help Meet Mission Needs. GAO-12-833. Washington, D.C.: September 18, 2012.