Homeland Security Acquisitions: Earlier Requirements Definition and Clear Documentation of Key Decisions Could Facilitate Ongoing Progress
Published: 2017-04-06T00:00:00

Fast Facts:
The Department of Homeland Security planned to spend $7 billion in fiscal year 2016 on major acquisitions—including ships, screening equipment, and surveillance technology.
We reviewed 26 of DHS's major acquisition programs and found that, for the first time, they all had approved cost, schedule, and performance baselines to measure progress. DHS has taken steps to strengthen management of its major acquisitions, but some programs continued to fall short of their goals.
We made recommendations to help DHS further improve its acquisition management, including that the Department define technical requirements earlier in the acquisition process.

Highlights:
section_title: Why GAO Did This Study
paragraphs: In fiscal year 2016, DHS planned to invest about $7 billion in major acquisitions. DHS's acquisition activities are on GAO's High Risk List, in part due to program management, requirements, and funding issues.
The Explanatory Statement accompanying the DHS Appropriations Act, 2015 included a provision for GAO to review DHS's major acquisitions. This report, GAO's third annual review, addresses the extent to which (1) DHS's major acquisition programs are on track to meet schedule and cost goals, (2) these programs are meeting KPPs, and (3) DHS has strengthened implementation of its acquisition policy.
GAO assessed DHS's 15 largest acquisition programs that were in the process of obtaining new capabilities as of May 2016, and 11 additional programs that GAO or DHS identified were at risk of poor outcomes. For all 26 programs, GAO reviewed key documentation, assessed performance against baselines established since DHS's 2008 acquisition policy, and met with program officials. GAO also met with DHS acquisition officials and assessed DHS's policies and practices against GAO acquisition best practices and federal internal control standards.
section_title: What GAO Found
paragraphs: For the first time since GAO began its annual assessments of the Department of Homeland Security's (DHS) major acquisitions, all 26 programs that were reviewed had a department-approved baseline. During 2016, over half of the programs reviewed (17 of the 26) were on track to meet their initial or revised schedule and cost goals. However, 7 of these 17 programs only recently established baselines, 6 of which operated for several years and deployed capabilities without approved baselines. The remaining 9 programs experienced schedule slips, including 4 that also experienced cost growth. The table shows the schedule and cost changes across all 26 programs reviewed, much of which was driven by changes in a few programs.
As of January 2017, 14 of the 26 programs deployed capabilities before meeting all key performance parameters (KPP)—the most important requirements that a system must meet. As a result, DHS may be deploying much-needed capabilities—such as border surveillance equipment and Coast Guard cutters—that do not work as intended. Programs did not meet KPPs for a variety of reasons, such as KPPs were not yet ready to be tested, systems failed to meet KPPs during testing, or KPPs were poorly defined. Contrary to acquisition best practices, DHS policy requires programs to establish schedule, cost, and performance baselines prior to gaining full knowledge about the program's technical requirements. As a result, DHS programs do not match their needs with available resources before starting product development, which increases programs' risk for cost growth, schedule slips, and inconsistent performance.
In 2016, DHS strengthened implementation of its acquisition policy by, for example, focusing on program staffing needs, requiring programs to obtain department-approval for key acquisition documents, and revising the process for when programs breach their cost goals, schedules, or KPPs. However, DHS could better document leadership's acquisition decisions to improve insight into cases that diverge from policy. For example, DHS approved six programs to proceed through the acquisition life cycle even though required documentation was not comprehensive or had not been approved, as required by DHS's policy. Senior DHS officials told GAO these decisions were also based on discussions held at the programs' formal acquisition reviews, but these considerations were not documented. Federal internal control standards require clear documentation of significant events. DHS leadership's decisions may be reasonable, but unless these decisions are documented, insight for internal and external stakeholders is limited. Furthermore, no programs reported a performance breach, even though some programs had not met KPPs. DHS's policy is not clear on how to determine whether a performance breach has occurred. As a result, DHS lacks insight into potential causes of performance issues that may contribute to poor outcomes.
section_title: What GAO Recommends
paragraphs: DHS should ensure that programs define technical requirements before setting baselines; document rationale for key acquisition decisions; and clarify when not meeting KPPs constitutes a breach. DHS concurred with GAO's recommendations.

section_title: Letter
paragraphs: Each year, the Department of Homeland Security (DHS) invests billions of  dollars in its major acquisition programs to help execute its many critical  missions. In fiscal year 2016 alone, DHS planned to spend approximately  $6.9 billion on these acquisition programs, and ultimately the department  will likely invest more than $210 billion in them. DHS and its underlying  components are acquiring systems to help secure the border, increase  marine safety, screen travelers, enhance cybersecurity, improve disaster  response, and execute a wide variety of other operations. Each of DHS’s  major acquisition programs generally costs $300 million or more and  spans multiple years.
To help manage these programs, DHS has established an acquisition  management policy that we have reported is generally sound, in that it  reflects key program management practices. However, we have found  shortfalls in executing the policy and have highlighted DHS acquisition  management issues in our high-risk updates since 2005. Over the past  decade, we have reported that department leadership has dedicated  additional resources to acquisition oversight and documented major  acquisition decisions in a more transparent and consistent manner, but  our work has also identified shortcomings in the department’s ability to  manage its portfolio of major acquisitions. For example, in March 2016  we found that 6 of the 25 programs we reviewed lacked a department- approved Acquisition Program Baseline (APB), which establishes a  program’s cost, schedule, and performance goals. We also found that  several of the acquisition programs faced staffing shortfalls and their  requirements had changed, with many of these programs citing poorly  defined requirements as a cause for changes. These challenges can  contribute to poor acquisition outcomes, such as cost increases or the  risk of end users—such as border patrol agents or first responders in a  disaster—receiving technologies that do not work as originally intended.
We have made many recommendations over the past decade to help  address these challenges. For example, we previously recommended that  DHS leadership specifically assess whether adequate funding is available  during all program reviews. In response, DHS has taken several steps to  improve acquisition management, such as dedicating additional  resources to acquisition oversight and requiring components to certify that  programs are affordable before they are approved to move through the  acquisition life cycle. Nonetheless, DHS has not fully addressed several  of our other recommendations. For example, we previously  recommended that DHS leadership ensure all major programs fully  comply with acquisition policy by obtaining department-level approval for  acquisition documents before the programs are allowed to proceed and  present any anticipated annual funding gaps for acquisition programs in  the annual funding plan submitted to Congress. DHS concurred with  these recommendations and has taken steps to address them.
The Explanatory Statement accompanying a bill to the DHS  Appropriations Act, 2015 contained a provision for GAO to develop a plan  for ongoing reviews of major DHS acquisition programs, as directed in the  Senate report. This is our third annual review of major DHS acquisition  programs. This report addresses the extent to which (1) DHS’s major  acquisition programs are on track to meet their schedule and cost goals,  (2) major acquisition programs are making progress in meeting key  performance parameters (KPP), and (3) DHS has taken actions to  strengthen implementation of its acquisition policy and to improve major  acquisition program outcomes.
We reviewed 26 of DHS’s 71 major acquisition programs, including 24  that we reviewed in 2016. We reviewed all 15 of DHS’s Level 1  acquisition programs—those with life-cycle cost estimates (LCCE) of $1  billion or more—that were in the process of obtaining new capabilities at  the initiation of our audit. To provide insight into some of the factors that  can lead to poor acquisition outcomes, we also included 11 other major  acquisition programs that we or DHS management identified were at risk  of not meeting their schedules, cost estimates, or capability requirements.  Six of these 11 programs were Level 1 acquisitions that had entered the  deployment phase of the acquisition life cycle, while the other five  programs were Level 2 acquisitions with LCCEs between $300 million  and $1 billion. In total, the 26 programs we reviewed were sponsored by  eight different DHS components.
For each of the 26 programs, we analyzed acquisition documentation,  such as APBs, which contain information on programs’ schedules, cost  estimates, and KPPs—the requirements a system must meet to fulfill its  fundamental purpose. Since the November 2008 update to DHS’s  overarching acquisition management directive, these documents have  required DHS-level approval; therefore, we used November 2008 as the  starting point for our analysis. We used these documents to construct a  data collection instrument for each program, identifying any schedule  slips, cost growth, and changes in KPP status. We subsequently shared  this information with each of the 26 program offices and met with program  officials to identify causes and effects associated with any schedule slips,  cost growth, and KPP status changes since (1) their initial baselines and  (2) January 2016—the data cut-off date of the report we issued last year.  We also reviewed DHS’s resource allocation policies and processes and  key funding documents—including affordability certification  memorandums and the Future Years Homeland Security Program  (FYHSP) report to Congress for fiscal years 2017-2021, which presents  5-year funding plans for each of DHS’s major acquisition programs—to  assess the affordability of the 26 programs we reviewed.
In addition, we reviewed test reports and any letters of assessment from  DHS’s Director, Office of Test and Evaluation (formerly Director of  Operational Test and Evaluation), which assess system performance  during operational testing, to assess programs’ progress in meeting  KPPs. Furthermore, we reviewed DHS’s acquisition policy and guidance;  acquisition decision memorandums issued in calendar year 2016; and  key acquisition documentation for major acquisition programs, including  APBs, breach notifications for cost, schedule, or performance that  exceeded baselines, and any remediation plans. We assessed DHS’s  acquisition management policies and processes against the Standards for  Internal Control in the Federal Government, as well as GAO’s best  practices for managing acquisition programs. Lastly, we interviewed  acquisition management officials from DHS headquarters to obtain their  perspectives on new and ongoing oversight initiatives intended to improve  the department’s management of major acquisition programs.
Appendix I presents individual assessments of each of the 26 programs  we reviewed. These assessments include key information such as  projected funding levels, staffing profiles, and progress against schedule  and cost goals. Our objective for the 2-page assessments is to provide  decision makers a means to quickly gauge the programs’ progress and  their potential cost, schedule, performance, or funding risks. Appendix II  provides detailed information on our scope and methodology.
We conducted this performance audit from May 2016 to April 2017 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: To help manage its multi-billion dollar acquisition investments, DHS has  established policies and processes for acquisition management, test and  evaluation, and resource allocation. The department uses these policies  and processes to deliver systems that are intended to close critical  capability gaps, helping enable DHS to execute its missions and achieve  its goals.
subsections: section_title: Acquisition Management Policy
paragraphs: DHS policies and processes for managing its major acquisition programs  are primarily set forth in Acquisition Management Directive (MD) 102-01  and DHS Instruction Manual 102-01-001, Acquisition Management  Instruction/Guidebook. DHS issued the initial version of this directive in  November 2008 in an effort to establish an acquisition management  system that effectively provides required capability to operators in support  of the department’s missions. DHS’s Under Secretary for Management  (USM) is currently designated as the department’s Chief Acquisition  Officer and, as such, is responsible for managing the implementation of  the department’s acquisition policies.
DHS’s USM serves as the decision authority for the department’s largest  acquisition programs: those with LCCEs of $1 billion or greater.  Component Acquisition Executives—the most senior acquisition  management officials within each of DHS’s component agencies—may  be delegated decision authority for programs with cost estimates between  $300 million and less than $1 billion. Table 1 identifies how DHS has  categorized the 26 major acquisition programs we review in this report,  and table 7 in appendix II specifically identifies the programs within each  level.
DHS acquisition policy establishes that a major acquisition program’s  decision authority shall review the program at a series of five  predetermined Acquisition Decision Events (ADE) to assess whether the  major program is ready to proceed through the acquisition life-cycle  phases. Depending on the program, these ADEs can occur within months  of each other, or be spread over several years. Figure 1 depicts the  acquisition life cycle established in DHS acquisition policy.
An important aspect of an ADE event is the decision authority’s review  and approval of key acquisition documents. See table 2 for a description  of the type of key acquisition documents requiring department-level  approval before a program moves to the next acquisition phase.
DHS acquisition policy establishes that the APB is the agreement  between program, component, and department-level officials establishing  how systems will perform, when they will be delivered, and what they will  cost. Specifically, the APB establishes a program’s schedule, costs, and  KPPs. DHS defines KPPs as a program’s most important and non- negotiable requirements that a system must meet to fulfill its fundamental  purpose. For example, a KPP for an aircraft may be airspeed and a KPP  for a surveillance system may be detection range.
The APB schedule, costs, and KPPs are defined in terms of an objective  and minimum threshold value. According to DHS policy, if a program fails  to meet any schedule, cost, or performance threshold approved in the  APB, it is considered to be in breach. Programs in breach are required to  notify their acquisition decision authority and develop a remediation plan  that outlines a time frame for the program to return to its APB parameters,  re-baseline—that is, establish new schedule, cost, or performance  goals—or have a DHS-led program review that results in  recommendations for a revised baseline.
In addition to the acquisition decision authority, other bodies and senior  officials support DHS’s acquisition management function:
The Acquisition Review Board (ARB) reviews major acquisition  programs for proper management, oversight, accountability, and  alignment with the department’s strategic functions at ADEs and other  meetings as needed. The ARB is chaired by the acquisition decision  authority or a designee and consists of individuals who manage  DHS’s mission objectives, resources, and contracts.
The Office of Program Accountability and Risk Management  (PARM) is responsible for DHS’s overall acquisition governance  process, supports the ARB, and reports directly to the USM. PARM  develops and updates program management policies and practices,  reviews major programs, provides guidance for workforce planning  activities, provides support to program managers, and collects  program performance data.
Component agencies, such as U.S. Customs and Border Protection  (CBP), the Transportation Security Administration (TSA), and the U.S.  Coast Guard (USCG) sponsor specific acquisition programs. The 26  programs we review in this report are sponsored by eight component  agencies.
Component Acquisition Executives within the components are  responsible for overseeing the execution of their respective  portfolios.
Program management offices, also within the components, are  responsible for planning and executing DHS’s individual  programs. They are expected to do so within the cost, schedule,  and performance parameters established in their APBs. If they  cannot do so, programs are considered to be in breach and must  take specific steps, as noted above.
Figure 2 depicts the relationship between acquisition managers at the  department, component, and program level.
section_title: Test and Evaluation Policy
paragraphs: In May 2009, DHS established policies and processes for testing the  capabilities delivered by the department’s major acquisition programs.  The primary purpose of test and evaluation is to provide timely, accurate  information to managers, decision makers, and other stakeholders to  reduce programmatic, financial, schedule, and performance risk. We  provide an overview of each of the 26 programs’ test activities in the  individual program assessments, presented in appendix I.
DHS testing policy assigns specific responsibilities to particular  individuals and entities throughout the department:
Program managers have overall responsibility for planning and  executing their programs’ testing strategies. They are responsible for  scheduling and funding test activities and delivering systems for  testing. They are also responsible for controlling developmental  testing. Programs use developmental testing to assist in the  development and maturation of products, product elements, or  manufacturing or support processes. Developmental testing includes  engineering-type tests used to verify that design risks are minimized,  substantiate achievement of contract technical performance, and  certify readiness for operational testing.
Operational test agents (OTA) are responsible for planning,  conducting, and reporting on operational testing, which is intended to  identify whether a system can meet its KPPs and provide the  acquisition decision authority with an evaluation of the operational  effectiveness and suitability of a system in a realistic environment.  Operational effectiveness refers to the overall ability of a system to  provide desired capability when used by representative personnel.  Operational suitability refers to the degree to which a system can be  placed in field use and sustained satisfactorily. The OTAs may be  organic to the component, another government agency, or a  contractor, but must be independent of the developer in order to  present credible, objective, and unbiased conclusions. For example,  the U.S. Navy Commander, Operational Test and Evaluation Force is  the OTA for the USCG National Security Cutter (NSC) program.
The Director, Office of Test and Evaluation (DOT&E) is responsible  for approving major acquisition programs’ OTAs, operational test  plans, and Test and Evaluation Master Plans (TEMP). A program’s  TEMP must describe the developmental and operational testing  needed to determine technical performance, and operational  effectiveness and suitability. As appropriate, DOT&E is also  responsible for participating in operational test readiness reviews,  observing operational tests, reviewing OTAs’ reports, and assessing  the reports. Prior to a program’s ADE 3, DOT&E provides the  program’s acquisition decision authority a letter of assessment that  includes an appraisal of the program’s operational test, a concurrence  or non-concurrence with the OTA’s evaluation, and any further  independent analysis.
As an acquisition program proceeds through its life cycle, the testing  emphasis moves gradually from developmental testing to operational  testing. See figure 3.
section_title: Resource Allocation Process
paragraphs: DHS has established a planning, programming, budgeting, and execution  (PPBE) process to allocate resources to acquisition programs and other  entities throughout the department. DHS’s PPBE process produces the  multi-year funding plans presented in the FYHSP, a database that  contains, among other things, 5-year funding plans for DHS’s major  acquisition programs. DHS guidance states that the 5-year plans in the  FYHSP should allow the department to achieve its goals more efficiently  than an incremental approach based on 1-year plans. DHS guidance also  states that the FYHSP articulates how the department will achieve its  strategic goals within fiscal constraints.
According to DHS guidance, at the outset of the annual PPBE process,  the department’s Office of Policy and Chief Financial Officer (CFO)  should provide planning and fiscal guidance, respectively, to the  department’s component agencies. In accordance with this guidance, the  components should submit 5-year funding plans to the CFO; these plans  are subsequently reviewed by DHS’s senior leaders, including the DHS  Secretary and Deputy Secretary. DHS’s senior leaders are expected to  modify the plans in accordance with their priorities and assessments, and  they document their decisions in formal resource allocation decision  memorandums. DHS submits the revised funding plans to the Office of  Management and Budget, which uses them to inform the President’s  annual budget request—a document sent to Congress requesting new  budget authority for federal programs, among other things. In some  cases, the funding appropriated to certain accounts in a given fiscal year  can be carried over to subsequent fiscal years. Figure 4 depicts DHS’s  annual PPBE process.
Federal law requires DHS to submit an annual FYHSP report to Congress  at or about the same time as the President’s budget request. This report  presents the 5-year funding plans in the FYHSP database at that time.
Within DHS’s Office of the CFO, the Office of Program Analysis and  Evaluation is responsible for establishing policies for the PPBE process  and overseeing the development of the FYHSP. In this role, the Office of  Program Analysis and Evaluation reviews the components’ 5-year funding  plans, advises DHS’s senior leaders on resource allocation issues,  maintains the FYHSP database, and submits the annual FYHSP report to  Congress.
section_title: During 2016, More than Half of 26 Programs Were on Track to Meet Their Schedules and Cost Goals
paragraphs: For the first time since we began our annual assessments of DHS’s major  acquisition programs, all of the programs included in our review had a  department-approved baseline. This allowed us to analyze schedule and  cost changes across the portfolio of the 26 programs we assessed, which  provides a foundation for measuring DHS’s acquisition performance going  forward. From January 2016 to January 2017, 17 of the 26 programs we  assessed were on track to meet their schedule and cost goals, including 2  that experienced either a schedule acceleration or cost decrease.  However, 7 of these 17 programs established their goals for the first time  since our last review and 9 others had previously revised their goals. The  remaining 9 of the 26 programs experienced schedule slips, including 4  that also experienced cost growth. The change in schedule for a key  program acquisition milestone in 2016 ranged from a 21-month  acceleration to a 75-month delay, which resulted in an average increase  of 6 months across the portfolio. Additionally, although 1 program had a  drop in costs, overall the total acquisition cost across the portfolio  increased by $988 million—or 1.6 percent—and the total LCCE across  the portfolio increased by nearly $1.6 billion—or 0.8 percent. The overall  schedule and cost changes were largely driven by increases experienced  by a few programs. For example, the full operational capability (FOC)  date for TSA’s Technology Infrastructure Modernization (TIM) program  slipped by more than 6 years when the program revised its acquisition  strategy—significantly delaying the delivery of some services to end  users.
Table 3 summarizes our findings and highlights those programs with  schedule or cost increases. We present more detailed information after  the table and in the individual assessments in appendix I.
subsections: section_title: During 2016, 17 Programs Were on Track
paragraphs: From January 2016 to January 2017, 17 programs were on track to meet  their schedules or cost goals. Eight of the 17 programs were on track  against their initial schedule and cost goals; that is, the schedules and  cost estimates in the baseline DHS leadership initially approved after the  department’s acquisition policy went into effect in November 2008. The  other 9 programs had re-baselined prior to January 2016 and were on  track against revised schedules and cost estimates that reflected past  schedule slips, cost growth, or both. However, most of the programs on  track in 2016 identified risks that may lead to schedule slips or cost  growth in the future.
subsections: section_title: On Track against Initial Baselines
paragraphs: Of the 8 programs on track against the schedules and cost goals in their  initial baselines, only 1 program received DHS approval of its initial  baseline prior to December 2015. Six of the remaining programs had  operated for several years without a DHS-approved baseline, which, in  addition to decreasing oversight, also increased the risk of end users not  getting required capabilities on time or at cost. For example, DHS  leadership approved the initial APB for CBP’s Non-Intrusive Inspection  (NII) Systems Program in January 2016, which was more than 13 years  after the program deployed initial capabilities to end users. This means  that, even though capabilities were delivered to end users, the program  had not followed the department’s November 2008 acquisition policy.  Since the NII Systems Program’s initial APB was approved, the program’s  acquisition cost estimate decreased by $190 million and its LCCE  decreased by $315 million. Program officials attributed these decreases  to achieving a reduction in NII system purchase and maintenance costs  and the replacement of some NII systems that were costly to maintain.  DHS leadership also recently approved the initial APB for a newer  program—the National Protection and Programs Directorate’s (NPPD)  Homeland Advanced Recognition Technology (HART)—in April 2016  when it entered the Obtain phase. Only 1 program—the Science and  Technology Directorate’s (S&T) National Bio and Agro-Defense Facility  (NBAF)—that we found was on track against its initial baselines in 2015  remained on track against its initial baselines in 2016.
section_title: On Track against Revised Baselines
paragraphs: For context, because many baselines had been approved only recently,  we also assessed the extent to which programs that were on track in  2016 had previously experienced problems. We found that 9 of these  programs had previously experienced schedule slips, cost growth, or  both. Specifically, all 9 of these programs had milestones that slipped an  average of 4.5 years, for a variety of reasons.
In addition, 6 of these 9 programs also experienced cost growth prior to  2016; in total, acquisition costs increased by $5 billion and LCCEs  increased by nearly $17 billion. Examples of programs with no changes  during 2016, but that had experienced past schedule slips and cost  growth, follow.
CBP’s Integrated Fixed Towers (IFT) program’s FOC date previously  slipped 5 years, which officials attributed to delays in awarding  contracts and to funding shortfalls.
From September 2010 to September 2014, NPPD’s Next Generation  Networks Priority Services (NGN-PS) program’s acquisition cost  increased by $447 million and LCCE increased by $386 million when  officials accounted for capabilities delivered under the voice phase’s  second increment. From September 2014 to August 2015, the  program’s acquisition costs subsequently decreased by $153 million  based on a refinement of the estimate, but the LCCE increased by an  additional $100 million when officials included all sustainment costs  funded by a separate program—NPPD’s Priority Telecommunications  Services program, which assumes responsibility for sustaining NGN- PS capabilities once they become operational—at the direction of  DHS headquarters.
On the other hand, 2 USCG programs—the Medium Range Surveillance  (MRS) Aircraft and Command, Control, Communications, Computers,  Intelligence, Surveillance, and Reconnaissance (C4ISR)—that  experienced past problems reported positive changes in 2016.
In August 2016, DHS approved a revised APB for the MRS program  that establishes initial schedule and cost goals for the restructured  program. Specifically, the department paused the number of HC-144A  aircraft at the 18 already procured and accounted for the transfer of  14 C-27J aircraft from the U.S. Air Force as directed by Congress in  fiscal year 2014. Prior to this restructuring, the MRS program’s FOC  date slipped from September 2020 to September 2025 when the  USCG reduced the number of HC-144A aircraft it planned to procure  annually in response to funding constraints. In addition, the program’s  LCCE increased by $16.4 billion when the USCG accounted for costs  over this additional 5-year period, among other things.
For C4ISR, USCG officials stated they now plan to complete the  transition away from using contractor-owned proprietary software by  the end of calendar year 2017, which is 21 months earlier than the  program’s revised APB. However, if completed by the new date, this  transition would still occur more than 5 years later than the C4ISR  program initially planned.
section_title: Risks That May Cause Schedule Slips, Cost Growth, or Both in the Future
paragraphs: Officials from most of the 17 programs on track in 2016 identified risks  that could cause schedule slips, cost growth, or both in the future. These  risks include testing issues, funding gaps, and technical challenges,  among other factors. For example, NPPD’s Continuous Diagnostics &  Mitigation (CDM) program is in the process of re-baselining to address  implementation challenges discovered in 2016, which officials anticipate  will increase the program’s cost and lead to potential schedule slips for  future capabilities. In addition, the USCG Long Range Surveillance  Aircraft is currently on track to meet schedule and cost goals, but  experienced significant cost increases and schedule slips from 2009 to  2012, which USCG officials primarily attributed to the decision to procure  additional HC-130J aircraft. Officials have said that the USCG would need  to acquire one to two HC-130J aircraft per year in order to meet the  program’s FOC date of March 2027. If the remaining aircraft are not  delivered at this rate, the program’s schedule could slip further. USCG  officials said the delivery rate is dependent on the amount of funding the  program receives, as the USCG has historically received HC-130Js  without including them in their budget requests.
section_title: Programs Not on Track During 2016
paragraphs: From January 2016 to January 2017, 9 of the 26 programs we assessed  experienced schedule slips, 4 of which also experienced cost growth. The  extent of these changes constituted breaches of schedules, cost goals, or  both, for 6 of the 9 programs. For these 9 programs, the average  schedule slip of 1.6 years was largely driven by changes in TSA’s TIM  program. As far as cost growth, increases of $1.2 billion and $1.9 billion  for acquisition and life-cycle costs, respectively, were also essentially  driven by one program, TSA’s Electronic Baggage Screening Program  (EBSP). More details follow.
subsections: section_title: Programs with Schedule Slips during 2016
paragraphs: During 2016, 9 of the 26 programs in our review had at least one major  acquisition milestone that slipped for various reasons. Across these  programs, the average schedule slip was 1.6 years, but that average was  significantly driven by a more than 6-year delay in the TSA’s TIM  program, which revised its acquisition strategy. Figure 5 identifies the 9  programs that experienced schedule slips and the extent to which their  major milestones slipped in 2016, as well as—for additional context—in  prior years. While there are various reasons for the schedule delays, the  effect is that end users may not have gotten needed capabilities when  they originally anticipated.
We identified several reasons why these key milestones slipped,  including the following:
New strategies or requirements: For example, TSA’s TIM program  re-baselined in September 2016 to reflect a new acquisition strategy  that is intended to address past program execution challenges that led  to the program breaching its initial APB in 2014. TIM’s new strategy  also includes integration with the Transportation Vetting System and  support for additional programs, such as TSA’s Pre-Check.  Additionally, TSA’s Passenger Screening Program (PSP) declared an  APB schedule breach in January 2016 because of delays in  incorporating new cybersecurity requirements in the Credential  Authentication Technology system prior to completing operational  testing.
Technical challenges: For example, the USCG’s H-65  conversion/sustainment program declared a schedule breach in  November 2016 after experiencing significant delays in developing a  portion of the avionics upgrades for the H-65, which officials primarily  attributed to an underestimation of the technical effort necessary to  meet requirements. As a result, the avionics initial production decision  has been delayed until September 2018, nearly 5 years later than  initially planned.
We elaborate on the reasons for all 9 programs’ schedule slips in the  individual assessments in appendix I.
section_title: Programs with Cost Growth during 2016
paragraphs: During 2016, 4 of the 26 programs in our review experienced growth in  both their acquisition cost estimates and LCCEs. In total, acquisition cost  estimates increased by a total of $1.2 billion and LCCEs increased by a  total of $1.9 billion, which reflects an approximately 8 percent increase in  both estimates when calculated across these 4 programs. The cost  growth is almost entirely driven by increases to TSA’s EBSP cost  thresholds to account for risk in its new estimate that reflects anticipated  funding shortfalls and planning for program succession. Table 4 identifies  the 4 programs with cost growth and the extent to which their estimates  increased in 2016.
We identified a number of reasons why cost estimates increased in 2016,  including the following:
Revised acquisition strategy: For example, DHS leadership  approved new APBs for TSA’s EBSP and TIM programs in May 2016  and September 2016, respectively, which increased the programs’  cost thresholds over their previous estimates to better account for  potential programmatic risks. EBSP updated its cost estimate in July  2015 in response to funding constraints and plans for a new  acquisition program to succeed EBSP in fiscal year 2028. In addition,  the TIM program’s cost estimates changed from its September 2015  estimate when it adopted its new acquisition strategy, as noted above.  Specifically, TIM’s acquisition cost estimate increased and LCCE  decreased. However, the establishment of new APB cost thresholds  in September 2016 that accounted for implementation risks  associated with the program’s new strategy resulted in an overall  increase in both estimates.
More realistic cost estimates: For example, officials from CBP’s  Automated Commercial Environment (ACE) program said the  program’s initial cost estimate underestimated the number and size of  the required development teams and included expected savings from  moving to a cloud environment. In addition, officials from the  Immigration and Customs Enforcement’s (ICE) TECS Modernization  program attributed their program’s acquisition increase to including  actuals for a contract awarded in 2016.
We elaborate on the reasons for all 4 programs’ cost growth in the  individual assessments in appendix I.
section_title: Funding Gaps Remain a Risk for Some Programs as DHS Continues to Address Affordability Issues
paragraphs: Some DHS programs continue to face funding challenges, which  increases the likelihood that they will cost more and take longer to deliver  capabilities to end users than expected. We found that 18 of the 26  programs we assessed in this review are projected to experience life- cycle funding gaps exceeding 10 percent through fiscal year 2021.  While DHS has continued to take steps to improve the affordability of its  major acquisition programs, this is 8 more programs than we found in our  prior review. In March 2016, we found that 10 of the 25 programs had a  projected 6-year funding gap. Similar to last year, we compared the  programs’ funding plans—documented in the FYHSP report to  Congress—to the programs’ yearly LCCEs in order to identify any  projected funding gaps for fiscal year 2016 through fiscal year 2021. We  also identified the funding from previous years that programs brought into  fiscal year 2016—known as carryover funding—to determine the extent to  which that carryover could offset any funding gaps.
Based on this analysis, we found various reasons for programs’ projected  funding gaps, such as unfunded activities, new requirements, or that a  sub-set of programs’ annual costs were funded by organizations outside  the program. In addition, the USCG’s cost estimates include operations  and maintenance (O&M) costs—which usually represent a majority of  program costs—but their funding plans do not. We first identified this  FYHSP reporting inconsistency in April 2015 and recommended that DHS  account for the O&M funding the USCG plans to allocate to each of its  acquisition programs in its future report. DHS concurred with the  recommendation, but the USCG has yet to take action. USCG officials  said they cannot resolve this issue until the USCG updates its financial  management system and transitions to DHS’s common appropriations  account structure, which they anticipate will occur in fiscal year 2020.  Similarly, DHS officials told us that the next FYSHP report, which will be  the first to include CBP’s Multi-Role Enforcement Aircraft (MEA) and  Medium Lift Helicopter (UH-60) as distinct programs, will also not include  funding allocated to cover these programs’ O&M costs because these  costs are funded through a separate, central account for all of CBP’s air  and marine assets. As a result of these reporting issues, any calculated  projected funding gap would likely be overstated for 9 USCG and CBP  programs we assessed.
Aside from these specific O&M issues, program officials identified  strategies to mitigate projected funding gaps, such as the following:
Using alternative funding sources: For example, TSA’s TIM  program anticipates receiving fees from vetting programs that will  cover the program’s anticipated funding shortfall;
Program tradeoffs: For example, officials from three CBP programs  noted that they planned to address their projected funding gaps with  actions such as performing only minimum maintenance, prioritizing  upgrades against operational needs, and service life extension efforts;  and  Increased funding allocation: For example, NPPD identified that  DHS plans to program additional funding to the HART program from  fiscal year 2017 through 2021.
However, officials from 7 programs said that projected funding gaps could  cause future program execution challenges, such as schedule slips or  cost growth. For example, officials from S&T’s NBAF program said that  although they were working with the component to mitigate a $38 million  funding gap, affordability challenges could cause delays in the operational  stand-up of the facility. We elaborate on programs’ projected funding  gaps in the individual program assessments in appendix I.
DHS officials recognize the need to address program affordability and,  since our last review, have continued to take actions through the  department’s acquisition management and annual budget development  processes to do so. For example, in March 2016, we found that DHS had  initiated a process to assess and address affordability trade-offs based on  a June 2014 requirement that components certify programs’ affordability  prior to ADEs. We also made several recommendations at that time to  enhance DHS leadership’s efforts to improve the affordability of the  department’s major acquisition portfolio. For example, we recommended  that components ensure their affordability certifications include details  such as cost estimates, funding streams, and the monetary value of  proposed tradeoffs. We also recommended that DHS review the  affordability of 11 programs that had not had an ADE since DHS’s new  funding certification requirements went into effect, and consider holding  ARBs to discuss the affordability of these programs, as necessary. DHS  concurred with both recommendations and now requires components to  provide explicit details on affordability prior to ARBs, as necessary, as  well as to submit more detailed information as a part of the annual budget  process. For example, to develop the President’s fiscal year 2018 budget  request, DHS required major acquisition programs to submit detailed data  on program affordability, such as identifying all funding sources, a  comparison to the program’s most recent cost estimate, and the impact of  any funding gaps on program schedule, cost, or performance. As a result,  officials said that they were able to address any potential funding gaps for  major acquisition programs through this process and determined that no  programs required an ARB specifically to discuss affordability in response  to our March 2016 recommendation.
In the near term, DHS officials said that they plan to publish programs’  annual acquisition cost estimates and any projected acquisition funding  gap in the FYHSP report for fiscal years 2018-2022, which had not yet  been submitted to Congress at the time of our review. They do not,  however, plan at this point to present annual LCCE gaps as we previously  recommended due to a lack of reliable information. While presenting  acquisition cost estimates and any projected funding gaps are important,  we continue to believe that DHS should also reflect annual LCCEs and  any overall funding gaps—including O&M data, not just acquisition—in its  future FYHSP reports. Adding this information would provide Congress  valuable insights into DHS’s total funding needs and clarify the potential  funding gaps for major acquisition programs. DHS officials acknowledged  the importance of communicating overall program funding gaps in the  FYHSP, including O&M data. They said that DHS’s efforts to implement a  common appropriations account structure across the department should  help them present this information in the future. We continue to monitor  DHS’s actions to address program affordability and, at the request of  Congress, have initiated a review to assess the extent to which DHS has  accounted for program’s O&M costs and funding.
section_title: Programs Generally Did Not Meet All KPPs before Deploying Capabilities and Late Requirements Definition May Affect Program Execution
paragraphs: Fourteen of the 26 programs we reviewed deployed capabilities prior to  meeting all of their department-approved KPPs—the most important  requirements that a system must meet to fulfill its purpose. As a result,  DHS faces increased risk of fielding capabilities that do not work as  intended. In some cases, it may be appropriate for programs to deploy  capabilities prior to meeting their KPPs, such as systems that develop  and test their capabilities incrementally. However, DHS’s acquisition  policy requires programs to conduct operational testing, which is intended  to demonstrate program performance, prior to receiving approval to  pursue full-rate production or to transition into sustainment. Program  officials identified multiple reasons that KPPs have not been met, such as  programs had not yet tested the KPPs or KPPs were poorly defined. We  found that DHS’s acquisition policy requires programs to establish an  initial baseline—including defined KPPs—prior to gaining full knowledge  about the program’s technical requirements. This timing is counter to  acquisition best practices, and may potentially cause programs to  experience cost growth, schedule slips, and inconsistent performance if  requirements are not firmly established at the time the baseline is set.
subsections: section_title: More than Half of 26 Programs Have Deployed Capabilities without Meeting All KPPs
paragraphs: Fourteen of the 26 programs we reviewed have deployed capabilities  prior to meeting all of their department-approved KPPs. All but 3 of these  14 programs have conducted some type of operational testing.  Programs evaluate KPPs during operational testing, which is intended to  help DHS determine how well a system will provide the desired capability  before the system is fully deployed. DHS’s acquisition policy requires  programs to conduct operational testing prior to receiving ADE 3  approval—the point where programs are authorized to pursue full-rate  production or to transition into sustainment—but the policy also allows  programs to initiate limited deployments of capabilities to support  operational testing under certain circumstances. In some cases,  programs deploy and test capabilities incrementally—an approach  commonly used by information technology (IT) programs. For example,  NPPD’s CDM program plans to provide sensors and tools for  strengthening the cybersecurity of the federal government’s computer  networks through a series of phases, which have their own KPPs that will  be deployed and tested separately. Of the 26 programs we assessed, 9  have met all of their KPPs and 3 are still relatively early in the acquisition  life cycle and have not yet deployed or operationally tested any  capabilities.
Table 5 identifies all 26 programs we assessed, whether they have  deployed or operationally assessed or tested capabilities, and their  progress in meeting department-approved KPPs as of January 2017.
DHS officials identified several reasons why programs have deployed  capabilities, but not met all of their department-approved KPPs. For  example, programs had not yet tested the KPPs or failed to meet the  KPPs when they were tested. Programs identified multiple reasons that  KPPs hadn’t been met, which are presented in figure 6 along with the  number of programs that identified them.
Examples for each of the categories of reasons that programs have not  met KPPs are presented below:
The program has not yet tested the KPP. For example, the USCG’s  C4ISR program no longer plans to independently conduct operational  testing against its KPPs and will instead test C4ISR systems in  conjunction with other USCG planes and vessels for which they are  installed. However, the C4ISR system’s KPPs were not specifically  assessed during prior HC-144, Fast Response Cutter (FRC), and  NSC tests. Future testing will focus only on the ability of the C4ISR  system to meet the NSC’s KPPs during the NSC’s follow-on  operational testing in fiscal years 2017 and 2018. This follow-on  testing, however, will only test one of the C4ISR system’s six KPPs.
The program failed to meet KPPs during testing, or testing was  not adequate to determine KPP status. For example, the U.S.  Citizenship and Immigration Services’ (USCIS) Transformation  program conducted an operational assessment on a sub-set of  deployed capabilities from March 2015 to August 2015. This  assessment evaluated seven of the program’s KPPs, and the program  failed to meet one of them—the reliability KPP—because of the  frequency of system failures. In another example, the Federal  Emergency Management Agency’s (FEMA) Logistics Supply Chain  Management System (LSCMS) program conducted operational  testing throughout calendar year 2013, but DOT&E concluded that  this testing was not adequate to determine whether the program had  met its KPPs. This program subsequently met two of its seven KPPs  through a performance test of a software release, and plans to  conduct additional operational testing in March 2018 once it  completes development of additional capabilities.
The KPPs are not ready to be tested because the required  technology or system capabilities are not yet available, or  because capabilities are being deployed and tested  incrementally. For example, the USCG’s MRS program cannot  demonstrate the C-27J’s seven KPPs until it installs an entire mission  system on the aircraft. Additionally, the program will not be able to  demonstrate two of these KPPs—the detection and interoperability  KPPs—identified in the joint operational requirements document (joint  with CBP) for the C-27J aircraft because the mission system  technology needed is not yet commercially available for this aircraft. In  April 2016, the USCG received approval to defer these capabilities  until the technology required to meet the detection KPP becomes  commercially available. DHS has also directed the program to revisit  requirements and, if appropriate, to initiate updating them prior to the  program’s next acquisition milestone. In another example, NPPD  National Cybersecurity Protection System (NCPS) officials told us that  the program has not yet met the five KPPs related to its Block 2.2  capabilities because these capabilities are still early in the  development phase and are not yet ready to be tested. The NCPS  program has met a majority of its KPPs for capabilities that have  previously been deployed and tested.
The KPP is poorly defined. For example, the USCG’s NSC program  indicated challenges in meeting three of its KPPs related to cutter- boat deployment in rough seas because the USCG and its OTA have  different interpretations of the cutter-boat requirements. In January  2016, we recommended the NSC program office clarify the KPPs for  the cutter boats, with which the USCG concurred. As of January  2017, the USCG was working on a resolution.
section_title: Setting Baselines before Establishing Technical Solution May Contribute to Program Execution Challenges
paragraphs: While we have previously found that DHS’s acquisition policy is sound, at  a more granular level we found an area for improvement. The policy  requires programs to obtain department-level approval for initial APBs— including KPPs, schedules, and cost goals—at ADE 2A, that is, prior to  gaining full knowledge about the program’s technical requirements. This  sequence is not consistent with acquisition best practices. GAO’s  acquisition best practices state that programs should pursue a  knowledge-based acquisition approach that ensures program’s needs are  matched with available resources—such as technical and engineering  knowledge, time, and funding—prior to starting product development.  While these initial APBs include KPPs that identify operational  requirements defined by the user prior to ADE 2A, programs have not yet  decomposed those KPPs into specific technical requirements or  conducted key engineering reviews to develop critical knowledge about  whether the proposed solution meets the user’s needs. This happens  after the baseline is approved and programs are officially initiated. Key  engineering reviews that should be conducted prior to establishing  program baselines include the following:
System definition review: establishes a functional baseline, which  identifies what the system is to perform.
Preliminary design review: assesses the preliminary design of the  system and determines whether the program is prepared to start  detailed design and test development.
A third review, called the critical design review, is appropriately conducted  after program initiation, which is consistent with acquisition best practices.  This is a key engineering review that demonstrates whether the system’s  final design is sufficiently complete to begin production.
Figure 7 compares GAO’s acquisition best practices to DHS’s acquisition  and systems engineering life-cycle phases. As shown, the system  definition and preliminary design reviews are to the left of program  initiation according to best practices, but are to the right of program  initiation within DHS’s acquisition life cycle.
By initiating programs without a well-developed understanding of system  needs, DHS increases the likelihood that programs will change their user- defined KPPs, costs, or schedules after establishing their baselines.  Changes such as this can be viewed as a natural occurrence as  requirements are better defined. For example, officials from NPPD’s  HART program told us that the cost and schedule goals in the program’s  approved APB may change once they award the initial contract and  receive the contractor’s technical solution for meeting the program’s  already-established KPPs. In addition, we found in March 2016 that  several programs had changed KPPs at least once since DHS’s current  acquisition policy went into effect in 2008, and that KPP changes were  associated with schedule slips and cost growth. We also found that 9 of the 12 programs that changed KPPs attributed those changes to poorly  defined or unattainable requirements, and officials from 12 programs said  that they may change KPPs in the future. Since March 2016, at least one  additional program—TSA’s TIM program—made changes to its KPPs and  we anticipate that more programs will need to make changes to KPPs in  the future to better reflect system requirements. For example, officials  from ICE’s TECS Modernization program said that they will not be able to  demonstrate the program’s concurrent user KPP because the minimum  goal far exceeds the current number of system users.
DHS leadership previously acknowledged that the department has had  difficulty defining KPPs, and senior DHS officials told us in December  2016 that they are continuing efforts to help programs define KPPs more  effectively. However, officials also noted that there is a lack of systems  engineering capability within the agency, which is an ongoing challenge.  Officials further agreed there is room to refine the acquisition processes  and told us that they are working with S&T to better align systems  engineering efforts with the acquisition life cycle. For example, DHS  officials said that they are working to adapt the acquisition processes for  agile development—the department’s preferred development approach  for IT programs—which is currently being piloted by some DHS major  acquisition programs. While DHS’s efforts may allow for increased S&T  involvement in the acquisition process, placement of the requirements  definition and key engineering reviews earlier in the acquisition life cycle  could yield better outcomes regardless of the development approach  pursued by programs. Without also matching the program’s technical  requirements and resources at the time KPPs are defined, DHS increases  the risk that programs will continue to experience execution challenges,  including cost growth, schedule slips, and inconsistent performance as  requirements change after programs are initiated. By accumulating more  knowledge before programs establish baselines and begin development,  per acquisition best practices, DHS can place major programs in a better  position to succeed, which ultimately means an increased likelihood of  end users obtaining the capabilities they need within expected costs and  time frames.
section_title: DHS Has Taken Steps to Strengthen Management of Its Major Acquisition Programs, but Leadership Decisions Not Always Fully Documented
paragraphs: In 2016, DHS made positive strides to strengthen its management of  major acquisition programs. For example, DHS established new  processes for assessing programs’ staffing needs and monitoring major  acquisition program progress. While promising, it is too soon to tell if  these processes will contribute to positive outcomes because DHS is still  working on how to implement them and use them to support more  forward-looking planning decisions. In addition, DHS revised the  instruction for implementing the department’s acquisition policy to reflect  changes made since the previous version was issued—some of which  reflect past GAO recommendations. In addition, the new instruction  includes changes to the documentation approvals needed before  programs advance through the acquisition life cycle and to DHS’s breach  policy.
Our analysis indicates that DHS made progress in implementing these  documentation requirements more consistently in 2016 than we have  found in the past. For example, DHS leadership generally approved all  the required key acquisition documentation prior to approving programs to  proceed through the acquisition process. However, DHS leadership could  better document its rationale for decisions made at ADEs to increase  insight the department and external stakeholders have into acquisition  management decisions. Further, we also found that no programs in our  review had reported performance breaches and that DHS’s policy does  not clearly define at what point not meeting KPPs constitutes a  performance breach. Without insight into potential performance issues  identified through breaches, DHS is at risk of fielding capabilities that do  not work as intended.
subsections: section_title: New DHS Processes Intended to Improve Acquisition Management
paragraphs: DHS has established new processes that could improve acquisition  management by addressing longstanding issues related to acquisition  workforce shortfalls and program execution challenges we have identified  in the past. Specifically, DHS revised its process for assessing major  acquisition program staffing needs and established a process to monitor  major acquisition program progress across a variety of factors and  categories DHS deemed were important for successful program  execution. However, it is too early to tell what impact these efforts will  have on program outcomes because DHS is still developing  implementation plans for these new processes.
subsections: section_title: Acquisition Program Staffing Assessments
paragraphs: We have highlighted DHS acquisition management issues in our high-risk  updates since 2005—most recently in February 2017—and identified five  outcomes that could strengthen DHS’s management of its acquisitions.  One of these outcomes is that DHS assess and address whether  sufficient numbers of trained acquisition personnel are in place at the  department and component levels. In addition, we previously found that  staffing shortfalls can impact a program’s ability to execute and may  introduce risks leading to schedule slips, cost growth, or both in the  future. For example, in March 2016, we found that staffing shortfalls  limited NPPD NCPS’s ability to perform testing, oversee contractors, and  manage finances. In response, DHS’s PARM initiated a process for  assessing the staffing needs of its major acquisition programs in fiscal  year 2014 and conducted a second assessment in fiscal year 2015.  PARM collected key information such as the total staffing needed— including positions identified as critical—actual staffing levels, and  mitigation strategies to fill any vacancies, among other items. However,  these assessments collected retrospective information on whether  programs were sufficiently staffed in those fiscal years and did not collect  current or future program staffing need data. In addition, some of the  fiscal year 2015 staffing assessments were not approved until January  2017, limiting the usefulness of the assessments given that the data was  over a year old.
In June 2016, the department began tracking only critical position  vacancies rather than assessing all acquisition-related positions. PARM  officials said they made this change to capture staffing data in a timely  manner, document progress in filling key staffing gaps, and help the  department mitigate remaining gaps. Consequently, some programs were  assessed as being sufficiently staffed because they had few or no critical  position vacancies, despite these programs identifying shortfalls in the  programs’ total staffing need. For example, NPPD’s CDM program  reported a total staffing need of 51 full-time positions, 19 of which were  considered critical. NPPD also reported that CDM had only 1 vacancy out  of its 19 critical positions. However, CDM officials told us they had only 31  of the 51 staff they needed in total, which represents a 39 percent shortfall overall. We present more information on programs’ staffing  profiles in the individual program assessments in appendix I.
After we raised questions about whether this approach would limit  department insight into programs’ total staffing needs in October 2016,  PARM revisited its decision to track only critical position vacancies and  revised its approach for future staffing assessments. In December 2016,  DHS approved a new staffing instruction that will require major acquisition  programs to submit and annually update staffing plans identifying total  staffing needs, but also track critical position vacancies quarterly, among  other things. According to PARM officials, the agency is developing  guidance and templates intended to bring clarity to the new policy and  limit potential inconsistencies in interpretation across the programs, such  as what positions programs determine to be critical.
In addition, the new staffing instruction requires programs to develop a  multi-year staffing plan that identifies future staffing needs. PARM officials  told us that they plan to pilot the new staffing assessment process in 2017  and hope to complete the first assessment in time to inform the  department’s fiscal year 2019 budget request. If implemented as  intended, the new staffing assessment process would improve PARM’s  insight into major acquisition program staffing needs and assist the  department in developing mitigation strategies to address current staffing  gaps and planning for future staffing needs.
section_title: Acquisition Program Health Assessment
paragraphs: In October 2016, DHS established the Acquisition Program Health  Assessment (APHA), a process intended to monitor major acquisition  programs’ progress. PARM initiated efforts to develop the APHA in  February 2015 after DHS’s Deputy USM directed it to lead development  of a holistic, objective, repeatable process for evaluating the department’s  major acquisition programs and reducing duplicative reports. PARM  established a working group with representatives from all ARB  stakeholder organizations—such as the CFO, Chief Information Officer  (CIO), Chief Procurement Officer, DOT&E, and the Joint Requirements  Council (JRC)—and each of DHS’s operational components, which  developed a weighted assessment methodology. The APHA assessment  methodology consists of a number of factors within several categories,  such as program management, financial management, contract  management, performance, and human capital, which DHS deemed were  important for successful program execution. Each factor was defined and  is rated by the stakeholder with primary responsibility for that area within  the department. For example, DOT&E defines and rates programs on the  factor related to operational testing, whereas the CFO defines and rates  programs on the factor related to LCCEs. The factor ratings are then used  to develop category ratings, which in turn, feed into a program’s single  overall APHA score.
DHS is still working on its implementation and it will take time to  determine whether it will be an effective acquisition management tool.  According to PARM officials, they plan to utilize the APHA results to  inform DHS leadership about major acquisition programs through monthly  briefings and quarterly reports, as well as reports to external  stakeholders. For example, the APHA will inform a section of the  department’s annual Comprehensive Acquisition Status Report to the  Senate and House appropriations committees starting in fiscal year 2017  and will provide the score that the DHS CIO reports for each major  acquisition program on the Office of Management and Budget’s IT  Dashboard.
However, senior DHS officials noted that while the department has made  progress in developing APHA, they still have work to do to refine and  strengthen the process, such as determining what constitutes a good  APHA score and turning it into a leading indicator of program health  versus a lagging indicator. DHS officials have shared information on the  department’s efforts to establish the APHA process with us, and we will  continue to review DHS’s efforts to evolve and implement the APHA  process moving forward.
section_title: DHS Revised Its Acquisition Policy Instruction to Be More Effective, but Has Not Fully Documented Decisions
paragraphs: In March 2016, DHS revised the acquisition policy instruction for  implementing MD-102 to provide guidance for successful program  planning, management, and execution. Some of the revisions reflect  changes DHS previously made in response to past GAO  recommendations, and the new instruction also includes changes to the  documentation that programs are required to get approved before  advancing through the acquisition life cycle. The revisions also set forth  the process programs must follow if they experience a breach. DHS has  made progress in implementing these documentation requirements more  consistently than we have found in the past, but DHS leadership could  better document its rationale for key acquisition decisions to increase  department and external stakeholder insight into acquisition management  decisions.
subsections: section_title: DHS Better Defines Responsibilities and Strengthens Oversight of Requirements
paragraphs: Over the past 3 years, DHS has made changes that reflect prior GAO  recommendations to clarify roles and responsibilities and provide better  oversight, which are now included in its revised acquisition policy  instruction. For example:
Clarifying roles and responsibilities. In March 2015, we found that  DHS’s acquisition policy did not clearly differentiate the roles and  responsibilities of DHS’s PARM and the Enterprise Business  Management Office in the Office of the CIO, which has the primary  responsibility for ensuring IT investments align with DHS’s missions  and objectives. We recommended that DHS clarify the roles and  responsibilities of PARM and other DHS oversight organizations to  improve coordination, limit overlap of responsibilities, and reduce  duplicative efforts at the component level. In April 2015, DHS’s Acting  Deputy USM issued an acquisition decision memorandum to clarify  the respective acquisition responsibilities of PARM, the Office of the  CIO, and other members of DHS’s ARB, and in March 2016, DHS  revised its policy instruction to reflect these changes.
Re-establishing the JRC. In November 2008, we found that DHS  had not effectively implemented or adhered to its review process for  major acquisitions and recommended that DHS reinstate the JRC to  review and approve acquisition requirements and assess potential  duplication of effort. In June 2014, the Secretary of Homeland  Security directed the creation of a joint requirements process, led by a  component-composed and chaired JRC, and in March 2016, DHS  revised its policy instruction to reflect the addition of the JRC as an  acquisition oversight body. Among other responsibilities, the JRC is to  provide requirements-related advice and validate key acquisition  documentation to prioritize requirements and inform DHS investment  decisions, such as the joint-operational requirements document  between USCG and CBP for a common aircraft mission system. In  October 2016, we found that the re-establishment of the JRC after  many years without such an active body is a positive demonstration of  senior-level commitment to improving the DHS-wide capabilities and  requirements processes and has the potential to help DHS reduce  duplication and make cost-effective investments across its portfolio  over time. However, the JRC is still developing a process to  prioritize requirements to inform budget decisions.
section_title: DHS Updates Acquisition Documentation Requirements, But Leadership Decisions Not Always Fully Documented
paragraphs: DHS’s March 2016 revision to the acquisition policy instruction also  included changes to the acquisition documentation required to inform  ADEs, but DHS leadership did not always document its rationale for key  acquisition decisions. In September 2012, we found that, in most  instances, DHS leadership had allowed programs to proceed with  acquisition activities without obtaining department-level approval of key  acquisition documentation—such as APBs, LCCEs, and operational  requirements documents—as required by its acquisition policy. As a  result, we recommended DHS ensure all programs obtain department- level approval for key acquisition documentation before approving their  movement through the acquisition life cycle to mitigate risks of execution  challenges, such as cost growth and schedule slips. DHS concurred with  this recommendation and we have continued to monitor the agency’s  progress in addressing this recommendation through our annual  assessments and high-risk updates. Key changes to the acquisition  documentation required to inform ADEs include:
ADE 2A: DHS now requires programs to obtain department-level  approval for program study plans for performing analysis of  alternatives and receive technical assessments conducted by S&T  and the CIO at this decision point.
ADE 2C: DHS now requires programs to update and obtain  department-level approval for several documents at ADE 2C,  including, but not limited to current APBs, LCCEs, and TEMPs. The  previous instruction had no formal documentation requirements for  this decision point.
We reviewed acquisition decision memorandums—the department’s  official repository for key acquisition management decisions—issued in  calendar year 2016 and identified that 14 major acquisition programs  received ADE approval in 2016. Half of these programs had ADEs before  DHS revised the acquisition policy instruction in March 2016, while the  other half had ADEs after March 2016. We reviewed the documentation  for each program compared to the requirements in place at the time of its  ADE and found that DHS leadership had generally approved the required  key acquisition documentation—including APBs, LCCEs, and operational  requirements documents—for all 14 programs according to the  requirements in place at the time. However, DHS had not approved some  of the required documentation for 4 programs—CBP’s Tactical  Communications (TACCOM) Modernization and UH-60, NPPD’s HART,  and TSA’s TIM.
CBP’s TACCOM program did not have a department-approved  Acquisition Plan when leadership granted it ADE 3 approval in  January 2016. CBP officials told us that the Acquisition Plan did not  complete the approval process prior to its ADE 3 because of  conflicting guidance delivered to the program regarding the content of  the plan. However, these officials stated that the program  subsequently updated the Acquisition Plan and submitted it for  department approval, which they expect to receive by early calendar  year 2017.
CBP’s UH-60 program did not have a department-approved  Integrated Logistics Support Plan, TEMP, or Systems Engineering  Life Cycle Tailoring Plan when DHS leadership granted it ADE 2B  approval in January 2016. DHS leadership required the program to  update its Integrated Logistics Support Plan and, as of December  2016, program officials said they had submitted a draft for signature.  Program officials also told us that DOT&E said that a TEMP was  unnecessary because the program completed operational testing in  2012 and DHS leadership only required that the program conduct  minimal flight checks on future aircraft. Program officials  acknowledged they had no Systems Engineering Life Cycle Tailoring  Plan for the UH-60 program, and noted that the systems engineering  reviews for the reconfigured aircraft are being performed by the U.S.  Army.
NPPD’s HART program received ADE 2A approval in May 2016, but  did not receive DHS approval for all of the new documentation  requirements under the March 2016 acquisition policy instruction  revision. Specifically, the program received a technical assessment  from S&T but not from DHS’s CIO, as was required. Program officials  noted that they were not aware of the requirement for a CIO technical  assessment, but that DHS’s CIO did review HART’s documentation  and is a part of the program’s source selection evaluation team.
TSA’s TIM program received a combined ADE 2A/2B approval in  October 2016, but did not receive approval for the Analysis of  Alternatives Study Plan, as required. However, TIM did receive DHS  approval of its new technical approach that was developed in close  collaboration with DHS’s CIO and subject matter experts from S&T,  among other organizations, prior to its ADE approval. A senior DHS  official stated that TIM’s new technical approach satisfied the Analysis  of Alternatives Study Plan requirement based on the activities  completed.
In all four cases, there is no acquisition decision memorandum granting  these programs approval to deviate from the documentation  requirements, as outlined in DHS policy.
While DHS made progress implementing its documentation requirements  in 2016, DHS leadership made some decisions that were inconsistent  with DHS’s acquisition policy for programs that did have all the required  documentation approved. For example, DHS leadership granted CBP’s  Land Border Integration (LBI) and NII programs ADE 3 approval while  simultaneously requiring CBP to identify a final year for each program. As  a result, DHS approved the programs to transition into sustainment based  on approved LCCEs that did not account for each programs’ full costs,  which is inconsistent with both the current and past versions of DHS’s  acquisition policy instruction. Senior DHS officials said that they had the  knowledge to support ADE 3 approval for the programs because the  approved LCCEs for both LBI and NII covered at least one cycle of  technology replacement past each program’s FOC dates and that they  had discussed plans for follow-on capabilities at each programs’ ADE.  Officials from both programs said they will update their programs’ LCCEs  in 2017 to reflect all costs through each programs’ identified end year.  Senior DHS officials acknowledged that the department could better  document these decisions and leadership’s rationale in acquisition  decision memorandums.
In other cases, we found that DHS leadership took steps to ensure  programs complied with its acquisition policy. For example, CBP’s ACE  program requested permission to waive the requirement to complete all  operational testing prior to FOC, but DHS leadership denied that request.  In addition, DHS leadership withheld ADE 1 approval for the USCG’s  Motor Lifeboat program until it received JRC validation of its mission  needs documentation and submitted it to DHS for approval, as required.
Federal internal control standards state that to achieve objectives and  respond to risks, agencies should clearly document and communicate  significant events in a manner that allows for effective oversight and  examination. DHS’s acquisition policy instruction indicates that  acquisition decision memorandums document acquisition decisions,  direction, guidance, and any assigned actions. However, the policy  instruction does not specify that leadership’s rationale for those actions  be included in the memorandums. DHS leadership’s decisions to approve  programs to proceed through the acquisition process without meeting all  acquisition policy instruction requirements may be reasonable in any  given case. For example, it can take months to obtain department-level  approval for key acquisition documentation, and it may take time for DHS  to build the capacity to conduct the new S&T and CIO assessments and  implement the policy across the department. However, unless the  rationale for these decisions is documented and communicated through  acquisition decision memorandums, effective oversight and insight into  approval decisions for internal and external stakeholders is limited.
section_title: Updated DHS Breach Policy Not Clear on Timing for Reporting Performance Breaches
paragraphs: DHS’s March 2016 revised acquisition policy instruction also includes  changes to the department’s breach policy, which applies to programs  that fail to meet any cost, schedule, or performance threshold in a  program’s approved APB. However, the policy instruction does not  specifically discuss how to determine whether a performance breach has  occurred, and we found that no programs had reported a performance  breach. Among other changes, DHS’s revision requires programs to notify  department- and component-level leadership via formal memorandum  within 30 calendar days of an identified breach (cost, schedule, or  performance). The revision also removed the requirement that programs  submit breach remediation plans to DHS leadership within 30 days of this  notification and take certain corrective actions—such as returning to its  APB parameters, re-baselining, or having a DHS-led program review that  results in recommendations for a revised baseline—within 90 days of the  breach occurrence. Under the revised instruction, programs are now  directed to work with the Component Acquisition Executive to determine  an appropriate timeframe in which to complete remediation planning after submitting a breach notification, and to take corrective actions within the  timeframe established by DHS as documented in an acquisition decision  memorandum approving the program’s remediation plan. In general,  programs continue to execute planned activities while conducting breach  remediation planning efforts, unless otherwise directed by DHS  leadership.
In calendar year 2016, 10 major acquisition programs—including 6 that  we reviewed in more depth—submitted schedule or cost breach  notification memorandums to component and DHS leadership. Three of  the programs declared the breaches before DHS revised the acquisition  policy instruction, while the rest declared breaches afterwards. These  programs took varying lengths of time to submit remediation plans, and  DHS approved the remediation plans for all programs. Table 6 depicts the  status (as of February 2017) of the 10 programs that had reported a cost  or schedule breach in 2016.
As a part of its review process, DHS requested that at least two programs  make revisions to their remediation plans before they were approved. For  example, DHS issued an acquisition decision memorandum in December  2016 disapproving the USCIS Transformation program’s remediation  plan, and directing that USCIS stop planning and development of new  capabilities and update its breach remediation plan, among other things.  DHS subsequently approved a revised breach remediation plan for the  Transformation program in February 2017. In addition, TSA submitted  three versions of its combined breach remediation plan for both PSP and  the Security Technology Integrated Program over the span of about 5  months, before DHS leadership ultimately approved the final plan in  January 2017. DHS issued an acquisition decision memorandum in July  2016 directing TSA to make significant changes to its initial breach  remediation plan submitted in May 2016. PARM officials confirmed they  received TSA’s revised breach remediation plan for these programs in  August 2016, but requested additional changes, which were reflected in a  final version submitted in October 2016. According to these officials, the  requested changes were made during a meeting with the program  managers and not documented in an acquisition decision memorandum.  They added that PARM is in communication with the component and  program as they develop their remediation plans, and also updates DHS  leadership on programs’ breach status on a monthly basis; however,  officials noted that the communication between DHS and the program is  informal and not always documented through acquisition decision  memorandums unless DHS leadership has significant concerns about the  breach. We will continue to monitor DHS’s implementation of its updated  breach policy, including documentation of the department’s  communication with programs during their breach remediation planning  efforts.
We also found that the revised acquisition policy instruction is not clear as  to how programs are to determine when a performance breach has  occurred. No program in our review had reported a performance breach  despite 14 programs not meeting KPPs, including 3 programs that DHS  had granted ADE 3 approval. Some program officials we spoke to said  that they did not report a performance breach to DHS headquarters  because the programs planned to meet all KPPs during future test  events. Senior DHS officials told us programs typically experience a cost  or schedule breach prior to a performance breach, and that they consider  the performance breach policy to apply towards the end of a program’s  acquisition life cycle, such as after it begins operational testing. In  addition, senior DHS officials said they frequently discuss program  performance at ARBs and prior to granting programs ADE 3 approval.  However, DHS’s acquisition policy instruction revision states that the  breach policy applies once a program’s initial APB is approved at ADE 2A  through FOC, and does not specify at what point during this timeframe  programs should have met KPPs.
Moreover, while some programs may experience schedule or cost  breaches earlier in the acquisition life cycle, these breaches or actions  programs take to remediate these breaches may not be related to  performance issues. For example, CBP’s IFT program experienced a  schedule breach in November 2012 due to delays in the initial contract  award process and anticipated funding shortfalls. DHS leadership  removed IFT from breach status in December 2015—one month after the  program’s OTA conducted a limited user test on equipment deployed on  the Arizona border. Based on the test data, the OTA was unable to  determine if the system met its identification range KPP. The program has  not declared a performance breach because the IFT program manager  did not concur with several of the test results due to testing limitations.  DHS granted the program ADE 3 approval in 2013 prior to this testing,  which means the program has the authority to continue fielding  equipment that may not work as intended.
In June 2014, we found that the USCG’s acquisition guidance did not  clearly specify the conditions—particularly the timing—that would  constitute a performance breach and that DHS approved two USCG  programs—FRC and MRS’s HC-144A—to enter full-rate production  without having demonstrated all of their KPPs. We recommended the  USCG revise its acquisition guidance to specify when performance  standards should be met and to clarify the performance data used to  determine whether a performance breach has occurred. The USCG  concurred with our recommendations and updated its component-level  policy in May 2015 to define a performance breach occurrence, specify  when performance standards should be met (such as in formal follow-on  operational testing), and to outline the actions a program must take  following a breach to resolve the performance shortfall. However, DHS’s  department-level policy does not contain similar guidance.
Until DHS clarifies its acquisition policy instruction, it may be difficult for  programs to determine when, or by what measure, a breach of its KPPs  has occurred and, therefore, when to notify DHS of the occurrence. By  allowing programs to continue re-testing capabilities that have failed to  meet KPPs without submitting performance breach notifications and  remediation plans, DHS lacks insight into the root causes of system  failures to address performance issues that may also impact a program’s  schedule and cost estimates moving forward. In addition, programs could  potentially continue to field capabilities that do not fully meet KPPs or test  and re-test indefinitely in an attempt to meet a KPP—scenarios in which  end users do not get the capabilities they need or in the timeframes that  they need them.
section_title: Conclusions
paragraphs: Since we began reviewing DHS’s portfolio of major acquisitions in 2015,  the agency has strengthened its ability to track the progress of its major  acquisitions. Significantly, this year, for the first time, all programs in our  review had approved baselines against which DHS can measure program  performance—an effort that has taken almost 8 years since DHS first  established this requirement. Nevertheless, DHS continues to face  challenges in managing its portfolio, and progress does not negate the  fact that many programs continue to cost more, take longer than  expected, or struggle to meet moving performance targets. Improving  information for DHS leadership that ensures a program’s needs are  matched with available resources—performance and technical  requirements, time, and funding—prior to approving programs to begin  development could reduce the risk that programs will continue to face  execution challenges, put programs in a better position to succeed, and  ensure the department is making wise investment decisions with its  limited resources.
DHS has made a concerted effort to refine its policies to reflect a more  disciplined management approach and adhere more closely to this  acquisition policy. This policy also affords acquisition decision makers a  certain amount of flexibility. As DHS leadership exercises this flexibility in  its oversight of acquisition programs, however, it is important that visibility  is maintained into whether programs are meeting established  requirements, that reasonable deviations are well documented, and that  feedback directly affecting a program’s ability to be successful—such as  remediating a breach of its goals—is consistently communicated to  programs through formal channels. Doing so will enable better  management of DHS’s major acquisition portfolio as a whole by retaining  organizational knowledge and providing useful insight for DHS decision  makers and external stakeholders. Additionally, as mature programs  continue to fall short of performance goals, it is not clear at what point  programs need to acknowledge to DHS that performance problems  constitute a breach. As a result, DHS may be missing opportunities for  oversight and correction of performance issues, and is at risk of fielding  systems that may not work as intended.
section_title: Recommendations
paragraphs: To mitigate the risk of poor acquisition outcomes and strengthen the  department’s investment decisions, we recommend the Secretary of  Homeland Security direct the Undersecretary for Management to take the  following three actions:  Update the acquisition policy to:
Require that major acquisition programs’ technical requirements are  well defined and key technical reviews are conducted prior to  approving programs to initiate product development and establishing  APBs, in accordance with acquisition best practices.
Specify that acquisition decision memorandums clearly document the  rationale of decisions made by DHS leadership, such as, but not  limited to, the reasons for allowing programs to deviate from the  requirement to obtain department approval for certain documents at  ADEs and the results of considerations or trade-offs.
Specify at what point minimum standards for KPPs should be met,  and clarify the performance data that should be used to assess  whether or not a performance breach has occurred.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this product to DHS for review and comment. In its  written comments, reproduced in appendix III, DHS concurred with all  three of our recommendations. In response to our first recommendation,  DHS provided an estimated completion date for a study on how to better  align the department’s systems engineering and acquisition life cycles  with GAO’s acquisition best practices. In response to our other two  recommendations, DHS requested that we consider them closed based  on recent actions taken. Specifically, the department stated that it has  begun expanding the information documented in programs’ acquisition  decision memorandums to include enhanced background information and  plans to include the status of acquisition documentation in the future. In  addition, the department has updated the handbook for PARM’s  component leads to include guidance on (1) including the information  noted above when writing acquisition decision memorandums and (2)  determining programs to be in performance breach if they have not met a  KPP prior to ADE 3. While these are positive steps for addressing the  intent of our recommendations, we continue to believe that DHS should  update its acquisition policy to ensure that these changes are clearly  communicated and implemented consistently throughout the department.  DHS also provided technical comments, which we incorporated as  appropriate.
We are sending copies of this report to the appropriate congressional  committees and the Secretary of Homeland Security. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or mackinm@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix IV.
section_title: Appendix I: Program Assessments
paragraphs: This appendix presents individual assessments for each of the 26  programs we reviewed. Each of these assessments is two pages and  presents information current as of January 2017. They include several  standard elements, including an image provided by the program office, a  brief program description, and a summary of the program’s progress in  meeting its key performance parameters. Each assessment also includes  the following four figures:
Projected Funding vs. Estimated Costs. This figure generally  compares the funding plan presented in the Future Years Homeland  Security Program report to Congress for fiscal years 2017-2021 to the  program’s current annual total cost estimate based on its department- approved life-cycle cost estimate. We use this funding plan because  the data are approved by the Department of Homeland Security  (DHS) and Office of Management and Budget, and was submitted to  Congress to inform the fiscal year 2017 budget process. As a result,  the data does not account for other potential funding sources, such as  carryover, cost-sharing agreements with other organizations, or fees.  In addition, the program’s current annual cost estimate accounts for  total costs attributable to the program, regardless of funding source.
Program Office Staffing Profile. This figure is generally based on  the staffing assessments conducted by the Office of Program  Accountability and Risk Management, which identify the number of  staff a program needs (measured in full time equivalents) including  how many are considered critical positions (measured in the number  of people) and how many staff the program actually has. This figure  and any discussion of programs’ efforts to address identified staffing  gaps or critical vacancies do not reflect the January 2017 presidential  order to freeze the hiring of federal civilian employees.
Schedule Changes over Time. This figure consists of two timelines.  The first timeline is generally based on the initial Acquisition Program  Baseline (APB) DHS leadership approved after the department’s  current acquisition policy went into effect in November 2008. Because  these APBs were approved at different times, the first as-of date  varies across programs. The second timeline identifies when that  program expected to reach its major milestones as of January 2017.  The second timeline also identifies any new major milestones that  were introduced after the initial APB was approved, such as the date  a new increment was scheduled to achieve initial operational  capability, or the date the program was re-baselined.
Cost Estimate Changes over Time. This figure generally compares  the program’s cost estimate in the initial APB approved after DHS’s  current acquisition policy went into effect to the program’s expected  costs as of January 2017. This figure also identifies how much funding  had been appropriated to the program through fiscal year 2016 and  how it compares to future funding needs.
These four figures are generally based on DHS headquarters-approved  documentation and data, as identified above. However, in some cases,  the figures are based on data the program office provided when it  commented on a draft of the assessment if, for example, the data were  more accurate or current.
Each program assessment also consists of a number of other sections  depending on issues specific to each program. These sections may  include: Program Governance, Acquisition Strategy, Program Execution,  Test Activities, and Other Issues. Lastly, each program’s assessment also  presents comments provided by the program office and identifies whether  the program provided technical comments, and presents GAO’s response  to these comments, as necessary.
subsections: section_title: Schedule Changes over TimeProjected Funding vs. Estimated CostsProgram Office Staffing Profile
paragraphs: Automated Commercial Environment (ACE)
Customs and Border Protection (CBP)
The ACE program is developing software that will electronically  collect and process information submitted by the international  trade community. ACE is intended to provide private and public  sector stakeholders access to this information, and enhance  the government’s ability to determine whether cargo should  be admitted into the United States. The ACE program aims to  increase the efficiency of operations at U.S. ports by eliminating  manual and duplicative trade processes, and enabling faster  decision making. CBP deployed ACE’s initial release in  February 2003, but struggled to develop capability for several  years. Department of Homeland Security (DHS) leadership  directed CBP to halt new development in August 2010, and did  not authorize CBP to restart development until it re-baselined  the program in August 2013. GAO previously reported on CBP’s  ACE program in March 2016 (GAO-16-338SP) and has an ongoing review to assess ACE’s implementation.
Staff needed: 196 full time equivalents (FTE)
CBP officials previously told GAO that three of ACE’s four key  performance parameters (KPP) were tested and successfully  demonstrated for deployed functionality in May 2015, including  the KPP for system availability. However, CBP officials  subsequently reported that the ACE program did not meet its  availability KPP in June 2016 when ACE became mandatory for  all manifest processing and system traffic increased. Officials  expect the availability KPP to temporarily decline again in  January 2017 when ACE is fully deployed. ACE will not be able  to demonstrate that it can meet its final KPP for full system  performance in an operational environment until the program  completes testing, which is now planned for April 2017.
When DHS leadership re-baselined ACE’s cost, schedule, and  performance parameters in August 2013, the program adopted  an agile software development methodology to accelerate  software creation and increase flexibility in the development  process. ACE’s agile method is defined by a series of 2-week  “sprints,” during which software is designed, developed,  integrated, and tested. Six ACE sprints constitute a program  increment. The program currently consists of 13 increments,  which are to be completed over a 3-year period. At the end of  each sprint, software developers demonstrate new capabilities  to ACE end users to obtain feedback and confirm that the new  capabilities meet requirements. The ACE program office serves  as the system integrator, overseeing 15 agile development  teams. Because the agile teams demonstrate capabilities after  each sprint, ACE program officials said they have opportunities  to closely monitor contractor performance and mitigate risks  through real-time management decisions. decommissioning the legacy program that ACE is replacing.  In December 2016, CBP reported that the program is able  to offset the projected gap in fiscal year 2017 with carryover  funds and that both CBP and DHS had realigned additional  funding to address the projected gap in the remaining years. In  August 2016, CBP officials told GAO that the ACE program is  developing a fee-for-service concept that could potentially be  used for system enhancements, among other things.
section_title: Program Execution
paragraphs: In April 2016, DHS’s Director, Office of Test and Evaluation  (DOT&E) approved a new Test Evaluation Master Plan (TEMP)  that reflected a more flexible testing approach. CBP officials  previously told GAO that they determined it would be more  feasible to test ACE’s KPPs in batches as capabilities were  deployed, rather than all at once as directed in its initial TEMP.  The program conducted its first operational test in June 2015,  but delayed a subsequent operational test from April 2016 to  July 2016 to allow all stakeholders more time to transition to  ACE prior to testing. In December 2016, program officials said  they plan to conduct follow-on testing for this event in February  and March 2017—after ACE’s final deployment in January  2017—and do not anticipate receiving final test results until May  2017.
In November 2016, DHS’s Under Secretary for Management  (USM) re-baselined the ACE program, removing it from breach  status after the program experienced schedule slips and cost  growth. In June 2016, CBP officials notified DHS leadership  that the program would not complete several key events as  planned, and that its costs would increase beyond its approved  thresholds. The program reported that its external stakeholders  raised concerns about meeting the mandatory transition date to  ACE. In response, the program delayed completion of two key  milestones: (1) decommissioning of the legacy entry system  slipped from March 2016 to July 2016 and (2) development  of ACE functionality slipped from May 2016 to September  2016. According to CBP officials, ACE functionality will be fully  developed and in use by January 2017. The delays affected  subsequent milestones including completion of operational test  and evaluation, which slipped from September 2016 to April  2017, and full operational capability (FOC), which slipped from  November 2016 to September 2017. Despite these delays,  CBP’s initial re-baseline draft did not delay the program’s  Acquisition Decision Event (ADE) 3 from its initial date of  November 2016, which could have allowed the program to  transition into sustainment without test and evaluation results  that confirmed successful performance of ACE’s full capabilities,  as required by DHS’s acquisition policy. However, the revised  baseline approved by DHS’s USM ultimately delayed ADE 3  to June 2017 until after operational testing is scheduled to be  complete.
The program’s final operational test event, which is the first  time the program will be able to test the functionality and  performance of the entire ACE system, was delayed from  September 2016 to April 2017. CBP officials said they requested  permission to waive the requirement to complete all operational  testing prior to FOC. DHS leadership denied the request and,  as reflected in the program’s new baseline, ACE will complete  operational testing prior to FOC.
The ACE program reported one critical vacancy for a Director of  Testing and Evaluation. In August 2016, CBP officials told GAO  that existing staff have covered the workload for this critical  vacancy and the position will no longer be required in the near  future.
Program Office Comments   The availability KPP is measured over any continuous  365-day period for a fully deployed system and reported to  DHS monthly. Although this KPP dipped slightly below its  threshold in June 2016, which is typical after a deployment  or mandatory use date, there is no indication that the  availability KPP will not be met once the system is fully  deployed. Program officials noted that they would declare  breach for performance prior to full system deployment  if they determine there is no chance of achieving a KPP.  Program officials also provided technical comments on  a draft of this assessment, which GAO incorporated as  appropriate.
Integrated Fixed Towers (IFT)
Customs and Border Protection (CBP)
The Department of Homeland Security (DHS) established  the IFT program in March 2012 to address the capability gap  left when the Secretary of Homeland Security canceled the  Secure Border Initiative Network (SBInet) program. CBP plans  to deliver approximately 53 fixed surveillance tower units  equipped with ground surveillance radar, infrared cameras,  and communications systems linking the towers to command  and control centers. CBP plans to deploy these units across six  areas of responsibility (AoR) in Arizona to help the Border Patrol  detect and track illegal entries in remote areas. DHS leadership  re-baselined the program in December 2015, approximately  3 years after CBP determined the program could not meet its  initial schedule goals. GAO previously reported on CBP’s IFT  program in March 2016 (GAO-16-338SP) and has an ongoing  review to assess IFT’s deployment along the Arizona border.
Staffing gap: 4 FTEs equivalents (FTE)
CBP officials previously told GAO that IFT met all 3 of its key  performance parameters (KPP) during a July 2015 systems  acceptance test in the Nogales AoR. These KPPs establish a  minimum acceptable range for detection and identification, and  the percentage of time the system must operate as intended.  In April 2016, however, testers found that IFT only met 2 of its  3 KPPs and experienced 5 operational deficiencies during a  November 2015 limited user test conducted in the same AoR.  IFT did not meet its KPP for identification range. IFT and Border  Patrol leadership did not concur with several of the test results  and reported deficiencies, but DHS’s Director, Office of Test and  Evaluation (DOT&E) did not formally assess the test results.
In January 2011, the Secretary of Homeland Security canceled  CBP’s SBInet program in response to cost, schedule, and  performance problems involving the acquisition of new  surveillance technologies. When CBP initiated the IFT program,  it decided to purchase a non-developmental system, and it  required that prospective contractors demonstrate their systems  prior to CBP awarding a contract. The program awarded the  contract to EFW, Inc. in February 2014, but this award was  protested. GAO sustained the protest, and CBP had to re- evaluate the offerors’ proposals before it again decided to  award the contract to EFW, Inc. As a result, EFW, Inc. did not  initiate work at the deployment sites until fiscal year 2015.  The contract is valued at $145 million and covers the entire  system acquisition cost for the six AoRs, as well as 7 years of  operations and maintenance. costs over the next 5 years. These officials added that they are  in the process of updating IFT’s cost estimate to account for  changes in the order of AoR deployments, but that the program  will carry over nearly $34 million in funding from fiscal year 2016  to help address any remaining gap.
According to CBP officials, the number of IFT units deployed  to a single AoR is subject to change based on assessments  by the Border Patrol. In April 2013, Border Patrol directed CBP  to reduce the number of planned IFT units from 50 to 38 and  reduce the AoRs from six to five. In January 2015, Border Patrol  directed CBP to increase the AoRs back to six, but instructed  CBP to replace 15 existing fixed tower systems deployed under  the SBInet program, rather than expanding IFT capabilities to  a new AoR as originally planned. In March 2016, Border Patrol  certified to the congressional appropriations committees that 7  of the 53 IFT units deployed to the first AoR in Nogales met the  program’s operational requirements—a prerequisite for CBP’s  deployment of additional IFT units. As of January 2017, CBP  officials said they had initiated the deployment of 15 additional  IFT units to two other AoRs, and planned to deliver the  remaining 31 IFT units across the other three AoRs.
The DOT&E-approved TEMP established that CBP would  conduct a limited user test to validate operational requirements  and determine how the IFT system contributes to CBP’s  mission. The program’s operational test agent (OTA) completed  a limited user test at the Nogales AoR in November 2015. This  test was delayed 2 months because, according to program  officials, CBP delayed systems acceptance so the contractor  could address problems identified with IFT’s cameras and  operator interfaces during a July 2015 test. In April 2016, the  OTA identified 5 operational deficiencies and recommended  the program take 11 actions to improve IFT system operations.  For example, the OTA found that the camera did not provide  sufficient video quality and the IFT system did not enable the  operator to consistently identify possible entries. In June 2016,  IFT’s program manager issued a memorandum identifying  his concerns with the OTA’s report and non-concurrence with  4 of the 5 deficiencies and 1 of the 11 actions. Border Patrol  leadership subsequently concurred with the IFT program  manager’s position.
section_title: Program Governance
subsections: section_title: In March 2012, DHS’s Under Secretary for Management (USM) approved the IFT Acquisition Program Baseline (APB), which established the program’s cost, schedule, and performance parameters. The USM also authorized the program to deploy all planned IFT units, but later clarified in June 2012 that this authorization was contingent on DHS’s DOT&E approving the IFT Test and Evaluation Master Plan (TEMP). In November 2012, CBP reported that IFT would breach its schedule because of delays in releasing the request for proposals and the source selection process, as well as anticipated funding shortfalls. Nonetheless, after DOT&E approved IFT’s TEMP, CBP deployed IFT units to the Nogales AoR in November 2014. Thirteen months later, in December 2015, the USM approved an updated APB that reflected the program’s schedule slips.
paragraphs: DOT&E reviewed the OTA’s test results, but decided not to  conduct a formal assessment because DHS leadership had  already authorized full deployment. In November 2016, a  DOT&E official who observed the limited user test told GAO  that he had concerns with how the test data were collected  and did not believe the test results were useful in assessing  IFT’s operational effectiveness, suitability, cybersecurity, or  contribution to CBP’s mission. Program officials told GAO that  Border Patrol is responsible for 5 of the 11 actions, and that  they are working with the contractor to address the remaining 6  actions identified during the limited user test, such as updating  the cameras to improve video quality. In January 2017, the IFT  program manager said the program plans to conduct further  testing and is working closely with DOT&E to determine the  scope and timing of future test events.
In January 2016, CBP reported that the IFT program had  a staffing gap of four full time equivalents. In August 2016,  program officials said they did not have problems executing  current IFT installations, but said they will encounter challenges  if CBP initiates subsequent AoR deployments simultaneously.
section_title: Program Execution
subsections: section_title: According to CBP officials, the IFT program is on track to meet the cost and schedule parameters in its December 2015 APB. However, from March 2012 to December 2015, IFT’s APB acquisition cost threshold increased by more than $50 million when CBP included the cost of contractor personnel supporting the program office, the cost of replacing SBInet systems and actual costs through fiscal year 2014, rather than estimates. Additionally, the program’s full operational capability (FOC) date slipped 5 years. CBP officials primarily attributed the FOC delay to funding shortfalls, and it appears that the program is projected to face a $130 million funding gap from fiscal year 2017 to fiscal year 2021. Program officials told GAO that the program’s costs include some items that are not funded by IFT. For example, operator costs are funded by Border Patrol and account for more than $61 million of the program’s estimated
paragraphs: Program Office Comments CBP officials non-concurred with GAO’s assessment that the IFT system failed a KPP in any phase of the program testing. There is no evidence in the limited user test report or other documentation showing IFT did not meet a KPP, specifically the KPP for identification range. CBP officials also provided technical comments on a draft of this assessment, which GAO incorporated as appropriate.
GAO Response  Based on the limited user test data, the OTA was unable to determine if the system met the identification range KPP.
Land Border Integration (LBI)
Customs and Border Protection (CBP)
The LBI program delivers license plate readers (LPR), radio  frequency identification readers, and other technologies to  122 land border locations. The program’s goal is to facilitate  legitimate trade and travel while enhancing border security.  LBI systems are intended to enhance the processing of  pedestrians, inbound and outbound vehicles at land border  crossings, as well as Border Patrol checkpoints. LBI leverages  technology delivered through a previous CBP acquisition  program designated the Western Hemisphere Travel Initiative  (WHTI), which sought to enhance inbound vehicle processing.  GAO previously reported on CBP’s LBI program in March 2016  (GAO-16-338SP).
Staff needed: 33.55 full time equivalents (FTE)
In September 2016, CBP officials reported that the program had  met its key performance parameter (KPP) for the checkpoint  LPR system. LBI previously relaxed the KPP threshold for the  checkpoint LPR system in November 2015 after determining  the original requirement was unrealistic and did not account for  challenges in the checkpoint operating environment. To achieve  the revised KPP, the program also replaced underperforming  LPR technology at 28 locations. Program officials previously told  GAO that the other LBI systems met their respective KPPs during  testing conducted in 2009, 2012, and 2015.
DHS’s Under Secretary for Management (USM) authorized  CBP to transition from WHTI to LBI in May 2011. At that  time, the USM transferred the inbound capabilities of WHTI  to LBI, authorized a limited deployment of LBI’s outbound,  pedestrian, and checkpoint capabilities, and informed CBP that  he planned to delegate acquisition decision authority for future  LBI deployments to CBP’s Component Acquisition Executive.  However, according to CBP officials, the USM never delegated  this authority. Nonetheless, program officials reported that CBP  expanded the deployment of LBI’s outbound, pedestrian, and  checkpoint capabilities without requesting formal authorization  from DHS leadership. CBP proceeded with these deployments  even though the USM had not approved an LBI Acquisition  Program Baseline (APB) establishing the program’s cost,  schedule, and performance parameters.
The program previously deferred some planned deployments  due to funding constraints. In December 2014, program officials  told GAO that LBI’s cost estimates had decreased significantly  from the nearly $2 billion estimated in August 2014. CBP  officials said they originally planned to execute the program  through three phases, which would allow CBP to enhance  LBI systems over time, and expand the deployment of certain  technologies to additional land border crossings. However,  program officials stated that subsequent funding constraints  forced CBP to defer some planned LBI deployments. CBP  prioritized subsequent deployments by identifying land border  crossings that would benefit the most from new technologies.  LBI officials also explained they no longer planned to deploy  Border Patrol checkpoint systems along the northern border,  and have purchased less expensive, less efficient equipment to  reduce costs.
In January 2016, the USM approved the program’s APB. Later  that month, DHS leadership granted the program Acquisition  Decision Event 3 approval, and simultaneously required  that CBP identify a final year for the program. CBP officials  subsequently identified fiscal year 2027 as the program’s end  date. However, LBI’s approved life-cycle cost estimate (LCCE)  includes planned costs only through fiscal year 2021—6  years short of the program’s final year. Nevertheless, DHS  approved the program to transition into sustainment without  an understanding of the program’s full costs, as required by its  acquisition policy.
Program Execution LBI achieved full operational capability (FOC) for its remaining  systems in 2016—more than 3 years later than officials  originally reported. Program officials previously told GAO that  all of LBI’s systems had achieved FOC by the end of August  2013. However, in August 2016, program officials reported that  none of the systems had achieved FOC until June 2015, when  the pedestrian systems reached this milestone. According to  program officials, the remaining systems reached FOC over the  next 15 months, with the inbound, outbound, and checkpoint  systems achieving this milestone in September 2015, June  2016, and September 2016, respectively. LBI’s approved APB  of January 2016 reflects these changes to the program’s FOC  dates.
In 2016, CBP continued to monitor the performance of the  checkpoint LPR system against its KPP, as directed by DHS’s  USM. In September 2016, CBP reported this system had met  its KPP. The program concluded formal testing prior to January  2016. DHS’s Director, Office of Test and Evaluation (DOT&E)  approved LBI’s Test and Evaluation Master Plan in November  2011, and the program conducted operational testing in January  2012. CBP officials reported that LBI systems met all of their  KPPs during the 2012 operational test with the exception of  the checkpoint LPR system. However, DOT&E did not validate  the test results because, as discussed above, the program  did not request formal authorization from DHS leadership  to expand LBI’s deployment. From July to September 2015,  CBP conducted an operational assessment of LBI’s deployed  outbound systems and declared them operationally effective  and suitable. In November 2015, DOT&E validated these  results.
Other Issues In January 2016, CBP reported the program needed 7.4 more  full time equivalents. In August 2016, CBP officials said the  program recently hired three new staff, and the remaining  staffing gap has had minimal effect on operations.
From January 2016 to January 2017, LBI’s cost estimates  remained stable. However, as noted above, the program’s  LCCE only reflects costs through fiscal year 2021 and does not  account for additional quantities, operations and maintenance,  or upgrade costs through the program’s end date of fiscal year  2027. In August 2016, CBP officials told GAO they plan to  update the program’s LCCE in fiscal year 2017, at which point  they will extend the estimate through the program’s end date of  fiscal year 2027.
From fiscal years 2017 to 2021, LBI’s yearly cost estimates  appear to exceed the program’s funding plan by $52 million. LBI  officials reported this funding gap is largely driven by the need  to refresh deployed technology. The program plans to mitigate  the funding gap by prioritizing upgrades against operational  needs, conducting preventive maintenance, and remotely  monitoring and correcting system issues, among other things.  Program officials stated that upgrades to LBI’s inbound systems  are most likely to be affected by future funding constraints, as  the program has already updated checkpoint and outbound  systems.
Program Office Comments   The LBI program formally achieved Produce/Deploy/ Support Phase in January 2016. The program satisfied  the outstanding checkpoint KPP with the refresh of  underperforming LPR technology. In September 2016  the program awarded a new primary technology support  contract with one base and four option years, without  protest. LBI will coordinate future activities with the CBP  Component Acquisition Executive to ensure compliance  with acquisition requirements. CBP officials also provided  technical comments on a draft of this assessment, which  GAO incorporated as appropriate.
Medium Lift Helicopter (UH-60)
Customs and Border Protection (CBP)
The UH-60 is a medium lift helicopter that CBP uses for law  enforcement and border security operations, air and mobility  support and transport, search and rescue operations, and other  missions. CBP’s UH-60 fleet consists of 20 aircraft acquired  from the U.S. Army in three different models. CBP previously  acquired 4 aircraft in the modern UH-60M model and converted  6 of its 16 older UH-60A aircraft into more capable UH-60L  models as a part of its Strategic Air and Marine Program  (StAMP). In July 2016, Department of Homeland Security (DHS)  leadership designated the UH-60 as a separate and distinct  level 1 acquisition program. The UH-60 program is currently  focused on converting the remaining 10 UH-60A aircraft. GAO  previously reported on the UH-60 aircraft as a part of StAMP in  March 2016 (GAO-16-338SP).
Staffing gap: 3 FTEs equivalents (FTE)
CBP determined that the converted UH-60L and UH-60M aircraft  met all five of their key performance parameters (KPP) through  operational testing conducted in fiscal years 2012 and 2014.  These KPPs establish requirements for communications and  specific mission capabilities, including interdiction, air mobility,  special operations, and search and rescue. However, DHS’s  Director, Office of Test and Evaluation (DOT&E) did not validate  these results because the UH-60 was not considered a major  acquisition when the tests were conducted.
CBP has obtained all 20 UH-60 aircraft through agreements  with the U.S. Army. CBP received the 16 UH-60A aircraft  through a loan agreement in January 2004. In March 2008, CBP  entered into an inter-agency agreement with the Army to convert  the UH-60A into UH-60L models to extend the aircraft’s service  life by an estimated 20 years, as well as to purchase and modify  the 4 new UH-60M aircraft. CBP completed acceptance of the  UH-60M aircraft in 2012.
Security Program report to Congress—the first report to include  UH-60 as a distinct major acquisition program—but said the  O&M funding will not be reflected for the reason stated. This issue limits insight into the program’s funding needs and may obscure the size of future funding gaps.
In November 2014, CBP proposed changing its acquisition  strategy for converting its UH-60A aircraft when it learned the  Army planned to divest several HH-60L aircraft, which could  more easily be reconfigured into UH-60L aircraft for CBP  missions. Specifically, CBP proposed concluding its UH-60A  conversions of the 6 aircraft it had initiated and trading the  remaining 10 aircraft for the Army’s newer HH-60L. Although the  Army would still have to reconfigure the HH-60L aircraft to meet  CBP’s needs, CBP officials anticipated this effort could reduce  the program’s costs by an estimated $70 million, accelerate its  schedule, and result in newer aircraft since the Army’s HH- 60L airframes had fewer operating hours than CBP’s existing  UH-60A aircraft. At that time, DHS’s Under Secretary for  Management (USM) directed CBP to further study its proposed  approach in consultation with DHS’s Aviation Governance  Board, and authorized CBP to initiate the transfer of a single  HH-60L aircraft for developing a prototype to validate and verify  its reconfiguration. In January 2016, DHS’s USM approved  CBP’s revised acquisition strategy based on the Aviation  Governance Board’s determination that the proposed plan  carries less risk and will result in overall cost savings. The USM  also approved the UH-60 program’s initial Acquisition Program  Baseline (APB) at that time, which established schedule,  cost, and performance parameters for the program’s revised  acquisition strategy.
Test Activities CBP conducted operational testing of the UH-60L and UH-60M  aircraft in fiscal year 2012. CBP testers assessed the UH-60L as  operationally effective and suitable in July 2012, but assessed  the UH-60M as operationally suitable and marginally effective in  April 2012 because it could not meet endurance requirements,  among other things. CBP completed modifications on the  UH-60M to address identified issues and conducted additional  operational testing in March 2014. In April 2014, CBP testers  assessed the UH-60M retrofits as operationally effective and  suitable. However, DOT&E did not validate CBP’s test results  for either aircraft variant because the UH-60 was not considered  a major acquisition when the tests were conducted.
In January 2016, DHS’s USM directed the program to conduct  acceptance functional flight checks on at least one reconfigured  HH-60L aircraft prior to receiving approval to proceed with the  remaining transfer and conversions. The program plans to  flight check the reconfigured HH-60L prototype in July 2017,  but officials said they do not plan to conduct further operational  testing because the HH-60L has minimal differences from the  UH-60L aircraft previously tested. However, not demonstrating  the reconfigured HH-60L in an operational environment may  increase the risk that the aircraft will not perform as intended or  be reliable once fielded.
section_title: Program Execution
subsections: section_title: CBP officials said the program is on track to meet the schedule and cost goals in its current APB. CBP accepted the sixth and final converted UH-60L—from the A model—in June 2016. The program plans to achieve initial operational capability by June 2018 upon acceptance and deployment of the reconfigured HH-60L prototype, and full operational capability by September 2022 once all remaining 10 reconfigured UH- 60L aircraft are accepted and deployed. In April 2015, DHS headquarters completed an independent cost estimate for the UH-60, which CBP adopted as the program’s life-cycle cost estimate (LCCE). Prior to 2015, the program never had a comprehensive cost estimate. For example, a draft 2007 APB— which was never department approved—reported acquisition costs of about $1 billion, but did not include any costs for operations and maintenance (O&M). The UH-60’s current LCCE is approximately $2 billion for all 20 aircraft, including both acquisition and O&M costs.
paragraphs: In September 2016, officials told GAO that CBP designated  a program manager to lead each former StAMP acquisition  program—including the UH-60—but that it maintained a  consolidated program office where the same staff from  StAMP continue to support all remaining acquisitions. Officials  explained that this matrixed approach works well because they  are able to leverage each team member’s particular subject  matter expertise. Officials added that the program’s prior  staffing challenges decreased significantly once they completed  UH-60’s required acquisition documentation, and officials did  not anticipate future staffing issues.
Program Office Comments CBP is committed to accurate reporting of all of its programs  and would like to clarify any misunderstanding in terms  of program affordability. O&M of the UH-60 is funded  separately, thus is not reflected in the acquisition funding.  This assessment reflects only the acquisition funding plan.  Additionally, CBP disagrees with GAO’s use of a 2007  draft APB. The program should be assessed according to  the APB signed in January 2016 that was provided. CBP  officials also provided technical comments on a draft of this  assessment, which GAO incorporated as appropriate.
GAO Response The draft 2007 APB provides perspective on the history of  the program; however, GAO did assess UH-60 against its  January 2016 APB as shown in the figures above.
Multi-Role Enforcement Aircraft (MEA)
Customs and Border Protection (CBP)
The MEA is a fixed-wing, multi-engine aircraft that replaces  CBP’s aging fleets of C-12, PA-42, and BE-20 aircraft. The  MEA can be configured to perform multiple missions, including  marine, air, and land interdiction; logistical support; and law  enforcement technical collection (LETC). The current MEA  configuration is equipped with marine search radar and an  electro-optical/infrared sensor to support maritime and land  surveillance and airborne tracking missions. CBP previously  acquired MEA aircraft as a part of its Strategic Air and Marine  Program (StAMP). In July 2016, Department of Homeland  Security (DHS) leadership designated the MEA as a separate  and distinct level 1 acquisition program. CBP plans to acquire  16 MEA aircraft in the current configuration and, as of January  2017, 12 had been delivered. GAO previously reported on  the MEA aircraft as a part of StAMP in March 2016 (GAO-16- 338SP).
Staffing gap: 3 FTEs equivalents (FTE)
The MEA program has met all five of its key performance  parameters (KPP). In March 2016, DHS’s Director, Office of  Test and Evaluation (DOT&E) determined that the MEA program  continued to meet four of its KPPs. Specifically, the MEA met two  KPPs related to the aircraft’s marine interdiction capabilities and  two KPPs related to air mobility. DOT&E previously determined  the MEA had met its fifth KPP for communications during initial  operational test and evaluation (IOT&E). Going forward, the  program plans to establish additional KPPs for air and land  interdiction, LETC operations, and suitability for future MEA  configurations.
CBP initially planned to procure 50 MEAs and awarded the  first production contract in September 2009. However, the  aircraft did not perform well during testing. In October 2014,  DHS leadership said CBP could not procure or accept transfer  of additional MEA without approval. CBP procured 12 aircraft  under the initial contract, which expired in March 2015. In  August 2015, DHS’s Under Secretary for Management (USM)  authorized CBP to procure 4 additional MEAs for a total of 16  and directed CBP to work with the Joint Requirements Council  (JRC) to determine the appropriate quantity and configuration  for future MEA procurements. In September 2016, CBP  awarded an indefinite delivery, indefinite quantity contract for  1 base year with four 1-year options, and issued a delivery  order for MEAs 13 and 14. Program officials plan to exercise  the first option year in fiscal year 2017 to procure MEAs 15  and 16, and the remaining option years once CBP receives  approval for additional quantities. In April 2016, CBP developed  a report that described capability gaps in multiple mission areas  and proposed future MEA quantities and configurations. In  September 2016, the JRC Chair endorsed CBP’s findings, but  stated additional analysis was necessary for the JRC to fully  validate them and recommended CBP develop a number of  acquisition documents including an operational requirements  document. reflected for the reason stated. This issue limits insight into the program’s funding needs and may obscure the size of future funding gaps.
In March 2016, DHS’s DOT&E determined that the MEA was  effective and had resolved issues found during prior testing.  DOT&E had assessed the program’s IOT&E results in 2013,  and concluded that additional testing was needed to assess  the MEA’s air interdiction capabilities. DOT&E also said CBP  needed to take 28 specific actions as soon as possible to  improve MEA performance and that CBP should prioritize  those that affect flight safety. CBP officials previously told GAO  that they began addressing flight safety issues in January  2014. In July 2015, the program’s operational test agent  (OTA) conducted an operational assessment and found that  CBP had addressed 24 of the 28 actions. However, the OTA  also made 15 additional recommendations to improve the  aircraft’s operational effectiveness and suitability, and offered  14 additional findings to improve the effectiveness of the MEA’s  new mission system. DOT&E concurred with the OTA’s findings,  and subsequently determined that the remaining 4 actions had  no operational impact or had been addressed by CBP. DOT&E  recommended the program develop a plan to address the OTA’s  recommendations, and consider the OTA’s additional findings to  improve the mission system. In September 2016, CBP officials  told GAO the program plans to conduct additional testing when  MEA 14 is delivered by September 2017.
CBP is replacing the mission system processor on the MEA with  a system used by the U.S. Navy and the U.S. Coast Guard. The  new processor is intended to enhance operator interface and  sensor management, as well as replace obsolete equipment.  CBP tested a prototype of the processor in July 2015. According  to program officials, MEAs 13-16 will be delivered with the  new mission system, and CBP will begin retrofitting previously  delivered MEAs in fiscal year 2017.
CBP officials said the program is on track to meet the goals in  its initial Acquisition Program Baseline (APB), which DHS’s USM  approved in January 2016. This APB established schedule,  cost, and performance parameters for the program’s approved  quantity of 16 MEAs. The program achieved initial operational  capability in June 2011 upon delivery and acceptance of the first  aircraft. The program plans to achieve full operational capability  (FOC) by December 2018 upon delivery and acceptance of  MEA 16. However, this is later than CBP previously planned.  For example, a draft 2007 APB—which was never department  approved—reported that the program planned to achieve FOC  by September 2016. CBP plans to revise its APB if it receives  approval to acquire future aircraft, which may delay FOC further  and increase costs.
Other Issues In September 2016, officials told GAO that CBP designated  a program manager to lead each former StAMP acquisition  program—including the MEA—but that it maintained a  consolidated program office where the same staff from  StAMP continue to support all remaining acquisitions. Officials  explained that this matrixed approach works well because they  are able to leverage each team member’s particular subject  matter expertise. Officials added that the program’s prior  staffing challenges decreased significantly once they completed  MEA’s required acquisition documentation, and officials did not  anticipate future staffing issues.
Program Office Comments  CBP is committed to accurate reporting of all of its  programs and would like to clarify any misunderstanding in  terms of program affordability. O&M of the MEA is funded  separately, thus is not reflected in the acquisition funding.  This assessment reflects only the acquisition funding plan.  Additionally, CBP disagrees with GAO’s use of a 2007  draft APB. The program should be assessed according to  the APB signed in January 2016 that was provided. CBP  officials also provided technical comments on a draft of this  assessment, which GAO incorporated as appropriate.
GAO Response  The draft 2007 APB provides perspective on the history of  the program; however, GAO did assess MEA against its  January 2016 APB as shown in the figures above.
Customs and Border Protection (CBP)
The NII Systems Program supports CBP’s interdiction of  weapons of mass destruction, contraband such as narcotics,  and illegal aliens being smuggled across U.S. borders, while  facilitating the flow of legitimate commerce. CBP officers in  the field utilize large- and small-scale NII systems at air, sea,  and land ports of entry, as well as border checkpoints and  international mail facilities. Large-scale NII systems use directed  beams of X-rays or gamma rays that allow officers to examine  the entire contents of containers and vehicles without breaching  them. Small-scale NII systems are used to perform inspections  of passenger baggage and cargo, view the inside of fuel tanks  and small compartments, and identify false walls in containers.  Small-scale NlI systems include X-ray systems and fiber optic  scopes, among other devices. GAO previously reported on  CBP’s NII Systems Program in March 2016 (GAO-16-338SP).
Staff needed: 53.4 full time equivalents (FTE)
In January 2016, the NII Systems Program reduced the number  of its key performance parameters (KPP) from 24 to 18.  According to officials, the program continues to meet all 18 KPPs  including one requiring CBP to examine 100 percent of cargo  identified for inspection. CBP previously reported challenges  obtaining examination data for this KPP in the land environment  because of data accessibility and compatibility issues. However,  in August 2016, CBP officials said they worked with stakeholders  to develop a standard methodology to report examination  data. That said, the Department of Homeland Security’s (DHS)  Director, Office of Test and Evaluation did not independently  validate CBP’s assertion that it has met this KPP.
CBP has been deploying NII systems since the 1980s, but  DHS leadership did not approve the NII Systems Program’s  Acquisition Program Baseline until January 2016. Later that  month, DHS leadership granted the program Acquisition  Decision Event (ADE) 3 approval, and simultaneously required  that CBP update NII’s life-cycle cost estimate (LCCE) and  identify a final year for the program. CBP officials subsequently  identified fiscal year 2035 as the program’s end date and  submitted an updated LCCE in February 2016. However, this  LCCE only updated costs estimated through fiscal year 2026—9  years short of the program’s final year. Nevertheless, DHS  approved the program to transition into sustainment without an  understanding of the program’s full costs, as required by DHS  acquisition policy.
NII systems are commercial-off-the-shelf products, and for  this reason, DHS leadership decided that the NII Systems  Program does not need a Test and Evaluation Master Plan.  However, the program regularly tests NII systems and plans to  conduct operational assessments through FOC in fiscal year  2024. In August 2016, CBP officials said that they assessed  the performance capabilities of deployed units earlier in  the year. Among other things, CBP compared two fielded  NII systems to determine their operational effectiveness in  detecting contraband in both empty and loaded containers. The  two systems were found to be equally effective at detecting  contraband in empty containers, but one was generally  determined to be a better option for loaded containers.
As a part of the program’s ADE 3 approval, DHS leadership  also required CBP to reassess future program requirements. In  response, CBP developed a Capabilities Analysis Study Plan in  March 2016 outlining the methodology for an 8-month analysis  that will assess current capability gaps to ascertain future  program requirements. CBP plans to complete the analysis by  December 2016.
The NII Systems Program has also conducted testing for future  capabilities. In 2015, the program assessed whether NII and  radiation detection technology could be combined to examine  rail cargo, and whether cameras are capable of detecting new  welding—indicating the possible presence of contraband—in  moving trains. In August 2016, CBP officials told GAO that  preliminary assessments of these tests were positive and  the results will be further evaluated on fielded systems to  validate the return on investment. This will better inform future  acquisitions or systems upgrades where practical. For example,  CBP is conducting operational testing on one of its rail systems  with the combined radiation detection technology. If successful,  future rail systems will incorporate this upgrade.
From January 2016 to January 2017, the program’s acquisition  cost estimate decreased from $1.9 billion to $1.7 billion, and  the LCCE decreased from $4.5 billion to $4.2 billion. NII’s  estimates previously increased when CBP extended the  program’s lifespan from fiscal year 2022 to fiscal year 2026;  increased the total procurement quantity for large- and small- scale systems, from 9,427 to 11,448; and increased the number  of planned replacement systems by more than 2,000 units.  CBP officials reported that the updated LCCE is lower because  of reduced NII system costs in newly awarded and anticipated  contracts, reduced maintenance costs resulting from fixed  price maintenance contracts, and the replacement of some  NII systems that were costly to maintain. However, as noted  above, the updated LCCE does not account for operations and  maintenance or replacement costs through the program’s end  date of fiscal year 2035. CBP officials said they plan to update  the program’s LCCE in 2017.
As of August 2016, the NII Systems Program continued to  face a staffing gap of approximately 44 percent. The largest  shortfalls were in the program management and life cycle  logistics disciplines. According to CBP officials, the current  staffing gap has reached a critical point because of the risk  of acquisition and deployment delays. Officials said that the  program is utilizing contractor support, but this approach comes  at a higher cost than filling the vacancies with government  employees. Program officials explained that CBP has not hired  additional staff because of an ongoing realignment of CBP’s  organizational structure, and CBP is placing a higher priority on  hiring officers, such as Border Patrol agents, versus program  staff.
From fiscal year 2017 to 2021, the NII Systems Program’s  yearly cost estimates appear to exceed the program’s funding  plan by $253 million. However, the yearly cost estimates over  this 5-year period also include $138 million for operating and  maintaining radiation detection equipment acquired by the  Domestic Nuclear Detection Office. According to CBP officials,  the program has instituted positive cost controls including  a service life extension for some NII systems to address  affordability challenges, but funding shortfalls continue to be  the program’s greatest risk. Officials stated that the program  is sustainable with these cost controls, but most of the NII  systems will reach the end of their expected service lives  within the next 5 years. Without funding for replacement of  these critical systems, CBP officials said they will not have the  capability to scan cargo and will have to inspect cargo manually.  The program may also experience further slips in reaching  full operational capability (FOC). As we found in March 2016,  funding shortfalls previously caused the program’s FOC to slip   5 years—from fiscal year 2019 to fiscal year 2024.
Program Office Comments CBP officials provided technical comments on a draft of this  assessment, which GAO incorporated as appropriate.
Customs and Border Protection (CBP)
The TACCOM program is intended to upgrade land mobile radio  (LMR) infrastructure and equipment. It is replacing obsolete  radio systems with modern digital systems in 20 different service  areas, linking 19 of these service areas to one another through  a nationwide network, and building new communications towers  to expand coverage in 5 of the 20 service areas. The program is  delivering LMR capability to approximately 95,000 users at CBP  and other federal agencies. GAO previously reported on the  TACCOM program in March 2016 and March 2015 (GAO-16- 338SP, GAO-15-201).
Staff needed: 50.6 full time equivalents (FTE)
In July 2016, CBP officials told GAO that the TACCOM program  continued to meet its two key performance parameters, which  measure coverage area and the percentage of time the systems  are available. In May 2014, the Department of Homeland  Security’s (DHS) Director, Office of Test and Evaluation (DOT&E)  determined that the TACCOM program’s systems met their  performance requirements. Going forward, the TACCOM program  plans to conduct annual assessments in select locations to  monitor systems performance.
The TACCOM program was initially intended to upgrade LMR  infrastructure and equipment in 20 different service areas,  replacing obsolete radio systems with modern digital systems.  The program was also intended to build new communications  towers in all 20 of those service areas to expand LMR coverage.  However, CBP subsequently decided to reduce the number of  service areas where it would build new communications towers  from 20 to 5 due to funding constraints. In the 15 remaining  service areas, the program will still replace obsolete analog  radio equipment with modern digital systems, but it will not  expand coverage. The funding needed for tower construction in  one service area was adequate to replace the radio systems in  the 15 remaining service areas. management in order to initiate the upgrades. If DHS does  not reach an agreement with the Department of Justice on  the ownership and maintenance of the San Diego system, the  program expects that the funding gap will increase.
In addition to upgrading LMR capabilities within the 20 service  areas, the TACCOM program is also responsible for connecting  19 service areas to one another. CBP plans to do so by  replacing the circuitry that connects these service areas to  an existing nationwide network. CBP officials said this effort  constitutes the majority of the program’s remaining work, which  they anticipate will be completed in December 2017.
The TACCOM program conducts operational assessments  annually in select locations where upgrades were recently  completed to determine whether the system is operating as  intended. From March to June 2016, the program’s operational  test agent (OTA) conducted an operational assessment in  the Houlton and Miami sectors and concluded the TACCOM  systems were operationally effective and operationally  suitable. However, the OTA noted some limitations, including  interoperability with external users and collecting performance  data for management review. The OTA recommended the  program office conduct periodic user reviews by sector to  identify and resolve coverage shortfalls and establish a system  to collect and report on TACCOM performance monthly, among  other things. In August 2016, program officials told GAO they  monitor performance of TACCOM systems regularly and report  outages to CBP’s Chief Information Officer daily.
section_title: Program Governance
subsections: section_title: In January 2016, DHS leadership approved the TACCOM program’s transition to sustainment at the same time that it approved the program’s first Acquisition Program Baseline (APB). The APB establishes the program’s cost, schedule, and performance parameters. DHS’s current acquisition policy, which was first established in 2008, states that a program’s APB should be approved before the program starts obtaining new capabilities. Further, CBP awarded contracts in 2010 worth a total of $145 million to initiate upgrades in 3 of the 20 service areas, but the DHS’s Under Secretary for Management did not approve the TACCOM program’s operational requirements until September 2013.
paragraphs: According to officials, the program will also conduct another  operational test after it has connected the 19 service areas to  one another. Program officials said the risk associated with this  effort is low, but they do not expect to determine whether the  capability meets mission needs until June 2017. CBP conducted  operational testing in the Rio Grande Valley in December 2013  after the program had replaced obsolete radio systems with  modern digital systems and built new communications towers.  DOT&E concluded that the new TACCOM systems were  operationally effective, and that the systems will likely prove  suitable over time.
section_title: Program Execution
paragraphs: In August 2016, program officials said they had hired two  business managers and were actively working to fill the  program’s remaining staffing gap.
From January 2016 to January 2017, the program’s cost  estimates remained unchanged. However, TACCOM’s cost  estimates in its January 2016 APB reflected changes from  the program’s previous internal estimates. The acquisition  cost estimate decreased from $467 million to $397 million,  but the life-cycle cost estimate increased from $959 million to  approximately $1.1 billion when the program added government  personnel costs.
The program is projected to have a funding shortfall of over  $100 million from fiscal years 2017 through 2021. In August  2016, program officials explained that they have taken steps  to mitigate the anticipated funding gap by cutting TACCOM’s  $4 million real properties budget in half; reducing manpower  support contracts, travel, and gas; and performing minimum  maintenance; among other things. However, they also explained  that the anticipated funding shortfall may have a substantial  long-term impact on operations and maintenance.
Program Office Comments The deployed system is consistently exceeding the objective  value for its operational availability key performance  parameter. The program implements a formal process  to review and update life-cycle cost estimates annually.  Program officials also provided technical comments on  a draft of this assessment, which GAO incorporated as  appropriate.
Customs and Border Protection (CBP)
TECS (not an acronym) is a law-enforcement information  technology system that helps CBP officials determine the  admissibility of persons wanting to enter the United States at  border crossings and ports of entry as well as pre-screening  sites located abroad. The legacy TECS system has been in  place since the 1980s and is obsolete, expensive to maintain,  and unable to support CBP’s evolving mission needs. In 2008,  the Department of Homeland Security (DHS) initiated efforts to  modernize TECS to provide its users with enhanced capabilities  for accessing and managing data. Immigration and Customs  Enforcement (ICE) is executing a separate TECS Modernization  program, which GAO is also assessing in this report. GAO  previously reported on CBP’s TECS Modernization program in  March 2016 (GAO-16-338SP).
Staff needed: 38.35 full time equivalents (FTE)
In August 2016, CBP officials told GAO the program had met its  remaining key performance parameter (KPP), which establishes  how quickly the system can create a new, searchable record.  CBP officials previously told GAO in August 2015 that the  program had met its other five KPPs, but DHS’s Director, Office  of Test and Evaluation (DOT&E) has not validated this assertion.  According to officials, the program will demonstrate its KPPs  through a series of four operational test events scheduled  between September 2016 and January 2017. After the final  event, DOT&E is to assess the test results to validate the  program’s performance.
To modernize TECS, CBP is replacing its legacy, mainframe- based platform with a combination of hardware, custom- developed and commercial software, and a web-based portal  that will allow TECS to deliver capabilities to users within CBP  and other DHS agencies. The TECS Modernization program  consists of five projects, and officials stated CBP initially used  an incremental acquisition approach for four of these projects.  However, CBP is now using an agile software development  methodology for all five of the projects. Under the agile software  development methodology, programs deliver software in short,  small increments—rather than long, sequential phases—which  allows programs to measure interim progress.
In April 2016, the program updated its life-cycle cost estimate,  which remained largely unchanged since November 2010.  According to CBP officials, the schedule delays have had  little to no effect on the program’s cost estimate or end users  because the legacy TECS system remains active.
According to program officials, the program leverages existing  CBP contracts to support TECS Modernization efforts. In June  2008, CBP awarded a contract to Bart & Associates, Inc. to  develop software and provide operations and maintenance  support. CBP exercised options on this contract from 2009 to  2012. However, the program experienced delays during this  period. Officials told GAO that, in 2013, CBP awarded a new  development and support contract to Northrop Grumman.  That February, Bart & Associates, Inc. and two other firms  submitted bid protests to GAO. CBP took corrective action,  and 20 months later awarded another contract to Northrop  Grumman in September 2014. Bart & Associates, Inc. submitted  a second protest, which GAO denied. In January 2015,  Northrop Grumman resumed work under the awarded contract  that is being used to support TECS Modernization application  development activities.
In May 2016, DOT&E approved a fourth version of CBP TECS  Modernization’s Test and Evaluation Master Plan that provided  additional information on how cybersecurity threats would  be addressed during operational testing starting in fall 2016.  According to officials, CBP conducted three operational test  events in September and November 2016—one event each  at a land border crossing, a seaport, and an airport—prior to  conducting a fourth operational test event in January 2017  that will verify final integration of the system’s hardware and  software at both the primary and secondary data centers. CBP  officials anticipate receiving preliminary results as testing is  conducted, but said a test report encompassing all four events  will be submitted to DHS’s DOT&E for assessment after the  final test is complete. They explained that the January 2017  operational test event is the program’s biggest challenge  because it will test integration of the TECS Modernization’s  hardware and software with DHS’s network.
In August 2016, CBP officials stated that staffing shortfalls  related to the previous bid protests have been resolved.  Officials do not plan to fill the remaining two full time equivalents  because they are for requirements analysts, which the program  no longer needs this late in the acquisition life cycle.
section_title: Program Execution
subsections: section_title: In the most recent APB, the program split FOC into two separate operational capability milestones at each data center. Operational capability at the primary data center is projected for December 2016 and includes transitioning all TECS users to the modernized system to support the retirement of the legacy system. However, program officials said CBP will need to keep the legacy system active through fiscal year 2017 to support other programs that use TECS data. Operational capability at the secondary data center is projected for June 2017 and is a planned enhancement to the legacy system that will provide CBP redundant TECS access to minimize downtime during system maintenance or unscheduled outages. In total, this represents a slip of 18 months from the program’s initial FOC date from its first APB in November 2010.
paragraphs: Program Office Comments CBP officials provided technical comments on a draft of this  assessment, which GAO incorporated as appropriate.
Logistics Supply Chain Management System (LSCMS)
Federal Emergency Management Agency (FEMA)
LSCMS is a computer-based tracking system that FEMA  officials use to track shipments during disaster-response  efforts. It is largely based on commercial-off-the-shelf software.  FEMA initially deployed LSCMS in 2005, and initiated efforts  to enhance the system in 2009. According to FEMA officials,  LSCMS can identify when a shipment leaves a warehouse and  the location of a shipment after it reaches a FEMA staging area  near a disaster location. However, LSCMS cannot track partner  organizations’ shipments—such as those by state and local  governments—en route to a FEMA staging area, and it lacks  automated interfaces with its partners’ information systems.  GAO previously reported on LSCMS in March 2016 (GAO-16- 338SP).
Staffing gap: 3 FTEs equivalents (FTE)
FEMA plans to conduct additional operational testing on the  system by March 2018, after the program completes anticipated  upgrades, including the capability to interface automatically with  its partners’ information systems. According to FEMA officials,  LSCMS previously demonstrated it could meet all seven of its  key performance parameters (KPP) through either operational or  developmental testing. However, the Department of Homeland  Security’s (DHS) Director, Office of Test and Evaluation (DOT&E)  noted that the testing was not adequate and recommended  FEMA retest LSCMS. FEMA subsequently met two of its KPPs  during a performance test for a single software release.
In March 2016, DHS leadership authorized LSCMS to resume  all development and acquisition efforts after a nearly 2-year  program pause. FEMA deployed the enhanced LSCMS  in 2013 without approval from the DHS Under Secretary  for Management (USM) or key documentation such as a  department-approved Acquisition Program Baseline (APB) or a  DOT&E letter of assessment, as required by DHS’s acquisition  policy. In April 2014, based on the preliminary results of a  DHS Office of Inspector General (OIG) report, the Acting USM  directed FEMA not to initiate the development of any new  LSCMS capabilities until further notice. The DHS OIG noted  that neither DHS nor FEMA leadership ensured the program  office identified all mission needs before selecting a solution,  and the Acting USM instructed FEMA to conduct an analysis of  alternatives to address LSCMS’s remaining capability gaps.
In June 2015, a contractor completed the analysis of  alternatives and recommended that FEMA pursue the current  version of LSCMS plus additional capabilities that would  improve coordination with partner organizations. On the basis  of this assessment, in August 2015, FEMA officials stated they  were planning to pursue an upgrade known as Electronic Data  Interchange (EDI), which would provide LSCMS with the ability  to automatically interface with its partners’ information systems.  In December 2015, DHS’s USM approved the program’s APB,  which established cost, schedule, and performance parameters  for LSCMS’s new capabilities.
In March 2016, DHS’s USM directed the program to select  a new operational test agent (OTA) and develop a Test and  Evaluation Master Plan (TEMP) to address issues identified  through past operational testing. Previously, FEMA deployed the  enhanced LSCMS in January 2013 before operationally testing  the system. When the operational test was conducted, DHS’s  DOT&E determined that the test was inadequate. The OTA at  the time—the Department of Defense’s Joint Interoperability  Test Command—conducted the operational testing throughout  calendar year 2013, leveraging performance data from the field,  including data collected during FEMA’s responses to real-world  disasters. The OTA’s conclusions were generally positive, but  DOT&E determined in June 2014 that these conclusions were  not supported by the test results, in part because the test’s  sample size was not adequate. DOT&E recommended that  the program conduct additional operational testing. In June  2016, DOT&E approved FEMA’s selection of a new OTA for  LSCMS. In November 2016, DOT&E approved the program’s  TEMP, which defines a new overall testing approach for  evaluating unresolved issues from previous testing along with  LSCMS’s new capabilities. The new TEMP also includes plans  for cybersecurity testing. FEMA plans to complete additional  operational testing by March 2018, once the security upgrades  and the addition of the EDI capability are complete.
section_title: Program Execution
subsections: section_title: LSCMS is on track to meet the schedule and cost parameters established in its December 2015 APB. In September 2016, officials told GAO they had initiated several efforts after receiving authority to resume development. For example, the program is working to implement security upgrades, add the EDI capability to the existing system, and integrate LSCMS with DHS’s asset management system for fixed property. After completing planned upgrades, FEMA expects LSCMS to reach full operational capability by December 2018—over 7 years later than the program planned when FEMA initiated efforts to enhance LSCMS in 2009. FEMA officials previously attributed this schedule slip to the need to address capability gaps identified by the DHS OIG and staffing shortages, among other factors.
paragraphs: In September 2016, FEMA officials told GAO that the LSCMS  program had 22 of the 25 full time equivalents (FTE) it needed  and was working to recruit additional staff. This represents  a significant improvement from fiscal year 2014, when GAO  found that the program had only 7 of the 22.5 FTEs it needed  (GAO-15-171SP). Officials previously attributed the program’s  governance and testing challenges in part to staffing shortages.  In September 2016, officials stated that the addition of new staff  has helped the program to update acquisition documents and  conduct business analyses that may help identify future cost  savings.
FEMA officials told GAO in September 2016 that they anticipate  meeting, or potentially coming in under, the program’s APB cost  threshold of $814 million, based on an April 2016 update to  the program’s approved life-cycle cost estimate. This estimate  represents a nearly $500 million increase from the program’s  initial 2009 estimate, which was never approved by DHS. FEMA  officials previously stated that the 2009 life-cycle cost estimate  did not account for costs beyond fiscal year 2017, among other  things.
From fiscal year 2017 through fiscal year 2021, LSCMS’s yearly  cost estimates exceed the program’s funding plan by almost  $29 million. However, the program’s updated life-cycle cost  estimate includes approximately $35 million in costs for some  services, such as personnel, that are funded by organizations  external to LSCMS. When excluding the externally funded  costs, the program is affordable during this 5-year period.
Program Office Comments FEMA officials reviewed a draft of this assessment and  provided no comments.
Immigration and Customs Enforcement (ICE)
ICE is responsible for investigating and enforcing border  control, customs, and immigration laws. The legacy TECS (not  an acronym) system has supported ICE’s mission since the  1980s, providing case management, intelligence reporting, and  information sharing capabilities. However, the legacy system is  obsolete, expensive to maintain, and unable to support ICE’s  growing mission needs. In 2009, ICE began efforts to modernize  aging TECS functionality and provide end users with additional  functionality required for mission execution. The Department of  Homeland Security’s (DHS) Customs and Border Protection is  executing a separate TECS Modernization program, which GAO  has also assessed in this report. GAO previously reported on  ICE’s TECS Modernization program in March 2016 (GAO-16- 338SP). equivalents (FTE)
The modernized ICE TECS system demonstrated two of its  three key performance parameters (KPP) during operational  testing conducted from August to October 2016. However,  DHS’s Director, Office of Test and Evaluation (DOT&E) has not  yet validated these results. The third KPP, related to concurrent  users, was not tested and ICE officials said it will be difficult for  the program to meet this KPP because the requirements are not  realistic. The current KPP threshold assumes 6,000 officers will  use the system simultaneously. In August 2016, officials said data  showed there are between 500 and 600 concurrent users. ICE  officials said they are working with end users to revise the KPP  threshold prior to full operational capability (FOC).
ICE initiated efforts to modernize the TECS system with a  custom-developed solution in 2011. By June 2013, ICE officials  determined that the existing TECS Modernization approach was  unfeasible and subsequently restructured the program. The  program now leverages commercial-off-the-shelf products, and  is no longer pursing a custom-developed solution. According  to the program manager, the program is acquiring capabilities  through four concurrent “work streams,” each of which delivers  discrete portions of the system’s total planned functionality.  Different contractors are responsible for different work streams,  and the program office is managing their efforts and integrating  their software. Program officials said that this approach is  intended to improve management visibility into each of the  contractor’s efforts. However, officials added that integrating  the program across the four work streams has presented  challenges and that ICE has utilized multiple techniques to  address these challenges including co-locating all work stream  teams, conducting daily coordination meetings, and establishing  a cross-program body of ICE and DHS technical experts to  address integration issues. data from the legacy system but deferred final evaluation of  the modernized system’s operational suitability, operational  effectiveness, and KPPs until further testing could be conducted  in a production environment. The program’s operational test  agent conducted initial operational test and evaluation from  August 2016 to October 2016. ICE initially planned to start  this testing in May 2016, but it slipped once IOC was delayed.  The operational test agent determined the modernized ICE  TECS system was operationally effective and operationally  suitable with limitations, and recommended the program  conduct additional tests related to cybersecurity prior to FOC,  among other things. The final operational test agent report was  released in December 2016, and DOT&E plans to complete an  assessment of the results by the end of February 2017.
ICE officials told GAO they plan to revise the program’s Test  and Evaluation Master Plan once FOC functionality is finalized  and conduct follow-on operational test and evaluation prior to  achieving FOC in September 2017. According to officials, final  testing will include threat-based cybersecurity testing.
section_title: Program Execution
subsections: section_title: The program achieved initial operational capability (IOC) in June 2016—3 months later than previously planned. The program office attributed the delay to resolving technical problems with Customs and Border Protection support services that emerged during final integration testing. In February 2016, ICE notified DHS that it would breach its re-baselined IOC threshold date of March 2016 and subsequently revised its Acquisition Program Baseline and life-cycle cost estimate (LCCE) to account for the delay. According to officials, IOC entailed delivering 80 percent of the modernized TECS system’s functionality to operators in the field and successfully transitioning ICE off the legacy system.
paragraphs: Prior to IOC, program officials stated the program conducted a  “soft launch” of the case management capabilities at the New  York field office, which allowed users to update their credentials,  conduct test searches, and insert test records into the  modernized TECS system. Program officials stated the exercise  helped users get comfortable with the new TECS system and  allowed the program to initiate the transfer of user provisions  from the legacy system to the modernized system. In August  2016, program officials told GAO that use of the modernized  TECS system since IOC has been consistent across all field  offices and they have received positive feedback from ICE  field agents that the system is meeting their day-to-day needs.  Program officials stated that ICE established a 24/7 Command  Center for the first 4 weeks following IOC implementation to  address end user problems and concerns. These officials  added that they continue to track help desk tickets on a weekly  basis and plan to release monthly updates to address identified  issues.
In achieving IOC, ICE has overcome past technical difficulties  and schedule delays. In June 2014, DHS’s Under Secretary  for Management re-baselined the program to reflect ICE’s  new acquisition approach. The program’s IOC date slipped  from December 2013 to March 2016, but the FOC date moved  forward, from December 2017 to September 2017. In August  2016, ICE officials told GAO that the program remains on  track to achieve its revised FOC date. However, at that time,  the program had not yet identified what FOC would entail  and officials stated that they were working with end users to  determine final FOC functionality. ICE officials subsequently  said they completed FOC planning activities in October 2016,  including confirming FOC functionality such as enhanced  system search capabilities.
From January 2016 to January 2017, the program’s acquisition  cost estimate increased by $4 million. ICE officials attributed  this increase to including actuals for a data center contract that  was awarded in 2016. However, the program’s cost estimates  previously decreased significantly when the program revised  its acquisition approach. In fiscal years 2017 and 2018, the  program is projected to face a $5 million funding gap. However,  ICE officials anticipate utilizing a multi-year appropriation to  cover the projected gap.
Program Office Comments ICE officials provided technical comments on a draft of this  assessment, which GAO incorporated as appropriate.
Continuous Diagnostics & Mitigation (CDM)
National Protection and Programs Directorate (NPPD)
The CDM program is intended to strengthen the cybersecurity  of the federal government’s computer networks by providing  sensors and dashboards to more than 65 participating civilian  departments and agencies. The sensors continually monitor  agency networks for vulnerabilities rooted in both hardware and  software, and automatically notify agency personnel through  dashboards when vulnerabilities are detected. CDM is also  delivering a government-wide dashboard to the Department  of Homeland Security (DHS), which will extract data from the  agency-level dashboards and enhance situational awareness  across the federal government. In June 2016, DHS leadership  directed the program to re-baseline for the third time to address  implementation challenges and to account for additional  capabilities. GAO previously reported on the CDM program in  March 2016 (GAO-16-338SP).
Staffing gap: 20 FTEs Actual staff: 31 FTEs equivalents (FTE)
CDM currently has 12 key performance parameters (KPP),  which it has not yet demonstrated. However, in August 2016,  NPPD officials told GAO that they were revising the program’s  operational requirements document as a part of the program’s  re-baseline effort. Officials said DHS leadership directed the  program to consolidate its existing 12 KPPs, but the program  may add KPPs to account for additional capabilities.
CDM plans to provide sensors and tools to the departments  and agencies in four phases. Phase 1 sensors will report  vulnerabilities in hardware and software; phase 2 tools will  report on user access controls; phase 3 tools will report on  department and agency efforts to prevent attacks and limit  the impact of ongoing attacks; and phase 4 tools will focus  on encryption and other data masking techniques to protect  data on the network. Phase 4 was added at the request of the  Office of Management and Budget (OMB) in December 2015  to address vulnerabilities on government networks that threats  may seek to exploit. The General Services Administration  (GSA) is administering CDM’s contracts using blanket purchase  agreements (BPA) established under vendors’ GSA Federal  Supply Schedule contracts. Through these BPAs, the program  issues task orders to acquire commercial-off-the-shelf software,  hardware, and services. In June 2016, GSA awarded the final  phase 1 task order—to deliver sensors for 44 agencies—as  well as the first of two task orders for phase 2 tools—to provide  tools for managing user network privileges at 69 agencies. GSA  awarded the second phase 2 task order in November 2016,  which will provide tools for verifying user network credentials at  26 agencies. GSA previously awarded five task orders to deliver  phase 1 sensors to 25 agencies and a separate task order for  the agency-level and government-wide dashboards.
DHS leadership approved three versions of CDM’s APB  between 2013 and 2015. In each new version, the program’s  cost estimates and schedule changed. The program’s third  APB, which DHS leadership approved in August 2015, reflected  schedule slips that officials largely attributed to contracting  delays. The program’s acquisition cost estimate increased to  $2.7 billion for several reasons, including increased staff levels  and costs for sensor replacement. In contrast, the program’s  LCCE decreased to $2.7 billion when NPPD determined it  did not need to support all of the sensors CDM offers at all  agencies and DHS leadership determined CDM would only fund  the operation and maintenance of CDM sensors, tools, and  dashboards for the first 2 years of deployment, rather than over  their entire life cycles.
section_title: Program Execution
paragraphs: CDM is only authorized to conduct testing on DHS networks,  which means the other departments and agencies are  responsible for testing the CDM sensors and dashboards on  their own networks. In August and October 2016, the contractor  providing phase 1 sensors for the DHS network conducted initial  testing to demonstrate their functional requirements. CDM’s  test team found that 65 percent of the requirements were not  demonstrated or not tested during these events. The program  plans to work with the contractor to identify and address  reasons why the requirements were not met or tested. In August  2016, NPPD officials said they had observed operational testing  conducted at three agencies and plan to revise CDM’s Test and  Evaluation Master Plan as a part of the programs’ re-baseline  effort.
In June 2016, DHS leadership directed CDM to re-baseline  for the third time to account for the addition of phase 4 and to  address challenges encountered during phase 1. Specifically,  contractors found large gaps for 12 of the agencies receiving  phase 1 sensors in the actual number of network-connected  devices needing coverage from what was originally reported.  The gaps in coverage ranged from 19 percent to 384 percent.  NPPD officials attributed the gaps to different interpretations  by some agencies of what devices should have been counted,  as well as a time lag between when the agencies reported  their coverage needs and when GSA awarded the task orders.  In August 2016, program officials said that DHS leadership  instructed CDM to self-fund the increased cost caused by  the gaps, which NPPD estimated to be at least $66 million to  support all agencies except the U.S. Postal Service (USPS).  USPS had the largest identified coverage gap, which NPPD  estimated would cost an additional $93 million to cover.  According to program officials, USPS will fund the cost of  covering its own phase 1 sensors, but NPPD will provide two  subject matter experts to support USPS’s efforts.
In December 2016, NPPD officials told GAO the program’s  authorized staffing levels had increased from 30 to 51 full time  positions, but that CDM continued to face significant staffing  shortages and needed a program manager. Officials said the  staffing gap of 20 full time positions—meaning actual personnel  rather than equivalents—forces the program to divert individuals  from their normal responsibilities to critical areas, such as  project management. NPPD is actively working to fill CDM’s  vacancies, but officials said they struggle to hire new staff due  to lengthy security clearance processes.
As of January 2017, NPPD had not yet completed the CDM   re-baseline effort, which officials said will include revisions to  the program’s Acquisition Program Baseline (APB) and life-cycle  cost estimate (LCCE). NPPD officials anticipate the program’s  cost estimates will increase and acknowledged that the  phase 1 gaps will likely delay the program’s ability to execute  subsequent phases. To cover the phase 1 gaps, NPPD officials  said they deferred $30 million of phase 2 funding by limiting the  number of agencies covered by phase 2 tools and used $36  million originally planned for phase 3. In fiscal year 2017, OMB  plans to allocate an additional $172 million to DHS to accelerate  deployment of CDM phase 3 capabilities and to support creation  of phase 4. Despite the challenges encountered with phase  1, CDM achieved initial operational capability by its revised  deadline of December 2016 after the program delivered sensors  and dashboards to five agencies.
Program Office Comments The program continues to re-baseline and is targeting April  2017 for completion. CDM continues to manage its budget  to ensure program costs match available funding. CDM is  leveraging the collective buying power of federal agencies  and strategic sourcing to achieve over $344 million in  government cost savings on CDM products—a 61 percent  savings compared to GSA’s Schedule 70. As of December  2016, CDM has deployed dashboards to nine agencies  and is planning to deploy the government-wide dashboard  in June 2017. CDM has received many accolades from  agencies and federal leaders. NPPD officials also provided  technical comments on this assessment, which GAO  incorporated as appropriate.
Homeland Advanced Recognition Technology (HART)
National Protection and Programs Directorate (NPPD)
HART is intended to replace and modernize the Department  of Homeland Security’s (DHS) legacy biometric identification  information system known as the Automated Biometric  Identification System (IDENT). Since 1994, IDENT has  enhanced national security and facilitated legitimate travel,  trade, and immigration by receiving, maintaining, and sharing  information on foreign nationals with DHS border management  organizations, other federal agencies, law enforcement, and  foreign partners. However, IDENT is at risk of failure because  it cannot keep pace with a growing number of daily system  transactions. In 2011, DHS initiated efforts to replace IDENT  with HART in order to provide users with enhanced capabilities  for accessing and managing biometric identification data.
Staff needed: 168 full time equivalents (FTE)
HART is still in a relatively early acquisition stage, and the  program has not yet demonstrated whether it can meet its eight  key performance parameters (KPP). The program plans to  demonstrate its KPPs as capabilities are developed. The first  two KPPs establish requirements for system availability and  a fingerprint biometric identification service. The next set of  four KPPs establishes requirements for multimodal biometric  verification services and interoperability with a Department of  Justice system. The program’s remaining two KPPs establish  requirements for web portal response time and reporting  capabilities.
Acquisition Strategy  HART plans to develop and deploy capabilities through 4  increments: increments 1 and 2 are intended to replace and  enhance existing IDENT system functionality, and increments  3 and 4 are intended to provide additional capabilities.  Specifically, increment 1 will provide the core infrastructure  including system hardware and basic functionality to  operate HART. Increment 2 will provide enhanced biometric  capabilities, such as facial and iris identification, and the full test  environment for measuring system performance. Increment 3  will introduce a web portal to improve system accessibility and  provide a holistic person-centric view of biometric identification  data. Increment 4 will provide additional tools for improved data  analysis and reporting capabilities.
DHS’s Director, Office of Test and Evaluation approved the  HART program’s Test and Evaluation Master Plan in September  2016, after the program incorporated feedback from DHS’s  Science and Technology Directorate (S&T) and HART’s  operational test agent, the Department of Defense’s Joint  Interoperability Test Command. For example, the program  revised its developmental test and evaluation strategy,  added risk assessment levels for planned tests, and aligned  cybersecurity objectives with requirements. HART plans to  conduct operational testing for increment 1 in June 2018 prior to  achieving IOC.
According to NPPD officials, the program is focused on  awarding an initial contract for the development and delivery of  increments 1 and 2, and plans to pursue separate contracts for  the development and delivery of increments 3 and 4.
Additionally, S&T’s Office of Systems Engineering completed  a technical assessment on HART in February 2016, and  concluded that the program had a moderate overall level of  technical risk. In October 2016, DHS’s USM directed HART to  work with S&T to monitor the risks identified in the technical  assessment, and directed S&T to conduct further analysis  following the program’s initial contract award for increments 1  and 2.
In April 2016, DHS’s Under Secretary for Management  (USM) approved HART’s Acquisition Program Baseline  (APB)—which established the program’s cost, schedule, and  performance parameters—and authorized the program to  initiate development efforts for increments 1 and 2 in October  2016. HART plans to achieve initial operational capability (IOC)  with the deployment of increment 1 in December 2018, at which  point program officials anticipate beginning to transition users  from the legacy IDENT system to HART. HART plans to achieve  full operational capability with the deployment of increment 4 by  September 2021.
NPPD reported the program had a staffing gap of 12 full time  equivalents, but in September 2016, program officials did not  attribute any negative affects to workforce shortages. Program  officials said that they plan to hire additional contractors to  support the new systems integrator, and will transition existing  staff to support HART efforts as the legacy IDENT system is  decommissioned. The program is also undergoing efforts to  determine future staffing needs. Program officials said they  proactively engaged the Office of Personnel Management to  conduct a workforce analysis. Additionally, DHS directed the  program to conduct a staffing analysis with assistance from  the department’s Chief Technology Officer to determine any  gaps, particularly in the cyber security field. The results of this  analysis are required to be completed by March 2017.
NPPD officials told GAO that the program’s schedule and  cost estimates may change once they award the contract for  increments 1 and 2 and receive the contractor’s proposed  technical solution. The program has experienced delays in  awarding the contract. In September 2016, NPPD officials  told GAO that the program received and incorporated industry  feedback into the request for proposal (RFP) in July 2016. In  October 2016, NPPD officials told GAO that the program was  resolving a potential issue with the final RFP and had released  a second draft RFP in order to maintain communication with  industry. Program officials anticipate releasing the final RFP  in January 2017. Subsequently, they plan to update HART’s  schedule and cost estimates once the contract for increments  1 and 2 is awarded because the contractor’s proposed solution  will assist officials in determining how much of the legacy IDENT  system can be reused for HART, a factor that may affect the  program’s cost estimate.
DHS proposed moving the IDENT and HART programs from  NPPD to Customs and Boarder Protection in its fiscal year 2017  budget submission. In September 2016, program officials told  GAO that the transition had not yet been approved and that  HART would remain with NPPD through at least the end of the  fiscal year 2017.
From fiscal year 2017 through fiscal year 2021, HART is  projected to face a $406 million funding gap. In April 2016,  NPPD identified that DHS plans to program an additional $335  million to the program over this 5-year period. In September  2016, program officials stated that they have taken steps  to mitigate remaining shortfalls. For example, the program  extended the planned schedule for technical refreshes from 5  years to 7 years, carried over $39 million into fiscal year 2016,  and identified approximately $27.3 million of no-year funding  in fiscal year 2016 that could be used to cover the anticipated  funding gap.
Program Office Comments NPPD officials provided technical comments on a draft of  this assessment, which GAO incorporated as appropriate.
National Cybersecurity Protection System (NCPS)
National Protection and Programs Directorate (NPPD)
NCPS is intended to defend the federal civilian government’s  information technology infrastructure from cyber threats. The  Department of Homeland Security (DHS) established the  program to acquire hardware, software, and services, and  NCPS delivers capabilities through a series of interdependent  upgrades designated “blocks.” Blocks 1.0, 2.0, and 2.1 are  fully deployed and collectively provide intrusion-detection and  analytic capabilities across government agencies. NCPS is  currently deploying EINSTEIN 3 Accelerated (EA), previously  designated block 3.0, which is intended to provide an intrusion- prevention capability. Going forward, NCPS plans to deliver  block 2.2 to improve information sharing across agencies. GAO  previously reported on the NCPS program in March 2016 and  January 2016 (GAO-16-338SP, GAO-16-294).
Staff needed: 176 full time equivalents (FTE)
In June 2015, DHS’s Director, Office of Test and Evaluation  (DOT&E) found EA testing in June 2016 and plans to initiate block 2.2  testing in March 2017.
The program originally planned to use government technology  to deliver block 3.0 intrusion-prevention capabilities, but in May  2012, it significantly changed its acquisition strategy, decided  to work directly with commercial internet service providers  (ISP), and designated the revised effort EA intrusion-prevention  capabilities are now primarily provided through sole source  contracts with the nation’s largest ISPs to maximize coverage.  However, in May 2015, NCPS decided to provide EA’s coverage, but the program developed a plan that  instead allowed it to expand its coverage. Program officials said  they awarded a contract to provide basic intrusion-prevention  services at a greater number of federal agencies and enable  the program to have the capacity to cover all federal email and  internet traffic. However, officials noted that providing intrusion- prevention services has some challenges, such as protecting  classified information used to identify threats on unclassified  networks and rolling out these services across the federal  government.
The EA had  a significant effect on the program’s schedule. Among other  things, the program delayed an acquisition decision to operate  deployed capabilities until July 2015—when DHS leadership  reviewed the results of EA until  December 2017.
In June 2015, DHS’s DOT&E evaluated the results of EA OA.  Program officials anticipate receiving final OA results at the end  of January 2017 and have begun planning for initial operational  test and evaluation, which is planned for late fiscal year 2017.
In December 2015, Congress required federal government  agencies and departments to adopt intrusion-prevention  services, such as NCPS’s EA at approximately 93 percent of  federal agencies and departments. Program officials cited legal  and network challenges as barriers to integration because they  must negotiate and customize EA services.
According to program officials, NCPS plans to conduct an  OA on block 2.2 capabilities in March 2017 after it completes  adoption with HSIN. The results of this OA will inform the  program’s Block 2.2 ADE 2C scheduled for December 2017.
Program Office Comments Since the last assessment, the NCPS program office has  made progress toward achieving program objectives.   Departments and agencies have continued to onboard EA  service. NCPS continues to work with agencies to provide  all available EINSTEIN protections. Also in 2016, the  NCPS program office developed and implemented a plan  to leverage an existing DHS investment to meet a portion  of the NCPS information sharing requirements (block  2.2), resulting in a cost savings for the program. Program  officials also provided technical comments on a draft of this  assessment, which GAO incorporated as appropriate.
Next Generation Networks Priority Services (NGN-PS)
National Protection and Programs Directorate (NPPD)
NGN-PS is intended to address an emerging capability gap  in the government’s emergency telecommunications service,  which prioritizes select officials’ phone calls when networks  are overwhelmed. NPPD executes the NGN-PS acquisition  program through commercial telecommunications service  providers, which address the government’s requirements as  they modernize their own networks. NPPD plans to execute  NGN-PS in phases—voice, video, and data—and is currently  focused on the voice phase. Once NGN-PS capabilities become  operational, NPPD’s Priority Telecommunications Services  (PTS) program assumes responsibility for sustaining them. The  cost and funding figures in this assessment account for both  NGN-PS and PTS in accordance with Department of Homeland  Security (DHS) guidance. GAO reported on the NGN-PS  acquisition program in March 2016 (GAO-16-338SP).
Actual staff: 12 FTEs equivalents (FTE)
In August 2016, NPPD officials told GAO that NGN-PS continued  to meet all six of its key performance parameters (KPP) for the  voice phase, but DHS’s Director, Office of Test and Evaluation  (DOT&E) has not yet validated the program’s performance.  NPPD officials noted that each emergency is unique and that  performance can be affected by damage to telecommunications  infrastructure. NPPD officials also stated that they are in the  process of developing additional KPPs for the video and data  capabilities of NGN-PS.
NGN-PS was established in response to an Executive  Order requiring the federal government to have the ability to  communicate at all times during all circumstances to ensure  national security and manage emergencies. The NGN-PS  program works with telecommunication service providers as  they enhance their carrier networks so they can provide select  government officials survivable telecommunications capability  nationwide through the PTS program. The NGN-PS voice  phase is divided into three increments: increment 1 includes  paying service providers to ensure their major core networks  can continue to prioritize government phone calls as needed;  increment 2 delivers wireless capabilities; and increment 3 is  intended to address landline capabilities. NGN-PS has initiated  the first two increments and awarded three base contracts in  2014, each of which includes 9 option years. In August 2016,  NPPD officials said they had begun planning for the third  increment.
2015, the full operational capability date for increment 1 slipped  from June 2017 to March 2019, which NPPD officials attributed  to funding shortfalls. In August 2016, NPPD officials said they  do not anticipate further schedule slips for planned increment  1 and 2 activities. The program plans to use surplus funding  expected in fiscal years 2019 through 2021 to implement new  services such as landline capabilities.
In July 2016, the White House issued a Presidential Policy  Directive that superseded previous directives requiring  continuous communication services for select government  officials.  NPPD officials said the new directive validates the  program’s requirements and that they do not expect it to affect  the program’s costs or schedule. NPPD officials noted that  they plan to update the Acquisition Program Baseline once the  impact of the new directive is understood, but could not provide  a timeframe for when this will be complete. The NGN-PS  data and video capabilities were initially planned as separate  phases, but in August 2016, NPPD officials told GAO that they  plan to acquire them together. NPPD officials explained that  it now makes more sense to consolidate the data and video  capabilities as a result of technological advancements achieved  since the program’s acquisition plan was developed in 2013.  The data and video phase is in the early planning stages and  NPPD officials said they plan to work with stakeholders to refine  requirements based on the July 2016 directive.
DHS’s DOT&E approved a revised Test and Evaluation Master  Plan for the NGN-PS program in June 2016, which clarified  the program’s existing testing approach. Specifically, NGN- PS capabilities are evaluated through developmental testing,  government acceptance testing, and operational assessments.  The service providers play a central role in NGN-PS test  activities because they conduct the developmental testing and  operational assessments on their own networks. NPPD officials  review the service providers’ test plans, oversee tests to verify  testing procedures are followed, and approve test results  to determine when testing is complete. The government’s  operational test agent (OTA)—the Department of Defense’s  Joint Interoperability Test Command—does not plan to conduct  a stand-alone operational test event for NGN-PS. Instead, the  OTA leverages the service providers’ test and actual operational  data to assess program performance. NPPD officials told GAO  that NGN-PS has performed well when its capabilities have  been tested and deployed. NPPD officials also said that they  continuously review actual NGN-PS performance and that all  service providers undergo annual network service verification  testing under the PTS program.
section_title: Program Execution
paragraphs: In January 2016, NPPD reported that NGN-PS’s required  staffing level decreased by approximately 5 full time  equivalents, and that the program no longer had a staffing gap.  In August 2016, NPPD officials said that these numbers only  account for funded positions and that NGN-PS also relies on  about 20 contracted staff to execute day-to-day activities. NPPD  officials also stated that the NGN-PS leverages support from  PTS program staff, as needed.
From January 2016 to January 2017, the NGN-PS program’s  department-approved cost and schedule goals remain  unchanged. However, NPPD officials stated that they are in  the process of revising the program’s life-cycle cost estimate  (LCCE) to clarify NGN-PS costs because past estimates had  double counted some operations and sustainment costs that  are funded by PTS. From September 2010 to September 2014,  NGN-PS’s LCCE increased to $1.1 billion when the program  accounted for the voice phase’s second increment. In August  2015, DHS’s Chief Financial Officer approved a revised cost  estimate that increased the LCCE to $1.2 billion. NPPD officials  attributed the increase to the inclusion of sustainment costs  for the PTS program, as requested by DHS headquarters. In  August 2016, NPPD officials told GAO they plan to specifically  identify operations and sustainment costs attributable only to  NGN-PS acquisition efforts in the updated LCCE. In addition,  program officials said the LCCE will account for changes related  to the new Presidential Policy Directive.
Program Office Comments The NGN-PS LCCE update is the refined analysis of  development service acquisition costs that will include  separate projections for the annual impact of validated  NGN-PS capabilities that are transferred to the PTS  operations program. The LCCE update will more accurately  represent NGN-PS technical support for authorized users to  have seamless priority services for critical communications  during crises while commercial service providers  evolve their infrastructure—while meeting or exceeding  performance metrics and managed under budget. NPPD  officials also provided technical comments on a draft of this  assessment, which GAO incorporated as appropriate.
National Bio and Agro-Defense Facility (NBAF)
Science and Technology Directorate (S&T)
The NBAF program is constructing a state-of-the-art laboratory  in Manhattan, Kansas, to enable the United States to conduct  comprehensive research, develop vaccines, and provide  enhanced diagnostic capabilities to protect against foreign  animal, emerging, and zoonotic diseases that threaten the  nation’s food supply, agricultural economy, and public health.  The facility will provide 574,000 square feet of laboratory  space to support the research missions of the Department of  Homeland Security (DHS) and the Department of Agriculture  (USDA). NBAF is intended to replace and expand upon the  capabilities provided at an existing facility called the Plum Island  Animal Disease Center, which is nearing the end of its useful  life. GAO previously reported on NBAF in March 2016 (GAO-16- 338SP).
Staff needed: 14.3 full time equivalents (FTE)
The NBAF program must commission several laboratory spaces  that meet different biosafety standards in order to meet its sole  key performance parameter (KPP). Program officials reported  that NBAF will not be able to demonstrate that it has met its KPP  until the facility is fully constructed and commissioned in May  2021.
S&T leadership, the NBAF program office, a facility design  team, and a construction manager are coordinating throughout  all phases of the NBAF program in an effort to ensure the  facility will be constructed as designed and within estimated  cost parameters. According to program officials, they selected  a construction manager early in the design process in order  to increase coordination between the design and construction  phases of the program, and to help reconcile cost and schedule  as the program progressed.
In July 2014, DHS’s Acting Under Secretary for Management  (USM) approved the NBAF Acquisition Program Baseline  (APB), which established the program’s cost, schedule, and  performance parameters. According to NBAF officials, the  program remains on track to meet these parameters. The  program awarded the contract for construction of the main  laboratory facility in May 2015, and is scheduled to commission  NBAF in May 2021. NBAF is scheduled to become fully  operational in December 2022, after it receives the certifications  needed to operate as a biocontainment facility.
In May 2013, DHS’s Director, Office of Test and Evaluation  determined he was not responsible for overseeing NBAF  because it was a facility as opposed to a system. According  to program officials, the NBAF program has implemented a  commissioning process for the facility to determine whether it  can meet its KPP and other requirements once construction  is complete. Program officials stated that a third-party  commissioning agent has been retained as a subcontractor  to the prime construction management contractor, and a  commissioning plan has been in place since 2012. The  commissioning agent will monitor and test the facility’s  equipment and building systems while construction is ongoing  to ensure they are properly installed and functioning according  to appropriate biosafety specifications. The commissioning  agent will report its findings directly to program officials and  coordinate with other entities involved in the commissioning  process, including the NBAF program office, the construction  management contractor, and end users, among others. Full  commissioning of the facility is scheduled for May 2021, 6  months after the planned completion of construction.
However, NBAF previously experienced significant cost  growth and schedule slips. Between August 2009, when the  Acting Under Secretary for Science and Technology approved  the initial version of NBAF’s APB, and July 2014, when the  Acting USM approved the current version of NBAF’s APB, the  program’s acquisition cost estimate increased from $725 million  to $1.3 billion, and the facility’s anticipated commissioning  date slipped by almost 6 years. In 2010, DHS and the National  Academy of Sciences both recommended the NBAF program  take a number of actions to mitigate its operational risks as  a biocontainment facility. Subsequently, at the direction of  Congress and DHS leadership, the program office revised  NBAF’s design in response to these recommendations, which  increased costs and caused delays.
S&T reported that the NBAF program office does not have a  staffing gap, and program officials told GAO the program had  recently completed the hiring of additional staff for the program’s  construction oversight team. According to NBAF officials, the  program office’s staffing requirements will change in the coming  years, as the NBAF program progresses through construction  and moves towards the operational stand-up of the facility.  For example, the program office reported it will need to hire  an Operations Director, Research and Development Director,  Business Manager, and Facility Engineer, among others, by  fiscal year 2018 for NBAF operations management.
Program officials reported that funding constraints between  2009 and 2014 exacerbated the cost growth and schedule slips,  and it appears the program continues to face a funding gap of  more than $38 million from fiscal year 2017 to fiscal year 2021.  According to program officials, the anticipated funding gap is  driven by the cost of operational stand-up activities for NBAF,  which are separate from facility construction. Operational stand- up activities are scheduled to ramp up in fiscal year 2018 and  include hiring additional operations management personnel;  preparing standard operating procedures; training laboratory  support personnel and researchers; and demonstrating  proficiency in biocontainment operations, among other things.  Program officials told GAO they are working with S&T to  mitigate the funding gap, but there is a risk these affordability  challenges could cause delays in the operational stand-up  of NBAF and, in turn, the transition from Plum Island Animal  Disease Center.
NBAF officials told GAO the program has received full funding  for facility construction efforts, through federal appropriations  and gift funds from the state of Kansas. DHS entered into a  cost-sharing agreement with Kansas’s state government to  reduce the federal government’s share of NBAF costs. Kansas’s  state government has contributed $307 million to NBAF, which  amounts to nearly 25 percent of the program’s estimated  acquisition cost.
Program Office Comments As noted in the assessment, all out-year funding requests  are for operational planning and operationalization activities.  Current funding gaps will be eliminated if the program is  funded to S&T requested amounts reflected in the next  Future Years Homeland Security Program update.
Electronic Baggage Screening Program (EBSP)
Transportation Security Administration (TSA)
TSA established EBSP in response to the terrorist attacks of  September 11, 2001. EBSP identifies, tests, procures, deploys,  installs, and sustains transportation security equipment across  approximately 440 U.S. airports to ensure 100 percent of  checked baggage is screened for explosives. The program’s  key objectives include: increasing threat detection capability,  improving the efficiency of checked baggage screening,  replacing aging equipment, and obtaining new screening  technologies. The program awarded contracts for 20 types of  baggage screening systems from 2002 to 2015. GAO previously  reported on EBSP in March 2016 and December 2015 (GAO- 16-338SP, GAO-16-117).
Staff needed: 104 full time equivalents (FTE)
TSA officials stated that EBSP has demonstrated that all  deployed systems can meet the minimum threshold for all of the  program’s key performance parameters including automated  threat detection, throughput, and operational availability. TSA  officials told GAO that two scanners underwent testing in fiscal  year 2016, and that two additional scanners are scheduled to  undergo testing in fiscal year 2017.
EBSP acquires explosives trace detectors and medium-speed  and reduced-size explosives detection systems through various  vendors. In 2002 and 2003, TSA deployed baggage screening  equipment to all federally regulated airports. Since then EBSP  has worked to deliver new systems with enhanced screening  capabilities and, according to program officials, development  efforts are primarily focused on software upgrades. As of  December 2016, EBSP had deployed 1,880 explosives  detection systems and 2,638 explosives trace detectors to  screen checked baggage nationwide. EBSP initially acquired  explosive detection systems during specific procurement  windows. In 2014, EBSP revised its acquisition strategy to  competitively procure systems on an ongoing basis using  qualified product lists. TSA officials told GAO this strategy  provides the program more flexibility in acquiring scanning  devices than its previous approach because vendors are able  to submit devices for consideration at any time. Additionally,  officials said this approach allows the program to keep better  pace with technology advancements. EBSP’s initial competitive  procurement of explosive detection systems will end in fiscal  year 2018, at which point TSA plans to initiate a second  competitive procurement.
DHS’s Director, Office of Test and Evaluation (DOT&E) has  assessed nine of EBSP’s systems and determined that six of  them are effective and suitable. As for the remaining three, TSA  is implementing a third party testing strategy to address system  failures during testing. TSA’s interim guidance, effective July  2014, states that TSA will not re-admit systems into testing until  vendors provide sufficient data from a third party tester that the  system meets the failed requirements. According to program  officials, an explosives detection system was the first to undergo  such testing after failing operational testing. After third party  testing of this system, DOT&E issued a memorandum stating  the system should be considered operationally suitable and  DHS approved full rate production in May 2015. In December  2015, GAO found that TSA has yet to finalize key aspects  of its third party testing strategy and recommended it do so  before implementing further third party testing requirements  for vendors to enter testing. In November 2016, TSA officials  said they now plan to implement the third party testing program  by the end of calendar year 2017—a full year later than  initially planned. These officials attributed the delay to the  need to reprioritize third party testing needs and challenges in  coordinating proposed strategy changes, among other things.
section_title: Program Execution
subsections: section_title: In May 2016, Department of Homeland Security (DHS) leadership approved a revised Acquisition Program Baseline (APB) for EBSP. According to TSA officials, the new APB reflects changes made to the program since it revised its acquisition strategy and aligns with the program’s most recent cost estimate that was approved by DHS’s Chief Financial Officer in July 2015. In EBSP’s revised APB, DHS leadership authorized TSA to increase the program’s cost thresholds by 10 percent over its July 2015 cost estimate to account for risk, which increased the program’s acquisition cost to approximately $14 billion and life-cycle cost estimate to approximately $19 billion. However, this reflects a nearly $545 million decrease in acquisition costs and $2.2 billion decrease in life-cycle costs from the program’s 2012 estimates. TSA officials said EBSP’s cost estimates decreased when the program was shortened to end in fiscal year 2027, rather than fiscal year 2030. TSA officials also said that the new cost estimate updated the program’s actual costs, which were lower than anticipated, and revised assumptions for future costs. For example, EBSP reduced the amount of systems it planned to recapitalize annually due to anticipated mechanical failures from 7 percent to 0.5 percent after DHS leadership approved a plan in December 2013 that re-evaluated the projected useful life of explosive detection systems from 10 years to 15 years. Additionally, TSA officials said the program plans to shift some costs for replacing equipment to airports. Officials explained that, in the past, EBSP funded not only the costs for replacing equipment at airports, but also infrastructure-related costs, such as reconfiguring the lanes where the equipment was installed. Going forward, EBSP will fund costs for replacing equipment, but infrastructure costs will generally be covered by the airports.
paragraphs: DOT&E approved EBSP’s Test and Evaluation Master Plan  (TEMP) in 2010. TSA officials previously told GAO that  they were updating the TEMP to reflect EBSP’s acquisition  strategy change, but subsequently decided to wait until the  start of EBSP’s second competitive procurement of explosives  detection systems before formally revising the TEMP, based on  discussion with DOT&E.
In June 2016, DHS reported that the program needed 20.5  full time equivalents (FTE) and did not have a staffing gap.  However, in December 2016, TSA officials told GAO that this  reflected only a subset of EBSP staff. These officials explained  that EBSP is supported by personnel from five different TSA  divisions and had a total staff need of 104 FTEs.
From January 2016 to January 2017, the date the program  planned to achieve initial operational capability for systems that  detect additional materials and provide enhanced homemade  explosives detection capabilities slipped. TSA officials  previously told GAO that they planned to achieve this milestone  in September 2016, but according to the program’s May 2016  APB, TSA has until September 2018 to achieve this milestone.  Previously, EBSP planned to award contracts for these systems  in September 2015 and September 2018, respectively.
Program Office Comments TSA officials provided technical comments on a draft of this  assessment, which GAO incorporated as appropriate.
Passenger Screening Program (PSP)
Transportation Security Administration (TSA)
The Department of Homeland Security (DHS) established PSP  in response to the terrorist attacks of September 11, 2001. PSP  identifies, tests, procures, deploys, and sustains transportation  security equipment across approximately 440 U.S. airports  to help TSA officers identify threats concealed on people and  in their carry-on items. The program’s key objectives include:  increasing threat detection capabilities, improving the efficiency  of passenger screening, and balancing passenger privacy and  security. The program has pursued 11 variants of passenger  screening systems since 2002, including 5 that TSA is currently  acquiring. GAO previously reported on PSP in March 2016 and  December 2015 (GAO-16-338SP, GAO-16-117).
Staffing gap: 15 FTEs equivalents (FTE)
PSP has faced challenges acquiring and deploying new  technologies, including the program’s newest technology: the  Credential Authentication Technology (CAT). However, TSA  officials stated that PSP has demonstrated that all deployed  systems can meet their key performance parameters. The  program is focused on addressing emerging threats with next  generation technologies as well as ensuring that deployed and  new technologies meet cybersecurity requirements.
TSA has acquired and deployed five variants of commercial- off-the-shelf passenger screening systems from multiple  contractors. One system—CAT—remains in development.  Program acquisition efforts are largely focused on upgrading  existing detection technology capabilities. In July 2016, TSA  identified an urgent operational need for automated screening  lanes to address increasing passenger wait times. constraints, significantly decreasing PSP’s acquisition costs  to $3.2 billion and its life-cycle cost estimate to $4.8 billion.  However, by January 2016, emerging threats drove TSA to  increase capability requirements, which in turn increased PSP’s  acquisition and life-cycle cost estimates by about $154 million  and $264 million, respectively.
The program employs two acquisition strategies to acquire  PSP systems. It has designated one the Qualified Product List  (QPL) approach and the other the Low Rate Initial Production  (LRIP) approach. PSP uses the QPL approach for established  and tested technologies, when capability requirements are  rigid and contractors’ systems are mature. For this approach,  any contractors’ systems that demonstrate they meet the  capability requirements are added to the QPL. TSA has used  this approach to acquire the second generation Advanced  Technology X-ray (AT-2) systems, Bottled Liquid Scanners,  and Explosive Trace Detectors. In May 2016, TSA published  its intent to establish a new QPL for the next generation of  Explosive Trace Detectors.
Alternatively, PSP uses the LRIP approach when capability  requirements are flexible and contractors’ systems are evolving.  With the LRIP approach, PSP uses a series of development  contracts to enhance systems’ capabilities over time. PSP is  currently using the LRIP approach to acquire CAT, which TSA  will use to verify the authenticity of passenger identification,  and confirm a passenger’s risk status. CAT is intended to help  TSA expand risk-based screening. PSP is also using the LRIP  strategy to acquire second generation Advanced Imaging  Technology (AIT-2).
DHS’s Director, Office of Test and Evaluation (DOT&E)  approved PSP’s Test and Evaluation Master Plan in 2010,  and each PSP system has its own approved addendum.  DOT&E has assessed seven PSP systems and determined  that three are effective and suitable. However, according to  TSA officials, many vendors’ systems cannot successfully pass  initial qualification testing because their technologies are not  mature, and some systems do not even get to the point in the  testing process where DOT&E would assess them. To address  this issue, TSA is implementing a third party testing strategy. In  December 2015, GAO found that TSA had yet to finalize key  aspects of its third party testing strategy and recommended it do  so before implementing further third party testing requirements  for vendors. Subsequently, TSA gathered and considered  industry feedback on potential third party test strategy changes  and identified potential third party test vendors. In November  2016, TSA officials said they now plan to implement the third  party testing program by the end of calendar year 2017—a full  year later than initially planned. These officials attributed the  delay to the need to reprioritize third party testing needs and  challenges in coordinating proposed strategy changes, among  other things.
section_title: Program Execution
subsections: section_title: Between 2008 and 2015, DHS leadership approved five versions of PSP’s Acquisition Program Baseline (APB). Each time, the program’s cost, schedule, and performance parameters changed. TSA’s plans to submit the sixth version of PSP’s APB to DHS’s Under Secretary for Management (USM) for approval have been delayed by over a year because it has taken longer than expected to update the program’s cost estimate and incorporate new cybersecurity requirements. In December 2016, TSA officials said the revised APB was in the final process of being submitted to DHS’s USM for approval.
paragraphs: DHS reported that PSP faced a staffing gap of 15 full time  equivalents (FTE)—a shortfall of nearly 30 percent. According  to TSA officials, the current staffing level hinders the program’s  response to emerging threats. This could affect the program’s  ability to meet the urgent operational need for automated  screening lanes that TSA identified in July 2016. Further, the  program projects the need for 38 percent more FTEs over the  current approved level, as TSA plans to initiate new checkpoint- related programs in 2018.
The program’s fifth APB—which the DHS USM approved in  February 2015—reflected schedule slips. The full operational  capability (FOC) dates for the AT-2 and AIT-2 both slipped  18 months due to testing issues. The FOC date for CAT also  slipped to June 2018—4 years later than initially planned—after  operational testing revealed performance issues. In January  2016, the PSP program declared an APB schedule breach of  a key CAT milestone—Acquisition Decision Event (ADE) 3,  which was scheduled to be complete by June 2016—because  of delays in incorporating new cybersecurity requirements  before completing operational testing. Program documentation  indicates CAT’s ADE 3 could be delayed by nearly 2 years,  which would directly affect follow-on events including FOC.
Program Office Comments TSA officials provided technical comments on a draft of this  assessment, which GAO incorporated as appropriate.
Technology Infrastructure Modernization (TIM)
Transportation Security Administration (TSA)
TSA conducts various threat assessment screening and  credentialing activities for millions of transportation workers  and travelers. However, these assessments are hindered  by stove-piped systems and duplicative processes which  cannot accommodate growing enrollment demand. In 2008,  TSA initiated the TIM program to address these shortfalls by  developing and operating a centralized system to manage  credential applications and the review process for three  segment populations: maritime, surface, and aviation. The  program delivered the maritime segment in May 2014, but  subsequently struggled to deliver additional capabilities. GAO  previously reported on the TIM program in March 2016 (GAO- 16-338SP) and has an ongoing review of the program’s current  efforts.
Staff needed: 40.0 full time equivalents (FTE)
In September 2016, Department of Homeland Security (DHS)  leadership approved a fourth key performance parameter (KPP)  for the program for enforcing system user access controls. The  program previously demonstrated TIM could meet two of its  KPPs—vetting response time and operational availability—during  initial operational test and evaluation (IOT&E) of the maritime  segment, but DHS’s Director, Office of Test and Evaluation  (DOT&E) concluded the system was extremely unreliable  due to frequent critical failures. DOT&E cannot assess TIM’s  other KPP—information reuse—until additional segments are  deployed.
In April 2016, DHS leadership approved a new technical  approach for the TIM program, which TSA developed in  collaboration with DHS’s Chief Information Officer (CIO) and  subject matter experts. In November 2015, DHS’s Under  Secretary for Management (USM) directed the CIO to work  with TSA to develop a new approach, after the CIO reported  he could not support TSA’s initial strategy for addressing the  TIM program’s execution challenges. Under the new approach,  TSA plans to replace the TIM system’s existing commercial-off- the-shelf applications with open source applications and move  to a new virtual environment. The program also adopted an  agile development methodology that relies on small teams to  rapidly develop, test, and deploy capabilities using an iterative,  rather than a sequential approach. TSA officials anticipate  that the agile approach will allow the program to accelerate  development, better respond to customer needs, and achieve  cost savings by eliminating expensive proprietary licensing  costs, among other things. The TIM program began piloting its  agile approach in May 2016 when developing fixes to address  issues identified during the maritime segment’s IOT&E. TSA  awarded two task orders totaling $17.6 million to the program’s  existing contractor in September 2016 for agile design and  development services, and plans to competitively award a new  contract in 2017. TSA officials expect to have multiple agile  development teams in place by early fiscal year 2017. program’s 6-year schedule slip. The TIM program’s yearly cost  estimates from fiscal year 2017 through 2021 exceed its funding  plan by almost $122 million. However, the program expects to  carry over almost $17 million into fiscal year 2017 and receive  nearly $106 million in fees from vetting programs during this  5-year period.
In September 2016, TSA officials identified several program  and technical risks associated with TIM’s new agile approach  that could affect the program’s schedule, cost, and performance  going forward. These risks include an increase in new  requirements or enrollments in TSA Pre-Check, implementation  of automated testing into its agile approach, and the availability  of knowledgeable contractor development staff. TSA officials  are working to mitigate these risks.
section_title: Program Execution
paragraphs: In September 2016, TSA officials told GAO they worked with  TIM customers to prioritize and address performance issues  identified during IOT&E of the maritime segment, which was  conducted from May to June 2015. DHS’s DOT&E assessed  the program’s IOT&E results in September 2015 and concluded  the system was not operationally effective or suitable, and was  not cyber-secure. According to TSA officials, the program’s  operational test agent completed follow-on operational test and  evaluation on the maritime segment in November 2016, but the  test results will not be available until March 2017.
In October 2016, DHS’s USM removed the TIM program  from breach status, which authorized TSA to resume new  development after a nearly 22-month program pause. TSA  notified DHS’s Acting USM in September 2014 that the TIM  program had breached its baseline due to significant cost,  schedule, and performance issues, and DHS leadership  directed the program to halt new development in January 2015  until TSA identified a strategy for addressing these issues.  TSA officials identified several causes for the breach, including  technical challenges and insufficient contractor performance. In  addition, the TIM program reported that TSA added significant  new requirements to TIM after DHS leadership had approved  the initial acquisition strategy.
In September 2016, DOT&E approved TSA’s proposed test  and evaluation strategy for the TIM program’s new approach.  However, DOT&E noted that DHS guidance for Test and  Evaluation Master Plans (TEMP) did not adequately address  programs using agile development. He reported his office was  leading an effort to develop such guidance and would work with  TSA officials to assist with revising the TIM program’s TEMP by  January 2017.
In June 2016, DHS reported that the TIM program’s staffing  need increased from 24 to 43 full time equivalents (FTE). TSA  officials explained that the additional FTEs were technical staff  funded by the TIM program, but matrixed from another TSA  office. In December 2016, TSA officials said the program had  only been authorized for 40 FTEs, 35.2 of which were filled.
Program Office Comments TSA continues to implement an agile strategy for the  completion of the TIM system development. Early agile  releases of the TIM system have shown the ability to  provide functionality that meets the immediate needs  of the mission operators in an accelerated timeframe  from traditional development approaches. TIM has also  partnered with DHS to form an Agile Integrated Product  Team. The role of this group is to take best practices of  agile development and policy from across DHS and tailor  it for use with the TIM program. TSA officials also provided  technical comments on a draft of this assessment, which  GAO incorporated as appropriate.
United States Coast Guard (USCG)
Command, Control, Communications, Computers, Intelligence,  Surveillance, and Reconnaissance (C4ISR) systems provide  situational awareness, data gathering and processing, and  information exchange tools that are installed in a variety of  USCG ships and aircraft. According to the current C4ISR  program’s baseline, the program encompasses the acquisition  of C4ISR systems tailored for the National Security Cutter  (NSC), Fast Response Cutter, Offshore Patrol Cutter, HC-130J  and HC-144 aircraft, and legacy vessels. However, USCG  officials told GAO the program is now primarily working on the  C4ISR system on the NSC. GAO previously reported on the  USCG’s C4ISR program in March 2016 and June 2014 (GAO- 16-338SP, GAO-14-450).
Staffing gap: 5 FTEs equivalents (FTE)
The USCG is no longer planning to operationally test its C4ISR  systems against its key performance parameters (KPP). Instead,  the C4ISR systems will be tested in conjunction with the USCG’s  planes and vessels to save money and avoid duplication.  However, the effectiveness and suitability of the C4ISR systems  were not specifically evaluated during the HC-144, Fast  Response Cutter, and NSC tests. Since this C4ISR system will  now only be used on the NSC, testing is focused on this asset.  The USCG plans to demonstrate the ability of the C4ISR system  to meet the NSC’s KPPs during follow-on operational testing,  which is scheduled to be completed in November 2017.
The USCG has significantly decreased the C4ISR program’s  scope since the Department of Homeland Security’s (DHS)  Under Secretary for Management (USM) approved the C4ISR  program’s first Acquisition Program Baseline (APB) in February  2011. This APB established the C4ISR program in broad terms,  namely that the program would improve the detection and  engagement of potential targets in the maritime domain through  better coordination and data sharing. However, the initial version  of the system relied on contractor-proprietary software, which  was in danger of becoming obsolete and too costly to maintain.  In November 2013, the USM approved a revised C4ISR APB  after lower than expected funding levels caused a schedule  breach. The new APB reflected a less comprehensive approach  to C4ISR, but established that the C4ISR program would still  deliver certain capabilities to specific cutters and aircraft. through fiscal year 2021. However, the gap may not be as great  as it appears. In April 2015, GAO found that the DHS funding  plan presented to Congress did not identify the operations  and maintenance funding the USCG plans to allocate for  each of its major acquisition programs—including the C4ISR  program—and recommended DHS account for this funding  in its future report (GAO-15-171SP). DHS concurred with the  recommendation, but has yet to take action.
section_title: Program Execution
subsections: section_title: USCG officials said the C4ISR program remains on track to meet the cost and schedule goals in its revised APB. The C4ISR program’s cost estimates significantly decreased from February 2011 and November 2013. It is likely the program’s costs have decreased further because the USCG continued to reduce the program’s scope. For example, the Fast Response Cutter and aviation programs decided to pursue their own solutions, which are being managed by the respective assets’ program offices. In September 2015, USCG officials confirmed the program is focused primarily on improving the C4ISR system for the NSC. Despite pursuing different systems across the USCG’s aviation and surface fleet, USCG officials stated that all of the systems are planned to be able to exchange information using common data formats.
paragraphs: The USCG initially planned to test the C4ISR system against  its KPPs separately from its planes and vessels, including the  NSC, but officials subsequently decided to test the C4ISR  system in conjunction with the planes and vessels to lower  costs and avoid duplication. However, the C4ISR system’s  KPPs were not specifically evaluated during the NSC’s initial  operational test and evaluation in April 2014, in part because  the necessary testing activities were not fully integrated into  the NSC’s test plan. The USCG also deferred testing of a  significant portion of C4ISR functionality on the NSC, including  cybersecurity capabilities and real-time tactical communications  with the Navy, to later dates. In June 2014, GAO recommended  the USCG fully integrate C4ISR assessments into other assets’  test plans or test the C4ISR program independently. The  USCG concurred with GAO’s recommendation and stated that  it planned to test the C4ISR system’s KPPs during follow-on  testing for the NSC. According to USCG officials and the current  follow-on testing plan, the USCG will test S2S2 to evaluate the  extent to which this improved system meets the NSC’s C4ISR- related KPPs, which the USCG will trace to the C4ISR KPPs.  However, the NSC’s KPPs only overlap with one of the C4ISR’s  six KPPs, so this testing will not demonstrate how the C4ISR  system performs against five of its KPPs. The USCG began  NSC’s follow-on operational test and evaluation in fiscal year  2015, but testing is not planned to be complete until the end of  calendar year 2017.
The USCG has developed a new C4ISR system for the NSC  known as segment 2 spiral 2 (S2S2). The S2S2 system  is intended to replace the NSC’s initial system to address  proprietary and obsolescence issues and, according to USCG  officials, to provide improved capabilities. In September 2016,  USCG officials told GAO that the S2S2 system performed  well during qualification testing conducted in August 2015  and that the USCG will install S2S2 on future NSCs. As of  January 2017, the USCG had installed S2S2 on three of the  five already-delivered NSCs and officials anticipated retrofitting  the remaining two NSCs by the end of calendar year 2017. If  completed, the USCG will have transitioned from contractor- proprietary software almost 2 years earlier than the deadline  established in the program’s revised APB, but more than 5  years later than initially planned. USCG officials previously  attributed delays in completing the transition to funding  shortfalls and difficulties scheduling S2S2 installations for when  the NSCs are in port.
In January 2016, the USCG reported that C4ISR had a staffing  gap of 5 full time equivalents, but in September 2016, program  officials did not attribute any negative effects to workforce  shortages.
Program Office Comments The acquisition program’s primary focus is on delivery of  the S2S2 baseline for the NSC class. Also, the acquisition  program continues to provide acquisition, technical, and  cyber security support to Offshore Patrol Cutter, Fast  Response Cutter, and other new asset acquisitions to tailor  C4ISR systems acquisition strategies and requirements to  meet respective platform milestones. The C4ISR acquisition  program plans to operationally test S2S2 in the next NSC  follow-on operational test and evaluation event. USCG  officials also provided technical comments on a draft of this  assessment, which GAO incorporated as appropriate.
Fast Response Cutter (FRC)
United States Coast Guard (USCG)
The USCG uses the FRC to conduct search and rescue,  migrant and drug interdiction, and other law enforcement  missions. The FRC replaces the USCG’s Island Class patrol  boat and carries one cutter boat onboard. It provides greater  fuel capacity, improved communications and surveillance  interoperability with other Department of Homeland Security  (DHS) and Department of Defense assets, and the ability to  conduct full operations in moderate sea conditions. The USCG  plans to acquire 58 FRCs, and as of October 2016, 20 had been  delivered. GAO previously reported on the FRC program in  March 2017, March 2016, and June 2014 (GAO-17-218, GAO- 16-338SP, GAO-14-450).
Staffing gap: 3 FTEs equivalents (FTE)
According to USCG officials, the FRC demonstrated all six of its  key performance parameters (KPP) during follow-on operational  test and evaluation (FOT&E) in July 2016. As of January 2017,  DHS’s Director, Office of Test and Evaluation (DOT&E) was in  the process of validating the FOT&E results and planned to issue  its assessment of the FRC’s performance in February 2017. The  FRC completed initial operational test and evaluation (IOT&E) in  fiscal year 2013 and partially met one of its six KPPs.
In September 2008, USCG officials awarded Bollinger  Shipyards Lockport a contract for 1 FRC with options to build  up to 33 more. GAO subsequently received a bid protest,  which was denied, and upheld the USCG’s contract award in  January 2009. In May 2014, the USCG established that it would  procure only 32 of the 58 FRCs through this contract. The  USCG subsequently purchased the technical specifications and  licenses from Bollinger that are necessary to build the FRC and  used this information to conduct a full and open competition for  the remaining 26 vessels. The USCG has designated this effort  as phase 2 of the program. In May 2016, the USCG awarded  the phase 2 contract, which officials stated has a potential value  of $1.42 billion to Bollinger Shipyards Lockport. According to  USCG officials, the phase 2 design will be similar to the phase  1 cutters with minimal changes to non-critical systems and  updates to address obsolescence issues. The phase 2 contract  is the same contract type as the phase 1—fixed price with  economic price adjustment—and includes the same warranty.  The USCG anticipates delivery of the first phase 2 cutter in  spring 2019.
FOT&E, which focused on resolving issues found during prior  testing. The USCG’s operational test agent (OTA) from the  U.S. Navy conducted IOT&E on the FRC in fiscal year 2013  and assessed three of the program’s six KPPs. At that time,  the FRC only partially met one of the KPPs tested. IOT&E  also revealed several major deficiencies, the most significant  of which involved the FRC’s cutter boat, which exhibited  problems operating in moderate sea conditions, and the FRC’s  main diesel engines, which had multiple equipment failures  during testing. Subsequently, independent testers concluded  the FRC was operationally effective, but not operationally  suitable. USCG officials told GAO they have improved the  FRC’s performance since IOT&E. For example, they replaced  and successfully tested the FRC’s cutter boat, worked with the  engine manufacturer to determine the root cause of equipment  failures, and have begun retrofitting the engines. However,  as recently as May 2016, three diesel engines were replaced  during production on two FRCs, indicating that the problems  with the diesel engines are ongoing.
section_title: Program Execution
subsections: section_title: According to USCG officials, the FRC program is on track to meet its revised schedule goals. Previously, the program’s initial operational capability date slipped from December 2012 to August 2013 because of the bid protest and the need for structural modifications. Additionally, the program’s full operational capability date slipped from September 2022 to March 2027 because, according to USCG officials, the procurement quantities for the FRC changed under the phase 1 contract. In fiscal years 2010 and 2011, the quantities decreased from six FRCs per year to four. Under the phase 2 contract, the USCG can procure four to six FRCs per option period. The USCG has established that the annual procurement quantity will be dictated by funding levels, and funding shortfalls could cause further delays going forward. A $1.5 billion gap appears to remain between the program’s projected funding levels and estimated costs from fiscal year 2017 through fiscal year 2021. However, the projected funding gap may not be this large. In April 2015, GAO found that the DHS funding plan presented to Congress did not identify the operations and maintenance funding the USCG plans to allocate for of its major acquisition programs—including the FRC—and recommended DHS account for this funding in its future report (GAO-15- 171SP). DHS concurred with the recommendation, but has yet to take action. If the USCG needs to order fewer FRCs per year, the program’s costs will likely increase. In June 2014, GAO found that the USCG estimated a decision to order two ships per year would likely increase the program’s costs by $600 million to $800 million beyond its current estimates.
paragraphs: The USCG completed FOT&E in July 2016 and the OTA found  that several deficiencies from IOT&E had been corrected.  For example, the OTA closed a severe deficiency related to  the engines based on modifications to the FRC’s main diesel  engines along with observing that the cutter achieved an  operational availability of 99 percent during FOT&E. Six major  deficiencies from IOT&E remain unresolved and the OTA  identified four new major deficiencies during FOT&E. Ultimately,  the OTA declared the FRC operationally effective and suitable.  As of January 2017, DOT&E was in the process of assessing  the FOT&E results to independently validate the program’s  performance.
In January 2016, the USCG reported that the FRC program  had a staffing gap of 3 full time equivalents. In August 2016,  program officials told GAO they had addressed the FRC’s  staffing gap and did not have any staffing vacancies.
The program continues to experience numerous problems  with the FRC’s main diesel engines. Twenty engines have  been replaced under the program’s warranty, which according  to officials has allowed the USCG to avoid $51.8 million  in potential costs. USCG officials said the program is also  conducting a 15-week dry-dock period for the first 13 cutters  to correct warranty items, which is also being covered by the  warranty. This effort began in January 2016 and is expected to  continue through November 2019.
Program Office Comments The FRC program is fully funded, executable, and on track  for full operational capability by March 2027, within baseline.  FRCs provided over 26,000 operational hours in support  of the USCG’s Western Hemisphere strategy in the last 12  months in which over 6,300 undocumented migrants were  rescued from unseaworthy vessels and 19,000 kg of illegal  narcotics trafficking was disrupted. The program office looks  forward to receiving DOT&E’s independent validation of  the program’s performance. USCG officials also provided  technical comments on a draft of this assessment, which  GAO incorporated as appropriate.
H-65 Conversion/Sustainment Projects (H-65)
United States Coast Guard (USCG)
The H-65 aircraft is a short-range helicopter that the USCG  uses in search and rescue, ports and waterways security,  ice-breaking, marine safety and environmental protection, and  defense readiness operations. The H-65 acquisition program  increased the USCG’s fleet size from 95 to 102 helicopters  and added armament capabilities, upgraded navigation  systems, and replaced all of the helicopters’ engines. The  program is focused on the final phase of upgrades to the radar  sensor system, the automatic flight control system (AFCS),  and avionics. The upgrades allow for greater reliability,  maneuverability, and interoperability between the H-65 and  other government assets. GAO previously reported on the H-65  program in March 2016 (GAO-16-338SP).
Staff needed: 24.5 full time equivalents (FTE)
According to USCG officials, the program has met 16 of  its 18 key performance parameters (KPP), but has not yet  demonstrated its 2 avionics KPPs. The USCG plans to  demonstrate these KPPs through developmental testing and an  operational assessment prior to installing the avionics upgrade  across the fleet, but the assessment has been delayed. USCG  officials stated that during actual operations, the aircraft have  not consistently met 3 of the 16 previously demonstrated KPPs,  which are related to operational availability. Program officials  previously attributed these shortfalls to difficulties maintaining  aging equipment, among other things, which the avionics  upgrades should address.
The USCG Aviation Logistics Center (ALC) is responsible  for procuring and integrating all the systems needed to  upgrade the H-65 aircraft. USCG leadership assigned the  ALC this responsibility because it was already responsible for  overhauling the H-65 aircraft every 4 years as part of normal  maintenance. The ALC has completed upgrades to the engines,  armament, and navigation systems on all flyable H-65 aircraft.  The ALC is in the process of testing the systems for the H-65  aircraft’s avionics and AFCS upgrades. the USCG plans to allocate for each of its major acquisition  programs—including the H-65—and recommended DHS  account for this funding in its future report (GAO-15-171SP).  DHS concurred with the recommendation, but has yet to take  action.
In June 2015, the Department of Homeland Security’s (DHS)  Under Secretary for Management (USM) authorized USCG  to award contracts for long-lead production materials for  the avionics and AFCS upgrades. Officials estimated these  materials will cost $20 million. In September 2016, USCG  officials told GAO they had awarded all but 2 of about 40 of  these contracts. According to officials, ordering long-lead  material was necessary to ensure that the ALC has all the  required parts to begin installing the upgrades during normal  aircraft maintenance once the program receives approval for  initial production.
According to USCG officials, the program has completed  several years of developmental testing on the avionics and  AFCS upgrades. In 2015, the program revised its Test and  Evaluation Master Plan (TEMP) at the request of the USM to  ensure the USCG has sufficient data to support approval for the  initial production of these upgrades. Specifically, the program  added an operational assessment conducted by the U.S. Navy  to collect more data about the upgrades prior to the production  decision. DHS’s Director, Office of Test and Evaluation  approved the TEMP in February 2016, but recommended  the program make further updates to reflect anticipated test  objective changes prior to program-wide IOT&E. IOT&E is  intended to test all the H-65 upgrades installed throughout the  life of the program to support approval for full-rate production.  Officials told GAO they would update the TEMP by August  2018, prior to when IOT&E was scheduled to begin in fiscal  year 2019. However, these activities will likely be rescheduled  because of the program’s delays.
The USCG experienced an over 12-month delay in developing  a portion of the avionics and AFCS upgrades that resulted in  the H-65 program declaring a schedule breach in November  2016. USCG officials told GAO in September 2016 that  several milestones for the avionics and AFCS upgrades  had been delayed. Specifically, the production readiness  review, completion of developmental testing, and operational  assessment—all of which were planned for summer 2016—had  been pushed into 2017. Program officials primarily attributed  these delays to an underestimation of the technical effort  necessary to meet requirements. As these activities support  approval for the avionics and AFCS initial production, this  decision was also delayed from the USCG’s target date of  December 2016. USCG officials anticipated receiving approval  for initial production by the program’s revised Acquisition  Program Baseline (APB) threshold date of March 2017, but  notified DHS leadership in November 2016 that it would not  meet this date. According to USCG officials, they now plan to  receive approval for initial production by September 2018— nearly 5 years later than the initial APB date of December  2013. The USCG plans to update the H-65’s APB by May 2017  to account for these delays, which will also reflect schedule  changes for subsequent milestones including initial operational  test and evaluation (IOT&E), the full-rate production decision,  and full operational capability.
In January 2016, the USCG reported the program had a staffing  gap of 4 full time equivalents. In September 2016, USCG  officials told GAO the program had closed this gap and was  sufficiently staffed. USCG officials also stated that they have  been able to address long-standing ALC contracting personnel  shortages by shifting some contracting duties from ALC to the  USCG contracting office.
As of October 2016, USCG officials reported two aircraft have  been lost during operational missions. As a result, the program’s  LCCE will likely decrease because the USCG no longer  needs to fund operations and maintenance costs for these  aircraft. However, if the USCG chooses to replace the aircraft,  officials said there will be no adverse effect on the program’s  schedule or acquisition costs because all of the materials for the  upgrades were previously purchased.
USCG officials told GAO they are also updating the program’s  life-cycle cost estimate (LCCE). The USCG anticipates that the  program’s schedule delays will result in minor cost increases  because of extended labor contracts and inflation, but that  these costs will remain within the program’s currently approved  cost thresholds. The program’s LCCE previously increased by  approximately $6 billion from 2011 to 2014 due to the USCG’s  decision to extend the aircraft’s operational life by 9 years, from  2030 to 2039.
Program Office Comments USCG officials provided technical comments on a draft of  this assessment, which GAO incorporated as appropriate.
Long Range Surveillance Aircraft (HC-130H/J)
United States Coast Guard (USCG)
The USCG uses HC-130H and HC-130J aircraft to conduct  search and rescue missions, transport cargo and personnel,  support law enforcement, and execute other operations. In  2009, the Department of Homeland Security’s (DHS) Under  Secretary for Management (USM) approved an Acquisition  Program Baseline (APB) for the HC-130H upgrade program,  and a separate APB for the acquisition of the more modern  and capable HC-130J aircraft. In 2012, the USM approved a  third APB that combined and re-baselined the two programs. In  October 2014, USCG officials told GAO they no longer planned  to upgrade any additional HC-130H aircraft, and that they  were pursuing an all-HC-130J fleet, in response to the addition  of C-27J aircraft into the USCG’s fleet of Medium Range  Surveillance Aircraft. GAO reported on the USCG’s HC-130H/J  program in March 2016 and March 2015 (GAO-16-338SP, GAO- 15-325).
Staffing gap: 3 FTEs equivalents (FTE)
The HC-130J will not be able to meet two of its seven key  performance parameters (KPP) until the USCG installs a  new mission system processor on the aircraft, an effort that  is underway. These two KPPs are related to the detection of  targets and the aircraft’s ability to communicate with other  assets. USCG officials said they installed a prototype of the new  HC-130J mission system processor in June 2016 and began  developmental testing. The USCG plans to conduct further  testing on the HC-130J’s mission system processor in fiscal year  2017. USCG officials previously told GAO that the HC-130H  aircraft met all six of its KPPs based on operational performance  during USCG missions.
The USCG plans to acquire 22 HC-130J aircraft, which will  eventually replace the existing HC-130H aircraft. After deciding  to pursue an all-HC-130J fleet in October 2014, the USCG  began to decrease the number of HC-130H aircraft in its fleet.  As of January 2017, the USCG had transferred or was in the  process of transferring 9 of its 23 existing HC-130H aircraft to  other organizations. For example, the USCG is transferring 7  of these aircraft to the U.S. Forest Service. USCG officials told  GAO that the USCG will continue to operate 14 of its HC-130H  aircraft until the end of their service lives or until they can be  replaced with new HC-130J aircraft. Officials anticipate retiring  all HC-130H aircraft by fiscal year 2022. As of January 2017,  USCG officials said they had received 10 HC-130J aircraft and  awarded contracts for 3 more. meet the full operational capability date of March 2027. If the  remaining aircraft are not delivered at this rate, the program’s  schedule could slip. USCG officials stated the delivery rate is  dependent on the amount of funding the program receives.  It appears that the program is facing a potential $2.2 billion  funding gap from fiscal year 2017 through fiscal year 2021.  However, the gap may not be this large, because the USCG  has historically received HC-130Js without including them in  its budget requests. Additionally, in April 2015, GAO found  that the DHS funding plan presented to Congress did not  identify the operations and maintenance funding the USCG  plans to allocate for each of its major acquisition programs— including the Long Range Surveillance Aircraft program—and  recommended DHS account for this funding in its future report  (GAO-15-171SP). DHS concurred with the recommendation,  but has yet to take action.
The USCG is also replacing the mission system processor on  all of its fixed-wing aircraft—including the HC-130J—with a  system used by the U.S. Navy and DHS’s Customs and Border  Protection. The new mission system processor is intended to  enhance operator interface and sensor management, as well  as replace obsolete equipment. Pending test results, the USCG  plans to install the new mission system processor on the 13 HC- 130J aircraft it plans to receive by the end of fiscal year 2020.  In September 2015, the USCG awarded a contract that will  cover retrofitting efforts for 7 of these aircraft for a total of $17.2  million.
In October 2016, USCG officials told GAO the program had  begun updating its life-cycle cost estimate to support a revised  APB that accounts for the cancellation of HC-130H upgrades,  the transition to an all-HC-130J fleet, and replacement of the  HC-130J’s mission system processor. However, officials said  they would not update the APB until the USCG completed  its multi-phased analysis of mission needs. Consistent with  congressional direction, the USCG conducted a multi-phased  analysis of its mission needs, including its flight-hour goals and  mix of fixed-wing assets, which the USCG is delivering through  both the Long Range Surveillance Aircraft program and the  Medium Range Surveillance Aircraft program, which GAO is  also assessing in this report. The USCG submitted the results of  this analysis to Congress in November 2016, which confirmed  the total quantity of 22 HC-130J aircraft the USCG plans to  acquire and an annual flight-hour goal of 800 hours per aircraft.
According to program officials, the USCG installed the HC-130J  mission system processor prototype, and began developmental  testing in June 2016. Once developmental testing is complete,  USCG officials said they plan to demonstrate the HC-130J’s  mission system functionality against its requirements through  performance testing conducted by the U.S. Navy in fiscal year  2017. USCG officials noted that this testing will be conducted in  various operational environments. However, formal operational  testing will not be conducted, which increases the risk that the  new mission system processor will not perform as intended  or be reliable once fielded. The USCG has not conducted  operational testing on either aircraft. In 2009, DHS’s Director,  Office of Test and Evaluation (DOT&E) and the USCG  determined the HC-130J did not need to operationally test the  airframe because the U.S. Air Force conducted operational  testing on the base C-130J airframe in 2005. Additionally,  DOT&E approved a Test and Evaluation Master Plan for the  HC-130H upgrades in 2010, but the USCG did not implement  the plan because it canceled the upgrade.
Despite reporting a staffing gap of 3 full time equivalents,  program officials did not attribute any negative effects to  workforce shortages.
section_title: Program Execution
subsections: section_title: In August 2016, USCG officials said they anticipated issuing a contract for additional HC-130J aircraft in fiscal year 2017, but did not identify how many. Officials previously told GAO that the USCG would need to acquire 1 to 2 aircraft per year in order to
paragraphs: Program Office Comments USCG officials provided technical comments on a draft of  this assessment, which GAO incorporated as appropriate.
Medium Range Surveillance Aircraft (HC-144A & C-27J)
United States Coast Guard (USCG)
In October 2014, Department of Homeland Security (DHS)  leadership directed the USCG to restructure its HC-144A  acquisition program to accommodate 14 C-27J aircraft from  the U.S. Air Force, and designated this combined acquisition  the Medium Range Surveillance (MRS) Aircraft program. All  32 aircraft—14 C-27J aircraft plus 18 previously purchased  HC-144A aircraft—are twin-engine propeller-driven platforms  that the USCG plans to use to conduct all types of Coast Guard  missions, including search and rescue and disaster response.  In August 2016, DHS leadership approved MRS’s Acquisition  Program Baseline (APB), which established the program’s  cost, schedule, and performance parameters. GAO previously  reported on the MRS program in March 2016 and the C-27J  aircraft in March 2015 (GAO-16-338SP, GAO-15-325).
Staffing gap: 6 FTEs equivalents (FTE)
The seven HC-144A key performance parameters (KPP) apply  to the C-27J aircraft. However, neither aircraft will be able to  meet two KPPs until the USCG installs a new mission system  processor, an effort that is underway, according to officials. These  two KPPs are related to the detection of targets and the aircraft’s  ability to communicate with other assets. The HC-144A previously  fully met three of its seven KPPs during testing conducted in July  2012. The C-27J aircraft will undergo testing once the USCG  installs an entire mission system, consisting of the processor  and sensor package, on the aircraft. However, the USCG has  deferred its detection KPP due to technology limitations.
The USCG initially planned to procure a total of 36 HC-144A  aircraft, but reduced that number to the 18 they had already  procured after Congress directed the U.S. Air Force to transfer  14 C-27J aircraft to the USCG in fiscal year 2014. As of October  2016, the USCG had accepted 9 C-27J aircraft. The USCG is  also replacing the mission system processor on all of its fixed- wing aircraft—including both the HC-144A and C-27J—with a  system used by the U.S. Navy and DHS’s Customs and Border  Protection. In August 2016, USCG officials told GAO they  expect to complete installation of the mission system processor  prototype on the HC-144A by December 2016, and plan to  outfit all 18 HC-144A aircraft by 2021. These officials said it will  take longer to complete installation of this system on the C-27J  because the aircraft first needs a sensor package—primarily a  radar and electro-optical camera—to meet its requirements.
In July 2012, U.S. Navy officials responsible for testing the  HC-144A aircraft reported that it was operationally effective and  suitable, but fully met only three of its seven KPPs. Program  officials previously stated that they are addressing the KPP  deficiencies by changing operational tactics until the USCG  installs a new mission system processor and other items. USCG  officials plan to test the upgraded aircraft through performance  testing conducted by the U.S. Navy in fiscal year 2017. USCG  officials noted that this testing will be conducted in various  operational environments. However, formal operational testing  will not be conducted, which may increase the risk that the new  mission system processor will not perform as intended or be  reliable once fielded.
section_title: Program Governance
section_title: Program Execution
paragraphs: In October 2014, DHS leadership directed the USCG to test the  C-27J mission system in an operational setting. In July 2016,  DHS’s Director, Office of Test and Evaluation approved the  program’s Test and Evaluation Master Plan for the C-27J, which  shows operational testing beginning in April 2021. However,  it is unclear when the C-27J will be able to meet its detection  KPP because the technology required does not yet exist for  this aircraft. In April 2016, the USCG received approval to defer  these capabilities until the technology becomes commercially  available.
Incorporating the C-27J into the USCG’s fleet revised the  MRS program’s full operational capability date to March 2025.  However, this reflects a 6-month acceleration from the USCG’s  revised APB date for the HC-144A. In 2012, the HC-144A’s  full operational capability date slipped from September 2020  to September 2025 when the USCG reduced the number of  aircraft purchased per year in response to funding constraints.
The USCG initially estimated that it may cost $600 million to  convert the C-27J aircraft to meet USCG mission needs, but  according to the MRS APB, it may cost $1 billion, bringing the  program’s total acquisition cost to $2.5 billion. These costs  include purchasing a sensor package, redesigning the aircraft  and installing the package, and customizing and testing the new  mission system processor. The MRS program’s life-cycle cost  estimate (LCCE) exceeds $15 billion, but this is an almost $13.6  billion decrease compared to the USCG’s revised estimates for  an all-HC-144A fleet. From 2009 to 2012, the HC-144A LCCE  increased from $12.3 billion to $28.7 billion when the USCG  accounted for 5 years of additional costs, among other things.  The MRS program’s LCCE decreased because of the reduced  number of aircraft acquired, a reduction in planned flight hours,  and the 15-year shorter service life of the C-27J compared to  the HC-144A. Nevertheless, the USCG will ultimately procure  fewer aircraft than initially planned at a higher cost.
The USCG still faces challenges in transitioning the C-27J into  the USCG fleet. In March 2015, GAO found that the successful  and cost-effective fielding of the C-27J aircraft is contingent on  the USCG’s ability to address three risk areas: (1) purchasing  spare parts, (2) accessing technical data, and (3) understanding  the condition of the aircraft. According to USCG officials,  purchasing spare parts remains the greatest risk. However, in  September 2016, the USCG awarded an $11 million contract for  spare parts. In December 2016, USCG officials also said they  had not yet received access to the aircraft’s technical data to  start the redesign effort.
In January 2016, the USCG reported that the program’s staffing  need increased from 15 full time equivalents to 81, much of  which was needed to establish a C-27J asset program office at  the USCG’s Aviation Logistics Center.
The MRS program is projected to face a $1.3 billion funding gap  from fiscal year 2017 through fiscal year 2021. However, the  funding gap may not be this large. In April 2015, GAO found that  the DHS funding plan presented to Congress did not identify  the operations and maintenance funding the USCG plans to  allocate for each of its major acquisition programs—including  the MRS program—and recommended DHS account for this  funding in its future report (GAO-15-171SP). DHS concurred  with the recommendation, but has yet to take action.
Program Office Comments USCG officials provided technical comments on a draft of  this assessment, which GAO incorporated as appropriate.
National Security Cutter (NSC)
United States Coast Guard (USCG)
The USCG uses the NSC to conduct search and rescue,  migrant and drug interdiction, environmental protection, and  other missions. The NSC replaces the USCG’s High Endurance  Cutters and is intended to provide improved capabilities over  this legacy asset. The NSC carries helicopters and cutter boats,  provides an extended on-scene presence at forward deployed  locations, and operates worldwide. As of January 2017, the  USCG had received six of eight originally planned NSCs, and  two were under construction. The Consolidated Appropriations  Act of 2016 stated that not less than $640 million shall be  immediately available and allotted to contract for the production  of a ninth NSC. Each NSC is designed to have a 30-year  service life. GAO previously reported on the NSC in March  2017, March 2016, and January 2016 (GAO-17-218, GAO-16- 338SP, GAO-16-148).
Staffing gap: 9 FTEs equivalents (FTE)
The USCG has been operating the NSC since 2010, but it has  not yet demonstrated that the NSC can fully meet 7 of its 19 key  performance parameters (KPP). The NSC’s unmet KPPs include  those related to unmanned aircraft, cutter-boat deployment, and  interoperability requirements. The USCG plans to demonstrate  all unmet KPPs during follow-on operational test and evaluation  (FOT&E) in fiscal years 2017 and 2018.
The USCG awarded a contract to produce the first three NSCs  to Integrated Coast Guard Systems—a joint venture between  Northrop Grumman and Lockheed Martin—as part of the now- defunct acquisition effort designated Deepwater. In 2006, the  USCG revised its Deepwater acquisition strategy, citing cost  increases, and took over the role of lead systems integrator,  acknowledging that it had relied too heavily on contractors. In  2010, the USCG awarded the production contract for the fourth  NSC to Northrop Grumman. In 2011, Northrop Grumman spun  off its shipbuilding sector as an independent company named  Huntington Ingalls Industries (HII). HII delivered the fourth,  fifth, and sixth NSCs, and is producing the seventh and eighth  NSCs. In December 2016, the USCG awarded HII a contract to  produce the ninth NSC, using the funding made available and  allotted by Congress for this purpose in December 2015. The  ninth NSC will be built to the same configurations as the eighth  NSC.
In June 2016, the Department of Homeland Security’s (DHS)  Director, Office of Test and Evaluation (DOT&E) approved  the NSC program’s revised Test and Evaluation Master Plan  (TEMP) in preparation for FOT&E. According to USCG officials,  FOT&E will focus on testing all unmet KPPs and resolving  deficiencies found during prior testing. The NSC completed its  initial operational testing in 2014, and DOT&E subsequently  found the NSC operationally effective and suitable. However,  the NSC did not fully demonstrate 7 of its 19 KPPs during  this testing, including those related to unmanned aircraft and  cutter-boat deployment in rough seas. USCG officials indicated  that challenges remain in determining a path forward to resolve  these KPPs because the USCG and its operational test agent  within the U.S. Navy have different interpretations of the cutter  boat requirements. In January 2016, GAO recommended the  NSC program office clarify the KPPs for the cutter boats, with  which the USCG concurred. As of January 2017, the USCG was  working on a resolution.
According to USCG officials, the NSC program is on track  to meet its revised schedule and cost goals for the first eight  NSCs. From 2008 to 2014, the program’s full operational  capability (FOC) date slipped 4 years. USCG officials attributed  this schedule slip to, among other things, funding shortfalls.  Additionally, the program’s acquisition cost estimate increased  nearly $1 billion due to lingering effects of Hurricane Katrina,  which in 2005 struck the region where the NSCs are built.  However, the program’s life-cycle cost estimate (LCCE)  decreased by $2.4 billion, which USCG officials attributed to  increasingly accurate cost estimates for personnel, materials,  and maintenance.
As of August 2016, the USCG was developing the test  scenarios that it will use to conduct FOT&E in fiscal years 2017  and 2018. Officials stated that, in January 2017, the NSC will  be the first USCG asset to undergo cyber security testing. The  USCG expects to complete installation of an unmanned aircraft  on the third NSC in December 2016, but it remains unclear  when the USCG will demonstrate the unmanned aircraft KPP.  In January 2016, GAO also recommended DHS specify when  the USCG must complete the NSC’s FOT&E and any further  actions the NSC program should take following FOT&E. The  USCG concurred and in April 2016, DHS issued a memorandum  outlining requirements for the program’s FOT&E including that it  be completed by March 2019. This memorandum also directed  the USCG to complete a study no later than December 2017 to  determine the root cause of the NSC’s propulsion system issues  such as high engine temperatures, cracked cylinder heads, and  overheating generator bearings that are impacting missions— issues GAO also reported on in January 2016.
The program’s costs include several design changes the USCG has had to implement on equipment with known issues aboard the NSC fleet. As of September 2016, 12 equipment systems required design changes costing over $1 million each, for an estimated total cost of $260 million. The estimated costs associated with these changes—such as structural enhancement work on the first two NSCs and the replacement of the gantry crane which aids in the deployment of the cutter boats—have increased by roughly $60 million since GAO reported on this issue in January 2016. Program officials attributed the increase to the revised cost of structural enhancements on NSCs 1 and 2 based on actual contract values and the addition of the ninth NSC. USCG officials told GAO they are updating the program’s Acquisition Program Baseline and LCCE to account for the ninth NSC, but these updates are not expected until September 2017. The USCG anticipates delivery of the ninth NSC in September 2020, which coincides with the program’s revised FOC date. It is unclear how the ninth NSC will affect the program’s costs.
In August 2016, USCG officials told GAO they have been able  to mitigate any effects of the program’s staffing shortfall with  existing staff and were in the hiring process for the program’s  remaining critical vacancy.
Despite receiving funding for the ninth NSC in fiscal year 2016,  the program is projected to face a $1.6 billion funding gap from  fiscal year 2017 to fiscal year 2021. However, the funding gap  may not be as large as it appears. In April 2015, GAO found that  the DHS funding plan presented to Congress did not identify  the operations and maintenance funding the USCG plans to  allocate for each of its major acquisition programs—including  the NSC—and recommended DHS account for this funding  in its future report (GAO-15-171SP). DHS concurred with the  recommendation, but has yet to take action.
Program Office Comments Cost estimates herein are threshold values from the NSC  Acquisition Program Baseline and do not reflect current  lower estimates based on award amounts for NSCs 7 and  8. The NSC program completed initial operational test and  evaluation (IOT&E) in 2014 and continues to work with  DHS to complete remaining testing and resolve pending  discrepancies. Despite not fully completing all aspects  of IOT&E, USCG operations, led by NSCs, seized more  cocaine in 2016 than any year prior—more than 416,600  pounds worth over $5.6 billion. USCG officials also provided  technical comments on a draft of this assessment, which  GAO incorporated as appropriate.
Offshore Patrol Cutter (OPC)
United States Coast Guard (USCG)
The USCG plans to use the OPC to conduct patrols for  homeland security, law enforcement, and search-and-rescue  operations. It will be designed for long-distance transit,  extended on-scene presence, and operations with deployable  aircraft and small boats. The OPC is intended to replace the  USCG’s aging Medium Endurance Cutters and to bridge the  operational capabilities provided by the USCG’s Fast Response  Cutters and National Security Cutters. The USCG plans to  procure 25 OPCs, and it expects to receive the first OPC in  2021. GAO previously reported on the OPC program in March  2016 and June 2014 (GAO-16-338SP, GAO-14-450).
Staffing gap: 7 FTEs equivalents (FTE)
Department of Homeland Security (DHS) leadership has  approved six key performance parameters (KPP) for the OPC,  establishing goals for the ship’s operating range and duration,  crew size, interoperability and maneuverability, and ability to  support operations in moderate to rough seas. The first OPC has  not yet been constructed, so the USCG has not yet demonstrated  whether it can meet these KPPs. The USCG plans to use  engineering reviews, and developmental and operational tests  throughout the acquisition to measure the OPC’s performance.
The USCG used a two-phased down-select strategy to  select a contractor to deliver the OPC. For phase 1, the  USCG conducted a full and open competition to select three  contractors to perform preliminary and contract design work,  and subsequently, in February 2014, the USCG awarded fixed- price contracts to Eastern Shipbuilding, Bollinger Shipyard,  and Bath Iron Works. For phase 2, the USCG selected one  of the three phase 1 contractors to develop a detailed design  of the OPC, and construct no more than the first 11 ships. In  September 2016, the USCG awarded the phase 2 contract to  Eastern Shipbuilding, worth approximately $110 million for the  detailed design and with separate options for each ship. The  options for ships 10 and 11 were unpriced and included in the  solicitation as an incentive to convert the contract type from  fixed price incentive to firm fixed price. These options will be  included in a re-pricing proposal submitted by the contractor  for ships 6-9 after delivery of the first ship. According to USCG  officials, the USCG will decide whether to exercise the option for  ships 10 and 11 based on the contractor’s re-pricing proposal  for ships 6-9. The USCG plans to re-compete the contract for  the remaining 14 or 16 ships.
USCG officials told GAO they are using a warranty similar to  that for the Fast Response Cutter (FRC). In March 2016, GAO  found that the FRC’s warranty improved cost and quality by  requiring the shipbuilder to pay to repair defects. The OPC’s  phase 2 contract includes a 2-year warranty for the lead ship  and a 1-year warranty for all other ships that includes provisions  that govern defects.
GAO previously found that the OPC’s existing cost estimate  raised questions about the program’s affordability. For example,  in September 2012, GAO found that the requirements and  mission for the National Security Cutter (NSC) and the OPC  programs have similarities, but the estimated acquisition unit  cost for the OPC was less than half the actual acquisition unit  cost for the NSC. At that time, USCG officials recognized that  the cost estimate for the OPC was still uncertain since the cutter  had yet to be designed. USCG officials also noted that any  delays, design issues, or contract oversight problems—all of  which were experienced during the procurement of the NSC— could increase the eventual cost of the OPC. In 2012, DHS’s  Chief Financial Officer also raised concerns that the OPC’s  costs could grow as other shipbuilding programs’ costs have  grown in the past, and could ultimately affect the affordability of  other USCG acquisition programs. In June 2014, GAO reported  that the OPC will absorb about two-thirds of the USCG’s  acquisition funding from 2018 to 2032, and recommended  that the USCG develop a 20-year fleet modernization plan  that identifies all acquisitions needed to maintain the current  service level, along with trade-offs if the funding needed to  execute the plan is not consistent with annual budgets. The  USCG concurred with this recommendation but did not identify  an estimated date for completing the plan. In September 2016,  USCG officials told GAO that significant investments in the NSC  and FRC will be phased out by fiscal year 2021 to support the  affordability of the OPC as it ramps up production.
section_title: Program Execution
subsections: section_title: According to USCG officials, the program is on track to meet the cost and schedule goals in its revised Acquisition Program Baseline, which was approved in September 2014. USCG re-baselined the OPC program to account for schedule slips resulting from delays in awarding the three preliminary and contract design contracts, and a subsequent bid protest that was filed with GAO. GAO denied the protest in June 2014. As a result, from 2012 to 2014, the program’s critical design review and initial operational test and evaluation (IOT&E) dates both slipped 12 months. Additionally, the program’s initial and full operational capability dates both slipped 15 months. Going forward, USCG officials have stated that additional OPC delays will decrease the USCG’s operational capacity because the aging Medium Endurance Cutters will require increased downtime for maintenance and other issues, reducing their availability.
paragraphs: DHS’s Director, Office of Test and Evaluation approved the OPC Test and Evaluation Master Plan (TEMP) in October 2011, which the USCG updated to reflect schedule changes resulting from the bid protest. In March 2016, the USCG issued a memo further refining the program’s test schedule and detailing plans for cybersecurity testing, among other things. The USCG plans to conduct developmental testing from fiscal years 2017 to 2022 before conducting IOT&E on the first OPC in fiscal year 2023.
In January 2016, the USCG reported that the program office  increased its required staffing level from 20 to 29 full time  equivalents (FTE), but still had a staffing gap of 7 FTEs. In  August 2016, program officials told GAO that the program had  closed its staffing gap to 3 FTEs. The 5 critical vacancies are  for additional USCG personnel who will oversee construction  and provide management of contract execution at Eastern  Shipbuilding’s shipyard once phase 2 activities ramp up.
The OPC’s acquisition and life-cycle cost estimates have not  changed since 2012. However, the acquisition cost estimate  had previously increased—GAO found in June 2014 that  this estimate had increased by $4 billion from 2007 to 2012.  USCG officials said the increase was largely due to invalid  assumptions in the earlier cost estimate, along with schedule  delays and inflation. The program is currently projected to  have a nearly $1.2 billion funding gap from fiscal years 2017  to 2021. However, it is unclear whether this assessment of the  gap is accurate because the USCG has not updated OPC’s  cost estimate to reflect the schedule delays experienced after  the 2012 cost estimate was approved. In addition, USCG  officials said that $231 million of the OPC’s costs over this  5-year period are funded by sources from outside the program.  DHS leadership directed the USCG to update OPC’s life-cycle  cost estimate by March 2017 following award of the phase 2  contract.
Program Office Comments USCG officials provided technical comments on a draft of  this assessment, which GAO incorporated as appropriate.
United States Citizenship and Immigration Services (USCIS)
USCIS spans more than 200 offices worldwide, and processes  tens of thousands of immigration and citizenship applications  each day. The Transformation program was established in  2006 to transition USCIS from a fragmented, paper-based  filing environment to a consolidated, paperless environment.  However, it struggled to deliver capability for several years, and  in 2013, the Department of Homeland Security (DHS) Under  Secretary for Management (USM) authorized USCIS to revise  its acquisition strategy. According to USCIS, the program is now  pursuing a simpler solution based on a new system architecture.  However, USCIS cannot use any of the architecture delivered  under the old strategy, despite having invested more than  $475 million in its development. GAO previously reported on  the Transformation program in March and July 2016 (GAO-16- 338SP, GAO-16-467).
Staffing gap: 17 FTEs Actual staff: 115.34 FTEs equivalents (FTE)
In April 2015, DHS leadership approved a revised set of 8 key performance parameters (KPP) after the program struggled to meet its requirements. USCIS will not be able to fully demonstrate these KPPs until it achieves full operational capability (FOC). In the interim, the program has conducted operational assessments of some deployed functionality. In November 2015, DHS’s Director, Office of Test and Evaluation (DOT&E) concluded that the system met 6 of the 7 tested KPPs during an assessment of the product line automating permanent resident card replacement applications. USCIS completed another assessment in March 2016 but, as of January 2017, DOT&E had not assessed these results.
In 2008, DHS awarded IBM a task order to deliver the original  solution through five software releases. The first release  was launched in May 2012, approximately 5 months behind  schedule. DHS attributed this delay to its decision to give a  single contractor too much responsibility, weak contractor  performance, pursuing an unnecessarily complex system, and  adopting a development methodology that did not allow DHS  to see problems early in the process. To address the delay,  the Office of Management and Budget, DHS, and USCIS  determined the program should implement a new acquisition  strategy, which allowed for an agile software development  methodology and increased competition for development  work. Under an agile software development methodology,  end users, subject matter experts, and testers collaborate  with developers, increasing visibility into interim progress. By  September 2014, USCIS had awarded four agile development  contracts, which expired in September 2016. USCIS told GAO  they awarded bridge contracts while the development contracts  are re-competed. In April 2015, the Acting Deputy USM formally  approved a program re-baseline. Currently, the program plans  to deliver capability through 14 releases that correspond to new  product lines. Each product line contributes to processing one  of four lines of business: Citizenship, Immigrant, Non-immigrant,  and Humanitarian.
The program’s yearly cost estimates appear to match its funding  plan from fiscal years 2017 through 2021, but it is actually  projected to have a sizable surplus. USCIS uses revenue  from premium processing fees to fund the Transformation  program. USCIS expected to carry over $468 million in premium  processing revenue into fiscal year 2017, and USCIS expects  it will still have $327 million in unobligated funds at the end of  fiscal year 2021.
section_title: Program Execution
subsections: section_title: From January 2016 to January 2017, the program deployed four product lines for the new system architecture. This brings the total to six product lines deployed since 2015, which collectively deliver functionality that supported approximately 24 percent of the total workload processed by USCIS in fiscal year 2016.
paragraphs: In March 2016, the program completed its third operational  assessment since adopting its new system architecture. The  assessment evaluated a software release deployed in 2015  that was intended to help USCIS customers submit immigrant  visa payments. In May 2016, the program’s operational test  agent (OTA)—a private industry firm—determined that the  product line had an overall low risk and should continue to  be developed and deployed in accordance with program  plans. However, the operational assessment only tested a  minor subset of the system’s FOC capability. As of January  2017, DHS’s DOT&E had not independently validated these  results. The OTA subsequently conducted a fourth operational  assessment intended to inform DHS leadership’s acceptance  of the Citizenship line of business. However, according to  program officials, the OTA extended the observation period for  this assessment once the program breached the Citizenship  line of business completion deadline. These officials said the  assessment will be completed in 2017, and DOT&E plans to  assess the results prior to DHS’s acceptance of the Citizenship  line of business. Going forward, the program plans to conduct  similar operational assessments several more times through  March 2019, when the program plans to achieve FOC.
USCIS completed data migration from the old system  architecture in March 2016, but subsequently encountered  challenges processing all applications as new product lines  were transitioned to the new system architecture. In August  2016, the program reverted back to the legacy system  for processing one of the Citizenship forms. As a result of  the switchover and other technical issues with the case  management system, the program did not complete deployment  of all the product lines associated with the Citizenship line  of business by its September 2016 deadline, resulting in a  schedule breach.
In January 2016, USCIS reported that the program added  approximately 30 full time equivalents (FTE), but still had a  staffing gap of 17 FTEs. In August 2016, program officials said  they had filled some vacant positions, including a division chief,  but had several new vacancies for support staff and one project  lead. However, program officials did not attribute any negative  effects as a result of staffing shortfalls.
In November 2016, USCIS submitted a breach remediation  plan to DHS leadership that identified several root causes  for the breach. These causes included that the program’s  schedule did not allow time to gather user feedback or  address complexities discovered during development; new  requirements were added; and there was no consistent  performance requirement from USCIS leadership on what  the program was supposed to accomplish for specific product  lines. In July 2016, GAO found that USCIS was not following  its own policies or leading practices when developing software,  including ensuring that software meets expectations prior  to deployment and development outcomes are defined.  GAO made 12 recommendations to improve Transformation  program management. USCIS planned to re-baseline the  program to account for the schedule delay and subsequently  proposed organizational changes. In December 2016, DHS  leadership directed USCIS to stop planning and development  for new product lines, update its breach remediation plan  and acquisition documentation, and brief leadership on the  program’s revised approach by February 2017.
Program Office Comments Since its introduction in March 2015, the enhanced system  architecture has taken in over 2.7 million cases and USCIS  also introduced four forms. USCIS continues to modernize  the processes based on internal user feedback and input.  USCIS is reassessing the program goals and schedule  and will re-baseline the program in fiscal year 2017. USCIS  officials also provided technical comments on a draft of this  assessment, which GAO incorporated as appropriate.
section_title: Appendix II: Objectives, Scope, and Methodology
paragraphs: The objectives of this audit were designed to provide congressional  committees insight into the Department of Homeland Security’s (DHS)  major acquisition programs. We assessed the extent to which (1) DHS’s  major acquisition programs are on track to meet their schedule and cost  goals, (2) major acquisition programs are making progress in meeting key  performance parameters (KPP), and (3) DHS has taken actions to  strengthen implementation of its acquisition policy and to improve major  acquisition program outcomes. To answer these questions, we reviewed  26 of DHS’s 71 major acquisition programs, including 24 that we reviewed  in 2016. We reviewed all 16 of DHS’s Level 1 acquisition programs— those with life-cycle cost estimates (LCCE) of $1 billion or more—that had  at least one project, increment, or segment in the Obtain phase—the  stage in the acquisition life cycle when programs develop, test, and  evaluate systems—at the initiation of our audit. Additionally, to provide  insight into some of the factors that can lead to poor acquisition  outcomes, we reviewed 10 other major acquisition programs—including 5  Level 1 programs beyond the Obtain phase and 5 Level 2 programs that  have LCCEs between $300 million and $1 billion—that we or DHS  leadership had identified were at risk of not meeting their cost estimates,  schedules, or capability requirements. We have reported on many of  these programs in our past work. As part of our scoping effort, we met  with representatives from DHS’s Office of Program Accountability and  Risk Management (PARM), DHS’s main body for acquisition oversight, to  determine which programs (if any) were facing difficulties in meeting their  cost estimates, schedules, or capability requirements. The 26 selected  programs were sponsored by eight different components, and they are  identified in table 7, along with our rationale for selecting them.
To determine the extent to which DHS’s major acquisition programs are  on track to meet their schedule and cost goals, we collected key  acquisition documentation for each of the 26 programs, including all  Acquisition Program Baselines (APB) approved at the department level  since DHS’s current acquisition policy went into effect in November 2008.  DHS policy establishes that all major acquisition programs should have a  department-approved APB, which establishes a program’s critical cost,  schedule, and performance parameters, before they initiate efforts to  obtain new capabilities. All 26 programs had one or more department- approved APB since November 2008. We used these APBs to establish  the initial and current cost and schedule goals for the 26 programs. We  then developed a data collection instrument to help validate the  information from the APBs. Specifically, for each program, we pre- populated a data collection instrument to the extent possible with the  schedule and cost information we had collected from the APBs and our  2016 assessment (if applicable) to identify cost growth and schedule  slips, if any, since the program’s initial baseline was approved. We shared  our data collection instruments with officials from the program offices to  confirm or correct our initial analysis and to collect additional information  to enhance the timeliness and comprehensiveness of our data sets.  Additionally, in June 2016, we collected program schedule and cost data  from DHS’s Investment Evaluation, Submission, and Tracking (INVEST)  System, which is the department’s system for information on its major  acquisition programs. We compared the information obtained through the  program offices’ data collection instrument responses and the INVEST  system to our 2016 assessment (if applicable) or the programs’ most  recent department-approved APB to identify schedule and cost changes,  if any, since January 2016—the data cut-off date of our 2016  assessment. We then met with program officials to identify causes and  effects associated with any identified schedule slips and cost growth.  Subsequently, we drafted preliminary assessments for each of the 26  programs, shared them with program and component officials, and gave  these officials an opportunity to submit comments to help us correct any  inaccuracies, which we accounted for as appropriate (such as when new  information was available). We also met with senior acquisition oversight  officials to share observations about trends and issues across the  portfolio. Through this process, we determined that our data elements  were sufficiently reliable for the purpose of this engagement.
In addition, we compared the cost data we collected for each of the 26  programs to DHS’s funding plans to identify any projected funding gaps— a challenge that increases the likelihood that acquisition programs will not  meet their schedule or cost goals. Specifically, we compared current  yearly cost estimates from department-approved LCCEs, INVEST, or  program office updates to the funding plan presented in the Future Years  Homeland Security Program (FYHSP) report to Congress for fiscal years  2017-2021, which presents 5-year funding plans for each of DHS’s major  acquisition programs, to assess the extent to which a program was  projected to have a funding gap from fiscal year 2016 through fiscal year  2021. These calculations also accounted for any fiscal year 2016  carryover funds, but did not include other funds that programs brought  into fiscal year 2016 from sources such as re-programming, fees, and  other reimbursable expenses. This analysis was consistent with the  methodology we used in our 2016 annual assessment, which allowed us  to make comparisons to our March 2016 findings. We shared our  analysis with officials from the program offices and components to  confirm or correct our calculations. We subsequently identified actions  DHS had taken or planned to take to address projected program funding  gaps by reviewing key documentation, such as certification of acquisition  funding memorandums for programs that had completed an Acquisition  Decision Event (ADE) in 2016 and DHS’s resource allocation policies and  processes. We also met with program officials to identify causes and  effects associated with any projected funding gaps, and interviewed  senior financial officials from DHS headquarters to discuss actions they  had taken to implement our prior recommendations on addressing  program affordability issues.
To determine the extent to which DHS’s major acquisition programs are  making progress in meeting their KPPs, we reviewed DHS’s acquisition  policy and guidance, as well as key acquisition documentation for all 26  programs, including APBs and operational requirements documents  approved at the department level since DHS’s current acquisition policy  went into effect in November 2008. An operational requirements  document provides a number of performance parameters, including the  KPPs, which must be met by a program to close an existing capability  gap and provide a useful capability to the operator. We used these  documents to establish the KPPs for the 26 programs. We included these  KPPs in our pre-populated data collection instrument along with the  status of each programs’ KPPs collected through our 2016 assessment (if  applicable) to identify changes, if any, in the programs’ KPPs over time.  We shared our data collection instruments with officials from the program  offices to confirm or correct our initial analysis and to collect additional  information to enhance the timeliness and comprehensiveness of our  data sets. We also collected test reports and any letters of assessment  from DHS’s Director, Office of Test and Evaluation (DOT&E), which  assess system performance during operational testing. Operational  testing is intended to identify whether a system can meet its KPPs and  provide an evaluation of the operational effective and suitability of a  system in an operationally realistic environment. For the purposes of our  review, we defined operational testing as initial or follow-on operational  test and evaluation events, operational assessments, and limited user  tests. We used the programs’ APBs, data collection instruments, and  other documents to identify whether the programs had deployed new  capabilities to operators. We then reviewed the programs’ test reports  and DOT&E letters of assessment to determine what KPPs were tested  and whether the system met all of the KPPs tested. We relied on  information provided by the program offices, such as in the data collection  instrument responses in instances where programs did not have test  reports and DOT&E letters of assessment, or if these documents did not  explicitly assess programs’ KPPs. We considered a program’s KPP met if  it achieved, at a minimum, the threshold value outlined in the programs’  APB or operational requirements document. We assessed DHS’s  acquisition policy, guidance, and practices against GAO’s acquisition best  practices for managing acquisition programs. We also met with officials  from the program offices to identify reasons why KPPs had not yet been  demonstrated, and interviewed senior officials from DHS headquarters  about the program’s performance breach policy and requirements  definition processes.
To determine the extent to which DHS has taken actions to improve major  acquisition program outcomes and to strengthen implementation of its  acquisition policy, we reviewed DHS’s acquisition policy and guidance,  including current and prior versions of the Acquisition Management  Directive Instruction 102-01-001; acquisition decision memorandums  issued in calendar year 2016; and key acquisition documentation for  major acquisition programs, such as APBs, LCCEs, operational  requirements documents, as well as breach notifications and remediation  plans. We used the acquisition policy and guidance to identify changes  made by DHS in 2016, such as establishing new oversight initiatives or  revisions to existing policies. We then used the acquisition decision  memorandums and program documentation to assess DHS’s  implementation of its acquisition policy in 2016. Specifically, for programs  that received DHS approval for an ADE in 2016, we compared the  acquisition documentation approved by DHS leadership for that event to  the documentation requirements in DHS’s acquisition policy. In addition,  we reviewed program breach notifications, breach remediation plans, and  acquisition decision memorandums for each of the programs that  reported a breach in calendar year 2016 against DHS’s acquisition policy.  We assessed DHS’s acquisition management policies, guidance, and  practices against the Standards for Internal Control in the Federal  Government. Lastly, we interviewed acquisition management officials  from DHS headquarters to obtain their perspectives on how new and  ongoing acquisition management initiatives are intended to improve  program outcomes, as well as key management decisions.
We conducted this performance audit from May 2016 through April 2017  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix III: Comments from the Department of Homeland Security
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact listed above, Richard A. Cederholm (Assistant  Director), Katherine Trimble (Assistant Director), Aryn Ehlow (Analyst-in- Charge), Peter Anderson, Mathew Bader, Steven Bagley, Jason Berman,  Carissa Bryant, Andrew Burton, Erin Butkowski, Lisa Canini, Jenny Chow,  Adam Couvillion, John Crawford, Lorraine Ettaro, Laurier R. Fish, Laura  Gibbons, Betsy Gregory-Hosler, Yvette Gutierrez, Leigh Ann Haydon,  Kirsten Leikem, Sarah Martin, John Mickey, Erin O’Brien, Alexis Olson,  Katherine Pfeiffer, John Rastler, Sylvia Schatz, Jillian Schofield, Charlie  Shivers III, Roxanna Sun, Lindsay Taylor, and Hai Tran made key  contributions to this report.
section_title: Related GAO Products
paragraphs: Homeland Security Acquisitions: Joint Requirements Council’s Initial  Approach Is Generally Sound and It Is Developing a Process to Inform  Investment Priorities. GAO-17-171. Washington, D.C.: Oct. 24, 2016.
Homeland Security Acquisitions: DHS Has Strengthened Management,  but Execution and Affordability Concerns Endure. GAO-16-338SP.  Washington, D.C.: Mar. 31, 2016.
National Security Cutter: Enhanced Oversight Needed to Ensure  Problems Discovered during Testing and Operations Are Addressed.  GAO-16-148. Washington, D.C.: Jan. 12, 2016.
TSA Acquisitions: Further Actions Needed to Improve Efficiency of  Screening Technology Test and Evaluation. GAO-16-117. Washington,  D.C.: Dec. 17, 2015.
Coast Guard Aircraft: Transfer of Fixed-Wing C-27J Aircraft Is Complex  and Further Fleet Purchases Should Coincide with Study Results.  GAO-15-325. Washington, D.C.: Mar. 26, 2015.
Homeland Security Acquisitions: Major Program Assessments Reveal  Actions Needed to Improve Accountability. GAO-15-171SP. Washington,  D.C.: Apr. 22, 2015.
Homeland Security Acquisitions: DHS Should Better Define Oversight  Roles and Improve Program Reporting to Congress. GAO-15-292.  Washington, D.C.: Mar. 12, 2015.
Coast Guard Acquisitions: Better Information on Performance and  Funding Needed to Address Shortfalls. GAO-14-450. Washington, D.C.:  June 5, 2014.
Homeland Security Acquisitions: DHS Could Better Manage Its Portfolio  to Address Funding Gaps and Improve Communications with Congress.  GAO-14-332. Washington, D.C.: Apr. 17, 2014.
Homeland Security: DHS Requires More Disciplined Investment  Management to Help Meet Mission Needs. GAO-12-833. Washington,  D.C.: Sept. 18, 2012.
Department of Homeland Security: Assessments of Selected Complex  Acquisitions. GAO-10-588SP. Washington, D.C.: June 30, 2010.
Department of Homeland Security: Billions Invested in Major Programs  Lack Appropriate Oversight. GAO-09-29. Washington, D.C.: Nov. 18,  2008.