Coast Guard Acquisitions: Polar Icebreaker Program Needs to Address Risks before Committing Resources
Released: 2018-09-04T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: To maintain heavy polar icebreaking capability, the Coast Guard and the Navy are collaborating to acquire up to three new heavy polar icebreakers through an integrated program office. The Navy plans to award a contract in 2019. GAO has found that before committing resources, successful acquisition programs begin with sound business cases, which include plans for a stable design, mature technologies, a reliable cost estimate, and a realistic schedule.

Section 122 of the National Defense Authorization Act for Fiscal Year 2018 included a provision for GAO to assess issues related to the acquisition of the icebreaker vessels. In addition, GAO was asked to review the heavy polar icebreaker program's acquisition risks. This report examines, among other objectives, the extent to which the program is facing risks to achieving its goals, particularly in the areas of design maturity, technology readiness, cost, and schedule. GAO reviewed Coast Guard and Navy program documents, analyzed Coast Guard and Navy data, and interviewed knowledgeable officials.

section_title: What GAO Found
paragraphs: The Coast Guard—a component of the Department of Homeland Security (DHS)—did not have a sound business case in March 2018, when it established the cost, schedule, and performance baselines for its heavy polar icebreaker acquisition program, because of risks in four key areas:

Design. The Coast Guard set program baselines before conducting a preliminary design review, which puts the program at risk of having an unstable design, thereby increasing the program's cost and schedule risks. While setting baselines without a preliminary design review is consistent with DHS's current acquisition policy, it is inconsistent with acquisition best practices. Based on GAO's prior recommendation, DHS is currently evaluating its policy to better align technical reviews and acquisition decisions.

Technology. The Coast Guard intends to use proven technologies for the program, but did not conduct a technology readiness assessment to determine the maturity of key technologies prior to setting baselines. Coast Guard officials indicated such an assessment was not necessary because the technologies the program plans to employ have been proven on other icebreaker ships. However, according to best practices, such technologies can still pose risks when applied to a different program or operational environment, as in this case. Without such an assessment, the program's technical risk is underrepresented.

Cost. The lifecycle cost estimate that informed the program's $9.8 billion cost baseline substantially met GAO's best practices for being comprehensive, well-documented, and accurate, but only partially met best practices for being credible. The cost estimate did not quantify the range of possible costs over the entire life of the program. As a result, the cost estimate was not fully reliable and may underestimate the total funding needed for the program.

Schedule. The Coast Guard's planned delivery dates were not informed by a realistic assessment of shipbuilding activities, but rather driven by the potential gap in icebreaking capabilities once the Coast Guard's only operating heavy polar icebreaker—the Polar Star —reaches the end of its service life (see figure).

GAO's analysis of selected lead ships for other shipbuilding programs found the icebreaker program's estimated construction time of 3 years is optimistic. As a result, the Coast Guard is at risk of not delivering the icebreakers when promised and the potential gap in icebreaking capabilities could widen.

section_title: What GAO Recommends
paragraphs: GAO is making six recommendations to the Coast Guard, DHS, and the Navy. Among other things, GAO recommends that the program conduct a technology readiness assessment, re-evaluate its cost estimate and develop a schedule according to best practices, and update program baselines following a preliminary design review. DHS concurred with all six of GAO's recommendations.

section_title: Letter
paragraphs: The Coast Guard, a component within the Department of Homeland  Security (DHS), is developing the first heavy icebreakers it has bought in  over 40 years. Overall, the Coast Guard and the Navy plan to invest up  to approximately $9.8 billion in lifecycle costs for the acquisition,  operations, and maintenance of three heavy polar icebreakers (HPIB).  These ships will enable the Coast Guard to maintain heavy polar  icebreaking capability and recapitalize its icebreaking fleet. Congressional  committees have expressed concern regarding the Coast Guard’s ability  to ensure year-round access to the Arctic and Antarctic with the current  fleet, which affects U.S. economic, maritime, and national security  interests in these regions. As the only operating HPIB nears the end of its  service life, the Coast Guard is planning for delivery of the lead ship by as  early as 2023 to avoid a gap in capability, with subsequent ship deliveries  anticipated in 2025 and 2026. In 2016, in response to a Congressional  report, the Navy and the Coast Guard established an integrated program  office (IPO) to leverage the Navy’s shipbuilding expertise for acquiring the  icebreakers for Coast Guard operations. In March 2018, the Navy  released the solicitation for a contract to design and construct up to three  HPIBs. The Navy indicated that it anticipates awarding the contract in the  third quarter of fiscal year 2019 with $270 million in Navy funding that  Congress has appropriated for the program.

Section 122 of the National Defense Authorization Act for Fiscal Year  2018 included a provision for us to assess issues related to the  procurement of the HPIB vessels. We were asked to review any risks to  the HPIB program’s ability to carry out its planned acquisition. This report  examines (1) the extent to which the HPIB program has taken steps to  develop mature designs and technologies consistent with best practices,  (2) the extent to which the HPIB program has taken steps to set realistic  cost and schedule estimates, and (3) the status of the HPIB program’s  contracting efforts and funding considerations.

To assess the extent to which the HPIB program has taken steps to  develop mature designs and technologies consistent with GAO-identified  best practices, we reviewed program performance and design  requirements, including the program’s operational requirements  documents, specifications, and technical baseline. We also reviewed the  program’s alternatives analysis study, tailored systems engineering plan,  test and evaluation master plan, model testing results; cooperative  agreements with Canada related to the HPIB; excerpts from industry  studies; and the March 2018 detail design and construction request for  proposals and subsequent amendments. We also reviewed relevant  DHS, Coast Guard, and Department of Defense (DOD) acquisition  guidance and instructions. From these documents, we determined the  program’s design and technology efforts and compared them to GAO’s  various best practices, including the knowledge-based approaches to  shipbuilding and major acquisitions in general, and approaches to  evaluating technology readiness.

To assess the extent to which the HPIB program has taken steps to set  realistic cost and schedule estimates, we determined the extent to which  the estimates were consistent with best practices as identified in GAO’s  Cost Estimating and Assessment and Schedule Assessment guides. To  assess the cost estimate, we reviewed the HPIB’s January 2018 lifecycle  cost estimate used to support the program’s initial cost baselines,  examined Coast Guard and Navy documentation supporting the estimate,  relevant program briefs to Coast Guard leadership, and HPIB program  documentation containing cost, schedule, and risk information, among  other steps. To assess the program’s schedule, we compared the HPIB  schedule documents, including the program’s initial schedule baselines,  delivery schedules from the HPIB’s request for proposals for the detail  design and construction contract, and integrated master schedule, to  selected GAO best practices for project schedules. These best practices  include establishing the duration of activities, conducting a schedule risk  analysis, and ensuring reasonable total buffer or margin. In addition, we  compared the HPIB program schedule to other shipbuilding programs’  schedules, among other steps.

To determine the status of the HPIB program’s contracting efforts and  funding considerations, we reviewed the program’s acquisition plan,  March 2018 request for proposals and subsequent amendments,  certification of funds memorandum, budget justifications, program  lifecycle cost estimate, and the Coast Guard’s fiscal year 2019 Capital  Investment Plan. For all objectives, we interviewed knowledgeable DHS,  Coast Guard, and Navy officials. Appendix I presents a more detailed  description of the scope and methodology of our review.

We conducted this performance audit from August 2017 to September  2018 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: History of the Polar Icebreakers and Icebreaking Capability Gap
paragraphs: The Coast Guard has been responsible for carrying out the nation’s polar  icebreaking missions since 1965—when it assumed primary responsibility  for the nation’s polar icebreaking fleet. The Coast Guard’s  responsibilities are outlined in various statutes, policies, and interagency  agreements.

A 2010 Coast Guard study identified gaps in the Coast Guard’s ability to  support and conduct missions in the Arctic and Antarctic. As a result, in  June 2013, the Coast Guard established the need for up to three heavy  polar icebreakers and three medium icebreakers to adequately meet  these Coast Guard mission demands. More recently, in November 2017,  Coast Guard officials reiterated that they will be able to fulfill all mission  requirements—which include support to agencies with Arctic  responsibilities such as DOD, the National Science Foundation (NSF),  Department of State, National Oceanic and Atmospheric Administration,  and National Aeronautics and Space Administration—with a fleet of three  heavy and three medium polar icebreakers. Coast Guard officials told us they are not currently assessing acquisition of the medium polar  icebreakers because they are focusing on the HPIB acquisition and plan  to assess the costs and benefits of acquiring medium polar icebreakers at  a later time.

The Coast Guard currently has two active polar icebreakers in its fleet— the Polar Star, a heavy icebreaker, and the Healy, a medium icebreaker.  An additional Coast Guard heavy icebreaker, the Polar Sea, has been  inactive since 2010 when it experienced a catastrophic engine failure.  Commissioned in 1976, the Polar Star is the world’s most powerful active  non-nuclear icebreaker. The less powerful Healy primarily supports Arctic  research. Although the Healy is capable of carrying out a wide range of  activities, it cannot operate independently in the ice conditions in the  Antarctic or ensure timely access to some Arctic areas in the winter. See  figure 1 for the Coast Guard’s active icebreakers.

The Coast Guard has faced challenges in meeting the government’s  icebreaking needs in recent years. For example, in June 2016, we found  that when neither the Polar Sea nor the Polar Star was active in 2011 and  2012, the Coast Guard did not maintain assured, year-round access to  both the Arctic and Antarctic, as the Healy cannot reach ice-covered  areas with more than 4½ feet of ice. According to a January 2017 Coast  Guard assessment, the Coast Guard does not plan to recommission the  Polar Sea because it would not be cost-effective.

section_title: Polar Star Sustainment Efforts
paragraphs: According to Coast Guard planning documents, the Polar Star’s service  life is estimated to end between fiscal years 2020 and 2023. This creates  a potential heavy polar icebreaker capability gap of about 3 years,  assuming the Polar Star’s service life ends in 2020 and the lead HPIB is  delivered by the end of fiscal year 2023 as planned. If the lead ship is  delivered later than planned in this scenario, the potential gap could be  more than 3 years. As a result, according to a 2017 polar icebreaking  bridging strategy, the Coast Guard is planning to recapitalize the Polar  Star’s key systems starting in 2020 to extend the service life of the ship  until the planned delivery of the second HPIB (see figure 2).

In September 2017, we found that the Coast Guard’s $75 million cost  estimate for the Polar Star life extension project may be unrealistic, in part  because it was based on the assumption of continuing to use parts from  the decommissioned Polar Sea, as has been done in previous  maintenance events. Because of the finite number of parts available  from the Polar Sea, the Coast Guard may have to acquire new parts for  the Polar Star that could increase the $75 million estimate. As a result, we  recommended that the Coast Guard complete a comprehensive cost  estimate and follow cost estimating best practices before committing to  the life extension project. The Coast Guard concurred with this  recommendation.

As of May 2018, Coast Guard officials told us they were still conducting  ship engineering inspections on the Polar Star to determine the details of  the work needed for the limited service life extension, which will then  inform the development of a cost estimate. In January 2018, the Coast  Guard completed its ship structures and machinery evaluation board  report. Coast Guard officials told us that this report will help to determine  the details of the work needed for the limited life extension. The January  2018 report estimated the remaining service life of the Polar Star as 5  years or less. In April 2018, the Coast Guard approved the Polar Star life  extension project to establish requirements and evaluate the feasibility of  alternatives that will achieve the requirements. Coast Guard officials  stated they completed a notional cost estimate in April 2018 and plan to  complete a detailed formal cost estimate by June 2020.

section_title: Coast Guard’s and Navy’s Roles in the Heavy Polar Icebreaker Program
paragraphs: The Coast Guard and the Navy established the IPO to collaborate and  develop a management approach to acquire three HPIBs. Through the  IPO, the Coast Guard planned to leverage the Navy’s shipbuilding  expertise and pursue an accelerated acquisition schedule. A Coast Guard  program manager heads the IPO, which includes embedded Navy  officials who provide acquisition, contracting, engineering, cost- estimating, and executive support to the program. The IPO has  responsibility for managing and executing the HPIB’s acquisition  schedule, acquisition oversight reviews, budget and communications, and  interagency coordination. In addition, the IPO coordinates with several  key organizations within the Coast Guard and Navy that contribute to the  HPIB program, including:

Coast Guard Capabilities Directorate: This directorate is  responsible for identifying and providing capabilities, competencies,  and capacity and developing standards to meet Coast Guard mission  requirements. The directorate sponsored the HPIB’s operational  requirements document, which provides the key performance  parameters the HPIB must meet—such as icebreaking, endurance,  and interoperability thresholds and objectives.

Ship design team: The ship design team includes Coast Guard and  Navy technical experts that develop ship specifications based on the  HPIB operational requirements document. The ship design team is  under the supervision of a Coast Guard ship design manager, who  provides all technical oversight for development of the HPIB design,  including development of “indicative,” or concept, designs used to  inform the ship’s specifications and the program’s lifecycle cost  estimate. Generally, the purpose of an indicative design is to  determine requirements feasibility, support cost estimating, and  provide a starting point for trade studies.

Naval Sea Systems Command (NAVSEA) Cost Engineering and  Industrial Analysis Group (NAVSEA 05C): The group developed  the HPIB lifecycle cost estimate, which informs the program’s cost  baselines and affordability constraints. NAVSEA 05C developed the  HPIB’s lifecycle cost estimate based on the ship design team’s  indicative design and the technical assumptions outlined in the  program cost estimating baseline document.

NAVSEA Contracts Directorate (NAVSEA 02): This directorate  includes the Navy contracting officer who released the HPIB detail  design and construction contract’s solicitation in March 2018 and  plans to award the contract under Navy authorities. The contracting  officer performs contract management services and provides  guidance to the IPO to help ensure the HPIB’s contract adheres to  DOD and Navy contracting regulations and guidance.

Figure 3 shows key organizations that support the HPIB program and  their responsibilities prior to the award of the contract.

Since establishing the IPO, the Coast Guard, DHS, and the Navy  formalized agreements on their approach for the HPIB acquisition in three  2017 memorandums of agreements and understanding. These  agreements define the Navy’s and Coast Guard’s roles in the HPIB  acquisition with respect to funding responsibilities, acquisition oversight  functions, and contracting and program management authorities, among  other things.

section_title: Heavy Polar Icebreaker Program’s Acquisition Framework
paragraphs: DHS, the Coast Guard, and the Navy have agreed to manage and  oversee the HPIB program using DHS’s acquisition framework, as Coast  Guard is a component within DHS. DHS’s acquisition policy establishes  that a major acquisition program’s decision authority shall review the  program at a series of predetermined acquisition decision events (ADE)  to assess whether the major program is ready to proceed through the  acquisition life-cycle phases (see figure 4).

As we found in April 2018, the Coast Guard and the Navy will adhere to a  tailored DHS acquisition framework for the HPIB program that  supplements DHS ADE reviews with additional “gate” reviews adopted  from the Navy’s acquisition processes. The DHS Under Secretary for  Management retains final decision authority for the HPIB’s ADEs as the  acquisition decision authority.

The HPIB program achieved a combined ADE 2A/2B in February 2018,  when DHS approved the program’s baselines and permitted the program  to enter into the Obtain Phase of the DHS acquisition framework. The  corresponding acquisition decision memorandum was signed in March  2018. The Coast Guard and the Navy plan to start detail design work for  the HPIB in June 2019, once the detail design and construction contract  is awarded. In Navy shipbuilding, detail design work can include outlining  the steel structure of the ship; determining the routing of systems, such as  electrical and piping, throughout the ship; and developing work  instructions for constructing elements of the ship, such as installation  drawings and material lists.

The program’s ADE 2C, or the low-rate initial production decision,  corresponds with the approval to start construction of the lead ship, which  is planned to begin no later than June 2021. Key steps typically taken in  the construction phase of a Navy ship include steel cutting and block  fabrication, assembly and outfitting of blocks, keel laying and block  erection, launch of the ship from dry dock, system testing and  commissioning, sea trials, and delivery and acceptance (see appendix II  for more detailed information on each shipbuilding phase). ADE 3,  scheduled to be held no later than March 2026, authorizes the program to  start follow-on test and evaluation.

Figure 5 shows the HPIB’s acquisition framework, including ADE  milestones and major program decision points, and how they relate to the  shipbuilding phases.

DHS acquisition policy establishes that the acquisition program baseline  is the fundamental agreement between programs, component, and  department-level officials establishing what will be delivered, how it will  perform, when it will be delivered, and what it will cost. Specifically, the  program baseline establishes a program’s schedule, costs, and key  performance parameters, and covers the entire scope of the program’s  lifecycle. The HPIB acquisition program baseline serves as an agreement  between the Coast Guard and DHS that the Coast Guard will execute the  acquisition within the bounds detailed in the document. The acquisition  program baseline establishes objective (target) and threshold (maximum  acceptable for cost, latest acceptable for schedule, and minimum  acceptable for performance) baselines. Tables 1, 2, and 3 show selected  cost, schedule, and performance baselines that DHS approved for the  HPIB program at ADE 2A/2B in March 2018.

After DHS approved the HPIB’s program baselines, the Navy released  the solicitation for the program’s detail design and construction contract in  March 2018. As revised, the solicitation requires offerors to submit their  technical proposals in August 2018 and their price proposals in October  2018. The Navy plans to competitively award the HPIB detail design and  construction contract to a single shipyard for all three ships in June 2019.  The contract award would include design (advance planning and  engineering) and long lead time materials, with separate options for detail  design and construction of each of the three ships. The HPIB contract  award and administration will follow DOD and Navy contracting  regulations and policies, including the Defense Federal Acquisition  Regulation Supplement. Although the Navy is planning to award the  contract, the source selection authority is from the Coast Guard, with both  Coast Guard and Navy personnel serving on the source selection  evaluation board.

section_title: Starting Programs with Sound Business Cases
paragraphs: Our prior work has found that successful programs start out with solid,  executable business cases before setting program baselines and  committing resources. For Coast Guard programs, such a business  case would be expected at ADE 2A/2B. A sound business case requires  balance between the concept selected to satisfy operator needs and the  resources—technologies, design knowledge, funding, and time—needed  to transform the concept into a product—or in the HPIB’s case, a ship. At  the heart of a business case is a knowledge-based approach—we have  found that successful shipbuilding programs build on attaining critical  levels of knowledge at key points in the shipbuilding process before  significant investments are made. We have previously found that key  enablers of a good business case include firm, feasible requirements;  plans for a stable design; mature technologies; reliable cost estimates;  and realistic schedule targets. Without a sound business case, acquisition programs are at risk of breaching the cost, schedule, and  performance baselines set when the program was initiated—in other  words, experiencing cost growth, schedule delays, and reduced  capabilities.

In November 2016, we found that a particular challenge for Congress is  the fact that committees must often consider requests to authorize and  fund a new program well ahead of program initiation—the point at which  key business case information would be presented. Given the time lag  between budget requests and the decision to initiate a new acquisition  program, Congress could be making critical funding decisions with limited  information about the soundness of the program’s business case.  Although the HPIB program has already proceeded through ADE 2A/2B  and established acquisition program baselines, information about the  soundness of the HPIB’s business case will be helpful for decision  makers as the Coast Guard and the Navy request funding in preparation  for the detail design and construction contract award in June 2019 and  anticipated construction start by the end of June 2021—two points at  which significant resource commitments will need to be made.

section_title: The Coast Guard Did Not Assess Design Maturity or Technology Risks Prior to Setting the Polar Icebreaker Program’s Baselines
paragraphs: The Coast Guard set the HPIB’s acquisition program baselines at ADE  2A/2B without conducting a preliminary design review to assess the  design maturity of the ship or a technology readiness assessment to  determine the maturity of key technologies. This approach meets DHS  acquisition policy requirements but is contrary to our best practices (see  figure 6).

While the Coast Guard is committed to a stable design prior to the start of  lead ship construction, it established baselines without clear knowledge of  the ship design because it does not plan to assess design maturity until  after the planned June 2019 award of the detail design and construction  contract. In addition, without a technology readiness assessment, the  Coast Guard does not have full insight into whether the technologies are  mature, potentially underrepresenting technical risk and increasing design  risk. As a result, the Coast Guard will be committing resources to the  HPIB program without key elements of a sound business case, increasing  the risk that the program will exceed its planned costs and schedule.
subsections: section_title: Polar Icebreaker Program Took Steps to Identify Design Risks but Did Not Assess Design Maturity Prior to Setting Baselines
subsections: section_title: Early Efforts to Identify Design Risks
paragraphs: To help inform the HPIB’s key performance parameters, specifications,  and design considerations prior to setting the acquisition program  baselines, the Coast Guard conducted design studies and partnered with  Canada (with which the United States has an existing cooperative  agreement) to gain knowledge on the HPIB’s design risks. For example:

Starting in November 2016, the HPIB ship design team developed an  indicative (or concept) design, which has undergone several revisions  as more information became available, completing a fifth iteration in  September 2017. To inform the HPIB indicative design, the ship  design team told us they used design elements with validated  characteristics, such as the hull form, from existing Coast Guard  icebreakers, including the Polar Star, Polar Sea, Healy, and the  Mackinaw (a Great Lakes icebreaker). Collectively, these icebreakers  informed elements of the indicative design such as the size and  producibility of the ship. The indicative design represents an  icebreaker design that meets the threshold key performance  parameter of breaking 6 feet of ice at a continuous speed of 3 knots  rather than the objective parameter of 8 feet of ice at a continuous  speed of 3 knots. Coast Guard officials stated that based on  preliminary analysis, a design that meets the HPIB’s objective key  performance parameters would be an entirely separate design and  would be too costly to construct. Coast Guard officials told us that in  addition to price, the shipbuilders’ HPIB proposals will be evaluated  on design factors, including how much the potential design exceeds  the threshold icebreaking performance parameters.

In February 2017, the Coast Guard contracted with five shipbuilders,  who teamed with icebreaker design firms, to conduct a series of  iterative design studies. These studies examined major design cost  drivers and technology risks for the HPIB program. Coast Guard  officials stated the results of these studies helped inform and refine  the ship’s specifications and provided them with a better  understanding of the technology risks and schedule challenges. As of  February 2018, each contract was valued at about $5.6 million. Under  these contracts, each shipbuilder completed five detailed industry  study iterations. For example, the shipbuilders analyzed various hull  forms, propulsion systems, cold weather operations, space  arrangements, and icebreaking enhancements.

In April 2017, the Coast Guard completed an alternatives analysis  study—an independent study required prior to ADE 2A that identifies  the most efficient method of addressing an identified capability gap.  The study examined various options, including whether existing  foreign icebreakers could meet the Coast Guard’s HPIB performance  requirements. The Coast Guard analyzed 18 domestic and foreign  icebreaker designs against the HPIB’s key performance parameters  and other requirements, such as seakeeping and habitability. The  icebreaker designs included a variety of icebreakers in terms of  propulsion power and size, such as nuclear-powered Russian  icebreakers and polar research and supply vessels from Australia,  Finland, and Germany.

The alternatives analysis found that only a Russian nuclear-powered  icebreaker and a design for a Canadian diesel-electric-powered  icebreaker, which has yet to be constructed, passed initial screening  for design maturity and performance requirements. Given a previous  independent study analyzing the cost-effectiveness of nuclear- powered icebreakers, the Coast Guard deemed a nuclear-powered  icebreaker design as infeasible. The alternatives analysis also noted  that the Canadian design met icebreaking requirements. However,  Coast Guard officials told us the Canadian design did not meet  requirements such as habitability and military-oriented multi-mission  tasks, but the design could potentially be modified to meet those  needs. In addition, IPO officials stated the Canadian design was  designed for science missions rather than military missions. The  Canadian design was considered among some of the shipbuilders as  a starting point in examining HPIB design risks.

From May to August 2017, the Coast Guard tested two scale models  of icebreakers at the Canadian National Research Council’s ice tank  facility in Newfoundland. Coast Guard officials told us the testing  helped to mitigate potential design risks with the hull form and  propulsors—a mechanical device that generates thrust to provide  propulsion for the ship. The Coast Guard tested the resistance,  powering, and maneuvering of the model icebreakers’ hull form and  propulsion to inform their indicative design and discovered that the  ship’s maneuverability was a challenge during model testing.  However, through model testing, the Coast Guard was able to validate  general characteristics of its indicative design, including power needs  and the hull form. In addition to model testing, Canadian Coast Guard  officials told us that the U.S. Coast Guard has engaged with them in  formal and informal exchanges regarding icebreaker acquisitions  more generally.

As a result of its indicative design, industry studies, and model testing  efforts, the Coast Guard identified the integrated power plant, propulsors,  and hull form as key design considerations for the HPIB. Because these  design elements work together to ensure the HPIB can meet its  icebreaking requirements, we determined that these are the HPIB’s main  design risks (see figure 7).

Although the Coast Guard undertook early efforts to identify design risks,  it did not conduct a preliminary design review for the HPIB program prior  to setting program acquisition baselines at ADE 2A/2B. These baselines  inform DHS’s and Coast Guard’s decisions to commit resources. Our best  practices for knowledge-based acquisitions state that before program  baselines are set, programs should hold key systems engineering events,  such as a preliminary design review, to help ensure that requirements are  defined and feasible and that the proposed design can be met within cost,  schedule, and other system constraints. Similarly, in November 2016,  we found that establishing a preliminary design through early detailed  systems engineering results in better program outcomes than doing so  after program start. During the HPIB’s preliminary design review, the  Coast Guard plans to verify that the contractor’s design meets the  requirement of the ship specifications and is producible, and the schedule  is achievable, among other activities.

The Coast Guard has yet to conduct a preliminary design review for the  HPIB program because DHS’s current acquisition policy does not require  programs to do so until after ADE 2A/2B. The Coast Guard plans to hold  the preliminary design review by December 2019, after the award of the  detail design and construction contract. Holding a preliminary design  review after ADE 2A/2B is consistent with DHS policy. However, in April  2017, we found that DHS’s sequencing of the preliminary design review is  not consistent with our acquisition best practices, which state that  programs should pursue a knowledge-based acquisition approach that  ensures program needs are matched with available resources—such as  technical and engineering knowledge, time, and funding—prior to setting  baselines. In that report, we found that by initiating programs without a  well-developed understanding of system needs through key engineering  reviews such as the preliminary design review, DHS increases the  likelihood that programs will change their user-defined key performance  parameters, costs, or schedules after establishing their baselines. As a  result, we recommended that DHS update its acquisition policy to require  key technical reviews, including the preliminary design review, to be  conducted prior to approving programs’ baselines. DHS concurred with  this recommendation and stated that it planned to initiate a study to  assess how to better align its processes for technical reviews and  acquisition decisions. Upon completion of the study, DHS plans to update  its acquisition policies, as appropriate.

Instead of establishing the HPIB program’s acquisition program baselines  after assessing the shipbuilder’s preliminary design, the Coast Guard  established cost baselines based on a cost estimate that used the ship  design team’s indicative design. Coast Guard officials told us that the  selected shipbuilder will develop its own HPIB design as part of the detail  design and construction contract, independent of the indicative design.  The ship design team noted that the indicative design informed the ship’s  specifications but is not meant to be an optimized design, does not  represent a design solution, and will not be provided to the shipbuilders.  Coast Guard officials stated that the shipbuilders that respond to the  request for proposals will propose their own designs based on their  production capabilities, which will drive where they will place components,  such as bulkheads, within the design. As a result, the shipbuilder’s design  will be different from the indicative design.

By setting the HPIB’s acquisition program baselines prior to gaining  knowledge on the shipbuilder’s design, the Coast Guard has established  cost, schedule, and performance baselines without a stable or mature  design. Although completing the preliminary design review after setting  program baselines is consistent with DHS policy, this puts the Coast  Guard at risk of breaching its established baselines and having to revise  them later in the acquisition process, after a contract has been signed  and significant resources have already been committed to the program.  At that point, the program will be well underway and it will be too late for  decision makers to make appropriate tradeoff decisions between  requirements and resources without causing disruptions to the program.

Consistent with DHS acquisition policy, DHS and the Coast Guard must  monitor the HPIB program against the acquisition program baselines set  at ADE 2A/2B; however, DHS acquisition policy does not require an  official update to the baseline unless the program breaches its baselines  or until the next major milestone, whichever occurs first. For the HPIB, the  next milestone is ADE 2C, which is currently planned for no later than  June 2021. ADE 2C corresponds to the approval of low-rate initial  production and in the case of the HPIB, the start of construction for the  lead ship. Evaluating the HPIB’s baselines at ADE 2C—immediately  before the shipbuilder is authorized to start construction—is too late  because the funding required for the construction phase likely would have  already been requested and provided. On the other hand, evaluating the  acquisition program baselines after the preliminary design review but  before ADE 2C would help to ensure that the knowledge gained during  the preliminary design review is used to inform the program baselines and  business case for investing in the HPIBs before significant resource  commitments are made.

Although the Coast Guard set the acquisition program baselines prior to  gaining knowledge on the feasibility of the selected shipbuilder’s design, it  has expressed a commitment to having a stable design prior to the start  of lead ship construction. In Navy shipbuilding, detail design typically  encompasses three design phases:

Basic design. Includes fixing the ship steel structure; routing all major  distributive systems, including electricity, water, and other utilities; and  ensuring the ship will meet the performance specifications.

Functional design. Includes providing further iteration of the basic  design, providing information on the exact position of piping and other  outfitting in each block, and completing a 3D product model.

Production design. Generating work instructions that show detailed  system information and including guidance for subcontractors and  suppliers, installation drawings, schedules, material lists, and lists of  prefabricated materials and parts.

Shipbuilding best practices we identified in 2009 found that design  stability is achieved upon completion of the basic and functional  designs. At this point of design stability, the shipbuilder has a clear  understanding of the ship structure as well as how every system is set up  and routed throughout the ship. Consistent with our best practices, prior  to the start of construction on the lead ship, the Coast Guard will require  the shipbuilder to complete basic and functional designs, develop a 3D  model output, and provide at least 6 months of production information to  support the start of construction. IPO officials have stated that they are  committed to ensuring that the HPIB’s design is stable before  construction of the lead ship begins, given the challenges prior Navy  shipbuilding programs have experienced when construction proceeded  before designs were completed.

section_title: Coast Guard Intends to Use Proven Technologies for the Polar Icebreaker Program but Has Not Assessed Their Maturity
paragraphs: The Coast Guard intends on using what it refers to as proven  technologies for the HPIB but has not conducted a technology readiness  assessment to determine maturity of key technologies that drive  performance of the ship prior to ADE 2A/2B, which is inconsistent with our  best practices. A technology readiness assessment is a systematic,  evidence-based process that evaluates the maturity of critical  technologies—hardware and software technologies critical to the  fulfillment of the key objectives of an acquisition program. This  assessment does not eliminate technology risk but, when done well, can  illuminate concerns and serve as a basis for realistic discussions on how  to mitigate potential risks. According to our best practices, a technology  readiness assessment should be conducted prior to program initiation.  DHS systems engineering guidance also recommends conducting a  technology readiness assessment before ADE 2A to help ensure that the  program’s technologies are sufficiently mature by the start of the program.

The Coast Guard intends on using what it has deemed “state-of-the- market” or “proven” technologies for the HPIB. DHS’s technical  assessment of the HPIB noted that the February 2017 design studies  resulted in industry producing designs that used commercially available,  state-of-the-market, and proven technologies. From the studies and  industry engagement, Coast Guard officials determined that the  technologies required for the HPIB, such as the integrated power plant  and azimuthing propulsors—thrusters that rotate up to 360 degrees and  provide propulsion to the ship—are available commercially and do not  need to be developed. Coast Guard officials further stated that the  integrated power plant is the standard power plant used on domestic and  foreign icebreakers. Coast Guard officials told us that similarly, market  survey data on azimuthing propulsors shows that ice-qualified azimuthing  propulsors in the power range have been used on foreign icebreakers.  The Coast Guard has also communicated to industry through the request  for proposals that the HPIB should have only proven technology and  plans to have the shipbuilders provide information on the maturity of the  technologies when they submit their proposals. As a result, Coast Guard  officials stated the HPIB program does not have any critical technologies,  as defined by DHS systems engineering guidance, and does not need to  conduct a technology readiness assessment.

However, according to DHS systems engineering guidance, a technology  element is considered critical if the system being acquired depends on  this technology to meet operational requirements, and if the technology or  its application is new, novel, or in an area that poses major technological  risk during detailed design or demonstration. The guidance further states  that technologies can become critical if they need to be modified from  prior successful use or expected to operate in an environment beyond  their original demonstrated capability. Similarly, according to our best  practices for assessing technology readiness, critical technologies are not  just technologies that are new or novel. Technologies used on prior  systems can also become critical if they are being used in a different  form, fit, or function.

Our technology readiness assessment guide notes that program officials  sometimes disregard critical technologies when they have longstanding  history, knowledge, or familiarity with them. The best practices guide cites  examples of organizations not considering a technology critical if it has  been determined to be mature, has already been fielded, or does not  currently pose a risk to the program. Additionally, our guide notes that  contractors may be overly optimistic about the maturity of critical  technologies, especially prior to contract awards. According to our best  practices guide, presuming a previously used technology as mature is  problematic when the technologies are being reapplied to a different  program or operational environment.

As a result, based on our analysis of available Coast Guard information,  we believe the HPIB’s planned integrated power plant and azimuthing  propulsors should be considered critical technologies given their criticality  in meeting key performance parameters, their use in a different  environment from prior ships, and the extent to which they pose major  cost risks (see table 4).

Without conducting a technology readiness assessment, the Coast Guard  does not have insight into how mature these critical technologies are.  According to our best practices, evaluating critical technologies requires  disciplined and repeatable steps and criteria to perform the assessment  and make credible judgments about their maturity. The evaluation of each  critical technology must be based on evidence such as data and test  results. In addition, the team that assesses the technologies must be  objective and ideally independent. Instead, the Coast Guard has relied  on industry to provide information on the maturity of the HPIB’s  technologies and uses terms such as “state-of-the-market” or “proven,”  which do not translate into meaningful measures for systematically  communicating the technology readiness, especially when discussing  new applications of existing technologies.

Additionally, even if the Coast Guard determines the maturity levels of the  HPIB’s technologies through an objective and independent technology  readiness assessment, the program’s planned level of maturity for the  ship’s technologies falls short of our best practices. According to the  HPIB’s systems engineering tailoring plan and request for proposals, the  program intends on implementing only proven technologies that have  been demonstrated in a relevant environment, commensurate with a  technology readiness level (TRL) of 6. However, our best practices do  not consider a technology to be mature until it has been demonstrated in  an operational environment, commensurate with a TRL 7. Specifically,  our best practices for shipbuilding recommend that programs should  require critical technologies to be matured into actual prototypes and  successfully demonstrated in an operational or a realistic environment  (TRL 7) before a contract is awarded for the detail design of a new ship.  DHS’s systems engineering guidance also states that critical technologies  below TRL 7 should be identified as technical risks.

By not conducting a technology readiness assessment and identifying,  assessing, and maturing its critical technologies prior to setting the  HPIB’s program baselines and prior to awarding the detail design  contract, the Coast Guard is underrepresenting the program’s technical  risks and understating its cost, schedule, and performance risks.  Technology risks that manifest later could require the shipbuilder to  redesign parts of the ship, which increases the risk of rework and  schedule delays during the construction phase.

section_title: The Coast Guard Based the Polar Icebreaker Program’s Baselines on a Cost Estimate That Is Not Fully Reliable and an Optimistic Schedule
paragraphs: The cost estimate and schedule that informed DHS’s decision to  authorize the HPIB program do not reflect the full scope of the program’s  risks. We found that while the Navy substantially adhered to a number of  best practices when it developed the HPIB’s cost estimate, the estimate is  not fully reliable, primarily because it does not reflect the full range of  possible costs over the HPIB’s 30-year lifecycle. We also found the HPIB  schedule was not informed by a realistic assessment of the work  necessary to construct the ship. Rather, the schedule was driven by the  potential gap in icebreaking capabilities once the Coast Guard’s only  operating HPIB reaches the end of its service life. Reliable cost estimates  and schedules are key elements of an executable business case, and are  needed at the outset of programs—when competitive pressures to obtain  funding for the program are high—to provide decision makers with insight  into how risks affect a program’s ability to deliver within its cost and  schedule goals.
subsections: section_title: Polar Icebreaker Program’s Cost Estimate Substantially Met Best Practices but Is Not Fully Reliable Because It Does Not Include Full Range of Possible Costs
paragraphs: We found that the lifecycle cost estimate used to inform the HPIB  program’s baselines substantially adheres to most cost estimating best  practices; however, the estimate is not fully reliable. The cost estimate  only partially met best practices for being credible primarily because it did  not quantify the range of possible costs over the entire life of the program.  We assessed the program’s lifecycle cost estimate, which was performed  by NAVSEA 05C, against our best practices for cost estimating. For our  reporting purposes, we collapsed 18 of our applicable best practices into  the four general characteristics of a reliable cost estimate:  comprehensive, well-documented, accurate, and credible. Figure 8  provides a summary of our assessment of the HPIB’s lifecycle cost  estimate.

Comprehensive. We found the HPIB cost estimate substantially met the  best practices for being comprehensive. For example, the estimate  includes government and contractor costs over the full lifecycle of all  three ships and contains sufficient levels of detail in the program’s work  breakdown structure—a hierarchical breakdown of the program into  specific efforts, including system engineering and ship construction. The  estimate also documents detailed ground rules and assumptions, such as  the learning curve used to capture expected labor efficiencies for follow- on ships. However, we found that the costs for disposal of the three ships  were not at a level of detail to ensure that all costs were considered and  not all assumptions, particularly regarding operating and support costs,  were varied to reflect the impact on cost should these assumptions  change.

Well-Documented. We also found the cost estimate substantially met the  best practices for being well-documented. Specifically, the cost estimate’s  documentation mostly captured the source data used as well as the  primary methods, calculations, results, rationales, and assumptions used  to generate each cost element. However, the documentation alone did  not provide enough information for someone unfamiliar with the cost  estimate to replicate what was done and arrive at the same results. For  example, NAVSEA officials discussed and showed us how historical data  from the analogous ships were used to create the estimate, but these  specific sources were not found in the cost estimate documentation.

Accurate. We found the estimate substantially met best practices for  being accurate. In particular, the estimate was properly adjusted for  inflation, and we did not find any mathematical errors in the estimate  calculations we inspected. Officials stated that labor and material cost  data from recent, analogous programs were used in the estimate. While  the documentation does not discuss the reliability, age, or relevance of  the cost data, NAVSEA officials provided us with additional information  regarding those data characteristics. Additionally, officials provided  documentation that demonstrated that they had updated the cost estimate  several times in the last 2 years.

Credible. We found the HPIB cost estimate partially met the best  practices associated with being credible. A credible cost estimate should  analyze the sensitivity of the program’s expected cost to changes among  key cost-driving assumptions and risks. It should also quantify the cost  impact of risks related to assumptions changing and variability in the  underlying data used to create the cost estimate. Credible cost estimates  should also be cross-checked internally and reconciled with an  independent cost estimate that is performed by an outside group. These  two best practices ensure that the estimate has been checked for any  potential bias. Our review of the HPIB cost estimate determined it  partially met the best practices for being credible due to the following:

Exclusions of major costs from sensitivity analysis and risk and  uncertainty analysis. The cost estimators conducted sensitivity  analysis as well as risk and uncertainty analysis on only a small  portion of the total lifecycle costs. For both the sensitivity analysis  and risk and uncertainty analysis, we found that NAVSEA only  modeled cost variation in the detail design and construction portion of  the program and excluded from its analyses any risk impacts related  to the remainder of the acquisition, operating and support, and  disposal phases, which altogether comprise about 75 percent of the  lifecycle cost. The cost estimate documents that the limited number of  active icebreakers and available data prevented NAVSEA from  identifying accurate risk bounds for the operating and support and  disposal phases. Further, NAVSEA officials told us because they used  historical data, including average maintenance costs from the Healy,  they felt that their estimate was reasonable. However, similar to how  NAVSEA consulted with the ship design team to establish high and  low-end costs using analogous ships, NAVSEA could have used cost  ranges in the historical data to develop risk bounds for the remaining  costs in the acquisition, operations and support, and disposal phases.  Without performing a sensitivity analysis on the entire life cycle cost of  the three ships, it is not possible for NAVSEA to identify key elements  affecting the overall cost estimate. Further, without performing a risk  and uncertainty analysis on the entire life cycle cost of the three ships,  it is not possible for NAVSEA to determine a level of confidence  associated with the overall cost estimate. By not quantifying important  risks, NAVSEA may have underestimated the range of possible costs  for about three-quarters of the entire program.

Lack of applied correlation in the risk and uncertainty analysis. In  its independent assessment of the HPIB cost estimate, the DHS Cost  Analysis Division similarly found that the results of the risk and  uncertainty analysis may understate the range of possible cost  outcomes for the HPIB. The DHS assessment noted that NAVSEA did  not use applied correlation—which links costs for related items so that  they rise and fall together during the analysis—in its cost model.  According to a joint agency handbook on cost risk and uncertainty,  applied correlation helps to ensure that cost estimates do not  understate the possible variation in total program costs. Omitting  applied correlation when assessing a cost estimate for risk can cause  an understated range of possible program costs and create a false  sense of confidence in the cost estimate. For example, absent applied  correlation, the DHS assessment noted that the Navy calculated with  a 99-percent level of confidence that the program will not exceed its  threshold (maximum acceptable) acquisition cost. Navy officials  explained that they will incorporate applied correlation in future  updates to the cost estimate when better data are available. However,  by applying correlation factors from the joint agency handbook to the  same data that NAVSEA used, DHS’s Cost Analysis Division  determined that NAVSEA overstated the likelihood of the program not  exceeding its threshold acquisition cost.

Cost estimate not fully reconciled with a comparable  independent cost estimate. While the Naval Center for Cost  Analysis performed an independent cost estimate of the HPIB  program, the office used a different methodology from NAVSEA’s, and  its estimate was based on an earlier version of the indicative ship  design and associated technical baseline. NAVSEA officials told us  that before the Coast Guard’s ship design team updated the indicative  ship design and technical baseline, NAVSEA met twice with Naval  Center for Cost Analysis to reconcile their results. However, NAVSEA  officials told us that due to the speed at which the program was  progressing, no reconciliation occurred after the ship design team  finalized the indicative ship design. While we did not find any specific  ground rules and assumptions that differed between the two  estimates, some ship characteristics had changed, such as the weight  estimates for propulsion and auxiliary systems, among others. The  use of two different technical baselines creates differences in the two  estimates and makes them less comparable to one another.

For additional details on our assessment of the HPIB’s cost estimate  against our 18 cost estimating best practices, see appendix III.

By excluding the majority of the HPIB program’s lifecycle costs from the  sensitivity analysis as well as the risk and uncertainty analysis, and  reconciling the estimate with an independent cost estimate based on a  different iteration of the ship design, the cost estimate does not provide a  fully credible range of costs the program may incur. Moreover, the  exclusion of applied correlation further provides a false sense of  confidence that the program will not exceed its threshold cost. As a result,  the estimate provides an overly optimistic assessment of the program’s  vulnerability to cost growth should risks be realized or current  assumptions change. This, in turn, may underestimate the lifecycle cost  of the program and calls into question the cost baselines DHS approved  and used to inform the HPIB’s budget request. Without a reliable cost  estimate to inform the business case for the HPIB prior to award of the  contract option for lead ship construction, Congress is at risk of  committing to a course of action without a complete understanding of the  program’s longer-term potential for cost growth.

section_title: Polar Icebreaker Program’s Optimistic Schedule Is Driven by Capability Gap and Does Not Reflect Robust Analysis
paragraphs: The Coast Guard set an optimistic schedule baseline for the HPIB based  on operational need, but its approach does not reflect a robust analysis of  what is realistic and feasible. According to DHS and Coast Guard  acquisition guidance, the goal of ADE 2A/2B is, among other things, to  ensure that the program’s schedule baseline is executable at an  acceptable cost. Rather than building a schedule based on  knowledge—including determining realistic schedule targets, analyzing  how much time to include in the schedule to buffer against potential  delays, and comprehensively assessing schedule risks—the Coast Guard  used the estimated end date of the Polar Star’s service life as the primary  driver to set the lead ship’s objective (or target) delivery date of  September 2023 and threshold (latest acceptable) delivery date of March  2024.
subsections: section_title: Analysis Conducted to Determine Lead Ship Construction Schedule Not Robust
paragraphs: The Coast Guard and the Navy did not conduct a robust analysis to  determine how realistic the 2.5- to 3-year construction cycle time is for the  lead HPIB before setting the schedule baseline. Our best practices for  developing project schedules state that, rather than meeting a particular  completion date, estimating how long an activity takes should be based  on the effort required to complete the activity and the resources  available. Doing so ensures that activity durations and completion dates  are realistic and supported by logic.

The Coast Guard and the Navy validated the reasonableness of the 2.5-  to 3-year construction time by comparing this duration to historical Navy  ship construction data. Program officials told us that they used 211 Navy  ships in their analysis and determined that the HPIB’s construction  schedule was within historical norms given its weight. However, program  officials told us they included both lead and follow-on ships in their  analysis. As we have found in our prior Navy shipbuilding work, schedule  delays tend to be amplified for lead ships in a class. Therefore, we  believe the program’s analysis for the lead ship was overly optimistic.

The Coast Guard also sought industry feedback to determine whether 2.5  to 3 years to build the lead HPIB was feasible. Design study information  provided to the Coast Guard by several shipbuilders estimated that they  would need between 2.5 to 3.5 years to build the lead ship. We  determined that the Coast Guard used the more optimistic estimate of 2.5  years for the objective delivery date and 3 years for the threshold delivery  date. Three years was also the time frame reflected in the request for  proposals for the detail design and construction contract. The request for  proposals lists December 2023 as the target delivery date for the lead  ship, which is approximately 3 years from the objective construction start  date.

Further, we compared the HPIB’s planned construction schedule to the  construction schedules of delivered lead ships for major Coast Guard and  Navy shipbuilding programs active in the last 10 years as well as the  Healy. We found that the HPIB’s lead ship construction cycle time of 2.5  to 3 years is optimistic, as only three of the ten ships in our analysis were  constructed in 3 years or less. For the purposes of our analysis, we  included information on each ship’s weight and classification, both of which can affect complexity and, therefore, construction times (see figure  9).

The Coast Guard also did not conduct any analysis to identify a  reasonable amount of margin to include in the program schedule baseline  to account for any delays. Estimating and documenting schedule margin  based on an analysis of schedule risks helps to ensure that a program’s  baseline schedule is achievable despite delays that may unexpectedly  arise. Program officials told us that the only margin included in the HPIB  schedule is the 6 months between the objective and threshold dates—the  maximum time between objective and threshold dates before DHS policy  requires additional rationale and justification. According to the request  for proposals, the winning shipbuilder will examine schedule risks while  preparing an integrated schedule. In addition, Coast Guard officials told  us that the current schedule will remain largely notional until the winning  shipbuilder provides detailed updates to the schedule.

Delays in project schedules, whether they are in the program’s control or  not, should be expected. For example, in prior shipbuilding programs we  have reviewed, we have found that delays have resulted from a number  of issues, including redesign work to address issues discovered during  pre-delivery testing, key system integration problems, and design quality  issues. Delays outside of the program’s control such as funding  instability, late material delivery, and bid protests have previously affected  a program’s ability to meet schedule. Program officials told us these and  other schedule risks are not accounted for in the HPIB schedule.

Further, our analysis of 12 selected shipbuilding acquisition programs  active in the last 10 years shows that the Navy and the Coast Guard have  delayed delivery of all but one lead ship from their original planned  delivery dates by more than 6 months, with delays occurring both before  and after the start of construction. The delays in lead ship deliveries  ranged from 9 months to 75 months. For the purposes of our analysis, we  included the lead ships of major Coast Guard and Navy shipbuilding  programs that have been active from 2008 to 2018. We excluded the  Navy submarines and aircraft carriers from our analysis because we  determined that their size and complexity did not make them reasonable  comparisons to the HPIB (see figure 10).

By supporting the lead ship construction time with overly optimistic  analysis and by not conducting analysis to estimate a reasonable amount  of margin, the Coast Guard’s HPIB schedule does not fully account for  likely or unforeseen delays, which would help ensure that the planned  delivery date for the lead ship is feasible.

section_title: Schedule Risks after Construction Start Not Identified
paragraphs: The Coast Guard has set the HPIB’s schedule baselines, including when  all three ships are planned to achieve full operational capability, but has  not yet identified risks for the program’s schedule that could occur after  the start of lead ship construction, such as risks related to the  construction schedule or concurrency between ship testing and  construction of subsequent ships. According to the HPIB risk  management plan, the program should formally track risks, which  includes developing risk mitigation plans and reporting risks to DHS. Prior  to setting its baselines, the Coast Guard formally tracked some schedule  risks that affect the program’s ability to start construction on time, such as  an aggressive schedule for releasing the request for proposals for the  detail design and construction contract. IPO officials told us they retired  that risk because the Navy released the request for proposals in March  2018. However, our analysis of the HPIB construction schedule and 6- month margin for delays found the program’s schedule was optimistic,  thereby warranting additional risk tracking and management.

The DHS Office of Systems Engineering also identified and  recommended the Coast Guard track and take steps to mitigate HPIB’s  schedule risks, including those related to concurrency. In its technical  assessment, this office noted that the program plans to deliver the first  two ships prior to completing initial operational testing and evaluation for  the lead ship. The assessment further noted that construction on the third  ship is planned to be nearly three-quarters finished prior to completing  initial operational testing and evaluation. DHS’s Office of Systems  Engineering found that this concurrency creates cost, schedule, and  technical risk resulting from rework that may be necessary to address  deficiencies found during initial testing. By not comprehensively and  formally tracking risks to the HPIB schedule that occur after the start of  lead ship construction, the program may not sufficiently identify and take  timely risk management actions to address this key phase in the  acquisition.

By not conducting a robust analysis to inform whether the HPIB’s  schedule baselines are feasible, the Coast Guard is not providing  Congress with realistic dates of when the ships may be delivered before  requesting funding for the construction of the lead ship. While the Coast  Guard is planning a service life extension of the Polar Star starting in  2020, as noted above, the HPIB’s optimistic schedule may put the Polar  Star at risk of needing to operate longer than planned. The HPIB  schedule’s optimism also puts the Coast Guard at risk of not fully  implementing a knowledge-based acquisition approach to meet its  aggressive schedule goals. Our prior work on shipbuilding programs has  shown that establishing optimistic program schedules based on  insufficient knowledge can create pressure for programs to make  sacrifices elsewhere. For example, we found that the Navy moved  forward with construction with incomplete designs and when key  equipment was not available when needed. Additionally, some Navy  programs pushed technology development into the design phase or  pushed design into the construction phase. These concurrencies often  result in costly rework to accommodate changes to the design, further  delays, or lower than promised levels of capability.

section_title: Polar Icebreaker Program’s Anticipated Contract May Be Funded by Both the Coast Guard and the Navy, but They Have Not Fully Documented Responsibility for Addressing Cost Growth
paragraphs: According to the IPO, the HPIB’s anticipated detail design and  construction contract may be funded by both Coast Guard and Navy  appropriations, but how certain types of cost growth will be addressed  between the Coast Guard and the Navy has not been fully documented.  The HPIB’s acquisition strategy anticipates award of a contract that will  have options, includes efforts aimed at mitigating cost risks, and  acknowledges the use of foreign suppliers to provide components and  design services as allowable under statute and regulation. Since 2013,  the program has received $360 million in funding, which includes both  Coast Guard and Navy appropriations. Moving forward, it is unclear how  much Coast Guard and Navy funding will be used to fund the contract.  The Coast Guard and the Navy have an agreement in place for funding  issues, but the agreement does not fully address how they plan to  address cost growth on the program.
subsections: section_title: Acquisition Strategy Anticipates Use of Contract Options, Ways to Mitigate Cost Risks, and Foreign Suppliers
paragraphs: As part of the HPIB’s acquisition strategy, the Navy structured the detail  design and construction of each of the ships as contract options in the  March 2018 request for proposals. Specifically, the request for proposals  structured the detail design and construction work into four distinct  contract line items, all under a fixed-price incentive (firm-target) contract  type. Generally, this contract type allows the government and shipbuilder  to share cost savings and risk through a specified profit adjustment  formula, also known as a share ratio; ties the shipbuilder’s ability to earn  a profit to performance by decreasing the shipbuilder’s profit after costs  reach the agreed upon target cost; and, subject to other contract terms,  fixes the government’s maximum obligation to pay at a ceiling price.  Table 5 provides information on the HPIB’s request for proposals as of  May 2018.

According to the request for proposals, in addition to potentially earning  profit by controlling costs, the shipbuilder may also earn up to $34 million  in incentives for achieving other programs goals, such as quality early  delivery, reducing operations and sustainment costs, and production  readiness. IPO officials stated that they based the incentives on prior  Navy shipbuilding contract examples. However, in March 2017, we found  that the Navy had not assessed the effectiveness of added incentives for  the reviewed fixed-price incentive contracts in terms of improved contract  outcomes across the applicable shipbuilding portfolio. As a result, we  recommended that DOD direct the Navy to conduct a portfolio-wide  assessment of the Navy’s use of additional incentives on fixed-price  incentive contracts across its shipbuilding programs. DOD concurred with  this recommendation, but the Navy has not yet taken steps to implement  it.

As part of the HPIB acquisition strategy, the IPO is striving to control  costs on the detail design and construction contract through the following:

A fixed-price incentive (firm-target) contract type. Because the  shipbuilder’s profit is linked to performance, fixed-price incentive  contracts provide an incentive for the shipbuilder to control cost. Most  of the Navy’s proposed share ratios and ceiling prices for the detail  design and construction work are consistent with DOD’s November  2010 Better Buying Power memo, which states a 50/50 share ratio  and 120 percent ceiling price should be the norm, or starting point, for  fixed-price incentive contracts.

Full and open competition. The Navy plans to competitively award  the HPIB’s detail design and construction contract. From market  research and industry engagement, the IPO determined that there  were multiple viable competitors. In March 2017, we found that  competition helped to strengthen the Navy’s negotiating position with  shipbuilders when setting contract terms, such as the share line and  ceiling price for fixed-price incentive type contracts.

Providing offerors the government’s estimated ship costs. The  request for proposals does not set affordability caps but does include  information on the government’s estimated cost for the ships,  including $746 million for the lead ship’s advance planning,  engineering, detail design, and construction, and an average ship  price of $615 million across all three ships. Navy contracting officials  explained that offers will not be disqualified from the source selection  solely for being higher than the estimated costs. Instead, the  estimated costs provide the offerors with cost bounds to help  appropriately scope the capabilities. For example, IPO officials stated  that they are striving to appropriately size the integrated power plant  so that it is generating sufficient power to meet key performance  parameters but not so much power that it drives up the cost.

Inquiries on block buys and economic order of quantity  purchases. The Navy gave offerors an opportunity to provide the  estimated savings that the government could achieve if it were to take  a “block buy” approach in purchasing the ships or purchasing supplies  in economic order quantities. The Navy did not include a definition of  “block buy” in the HPIB request for proposals synopsis. Based on our  prior work, block buy contracting generally refers to special legislative  authority that agencies seek on an acquisition-by-acquisition basis to  purchase more than one year’s worth of requirements. The request  for proposals synopsis stated a preference for submission of the  estimated savings within 60 days of the release of the request for  proposals, or by May 2018. As of June 2018, the Navy had not  received any formal responses from industry on potential savings from  block buys or economic order quantities. For the HPIB request for  proposals, the Navy stated that any information on block buys or  economic order of quantities would be optional and would not be used  as part of the evaluation of proposals submitted by offerors.

Our prior work on block buy contracting approaches for the Littoral  Combat Ship and F-35 Joint Strike Fighter programs found that the  terms and conditions of the contracts affect the extent to which the  government achieves savings under a block buy approach. For  example, the Littoral Combat Ship’s block buy contracts indicated that  a failure to fully fund the purchase of a ship in a given year would  make the contract subject to renegotiation. DOD has pointed to this as  a risk that the contractors would demand higher prices if DOD  deviated from the agreed to block buy plan.

In its HPIB acquisition strategy, the IPO has also considered the use of  foreign suppliers as allowable under the law. According to the February  2018 HPIB acquisition plan, the HPIB must be constructed in a U.S.  shipyard given statutory restrictions, including restrictions on construction  of Coast Guard vessels and major components in foreign shipyards  unless authorized by the President. However, foreign suppliers will be  permitted to provide components and design services to the extent  applicable statutes and regulations allow. According to Coast Guard  officials, foreign design firms have extensive expertise and knowledge to  produce the design for HPIBs. As a result, the U.S. shipbuilders planning  to submit proposals on the HPIB solicitation may partner with these  foreign design firms when submitting proposals. Similarly, Coast Guard  officials stated that the azimuthing propulsors that have the necessary  power and ice classification for the HPIB are manufactured by foreign  companies. Therefore, the selected shipbuilder may subcontract with  these companies to acquire the propulsors.

In addition, Navy contracting officials stated that the program did not need  to obtain a waiver from the Buy American Act—which generally requires  federal agencies to purchase domestic end products when supplies are  acquired for use in the United States, and use domestic construction  materials on contracts performed in the United States—for certain  components. The Act includes exceptions, such as when the domestic  end products or construction materials are unavailable in sufficient and  reasonably available commercial quantities and of a satisfactory quality.

section_title: Program Has Received Both Coast Guard and Navy Funds, but Unclear How Program Will Be Funded Moving Forward
paragraphs: From 2013 through 2018, the program has received $360 million in  funding—$60 million in the Coast Guard’s Acquisition, Construction, and  Improvement appropriations (hereafter referred to as Coast Guard  funding) and $300 million in Navy’s Shipbuilding and Conversion, Navy  advance procurement appropriations (hereafter referred to as Navy  appropriations). In addition, according to Coast Guard officials, in fiscal  year 2017, Coast Guard reprogrammed $30 million in fiscal year 2016  appropriations for the HPIB from another program (see figure 11).

According to IPO and Navy contracting officials, the Navy plans to use  $270 million of the $300 million in Navy appropriations to award the detail  design and construction contract in fiscal year 2019, which would fund the  advanced engineering, long lead time materials, and detail design work.  Navy officials stated the remaining $30 million in Navy appropriations will  be held in reserves for potential scope changes. Of the $60 million in  Coast Guard funding, the IPO has used $41 million for program office  costs and the February 2017 design study contracts, and plans to use the  remaining $19 million for program office costs. Coast Guard officials  stated that they used the $30 million in reprogrammed 2016  appropriations to fund the design studies, model testing, and Navy  warfare center support.

As the program prepares to award a contract worth billions of dollars if all  the options are exercised, Congress, the Coast Guard, and the Navy face  key funding considerations. These include the extent to which the  program will be funded using Coast Guard and Navy appropriations in the  future and whether each of the ships will be fully or incrementally  funded. Navy contracting officials stated that by structuring the  contract’s construction work as options, the contract has flexibility to  accommodate any type of additional funding the program may receive.

The National Defense Authorization Act for Fiscal Year 2018 authorized  procurement of one Coast Guard heavy polar icebreaker vessel. The  Navy did not request any funding in fiscal year 2019 for the HPIB, while  Coast Guard requested $30 million. Subsequently, after discretionary  budget caps were relaxed by Congress, the Administration’s fiscal year  2019 budget addendum requested an additional $720 million in fiscal year  2019 Coast Guard appropriations for the program. Although the Navy  did not request fiscal year 2019 funding for the lead ship, and Navy  officials told us they have no plans to budget for the HPIB program  moving forward, Congress may still choose to appropriate funds for the  HPIB to the Navy. For example, in fiscal years 2017 and 2018, the Navy  did not request funding but received $150 million in appropriations each  year for the HPIB (see figure 12).

Additionally, the Coast Guard has been expressly authorized to use  incremental funding for the HPIB. This authorization is reflected in the  Coast Guard’s January 2018 affordability certification memo, submitted to  DHS leadership. These memos are required to certify that a program’s  funding levels are adequate and identify tradeoffs needed to address any  funding gaps. However, as noted above, with the addition of the  Administration’s fiscal year 2019 budget request addendum, the Coast  Guard requested $750 million in full funding for the lead ship. The Navy  has informed us that it plans to award the advance planning, design,  engineering, long lead time material contract line item with its $270 million  in appropriations. Navy officials also told us they are in the process of  determining whether it needs to be authorized by Congress to use an  incremental funding approach to fund the detail design and construction  options if full funding is not received by the Navy.

According to the Office of Management and Budget’s A-11 budget  circular, full funding helps to ensure that all costs and benefits of an  acquisition are fully taken into account at the time decisions are made to  provide resources. The circular goes on to say that when full funding is  not followed, without certainty if or when future funding will be available,  the result is sometimes poor planning, higher acquisition costs,  cancellation of major investments, or the loss of sunk costs. The circular,  however, also notes that Congress may change the agency’s request for  full funding to incremental funding to accommodate more projects in a  year than would be allowed with full funding.

section_title: Plans to Address Cost Growth Not Fully Documented
paragraphs: Regardless of the funding strategy and which service funds the contract,  the Coast Guard and the Navy do not have a clear agreement on how  certain types of cost growth within the program will be addressed. The  budgeting and financial management appendix of the July 2017  agreement between the Coast Guard and Navy for the HPIB notes that  any cost overruns will be funded by the originating source of the  appropriation and be the responsibility of the organization that receives  the funding. However, the Coast Guard and the Navy have interpreted  “cost overruns” differently in the context of the agreement.

Coast Guard and Navy officials are in agreement that given the fixed- price incentive contract type, the government’s share of cost overruns  between the target cost and ceiling price (based on the share ratio) will be  the responsibility of the organization that provided the funding for the  contract line item. Navy officials also noted that because the contract type  is fixed-price incentive, any cost overruns above the ceiling price are  generally the responsibility of the contractor, not the government.

However, the Coast Guard and the Navy have not addressed in an  agreement how they plan to handle any cost growth stemming from  changes to the scope, terms, and conditions of the HPIB detail design  and construction contract. For example, if the Coast Guard or the Navy  revises the program’s requirements, this could increase the scope and  value of the contract and result in additional contract costs. It is unclear in  this instance, which organization is responsible for paying for the  additional costs. Further, our 2005 work on Navy shipbuilding programs  found that the most common causes of cost growth in these programs  were related to design modifications, the need for additional and more  costly materials, and changes in employee pay and benefits, some of  which required changes in contract scope.

IPO officials told us that unplanned changes to the program’s scope and  any corresponding funding requests for unanticipated cost growth would  require discussions and agreements with both Coast Guard and Navy  leadership. Coast Guard and Navy officials stated that they are in the  process of reviewing the July 2017 budget appendix of the agreement to  clarify the definition of cost overruns and plan to finalize revisions no later  than September 2018. Our prior work on implementing interagency  collaborative mechanisms found that agencies that articulate their  agreements in formal documents can strengthen their commitment to  working collaboratively, which can help better overcome significant  differences when they arise. Different interpretations or disagreements  on financial responsibility between the Coast Guard and the Navy on cost  growth for the HPIB program could result in funding instability for the  program, which could affect the program’s ability to meet its cost and  schedule goals.

section_title: Conclusions
paragraphs: In the last several years, the Coast Guard and the Navy have made  significant strides in their efforts to acquire heavy polar icebreakers. It has  been over 40 years since the United States has recapitalized its aging  heavy polar icebreaker fleet, and Congress has expressed the need for  investment in the HPIB program to help ensure our continued presence in  the polar regions. The Coast Guard and the Navy have taken steps to  examine design risks and expressed commitment to design maturity  before starting construction on the lead ship.

However, the Coast Guard and the Navy did not take key steps to reduce  risks on the HPIB program before setting the HPIB’s program baselines— namely, conducting a preliminary design review, conducting a technology  readiness assessment, developing a fully reliable cost estimate, and  conducting analysis to determine a realistic schedule and risks to that  schedule. By setting the program’s baselines prior to obtaining sufficient  knowledge in the design, technologies, cost, and schedule of the HPIB,  DHS, the Coast Guard, and the Navy are not establishing a sound  business case for investing in the HPIB nor putting the program in a  position to succeed. There is risk that the program will cost more than the  planned $9.8 billion and the lead ship will not be delivered by 2023 as  planned. Further, without clear agreement between the Coast Guard and  the Navy on which service will be responsible for any cost growth on the  HPIB, the program is at further risk of not meeting its ambitious goals. In  the current budget environment, it is imperative that the Coast Guard and  the Navy obtain sufficient acquisition knowledge and put together a sound  business case before asking Congress and taxpayers to commit  significant resources to the HPIB program.

section_title: Recommendations for Executive Action
paragraphs: We are making six recommendations total to the Coast Guard, DHS, and  the Navy:

The Commandant of the Coast Guard should direct the polar  icebreaker program to conduct a technology readiness assessment in  accordance with best practices for evaluating technology readiness,  identify critical technologies, and develop a plan to mature any  technologies not designated to be at least TRL 7 before detail design  of the lead ship begins. (Recommendation 1)

The Commandant of the Coast Guard, in collaboration with the  Secretary of the Navy, should direct the polar icebreaker program and  NAVSEA 05C to update the HPIB cost estimate in accordance with  best practices for cost estimation, including (1) developing risk bounds  for all phases of the program lifecycle, and on the basis of these risk  bounds, conduct risk and uncertainty analysis, as well as sensitivity  analysis, on all phases of the program lifecycle, and (2) reconciling  the results with an updated independent cost estimate based on the  same technical baseline before the option for construction of the lead  ship is awarded. (Recommendation 2)

The Commandant of the Coast Guard should direct the polar  icebreaker program office to develop a program schedule in  accordance with best practices for project schedules, including  determining realistic durations of all shipbuilding activities and  identifying and including a reasonable amount of margin in the  schedule, to set realistic schedule goals for all three ships before the  option for construction of the lead ship is awarded. (Recommendation  3)

The Commandant of the Coast Guard should direct the polar  icebreaker program office to analyze and determine appropriate  schedule risks that could affect the program after construction of the  lead ship begins to be included in its risk management plan and  develop appropriate risk mitigation strategies. (Recommendation 4)

The DHS Under Secretary for Management should require the Coast  Guard to update the HPIB acquisition program baselines prior to  authorizing lead ship construction, after completion of the preliminary  design review, and after it has gained the requisite knowledge on its  technologies, cost, and schedule, as recommended above.  (Recommendation 5)

The Commandant of the Coast Guard, in collaboration with the  Secretary of the Navy, should update the financial management and  budget execution appendix of the memorandum of agreement  between the Coast Guard and the Navy to clarify and document  agreement on how all cost growth on the HPIB program, including  changes in scope, will be addressed between the Coast Guard and  the Navy. (Recommendation 6)

section_title: Agency Comments
paragraphs: We provided a draft of this report to DHS and DOD for review and  comment. In its comments, reproduced in appendix IV, DHS concurred  with all six of our recommendations and identified actions it planned to  take to address them. The Navy stated that it deferred to DHS and the  Coast Guard on responding to our recommendations. DHS, the Coast  Guard, and the Navy also provided technical comments, which we  incorporated as appropriate.

We are sending copies of this report to the appropriate congressional  committees, the Secretary of Defense, the Secretary of Homeland  Security, the Commandant of the Coast Guard, the Secretary of the Navy,  and other interested parties. In addition, the report is available at no  charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or makm@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to the report  are listed in appendix V.

section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report examines (1) the extent to which the heavy polar icebreaker  (HPIB) program has taken steps to develop mature designs and  technologies consistent with best practices, (2) the extent to which the  HPIB program has taken steps to set realistic cost and schedule  estimates, and (3) the status of the HPIB program’s contracting efforts  and funding considerations.

To assess the extent to which the HPIB program has taken steps to  develop mature designs and technologies consistent with GAO-identified  best practices, we reviewed program performance and design  requirements, including the program’s operational requirements  documents, system specifications such as the power plant, propulsion  system, and hull, and technical baseline; the program’s alternatives  analysis study, tailored systems engineering plan, test and evaluation  master plan, and model testing results; cooperative agreements with  Canada related to the HPIB; excerpts from industry studies; and the  March 2018 detail design and construction request for proposals and  subsequent amendments. We also reviewed relevant Department of  Homeland Security (DHS), Coast Guard, and Department of Defense  (DOD) acquisition guidance and instructions. From these documents, we  determined the program’s design and technology efforts and compared  them to GAO’s various best practices, including using a knowledge-based  approach to shipbuilding, knowledge-based approach to major  acquisitions, and evaluating technology readiness. We also interviewed  knowledgeable officials from the Coast Guard’s Capabilities Directorate,  Research and Development Center, and Marine Transportation Systems  Directorate; DHS’s Science and Technology Directorate’s Office of  Systems Engineering; the Canadian Coast Guard; and the National  Science Foundation.

To assess the extent to which the HPIB program has taken steps to set  realistic cost and schedule estimates, we determined the extent to which  the estimates were consistent with best practices as identified in GAO’s  Cost Estimating and Assessment and Schedule Assessment guides. To  assess the cost estimate, we reviewed the HPIB’s January 2018 lifecycle  cost estimate used to support the program’s initial cost baselines, Coast  Guard and Navy documentation supporting the estimate, relevant  program briefs to Coast Guard leadership, and HPIB program  documentation containing cost, schedule, and risk information. We met  with Naval Sea Systems Command (NAVSEA) officials responsible for  developing the cost estimate to understand the processes used by the  cost estimators, clarify information, and request additional documentation  to support the estimate. Because we did not have direct access to the  HPIB cost model, we observed portions of the model during a  presentation and discussion with Navy cost estimators. We also reviewed  the Naval Center for Cost Analysis’ September 2017 independent cost  estimate for the HPIB program, the DHS Cost Analysis Division’s January  2018 independent cost assessment of the HPIB lifecycle cost estimate,  and DHS Office of Systems Engineering’s January 2018 technical  assessment of the HPIB program. We also conducted interviews with  officials from the Naval Center for Cost Analysis, DHS Cost Analysis  Division, and the DHS Office of Systems Engineering.

To assess the program’s schedule, we compared the HPIB program’s  schedule, including the program’s initial schedule baselines, delivery  schedules from the HPIB’s request for proposals for the detail design and  construction contract, and integrated master schedule, to selected GAO  best practices for project schedules, including establishing the duration of  activities, ensuring reasonable total buffer or margin, and conducting a  schedule risk analysis. To specifically assess the HPIB lead ship’s 3- year construction schedule estimate, we reviewed the Coast Guard’s and  the Navy’s analysis supporting the HPIB schedule. We did not assess the  reliability of the historical ship construction data the Coast Guard and  Navy used for this analysis. We also compared the HPIB lead ship’s 3- year construction schedule to historical construction cycle times of lead  ships among a nongeneralizable sample of major Navy and Coast Guard shipbuilding programs. We selected programs that were active within the  last 10 years and have completed construction of the lead ship. We also  included the Coast Guard’s Healy medium polar icebreaker, even though  it is not a recent shipbuilding program, because it is the most recent polar  icebreaker to be built in the United States. We excluded the Coast Guard  Fast Response Cutter, Navy submarines, and Navy aircraft carriers  because we determined that their size and complexity did not make them  reasonable comparisons to the HPIB for construction times. This resulted  in an analysis of construction schedules for 10 shipbuilding programs. We  obtained data on these programs’ construction schedules from program  documentation, such as acquisition program baselines, Navy selected  acquisition reports, and Navy and Coast Guard budget documentation.  We selected only lead ships for comparison because we have found in  our prior work that schedule delays are amplified for lead ships in a class.  Lead ships are thus more comparable to the HPIB lead ship than follow- on ships. We reviewed ship displacement data from the Naval Vessel  Registry and the Coast Guard to control for the size of the ships. To  assess the reliability of Naval Vessel Registry data, we reviewed the  Navy’s data collection and database maintenance documentation, cross- checked select data across Navy websites, and interviewed cognizant  Navy officials regarding internal controls for the database. We determined  the ship displacement data were reliable for our purposes. To assess the  degree to which the 6-month schedule margin that the HPIB baseline  affords the lead ship is in keeping with historical ship delivery delays, we  reviewed Coast Guard, Navy, and DHS acquisition documentation from a  nongeneralizable sample of major Navy and Coast Guard shipbuilding  programs. We selected programs active within the last 10 years and  analyzed changes in lead ship delivery dates. We excluded Navy  submarines and aircraft carriers because we determined that their size  and complexity did not make them reasonable comparisons to the HPIB  for delivery delays. We included programs that have not yet delivered  their lead ships. This resulted in an analysis of construction schedules for  12 shipbuilding programs. For delivered ships, we used the actual  delivery date; for ships not yet delivered, such as the Offshore Patrol  Cutter and DDG 1000, we used the most recent, planned delivery date in  the program baseline.

To determine the status of the HPIB program’s contracting efforts and  funding considerations, we reviewed the program’s acquisition plan,  March 2018 request for proposals and subsequent amendments,  certification of funds memorandum, budget justifications, lifecycle cost  estimate, and the Coast Guard’s fiscal year 2019 Capital Investment  Plan. We also interviewed knowledgeable officials from the Coast Guard’s  Office of Budget and Programs, NAVSEA Contracts Directorate, NAVSEA  Comptroller Directorate, and the Office of the Assistant Secretary of  Navy’s Financial Management and Comptroller.

For all objectives, we reviewed relevant DHS and Coast Guard policies  and interviewed knowledgeable officials from DHS, the Coast Guard’s  and the Navy’s HPIB integrated program office, and ship design team.

We conducted this performance audit from August 2017 to September  2018 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

section_title: Appendix II: Shipbuilding Phases
paragraphs: There are four primary phases in shipbuilding: pre-contracting activities  and contract award, detail design and planning, construction, and post- delivery activities (see table 6).

section_title: Appendix III: Summary of Results of Heavy Polar Icebreaker Program’s Cost Estimate Assessed against GAO’s Best Practices
paragraphs: The GAO Cost Estimating and Assessment Guide (GAO-09-3SP) was  used as criteria in this analysis. Using this guide, GAO cost experts  assessed the heavy polar icebreaker (HPIB) program’s lifecycle cost  estimate against measures consistently applied by cost-estimating  organizations throughout the federal government and industry that are  considered best practices for developing reliable cost estimates. For our  reporting purposes, we grouped these best practices into four  categories—or characteristics—associated with a reliable cost estimate:  comprehensive, accurate, well documented, and credible. A cost estimate  is considered reliable if the overall assessment ratings for each of the four  characteristics are substantially or fully met. If any of the characteristics  are not met, minimally met, or partially met, then the cost estimate does  not fully reflect the characteristics of a high-quality estimate and cannot  be considered reliable. After reviewing documentation the Navy submitted  for its cost estimate, conducting interviews with the Navy’s cost  estimators, and reviewing other relevant HPIB cost documents, we found  the HPIB lifecycle cost estimate substantially met three and partially met  one characteristic of reliable cost estimates.

We determined the overall assessment rating by assigning each  individual rating a number: Not Met = 1, Minimally Met = 2, Partially Met =  3, Substantially Met = 4, and Met = 5. Then, we took the average of the  individual assessment ratings to determine the overall rating for each of  the four characteristics. The resulting average becomes the Overall  Assessment as follows: Not Met = 1.0 to 1.4, Minimally Met = 1.5 to 2.4,  Partially Met = 2.5 to 3.4, Substantially Met = 3.5 to 4.4, and Met = 4.5 to  5.0. See table 7 for a high level summary of each best practice and the  reasons for the overall scoring.

section_title: Appendix IV: Comments from the Department of Homeland Security

section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition the contact named above, the following staff members made  key contributions to this report: Rick Cederholm (Assistant Director),  Claire Li (Analyst-in-Charge), Peter Anderson, Brian Bothwell, Juaná  Collymore, Laurier Fish, Kristine Hassinger, Karen Richey, Miranda  Riemer, Roxanna Sun, David Wishard, and Samuel Woo.

Fast facts: The Coast Guard and Navy are planning to invest up to $9.8 billion to build and maintain three heavy polar icebreakers to conduct missions in the Antarctic and Arctic. The first ship is scheduled for delivery in 2023.

We found the icebreaker program faces risks in 3 key areas:

the program has not fully assessed how well key technologies will work in this particular effort,

its cost estimate may underestimate the funding needed, and

it set ship delivery dates that are optimistic and not based on a realistic shipbuilding assessment.

We made six recommendations to address these and other risks.

Question-summary annotations: questions: question: What initial mistakes did the Coast Guard make?
answer: The Coast Guard set program baselines before conducting a preliminary design review, which puts the program at risk of having an unstable design, thereby increasing the program's cost and schedule risks.
child_questions: question: In what way does this practice fit with DHS's current acquisition policy and acquisition best practices?
answer: While setting baselines without a preliminary design review is consistent with DHS's current acquisition policy, it is inconsistent with acquisition best practices.

question: What were the implications of these mistakes on DHS policy?
answer: Based on GAO's prior recommendation, DHS is currently evaluating its policy to better align technical reviews and acquisition decisions.

questions: question: In the Coast Guard, how have technology readiness assessments influenced baselines?
answer: The Coast Guard intends to use proven technologies for the program, but did not conduct a technology readiness assessment to determine the maturity of key technologies prior to setting baselines.
child_questions: question: Why did Coast Guard officials not feel it was necessary to conduct a technology readiness assessment?
answer: Coast Guard officials indicated such an assessment was not necessary because the technologies the program plans to employ have been proven on other icebreaker ships.

question: Were the Coast Guard officials correct in their assumption that this assessment was unnecessary?
answer: However, according to best practices, such technologies can still pose risks when applied to a different program or operational environment, as in this case. Without such an assessment, the program's technical risk is underrepresented.

questions: question: Did the financial estimate fully meet GAO's standards and best practices?
answer: The lifecycle cost estimate that informed the program's $9.8 billion cost baseline substantially met GAO's best practices for being comprehensive, well-documented, and accurate, but only partially met best practices for being credible.

question: What was lacking in the cost estimate?
answer: The cost estimate did not quantify the range of possible costs over the entire life of the program.
child_questions: question: What are the implications of the flaws in the cost estimate?
answer: As a result, the cost estimate was not fully reliable and may underestimate the total funding needed for the program.

questions: question: Why are the Coast Guard and Navy collaborating?
answer: To maintain heavy polar icebreaking capability, the Coast Guard and the Navy are collaborating to acquire up to three new heavy polar icebreakers through an integrated program office.

question: What does GAO recommend for successful acquisition programs?
answer: GAO has found that before committing resources, successful acquisition programs begin with sound business cases, which include plans for a stable design, mature technologies, a reliable cost estimate, and a realistic schedule.

questions: question: What important information was included in Section 122?
answer: Section 122 of the National Defense Authorization Act for Fiscal Year 2018 included a provision for GAO to assess issues related to the acquisition of the icebreaker vessels.

question: What were the most important objectives of the report?
answer: This report examines, among other objectives, the extent to which the program is facing risks to achieving its goals, particularly in the areas of design maturity, technology readiness, cost, and schedule.

question: How did GAO conduct its review?
answer: GAO reviewed Coast Guard and Navy program documents, analyzed Coast Guard and Navy data, and interviewed knowledgeable officials.