text: Background History of the Polar Icebreakers and Icebreaking Capability Gap The Coast Guard has been responsible for carrying out the nation’s polar icebreaking missions since 1965—when it assumed primary responsibility for the nation’s polar icebreaking fleet. The Coast Guard’s responsibilities are outlined in various statutes, policies, and interagency agreements. A 2010 Coast Guard study identified gaps in the Coast Guard’s ability to support and conduct missions in the Arctic and Antarctic. As a result, in June 2013, the Coast Guard established the need for up to three heavy polar icebreakers and three medium icebreakers to adequately meet these Coast Guard mission demands. More recently, in November 2017, Coast Guard officials reiterated that they will be able to fulfill all mission requirements—which include support to agencies with Arctic responsibilities such as DOD, the National Science Foundation (NSF), Department of State, National Oceanic and Atmospheric Administration, and National Aeronautics and Space Administration—with a fleet of three heavy and three medium polar icebreakers. Coast Guard officials told us they are not currently assessing acquisition of the medium polar icebreakers because they are focusing on the HPIB acquisition and plan to assess the costs and benefits of acquiring medium polar icebreakers at a later time. The Coast Guard currently has two active polar icebreakers in its fleet— the Polar Star, a heavy icebreaker, and the Healy, a medium icebreaker. An additional Coast Guard heavy icebreaker, the Polar Sea, has been inactive since 2010 when it experienced a catastrophic engine failure. Commissioned in 1976, the Polar Star is the world’s most powerful active non-nuclear icebreaker. The less powerful Healy primarily supports Arctic research. Although the Healy is capable of carrying out a wide range of activities, it cannot operate independently in the ice conditions in the Antarctic or ensure timely access to some Arctic areas in the winter. See figure 1 for the Coast Guard’s active icebreakers. The Coast Guard has faced challenges in meeting the government’s icebreaking needs in recent years. For example, in June 2016, we found that when neither the Polar Sea nor the Polar Star was active in 2011 and 2012, the Coast Guard did not maintain assured, year-round access to both the Arctic and Antarctic, as the Healy cannot reach ice-covered areas with more than 4½ feet of ice. According to a January 2017 Coast Guard assessment, the Coast Guard does not plan to recommission the Polar Sea because it would not be cost-effective. Polar Star Sustainment Efforts According to Coast Guard planning documents, the Polar Star’s service life is estimated to end between fiscal years 2020 and 2023. This creates a potential heavy polar icebreaker capability gap of about 3 years, assuming the Polar Star’s service life ends in 2020 and the lead HPIB is delivered by the end of fiscal year 2023 as planned. If the lead ship is delivered later than planned in this scenario, the potential gap could be more than 3 years. As a result, according to a 2017 polar icebreaking bridging strategy, the Coast Guard is planning to recapitalize the Polar Star’s key systems starting in 2020 to extend the service life of the ship until the planned delivery of the second HPIB (see figure 2). In September 2017, we found that the Coast Guard’s $75 million cost estimate for the Polar Star life extension project may be unrealistic, in part because it was based on the assumption of continuing to use parts from the decommissioned Polar Sea, as has been done in previous maintenance events. Because of the finite number of parts available from the Polar Sea, the Coast Guard may have to acquire new parts for the Polar Star that could increase the $75 million estimate. As a result, we recommended that the Coast Guard complete a comprehensive cost estimate and follow cost estimating best practices before committing to the life extension project. The Coast Guard concurred with this recommendation. As of May 2018, Coast Guard officials told us they were still conducting ship engineering inspections on the Polar Star to determine the details of the work needed for the limited service life extension, which will then inform the development of a cost estimate. In January 2018, the Coast Guard completed its ship structures and machinery evaluation board report. Coast Guard officials told us that this report will help to determine the details of the work needed for the limited life extension. The January 2018 report estimated the remaining service life of the Polar Star as 5 years or less. In April 2018, the Coast Guard approved the Polar Star life extension project to establish requirements and evaluate the feasibility of alternatives that will achieve the requirements. Coast Guard officials stated they completed a notional cost estimate in April 2018 and plan to complete a detailed formal cost estimate by June 2020. Coast Guard’s and Navy’s Roles in the Heavy Polar Icebreaker Program The Coast Guard and the Navy established the IPO to collaborate and develop a management approach to acquire three HPIBs. Through the IPO, the Coast Guard planned to leverage the Navy’s shipbuilding expertise and pursue an accelerated acquisition schedule. A Coast Guard program manager heads the IPO, which includes embedded Navy officials who provide acquisition, contracting, engineering, cost- estimating, and executive support to the program. The IPO has responsibility for managing and executing the HPIB’s acquisition schedule, acquisition oversight reviews, budget and communications, and interagency coordination. In addition, the IPO coordinates with several key organizations within the Coast Guard and Navy that contribute to the HPIB program, including: Coast Guard Capabilities Directorate: This directorate is responsible for identifying and providing capabilities, competencies, and capacity and developing standards to meet Coast Guard mission requirements. The directorate sponsored the HPIB’s operational requirements document, which provides the key performance parameters the HPIB must meet—such as icebreaking, endurance, and interoperability thresholds and objectives. Ship design team: The ship design team includes Coast Guard and Navy technical experts that develop ship specifications based on the HPIB operational requirements document. The ship design team is under the supervision of a Coast Guard ship design manager, who provides all technical oversight for development of the HPIB design, including development of “indicative,” or concept, designs used to inform the ship’s specifications and the program’s lifecycle cost estimate. Generally, the purpose of an indicative design is to determine requirements feasibility, support cost estimating, and provide a starting point for trade studies. Naval Sea Systems Command (NAVSEA) Cost Engineering and Industrial Analysis Group (NAVSEA 05C): The group developed the HPIB lifecycle cost estimate, which informs the program’s cost baselines and affordability constraints. NAVSEA 05C developed the HPIB’s lifecycle cost estimate based on the ship design team’s indicative design and the technical assumptions outlined in the program cost estimating baseline document. NAVSEA Contracts Directorate (NAVSEA 02): This directorate includes the Navy contracting officer who released the HPIB detail design and construction contract’s solicitation in March 2018 and plans to award the contract under Navy authorities. The contracting officer performs contract management services and provides guidance to the IPO to help ensure the HPIB’s contract adheres to DOD and Navy contracting regulations and guidance. Figure 3 shows key organizations that support the HPIB program and their responsibilities prior to the award of the contract. Since establishing the IPO, the Coast Guard, DHS, and the Navy formalized agreements on their approach for the HPIB acquisition in three 2017 memorandums of agreements and understanding. These agreements define the Navy’s and Coast Guard’s roles in the HPIB acquisition with respect to funding responsibilities, acquisition oversight functions, and contracting and program management authorities, among other things. Heavy Polar Icebreaker Program’s Acquisition Framework DHS, the Coast Guard, and the Navy have agreed to manage and oversee the HPIB program using DHS’s acquisition framework, as Coast Guard is a component within DHS. DHS’s acquisition policy establishes that a major acquisition program’s decision authority shall review the program at a series of predetermined acquisition decision events (ADE) to assess whether the major program is ready to proceed through the acquisition life-cycle phases (see figure 4). As we found in April 2018, the Coast Guard and the Navy will adhere to a tailored DHS acquisition framework for the HPIB program that supplements DHS ADE reviews with additional “gate” reviews adopted from the Navy’s acquisition processes. The DHS Under Secretary for Management retains final decision authority for the HPIB’s ADEs as the acquisition decision authority. The HPIB program achieved a combined ADE 2A/2B in February 2018, when DHS approved the program’s baselines and permitted the program to enter into the Obtain Phase of the DHS acquisition framework. The corresponding acquisition decision memorandum was signed in March 2018. The Coast Guard and the Navy plan to start detail design work for the HPIB in June 2019, once the detail design and construction contract is awarded. In Navy shipbuilding, detail design work can include outlining the steel structure of the ship; determining the routing of systems, such as electrical and piping, throughout the ship; and developing work instructions for constructing elements of the ship, such as installation drawings and material lists. The program’s ADE 2C, or the low-rate initial production decision, corresponds with the approval to start construction of the lead ship, which is planned to begin no later than June 2021. Key steps typically taken in the construction phase of a Navy ship include steel cutting and block fabrication, assembly and outfitting of blocks, keel laying and block erection, launch of the ship from dry dock, system testing and commissioning, sea trials, and delivery and acceptance (see appendix II for more detailed information on each shipbuilding phase). ADE 3, scheduled to be held no later than March 2026, authorizes the program to start follow-on test and evaluation. Figure 5 shows the HPIB’s acquisition framework, including ADE milestones and major program decision points, and how they relate to the shipbuilding phases. DHS acquisition policy establishes that the acquisition program baseline is the fundamental agreement between programs, component, and department-level officials establishing what will be delivered, how it will perform, when it will be delivered, and what it will cost. Specifically, the program baseline establishes a program’s schedule, costs, and key performance parameters, and covers the entire scope of the program’s lifecycle. The HPIB acquisition program baseline serves as an agreement between the Coast Guard and DHS that the Coast Guard will execute the acquisition within the bounds detailed in the document. The acquisition program baseline establishes objective (target) and threshold (maximum acceptable for cost, latest acceptable for schedule, and minimum acceptable for performance) baselines. Tables 1, 2, and 3 show selected cost, schedule, and performance baselines that DHS approved for the HPIB program at ADE 2A/2B in March 2018. After DHS approved the HPIB’s program baselines, the Navy released the solicitation for the program’s detail design and construction contract in March 2018. As revised, the solicitation requires offerors to submit their technical proposals in August 2018 and their price proposals in October 2018. The Navy plans to competitively award the HPIB detail design and construction contract to a single shipyard for all three ships in June 2019. The contract award would include design (advance planning and engineering) and long lead time materials, with separate options for detail design and construction of each of the three ships. The HPIB contract award and administration will follow DOD and Navy contracting regulations and policies, including the Defense Federal Acquisition Regulation Supplement. Although the Navy is planning to award the contract, the source selection authority is from the Coast Guard, with both Coast Guard and Navy personnel serving on the source selection evaluation board. Starting Programs with Sound Business Cases Our prior work has found that successful programs start out with solid, executable business cases before setting program baselines and committing resources. For Coast Guard programs, such a business case would be expected at ADE 2A/2B. A sound business case requires balance between the concept selected to satisfy operator needs and the resources—technologies, design knowledge, funding, and time—needed to transform the concept into a product—or in the HPIB’s case, a ship. At the heart of a business case is a knowledge-based approach—we have found that successful shipbuilding programs build on attaining critical levels of knowledge at key points in the shipbuilding process before significant investments are made. We have previously found that key enablers of a good business case include firm, feasible requirements; plans for a stable design; mature technologies; reliable cost estimates; and realistic schedule targets. Without a sound business case, acquisition programs are at risk of breaching the cost, schedule, and performance baselines set when the program was initiated—in other words, experiencing cost growth, schedule delays, and reduced capabilities. In November 2016, we found that a particular challenge for Congress is the fact that committees must often consider requests to authorize and fund a new program well ahead of program initiation—the point at which key business case information would be presented. Given the time lag between budget requests and the decision to initiate a new acquisition program, Congress could be making critical funding decisions with limited information about the soundness of the program’s business case. Although the HPIB program has already proceeded through ADE 2A/2B and established acquisition program baselines, information about the soundness of the HPIB’s business case will be helpful for decision makers as the Coast Guard and the Navy request funding in preparation for the detail design and construction contract award in June 2019 and anticipated construction start by the end of June 2021—two points at which significant resource commitments will need to be made. The Coast Guard Did Not Assess Design Maturity or Technology Risks Prior to Setting the Polar Icebreaker Program’s Baselines The Coast Guard set the HPIB’s acquisition program baselines at ADE 2A/2B without conducting a preliminary design review to assess the design maturity of the ship or a technology readiness assessment to determine the maturity of key technologies. This approach meets DHS acquisition policy requirements but is contrary to our best practices (see figure 6). While the Coast Guard is committed to a stable design prior to the start of lead ship construction, it established baselines without clear knowledge of the ship design because it does not plan to assess design maturity until after the planned June 2019 award of the detail design and construction contract. In addition, without a technology readiness assessment, the Coast Guard does not have full insight into whether the technologies are mature, potentially underrepresenting technical risk and increasing design risk. As a result, the Coast Guard will be committing resources to the HPIB program without key elements of a sound business case, increasing the risk that the program will exceed its planned costs and schedule. Polar Icebreaker Program Took Steps to Identify Design Risks but Did Not Assess Design Maturity Prior to Setting Baselines Early Efforts to Identify Design Risks To help inform the HPIB’s key performance parameters, specifications, and design considerations prior to setting the acquisition program baselines, the Coast Guard conducted design studies and partnered with Canada (with which the United States has an existing cooperative agreement) to gain knowledge on the HPIB’s design risks. For example: Starting in November 2016, the HPIB ship design team developed an indicative (or concept) design, which has undergone several revisions as more information became available, completing a fifth iteration in September 2017. To inform the HPIB indicative design, the ship design team told us they used design elements with validated characteristics, such as the hull form, from existing Coast Guard icebreakers, including the Polar Star, Polar Sea, Healy, and the Mackinaw (a Great Lakes icebreaker). Collectively, these icebreakers informed elements of the indicative design such as the size and producibility of the ship. The indicative design represents an icebreaker design that meets the threshold key performance parameter of breaking 6 feet of ice at a continuous speed of 3 knots rather than the objective parameter of 8 feet of ice at a continuous speed of 3 knots. Coast Guard officials stated that based on preliminary analysis, a design that meets the HPIB’s objective key performance parameters would be an entirely separate design and would be too costly to construct. Coast Guard officials told us that in addition to price, the shipbuilders’ HPIB proposals will be evaluated on design factors, including how much the potential design exceeds the threshold icebreaking performance parameters. In February 2017, the Coast Guard contracted with five shipbuilders, who teamed with icebreaker design firms, to conduct a series of iterative design studies. These studies examined major design cost drivers and technology risks for the HPIB program. Coast Guard officials stated the results of these studies helped inform and refine the ship’s specifications and provided them with a better understanding of the technology risks and schedule challenges. As of February 2018, each contract was valued at about $5.6 million. Under these contracts, each shipbuilder completed five detailed industry study iterations. For example, the shipbuilders analyzed various hull forms, propulsion systems, cold weather operations, space arrangements, and icebreaking enhancements. In April 2017, the Coast Guard completed an alternatives analysis study—an independent study required prior to ADE 2A that identifies the most efficient method of addressing an identified capability gap. The study examined various options, including whether existing foreign icebreakers could meet the Coast Guard’s HPIB performance requirements. The Coast Guard analyzed 18 domestic and foreign icebreaker designs against the HPIB’s key performance parameters and other requirements, such as seakeeping and habitability. The icebreaker designs included a variety of icebreakers in terms of propulsion power and size, such as nuclear-powered Russian icebreakers and polar research and supply vessels from Australia, Finland, and Germany. The alternatives analysis found that only a Russian nuclear-powered icebreaker and a design for a Canadian diesel-electric-powered icebreaker, which has yet to be constructed, passed initial screening for design maturity and performance requirements. Given a previous independent study analyzing the cost-effectiveness of nuclear- powered icebreakers, the Coast Guard deemed a nuclear-powered icebreaker design as infeasible. The alternatives analysis also noted that the Canadian design met icebreaking requirements. However, Coast Guard officials told us the Canadian design did not meet requirements such as habitability and military-oriented multi-mission tasks, but the design could potentially be modified to meet those needs. In addition, IPO officials stated the Canadian design was designed for science missions rather than military missions. The Canadian design was considered among some of the shipbuilders as a starting point in examining HPIB design risks. From May to August 2017, the Coast Guard tested two scale models of icebreakers at the Canadian National Research Council’s ice tank facility in Newfoundland. Coast Guard officials told us the testing helped to mitigate potential design risks with the hull form and propulsors—a mechanical device that generates thrust to provide propulsion for the ship. The Coast Guard tested the resistance, powering, and maneuvering of the model icebreakers’ hull form and propulsion to inform their indicative design and discovered that the ship’s maneuverability was a challenge during model testing. However, through model testing, the Coast Guard was able to validate general characteristics of its indicative design, including power needs and the hull form. In addition to model testing, Canadian Coast Guard officials told us that the U.S. Coast Guard has engaged with them in formal and informal exchanges regarding icebreaker acquisitions more generally. As a result of its indicative design, industry studies, and model testing efforts, the Coast Guard identified the integrated power plant, propulsors, and hull form as key design considerations for the HPIB. Because these design elements work together to ensure the HPIB can meet its icebreaking requirements, we determined that these are the HPIB’s main design risks (see figure 7). Although the Coast Guard undertook early efforts to identify design risks, it did not conduct a preliminary design review for the HPIB program prior to setting program acquisition baselines at ADE 2A/2B. These baselines inform DHS’s and Coast Guard’s decisions to commit resources. Our best practices for knowledge-based acquisitions state that before program baselines are set, programs should hold key systems engineering events, such as a preliminary design review, to help ensure that requirements are defined and feasible and that the proposed design can be met within cost, schedule, and other system constraints. Similarly, in November 2016, we found that establishing a preliminary design through early detailed systems engineering results in better program outcomes than doing so after program start. During the HPIB’s preliminary design review, the Coast Guard plans to verify that the contractor’s design meets the requirement of the ship specifications and is producible, and the schedule is achievable, among other activities. The Coast Guard has yet to conduct a preliminary design review for the HPIB program because DHS’s current acquisition policy does not require programs to do so until after ADE 2A/2B. The Coast Guard plans to hold the preliminary design review by December 2019, after the award of the detail design and construction contract. Holding a preliminary design review after ADE 2A/2B is consistent with DHS policy. However, in April 2017, we found that DHS’s sequencing of the preliminary design review is not consistent with our acquisition best practices, which state that programs should pursue a knowledge-based acquisition approach that ensures program needs are matched with available resources—such as technical and engineering knowledge, time, and funding—prior to setting baselines. In that report, we found that by initiating programs without a well-developed understanding of system needs through key engineering reviews such as the preliminary design review, DHS increases the likelihood that programs will change their user-defined key performance parameters, costs, or schedules after establishing their baselines. As a result, we recommended that DHS update its acquisition policy to require key technical reviews, including the preliminary design review, to be conducted prior to approving programs’ baselines. DHS concurred with this recommendation and stated that it planned to initiate a study to assess how to better align its processes for technical reviews and acquisition decisions. Upon completion of the study, DHS plans to update its acquisition policies, as appropriate. Instead of establishing the HPIB program’s acquisition program baselines after assessing the shipbuilder’s preliminary design, the Coast Guard established cost baselines based on a cost estimate that used the ship design team’s indicative design. Coast Guard officials told us that the selected shipbuilder will develop its own HPIB design as part of the detail design and construction contract, independent of the indicative design. The ship design team noted that the indicative design informed the ship’s specifications but is not meant to be an optimized design, does not represent a design solution, and will not be provided to the shipbuilders. Coast Guard officials stated that the shipbuilders that respond to the request for proposals will propose their own designs based on their production capabilities, which will drive where they will place components, such as bulkheads, within the design. As a result, the shipbuilder’s design will be different from the indicative design. By setting the HPIB’s acquisition program baselines prior to gaining knowledge on the shipbuilder’s design, the Coast Guard has established cost, schedule, and performance baselines without a stable or mature design. Although completing the preliminary design review after setting program baselines is consistent with DHS policy, this puts the Coast Guard at risk of breaching its established baselines and having to revise them later in the acquisition process, after a contract has been signed and significant resources have already been committed to the program. At that point, the program will be well underway and it will be too late for decision makers to make appropriate tradeoff decisions between requirements and resources without causing disruptions to the program. Consistent with DHS acquisition policy, DHS and the Coast Guard must monitor the HPIB program against the acquisition program baselines set at ADE 2A/2B; however, DHS acquisition policy does not require an official update to the baseline unless the program breaches its baselines or until the next major milestone, whichever occurs first. For the HPIB, the next milestone is ADE 2C, which is currently planned for no later than June 2021. ADE 2C corresponds to the approval of low-rate initial production and in the case of the HPIB, the start of construction for the lead ship. Evaluating the HPIB’s baselines at ADE 2C—immediately before the shipbuilder is authorized to start construction—is too late because the funding required for the construction phase likely would have already been requested and provided. On the other hand, evaluating the acquisition program baselines after the preliminary design review but before ADE 2C would help to ensure that the knowledge gained during the preliminary design review is used to inform the program baselines and business case for investing in the HPIBs before significant resource commitments are made. Although the Coast Guard set the acquisition program baselines prior to gaining knowledge on the feasibility of the selected shipbuilder’s design, it has expressed a commitment to having a stable design prior to the start of lead ship construction. In Navy shipbuilding, detail design typically encompasses three design phases: Basic design. Includes fixing the ship steel structure; routing all major distributive systems, including electricity, water, and other utilities; and ensuring the ship will meet the performance specifications. Functional design. Includes providing further iteration of the basic design, providing information on the exact position of piping and other outfitting in each block, and completing a 3D product model. Production design. Generating work instructions that show detailed system information and including guidance for subcontractors and suppliers, installation drawings, schedules, material lists, and lists of prefabricated materials and parts. Shipbuilding best practices we identified in 2009 found that design stability is achieved upon completion of the basic and functional designs. At this point of design stability, the shipbuilder has a clear understanding of the ship structure as well as how every system is set up and routed throughout the ship. Consistent with our best practices, prior to the start of construction on the lead ship, the Coast Guard will require the shipbuilder to complete basic and functional designs, develop a 3D model output, and provide at least 6 months of production information to support the start of construction. IPO officials have stated that they are committed to ensuring that the HPIB’s design is stable before construction of the lead ship begins, given the challenges prior Navy shipbuilding programs have experienced when construction proceeded before designs were completed. Coast Guard Intends to Use Proven Technologies for the Polar Icebreaker Program but Has Not Assessed Their Maturity The Coast Guard intends on using what it refers to as proven technologies for the HPIB but has not conducted a technology readiness assessment to determine maturity of key technologies that drive performance of the ship prior to ADE 2A/2B, which is inconsistent with our best practices. A technology readiness assessment is a systematic, evidence-based process that evaluates the maturity of critical technologies—hardware and software technologies critical to the fulfillment of the key objectives of an acquisition program. This assessment does not eliminate technology risk but, when done well, can illuminate concerns and serve as a basis for realistic discussions on how to mitigate potential risks. According to our best practices, a technology readiness assessment should be conducted prior to program initiation. DHS systems engineering guidance also recommends conducting a technology readiness assessment before ADE 2A to help ensure that the program’s technologies are sufficiently mature by the start of the program. The Coast Guard intends on using what it has deemed “state-of-the- market” or “proven” technologies for the HPIB. DHS’s technical assessment of the HPIB noted that the February 2017 design studies resulted in industry producing designs that used commercially available, state-of-the-market, and proven technologies. From the studies and industry engagement, Coast Guard officials determined that the technologies required for the HPIB, such as the integrated power plant and azimuthing propulsors—thrusters that rotate up to 360 degrees and provide propulsion to the ship—are available commercially and do not need to be developed. Coast Guard officials further stated that the integrated power plant is the standard power plant used on domestic and foreign icebreakers. Coast Guard officials told us that similarly, market survey data on azimuthing propulsors shows that ice-qualified azimuthing propulsors in the power range have been used on foreign icebreakers. The Coast Guard has also communicated to industry through the request for proposals that the HPIB should have only proven technology and plans to have the shipbuilders provide information on the maturity of the technologies when they submit their proposals. As a result, Coast Guard officials stated the HPIB program does not have any critical technologies, as defined by DHS systems engineering guidance, and does not need to conduct a technology readiness assessment. However, according to DHS systems engineering guidance, a technology element is considered critical if the system being acquired depends on this technology to meet operational requirements, and if the technology or its application is new, novel, or in an area that poses major technological risk during detailed design or demonstration. The guidance further states that technologies can become critical if they need to be modified from prior successful use or expected to operate in an environment beyond their original demonstrated capability. Similarly, according to our best practices for assessing technology readiness, critical technologies are not just technologies that are new or novel. Technologies used on prior systems can also become critical if they are being used in a different form, fit, or function. Our technology readiness assessment guide notes that program officials sometimes disregard critical technologies when they have longstanding history, knowledge, or familiarity with them. The best practices guide cites examples of organizations not considering a technology critical if it has been determined to be mature, has already been fielded, or does not currently pose a risk to the program. Additionally, our guide notes that contractors may be overly optimistic about the maturity of critical technologies, especially prior to contract awards. According to our best practices guide, presuming a previously used technology as mature is problematic when the technologies are being reapplied to a different program or operational environment. As a result, based on our analysis of available Coast Guard information, we believe the HPIB’s planned integrated power plant and azimuthing propulsors should be considered critical technologies given their criticality in meeting key performance parameters, their use in a different environment from prior ships, and the extent to which they pose major cost risks (see table 4). Without conducting a technology readiness assessment, the Coast Guard does not have insight into how mature these critical technologies are. According to our best practices, evaluating critical technologies requires disciplined and repeatable steps and criteria to perform the assessment and make credible judgments about their maturity. The evaluation of each critical technology must be based on evidence such as data and test results. In addition, the team that assesses the technologies must be objective and ideally independent. Instead, the Coast Guard has relied on industry to provide information on the maturity of the HPIB’s technologies and uses terms such as “state-of-the-market” or “proven,” which do not translate into meaningful measures for systematically communicating the technology readiness, especially when discussing new applications of existing technologies. Additionally, even if the Coast Guard determines the maturity levels of the HPIB’s technologies through an objective and independent technology readiness assessment, the program’s planned level of maturity for the ship’s technologies falls short of our best practices. According to the HPIB’s systems engineering tailoring plan and request for proposals, the program intends on implementing only proven technologies that have been demonstrated in a relevant environment, commensurate with a technology readiness level (TRL) of 6. However, our best practices do not consider a technology to be mature until it has been demonstrated in an operational environment, commensurate with a TRL 7. Specifically, our best practices for shipbuilding recommend that programs should require critical technologies to be matured into actual prototypes and successfully demonstrated in an operational or a realistic environment (TRL 7) before a contract is awarded for the detail design of a new ship. DHS’s systems engineering guidance also states that critical technologies below TRL 7 should be identified as technical risks. By not conducting a technology readiness assessment and identifying, assessing, and maturing its critical technologies prior to setting the HPIB’s program baselines and prior to awarding the detail design contract, the Coast Guard is underrepresenting the program’s technical risks and understating its cost, schedule, and performance risks. Technology risks that manifest later could require the shipbuilder to redesign parts of the ship, which increases the risk of rework and schedule delays during the construction phase. The Coast Guard Based the Polar Icebreaker Program’s Baselines on a Cost Estimate That Is Not Fully Reliable and an Optimistic Schedule The cost estimate and schedule that informed DHS’s decision to authorize the HPIB program do not reflect the full scope of the program’s risks. We found that while the Navy substantially adhered to a number of best practices when it developed the HPIB’s cost estimate, the estimate is not fully reliable, primarily because it does not reflect the full range of possible costs over the HPIB’s 30-year lifecycle. We also found the HPIB schedule was not informed by a realistic assessment of the work necessary to construct the ship. Rather, the schedule was driven by the potential gap in icebreaking capabilities once the Coast Guard’s only operating HPIB reaches the end of its service life. Reliable cost estimates and schedules are key elements of an executable business case, and are needed at the outset of programs—when competitive pressures to obtain funding for the program are high—to provide decision makers with insight into how risks affect a program’s ability to deliver within its cost and schedule goals. Polar Icebreaker Program’s Cost Estimate Substantially Met Best Practices but Is Not Fully Reliable Because It Does Not Include Full Range of Possible Costs We found that the lifecycle cost estimate used to inform the HPIB program’s baselines substantially adheres to most cost estimating best practices; however, the estimate is not fully reliable. The cost estimate only partially met best practices for being credible primarily because it did not quantify the range of possible costs over the entire life of the program. We assessed the program’s lifecycle cost estimate, which was performed by NAVSEA 05C, against our best practices for cost estimating. For our reporting purposes, we collapsed 18 of our applicable best practices into the four general characteristics of a reliable cost estimate: comprehensive, well-documented, accurate, and credible. Figure 8 provides a summary of our assessment of the HPIB’s lifecycle cost estimate. Comprehensive. We found the HPIB cost estimate substantially met the best practices for being comprehensive. For example, the estimate includes government and contractor costs over the full lifecycle of all three ships and contains sufficient levels of detail in the program’s work breakdown structure—a hierarchical breakdown of the program into specific efforts, including system engineering and ship construction. The estimate also documents detailed ground rules and assumptions, such as the learning curve used to capture expected labor efficiencies for follow- on ships. However, we found that the costs for disposal of the three ships were not at a level of detail to ensure that all costs were considered and not all assumptions, particularly regarding operating and support costs, were varied to reflect the impact on cost should these assumptions change. Well-Documented. We also found the cost estimate substantially met the best practices for being well-documented. Specifically, the cost estimate’s documentation mostly captured the source data used as well as the primary methods, calculations, results, rationales, and assumptions used to generate each cost element. However, the documentation alone did not provide enough information for someone unfamiliar with the cost estimate to replicate what was done and arrive at the same results. For example, NAVSEA officials discussed and showed us how historical data from the analogous ships were used to create the estimate, but these specific sources were not found in the cost estimate documentation. Accurate. We found the estimate substantially met best practices for being accurate. In particular, the estimate was properly adjusted for inflation, and we did not find any mathematical errors in the estimate calculations we inspected. Officials stated that labor and material cost data from recent, analogous programs were used in the estimate. While the documentation does not discuss the reliability, age, or relevance of the cost data, NAVSEA officials provided us with additional information regarding those data characteristics. Additionally, officials provided documentation that demonstrated that they had updated the cost estimate several times in the last 2 years. Credible. We found the HPIB cost estimate partially met the best practices associated with being credible. A credible cost estimate should analyze the sensitivity of the program’s expected cost to changes among key cost-driving assumptions and risks. It should also quantify the cost impact of risks related to assumptions changing and variability in the underlying data used to create the cost estimate. Credible cost estimates should also be cross-checked internally and reconciled with an independent cost estimate that is performed by an outside group. These two best practices ensure that the estimate has been checked for any potential bias. Our review of the HPIB cost estimate determined it partially met the best practices for being credible due to the following: Exclusions of major costs from sensitivity analysis and risk and uncertainty analysis. The cost estimators conducted sensitivity analysis as well as risk and uncertainty analysis on only a small portion of the total lifecycle costs. For both the sensitivity analysis and risk and uncertainty analysis, we found that NAVSEA only modeled cost variation in the detail design and construction portion of the program and excluded from its analyses any risk impacts related to the remainder of the acquisition, operating and support, and disposal phases, which altogether comprise about 75 percent of the lifecycle cost. The cost estimate documents that the limited number of active icebreakers and available data prevented NAVSEA from identifying accurate risk bounds for the operating and support and disposal phases. Further, NAVSEA officials told us because they used historical data, including average maintenance costs from the Healy, they felt that their estimate was reasonable. However, similar to how NAVSEA consulted with the ship design team to establish high and low-end costs using analogous ships, NAVSEA could have used cost ranges in the historical data to develop risk bounds for the remaining costs in the acquisition, operations and support, and disposal phases. Without performing a sensitivity analysis on the entire life cycle cost of the three ships, it is not possible for NAVSEA to identify key elements affecting the overall cost estimate. Further, without performing a risk and uncertainty analysis on the entire life cycle cost of the three ships, it is not possible for NAVSEA to determine a level of confidence associated with the overall cost estimate. By not quantifying important risks, NAVSEA may have underestimated the range of possible costs for about three-quarters of the entire program. Lack of applied correlation in the risk and uncertainty analysis. In its independent assessment of the HPIB cost estimate, the DHS Cost Analysis Division similarly found that the results of the risk and uncertainty analysis may understate the range of possible cost outcomes for the HPIB. The DHS assessment noted that NAVSEA did not use applied correlation—which links costs for related items so that they rise and fall together during the analysis—in its cost model. According to a joint agency handbook on cost risk and uncertainty, applied correlation helps to ensure that cost estimates do not understate the possible variation in total program costs. Omitting applied correlation when assessing a cost estimate for risk can cause an understated range of possible program costs and create a false sense of confidence in the cost estimate. For example, absent applied correlation, the DHS assessment noted that the Navy calculated with a 99-percent level of confidence that the program will not exceed its threshold (maximum acceptable) acquisition cost. Navy officials explained that they will incorporate applied correlation in future updates to the cost estimate when better data are available. However, by applying correlation factors from the joint agency handbook to the same data that NAVSEA used, DHS’s Cost Analysis Division determined that NAVSEA overstated the likelihood of the program not exceeding its threshold acquisition cost. Cost estimate not fully reconciled with a comparable independent cost estimate. While the Naval Center for Cost Analysis performed an independent cost estimate of the HPIB program, the office used a different methodology from NAVSEA’s, and its estimate was based on an earlier version of the indicative ship design and associated technical baseline. NAVSEA officials told us that before the Coast Guard’s ship design team updated the indicative ship design and technical baseline, NAVSEA met twice with Naval Center for Cost Analysis to reconcile their results. However, NAVSEA officials told us that due to the speed at which the program was progressing, no reconciliation occurred after the ship design team finalized the indicative ship design. While we did not find any specific ground rules and assumptions that differed between the two estimates, some ship characteristics had changed, such as the weight estimates for propulsion and auxiliary systems, among others. The use of two different technical baselines creates differences in the two estimates and makes them less comparable to one another. For additional details on our assessment of the HPIB’s cost estimate against our 18 cost estimating best practices, see appendix III. By excluding the majority of the HPIB program’s lifecycle costs from the sensitivity analysis as well as the risk and uncertainty analysis, and reconciling the estimate with an independent cost estimate based on a different iteration of the ship design, the cost estimate does not provide a fully credible range of costs the program may incur. Moreover, the exclusion of applied correlation further provides a false sense of confidence that the program will not exceed its threshold cost. As a result, the estimate provides an overly optimistic assessment of the program’s vulnerability to cost growth should risks be realized or current assumptions change. This, in turn, may underestimate the lifecycle cost of the program and calls into question the cost baselines DHS approved and used to inform the HPIB’s budget request. Without a reliable cost estimate to inform the business case for the HPIB prior to award of the contract option for lead ship construction, Congress is at risk of committing to a course of action without a complete understanding of the program’s longer-term potential for cost growth. Polar Icebreaker Program’s Optimistic Schedule Is Driven by Capability Gap and Does Not Reflect Robust Analysis The Coast Guard set an optimistic schedule baseline for the HPIB based on operational need, but its approach does not reflect a robust analysis of what is realistic and feasible. According to DHS and Coast Guard acquisition guidance, the goal of ADE 2A/2B is, among other things, to ensure that the program’s schedule baseline is executable at an acceptable cost. Rather than building a schedule based on knowledge—including determining realistic schedule targets, analyzing how much time to include in the schedule to buffer against potential delays, and comprehensively assessing schedule risks—the Coast Guard used the estimated end date of the Polar Star’s service life as the primary driver to set the lead ship’s objective (or target) delivery date of September 2023 and threshold (latest acceptable) delivery date of March 2024. Analysis Conducted to Determine Lead Ship Construction Schedule Not Robust The Coast Guard and the Navy did not conduct a robust analysis to determine how realistic the 2.5- to 3-year construction cycle time is for the lead HPIB before setting the schedule baseline. Our best practices for developing project schedules state that, rather than meeting a particular completion date, estimating how long an activity takes should be based on the effort required to complete the activity and the resources available. Doing so ensures that activity durations and completion dates are realistic and supported by logic. The Coast Guard and the Navy validated the reasonableness of the 2.5- to 3-year construction time by comparing this duration to historical Navy ship construction data. Program officials told us that they used 211 Navy ships in their analysis and determined that the HPIB’s construction schedule was within historical norms given its weight. However, program officials told us they included both lead and follow-on ships in their analysis. As we have found in our prior Navy shipbuilding work, schedule delays tend to be amplified for lead ships in a class. Therefore, we believe the program’s analysis for the lead ship was overly optimistic. The Coast Guard also sought industry feedback to determine whether 2.5 to 3 years to build the lead HPIB was feasible. Design study information provided to the Coast Guard by several shipbuilders estimated that they would need between 2.5 to 3.5 years to build the lead ship. We determined that the Coast Guard used the more optimistic estimate of 2.5 years for the objective delivery date and 3 years for the threshold delivery date. Three years was also the time frame reflected in the request for proposals for the detail design and construction contract. The request for proposals lists December 2023 as the target delivery date for the lead ship, which is approximately 3 years from the objective construction start date. Further, we compared the HPIB’s planned construction schedule to the construction schedules of delivered lead ships for major Coast Guard and Navy shipbuilding programs active in the last 10 years as well as the Healy. We found that the HPIB’s lead ship construction cycle time of 2.5 to 3 years is optimistic, as only three of the ten ships in our analysis were constructed in 3 years or less. For the purposes of our analysis, we included information on each ship’s weight and classification, both of which can affect complexity and, therefore, construction times (see figure 9). The Coast Guard also did not conduct any analysis to identify a reasonable amount of margin to include in the program schedule baseline to account for any delays. Estimating and documenting schedule margin based on an analysis of schedule risks helps to ensure that a program’s baseline schedule is achievable despite delays that may unexpectedly arise. Program officials told us that the only margin included in the HPIB schedule is the 6 months between the objective and threshold dates—the maximum time between objective and threshold dates before DHS policy requires additional rationale and justification. According to the request for proposals, the winning shipbuilder will examine schedule risks while preparing an integrated schedule. In addition, Coast Guard officials told us that the current schedule will remain largely notional until the winning shipbuilder provides detailed updates to the schedule. Delays in project schedules, whether they are in the program’s control or not, should be expected. For example, in prior shipbuilding programs we have reviewed, we have found that delays have resulted from a number of issues, including redesign work to address issues discovered during pre-delivery testing, key system integration problems, and design quality issues. Delays outside of the program’s control such as funding instability, late material delivery, and bid protests have previously affected a program’s ability to meet schedule. Program officials told us these and other schedule risks are not accounted for in the HPIB schedule. Further, our analysis of 12 selected shipbuilding acquisition programs active in the last 10 years shows that the Navy and the Coast Guard have delayed delivery of all but one lead ship from their original planned delivery dates by more than 6 months, with delays occurring both before and after the start of construction. The delays in lead ship deliveries ranged from 9 months to 75 months. For the purposes of our analysis, we included the lead ships of major Coast Guard and Navy shipbuilding programs that have been active from 2008 to 2018. We excluded the Navy submarines and aircraft carriers from our analysis because we determined that their size and complexity did not make them reasonable comparisons to the HPIB (see figure 10). By supporting the lead ship construction time with overly optimistic analysis and by not conducting analysis to estimate a reasonable amount of margin, the Coast Guard’s HPIB schedule does not fully account for likely or unforeseen delays, which would help ensure that the planned delivery date for the lead ship is feasible. Schedule Risks after Construction Start Not Identified The Coast Guard has set the HPIB’s schedule baselines, including when all three ships are planned to achieve full operational capability, but has not yet identified risks for the program’s schedule that could occur after the start of lead ship construction, such as risks related to the construction schedule or concurrency between ship testing and construction of subsequent ships. According to the HPIB risk management plan, the program should formally track risks, which includes developing risk mitigation plans and reporting risks to DHS. Prior to setting its baselines, the Coast Guard formally tracked some schedule risks that affect the program’s ability to start construction on time, such as an aggressive schedule for releasing the request for proposals for the detail design and construction contract. IPO officials told us they retired that risk because the Navy released the request for proposals in March 2018. However, our analysis of the HPIB construction schedule and 6- month margin for delays found the program’s schedule was optimistic, thereby warranting additional risk tracking and management. The DHS Office of Systems Engineering also identified and recommended the Coast Guard track and take steps to mitigate HPIB’s schedule risks, including those related to concurrency. In its technical assessment, this office noted that the program plans to deliver the first two ships prior to completing initial operational testing and evaluation for the lead ship. The assessment further noted that construction on the third ship is planned to be nearly three-quarters finished prior to completing initial operational testing and evaluation. DHS’s Office of Systems Engineering found that this concurrency creates cost, schedule, and technical risk resulting from rework that may be necessary to address deficiencies found during initial testing. By not comprehensively and formally tracking risks to the HPIB schedule that occur after the start of lead ship construction, the program may not sufficiently identify and take timely risk management actions to address this key phase in the acquisition. By not conducting a robust analysis to inform whether the HPIB’s schedule baselines are feasible, the Coast Guard is not providing Congress with realistic dates of when the ships may be delivered before requesting funding for the construction of the lead ship. While the Coast Guard is planning a service life extension of the Polar Star starting in 2020, as noted above, the HPIB’s optimistic schedule may put the Polar Star at risk of needing to operate longer than planned. The HPIB schedule’s optimism also puts the Coast Guard at risk of not fully implementing a knowledge-based acquisition approach to meet its aggressive schedule goals. Our prior work on shipbuilding programs has shown that establishing optimistic program schedules based on insufficient knowledge can create pressure for programs to make sacrifices elsewhere. For example, we found that the Navy moved forward with construction with incomplete designs and when key equipment was not available when needed. Additionally, some Navy programs pushed technology development into the design phase or pushed design into the construction phase. These concurrencies often result in costly rework to accommodate changes to the design, further delays, or lower than promised levels of capability. Polar Icebreaker Program’s Anticipated Contract May Be Funded by Both the Coast Guard and the Navy, but They Have Not Fully Documented Responsibility for Addressing Cost Growth According to the IPO, the HPIB’s anticipated detail design and construction contract may be funded by both Coast Guard and Navy appropriations, but how certain types of cost growth will be addressed between the Coast Guard and the Navy has not been fully documented. The HPIB’s acquisition strategy anticipates award of a contract that will have options, includes efforts aimed at mitigating cost risks, and acknowledges the use of foreign suppliers to provide components and design services as allowable under statute and regulation. Since 2013, the program has received $360 million in funding, which includes both Coast Guard and Navy appropriations. Moving forward, it is unclear how much Coast Guard and Navy funding will be used to fund the contract. The Coast Guard and the Navy have an agreement in place for funding issues, but the agreement does not fully address how they plan to address cost growth on the program. Acquisition Strategy Anticipates Use of Contract Options, Ways to Mitigate Cost Risks, and Foreign Suppliers As part of the HPIB’s acquisition strategy, the Navy structured the detail design and construction of each of the ships as contract options in the March 2018 request for proposals. Specifically, the request for proposals structured the detail design and construction work into four distinct contract line items, all under a fixed-price incentive (firm-target) contract type. Generally, this contract type allows the government and shipbuilder to share cost savings and risk through a specified profit adjustment formula, also known as a share ratio; ties the shipbuilder’s ability to earn a profit to performance by decreasing the shipbuilder’s profit after costs reach the agreed upon target cost; and, subject to other contract terms, fixes the government’s maximum obligation to pay at a ceiling price. Table 5 provides information on the HPIB’s request for proposals as of May 2018. According to the request for proposals, in addition to potentially earning profit by controlling costs, the shipbuilder may also earn up to $34 million in incentives for achieving other programs goals, such as quality early delivery, reducing operations and sustainment costs, and production readiness. IPO officials stated that they based the incentives on prior Navy shipbuilding contract examples. However, in March 2017, we found that the Navy had not assessed the effectiveness of added incentives for the reviewed fixed-price incentive contracts in terms of improved contract outcomes across the applicable shipbuilding portfolio. As a result, we recommended that DOD direct the Navy to conduct a portfolio-wide assessment of the Navy’s use of additional incentives on fixed-price incentive contracts across its shipbuilding programs. DOD concurred with this recommendation, but the Navy has not yet taken steps to implement it. As part of the HPIB acquisition strategy, the IPO is striving to control costs on the detail design and construction contract through the following: A fixed-price incentive (firm-target) contract type. Because the shipbuilder’s profit is linked to performance, fixed-price incentive contracts provide an incentive for the shipbuilder to control cost. Most of the Navy’s proposed share ratios and ceiling prices for the detail design and construction work are consistent with DOD’s November 2010 Better Buying Power memo, which states a 50/50 share ratio and 120 percent ceiling price should be the norm, or starting point, for fixed-price incentive contracts. Full and open competition. The Navy plans to competitively award the HPIB’s detail design and construction contract. From market research and industry engagement, the IPO determined that there were multiple viable competitors. In March 2017, we found that competition helped to strengthen the Navy’s negotiating position with shipbuilders when setting contract terms, such as the share line and ceiling price for fixed-price incentive type contracts. Providing offerors the government’s estimated ship costs. The request for proposals does not set affordability caps but does include information on the government’s estimated cost for the ships, including $746 million for the lead ship’s advance planning, engineering, detail design, and construction, and an average ship price of $615 million across all three ships. Navy contracting officials explained that offers will not be disqualified from the source selection solely for being higher than the estimated costs. Instead, the estimated costs provide the offerors with cost bounds to help appropriately scope the capabilities. For example, IPO officials stated that they are striving to appropriately size the integrated power plant so that it is generating sufficient power to meet key performance parameters but not so much power that it drives up the cost. Inquiries on block buys and economic order of quantity purchases. The Navy gave offerors an opportunity to provide the estimated savings that the government could achieve if it were to take a “block buy” approach in purchasing the ships or purchasing supplies in economic order quantities. The Navy did not include a definition of “block buy” in the HPIB request for proposals synopsis. Based on our prior work, block buy contracting generally refers to special legislative authority that agencies seek on an acquisition-by-acquisition basis to purchase more than one year’s worth of requirements. The request for proposals synopsis stated a preference for submission of the estimated savings within 60 days of the release of the request for proposals, or by May 2018. As of June 2018, the Navy had not received any formal responses from industry on potential savings from block buys or economic order quantities. For the HPIB request for proposals, the Navy stated that any information on block buys or economic order of quantities would be optional and would not be used as part of the evaluation of proposals submitted by offerors. Our prior work on block buy contracting approaches for the Littoral Combat Ship and F-35 Joint Strike Fighter programs found that the terms and conditions of the contracts affect the extent to which the government achieves savings under a block buy approach. For example, the Littoral Combat Ship’s block buy contracts indicated that a failure to fully fund the purchase of a ship in a given year would make the contract subject to renegotiation. DOD has pointed to this as a risk that the contractors would demand higher prices if DOD deviated from the agreed to block buy plan. In its HPIB acquisition strategy, the IPO has also considered the use of foreign suppliers as allowable under the law. According to the February 2018 HPIB acquisition plan, the HPIB must be constructed in a U.S. shipyard given statutory restrictions, including restrictions on construction of Coast Guard vessels and major components in foreign shipyards unless authorized by the President. However, foreign suppliers will be permitted to provide components and design services to the extent applicable statutes and regulations allow. According to Coast Guard officials, foreign design firms have extensive expertise and knowledge to produce the design for HPIBs. As a result, the U.S. shipbuilders planning to submit proposals on the HPIB solicitation may partner with these foreign design firms when submitting proposals. Similarly, Coast Guard officials stated that the azimuthing propulsors that have the necessary power and ice classification for the HPIB are manufactured by foreign companies. Therefore, the selected shipbuilder may subcontract with these companies to acquire the propulsors. In addition, Navy contracting officials stated that the program did not need to obtain a waiver from the Buy American Act—which generally requires federal agencies to purchase domestic end products when supplies are acquired for use in the United States, and use domestic construction materials on contracts performed in the United States—for certain components. The Act includes exceptions, such as when the domestic end products or construction materials are unavailable in sufficient and reasonably available commercial quantities and of a satisfactory quality. Program Has Received Both Coast Guard and Navy Funds, but Unclear How Program Will Be Funded Moving Forward From 2013 through 2018, the program has received $360 million in funding—$60 million in the Coast Guard’s Acquisition, Construction, and Improvement appropriations (hereafter referred to as Coast Guard funding) and $300 million in Navy’s Shipbuilding and Conversion, Navy advance procurement appropriations (hereafter referred to as Navy appropriations). In addition, according to Coast Guard officials, in fiscal year 2017, Coast Guard reprogrammed $30 million in fiscal year 2016 appropriations for the HPIB from another program (see figure 11). According to IPO and Navy contracting officials, the Navy plans to use $270 million of the $300 million in Navy appropriations to award the detail design and construction contract in fiscal year 2019, which would fund the advanced engineering, long lead time materials, and detail design work. Navy officials stated the remaining $30 million in Navy appropriations will be held in reserves for potential scope changes. Of the $60 million in Coast Guard funding, the IPO has used $41 million for program office costs and the February 2017 design study contracts, and plans to use the remaining $19 million for program office costs. Coast Guard officials stated that they used the $30 million in reprogrammed 2016 appropriations to fund the design studies, model testing, and Navy warfare center support. As the program prepares to award a contract worth billions of dollars if all the options are exercised, Congress, the Coast Guard, and the Navy face key funding considerations. These include the extent to which the program will be funded using Coast Guard and Navy appropriations in the future and whether each of the ships will be fully or incrementally funded. Navy contracting officials stated that by structuring the contract’s construction work as options, the contract has flexibility to accommodate any type of additional funding the program may receive. The National Defense Authorization Act for Fiscal Year 2018 authorized procurement of one Coast Guard heavy polar icebreaker vessel. The Navy did not request any funding in fiscal year 2019 for the HPIB, while Coast Guard requested $30 million. Subsequently, after discretionary budget caps were relaxed by Congress, the Administration’s fiscal year 2019 budget addendum requested an additional $720 million in fiscal year 2019 Coast Guard appropriations for the program. Although the Navy did not request fiscal year 2019 funding for the lead ship, and Navy officials told us they have no plans to budget for the HPIB program moving forward, Congress may still choose to appropriate funds for the HPIB to the Navy. For example, in fiscal years 2017 and 2018, the Navy did not request funding but received $150 million in appropriations each year for the HPIB (see figure 12). Additionally, the Coast Guard has been expressly authorized to use incremental funding for the HPIB. This authorization is reflected in the Coast Guard’s January 2018 affordability certification memo, submitted to DHS leadership. These memos are required to certify that a program’s funding levels are adequate and identify tradeoffs needed to address any funding gaps. However, as noted above, with the addition of the Administration’s fiscal year 2019 budget request addendum, the Coast Guard requested $750 million in full funding for the lead ship. The Navy has informed us that it plans to award the advance planning, design, engineering, long lead time material contract line item with its $270 million in appropriations. Navy officials also told us they are in the process of determining whether it needs to be authorized by Congress to use an incremental funding approach to fund the detail design and construction options if full funding is not received by the Navy. According to the Office of Management and Budget’s A-11 budget circular, full funding helps to ensure that all costs and benefits of an acquisition are fully taken into account at the time decisions are made to provide resources. The circular goes on to say that when full funding is not followed, without certainty if or when future funding will be available, the result is sometimes poor planning, higher acquisition costs, cancellation of major investments, or the loss of sunk costs. The circular, however, also notes that Congress may change the agency’s request for full funding to incremental funding to accommodate more projects in a year than would be allowed with full funding. Plans to Address Cost Growth Not Fully Documented Regardless of the funding strategy and which service funds the contract, the Coast Guard and the Navy do not have a clear agreement on how certain types of cost growth within the program will be addressed. The budgeting and financial management appendix of the July 2017 agreement between the Coast Guard and Navy for the HPIB notes that any cost overruns will be funded by the originating source of the appropriation and be the responsibility of the organization that receives the funding. However, the Coast Guard and the Navy have interpreted “cost overruns” differently in the context of the agreement. Coast Guard and Navy officials are in agreement that given the fixed- price incentive contract type, the government’s share of cost overruns between the target cost and ceiling price (based on the share ratio) will be the responsibility of the organization that provided the funding for the contract line item. Navy officials also noted that because the contract type is fixed-price incentive, any cost overruns above the ceiling price are generally the responsibility of the contractor, not the government. However, the Coast Guard and the Navy have not addressed in an agreement how they plan to handle any cost growth stemming from changes to the scope, terms, and conditions of the HPIB detail design and construction contract. For example, if the Coast Guard or the Navy revises the program’s requirements, this could increase the scope and value of the contract and result in additional contract costs. It is unclear in this instance, which organization is responsible for paying for the additional costs. Further, our 2005 work on Navy shipbuilding programs found that the most common causes of cost growth in these programs were related to design modifications, the need for additional and more costly materials, and changes in employee pay and benefits, some of which required changes in contract scope. IPO officials told us that unplanned changes to the program’s scope and any corresponding funding requests for unanticipated cost growth would require discussions and agreements with both Coast Guard and Navy leadership. Coast Guard and Navy officials stated that they are in the process of reviewing the July 2017 budget appendix of the agreement to clarify the definition of cost overruns and plan to finalize revisions no later than September 2018. Our prior work on implementing interagency collaborative mechanisms found that agencies that articulate their agreements in formal documents can strengthen their commitment to working collaboratively, which can help better overcome significant differences when they arise. Different interpretations or disagreements on financial responsibility between the Coast Guard and the Navy on cost growth for the HPIB program could result in funding instability for the program, which could affect the program’s ability to meet its cost and schedule goals. Conclusions In the last several years, the Coast Guard and the Navy have made significant strides in their efforts to acquire heavy polar icebreakers. It has been over 40 years since the United States has recapitalized its aging heavy polar icebreaker fleet, and Congress has expressed the need for investment in the HPIB program to help ensure our continued presence in the polar regions. The Coast Guard and the Navy have taken steps to examine design risks and expressed commitment to design maturity before starting construction on the lead ship. However, the Coast Guard and the Navy did not take key steps to reduce risks on the HPIB program before setting the HPIB’s program baselines— namely, conducting a preliminary design review, conducting a technology readiness assessment, developing a fully reliable cost estimate, and conducting analysis to determine a realistic schedule and risks to that schedule. By setting the program’s baselines prior to obtaining sufficient knowledge in the design, technologies, cost, and schedule of the HPIB, DHS, the Coast Guard, and the Navy are not establishing a sound business case for investing in the HPIB nor putting the program in a position to succeed. There is risk that the program will cost more than the planned $9.8 billion and the lead ship will not be delivered by 2023 as planned. Further, without clear agreement between the Coast Guard and the Navy on which service will be responsible for any cost growth on the HPIB, the program is at further risk of not meeting its ambitious goals. In the current budget environment, it is imperative that the Coast Guard and the Navy obtain sufficient acquisition knowledge and put together a sound business case before asking Congress and taxpayers to commit significant resources to the HPIB program. Recommendations for Executive Action We are making six recommendations total to the Coast Guard, DHS, and the Navy: The Commandant of the Coast Guard should direct the polar icebreaker program to conduct a technology readiness assessment in accordance with best practices for evaluating technology readiness, identify critical technologies, and develop a plan to mature any technologies not designated to be at least TRL 7 before detail design of the lead ship begins. (Recommendation 1) The Commandant of the Coast Guard, in collaboration with the Secretary of the Navy, should direct the polar icebreaker program and NAVSEA 05C to update the HPIB cost estimate in accordance with best practices for cost estimation, including (1) developing risk bounds for all phases of the program lifecycle, and on the basis of these risk bounds, conduct risk and uncertainty analysis, as well as sensitivity analysis, on all phases of the program lifecycle, and (2) reconciling the results with an updated independent cost estimate based on the same technical baseline before the option for construction of the lead ship is awarded. (Recommendation 2) The Commandant of the Coast Guard should direct the polar icebreaker program office to develop a program schedule in accordance with best practices for project schedules, including determining realistic durations of all shipbuilding activities and identifying and including a reasonable amount of margin in the schedule, to set realistic schedule goals for all three ships before the option for construction of the lead ship is awarded. (Recommendation 3) The Commandant of the Coast Guard should direct the polar icebreaker program office to analyze and determine appropriate schedule risks that could affect the program after construction of the lead ship begins to be included in its risk management plan and develop appropriate risk mitigation strategies. (Recommendation 4) The DHS Under Secretary for Management should require the Coast Guard to update the HPIB acquisition program baselines prior to authorizing lead ship construction, after completion of the preliminary design review, and after it has gained the requisite knowledge on its technologies, cost, and schedule, as recommended above. (Recommendation 5) The Commandant of the Coast Guard, in collaboration with the Secretary of the Navy, should update the financial management and budget execution appendix of the memorandum of agreement between the Coast Guard and the Navy to clarify and document agreement on how all cost growth on the HPIB program, including changes in scope, will be addressed between the Coast Guard and the Navy. (Recommendation 6) Agency Comments We provided a draft of this report to DHS and DOD for review and comment. In its comments, reproduced in appendix IV, DHS concurred with all six of our recommendations and identified actions it planned to take to address them. The Navy stated that it deferred to DHS and the Coast Guard on responding to our recommendations. DHS, the Coast Guard, and the Navy also provided technical comments, which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, the Secretary of Homeland Security, the Commandant of the Coast Guard, the Secretary of the Navy, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-4841 or makm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to the report are listed in appendix V. Appendix I: Objectives, Scope, and Methodology This report examines (1) the extent to which the heavy polar icebreaker (HPIB) program has taken steps to develop mature designs and technologies consistent with best practices, (2) the extent to which the HPIB program has taken steps to set realistic cost and schedule estimates, and (3) the status of the HPIB program’s contracting efforts and funding considerations. To assess the extent to which the HPIB program has taken steps to develop mature designs and technologies consistent with GAO-identified best practices, we reviewed program performance and design requirements, including the program’s operational requirements documents, system specifications such as the power plant, propulsion system, and hull, and technical baseline; the program’s alternatives analysis study, tailored systems engineering plan, test and evaluation master plan, and model testing results; cooperative agreements with Canada related to the HPIB; excerpts from industry studies; and the March 2018 detail design and construction request for proposals and subsequent amendments. We also reviewed relevant Department of Homeland Security (DHS), Coast Guard, and Department of Defense (DOD) acquisition guidance and instructions. From these documents, we determined the program’s design and technology efforts and compared them to GAO’s various best practices, including using a knowledge-based approach to shipbuilding, knowledge-based approach to major acquisitions, and evaluating technology readiness. We also interviewed knowledgeable officials from the Coast Guard’s Capabilities Directorate, Research and Development Center, and Marine Transportation Systems Directorate; DHS’s Science and Technology Directorate’s Office of Systems Engineering; the Canadian Coast Guard; and the National Science Foundation. To assess the extent to which the HPIB program has taken steps to set realistic cost and schedule estimates, we determined the extent to which the estimates were consistent with best practices as identified in GAO’s Cost Estimating and Assessment and Schedule Assessment guides. To assess the cost estimate, we reviewed the HPIB’s January 2018 lifecycle cost estimate used to support the program’s initial cost baselines, Coast Guard and Navy documentation supporting the estimate, relevant program briefs to Coast Guard leadership, and HPIB program documentation containing cost, schedule, and risk information. We met with Naval Sea Systems Command (NAVSEA) officials responsible for developing the cost estimate to understand the processes used by the cost estimators, clarify information, and request additional documentation to support the estimate. Because we did not have direct access to the HPIB cost model, we observed portions of the model during a presentation and discussion with Navy cost estimators. We also reviewed the Naval Center for Cost Analysis’ September 2017 independent cost estimate for the HPIB program, the DHS Cost Analysis Division’s January 2018 independent cost assessment of the HPIB lifecycle cost estimate, and DHS Office of Systems Engineering’s January 2018 technical assessment of the HPIB program. We also conducted interviews with officials from the Naval Center for Cost Analysis, DHS Cost Analysis Division, and the DHS Office of Systems Engineering. To assess the program’s schedule, we compared the HPIB program’s schedule, including the program’s initial schedule baselines, delivery schedules from the HPIB’s request for proposals for the detail design and construction contract, and integrated master schedule, to selected GAO best practices for project schedules, including establishing the duration of activities, ensuring reasonable total buffer or margin, and conducting a schedule risk analysis. To specifically assess the HPIB lead ship’s 3- year construction schedule estimate, we reviewed the Coast Guard’s and the Navy’s analysis supporting the HPIB schedule. We did not assess the reliability of the historical ship construction data the Coast Guard and Navy used for this analysis. We also compared the HPIB lead ship’s 3- year construction schedule to historical construction cycle times of lead ships among a nongeneralizable sample of major Navy and Coast Guard shipbuilding programs. We selected programs that were active within the last 10 years and have completed construction of the lead ship. We also included the Coast Guard’s Healy medium polar icebreaker, even though it is not a recent shipbuilding program, because it is the most recent polar icebreaker to be built in the United States. We excluded the Coast Guard Fast Response Cutter, Navy submarines, and Navy aircraft carriers because we determined that their size and complexity did not make them reasonable comparisons to the HPIB for construction times. This resulted in an analysis of construction schedules for 10 shipbuilding programs. We obtained data on these programs’ construction schedules from program documentation, such as acquisition program baselines, Navy selected acquisition reports, and Navy and Coast Guard budget documentation. We selected only lead ships for comparison because we have found in our prior work that schedule delays are amplified for lead ships in a class. Lead ships are thus more comparable to the HPIB lead ship than follow- on ships. We reviewed ship displacement data from the Naval Vessel Registry and the Coast Guard to control for the size of the ships. To assess the reliability of Naval Vessel Registry data, we reviewed the Navy’s data collection and database maintenance documentation, cross- checked select data across Navy websites, and interviewed cognizant Navy officials regarding internal controls for the database. We determined the ship displacement data were reliable for our purposes. To assess the degree to which the 6-month schedule margin that the HPIB baseline affords the lead ship is in keeping with historical ship delivery delays, we reviewed Coast Guard, Navy, and DHS acquisition documentation from a nongeneralizable sample of major Navy and Coast Guard shipbuilding programs. We selected programs active within the last 10 years and analyzed changes in lead ship delivery dates. We excluded Navy submarines and aircraft carriers because we determined that their size and complexity did not make them reasonable comparisons to the HPIB for delivery delays. We included programs that have not yet delivered their lead ships. This resulted in an analysis of construction schedules for 12 shipbuilding programs. For delivered ships, we used the actual delivery date; for ships not yet delivered, such as the Offshore Patrol Cutter and DDG 1000, we used the most recent, planned delivery date in the program baseline. To determine the status of the HPIB program’s contracting efforts and funding considerations, we reviewed the program’s acquisition plan, March 2018 request for proposals and subsequent amendments, certification of funds memorandum, budget justifications, lifecycle cost estimate, and the Coast Guard’s fiscal year 2019 Capital Investment Plan. We also interviewed knowledgeable officials from the Coast Guard’s Office of Budget and Programs, NAVSEA Contracts Directorate, NAVSEA Comptroller Directorate, and the Office of the Assistant Secretary of Navy’s Financial Management and Comptroller. For all objectives, we reviewed relevant DHS and Coast Guard policies and interviewed knowledgeable officials from DHS, the Coast Guard’s and the Navy’s HPIB integrated program office, and ship design team. We conducted this performance audit from August 2017 to September 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Shipbuilding Phases There are four primary phases in shipbuilding: pre-contracting activities and contract award, detail design and planning, construction, and post- delivery activities (see table 6). Appendix III: Summary of Results of Heavy Polar Icebreaker Program’s Cost Estimate Assessed against GAO’s Best Practices The GAO Cost Estimating and Assessment Guide (GAO-09-3SP) was used as criteria in this analysis. Using this guide, GAO cost experts assessed the heavy polar icebreaker (HPIB) program’s lifecycle cost estimate against measures consistently applied by cost-estimating organizations throughout the federal government and industry that are considered best practices for developing reliable cost estimates. For our reporting purposes, we grouped these best practices into four categories—or characteristics—associated with a reliable cost estimate: comprehensive, accurate, well documented, and credible. A cost estimate is considered reliable if the overall assessment ratings for each of the four characteristics are substantially or fully met. If any of the characteristics are not met, minimally met, or partially met, then the cost estimate does not fully reflect the characteristics of a high-quality estimate and cannot be considered reliable. After reviewing documentation the Navy submitted for its cost estimate, conducting interviews with the Navy’s cost estimators, and reviewing other relevant HPIB cost documents, we found the HPIB lifecycle cost estimate substantially met three and partially met one characteristic of reliable cost estimates. We determined the overall assessment rating by assigning each individual rating a number: Not Met = 1, Minimally Met = 2, Partially Met = 3, Substantially Met = 4, and Met = 5. Then, we took the average of the individual assessment ratings to determine the overall rating for each of the four characteristics. The resulting average becomes the Overall Assessment as follows: Not Met = 1.0 to 1.4, Minimally Met = 1.5 to 2.4, Partially Met = 2.5 to 3.4, Substantially Met = 3.5 to 4.4, and Met = 4.5 to 5.0. See table 7 for a high level summary of each best practice and the reasons for the overall scoring. Appendix IV: Comments from the Department of Homeland Security Appendix V: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition the contact named above, the following staff members made key contributions to this report: Rick Cederholm (Assistant Director), Claire Li (Analyst-in-Charge), Peter Anderson, Brian Bothwell, Juaná Collymore, Laurier Fish, Kristine Hassinger, Karen Richey, Miranda Riemer, Roxanna Sun, David Wishard, and Samuel Woo.
target: To maintain heavy polar icebreaking capability, the Coast Guard and the Navy are collaborating to acquire up to three new heavy polar icebreakers through an integrated program office. The Navy plans to award a contract in 2019. GAO has found that before committing resources, successful acquisition programs begin with sound business cases, which include plans for a stable design, mature technologies, a reliable cost estimate, and a realistic schedule. Section 122 of the National Defense Authorization Act for Fiscal Year 2018 included a provision for GAO to assess issues related to the acquisition of the icebreaker vessels. In addition, GAO was asked to review the heavy polar icebreaker program's acquisition risks. This report examines, among other objectives, the extent to which the program is facing risks to achieving its goals, particularly in the areas of design maturity, technology readiness, cost, and schedule. GAO reviewed Coast Guard and Navy program documents, analyzed Coast Guard and Navy data, and interviewed knowledgeable officials. The Coast Guard—a component of the Department of Homeland Security (DHS)—did not have a sound business case in March 2018, when it established the cost, schedule, and performance baselines for its heavy polar icebreaker acquisition program, because of risks in four key areas: Design. The Coast Guard set program baselines before conducting a preliminary design review, which puts the program at risk of having an unstable design, thereby increasing the program's cost and schedule risks. While setting baselines without a preliminary design review is consistent with DHS's current acquisition policy, it is inconsistent with acquisition best practices. Based on GAO's prior recommendation, DHS is currently evaluating its policy to better align technical reviews and acquisition decisions. Technology. The Coast Guard intends to use proven technologies for the program, but did not conduct a technology readiness assessment to determine the maturity of key technologies prior to setting baselines. Coast Guard officials indicated such an assessment was not necessary because the technologies the program plans to employ have been proven on other icebreaker ships. However, according to best practices, such technologies can still pose risks when applied to a different program or operational environment, as in this case. Without such an assessment, the program's technical risk is underrepresented. Cost. The lifecycle cost estimate that informed the program's $9.8 billion cost baseline substantially met GAO's best practices for being comprehensive, well-documented, and accurate, but only partially met best practices for being credible. The cost estimate did not quantify the range of possible costs over the entire life of the program. As a result, the cost estimate was not fully reliable and may underestimate the total funding needed for the program. Schedule. The Coast Guard's planned delivery dates were not informed by a realistic assessment of shipbuilding activities, but rather driven by the potential gap in icebreaking capabilities once the Coast Guard's only operating heavy polar icebreaker—the Polar Star —reaches the end of its service life (see figure). GAO's analysis of selected lead ships for other shipbuilding programs found the icebreaker program's estimated construction time of 3 years is optimistic. As a result, the Coast Guard is at risk of not delivering the icebreakers when promised and the potential gap in icebreaking capabilities could widen.