Columbia Class Submarine: Immature Technologies Present Risks to Achieving Cost Schedule and Performance Goals
Released: 2017-12-21T00:00:00

Letter

Background

The U.S. strategic nuclear deterrent is spread among three legs, as depicted in figure 1.

DOD has continued to reinforce the high priority of the Columbia class program to the nation’s long-term defense. SSBNs are designed to maximize stealth to remain undetected while on patrol at sea. This survivability gives the United States a credible ability to retaliate if faced with an attack targeting other legs of the triad, and explains DOD’s decision to ultimately deploy up to 70 percent of the nation’s nuclear warheads on SSBNs.

As stated in its April 2010 Nuclear Posture Review Report, DOD determined that ensuring a survivable U.S. deterrent requires continuous at-sea deployments of SSBNs in both the Atlantic and Pacific oceans, as well as the ability to surge additional submarines in crisis. Currently, 14 Ohio class SSBNs provide the sea-based strategic deterrent. The Navy commissioned the lead ship of this fleet in 1981. The first Ohio class SSBN to retire—SSN 730—will leave service in 2027 and plans are to retire one per year following this. When these submarines retire, they will have been in service over 40 years, longer than any previous submarines. Navy officials have stated that the legacy Ohio fleet cannot be life-extended any longer than what is planned due to aging issues.

The U.S. Strategic Command (STRATCOM) retains operational control of the strategic triad and determines how many SSBNs are needed to patrol on a day-to-day basis. STRATCOM and the Navy have determined that 10 operationally available SSBNs are needed to meet mission requirements. As a result, the lead Columbia class submarine must be available for its first deterrent patrol in the first quarter of fiscal year 2031 to coincide with the planned 2031 retirement of SSN 734, or the Navy will not have 10 operationally available SSBNs, thereby requiring DOD to identify other steps to ensure it can meet current deterrent requirements.

The Navy expects that it can meet mission requirements with 12 Columbia class submarines carrying 16 missile tubes (equating to a total of 192 available tubes) in lieu of 14 Ohio class submarines carrying 24 tubes (336 total available tubes). Currently, it takes 14 Ohio class submarines to provide 10 operationally available SSBNs due to maintenance needs that can take up to 4 submarines out of the patrol rotation at any given time. The Navy plans to reduce the number and duration of required maintenance periods for the Columbia class, allowing just 12 Columbia class submarines to provide the required 10 operational submarines at all times. Between fiscal year 2031-2040, the Navy plans to have a mix of 10 operationally available Columbia and Ohio class submarines. In fiscal year 2041, with the retirement of the final Ohio class submarine, this is to increase to 11 Columbia class, and finally to 12 operationally available Columbia class submarines by fiscal year 2042.

Columbia Class Technology Efforts

The Columbia class program is comprised of several major lines of effort—hull and supporting systems, the strategic weapons system; and the nuclear reactor-based propulsion plant—which are managed by different program offices, as depicted in figure 2.

The Navy is introducing new technologies to improve capabilities where required while leveraging systems from existing submarine programs— the Virginia and Seawolf attack submarines and the Ohio class SSBNs— in order to ensure commonality with the submarine fleet and reduce development needs for the Columbia class to limit technical risk. For example, the program is re-using over 19,000 Virginia class standard parts including fittings, valves, and switches and leveraging the Navy’s Submarine Warfare Federated Tactical System program, which integrates more than 40 independent electronics systems into a common combat system for use by multiple program offices.

The Navy has identified several key technical efforts for the Columbia class program: (1) the Common Missile Compartment, (2) Integrated Power System, (3) Stern Area System, and (4) propulsor. Other systems that we consider key technical efforts include the nuclear reactor and the coordinated stern, a system-of-systems that includes the propulsor and submarine maneuvering components. These areas are depicted in Figure 3 and defined below.

Since 2008, the United States and the United Kingdom (U.K.) have been jointly developing a common system to house the tubes that will carry submarine launched ballistic missiles. Columbia class SSBNs and U.K.

SSBNs will carry the Trident II D-5 missile for the first portion of their respective operational lives; the U.S. missiles armed with nuclear warheads which are maintained by the Department of Energy (DOE). Figure 4 shows a notional example of the CMC.

In addition to the missile tubes, the CMC also provides systems to support the missiles and the launch equipment, including power, cooling, gas venting, and launch hardware and software. The Navy’s Strategic Systems Program is responsible for CMC development efforts.

Integrated Power System (IPS) and Nuclear Reactor

The IPS includes an electric drive system to propel the submarine through the water, unlike other current U.S. submarines which use a mechanical drive system. IPS is powered by the nuclear reactor, which is a separate system. As shown in figure 5, with a nuclear electric drive system, steam from the nuclear reactor turns a turbine creating electricity, which is then directly used to power electric motors. This is in contrast with a nuclear mechanical propulsion system, where steam from the nuclear reactor turns a turbine creating high-speed rotation; a reduction gear then slows the speed of this rotation to a speed that is suitable for use by the propulsor.

To provide power to the electric drive, the Columbia class nuclear propulsion plant relies on a life-of-the-ship reactor core—called S1B—that is planned to remain in service without refueling, almost 10 years longer than current U.S. Navy nuclear reactors. The Virginia class also uses a life-of the-ship reactor core, but the Columbia class reactor needs to be more powerful to drive the larger submarine, and needs to last longer to allow for the 42.5-year Columbia class service life of versus 33 years for the Virginia class. By using a life-of-the-ship reactor, the Columbia class will not require a mid-life refueling. This will reduce the mid-life maintenance period from 27 months for Ohio class to 16 months for Columbia class. This reactor is being developed by the Naval Nuclear Propulsion Program (also known as Naval Reactors) and the Naval Nuclear Laboratory (operated by Bechtel Marine Propulsion Corporation).

Stern Area System (SAS)

SAS is a technical feature of the stern that is comprised of three subcomponents; details of which are classified.

Propulsor/Coordinated Stern

The Columbia class will use a propulsor instead of a propeller to drive the submarine through the water. The design of the propulsor relies on several other technical features that form a system-of-systems, sometimes referred to as the coordinated stern. The coordinated stern is where the rudder and other control surfaces are mounted; these control surfaces are used for submarine maneuvering and are critical to submarine performance. The coordinated stern consists of interrelated technology elements, including the propulsor and advanced propulsor bearing, the stern control surface configuration, and the propulsor shaft and bearing.

The propulsion shaft and bearing connects the propulsion system to the propulsor, transferring energy from the propulsion system to the propulsor to drive the submarine through the water. The Navy plans to use a new design “X-stern” configuration instead of the cruciform stern used in other submarines. Figure 6 depicts the major components of the coordinated stern, omitting a depiction of the classified Stern Area System.

Acquisition Strategy for the Columbia Class

The Navy expects to require over $267 billion (then-year dollars) in total life-cycle costs for the Columbia class program. Figure 7 shows the break-down of this amount between operations and support costs and acquisition costs, as well as the elements comprising the $128 billion in acquisition costs.

The approximately $128 billion total acquisition cost includes funding the Navy expects it will need to research, develop, and build its Columbia class SSBN.

Due to their size and complexity, submarines require funding for design, long-lead materials (such as nuclear propulsion plant components), and construction over many years. To accomplish these activities, the Navy awards contracts over several phases of design and construction. Figure 8 outlines major acquisition plans for the Columbia class.

In 2014, Congress created a National Sea-based Deterrence Fund to provide DOD with greater discretion to fund the design, construction, and purchase of the Columbia class. Since then, Congress has provided the Navy with enhanced acquisition authorities to buy and construct submarines and certain key components early, in bulk, and continuously.

The Columbia class program entered the Technology Development phase of the defense acquisition process in January 2011. The schedule to acquire the Columbia class was shifted in 2011 when the Navy decided to delay the start of construction of the lead submarine by 2 years—from 2019 to 2021—due to budget constraints. The first patrol date for the lead ship was also shifted from fiscal year 2029 to fiscal year 2031. In January 2017, the Columbia class program achieved Milestone B—considered the official start of a DOD acquisition program—and moved into the Engineering and Manufacturing Development phase of the acquisition process. The program does not envision holding a Milestone C, which typically denotes a program’s approval to enter the production and deployment phase as shown in figure 9, but does plan to have an OSD- level review prior to authorizing the construction of the lead ship.

Shipbuilding programs have slightly different decision points than other DOD weapon systems, partly because of the timing of the Milestone B decision for ships. Milestone B for ship programs usually occurs after development of ship specifications and system diagrams is well under way.

As part of the Columbia class Milestone B decision, OSD approved a Low Rate Initial Production quantity of 12 submarines, the total quantity expected for the class. According to the Navy, the program awarded a $5.1 billion detail design contract to Electric Boat in September 2017 for work including design completion, component and technology development, and prototyping efforts. Detail design is typically funded with Shipbuilding and Conversion, Navy funds (the Navy’s procurement fund for buying ships) and represents a further refinement of the design of the ship and ultimately generation of work instructions needed by the shipyard in advance of lead ship construction. The program was granted approval to begin early detail design work in January 2017.

In shipbuilding, the design phase generally encompasses three activities: basic design, functional design, and detail design. These steps occur after the Navy sets the technical requirements for the ship. At a high level: basic design serves to outline the steel structure of the ship; functional design routes distributive systems—such as electrical or piping systems—throughout the ship; a three-dimensional (3D) computer-aided design model is often generated; and detail design completes the design work for even the lowest-level items, and ultimately furnishes the work instructions for the shipyard workers to use in constructing the ship. During this phase, all aspects of the ship are defined, and two-dimensional paper or 3D electronic drawings (also called work instructions) are generated.

For the Columbia class program, the Navy defines design in two phases: arrangements, which program officials describe as a combination of basic and functional design; and disclosures, which they describe as a combination of detail design and generation of work instructions. Figure 10 shows the phases of design for the program as compared with typical surface ship terminology.

Two shipbuilders—General Dynamics Electric Boat and Huntington Ingalls Industries Newport News—are responsible for designing and building nuclear submarines. For the Columbia class program, Electric Boat is the prime contractor for design and construction, with Newport News as a subcontractor. Similar to the Virginia class program, each shipyard will build modules of the submarine, but Electric Boat will be responsible for final delivery of the submarine to the Navy.

Technology Readiness Assessment

For more than a decade, our work on major acquisitions has shown that part of an effective management process is assessing how far a technology has matured and how it has been demonstrated, which indicates the technology’s readiness to be integrated into a system and the degree of program risk. DOD acquisition instruction requires that programs complete a technology readiness assessment (TRA) at Milestone B. A TRA is a systematic, evidence-based process that evaluates the maturity of hardware and software technologies critical to the performance of a larger system or the fulfillment of the key objectives of an acquisition program. A reliable TRA illuminates concerns and serves as the basis for realistic discussions on how to mitigate potential risks as programs move from the early stages of technology development.

TRAs do not eliminate technology risk but, when done well, can illuminate concerns and serve as the basis for realistic discussions on how to mitigate potential risks as programs move from the early stages of technology development, where resource requirements are relatively modest, to system development and beyond, where resource requirements are often substantial. In addition, TRAs help legislators, government officials, and the public hold government program managers accountable for achieving their technology performance goals.

A main element of a TRA is the identification of critical technology elements (CTE) and assessment of the appropriate Technology Readiness Level (TRL), used to measure the readiness of technologies to be incorporated into a weapon or other type of system. TRLs range from 1 (least mature) to 9 (most mature), as shown in table 1.

Current DOD guidance assigns the program manager responsibility for identifying CTEs. The program manager identifies possible technologies, then, in consultation with officials from the Assistant Secretary of Defense for Research and Engineering—ASD(R&E)—and with the program executive office and component acquisition executive approval, identifies the subject matter experts needed to perform the TRA. For the Columbia class TRA, the expert team was comprised of Navy program management and technical personnel. ASD(R&E) reviews the list of critical technologies provided by the program manager and recommends technologies to add or delete. Ultimately, the program submits the TRA report to ASD(R&E), who independently assesses the maturity of the technologies. The ASD(R&E) prepares a memorandum based on the assessment that is transmitted to the milestone decision authority, along with the TRA Report.

The TRA is also an element of the Milestone B approval process. Section 2366b, title 10, U.S. code states that a major defense acquisition program may not receive Milestone B approval until the milestone decision authority has, among other things, certified that the CTE has been demonstrated at a TRL 6. A program may request a waiver from OSD if the maturity provision cannot be met. The statute requires that:

Every waiver determination must be submitted in writing to the congressional defense committees within 30 days after the waiver request by the program is authorized.

The milestone decision authority reviews the program not less often than annually until the milestone decision authority determines that the program satisfies all certification and determination components.

In addition, in 2015 Congress required program acquisition strategies to include a comprehensive approach to risk management, including the consideration of techniques such as technology demonstrations and decision points for disciplined transition of planned technologies into programs or the selection of alternative technologies.

Recognizing the importance of the TRA to risk management, in 2016, GAO developed a Technology Readiness Assessment Guide. This guide has two purposes: (1) to describe generally accepted best practices for conducting effective evaluations of technology developed for systems or acquisition programs; and (2) to provide program managers, technology developers, and governance bodies with the tools they need to more effectively mature technology, determine its readiness, and manage and mitigate risk. As noted above, we developed the guide by drawing heavily from DOD, DOE, and NASA best practices, terminology, examples, and credible resources, materials, and tools developed and applied by experts and organizations in order to capture the current thinking on technology readiness and maturity. In our guide, we identify criteria for a CTE, namely that it is a technology that is “new or novel, and needed for a system to meet its anticipated operational performance requirements; or that poses major cost, schedule, or performance risk during design or demonstration”. According to our guide, re-used existing technologies can also become critical if they are being used in a different form, fit, or function—as is the case with the propulsor and coordinated stern.

Major Funding Commitments Planned, but Reporting on the Progress of Several Key Immature Technologies Is Not Required

Several key technical efforts remain immature as the Columbia class program moves into its design phase—a practice counter to best practices we have previously identified. These efforts include the integrated power system, nuclear reactor, propulsor/coordinated stern, stern area system, and common missile compartment. While the Navy made progress in some areas—such as prototyping efforts for the missile compartment and nuclear reactor—all of these systems continue to require development and testing to mature them to TRL 7, the point at which GAO’s technology readiness guide considers a technology mature. Any challenges in development could put the program at risk of costing more, taking longer to develop, or jeopardizing the program’s ability to meet its expected performance requirements. However, the Navy identified only two of the submarine’s technologies as “critical” in the program’s 2015 TRA, thereby underrepresenting the technology risk in the program. Underreporting technical risks can hinder Congress’ and other decision makers’ full understanding of the program’s progress. This is especially important because the Navy has already requested $1.6 billion for advanced procurement and recently awarded the detail design contract. Moreover, there is no requirement that the Navy report to Congress on its progress in developing and testing the technologies until after the program completes its production readiness review in May 2020 after the Navy requests another $8.7 billion in funding for the construction of the lead submarine.

Several Technologies Remain Immature as Detail Design Begins

Demonstrating Technology Maturity Based on our work on best practices in weapon system acquisitions, we have previously recommended that programs fully mature technologies to TRL 7—versus TRL 6 as required by DOD—prior to passing Milestone B and entering the engineering and manufacturing development phase. TRL 7 represents a major step up from TRL 6, requiring demonstration of an actual system prototype in an operational environment such as in an aircraft, vehicle, or space. We have previously identified that demonstrating technologies in an operational environment provides a higher level of technology understanding and reduces risk prior to starting product development. DOD has historically disagreed with this recommended practice. added that modeling and simulation should be considered appropriate in some cases in lieu of actual prototype testing.

While the Navy has made progress in reducing technical risks in many areas, such as starting construction of the first CMC, the program (according to the Navy) awarded a detail design contract in September 2017, with several key technologies not yet at a TRL 7.

The nuclear reactor, IPS, propulsor and coordinated stern, and SAS all have potentially significant effects on design and construction of the Columbia class because they encompass much of the design and physical structure of the submarine. Based on our analysis, we found that IPS, SAS, the propulsor and coordinated stern are not yet at a TRL 7, as depicted in figure 11. The nuclear reactor and CMC are further along in prototyping work but still require testing in an operational environment to achieve a TRL 7.

If any of these systems do not develop as planned, the Navy and the shipyards could be required to complete some redesign, or, if risks manifest later, they may force costly workarounds or construction rework. In addition, these systems also enable many performance attributes ranging from weapon launch to speed and maneuverability, so performance could be negatively affected. The status of these technologies is discussed in detail below.

Integrated Power System

According to officials from Naval Reactors, the permanent magnet motor- based electric drive system—a key component of IPS for the Columbia class—is at a TRL 6, below the TRL 7 recommended by our work on best practices. Naval Reactors has yet to develop an IPS prototype that is near or at the planned operational system configuration (integrated and full-size) and has been tested in an operational environment. The Navy has experimented with electric drive technology on submarines in the past with two now-decommissioned nuclear-powered attack submarines, but these submarines used different motor technology than what is planned for the Columbia class, and thus are not representative. The T- AKE 1 Lewis and Clark class of dry-cargo ammunition ships and DDG 1000 Zumwalt class destroyer are current U.S. Navy electric drive ships in operation, but these two systems are somewhat different than what is planned for the Columbia class and neither is powered by a nuclear reactor. The Navy is currently developing the IPS and producing a number of pre-production prototypes.

Naval Reactors officials told us that they are confident that the IPS will meet requirements based on 20 years of development and testing of the underlying permanent magnet motor technology. They also noted that this technology is proven based on testing of the smaller-scale prototype motor to validate the main propulsion motor design. However, Naval Reactors is still developing and producing the system’s major components. Testing of a full-scale prototype under full power, which we would consider evidence that the technology is mature, is not scheduled to occur until fiscal years 2018-2020. In a land-based test facility, the Navy plans to integrate all the IPS systems in a ship-representative layout. Successful completion of this testing is an important step in mitigating risk. In contrast, the DDG 1000 program only tested its electric drive system at the land based test facility at one-half of the ship’s power generation and electric propulsion system configuration, and as a result performance problems were not discovered until well after installation and when system testing on the ship was run at full power. Thus, the Navy’s planned full-scale prototype testing for Columbia class should prevent a similar experience, since it will test a full-sized and full-power system rather than a partial system.

Nuclear Reactor

According to officials from Naval Reactors, as a result of its statutory mandate, its programs follow a different development process than typical DOD programs and do not use documents typical of other Navy programs, such as an Integrated Master Schedule or a Test and Evaluation Master Plan. Instead, officials from Naval Reactors told us that they use a rigorous process to assess, manage and control technical risk during development and testing to manage its day-to-day technical efforts. Based on descriptions provided by Naval Reactors officials, the Navy has been operating a Columbia-like experimental reactor in a land- based environment for many years to demonstrate some Columbia class submarine systems. Naval Reactors officials said that this experience gives them confidence that the Columbia class reactor will be delivered to the shipyard on time and will meet all requirements.

Naval Reactors has design and development work remaining before it awards the contract for reactor core production in fiscal year 2019. Naval Reactors budget documentation shows that reactor design work is planned to be 65 percent complete in fiscal year 2018. While we recognize that it would not be realistic to expect Naval Reactors to test the reactor in a submarine to achieve a TRL 7, a completed design would still be required to produce a final configuration to demonstrate technology maturity.

Propulsor/Coordinated Stern

Neither the propulsor nor other related components of the coordinated stern have been demonstrated through testing in a near or planned operational system configuration, a key element for achieving TRL 7. Navy officials told us that the propulsor effort is based on prior experience with propulsors and that it will resemble the Virginia-class propulsor design. However, according to Navy documentation, the propulsor will be different in form, fit, and function than prior propulsors, and the final configuration has yet to be selected or tested. Specifically, the following components require additional design work and testing prior to demonstrating a representative prototype:

Propulsor: The Navy is working with various partners to refine two different high-level propulsor designs. The program also faced a year delay in completing the first phase of design work, which subsequently delayed large-scale vehicle testing. Further, the Navy still has to complete large-scale prototype testing of different propulsor designs that are being evaluated for an eventual down-select to one vendor for production.

Propulsor shaft: The system that connects the propulsion to the motors—which the Navy states is similar to shafting systems used on previous submarine classes but with different materials and size and weight—is still in concept and preliminary design phases. Main shaft design development and testing is being performed to select materials and inform design efforts.

Advanced propulsor bearing: The Navy has yet to complete the preliminary design of the advanced propulsor bearing, with prototype test in a full scale configuration planned to begin in fiscal year 2019. Navy officials told us that they believe that the final design and material selections will exceed the reserved weight and size margins of the shafting or bearing system.

X-stern: the final X-stern configuration has yet to be tested with a final design propulsor.

Our assessment of the propulsor and coordinated stern system design indicates that it is not yet mature enough to provide the basis for a prototype in final form, fit, and function—key elements of achieving TRL 7.

Stern Area System

The Navy identified the SAS as a TRL 4 at Milestone B. The preliminary design review for the SAS is planned for March 2018. This review establishes the baseline (hardware, software, human/support systems) and underlying architectures to ensure that the system has a reasonable expectation of satisfying requirements within the current budget and schedule. The critical design review—a technical review that ensures that a system can proceed into fabrication and demonstration and can meet stated performance requirements within cost, schedule, and risk—is not planned until March 2020.

A TRL 4 represents a relatively low level of maturity compared to the eventual system. At this low level of maturity, there are no assurances that the SAS will work as planned, which would likely result in the Columbia class not meeting certain requirements or in cost and schedule increases. The Navy plans to hold a critical design review for SAS in fiscal year 2019. The Navy has identified existing fleet technologies as backups for two SAS components, but officials noted that if these are used the submarine will not meet current requirements. According to the program office, there is no backup technology for one other SAS component, and, if that element—currently a TRL 4—does not develop as planned, it will be omitted, meaning that the program will lack that capability.

Specific details of SAS are classified and cannot be included in this report.

Common Missile Compartment

The shipbuilders and the Navy have described CMC as complex to build. The Navy and the two shipyards—with consultation from the United Kingdom, which will also leverage the CMC design on its new SSBN— have conducted risk-reducing prototyping work and are building a representative CMC to demonstrate production processes. In fact, Columbia class representative missile tubes will be first installed on a United Kingdom submarine, scheduled for mid-2020. The Navy has plans for a robust land-based test procedure for both the missile tubes and the CMC as a system that will provide an operationally similar environment to a submarine; however, this testing has yet to start and will not conclude for several years.

The Navy Has Not Appropriately Identified Technologies as Critical, Which Underrepresents the Program’s Technical Risk

While the Navy conducted the 2015 Columbia class TRA in accordance with a DOD-approved plan, it did not follow our identified best practices for identifying all critical technology elements (CTE), resulting in an underrepresentation of the technical risk facing the program. Specifically, the TRA only identified 2 CTEs: the SAS and a carbon dioxide removal system. CTEs are required to be at TRL 6 at Milestone B (the official start of a program). For the Columbia class program, OSD approved Milestone B in January 2017. The Navy received a waiver at Milestone B for the SAS because the system was still immature, as discussed above. The carbon dioxide removal system has matured since the TRA following demonstration on an operational submarine, and no longer requires active risk mitigation efforts.

We compared the Navy’s 2015 Columbia class TRA to criteria documented in GAO’s TRA Guide and DOD’s own guidance. In doing so, we found that 4 additional key technical efforts—IPS, nuclear reactor, and propulsor/coordinated stern, and the CMC—meet the criteria for a CTE. Since the Navy did not identify these technologies in the TRA, it also did not assign them a TRL. Their exclusion is significant because the 2015 TRA represents a key independent review and technical risk assessment used by DOD to certify to Congress that the Columbia class program’s technologies had been demonstrated in a relevant environment (TRL 6) at Milestone B. Because not all of the CTEs were identified, DOD and Congress lack an important oversight tool for assessing technology maturity and evaluating program risk. Further, this certification is the only required reporting on technology development prior to the Navy requesting authorization for construction of the lead ship. Some of the concerns that we identified are discussed in detail below.

Conflicting Criteria for Identifying Critical Technologies

The team responsible for preparing the 2015 Columbia class TRA did not identify all appropriate CTEs because it used a more restrictive definition of a CTE than that recommended in our best practices guide and DOD’s 2011 TRA guide. Table 2 compares the criteria in the three sources.

As reflected in table 2, not only does the Navy’s TRA definition require a technology to meet a number of criteria to be considered a CTE, it also has to be considered a technology development effort. According to the Columbia class program office, the TRA team based this definition on a 2011 OSD AT&L memorandum issued contemporaneously with the 2011 TRA guidance that states: “TRAs should focus only on technology maturity, as opposed to engineering and integration risk.” However, our analysis of this memo found that it also directs programs to use DOD’s TRA guidance and CTE definition, which are broader and more consistent with our definition of a CTE. The 2015 Columbia class TRA does not further define what constitutes a technology development effort, with the Navy applying this as a criterion without defining what the criteria actually meant. Moreover, the TRA does not provide any definition or criteria for what it considers engineering and integration risk. We determined that the Navy under-identified program technical risks because the Navy’s criteria were more restrictive than GAO’s CTE definition.

Several Critical Technologies Not Identified

We further assessed the specific technologies in the Columbia class program against our technology readiness criteria for a CTE, as shown in table 3.

As shown in table 3, by applying the additional “technology development effort” criteria in the 2015 Columbia class TRA, the TRA team eliminated several systems from CTE consideration without criteria or a definition of what constitutes a technology development effort. Some of these systems were previously identified as CTEs in other recent Navy documentation. The TRA team did not identify the nuclear reactor as a CTE because this system is under the cognizance of Naval Reactors and not the Columbia class program office. Officials from Naval Reactors told us that they do not conduct TRAs, but rather follow a different and more iterative process to manage their technology development efforts.

While the Navy did not identify all of the program’s CTEs as compared with the TRA criteria in our guide, it is tracking these efforts to manage technology risks. For example, 3 of the 4 CTEs we identified are also identified in Navy documents as “key technical efforts” with active risk mitigation plans. We will continue to track the progress of these efforts in our future work.

Required Report to Congress on Technology Efforts Will Not Occur Until after Lead Ship Authorization

As the Columbia class program moves into its detail design and construction phase, it will be more than 2 years before the next requirement for a formal DOD report to Congress on the progress of the technology efforts. This will occur at some point after the program’s Production Readiness Review is completed in May 2020. In the meantime, the Navy plans to request another $8.7 billion (in addition to the $1.6 billion already requested) for lead ship construction. If a typical budget schedule is followed, this request will come before Congress in February 2020. The Navy plans to begin construction of the lead submarine starting in fiscal year 2020. Congress will be asked to approve lead ship construction absent key information on the maturity of the critical technologies that, at present, are not up to the maturity levels that would provide assurance they will work as intended. Without additional updates on the progress of technology maturity between now and 2020, we believe Congress will not have information it needs to evaluate technical risk in advance of the Navy’s requests for considerable increases in program funding. As previously discussed, there is currently no DOD requirement to submit such reports to congressional oversight committees.

The Navy Plans to Leverage Completed Design to Mitigate Aggressive Schedule, but Ongoing Technology Development Likely to Undermine This Goal

The Navy is prioritizing design completion before starting construction, which is a good practice that is in accordance with our work on best practices because it helps reduce cost and schedule challenges in construction. However, since some of the key technologies are not fully matured, detail design work is proceeding with notional or placeholder data representing these key systems. As a result, the design will likely remain immature once construction starts in fiscal year 2021. We have previously reported that concurrency of technology development and design increases the risk of design rework—or having to make modifications to design drawings to accommodate any changes needed as a result of technologies changing size, shape, or weight as they mature—and potentially can result in negative cost and schedule impacts. Further, the Navy faces an aggressive production schedule in order to deliver the lead submarine by fiscal year 2031, which will be required to prevent a gap in U.S. nuclear deterrent capabilities. According to our analysis of previous submarine program schedules, the Columbia class program’s schedule is aggressive in its expected short duration to build the lead submarine. The program office intends to mitigate this schedule challenge, in part, by starting construction of portions of the submarine earlier than initially planned. If this early construction occurs and the Navy does not alter design plans, construction of some parts of the lead submarine could outpace a finalized design for developing other components, which increases the risk of rework during construction and could further delay completion.

Consistent with Best Practices, Program Has Prioritized Design Completion, but Immature Technologies May Compromise Design Maturity

The Columbia class program is prioritizing a high level of design completion prior to the start of construction of the lead submarine of the class. The program plans to complete 100 percent of design arrangements and 83 percent of design disclosures prior to the start of construction of the lead submarine. In our 2009 report on best practices in shipbuilding, we identified design maturity as important step in reducing cost and schedule risk. As such, we recommended that the design be stabilized through completion of basic and functional design and 3D product modeling prior to the start of construction for a new ship. Because, as mentioned previously, the Navy defined design arrangements on the Columbia class program as being equivalent to basic and functional design, having 100 percent of the arrangements completed prior to the start of Columbia class construction would meet the intent of our prior recommendation.

Further, our analysis found that the Columbia class program’s planned level of design completion prior to starting construction is much higher than most recent Navy shipbuilding programs. For example, the Virginia class attack submarine program started construction with only 43 percent of the design complete compared with a planned 83 percent completion for the Columbia class. The Columbia class program also plans to have a 52 week buffer between the completion of design for an area of the submarine and the start of construction on that area, which is intended to allow time to address any challenges that may arise and thus minimize schedule delays. Additionally, the Navy plans to have all components fully developed 8 months before they are required in the shipyard, which will provide some additional schedule buffer to address challenges before the components are actually needed for construction.

To facilitate design completion, the Navy made a commitment at the start of the program to set realistic and reasonable requirements and to keep those requirements stable throughout the program. This approach is also in keeping with our previously identified best practices, which highlight the importance of demonstrating balance among program requirements, technology demands, and cost considerations. The Columbia class program has not had any significant requirements changes since DOD’s Joint Requirements Oversight Council validated the Capability Development Document in 2015. Setting realistic and reasonable requirements also permitted the Navy and shipyards to reuse some design elements for components of the submarine that are similar in design and function to the Virginia class instead of requiring new design work. Similarly, the program has worked to keep stable ship specifications to minimize design disruptions.

The technical specifications for the ship have been set since 2014, and the program manager maintains personal visibility and accountability over any proposed deviations or changes to the specifications. According to the program manager, to date there have been minimal changes made to the technical baseline. These steps help to minimize design rework that can be caused by changing requirements, as was seen on the Littoral Combat Ship program, and that can lead to cost increases or scheduled delays. The program has also conducted some prototyping efforts— including building representative portions of the submarine to demonstrate that its design tool can send the correct information to the shop floor to build the ship—and has plans for more.

However, based on our analysis of the program’s current technology development plan and status, it is unlikely that the Navy’s planned 83 percent of design disclosures will be finalized at the time construction begins for the lead ship in 2021. Similar to many shipbuilding programs, the Columbia class program plans to continue to mature technologies into their final form while detail design is underway. As we have previously reported, to offset this risk, shipbuilding programs, including the Columbia class, often include design “reservations” for space, weight, power, cooling, and other key attributes to reserve a footprint for components. As contractors or government employees develop and refine technologies or systems, they provide vendor furnished information (VFI) or government furnished information (GFI) to the shipyards to update the design. Completion of the detail design of the submarine—and subsequent achievement of design stability to support a properly sequenced construction phase—requires shipbuilders to have final information on the form and fit of each system that will be installed on the ship, including the system’s weight and its demand for power, cooling, and other supporting elements.

As development proceeds on a new technology, initial assumptions about size, shape, weight, and power and cooling requirements can change, potentially significantly. These changes in VFI or GFI—if not resolved early in the design phase—can introduce considerable volatility to the design process for a lead ship. As such, in our May 2009 report, we recommended that, to attain the level of knowledge needed to retire design risk and reduce construction disruptions, complete—versus notional—VFI or GFI must be incorporated for the design to be truly stable. DOD concurred with this recommendation. We have previously reported that other Navy programs have run into difficulties, including out- of-sequence or more costly construction work, when space, weight, power, and cooling reservations are based on immature or ill-defined technologies or components that have changed in size, weight, or other attributes when they are finalized. Ramifications from such changes can ripple through much of the ship design. For example, we reported in 2009 that during construction of the Seawolf-class attack submarine, the AN/BSY-2 combat system did not fit into the space and weight reservations that the Navy had allocated within the submarine’s design. As a result, a portion of the submarine had to be redesigned at additional cost.

However, the Navy has entered the detail design phase for the Columbia class with incomplete technical data on several key components that are either significant in size relative to the submarine or spread throughout a number of spaces of the submarine. These components include IPS, the nuclear reactor, the propulsor and coordinated stern, and SAS. This situation is problematic because even if the Columbia class design is 83 percent complete, if it contains many reservations for systems that are not fully developed the design will continue to be immature and subject to change. Thus, the 83 percent completion metric may be somewhat meaningless since elements of the design are uncertain and could change because of the incomplete technology development efforts.

As shown in figure 12, the Columbia class program has entered the detail design phase with a number of technologies still in development or design finalization, which means that the VFI/GFI for these systems are not yet final. This figure also depicts our recommended knowledge points for shipbuilding programs, which align with contract award for detail design and the start of lead ship construction. The concurrency depicted between phases could be further exacerbated if the Navy pursues plans to start construction of some components early.

As is shown in figure 12, the Navy plans to continue technology development while executing detail design; this concurrency may potentially extend through construction if the Navy pursues its plans for early construction. For example, the Navy and the shipyards are currently designing the stern of the submarine—with 95 percent of stern arrangements planned to be complete by December 2017—but the final configuration of the propulsor has yet to be determined. As currently planned, the Navy will not complete prototype testing until the third quarter of fiscal year 2020, and development and design of the SAS is planned to continue until the end of fiscal year 2021—almost a year after the start of lead ship construction. The Navy believes it is managing this stern risk by controlling the interfaces through an Interface Control Document that identifies set design constraints. According to Navy officials, all aspects of the propulsor design that could impact the overall ship design such as size, weight, and arrangements of major sub- assemblies of the propulsor are already finalized, and that the systems are currently tracking to the reservation allowances. However, until a final representative prototype is tested as a system, the possibility of design changes and broader design impacts remains. Although the Navy plans to have arrangements for the stern 100 percent complete at construction start, the VFI or GFI for these important systems will not be finalized until later after these systems finish development.

Additionally, the electric drive of IPS has already experienced manufacturing problems that could compromise its ability to meet its schedule if further challenges arise. According to Naval Reactors officials, a manufacturing defect was identified in February 2017 that affected the assembly of the first production-representative propulsion motor intended for installation in the land-based test facility to prove out the integration of all the electric drive components. The officials explained that the vendor responsible for the motor is in the process of repairing the defect—a process that will take up to 9 months to complete. As a result, Naval Reactors is now executing a schedule recovery plan to regain some schedule margin. Part of this plan involves using a smaller scale prototype motor in initial land-based test facility testing to prove out system integration. This plan means that initial full-scale system testing will be conducted with a different motor, albeit one with the same electromagnetic properties. Further, this delay will leave less margin to account for any unexpected challenges encountered in developmental testing.

Aggressive Construction Schedule for Lead Submarine Unprecedented

The Columbia class program has an aggressive schedule to deliver the lead submarine in time to begin patrols in fiscal year 2031. The Navy plans for 84 months, or 7 years, to build the lead submarine. While imperatives associated with our nation’s nuclear deterrent are driving this planned schedule, our analysis shows that it is significantly shorter than what the Navy has achieved on any recent lead submarine construction effort—including during high levels of Cold War submarine production. The Navy expects that the Columbia class will be built in the same timeframe as was planned for the lead Virginia class submarine—a submarine that is one and a half times smaller and has less estimated construction man hours than the Columbia class. Figure 13 shows the estimated and actual timeframes for constructing prior lead submarines as compared with the 84 month estimate for the Columbia class lead submarine.

Further, there are industrial base implications to this aggressive schedule. The Navy and the two shipyards will be trying to attain this level of unprecedented schedule performance with the lead submarine while the shipbuilders are also starting work on the first few Virginia class submarines built in a new Block V configuration. Virginia class program officials told us that the ramp-up to building two attack submarines per year has resulted in recent cost and schedule growth at the shipyards. The addition of Block V and Columbia-class will likely create additional schedule pressures with the increase in workload required to build those submarines compared with non-Block V version submarines.

In an effort to mitigate the risks associated with its aggressive delivery schedule, the Navy is planning to start construction of a number of parts of the structure of the lead submarine years earlier than the date of lead ship authorization in fiscal year 2021. This plan, called advanced construction, would use expanded acquisition authorities provided by Congress in the National Sea-Based Deterrence Fund. The Navy and its shipbuilders intend to start construction as early as 2019 on numerous areas of the submarine’s structure. Specifically, the Navy and shipyards plan to start building the stern, bow and missile command and control module as early as 6 months before fiscal year 2021, citing the disruptive effects of delays to these three “super-modules” that are also critical to ensuring an on-time delivery. These super-modules also comprise vital areas of the submarine, including the CMC, IPS and the coordinated stern. The shipyards have proposed moving 500,000-600,000 labor hours of construction work to before ship authorization. Figure 14 shows the super-modules of the submarine that the Navy plans to start early.

However, the Navy has yet to finalize or fund the approach for this type of early work. Starting construction early for the lead and follow submarines provides schedule relief to the Navy and shipbuilders, but these plans may further exacerbate the existing overlap of technology development and design and construction, which was discussed above. Moving construction earlier could challenge the Navy’s goal to have all components developed 7 months before they are required in the shipyard. Further, the shipbuilders acknowledge that early construction plans will result in increased overlap between various stages of design activities in certain areas, including the bow and stern. If Congress funds the Navy’s plans to fund advanced construction work, this incomplete VFI/GFI situation will likely be worsened and could disrupt the optimal build strategy.

We have previously reported that programs starting construction of the lead ship of a class without a mature, stable design has been a major source of cost growth and schedule delays in Navy programs. We have also reported that when a schedule is set that cannot accommodate program scope, delivering an initial capability is often delayed and higher costs are incurred because problems typically occur that cannot be resolved within compressed, optimistic schedules. The Navy’s Columbia class plans put the program at risk of cost and schedule growth. However, its options for reducing concurrency are, at this point, limited due to the schedule imperatives driven by the lead ship patrol deadline.

Columbia Class Is Not Funded Adequately to Address Program Risks

Our analysis determined that it is more likely than not that the Columbia class program will exceed the Navy’s $128 billion (then-year dollars) estimate of total acquisition cost to which the program will be funded. Specifically, the program’s 2017 Milestone B cost estimates are optimistic because they do not account for a sufficient amount of program risk due to ongoing technology development, as well as the likely costs to design and construct the submarines. In addition, the Navy has budgeted the program to a confidence level for the program that is lower than what experts recommend, with a particularly optimistic estimate for the lead ship. While there may be situations when this would be appropriate, this is not the case for the Columbia class program due to the technical and design risks that we identified above. As a result, program costs will more likely than not exceed requested funding, particularly for lead ship construction. Due to the significant level of funding required for this program, even a small percentage of cost growth could have far-reaching consequences on the Navy’s long-range plans to fund construction of its future fleet. For this review we conducted an initial analysis of the Navy’s cost estimate but did not assess if it was conducted in accordance with all of the best practices identified in our cost estimating guide. We plan to more fully assess the Navy’s life-cycle cost estimate for the entire Columbia class, including the program’s risk analyses, in future work.

Confidence Levels and the Navy’s Estimate

Confidence Levels A confidence level is stated as a percentage depicting the probability that the program’s cost will actually be at some value or lower, calculated after conducting a risk analysis to identify and quantity program risks and determine the effects of these risks on its point estimates.

From early on, the Navy recognized the need to control costs for the Columbia class. In fact, the program’s cost estimates have decreased significantly since the program’s inception due to Navy decisions early in the program to trade off some capabilities and the incorporation of updated actual cost data from the continued procurement of Virginia class submarines. At Milestone B, OSD determined that Columbia class procurement costs had fallen almost 40 percent since the program’s original estimate. However, while the Navy did conduct a risk analysis for its recent Columbia class cost estimates, the confidence level of the Navy’s estimate at Milestone B for acquisition of the entire class is 45 percent. This means that it is more likely than not that actual costs to research, develop, and buy the submarines will exceed the Navy’s $128 billion estimate.

This situation is particularly apparent at this point with regard to costs to design the class and build the lead submarine. Any difficulties in ongoing technology development efforts would likely worsen the picture. At Milestone B, the Navy’s point estimate to develop the technologies, design the class, and build the lead Columbia was at a 43 percent confidence level.

Experts agree that programs should be budgeted to at least the 50 percent confidence level, but budgeting to a higher level (e.g., 70 to 80 percent, or the mean) is a common practice to cover increased costs resulting from unexpected design complexity and technology uncertainty, among other things. Navy cost guidance recommends using the “risk adjusted mean” for the cost for the program, which usually lies between 50 and 60 percent. If the Navy budgeted to an estimate at a higher confidence level like the risk adjusted mean, its Milestone B point estimates—meaning the selected estimate of cost—would be higher, reducing the probability of overruns occurring. According to Navy cost analysts, the program’s total acquisition cost, which the Navy estimated at Milestone B at $128 billion (then-year dollars), would exceed $131 billion (then-year dollars) at 50 percent confidence, which is the bottom range of the risk adjusted mean confidence level.

Cost Growth Potential Based on the Navy’s Estimate

Even if the Navy budgeted to the 90 percent—a “worst-case” scenario where significant programmatic challenges are realized and the probability of cost overruns is low—confidence level, Columbia class lead ship costs would not be dissimilar to cost outcomes on other lead ship programs. We have observed in prior work that cost growth for recent lead ships across the Navy’s shipbuilding portfolio is 28 percent on average. For example, the Navy’s lead Virginia class submarines (SSN 774 and SSN 775)—the most similar class to Columbia in terms of technology and component development as well as aspects of its design and build plans—experienced 15 and 24 percent budget growth respectively, with average cost growth of 28 percent for the three most recent lead submarines (see figure 15).

The 28 percent cost growth we have observed is slightly more than the 22 percent cost increase between the Navy’s point estimate and the 90 percent confidence level, meaning that even if the Navy budgeted the program to the 90 percent confidence level there would still be historical shipbuilding precedence for further cost growth. In particular, if costs to build the lead Columbia class submarine grow similar to the lead Seawolf and Virginia class submarines, the cost to construct the submarine would exceed the Navy’s Milestone B estimate by more than $2.5 billion. This would represent a total approaching $12 billion (then-year dollars) versus the current estimate of $9.2 billion for the lead submarine. Due to the magnitude of the Columbia class program’s expected cost, any cost growth, including for design and construction of the lead ship could impact the availability of funds for other Navy priorities.

The Congressional Budget Office (CBO) and CAPE also analyzed Columbia class program costs. CBO predicted higher costs than the Navy estimate. In its 2017 assessment of the Navy’s long-term shipbuilding plans, CBO concluded that the Navy underestimated the cost of the total Columbia class procurement by $8 billion (2017 dollars).

CAPE estimated a lower cost, but also identified areas where reliable cost data were unavailable. The independent cost estimate prepared by CAPE in support of the program’s Milestone B reflects a 3 percent lower total program life-cycle cost (2017 dollars) than the Navy estimate. In setting the program baseline in January 2017, DOD pragmatically opted to use the Navy’s higher estimate ($7.3 billion) instead of CAPE’s $7 billion estimate for the average unit cost to procure a Columbia class submarine (calendar year 2017 dollars). According to CAPE officials, this difference in estimates is largely due to CAPE incorporating more recent Virginia class actual cost data into its estimate than the Navy. However, CAPE also identified that there is a lack of reliable cost data on some contractor- furnished materials and government furnished equipment (GFE) for the Columbia class program, which limited the quality of the estimate. GFE comprises critical areas of the Columbia class submarine, including the strategic weapon system managed by Strategic Systems Program and the IPS developed by Naval Reactors.

Conclusions

The Columbia class submarine will be a significant DOD acquisition for the next several decades due to cost and mission importance in guaranteeing the nation’s strategic deterrence. Failure to meet the aggressive patrol dates required of the program could challenge the Navy in effectively meeting strategic patrol requirements, and not delivering the required level of performance could compromise the Navy’s plan to operate this class through 2080.

Given the risks facing the program and the significance of potential delays or cost growth, we believe this program warrants increased attention to and scrutiny over what we consider to be its critical technologies (inclusive of the program’s stated technology development efforts), several of which remain immature. Specifically, technologies such as IPS and the propulsor and coordinated stern demand more specific congressional visibility to ensure they stay on track. These areas also warrant specific assurances from the Navy that they will be delivered on time and will perform as required. This assurance could augment the Milestone B certifications which were predicated on a TRA that was not representative of the technical risk facing the program. Further, such information would help bolster confidence for Congress that the program technologies will be matured in time to support construction, which is especially important as the Navy pursues plans to start construction of the lead ship early. Without putting in place a requirement for the Navy to provide these assurances on a periodic basis, Congress will not have the information until after the Navy has asked for another $8.7 billion in funding for lead ship construction. It is also important for Congress to be informed of the impact on performance requirements if technologies are delayed or fail to mature as planned.

The Columbia class program is also facing risks from its aggressive and concurrent schedule as a result of the continued and pressing need for it to meet the Navy’s nuclear deterrent requirements as the legacy submarine fleet that cannot be life extended any longer. Typically addressing risks of such concurrency is accomplished by, among other things, delaying milestones until more knowledge is obtained. Doing so helps reduce concurrency and bring more stability to the design before construction activities begin. Recognizing the mission imperatives that are driving Columbia class’s aggressive and concurrent schedule it is unlikely that the Navy will have the ability to slow the pace of the program in order to reduce cost and schedule risk. Therefore, additional reporting to decisionmakers on the status of key technologies could help ensure they fully understand the risks of such an approach and account for such risks when making programmatic decisions.

Matter for Congressional Consideration

In our draft report we had suggested a matter for congressional consideration related to additional Navy reporting on the Columbia class technologies, but we have since removed it because the recently passed National Defense Authorization Act (NDAA) for 2018 includes Navy reporting requirements for the Columbia class program that would achieve the intent of our matter.

Agency Comments and our Evaluation

We provided a draft of this product to DOD for comment. The Navy provided technical comments earlier in the review process which we incorporated where appropriate. In its written comments, reproduced in appendix III, DOD’s position was that there is not a need for additional congressional reporting on the Columbia class program because there are new reporting requirements in the conference report accompanying the NDAA for fiscal year 2018. We agree that the reporting requirements in the section 231 of the NDAA for Fiscal Year 2018 meet the intent of our matter for congressional consideration. These new reporting requirements for the Navy became law on December 12, 2017, after we sent the report to DOD and appropriate congressional committees. We agree that the reporting requirements meet the intent of our matter for congressional consideration. Accordingly, we have removed our matter from this report.

In addition, DOD also disagreed with our characterization of technical risks facing the Columbia class program and its TRA. Specifically, DOD stated that the program is meeting statutory and DOD maturity standards and met or exceeded DOD technology maturity requirements. DOD also stated that the program’s TRA was conducted in accordance with a 2011 DOD policy memo that directed TRA’s should focus only on “technology maturity, as opposed to engineering and integration risk.” However, neither this policy memo nor the Columbia class TRA define what constitutes engineering and integration risk and it is unclear what criteria the Navy used in making these determinations. Our report acknowledges that DOD followed statutory and DOD requirements for the two technologies that the Navy identified as critical technologies in the program’s TRA. However, our report also identifies several other technologies that we believe should have also been subject to these requirements had the Navy conducted a TRA in accordance with our identified best practices. By applying our identified best practices, we believe these efforts would have been considered critical technologies and would have been subject to an evaluation of technology maturity levels, additional reporting requirements and, potentially, identification of additional risk mitigation efforts.

DOD also disagreed with our criteria for identifying a critical technology and assessing maturity. DOD asserted that applying our criteria would result in nearly every system on a submarine becoming a critical technology. We disagree. Our criteria are consistent with DOD’s own criteria for identifying critical technologies, and only focus on those that are most significant to a program. Given the program’s cost and schedule risks and operational imperatives, we believe that appropriately identifying the critical technologies is an important step in acknowledging and mitigating program risk. DOD also stated that achieving a TRL 7 by milestone B would be unrealistic because of the difficulties in testing some systems in an operational environment prior to launching the submarine. We agree that in some cases testing at sea is not practical and testing in a relevant environment may be sufficient to demonstrate maturity. However, achieving a TRL 7 is not only based on the test environment; it is also based on demonstrating a prototype near or at the planned operational system configuration, which requires a design resembling the final configuration. The Columbia class program has yet to complete this type of prototype for the key systems we identified. As we stated in the report some systems, like the propulsor, do not yet have a final design. While we do not expect the Navy to test every critical technology on a submarine at sea to demonstrate maturity, we would expect testing of a prototype near or at the planned operational system configuration prototype in a relevant environment. For example, prototype testing of the electric drive at a land-based test facility would demonstrate maturity—but is not planned for several years—well after the submarine’s design and potentially construction is underway. While such concurrency introduces cost, schedule and technical risk, we have previously reported that programs may choose to move forward with these risks, but should acknowledged and appropriately resource the program to address the risks should they materialize. As we stated in the report, this is not the case for the Columbia class program: some risks have not been properly identified and the cost estimate does not fully account for the margin of technical and schedule risks facing the program.

DOD also provided a table of Columbia class practices, reprinted with our comments in appendix III.

We are sending copies of this report to the appropriate congressional committees; the Secretary of Defense, the Secretary of the Navy, and other interested parties. This report will also be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff members have any questions regarding this report, please contact me at (202) 512-4841 or oakleys@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to the report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

This report examines (1) the status of key Columbia class technologies and congressional reporting requirements on this status, (2) risks, if any, with the Navy’s planned approach for design and construction, and (3) whether expected funding levels for the Columbia class will be adequate moving forward.

To assess the status of key Columbia class technologies, we reviewed the Navy’s technology development plan and the planned technical approach and the status of key prototyping efforts to all of the systems that comprise the program, focusing on the technology readiness level of the major components that are key to enabling program success and that are key cost and schedule drivers. We also compared technology development efforts with program requirements and with GAO’s identified best practices for shipbuilding programs. We also evaluated the program’s Technology Readiness Assessment, which included applying the GAO-developed criteria documented in GAO’s Technology Assessment Guide. GAO’s guide draws heavily from the Department of Defense (DOD), Energy (DOE), and National Aeronautics and Space Administration (NASA) best practices, and establishes a methodology based on those best practices that can be used across the federal government for evaluating technology maturity, particularly as it relates to determining a program or project’s readiness to move past key decision points that typically coincide with major commitments of resources. We also interviewed relevant officials from the Navy’s Columbia class submarine program office; the Office of the Chief of Naval Operations- Undersea Warfare; Naval Sea Systems Command Naval Nuclear Propulsion Program; Navy Strategic Systems Program; Naval Undersea Warfare Center Newport; Naval Surface Warfare Center Carderock Division; Office of the Secretary of Defense (OSD) Director Operational Test and Evaluation; OSD Acquisition, Technology, and Logistics (AT&L); OSD Cost Analysis and Program Evaluation (CAPE); and the prime contractor shipyard General Dynamics Electric Boat and their sub- contractor Huntington Ingalls Industries Newport News Shipbuilding. To determine the congressional reporting requirements on this status we reviewed relevant DOD acquisition instructions and statute.

To assess the risks, if any, with the Navy’s planned approach for design and construction, we compared the status of design maturity with Navy and shipyard plans to identify any delays, and compared planned design maturity and schedule projections with those of prior U.S. submarine efforts (the Virginia, Seawolf, and Ohio classes) to assess realism of Columbia class estimates. We also interviewed and analyzed available documentation from Naval Reactors (NAVSEA 08) related to nuclear reactor and Integrated Power System status. We also interviewed relevant officials from the Navy’s Columbia class submarine program office; Naval Sea Systems Command Naval Nuclear Propulsion Program; Naval Surface Warfare Center Carderock Division, and the prime contractor shipyard General Dynamics Electric Boat and their sub- contractor Huntington Ingalls Industries Newport News Shipbuilding. We also assessed the Navy’s acquisition strategy and the Integrated Enterprise Plan that tracks shipyard workload across the Columbia and Virginia class submarines and the Ford class aircraft carrier to identify any factors related to potential schedule challenges.

To assess whether expected funding levels for the Columbia class will be adequate moving forward, we compared program cost estimates prepared at Milestone B to historical data on lead ships and submarine estimates and actuals to assess the realism of these requirements. We also analyzed program documentation to identify risk factors, if any, related to cost projections, including the program’s Independent Cost Estimate created by the OSD Cost Analysis and Program Evaluation, and the Navy’s Service Cost Position and Program Life Cycle Cost Estimate. This evaluation leverages, among other things, prior GAO work on cost estimating and the Navy’s acquisition of lead ships.

We also interviewed relevant officials from the Navy’s Columbia class submarine program office; the Office of the Chief of Naval Operations- Undersea Warfare; Naval Sea Systems Command Naval Nuclear Propulsion Program; Naval Undersea Warfare Center; Naval Surface Warfare Center Carderock Division; OSD Director Operational Test and Evaluation; OSD AT&L; CAPE; and the prime contractor shipyard General Dynamics Electric Boat and their sub-contractor Huntington Ingalls Industries Newport News Shipbuilding.

We conducted this performance audit from May 2016 to December 2017 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Department of Defense Technology Readiness Levels

Appendix II: Department of Defense Technology Readiness Levels Description Lowest level of technology readiness. Scientific research begins to be translated into applied research and development (R&D). Examples might include paper studies of a technology’s basic properties.

Invention begins. Once basic principles are observed, practical applications can be invented. Applications are speculative and there may be no proof or detailed analysis to support the assumptions. Examples are limited to analytic studies.

Active R&D is initiated. This includes analytical studies and laboratory studies to physically validate the analytical predictions of separate elements of the technology. Examples include components that are not yet integrated or representative.

Basic technological components are integrated to establish that they will work together. This is relatively “low fidelity” compared with the eventual system. Examples include integration of “ad hoc” hardware in the laboratory.

Fidelity of breadboard technology increases significantly. The basic technological components are integrated with reasonably realistic supporting elements so they can be tested in a simulated environment. Examples include “high-fidelity” laboratory integration of components.

Representative model or prototype system, which is well beyond that of TRL 5, is tested in a relevant environment. Represents a major step up in a technology’s demonstrated readiness. Examples include testing a prototype in a high-fidelity laboratory environment or in a simulated operational environment.

Prototype near or at planned operational system. Represents a major step up from TRL 6 by requiring the demonstration of an actual system prototype in an operational environment (e.g., in an aircraft, in a vehicle, or in space.

Technology has been proven to work in its final form and under expected conditions. In almost all cases, this TRL represents the end of the true system development. Examples include developmental test and evaluation of the system in its intended weapon system to determine if it meets design specifications.

Actual system proven through successful mission operations.

Actual application of the technology in its final form and under mission conditions, such as those encountered in operational test and evaluations. Examples include using the system under operational conditions.

Appendix III: Comments from the Department of Defense

GAO Comments

DOD also provided the above table of Columbia class practices. These practices align with GAO’s identified best practices in shipbuilding—stable requirements, design maturity at construction start, and manufacturing readiness. However, we have several observations on the DOD’s statements: Stable Operational and Technical Requirements:

We have previously identified maintaining stable requirements as a best practice; in this report we note that the Navy has provided a stable basis for the Columbia class program by adhering to this practice.

High Design Maturity at Construction Start:

While we give credit to the program for striving for a high level of design maturity at construction start for the Columbia class program, we identify in this report that we have concerns about the Navy’s ability to stabilize design drawings while technology development continues.

As we point out in this report, we are concerned with the maturity of the Columbia class design due to the unknowns with key technologies. In this table the Department identifies that the program is leveraging proven Virginia class technology for the propulsor, which it identifies as a TRL 9. Although this technology is indeed mature in the context of Virginia class submarines (i.e., not new or novel), it is nevertheless novel in the context of Columbia class submarines and should thus be considered a CTE to be evaluated and risk managed. As such, we dispute the Navy’s assertion that the Virginia class propulsor is TRL 9 in the context of the Columbia class program, since the Navy has yet to complete a design for the propulsor nor has it tested a production representative prototype, which would achieve a TRL 6 or 7 (depending on the test environment).

Manufacturing and Construction Readiness:

We have not conducted adequate work in this area to comment on DOD’s statements of manufacturing and construction readiness; we plan to address this in future work.

Aggressive Action to Reduce Costs:

While the Navy has made significant progress in reducing potential costs for the Columbia class program, we believe that the risks identified in this report, coupled with the optimistic cost estimate and aggressive schedule, could result in cost growth that reduces the actual savings identified by the program.

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition the contact name above, the following staff members made key contributions to this report: Diana Moldafsky, Assistant Director; C. James Madar; Jacob Leon Beier; Brian Bothwell; Herb Bowsher; Kurt Gurka; Stephanie Gustafson; Tim Persons; and Robin Wilson.