Defense Acquisitions: Navy's Ability to Overcome Challenges Facing the Littoral Combat Ship Will Determine Eventual Capabilities
Published: 2010-08-31T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Navy's Littoral Combat Ship (LCS) is envisioned as a reconfigurable vessel able to meet three missions: surface warfare, mine countermeasures, and anti-submarine warfare. It consists of the ship (seaframe) and the mission package it carries and deploys. The Navy plans to invest over $25 billion through fiscal year 2035 to acquire LCS. However, recurring cost growth and schedule delays have jeopardized the Navy's ability to deliver promised LCS capabilities. Based on a congressional request, GAO (1) identified technical, design, and construction challenges to completing the first four ships within current cost and schedule estimates, (2) assessed the Navy's progress developing and fielding mission packages, and (3) evaluated the quality of recent Navy cost analyses for seaframes and their effect on program progress. GAO's findings are based on an analysis of government and contractor-generated documents, and discussions with defense officials and key contractors. This product is a public version of a For Official Use Only report, GAO-10-1006SU , also issued in August 2010.
section_title: What GAO Found
paragraphs: The Navy faces technical, design, and construction challenges to completing the first four seaframes within current cost and schedule estimates. The Navy and its shipbuilders have learned lessons from construction of the first two seaframes that have positioned them to more effectively construct future vessels. However, technical issues with the first two seaframes have yet to be fully resolved. Addressing these technical issues has required the Navy to implement design changes at the same time LCS 3 and LCS 4 are being built. Incorporating changes during this phase will likely require additional labor hours beyond current forecasts. Together, these challenges may hinder the ability of shipbuilders to apply lessons learned to follow-on ships and could undermine anticipated benefits from recent capital investments in the LCS shipyards. Challenges developing mission packages have delayed the timely fielding of promised capabilities, limiting the ships' utility to the fleet during initial deployments. Until these challenges are resolved, it will be difficult for the Navy to align seaframe purchases with mission package procurements and execute planned tests. Key mine countermeasures and surface warfare systems encountered problems in operational and other testing that delayed their fielding. For example, four of six Non-Line-of-Sight Launch System missiles did not hit their intended targets in recent testing, and the Department of Defense has since canceled the program. Further, Navy analysis of anti-submarine warfare systems has shown the planned systems do not contribute significantly to the anti-submarine warfare mission. These combined challenges have led to procurement delays for all three mission packages. Mission package delays have also disrupted program test schedules--a situation exacerbated by early deployments of initial ships--limiting their availability for operational testing. In addition, these delays could disrupt program plans for simultaneously acquiring seaframes and mission packages. Until mission packages are proven, the Navy risks investing in a fleet of ships that does not deliver promised capability. The Navy entered contract negotiations in 2009 for fiscal year 2010 funded seaframes with an incomplete understanding of LCS program costs. These contract negotiations proved unsuccessful, prompting the Navy to revise its acquisition strategy for the program. The contractors' proposals for construction of the next three ships exceeded the approximate $1.4 billion in funds the Navy had allocated in its fiscal year 2010 budget. In response, the Navy revised its strategy to construct one seaframe design instead of two for fiscal year 2010 ships and beyond in an effort to improve affordability. Navy cost analyses completed prior to the failed negotiations in 2009 lack several characteristics essential to a high-quality cost estimate. These characteristics include the completion of sensitivity and uncertainty analyses and an independent review of the cost estimate. The Navy plans to complete a more comprehensive cost estimate before award of additional ship contracts in 2010.
section_title: What GAO Recommends
paragraphs: GAO recommends the Secretary of Defense take actions to ensure more realistic cost estimates, timely incorporation of design changes, and coordination of seaframe and mission package acquisition. The Department of Defense concurred with each of these recommendations.

section_title: Letter
paragraphs: The Navy’s Littoral Combat Ship (LCS) is envisioned as a vessel able to be  reconfigured to meet three different mission areas: mine countermeasures,  surface warfare, and anti-submarine warfare. Its design concept consists  of two distinct parts—the ship itself (seaframe) and the mission package it  carries and deploys. The Navy currently plans to invest over $25 billion to  acquire LCS seaframes and mission packages through fiscal year 2035.  However, recurring cost growth and schedule delays in the program have  jeopardized the Navy’s ability to deliver promised LCS capabilities.
In light of these developments, you asked us to evaluate LCS planning and  implementation efforts. In response to this request, we (1) identified  technical, design, and construction challenges to completing the first four  seaframes within current cost and schedule estimates; (2) assessed the  Navy’s progress developing and fielding mission packages; and (3)  evaluated the quality of recent Navy cost analyses for seaframes and their  effect on program progress. This product is a public version of a For  Official Use Only report, GAO-10-1006SU, also issued in August 2010.
To identify challenges to completing the first four seaframes, we analyzed  Department of Defense and contractor-generated documents that  addressed technical challenges and cost and schedule performance for  LCS seaframes including sea trial reports for the first two ships,  construction progress briefings, and monthly contract performance  reports. We corroborated this information through discussions with  officials responsible for managing LCS design and construction activities  including Navy program officials, technical authorities, and requirements  officers; LCS prime contractors and shipbuilders; and the Office of the  Secretary of Defense. To assess the Navy’s progress developing and  fielding mission packages, we reviewed documents that outline LCS  mission package plans and performance including program schedules and  recent test reports. We also held discussions with Navy program offices  and Department of Defense agencies responsible for acquiring and testing  key LCS mission systems to gather additional information on remaining  risks to mission package development and integration. To evaluate the  quality of recent Navy cost analyses, we compared the Navy’s total  ownership cost baseline estimate for the LCS program against the  characteristics inherent in high-quality cost estimates as outlined in our  cost estimating and assessment guide. In addition, we interviewed LCS  cost analysts and program officials to supplement our analysis and gain  additional visibility into the Navy’s process for developing its cost  estimate. A more detailed description of our scope and methodology is  presented in appendix I.
We conducted this performance audit from July 2009 to August 2010 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: LCS is designed to move fast and transport manned and unmanned mine  countermeasures, surface warfare, and anti-submarine warfare systems  into theater. For LCS, the seaframe consists of the hull; command and  control systems; automated launch, handling, and recovery systems; and  certain core combat systems like an air defense radar and 57-millimeter  gun. The Navy is procuring the first four ships in two different designs  from shipbuilding teams led by Lockheed Martin and General Dynamics.  Lockheed Martin and General Dynamics currently build their designs at  Marinette Marine and Austal USA shipyards, respectively. Figure 1 shows  the first two LCS seaframes, USS Freedom (LCS 1) and USS Independence  (LCS 2).
The Navy is embedding LCS’s mine countermeasures, surface warfare, and  anti-submarine warfare capabilities within mission packages. These  packages are comprised of unmanned underwater vehicles, unmanned  surface vehicles, towed systems, and hull- and helo-mounted weapons.  Table 1 identifies the systems included in the LCS mission packages.
Fundamental to the capability of the LCS seaframe is its ability to move  quickly ahead of other ships and deploy its offboard sensors to secure  lanes of transit. To deploy LCS’s mine countermeasures and anti- submarine warfare systems, the Navy will rely extensively on (1)  automated launch, handling, and recovery systems embedded in each  seaframe and (2) helicopters and unmanned aerial vehicles. The Navy’s  acquisition approach is to populate initial versions of mission packages  with a mixture of developmental and production-representative systems,  gradually moving to all production-representative systems that constitute  the baseline configuration for each package. The Navy plans to procure 55  seaframes and 64 mission packages (24 mine countermeasures, 24 surface  warfare, and 16 anti-submarine warfare) as part of the LCS program.
The Navy has required LCS seaframes to meet Level 1 survivability  standards. Ships built to Level 1 are expected to operate in the least severe  environment, away from the area where a carrier group is operating or the  general war-at-sea region. These vessels should also maintain good  handling in bad weather—including seas above 30 feet high (sea state  8)—and have systems for fighting fires on board the ships, hardening  against electromagnetic pulses, and protection against chemical,  biological, or radiological contamination. Unlike surface warships like  cruisers and destroyers, Level 1 ships (including LCS) are not designed to  maintain their mission capabilities after incurring substantive damage.  Current ships in the fleet built to the Level 1 standard include material  support ships, mine-warfare vessels, and patrol combatants.
Two broad categories of contract types are available for government  procurements, including ship procurement: fixed-price and cost- reimbursement. Fixed-price contracts provide for a firm price or, in  appropriate cases, an adjustable price that may include a ceiling price, a  target price, or both. This contract type places the risk on the contractor,  who generally bears the responsibility of increased costs of performance.  Cost-reimbursement contracts provide for payment of allowable incurred  costs, to the extent prescribed in the contract. This contract type places  most of the risk on the government, which may pay more than budgeted  should incurred costs be more than expected when the contract was  signed.
The Navy awarded cost-reimbursable contracts for detail design and  construction of the first two seaframes—LCS 1 and LCS 2—in December  2004 and October 2005 for $188.2 million and $223.2 million, respectively.  It later exercised options on each of these contracts in June and December  2006 for construction of the third and fourth ships (LCS 3 and LCS 4).  However, changing technical requirements, evolving designs, and  construction challenges increased the government’s estimated prices at  completion for the LCS 1 and LCS 2 seaframes to about $500 million each.  This cost growth precipitated concern within the Navy that similar  outcomes were possible for LCS 3 and LCS 4. In response, the Navy  reassessed program costs and structure, revisited the acquisition strategy  for future ships, and entered into negotiations with its shipbuilders to  convert the LCS 3 and LCS 4 contracts into fixed-price contracts. The Navy  was unable to reach agreement with its shipbuilders on fixed-price terms  for these ships, subsequently leading the Navy to terminate, in part, the  LCS 3 and LCS 4 contracts in April and November 2007 for the  convenience of the government. In March and May 2009, the Navy  awarded new fixed-price contracts for LCS 3 and LCS 4. According to the  Navy, work completed and materials procured under the terminated  original contract options for LCS 3 and LCS 4—totaling approximately  $192 million—are not included in the current contract values for those  ships.
In our work on shipbuilding best practices, we found that achieving design  stability before start of fabrication is a key step that leading commercial  shipbuilders and ship buyers follow to ensure their vessels deliver on-time,  within planned costs, and with planned capabilities. Leading commercial  firms assess a ship design as stable once all basic and functional design  activities have been completed. Basic and functional design refers to two- dimensional drawings and three-dimensional, computer-aided models  (when employed) that fix the ship’s hull structure; set the ship’s  hydrodynamics; route all major distributive systems including electricity,  water, and other utilities; and identify the exact positioning of piping and  other outfitting within each block of the ship. At the point of design  stability, the shipbuilder has a clear understanding of both ship structure  as well as ship electrical, piping, and other systems that traverse individual  blocks of the ship. To achieve design stability, shipbuilders need suppliers  (also called vendors) to provide complete, accurate system information  prior to entering basic design. This vendor-furnished information  describes the exact dimensions of a system or piece of equipment going  into a ship, including space and weight requirements, and also  requirements for power, water, and other utilities that will have to feed the  system.
As is typical for all ships, the LCS construction phase includes several  steps: block fabrication, assembly and outfitting of blocks, block erection,  launch, and delivery. During block fabrication, metal plates are welded  together into elements called blocks. Blocks are the basic building units  for a ship, and when completed they will form completed or partial  compartments, including accommodation space, engine rooms, and  storage areas. Blocks are generally outfitted with pipes, brackets for  machinery or cabling, ladders, and any other equipment that may be  available for installation at this early stage of construction. This allows a  block to be installed as a completed unit when it is welded to the hull of  the ship. Installing equipment at the block stage of construction is  preferable because access to spaces is not limited by doors or machinery,  unlike at later phases. Blocks are welded together to form grand blocks  and then erected with other grand blocks in a drydock or, in the case of  LCS, in a building hall. Finally, once the ship is watertight and the decision  is made to launch—or float the ship in water—the ship is then towed into  a quay or dock area where final outfitting and testing of machinery and  equipment like main engines will occur. Afterwards, the ship embarks on  sea trials where performance is evaluated against the contractually  required specifications and overall quality is assessed. Following sea trials,  the shipyard delivers the ship to the buyer.
LCS 1 was delivered to the Navy in September 2008, with LCS 2 following  in December 2009. The Navy has also accepted delivery of five partial  mission packages to date. Currently, LCS 1 is on deployment, LCS 2 is  undergoing post-delivery work, and LCS 3 and LCS 4 remain in different  stages of construction. In addition, development and testing activities for  the mine countermeasures, anti-submarine warfare, and surface warfare  mission packages continue. The Navy deployed LCS 1 two years ahead of  its previous schedule and prior to the ship completing initial operational  test and evaluation. The Navy also stated that early deployment is possible  for LCS 2.
Initial operational test and evaluation is intended to assess a weapon  system’s capability in a realistic environment when maintained and  operated by sailors, subjected to routine wear-and-tear, and employed in  typical combat conditions against a simulated enemy who fights back.  During this test phase, the weapon system is exposed to as many actual  operational scenarios as possible—a process that reveals the weapon  system’s capabilities under stress. Once the fleet has attained the ability to  effectively employ and operate the weapon system, initial operational  capability is achieved.
Until September 2009, the Navy planned to continue buying both ship  designs. In September 2009, the Navy announced it was revising the LCS  program’s acquisition strategy and would select one seaframe design  before awarding contracts for any additional ships. In the National  Defense Authorization Act for Fiscal Year 2010, Congress mandated a $480  million cost cap for each LCS, starting with fiscal year 2011 funded  seaframes. In an effort to comply with this mandate, Navy officials have  stated that a major program review (milestone B)—and completion of an  independent cost estimate—will precede further contract awards in the  program.
Cost estimates are necessary for government acquisition programs, like  LCS, for many reasons: to support decisions about funding one program  over another, to develop annual budget requests, to evaluate resource  requirements at key decision points, and to develop performance  measurement baselines. A cost estimate is a summation of individual cost  elements, using established methods and valid data, to estimate the future  costs of a program, based on what is known today. The management of a  cost estimate involves continually updating the cost estimate with actual  data as they become available, revising the estimate to reflect changes,  and analyzing differences between estimated and actual costs—for  example, using data from a reliable earned value management system.
section_title: Ongoing Development of Key Seaframe Systems Could Impede Efficient Construction of Initial Follow-On Ships
paragraphs: The Navy faces technical, design, and construction challenges to  completing the first four seaframes within current cost and schedule  estimates. The Navy and its shipbuilders have learned lessons from  construction of the first two seaframes that can be applied to construction  of future vessels. However, technical issues with the first two seaframes  have yet to be fully resolved, posing risk of design changes to follow on  ships already under construction. Addressing these technical issues has  required the Navy to implement design changes at the same time LCS 3  and LCS 4 are being built. Incorporating changes during this phase may  disrupt the optimal construction sequence for these ships, requiring  additional labor hours beyond current forecasts. Together, these  challenges may hinder the ability of shipbuilders to apply lessons learned  to follow on ships and could undermine anticipated benefits from recent  capital investments in the LCS shipyards.
subsections: section_title: Cost Growth and Schedule Delays Have Hampered Construction of the First Four Ships
paragraphs: Initial LCS seaframes have required more funding and taken longer to  construct than the Navy originally planned. The Navy has accepted  delivery of the first two ships (LCS 1 and LCS 2), which, according to the  Navy, reduces the likelihood of additional cost increases and schedule  delays on those ships. Further, the Navy’s decision to partially terminate,  and later re-award, construction contracts for follow-on ships (LCS 3 and  LCS 4) changed the planned delivery dates for those ships. Tables 2 and 3  highlight the cost growth and schedule delays associated with the first  four ships of the class.
section_title: First Two Seaframes Delivered to the Fleet, but Technical Challenges Currently Limit Their Capabilities
paragraphs: The Navy accepted delivery of LCS 1 and LCS 2 with both seaframes in an  incomplete state and with outstanding technical issues. After experiencing  significant cost increases and schedule delays on these ships, the Navy  judged it more cost efficient to accept the incomplete ships and resolve  remaining issues post-delivery. According to Navy officials, this step  afforded the Navy more control over remaining work and provided the  ability to use repair yards that charge less than the builder in some  instances. Although the ships are currently in service, the Navy continues  to address technical issues on each seaframe. Addressing these issues has  required the Navy to schedule extensive post-delivery work periods for  each ship, which were not fully anticipated at the time of lead ship  contract awards. For instance, to resolve the LCS 1 issues, the Navy  allocated several months for two industrial post-delivery availability  periods in 2009. A similar schedule is planned for LCS 2.
The Navy has made significant progress resolving LCS 1 deficiencies.  While challenges with several systems were identified at delivery, the Navy  deferred testing of other systems until after delivery. The shipbuilder had  not completed installation of several LCS 1 systems prior to delivery,  contributing to Navy decisions to defer key elements of the ship’s  acceptance trials until later. Most notably, the Navy deferred testing of  the ship’s launch, handling, and recovery system—a system instrumental  to deploying and recovering mission package elements that, if not  performing adequately, will impair LCS capability. To date, a full  demonstration of this system remains incomplete. Navy simulations to  date have identified risks in safely launching and recovering mission  systems that experience pendulous motion during handling—such as the  remote multi-mission vehicle and unmanned surface vehicle systems. Navy  officials stated, however, that the fleet successfully demonstrated  operation and movement of an embarked 11-meter rigid-hull inflatable  boat, having used one extensively for counterdrug operations in the  Caribbean in March 2010.
Another challenge for LCS 1 launch, handling, and recovery is the potential  for unacceptably high water levels during high sea states in the  waterborne mission zone—the area at the stern end of the ship designed to  launch watercraft through stern doors and down a ramp directly into the  water. Further, LCS 1’s launch, handling, and recovery system has also  experienced difficulty safely moving payloads on the ship. Most notably,  payload handling cranes will not be installed until a future maintenance  period in fiscal year 2013.
Like LCS 1, the Navy identified several significant technical deficiencies on  LCS 2 during that ship’s acceptance trials. However, because LCS 2 was  only recently delivered (December 2009), the Navy remains in the early  stages of addressing the issues facing that ship. Similar to LCS 1, the Navy  chose to accept delivery of LCS 2 prior to the shipbuilder completing  installation of key systems. The incomplete condition of the ship  contributed to Navy decisions to defer key elements of acceptance trials  until after delivery. As was the case with LCS 1, these deferments included  testing of the LCS 2 launch, handling, and recovery system for mission  watercraft.
LCS 2 is designed to employ a twin boom extensible crane system to  launch, handle, and recover mission watercraft. This system includes the  crane, synthetic lift lines, and a straddle carrier. The synthetic lift lines  attach to the crane to retrieve watercraft, but remain an unproven, new  capability to the Navy. Another risk to the system is the ability of the  straddle carrier to interface with, maneuver, and return to stowage the  rigid-hull inflatable boat, remote multi-mission vehicle, and unmanned  surface vehicle systems—three of the largest watercraft the Navy plans to  embark on LCS 2. According to the Navy, the straddle carrier was used to  successfully move the remote multi-mission vehicle and unmanned surface  vehicle during onboard trials in March 2010.
section_title: Continuing Design Changes Could Hinder Efficient Construction of LCS 3 and LCS 4
paragraphs: The Navy’s efforts to resolve technical issues affecting LCS 1 and LCS 2,  implement cost reduction measures, and increase mission capability have  led to design changes for LCS 3 and LCS 4, several of which are not yet  complete. These design changes have affected the configuration of several  major ship systems for LCS 3 and LCS 4 including propulsion,  communications, electrical, and navigation. The Navy is working to  implement these design changes concurrent with LCS 3 and LCS 4  construction activities. Incorporating design changes on the lead  seaframes while the follow-on ships are under construction may disrupt  the optimal construction sequence for LCS 3 and LCS 4, requiring  additional labor hours beyond current forecasts. As we have previously  reported, by delaying construction start until basic and functional design is  completed and a stable design is achieved, shipbuilders minimize the risk  of design changes and the subsequent costly rework and out-of-sequence  work these changes can drive.
section_title: Benefits Derived from Recent Process Improvements and Capital Investments in the LCS Shipyards May Not Be Fully Realized on Early Follow-On Ships
paragraphs: The Navy and its shipbuilders learned valuable lessons from the  construction of the lead ships that can save time and money on the  construction of follow-on ships. The shipbuilding teams have implemented  process improvements based on these lessons and made capital  investments in their yards in an effort to increase efficiency. Despite the  various improvements to capacity and processes at the shipyards,  capitalizing on these improvements might be challenging given the  significant design changes still occurring in the program. As technical  issues are resolved on the lead seaframes, this, in general, leads to  redesign—and potentially costly rework—for initial follow-on ships. Thus,  while efficiencies will be gained as a result of the shipyards’ improvement,  remaining technical issues on the seaframes will likely continue to  jeopardize the Navy’s ability to complete the first four seaframes within  planned cost and schedule estimates.
section_title: Mission Package Delays Limit Ship Capabilities in the Near Term and Pose Risk to Efficient Execution of Program Acquisition and Test Plans
paragraphs: Challenges developing and procuring mission packages have delayed the  timely fielding of promised capabilities, limiting the ships’ utility to the  fleet during initial deployments. Until these challenges are resolved, it will  be difficult for the Navy to align seaframe purchases with mission package  procurements and execute planned tests. Key mine countermeasures and  surface warfare systems have encountered technical issues that have  delayed their development and fielding. Further, Navy analysis of LCS anti- submarine warfare systems found these capabilities did not contribute  significantly to the anti-submarine warfare mission. These challenges have  led to procurement delays for all three mission packages. For instance,  key elements of the surface warfare package remain in development,  requiring the Navy to deploy a less robust capability on LCS 1. Mission  package delays have also disrupted program test schedules—a situation  exacerbated by decisions to deploy initial ships early, which limit their  availability for operational testing. In addition, these delays could disrupt  program plans for simultaneously acquiring seaframes and mission  packages. Until mission package performance is proven, the Navy risks  investing in a fleet of ships that does not deliver promised capability.
subsections: section_title: Challenges Developing LCS Mission Package Systems Have Delayed Their Planned Fielding Dates
paragraphs: Development efforts for most of these systems predate the LCS program— in some cases by 10 years or more. Recent testing of mission package  systems has yielded less than desirable results. To date, most LCS mission  systems have not demonstrated the ability to provide required capabilities.  Further, the Navy has determined that an additional capability will be  incorporated into future anti-submarine warfare mission packages. The  existing anti-submarine warfare mission package procurement is  temporarily suspended, and performance will be assessed during at-sea  testing in 2010. In addition to the sensors, vehicles, and weapons included  in each mission package, each LCS will rely on aircraft and their support  systems to complete missions.
subsections: section_title: Mine Countermeasures
paragraphs: Mine countermeasures missions for LCS will involve detecting, classifying,  localizing, identifying, and neutralizing enemy sea mines in areas ranging  from deep water through beach zones. We have previously reported on  challenges the Navy faces in transitioning the mine countermeasures  mission to LCS. Figure 2 illustrates how the Navy plans to employ the LCS  mine countermeasures systems against mine threats. m )
Table 4 shows the status of mine countermeasures mission package  systems.
For two of the LCS mine countermeasures systems—the Remote  Minehunting System and the Airborne Laser Mine Detection System—the  Navy has delayed further production pending successful resolution of  developmental challenges. These systems both entered production in 2005.  According to Navy officials, relaxing the performance requirements for the  Remote Minehunting System and the Airborne Laser Mine Detection  System is one option under consideration.
Airborne Laser Mine Detection System: Testing of this system has  revealed problems detecting mines at the required maximum depth  and classifying mines at surface depths. According to Navy officials,  the system’s required maximum detection depth could be reduced  because the system can currently detect mine-like objects at depths  that extend below the keels of all ships in the fleet. According to  Director, Operational Test and Evaluation officials, however, the  system is currently incapable of providing this capability with the  required accuracy. Further, Navy officials report that the Remote  Minehunting System could provide coverage in near-surface areas of  the water that the Airborne Laser Mine Detection System currently  cannot reach.
Remote Minehunting System: Operational testing of this system in  2008 revealed significant reliability shortfalls associated with the  underwater vehicle. Most notably, the system was only able to function  for 7.9 hours before failing—far short of its minimum requirement.  Director, Operational Test and Evaluation officials report that since  the 2008 event, the Navy’s estimated mean time between failures for  the system has increased to 45 hours. According to Navy officials,  testing and design changes are expected to last into 2011. While the  Navy is actively exploring ways to improve Remote Minehunting  System reliability, it is also considering reducing the reliability  requirement by half.
section_title: Surface Warfare
paragraphs: Surface warfare for LCS involves detecting, tracking, and engaging small  boat threats; escorting ships; and protecting joint operating areas. Figure 3  illustrates how the surface warfare mission package functions.
Table 5 shows the status of surface warfare mission systems.
The surface warfare package remains unproven as a key system, the Non- Line-of-Sight Launch System, was recently canceled prior to completing  development. The system—developed under the Army’s Future Combat  System program—progressed slower than anticipated due to technical  challenges and associated test failures. These issues—along with Army  fiscal year 2011 budget estimates showing missiles could cost up to  $466,000 each—prompted the Army to revisit its commitment to the  program. In May 2010, this process culminated with the Under Secretary of  Defense for Acquisition, Technology and Logistics approving the Army’s  request to cancel the program. In response, Navy officials report they are  now evaluating potential alternatives to the Non-Line-of-Sight Launch  System—including modifications to existing missile systems—to  substitute into the surface warfare mission package.
section_title: Support Aircraft
paragraphs: In addition to systems outlined above, the Navy plans to employ aircraft in  different configurations to execute LCS missions. Table 6 highlights the  status of key mission package support aircraft.
In addition to these technical challenges, other factors may constrain  availability of the three LCS support aircraft.
MH-60R Helicopter: According to Navy officials, while this system  has completed its first deployment with a carrier strike group, the  earliest possible date that it will deploy onboard an LCS is the end of  fiscal year 2013. Navy officials reported that because of fleet demand  for the helicopter, initial MH-60Rs will be assigned to the carrier strike  group elements (e.g., destroyers, cruisers, and frigates) before  deploying with an LCS. As of January 2010, the Navy had accepted  delivery of 46 MH-60Rs of a planned quantity of 252.
MH-60S Helicopter: The Navy has certified flight operations of the  armed variant of this helicopter from LCS 1. Previous plans called for  only the mine countermeasures variant of the MH-60S to fly from the  LCS. As of December 2009, 159 of 275 MH-60S helicopters had been  delivered.
MQ-8B Vertical Take-off and Landing Tactical Aerial Vehicle  (known as Fire Scout): The Navy is conducting operational testing  of Fire Scout onboard a frigate, but intends to field the system with  LCS exclusively. Reliability and availability issues uncovered during  fiscal year 2010 testing have delayed the program’s fielding schedule.  Previously, the Navy expected to reach a full-rate production decision  on Fire Scout in March 2010. The Navy has since deferred this decision  to May 2011.
section_title: Developmental Challenges with Individual Systems Have Contributed to Mission Package Procurement Delays
paragraphs: The Navy is buying mission packages through an incremental approach by  incorporating systems into the respective mission packages when systems  achieve minimum performance requirements. Figure 4 illustrates the effect  that recent developmental challenges have had upon mission package  procurement plans.
In 2007, the Navy anticipated that a total of 16 mission packages would be  delivered by 2012 with all three types of mission packages reaching their  full configuration by that date. In recent years, the Navy has deferred  planned procurements of LCS mission packages due, in part, to  developmental challenges facing mission systems. Under the Navy’s 2010  plan, the Navy plans to acquire 8 mission packages by 2012 with no  mission package having achieved its full configuration of capabilities.
To date, the Navy has taken delivery of five partial mission packages—two  surface warfare, two mine countermeasures, and one anti-submarine  warfare. The Navy delayed procurement of the fiscal year 2009 funded  mine countermeasures package due to technical issues and resulting  operational testing delays. Delivery of the two partial surface warfare  mission packages occurred in July 2008 and March 2010, respectively. The  first surface warfare package included two engineering development  models for the 30-millimeter gun, but did not include the launcher or  missiles for the Non-Line-of-Sight Launch System. The second surface  warfare mission package included the 30-millimeter gun module and the  launcher component—but no missiles—for the Non-Line-of-Sight Launch  System.
Because of planned configuration changes to the anti-submarine warfare  mission package, the Navy does not plan to buy additional quantities of  this package until the new configuration is settled. According to Navy  officials, recent warfighting analyses showed that the baseline anti- submarine warfare package did not contribute significantly to the anti- submarine warfare mission. The first package will undergo developmental  testing, with results used to inform decisions on future configuration. The  Navy conducted end-to-end testing on the first anti-submarine warfare  mission package in April 2009 and plans to continue developmental testing  in fiscal year 2010. During the 2009 end-to-end test, the Navy found that  the unmanned surface vehicle and its associated sensors will require  reliability and interface improvements to support sustained undersea  warfare.
section_title: Mission Package Procurement Delays Could Disrupt Program Plans for Simultaneously Acquiring Seaframes and Mission Packages and Will Limit the Ships’ Utility During Initial Deployments
paragraphs: While the Navy now plans to purchase 17 ships and 13 mission packages  between fiscal year 2011 and fiscal year 2015, developmental delays facing  key mission package systems have positioned the Navy to acquire  significant numbers of seaframes before mission packages are proven.  This development represents a reversal for the LCS program. In prior  years, the Navy deferred purchase of mission packages to account for  delays in constructing seaframes. However, as key mission package  systems entered operational testing—producing less than successful  results—and seaframe design and construction progressed, planned  alignment of seaframe and mission package purchases suffered further  disruption.
Until mission package performance is proven, the Navy risks investing in a  fleet of ships that does not deliver its promised capability. As the Navy  stated, the underlying strength of the LCS lies in its innovative design— interchangeable mission equipment that allows the ship to be used for  different missions. Fundamental to this approach is the capability to  rapidly install interchangeable mission packages into the seaframe. Absent  significant capability within its mission packages, seaframe functionality is  largely constrained to self-defense as opposed to mission-related tasks.
In addition, the Navy has deployed LCS 1 earlier than originally scheduled  and is evaluating a similar course for LCS 2. For these deployments, the  Navy is employing hybrid—limited and incomplete—versions of mission  packages. The package on LCS 1 includes one mission system (two 30- millimeter guns), the armed variant of the MH-60S helicopter, and the  maritime security module. According to Navy officials, deploying LCS 1  two years ahead of schedule allows them to incorporate lessons that can  only be learned in a deployment setting. LCS 1 will conduct operations  where it will be able to take advantage of its speed and will be equipped  with an incomplete version of the surface warfare package. Because the  surface warfare mission package is incomplete, the range of missions LCS  1 is capable of executing will be constrained during its initial deployment.
Furthermore, the surface warfare mission package onboard LCS 1 has yet  to be fully integrated with the seaframe and lacks key capabilities  necessary to defeat surface threats. For example, the 30-millimeter guns  have undergone testing with the LCS 1 seaframe, but have yet to be fully  integrated with the ship’s combat suite. Also, while the guns provide a  close range self-defense capability, Navy officials report LCS 1 is currently  unable to automatically transfer tracking data from the ship’s radar to the  30-millimeter guns. In addition, because of excess MH-60R helicopter  demands, the Navy has assigned the armed helo variant of the MH-60S  helicopter to LCS 1 for its maiden deployment. Although this MH-60S  variant carries air-to-surface missiles and crew-served side machine guns  (among other offensive capabilities), it does not have the multi-mode radar  found on the MH-60R—a shortfall that could constrain LCS 1’s ability to  execute surface warfare missions.
section_title: Mission Package Delays Coupled with Early Deployment of LCS Seaframes Require Deferral of Key Test Events
paragraphs: Neither LCS seaframe design—nor any of the three LCS mission  packages—has completed initial operational test and evaluation. Normally  after a lead ship completes its post-delivery maintenance period, initial  operational test and evaluation occurs—generally planned for within 1  year of ship delivery. In the case of LCS, initial operational test and  evaluation will encompass both seaframes and mission packages. Under  the program’s 2008 Test and Evaluation Master Plan, LCS 1 was to begin  operational testing with the initial systems in the mine countermeasures  mission package in the second quarter of fiscal year 2010. The mission  systems that were to be tested included the Remote Minehunting System,  the MH-60S helicopter (mine countermeasures variant), AN/AQS-20A mine  detecting sonar, Unmanned Surface Vehicle with Unmanned Surface  Sweep System, Airborne Mine Neutralization System, and Airborne Laser  Mine Detection System. However, as noted above, these mission systems  have experienced developmental challenges and none are ready to be  tested on the seaframe. The Navy has since postponed operational testing  of the mine countermeasures mission package to the second quarter of  fiscal year 2013, now planned to occur onboard LCS 2.
In addition, early deployments of seaframes postpone their availability to  complete planned testing events. For example, although LCS 1 deployed 2  years earlier than scheduled, its first operational testing event with a  mission package was delayed by 3 years. The Navy faces several risks in  deploying the LCS 1 before it has completed initial operational test and  evaluation. For example, Department of Defense testing officials stated  that because LCS 1 and LCS 2 are such revolutionary designs, the lead  ships should be put through a rigorous testing and evaluation process—in  a controlled environment—to best understand their capabilities and  limitations. Additionally, the testing officials reported that the ship’s  maintenance and support strategy was of significant concern to them.  According to the testing officials, LCS—more so than other ships—will  have to rely heavily on shore-based support, which is an unproven  concept.
section_title: Incomplete Cost Analyses in the LCS Program Have Undermined Program Progress
paragraphs: The Navy entered contract negotiations in 2009 for fiscal year 2010 funded  seaframes with an incomplete understanding of LCS program costs. These  contract negotiations proved unsuccessful, prompting the Navy to revise  its acquisition strategy for the program. The contractors’ proposals for  construction of the next three ships exceeded the approximate $1.4 billion  in funds the Navy had allocated in its fiscal year 2010 budget. In response,  the Navy revised its strategy to construct one seaframe design instead of  two for fiscal year 2010 ships and beyond in an effort to improve  affordability. Navy cost analyses completed prior to the failed negotiations  in 2009 lack several characteristics essential to a high quality cost  estimate. These characteristics include the completion of sensitivity and  uncertainty analyses and an independent review of the cost estimate. The  Navy plans to complete a more comprehensive cost estimate before award  of additional ship contracts in 2010.
subsections: section_title: Unsuccessful Contract Negotiations in Late 2009 Prompted the Navy to Restructure Its Acquisition Strategy for the LCS Program
paragraphs: The Navy budgeted $1.38 billion in fiscal year 2010 for construction of  three ships (LCS 5, LCS 6, and LCS 7) at a cost of $460 million each. The  Navy planned to award construction contracts for these ships in  November 2009, to include the purchase of at least one ship of each LCS  seaframe design. Navy officials were confident they had gained sufficient  knowledge from construction activities associated with the first four  seaframes to support a cost efficient, dual design strategy going forward.  As part of this strategy, the Navy solicited proposals from each of the LCS  prime contractors for construction of up to three ships on a fixed-price  basis. The industry teams returned their proposals in August 2009 and,  according to Navy officials, included pricing significantly above the Navy’s  expectations. Lockheed Martin and General Dynamics officials stated that  the fixed-price terms the Navy sought prompted a forthright assessment of  remaining program risks—including technical, design, and funding  uncertainties—and subsequent pricing of that risk in their proposals.
Due to the sharp differences between the Navy cost expectations and the  contractor proposals with respect to LCS pricing, the Navy concluded the  negotiations without awarding contracts for any new ships. Further, Navy  leadership stated it had no reasonable basis to find that the LCS program  would be executable going forward under the current acquisition strategy,  which prompted the Navy to outline a new acquisition strategy in  September 2009 aimed at improving program affordability by selecting one  design for the fiscal year 2010 ships and beyond. Under the terms of the  new strategy, the Navy will contract with a single source on a fixed-price  basis for up to 10 ships (2 ships awarded per year) through fiscal year  2014. The strategy also outlines plans to issue a second solicitation for up  to another 5 ships to be constructed at a separate yard with awards  planned between fiscal years 2012 and 2014. The first source will provide  the combat systems for the 5 additional ships constructed by the second  shipyard. Navy officials stated that the new acquisition approach will  produce cost benefits attributable to near-term competitive pricing  pressures between the two current LCS shipbuilding teams, economic  order quantity purchases of key materials, efficiencies associated with  potentially moving to a single, common combat system, and significantly  reduced total ownership costs for the Navy.
While the new acquisition strategy for the LCS program promises  improved affordability, the Navy’s failure to recognize the unexecutable  nature of the previous strategy—before engaging in the costly, time  consuming process of requesting proposals—has not come without  penalty. Contract awards for LCS 5, LCS 6, and LCS 7 have been delayed  by at least 9 months, subsequently disrupting planned workloads—and  potentially increasing the overhead costs charged to existing Navy  contracts—in the LCS shipyards. Most importantly, however, the  unsuccessful negotiations and revised strategy represent the latest delay  to delivery of promised capabilities to the fleet, which is depending heavily  on LCS to take over several current and future missions. Figure 5  highlights the resequencing of key LCS program events following the  revised acquisition strategy, which delayed contract awards for the fiscal  year 2010 funded ships.
Under the new acquisition strategy, the LCS program must complete a  major program review (milestone B) before award of fiscal year 2010 ship  contracts. The previous acquisition strategy deferred this review until June  2011—after award of the three fiscal year 2010 ship contracts. While  holding the milestone B decision earlier than planned is an improvement,  most ship programs align milestone B with the decision to authorize the  start of detail design—a decision that dates back to 2004 in the LCS  program.
In support of the milestone B review, Department of Defense policy  requires the Navy to provide its own cost estimate for the program. In  response, the Navy plans to submit a program life cycle cost estimate,  which will be completed by Naval Sea Systems Command’s Cost  Engineering and Industrial Analysis division. In addition, federal statute  requires that for major defense acquisition programs (including LCS), an  independent estimate of life cycle costs be prepared and provided to the  milestone decision authority before the approval is given to proceed with  Engineering and Manufacturing Development. The responsibility for the  independent cost estimate is assigned to the Director, Cost Assessment  and Program Evaluation within the Department of Defense.
section_title: Navy Cost Analyses Completed Ahead of Contract Negotiations in 2009 Lacked Key Elements Needed to Ensure High Quality
paragraphs: Because the Department of Defense has not yet completed a milestone B  review of the LCS program—to include development and evaluation of a  comprehensive Navy cost estimate and independent cost estimate—the  typical mechanisms and processes for assessing program affordability  were not carried out ahead of the Navy’s 2009 contract negotiations, which  turned out to be unsuccessful. Instead, Navy officials reported their cost  expectations for fiscal year 2010 funded seaframes were largely framed by  the pricing agreements reached with Lockheed Martin and General  Dynamics for LCS 3 and LCS 4 construction, respectively. Apart from this  data, the Navy had few alternative sources available that forecast LCS  program costs. Most prominent of these sources was analysis completed  in June 2009 as part of the LCS total ownership cost baseline estimate.  The Navy chartered this estimate to investigate ways it could alter the LCS  seaframe designs to reduce its total ownership cost in the program. The  estimate reflected the life cycle costs of the proposed two-design, 55 ship  LCS class—not including mission packages—and was divided into  sections covering (1) research, development, testing, and evaluation; (2)  procurement; and (3) operations and support costs. The estimate also  updated a set of previous Navy and Department of Defense estimates for  LCS 3 and LCS 4 that were completed in November 2008 in support of the  program’s milestone A-Prime review and March and May 2009 contract  awards for those ships.
Our analysis of the procurement section of the LCS total ownership cost  baseline found the estimate lacks several characteristics essential to a  high-quality cost estimate. To complete this analysis, we compared the  Navy’s estimate to best practices criteria, as outlined in GAO’s Cost  Estimating and Assessment Guide and presented in appendix II. These  criteria characterize high-quality cost estimates as those that are credible,  comprehensive, well-documented, and accurate. Table 7 highlights the key  findings of our analysis.
In developing the LCS total ownership cost baseline estimate, the Navy  excluded certain key costs, used overly optimistic assumptions,  inadequately documented its analyses, and did not perform analyses  needed to identify levels of confidence and certainty in the cost estimate.  As a result of these weaknesses, the LCS program total ownership cost  baseline estimate is not reliable for decision making.
Cost estimates are well-documented when they can be easily repeated or  updated and can be traced to original sources through auditing. Rigorous  documentation increases the credibility of an estimate and helps support  an organization’s decision making process. The documentation should  explicitly identify the primary methods, calculations, results, rationales,  assumptions, and sources of the data used to generate each cost element.  All the steps involved in developing the estimate should be documented so  that a cost analyst unfamiliar with the program can recreate the estimate  with the same result.
The level of documentation detailing the Navy’s LCS procurement cost  model is insufficient for someone unfamiliar with the program to easily  recreate the estimate. The level of detail in the total ownership cost  estimate does not reflect the full level of detail available for the LCS  program, leaving managers with incomplete information on which to base  program decisions. The Navy documented the LCS technical baseline in a  July 2006 report—the Cost Analysis Requirements Description— describing program requirements, purpose, technical characteristics,  development plan, acquisition strategy, operational plan, and risks. Navy  estimators, however, did not rely upon this document in developing their  estimate because the Cost Analysis Requirements Document is out of date  and does not reflect current program approaches, seaframe  configurations, and developmental challenges. Alternatively, Navy cost  estimators relied on the current LCS build specification, ship weight  reports, and known design changes to inform their understanding of the  ships’ technical characteristics. Since then, the Navy reports it has updated  the LCS Cost Analysis Requirements Document in anticipation of the  program’s milestone B review.
In addition, the Navy divided the procurement content of its estimate into  cost categories for plans, basic construction, change orders, government  furnished equipment, other items, and outfitting and post-delivery. For  most of these elements, Navy cost estimators relied on historical data and  subject matter experts as sources of data. For the basic construction cost  element, the Navy relied primarily on contractor cost proposals dating to  May 2008 and November 2008. According to the Navy, the proposal data  provided an accurate starting point for basic construction cost modeling  because industry teams were instructed to base their proposals on their  actual labor and material costs and/or estimates for lead ships. The Navy  reports it traced the proposal data to November 2008 cost performance  reports submitted by the contractors. The Navy then adjusted the  proposals’ estimates for labor hours and materials to reflect, for instance,  known changes in pricing.
However, as we recently reported, earned value management systems in  each of the LCS shipyards do not meet Defense Contract Management  Agency requirements for validation. Consequently, cost and schedule  data reported by the prime contractors cannot be considered fully  reliable. The LCS cost estimate does not include any evidence that the  Navy adjusted its basic construction cost estimates to account for this  uncertainty.
subsections: section_title: Comprehensive
paragraphs: Estimates are comprehensive when they contain a level of detail that  ensures that all pertinent costs are included and no costs are double- counted. It is important to ensure the completeness, consistency, and  realism of the information contained in the cost estimate.
The Navy chartered a working group at the outset of the total ownership  cost baseline estimate. Group members included representatives from the  Naval Center for Cost Analysis; Naval Sea Systems Command’s Ship  Engineering and Logistics, Maintenance, and Industrial Operations  directorates; Chief of Naval Operations’ Surface Warfare directorate,  Space and Naval Warfare Systems Command; Program Executive Office  for Integrated Warfare Systems; and the RAND Corporation. This group  developed a study plan for the LCS estimate that identified goals, set  deadlines for completing key tasks, and outlined required resources. In  addition, the group met biweekly and provided cross-checks and  verification to the Navy cost team’s estimating assumptions and results.  Upon completion of the procurement cost analysis, Navy estimators  briefed out results to the Commander, Naval Sea Systems Command, in  June 2009.
Further, our analysis found that the Navy’s LCS estimate identifies ground  rules and assumptions from which the estimate is derived. However, it  does not identify potential effects that changes to key assumptions—such  as allocation of ships between contractors and changes to the technical  baseline of the ship—could have upon the cost estimate. Also, the estimate  does not identify how budget constraints could affect program plans or the  potential effects that continued design refinements to the lead ships may  have upon construction cost outcomes of follow-on ships.
In addition, the Navy did not complete evaluations of risk distributions for  its cost estimating assumptions. Alternatively, cost estimators relied on  discussions with ship designers, engineers, and technicians from the Navy  and contractors to identify the scope of certain procurement cost  elements. Navy cost analysts report they will complete evaluations of risk  distributions for assumptions used in the LCS program life cycle cost  estimate being developed for the milestone B review.
The Navy also relied upon industry-provided work breakdown structures  for LCS 1, LCS 2, and LCS 3—that it then mapped to its own ship work  breakdown structure—to identify the work tasks necessary to deliver LCS  seaframes. Following completion of the total ownership cost estimate, the  Navy received the LCS 4 work breakdown structure from industry, which  it will use in developing the milestone B program life cycle cost estimate.
section_title: Accurate
paragraphs: Estimates are accurate when they are based on an assessment of the costs  most likely to be incurred. Therefore, when costs change, best practices  require that the estimate be updated to reflect changes in technical or  program assumptions and new phases or milestones.
The total ownership cost baseline estimate discretely estimated LCS 3 and  LCS 4 basic construction costs separate from the cost performance  outcomes being realized on LCS 1 and LCS 2. This approach was used  because, according to Navy cost estimators, distinguishing nonrecurring  work on the first two ships from work that would recur on future ships—a  necessary step for deriving follow-on ship costs—was too challenging an  undertaking given the major design changes and construction rework that  occurred on the lead ships. Further, LCS 3 and LCS 4 represented a slightly  different technical baseline from LCS 1 and LCS 2, leading the Navy to  judge it more appropriate to use the initial follow-on ships as the basis of  estimates for future ships.
After developing their cost estimates for LCS 3 and LCS 4, the Navy  estimators applied a 94 percent learning curve to the basic construction  cost elements to arrive at construction estimates for future ships of each  design. However, because Austal USA and Marinette Marine have only  recently begun building complex Navy ships such as LCS, the historical  data available to Navy cost analysts for deriving an accurate learning curve  was constrained. As a result, the Navy developed its LCS learning curve  based primarily on construction outcomes for alternative vessels and, in  some cases, alternative shipyards including (1) 14 Coast Guard buoy  tenders built at Marinette Marine, (2) 13 Cyclone-class coastal patrol ships  built at Bollinger Shipyards, and (3) 10 patrol craft built at Austal facilities  in Australia. According to the Navy, this data produced a widely varied  range of potential LCS learning curves, which contributed to the Navy’s  decision to arrive at a curve for LCS that was toward the high end of the  range (i.e., more conservative).
Navy cost estimators stated that their current work to develop a program  life cycle cost estimate for LCS precludes future updates to the total  ownership cost baseline. Additional cost estimates post-milestone B in the  program will rely on the program life cycle cost estimate—and  corresponding independent cost analysis—as starting points.
section_title: Credible
paragraphs: Estimates are credible when they have been cross-checked with an  independent cost estimate and when a level of uncertainty associated with  the estimate has been identified. An independent cost estimate provides  the estimator with an unbiased test of the reasonableness of the estimate  and reduces the cost risk associated with the project by demonstrating  that alternative methods generate similar results.
Our analysis found that the total ownership cost baseline was not  compared to an independent cost estimate. Comparing against an  independent cost estimate provides an unbiased test of whether a program  office or service-level cost estimate is reasonable. It is also used to identify  risks related to budget shortfalls or excesses. According to Navy officials,  the total ownership cost estimate was not tied to a major program  milestone. As such, development of a corresponding independent cost  estimate was not required. The Navy did, however, complete peer reviews,  engage in discussions with program officials for classes of ships used as  analogies in the estimate, and utilize expert opinion and work groups to  cross-check for accuracy and omissions.
In addition, we found the Navy did not complete sensitivity or uncertainty  analyses for LCS procurement cost elements. A sensitivity analysis  provides a range of costs that span a best and worst case spread. In  general, it is better for decision makers to know the range of potential  costs that surround a point estimate and the reasons behind what drives  that range than to just have a point estimate from which to make  decisions. Sensitivity analysis can provide a clear picture of both the high  and low costs that can be expected, with discrete reasons for what drives  them. Uncertainty analysis provides the basis for adjusting estimates to  reflect unknown facts and circumstances that could affect costs, and it  identifies risk associated with the cost estimate. In order to inform  decision makers about the likelihood of success, an uncertainty analysis  should be performed for every cost estimate, as an organization varies the  effects of multiple elements on costs, and as a result, can express a level of  confidence in the point estimate. Further, because numerous risks can  influence the estimate, they should be examined for their sources of  uncertainty and potential effect, and they should be modeled to determine  how they can affect the uncertainty of the cost estimate.
Navy cost estimators identified major procurement cost drivers in the  program, but do not plan to complete sensitivity or uncertainty analyses  for those drivers until shortly before milestone B. Navy officials cited the  change to the program’s acquisition strategy shortly following completion  of the total ownership cost estimate as their rationale for deferring these  analyses. Without sensitivity and uncertainty analyses, the Navy cannot  fully account for the effect various risks can have on the overall total  ownership cost estimate.
section_title: Conclusions
paragraphs: The Navy is counting on LCS as its primary means to defeat sea mines,  counter low-end surface threats, and prosecute enemy submarines in  coastal waters. Further, LCS—with its planned 55 seaframes and 64  mission packages—represents a large component of the Navy’s future  surface fleet, making it key to the Navy’s ability to maintain global  presence.
However, the Navy’s ability to deliver a capable, affordable LCS remains  unproven. Staying within budget will require the Navy to achieve design  stability before beginning construction of future ships. Moreover, LCS  testing remains in its infancy, as the first operational testing event  involving a seaframe and partial mission package has been deferred to  fiscal year 2013. In addition, the Navy now expects individual mission  package systems to remain in development through 2017. Until LCS  capabilities are demonstrated through operational testing, the Navy cannot  be certain that the seaframes and mission packages it is buying will be  able to execute the missions that the fleet plans to assign to LCS vessels.
Further, decisions to deploy the lead ship early and complete previously  unplanned maintenance periods have rendered current program test plans  obsolete. Testing delays to key mission package elements—followed by  prudent Navy decisions to defer new procurements of these systems— have created an imbalance between seaframe and mission package  acquisition plans. For example, the Navy now plans to fund construction  of 17 seaframes between fiscal years 2011 and 2015—whereas only 13  mission packages will be purchased during that time. This situation could  be exacerbated should the Navy encounter additional difficulties resolving  the substantial technical issues facing the mine countermeasures package.
In addition, although the Navy has emphasized the importance of  affordability to successful outcomes in the LCS program, it continues to  make key investment decisions without a clear understanding of program  costs. For LCS, the Navy determined it appropriate to award contracts for  four ships and conduct negotiations for three more without completing a  detailed, programwide, independent cost estimate—a strategy that  contributed to less than optimal results. High-quality cost estimates are  well-documented, comprehensive, accurate, and credible—characteristics  that are not fully embodied in the Navy’s most recent cost estimate for  LCS procurement. Shortfalls include the lack of sensitivity and uncertainty  analyses and an independent review of the cost estimate. Continuing  technical challenges and design changes on initial seaframes further  complicate the Navy’s efforts to identify future LCS costs. Until these  issues are resolved, and a high-quality estimate of program costs is  developed, the Navy cannot be confident that the LCS capabilities it  promises can be attained at prices it is willing to pay.
section_title: Recommendations for Executive Action
paragraphs: We recommend the Secretary of Defense take the following four actions:  To attain the level of knowledge needed to retire design risk and reduce  construction disruptions, ensure changes identified in building and testing  the first four ships are incorporated into the basic and functional design by  the start of construction for future LCS seaframes.
To provide a meaningful framework for evaluating seaframe and mission  package performance, update the LCS test and evaluation master plan to  (1) account for any early deployments of seaframes and the significant  developmental challenges faced by key mission package systems and (2)  identify alternative approaches for completing seaframe and mission  package initial operational test and evaluation.
To safeguard against excess quantities of ships and mission packages  being purchased before their combined capabilities are demonstrated,  update the LCS acquisition strategy to account for operational testing  delays in the program and resequence planned purchases of ships and  mission packages, as appropriate.
To provide a sound basis for future LCS investment decisions, ensure that  future LCS cost estimates—including the program life cycle cost estimate  currently planned for milestone B—are well-documented, comprehensive,  accurate, and credible.
section_title: Agency Comments and Our Evaluation
paragraphs: The Department of Defense agreed with all of our recommendations.  However, in responding to our recommendation to ensure changes  identified in building and testing the first four ships are incorporated into  the basic and functional design by the start of construction for future LCS  seaframes, the department stated that the program can use existing ship  and class design services contracts to execute additional changes after  contract award. As our prior work has shown, however, this practice has  been tried before in Navy shipbuilding programs and has consistently  contributed to ship deliveries that are over cost and behind schedule. As  such, we would expect the Navy to set the bar extremely high for making  design changes to ships that are already under construction.
The department’s written comments can be found in appendix III of this  report. The department also provided technical comments, which were  incorporated into the report as appropriate.
We are sending copies of this report to interested congressional  committees, the Secretary of Defense, and the Secretary of the Navy. The  report is also available at no charge on the GAO Web site at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-4841 or martinb@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix IV.
section_title: Appendix I: Scope and Methodology
paragraphs: This report evaluates Littoral Combat Ship (LCS) planning and  implementation efforts. Specifically, we (1) identified technical, design,  and construction challenges to completing the first four seaframes within  current cost and schedule estimates, (2) assessed the Navy’s progress  developing and fielding mission packages, and (3) evaluated the quality of  recent Navy cost analyses for seaframes and their effect on program  progress.
To identify challenges in completing the first four seaframes, we analyzed  Department of Defense and contractor-generated documents that  addressed technical challenges and cost and schedule performance for  LCS seaframes including Navy test reports; Navy Supervisor of  Shipbuilding reports; monthly contract performance reports; integrated  baseline reviews; reports to Congress; sea trial reports for the first two  ships; and construction progress briefings. To identify design changes and  to understand the impact of these changes to the construction processes  for seaframes, we reviewed LCS contracts and change orders; program  schedules for LCS 3 and LCS 4; monthly contract performance reports;  weekly Supervisor of Shipbuilding reports; and quarterly ship production  progress conference briefings. We also reviewed information from  contractors outlining process improvements and capital investments at  each of the LCS shipyards aimed at increasing capability and capacity  needed to support efficient construction of LCS seaframes. To further  corroborate documentary evidence and gather additional information in  support of our review, we conducted interviews with relevant Navy and  industry officials responsible for managing the design and construction of  LCS seaframes, such as the LCS Seaframe program office; Program  Executive Office, Ships; Supervisor of Shipbuilding officials; Lockheed  Martin and General Dynamics (LCS prime contractors); Marinette Marine  and Austal USA (LCS shipbuilders); and MTU (LCS 2 and LCS 4 diesel  engine vendor). We also held discussions with LCS technical authorities,  testing agents, and requirements officers from Naval Sea Systems  Command’s Ship Engineering directorate; Director, Operational Test and  Evaluation; American Bureau of Shipping; Commander, Navy Operational  Test and Evaluation Force; Office of the Chief of Naval Operations’  Surface Warfare directorate; and Naval Surface Warfare Center-Panama  City division.
To assess the Navy’s progress developing and fielding mission packages,  we analyzed documents outlining LCS mission package development plans  and performance including program schedules, test reports, and budget  submissions. In order to evaluate the realism of mission package testing  and procurement plans, we analyzed and compared mission package  development schedules and test reports against LCS seaframe  construction, delivery, and testing schedules. To further corroborate  documentary evidence and gather additional information in support of our  review, we held discussions with Navy program offices and Department of  Defense agencies responsible for acquiring and testing key LCS mission  systems including the Program Executive Office for Littoral and Mine  Warfare; LCS Mission Modules program office; Unmanned Maritime  Vehicles program office; and the Mine Warfare program office; Director,  Operational Test and Evaluation; Commander, Operational Test and  Evaluation Force, Navy. To gather additional information on remaining  risks to mission package development and integration, we interviewed  relevant Navy officials from the MH-60R Helicopter program office, MH- 60S Helicopter program office, Navy and Marine Corps Tactical Multi- Mission Unmanned Aerial Systems program office, and Naval Surface  Warfare Center-Panama City division.
To evaluate the quality of recent Navy cost analyses for LCS seaframes, we  compared the Navy’s total ownership cost baseline estimate for LCS  procurement to best practices criteria as outlined in GAO’s Cost  Estimating and Assessment Guide. These criteria characterize high-quality  cost estimates as those that are credible, comprehensive, well- documented, and accurate. To supplement our analysis and gain additional  visibility into the Navy’s process for developing its LCS estimate, we  interviewed officials from Naval Sea Systems Command’s Cost  Engineering and Analysis directorate; LCS Seaframe program officials; and  the Director, Cost Assessment and Program Evaluation.
We conducted this performance audit from July 2009 to August 2010 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix II: GAO Methodology Used to Perform Cost Estimating Analysis
paragraphs: To evaluate estimated Littoral Combat Ship (LCS) seaframe procurement  costs as outlined in the Navy’s total ownership cost baseline (August  2009), GAO employed criteria from our Cost Estimating and Assessment  Guide. In developing this guide, GAO cost experts identified 12 steps  consistently applied by cost-estimating organizations throughout the  federal government and industry and considered best practices for the  development of reliable cost estimates. These 12 steps—and their related  measures—are identified below.
Step One: Define the Estimate’s Purpose    Are the purpose and scope of the cost estimate defined and  documented?  a.  Is the level of detail the estimate is conducted at consistent with  the level of detail available for the program?  b.  Have all applicable costs been estimated, including life cycle costs?  c.  Is the scope of the estimate defined and documented?
Step Two: Develop the Estimating Plan    Did the team develop a written study plan that:  a.  Determined the estimating team’s composition and whether the  team is from a centralized office;  b.  Identified which subject matter experts the team will rely on for  c.  Outlined the estimating approach (see Step four);  d.  Identified a master schedule for completing the estimate that  provided adequate time to do the work.
Step Three: Define the Program Characteristics  Is there a documented technical baseline description that is contained  in a single document? If yes, does it include the following:  a.  What the program is supposed to do—requirements;  b.  How the program will fulfill its mission—purpose;  c.  What it will look like—technical characteristics;  d.  Where and how the program will be built—development plan;  e.  How the program will be acquired—acquisition strategy;  f.  How the program will operate—operational plan;  g.  Which characteristics affect cost the most—risk.
Step Four: Determine the Estimating Structure  Is there a defined work breakdown structure (WBS) and/or cost  element structure?  a.  Is the WBS product-oriented, traceable to the statement of work,  and at an appropriate level of detail to ensure that cost elements  are neither omitted nor double-counted?  b.  Is the WBS standardized so that cost data can be collected and  used for estimating future programs?  c.  Does the cost estimate WBS match the schedule and earned value  management (EVM) WBS?  d.  Is the WBS updated as the program becomes better defined and to  reflect changes as they occur?  e.  Is there a WBS dictionary that defines what is included in each  element and how it relates to others in the hierarchy?
Step Five: Identify Ground Rules and Assumptions    Are there defined ground rules and assumptions that document the  rationale and any historical data to back up any claims?  a.  Have risks associated with any assumptions been identified and  traced to specific WBS elements?  b.  Have budget constraints, as well as the effect of delaying program  content, been defined?  c.  Have inflation indices and their source been identified?  d.  If the program depends on a participating agency or agency’s  equipment have the effects of these assumptions not holding been  identified?  e.  Have items excluded from the estimate been documented and  f. explained?  If technology maturity has been assumed, does the estimate  address the effect of the assumption’s failure on cost and  schedule?  g.  Did cost estimators meet with technical staff to determine risk  distributions for all assumptions so they could use this information  for sensitivity and uncertainty analysis?
Step Six: Obtain the Data    Was the data gathered from valid historical actual cost, schedule, and  program and technical sources?  a.  Do the data apply to the program being estimated and have they  been analyzed for cost drivers?  b.  Have the data been collected from primary sources and adequately  documented as to the source, content, time, units, an assessment  of the accuracy of the data and reliability, and any circumstances  affecting the data?  c.  Are data continually collected and stored for future use?  d.  Did analysts meet with the data sources to better understand the  program and ask them about the data?  e.  Were the data reviewed and benchmarked against historical data  for reasonableness?  f.  Were the data analyzed using scatterplots and descriptive statistics  and were they normalized to account for cost, sizing units, etc. so  they are consistent for comparisons?
Step Seven: Develop the Point Estimate and Compare It to an Independent  Cost Estimate    Did the cost estimator consider various cost estimating methods like  analogy, engineering build up, parametric, extrapolating from actual  costs, and expert opinion (if none of the other methods can be used)?  a.  If the parametric method was used as the estimating method, were  the cost estimating relationships (CER) statistics examined to  determine its quality?  b.  Were learning curves used if there was much manual labor  associated with production and were production rate and breaks in  production considered?  c.  Was the point estimate developed by aggregating the WBS cost  estimates by one of the cost estimating methods?  d.  Were results checked for accuracy, double-counting, and  omissions and were validated with cross checks and independent  cost estimates?  e.  If software is a major component of the cost estimate were  software estimating best practices from Chapter 12 of the Guide  addressed?
Step Eight: Conduct a Sensitivity Analysis    Did the cost estimate include a sensitivity analysis that, using a range  of possible costs, identified the effects of changing key cost driver  assumptions or factors? Were the following steps taken:  a.  Key cost drivers were identified;  b.  Cost elements representing the highest percentage of cost were  determined and their parameters and assumptions were examined;  c.  The total cost was reestimated by varying each parameter between  its minimum and maximum range;  d.  Results were documented and the reestimate repeated for each  parameter that was a key cost driver;  e.  Outcomes were evaluated for parameters most sensitive to change.
Step Nine: Conduct Risk and Uncertainty Analysis    Was a risk and uncertainty analysis conducted that quantified the  imperfectly understood risks and identified the effects of changing key  cost driver assumptions and factors? Were the following steps  performed:  a.  A probability distribution was modeled for each cost element’s  uncertainty based on data availability, reliability, and variability;  b.  The correlation (i.e., relationship) between cost elements was  accounted for to capture risk;  c.  A Monte Carlo simulation model was used to develop a distribution  of total possible costs and an S curve showing alternative cost  estimate probabilities;  d.  The probability associated with the point estimate was identified.  e.  Contingency reserves were recommended for achieving the desired  f.  The risk-adjusted cost estimate was allocated, phased, and  converted to then year dollars for budgeting, and high-risk  elements were identified to mitigate risks;  g.  A risk management plan was implemented jointly with the  contractor to identify and analyze risk, plan for risk mitigation, and  continually track risk.
Step Ten: Document the Estimate    Did the documentation describe the cost estimating process, data  sources, and methods step by step so that a cost analyst unfamiliar  with the program could understand what was done and replicate it?  a.  Are supporting data adequate for easily updating the estimate to  reflect actual costs or program changes and using them for future  estimates?  b.  Did the documentation describe the estimate with narrative and  cost tables and did it contain an executive summary, introduction,  and descriptions of methods, with data broken out by WBS cost  elements, sensitivity analysis, risk and uncertainty analysis,  management approval, and updates that reflect actual costs and  changes?  c.  Did the detail address best practices and the 12 steps of high- quality estimates?  d.  Was the documentation mathematically sensible and logical?  e.  Did it discuss contingency reserves and how they were derived  from risk and uncertainty analysis?  f.  Did the documentation include access to an electronic copy of the  cost model and are both the documentation and the cost model  stored so that authorized personnel can easily find and use them  for other cost estimates?
Step Eleven: Present the Estimate to Management for Approval    Was there a briefing to management that included a clear explanation  of the cost estimate so as to convey its level of competence?  a.  Did the briefing illustrate the largest cost drivers by presenting  them logically with backup charts for responding to more probing  questions?  b.  Did the briefing include an overview of the program’s technical  foundation and objectives, the life cycle cost estimate in time- phased constant year dollars, a discussion of ground rules and  assumptions, the method and process for each WBS cost element  estimate including data sources, the results of sensitivity and  risk/uncertainty analysis along with a confidence interval, the  comparison of the point estimate to an independent cost estimate  with a discussion of any differences and whether the point  estimate is reasonable, an affordability analysis based on funding  and contingency reserves, a discussion of any concerns or  challenges, conclusions, and recommendations for approval?  c.  Was any feedback from the briefing including management’s  acceptance of the estimate acted on and recorded in the cost  estimate documentation?
Step Twelve: Update the Estimate to Reflect Actual Costs and Changes  Is there a process for the estimating team to update the estimate with  actual costs as it becomes available?  a.  Was the estimate updated to reflect changes in technical or  program assumptions and was there a discussion how these  changes affected the cost estimate?  b.  Did the cost estimates get replaced with actual costs? Were the  actual costs from an EVM system?  c.  Did the estimate discuss lessons learned for elements whose actual  costs or schedules differed from the estimate?
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact named above, key contributors to this report  were Karen Zuckerstein, Assistant Director; Greg Campbell; Christopher  R. Durbin; Kristine Hassinger; Jeremy Hawk; Jasmin Jahanshahi; Julia P.  Jebo; Jason Kelly; and Amber N. Keyser.