F-35 Joint Strike Fighter: Problems Completing Software Testing May Hinder Delivery of Expected Warfighting Capabilities
Published: 2014-03-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The F-35 Lightning II, also known as the Joint Strike Fighter, is DOD's most costly and ambitious acquisition program. The program seeks to develop and field three aircraft variants for the Air Force, Navy, and Marine Corps and eight international partners. The F-35 is integral to U.S. and international plans to replace existing fighter aircraft and support future combat operations. Total U.S. planned investment in the F-35 program is approaching $400 billion to develop and acquire 2,457 aircraft through 2037, plus hundreds of billions of dollars in long-term spending to operate and maintain the aircraft.
The National Defense Authorization Act for Fiscal Year 2010 mandated that GAO review the F-35 acquisition program annually for 6 years. In this, GAO's fifth annual report on the F-35, GAO assesses the program's (1) ongoing development and testing, (2) long-term affordability, and (3) manufacturing progress.
GAO reviewed and analyzed manufacturing data through December 2013, program test plans, and internal DOD analyses, and spoke with DOD, program, and contractor officials.
section_title: What GAO Found
paragraphs: Delays in developmental flight testing of the F-35's critical software may hinder delivery of the warfighting capabilities the military services expect. F-35 developmental flight testing comprises two key areas: mission systems and flight sciences. Mission systems testing verifies that the software-intensive systems that provide critical warfighting capabilities function properly and meet requirements, while flight sciences testing verifies the aircraft's basic flying capabilities. Challenges in development and testing of mission systems software continued through 2013, due largely to delays in software delivery, limited capability in the software when delivered, and the need to fix problems and retest multiple software versions. The Director of Operational Test and Evaluation (DOT&E) predicts delivery of warfighting capabilities could be delayed by as much as 13 months. Delays of this magnitude will likely limit the warfighting capabilities that are delivered to support the military services' initial operational capabilities—the first of which is scheduled for July 2015—and at this time it is not clear what those specific capabilities will be because testing is still ongoing. In addition, delays could increase the already significant concurrency between testing and aircraft procurement and result in additional cost growth. Without a clear understanding of the specific capabilities that will initially be delivered, Congress and the military services may not be able to make fully informed resource allocation decisions. Flight sciences testing has seen better progress, as the F-35 program has been able to accomplish nearly all of its planned test flights and test points. Testing of the aircraft's operational capabilities in a realistic threat environment is scheduled to begin in 2015. The program has continued to make progress in addressing some key technical risks.
To execute the program as planned, the Department of Defense (DOD) will have to increase funds steeply over the next 5 years and sustain an average of $12.6 billion per year through 2037; for several years, funding requirements will peak at around $15 billion. Annual funding of this magnitude clearly poses long-term affordability risks given the current fiscal environment. The program has been directed to reduce unit costs to meet established affordability targets before full-rate production begins in 2019, but meeting those targets will be challenging as significant cost reductions are needed. Additionally, the most recent cost estimate for operating and supporting the F-35 fleet is more than $1 trillion, which DOD officials have deemed unaffordable. This estimate reflects assumptions about key cost drivers the program can control, like aircraft reliability, and those it cannot control, including fuel costs, labor costs, and inflation rates. Reliability is lower than expected for two variants, and DOT&E reports that the F-35 program has limited additional opportunities to improve reliability.
Aircraft manufacturing continued to improve in 2013, and management of the supply chain is evolving. As the number of aircraft in production has increased, critical learning has taken place and manufacturing efficiency has improved. For example, the prime contractor has seen reductions in overall labor hours needed to manufacture the aircraft, as expected. In 2013, the contractor delivered 35 aircraft to the government, 5 more than it delivered in 2012 and 26 more than it delivered in 2011. The prime contractor has put in place a supplier management system to oversee key supplier performance.
section_title: What GAO Recommends
paragraphs: GAO recommends that DOD assess and identify the specific capabilities that realistically can be delivered to the military services to support their respective initial operational capabilities, and share its findings with the Congress and military services prior to July 2015. DOD concurred with this recommendation.

section_title: Letter
paragraphs: With estimated acquisition costs approaching $400 billion, the F-35  Lightning II—also known as the Joint Strike Fighter—is the Department of  Defense’s (DOD) most costly and ambitious acquisition program. The  program is developing and fielding a family of next generation fighter  aircraft, incorporating low observable (stealth) technologies as well as  advanced sensors and computer networking capabilities for the United  States Air Force, Navy, and Marine Corps as well as eight international  partners. The F-35 family is comprised of three aircraft variants: (1) a  conventional takeoff and landing (CTOL) variant, (2) a short takeoff and  vertical landing (STOVL) variant, and (3) a carrier-suitable variant (CV).  The F-35 is integral to U.S. and partner plans to replace existing fighter  aircraft and support future combat operations. According to current plans,  the U.S. portion of the program will require annual acquisition funding of  more than $12 billion on average through 2037 to complete development  and procure a total of 2,457 aircraft. In addition, the F-35 fleet is  estimated to cost around $1 trillion to operate and support over its  lifetime. In a time of austere federal budgets, cost projections of this  magnitude pose significant fiscal challenges.
See related GAO products at the end of this report.
DOD completed an extensive restructuring of the F-35 program and we  reported in June 2012 and again in March 2013 that the restructuring  actions should lead to more achievable and predictable outcomes, albeit  at higher cost and longer timeframes for testing and delivering capabilities  to the warfighter than originally planned.
The National Defense Authorization Act for Fiscal Year 2010 mandated  GAO to review the F-35 acquisition program annually for 6 years. This is  our fifth report under that mandate, and in it, we assess the program’s (1)  ongoing development and testing, (2) funding and long-term affordability,  and (3) manufacturing progress.
To conduct our work, we reviewed and analyzed program briefings,  management reports, program test results, and internal DOD program  analyses. We discussed key aspects of F-35 performance with both  military and private contractor test pilots. We interviewed F-35 program  and aircraft prime contractor officials to discuss developmental testing.  We also collected developmental test plans, and data on test  achievements to assess program progress through December 2013. We  obtained current program acquisition and life-cycle sustainment cost  estimates, reviewed the supporting documentation and discussed the  development of those estimates with DOD and prime contractor officials  instrumental in producing them. We toured F-35 manufacturing and test  facilities and obtained and analyzed production and supply chain data as  of December 2013. We assessed the reliability of DOD and contractor  data by reviewing existing information about the data, and interviewing  agency officials knowledgeable about the data. We determined that the  data were sufficiently reliable for the purposes of this report. We also  discussed ongoing manufacturing process improvements with prime  contractor and Defense Contract Management Agency (DCMA) officials.  Appendix Il contains a more detailed description of our scope and  methodology.
We conducted this performance audit from June 2013 to March 2014 in  accordance with generally accepted government auditing standards.  Those standards required that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: program baseline was finalized in March 2012, and since that time costs  have remained relatively stable. Table 1 notes the significant cost,  quantity, and schedule changes from the original program baseline and  the relative stability since the new baseline was established.
At the time the new F-35 acquisition program baseline was finalized, it did  not identify new initial operational capability (IOC) dates for the three  military services. The following year DOD issued a memorandum noting  that Marine Corps and Air Force were planning to field initial operational  capabilities in 2015 and 2016, respectively, and that the Navy planned to  field its initial capability in 2018. The memorandum emphasized that the  Marine Corps and Air Force initial operational capabilities would be  achieved with aircraft that possess initial combat capabilities, and noted  that those aircraft would need additional lethality and survivability  enhancements to meet the full spectrum of warfighter requirements in the  future. These new parameters represented a delay of 5 to 6 years from  the program’s initial 2001 baseline and a reduction in the capabilities  expected at IOC.
In March 2005 we recommended that DOD implement an evolutionary,  incremental approach to developing and fielding the F-35—then known as  the Joint Strike Fighter—to ensure that the warfighters would receive an  initial combat capability that, at a minimum, would meet their most  immediate needs. Again in March 2010, we recommended that DOD  identify the absolute minimum combat capabilities that would be  acceptable by each of the military services to field their initial operational  capabilities and establish reasonable, realistic timeframes for achieving  those requirements. In both instances, we noted that the military  services should consider trading off desired capabilities in order to more  rapidly field aircraft with an initial set of useable capabilities and that any  capabilities not needed to meet immediate warfighting needs should be  deferred to a future development increment. In both instances, DOD  agreed with the intent of our recommendation, but believed that its  existing program management practices were sufficient.
section_title: Testing of Critical Software Poses a Significant Challenge to the Delivery of F- 35 Warfighting Capabilities
paragraphs: Delays in the testing of critical mission systems software have put the  delivery of expected warfighting capabilities to the Marine Corps at risk,  and could affect the delivery of capabilities to the Air Force and Navy as  well. F-35 developmental flight testing is separated into two key areas:  mission systems and flight sciences. Mission systems testing is done to  verify that the software and systems that provide critical warfighting  capabilities function properly and meet requirements, while flight science  testing is done to verify the aircraft’s basic flying capabilities. In a March  2013 report we found that development and testing of mission systems  software was behind schedule, due largely to delayed software deliveries,  limited capability in the software when delivered, and the need to fix  problems and retest multiple software versions. These same challenges  continued thorough 2013, and as a result progress in mission systems  testing has been limited. The Director of Operational Test and Evaluation  (DOT&E) predicts that the delivery of expected warfighting capabilities to  the Marine Corps could be delayed by as much as 13 months. Delays of  this magnitude could also increase the already significant concurrency  between testing and aircraft procurement and result in additional cost  growth. Although mission systems testing is behind, the F-35 program  has been able to accomplish nearly all of its planned flight science  testing. The program also continued to make progress in addressing key  technical risks, although some of that progress has been limited.
subsections: section_title: Limited Progress in Mission Systems Software Testing May Cause Delays and Add Cost to F-35 Development
paragraphs: While the F-35 program was able to accomplish all of the mission system  test flights it had planned in 2013, it did not accomplish all of the planned   falling short by 11 percent. The F-35 program planned to fly  test points,329 mission systems test flights and accomplish 2,817 test points in  2013. The program actually flew 352 test flights, exceeding the goal, but  only accomplished 2,518 test points. According to program and contractor  officials, slow progress in developing, delivering, and testing mission  systems software continues to be the program’s most significant risk  area. The F-35 program is developing and fielding mission systems  software capabilities in blocks: (1) Block 1, (2) Block 2A, (3) Block 2B, (4)  Block 3i, and (5) Block 3F. Each subsequent block builds on the  capabilities provided in the preceding blocks. Blocks 1 and 2A provide  training capabilities and are essentially complete, with some final  development and testing still underway. Blocks 2B and 3i provide initial  warfighting capabilities and are needed by the Marine Corps and Air  Force, respectively, to achieve initial operational capability. Block 3F is  expected to provide the full suite of warfighting capabilities, and is the  block the Navy expects to have to achieve its initial operational capability.
Developmental testing of Block 2B software is behind schedule and will  likely delay the delivery of expected warfighting capabilities. The delivery  of this software capability is of high near-term importance because it  provides initial warfighting capability for the overall F-35 program, and is  needed by the Marine Corps to field its initial operational capability in July  2015. As of January 2014, the program planned to have verified the  functionality of 27 percent of the software’s capability on-board the  aircraft, but had only been able to verify 13 percent. This leaves a  significant amount of work to be done before October 2014, which is  when the program expects to complete developmental flight testing of this  software block. According to DOT&E, Block 2B developmental testing will  not be completed as scheduled and could be delayed by as much as 13  months, as the program has had to devote time and resources to address  problems and completing development of prior software blocks. Delays of  this magnitude would mean that the Marine Corps will not likely have all  of the capabilities it expects in July 2015, its planned initial operational  capability date. At this time it is not clear exactly which of the expected  capabilities will be available as testing is still ongoing. The effects of these  delays compound as they also put the timely delivery of Air Force and  Navy initial operational capabilities at risk. The Air Force expects to field  its initial operational capability in August 2016 with Block 3i aircraft that  possess the same warfighting capabilities as the Marine Corps aircraft,  but have upgraded computer hardware. The Navy plans to achieve its  initial operational capability in August 2018 with Block 3F aircraft that  possess the full suite of F-35 warfighting capabilities.
Program and contractor officials have stated that while they recognize  that the program faces software risks, they still expect to deliver all of the  planned F-35 software capabilities to the military services as currently  scheduled. However, given the uncertainty in mission systems software  testing and the significance of the F-35 to future force structure plans, it is  important that the military services have a clear understanding of the  specific capabilities that they can realistically expect to receive, and those  capabilities that are not likely to be delivered by their initial operational  capability dates—the first of which is scheduled for July 2015. In addition,  because the F-35 is DOD’s most costly and ambitious acquisition  program, it is important that DOD keep Congress informed about the  status of the program. Without a clear understanding of the specific  capabilities that will initially be delivered, Congress and the military  services may not be able to make fully informed resource allocation  decisions.
Delays in mission systems software testing could also increase costs. As  currently planned, DOD expects to complete developmental flight testing  in 2017. If the flight test schedule is extended, the program may have to  retain testing and engineering personnel longer than currently expected,  which would increase development cost. DOD currently expects to have  invested $70.2 billion to procure 359 aircraft by 2017 when developmental  flight testing is scheduled to end. Our past reports have concluded that  purchasing aircraft while concurrently conducting developmental flight  testing increases the risk that problems will be discovered late in testing  and additional funding will be needed to rework aircraft that have already  been purchased. If F-35 procurement plans remain unchanged and  developmental testing continues into 2018, the cost risks associated with  concurrency will likely increase as DOD expects to have invested $83.4  billion in 459 aircraft by that point in time.
The F-35 contractor recognizes that additional testing efficiencies are  important in order to deliver capabilities on schedule and cost. One way it  plans to gain efficiency is to use test results from one F-35 variant to  close out test points for the other two variants in instances in which the  variants have common functions. According to test officials, most mission  systems testing can be accomplished on any variant and only a limited  amount of variant-specific testing is required. Contractor officials pointed  out that this type of efficiency would help mitigate some testing risk, but  they also recognized that it will still be difficult to make up the lost time in  the test program. They noted that delays in specific test events generally  impact the entire test schedule because the ability to conduct future  testing is often dependent on the completion of the earlier events.
section_title: Program Accomplished Most Flight Science Test Goals in 2013
paragraphs: The program accomplished nearly all of the flight sciences testing,  including weapons testing, it had planned for 2013. As of December  2013, the program had achieved half of the total number of test points  required to complete all flight science testing for the program. Figure 1  below shows the number of flight science test points planned and  accomplished for each F-35 variant as of December 2013, and also  identifies the total number of test points required for each variant to  complete flight science testing.
The program made progress despite the fact that flight testing was halted  twice at the beginning of the year to investigate and fix cracks in an  engine fan blade and leaky fuel hoses. In addition, program and  contractor officials emphasized that employee furloughs that occurred in  2013, due to mandatory sequestration, limited the amount of flight testing  that could be done during that time as well.science and weapons testing accomplishments included:   Some of the key flight
Conventional takeoff and landing variant – The program successfully  demonstrated the variant’s ability to launch AIM-120 missiles from its  internal weapons bay and to refuel while in flight. The program also  continued testing the aircraft’s ability to function at high vertical flight  angles, although program officials noted that the testing took longer  than expected. As of December 2013, the program had accomplished  59 percent of its total expected flight science test points for this  variant.
Short takeoff and vertical landing variant – The program successfully  demonstrated the STOVL’s ability to takeoff vertically, launch  weapons from its internal weapons bay, and dump fuel when needed.  In addition, the program conducted some testing of the variant at sea  on an amphibious assault ship—specifically the USS WASP. As of  December 2013, the program had accomplished 49 percent of its total  expected flight science test points for this variant.
Carrier-suitable variant – The program began testing the capability of  the aircraft to function at high vertical flight angles. In addition, the  program successfully demonstrated the aircraft’s ability to dump fuel  when needed. Program and contractor officials noted that the program  also made progress to begin testing to verify that the aircraft’s new  arresting hook system could successfully catch a cable on a set of  carrier arresting gear installed onshore at the Lakehurst facility. As of  December 2013, the program had accomplished 43 percent of its total  expected flight science test points for this variant.
Developmental testing is not the only testing that the program still has to  complete. The F-35 program is also scheduled to begin operational  testing in June 2015, to determine that the aircraft variants can effectively  perform their intended missions in a realistic threat environment.
section_title: Program Progress in Addressing Key Technical Risks Has Been Mixed
paragraphs: While the F-35 program made progress addressing some key technical  risks in 2013, it continued to encounter slower than expected progress in  developing the Autonomic Logistics Information System (ALIS). Over  time, we have reported on 4 areas of technical and structural risk that the  program identified during flight, ground, and lab testing that if not  addressed, could result in substantially degraded capabilities and mission  effectiveness. In 2013, we found that the program made the following  progress in each of those areas:
Helmet mounted display - provides flight data, targeting, and other  sensor data to the pilot, and is integral to reducing pilot workload and  achieving the F-35’s concept of operations. The original helmet  mounted display encountered significant technical deficiencies,  including display jitter, the undesired shaking of the visor display, and  latency, the perceivable lag that occurs in transmitting sensor data,  and did not meet warfighter requirements. The program made  adjustments to the helmet design, including adding sensors to lessen  the display jitter, and redesigning elements to minimize latency. The  program tested these design changes in 2013 and found that most of  the technical deficiencies had been adequately addressed, and that  the helmet’s performance was sufficiently suitable to support Marine  Corps initial operational capability in 2015. DOT&E and program test  pilots noted that the current night vision camera continues to have  problems. The program has identified a new camera that it believes  will address those problems, but that camera has not been fully tested  to verify its capabilities.
Arresting Hook System - allows the F-35 carrier-suitable variant to  engage landing wires on aircraft carriers, was redesigned after the  original hook system was found to be deficient. The program  determined that the original hook assembly was not strong enough to  reliably catch the wire and stop the airplane. As a result, the program  modified the hook system’s hydraulic components, and made  structural modifications to the plane. In March 2013, the program  completed a critical design review of the hook system to verify that the  new design is sound. Land testing of the redesigned system has been  successful, and the program anticipates that it will be ready for carrier  testing in October 2014.
Durability - structural and durability testing of the aircraft continued in  2013, and the program completed the first round of this testing on all  three variants. The conventional takeoff and landing variant and the  short takeoff and vertical landing variants have also started their  second round of testing. During this second round of testing, the short  takeoff and vertical landing test aircraft developed bulkhead cracks at  the equivalent of 17 years of service life. Contractor officials noted  that they were working to develop a solution to those cracks, but the  total cost and schedule impacts of these bulkhead cracks are  unknown at this time.
Autonomic Logistics Information System - an important tool to  predict and diagnose maintenance and supply issues, automate  logistics support processes and provide decision aids aimed at  reducing life-cycle sustainment costs and improving force readiness.  ALIS is being developed and fielded in increments. In 2013, the  program had to release an update to the first increment because  problems were discovered after the increment was released to the  testing locations. The additional time to develop and field this update  will likely delay the delivery of future increments. The program  completed site activation of ALIS systems at some training and testing  locations, and is in the process of adding capabilities and maturing  ALIS in a second increment to support the Marine Corps’ initial  operational capability. DOT&E notes that, although the second  increment is scheduled to be delivered in time to support the Marine  Corps’ initial operational capability, there is no margin for error in the  development schedule. Testing of this ALIS increment is about two  months behind largely due to a lack of test facilities. Program officials  note that they are in the process of adding facilities. The third, and  final, increment of ALIS that provides full capability is not expected to  be released until 2016.
section_title: Program Funding Projections and Unit Cost Targets May Not Be Achievable
paragraphs: The F-35 program’s high projected annual acquisition funding levels  continue to put the program’s long-term affordability at risk. Currently the  acquisition program requires $12.6 billion per year through 2037, which  does not appear to be achievable given the current fiscal environment.  The program is reducing unit costs to meet targets, but a significant  amount of additional cost reduction is needed if it expects to meet those  targets before the beginning of full rate production—currently scheduled  for 2019. Additionally, the most recent life-cycle sustainment cost  estimate for the F-35 fleet is more than $1 trillion, which DOD officials  have deemed unaffordable. The program’s long term sustainment  estimates reflect assumptions about key cost drivers that the program  does not control, including fuel costs, labor costs, and inflation rates. The  program is also focusing on product reliability, which is something that the  program can control, and something we have found in our prior best  practices work to be a key to driving down sustainment costs. According  to program reliability data, each F-35 variant was tracking closely to its  reliability plan as of December 2013, although the program has a long  way to go to achieve its reliability goals.
subsections: section_title: Long-term Acquisition Funding Projections Pose Affordability Challenges
paragraphs: The overall affordability of the F-35 acquisition program remains a  significant concern. As of March 2013, the program office estimated that  the total acquisition cost will be $390.4 billion. DOD’s estimated annual  funding levels to finish development and procurement of the F-35 are  shown in figure 2.
From fiscal years 2014 to 2018, DOD plans to increase development and  procurement funding for the F-35 from around $8 billion to around $13  billion, an investment of more than $50 billion over that 5-year period.  This build-up will occur during years of potential reductions in DOD’s  budget as a result of sequestration. From fiscal year 2014 through fiscal  year 2037, the program projects that it will require, on average,  development and procurement funding of $12.6 billion per year, with  several peak years at around $15 billion. Such a high average annual  cost requirement poses affordability risks. At $12.6 billion a year, the F-35  acquisition program alone would consume around one-quarter of all of  DOD’s annual major defense acquisition funding. Therefore, any change  in F-35 funding is likely to affect DOD’s ability to fully fund its other major  acquisition programs. In addition, maintaining this level of sustained  funding will be difficult in a period of declining or flat defense budgets and  competition with other large acquisition programs such as the KC-46  tanker and a new bomber. These costs do not include the costs to  operate and maintain the F-35s as they are produced and fielded.
Recognizing the affordability challenges posed by the F-35 program, the  Under Secretary of Defense for Acquisition, Technology, and Logistics  established affordability unit cost targets for each F-35 variant to be met  by the start of full rate production in 2019. The program is likely to be  challenged to meet those targets, as the three variants still require  anywhere from $41 million to $49 million in unit cost reductions (see table  2). In addition, the program’s current funding and quantity projections  indicate that unit costs in 2019 could actually be higher than the targets.  The Under Secretary issued a memorandum in April 2013 explaining that  affordability constraints are intended to force prioritization of  requirements, drive performance and cost trades, and ensure that  unaffordable programs do not enter the acquisition process. The  memorandum goes on to state that “if affordability caps are breached,  costs must be reduced or else program cancelation can be expected.”
The F-35 program made progress this year in decreasing the unit costs of  the conventional take-off and landing and carrier-suitable variants, but the  unit cost of the short takeoff and vertical landing variant increased by  nearly $10 million. According to program officials, the unit cost of the  short takeoff and vertical landing variant increased because the program  had to delay the procurement of a number of aircraft into the future, which  reduced near-term quantities and made each individual unit more costly,  and engine costs were higher than originally estimated. There is still  uncertainty surrounding these estimates depending upon how DOD  chooses to implement sequestration in future budgets.
section_title: Estimated Sustainment Costs Remain High but Emphasis on Improving Reliability Could Reduce Costs
paragraphs: In addition to the concerns about the affordability of the F-35 acquisition  program, there are also significant concerns about the cost of operating  and supporting the F-35 fleet over the coming decades. Currently, the  Cost Assessment and Program Evaluation (CAPE) office, within the  Office of the Secretary of Defense, estimates that the cost to operate and  support the fleet over 30 years is likely to exceed $1 trillion, which is 3  times higher than what was projected when the development program  began in 2001. CAPE’s estimates also indicate that F-35 operations and  support costs could surpass the average cost of legacy aircraft by 40  percent or more, when original estimates indicated that the F-35 would  cost less than the legacy aircraft. Program officials recently stated that  their estimates indicate that operation and support costs are likely to be  closer to $860 billion, and not the $1 trillion estimated by CAPE.  According to CAPE, program, and contractor officials, F-35 sustainment  cost estimates differ as the assumed future values for key cost drivers,  like inflation rates and fuel costs, vary among cost estimators. CAPE  officials emphasize that the difference between cost estimates is almost  entirely attributable to the use of different inflation indices. Table 3 below  lists the top cost drivers in the F-35 operation and sustainment estimates.  While it is important for the program to consider potential reductions or  increases in these variables listed below as it estimates the F-35’s long- term operation and sustainment costs, some of those variables can be  directly controlled by the program office while others like inflation rates  and fuel costs cannot.
The F-35 program office and prime contractor are working to make the  long-term program more affordable. Starting in September 2013, they  established a sustainment cost initiative team to meet regularly and  discuss options for driving down sustainment costs. According to  contracting officials, they also developed a management team dedicated  to improving the aircraft’s prognostics and health management system.  Additionally, the program is awaiting the results of a business case  analysis of the costs and benefits of various sustainment options. The first  phase of that analysis, completed in 2012, found that relying on  government personnel for sustainment processes would be less costly  than having contractors do the work. The second phase, which began in  2013, is examining multiple aspects of the F-35 sustainment strategy,  including the program’s approach to maintenance management and  supply chain support. The findings of this second phase are expected to  identify affordability opportunities and areas for additional future analysis.  Those findings are expected to be provided to the program by March  2014.
As the program faces key decisions about its F-35 operation and support  strategy reliability is still a significant concern. Our past work has found  that weapon system operating and support costs are directly correlated to  weapon system reliability, which is something the program can affect.  We found that lower reliability causes an imbalance in the relationship  between readiness and operating costs, and lends toward the need for  high costs to maintain readiness, as seen in figure 3 below. We also  previously found that reliability problems identified in DOD weapon  systems resulted in cost overruns and schedule delays.
DOD and the contractor use various measures to track and improve F-35  reliability, including average flying hours between failures, which is  defined as the number of flying hours achieved divided by the number of  failures incurred. As indicated in figure 4, the conventional takeoff and  landing variant and the short takeoff and vertical landing variant were not  meeting expected reliability as of September 2013, while the carrier- suitable variant was performing better than expected.
DOT&E’s recent report noted concerns about the program’s ability to  achieve its reliability goals by the time each of the F-35 variants reaches  maturity—defined as 75,000 flight hours for the CTOL and STOVL  variants; and 50,000 flight hours for the CV. DOT&E also noted that the  F-35 design is becoming more stable, and although the program still has  a large number of flight hours to go until system maturity, additional  reliability growth is not likely to occur without a focused, aggressive and  well-resourced effort.
section_title: F-35 Aircraft Manufacturing Efficiency Continues to Improve
paragraphs: F-35 manufacturing has improved and the contractor’s management of its  suppliers is evolving. As the number of aircraft in production has  increased, learning has taken place and manufacturing efficiency has  improved. For example, the prime contractor has seen reductions in  overall labor hours needed to manufacture the aircraft. The number of F- 35 aircraft produced and delivered annually by the prime contractor has  steadily increased since the first low rate production aircraft were  delivered in 2011. In 2013, the contractor delivered 35 aircraft to the  government, 5 more than it delivered in 2012 and 26 more than it  delivered in 2011. The prime contractor has put in place a supplier  management system to oversee key supplier performance allowing them  to identify poor performers and take appropriate action to address issues  such as part shortages and poor quality. According to contractor officials,  actions taken as a result of this system contributed to improvements in  supplier performance over the past year.
subsections: section_title: Program Manufacturing Processes Continue to Show Improvement
paragraphs: The prime contractor continues to gain efficiencies in the manufacturing  process as it learns more about manufacturing the aircraft. Reductions in  the amount of time spent on work completed out of its specific work  station have contributed to overall labor hour reductions. Aircraft delivered  in 2012 averaged about 93 hours of out of station work per aircraft, while  in 2013 about 8 hours of out of station work were expended per aircraft  on average. While these gains in efficiency have moved the program  closer to meeting its established labor hour goals, there is still a long way  to go. In 2013, the prime contractor was unable to reach labor hour goals  for both the CTOL and STOVL variants. By the end of 2014, the prime  contractor expects to significantly reduce the average labor hours to  produce the CTOL and STOVL variants. However, in order to meet its  goal, the program will have to reduce the average number of hours per  aircraft for CTOL production by nearly 20,000 and the average number of  hours per aircraft for STOVL production by more than 14,000 over the  next year. While challenging, this goal appears to be achievable given  that the program has been able to reduce the average labor hours for  CTOL and STOVL production by more than 20,000 hours annually since  2011. Figure 5 identifies the prime contractor’s trend in reduction of labor  hours since the beginning of low-rate initial production as well as the  contractor’s plans for 2014.
As manufacturing efficiency has improved, the prime contractor has also  been able to increase throughput, delivering more aircraft year over  year—9 in 2011, 30 in 2012, and 35 in 2013. Over the past year, the  prime contractor continued to deliver aircraft closer to contracted delivery  dates. Last year we found that deliveries averaged 11 months late, but  had improved considerably.deliveries in 2013 averaging about 5 months late. While deliveries in 2013  were later than planned, the trend continued to move in the right direction.  Figure 6 tracks the actual delivery dates against the dates specified in the  contract.
section_title: Prime Contractor Supplier Management System
paragraphs: The prime contractor is responsible for managing a complex supply chain  made up of a large number of national and international suppliers.  Currently, the prime contractor oversees about 1,500 domestic suppliers  and 80 international suppliers spread across 11 countries. Figure 7  identifies those countries participating in F-35 production. The number of  suppliers has grown significantly over the past three years. Since 2011,  153 new suppliers have been added to the supply chain, 67 of those were  added between 2012 and 2013. The prime contractor expects more  suppliers to be added as production increases and the program  progresses.
The prime contractor’s insight into the performance of 52 of its key  suppliers through its Supplier Integration Management system has led to  actions that have improved supplier performance. The system tracks  supplier performance data in 23 areas including, but not limited to, cost  growth, parts shortage occurrences, and the number of corrective action  reports filed. That data is reviewed and scored on a monthly basis, with  each supplier receiving an overall score based on their performance.  According to the prime contractor, a score of 80 or above, out of 100, is  considered good performance. All 52 of the key suppliers tracked using  this system were considered good performers as of December 2013. In  addition, 15 showed improvement in performance over the last year.  According to contractor officials, the system identifies poor performers  who are counseled and corrective actions are identified and implemented.  For example, according to officials from the prime contractor, one supplier  was identified through this process as having a large number of parts that  did not conform to specifications. The prime contractor held meetings with  and provided direction to that supplier. As a result of the prime  contractor’s actions, the supplier’s performance has improved over the  last year. In addition, officials from the prime contractor have identified  part shortages—parts that are late to production need—as a major  concern. The number of part shortages has slightly increased over the  last year. Although the root cause of the shortages is still being assessed,  officials from the prime contractor stated they are currently working on  ways to improve part availability.
section_title: Conclusions
paragraphs: Since the F-35 program restructuring was completed in March 2012,  acquisition cost and schedule estimates have remained relatively stable,  and the program has made progress in key areas. However, persistent  software problems have slowed progress in mission systems flight  testing, which is critical to delivering the warfighting capabilities expected  by the military services. These persistent delays put the program’s  development cost and schedule at risk. As a result, DOT&E now projects  that the warfighting capabilities expected by the Marine Corps in July  2015, will not likely be delivered on time, and could be delayed as much  as 13 months. This means that the Marine Corps may initially receive less  capable aircraft than it expects, and if progress in mission systems  software testing continues to be slower than planned, Air Force and Navy  initial operational capabilities may also be affected. The program may  also have to extend its overall developmental flight test schedule, which  would increase concurrency between testing and production and could  result in additional development cost growth. In addition to software  concerns, the current funding plans may be unaffordable, given current  budget constraints. This situation could worsen if unit cost targets are not  met. Finally, the estimated cost of operating and supporting the fleet over  its life-cycle continues to be high and could increase further if aircraft  reliability goals are not met.
DOD has already made a number of difficult decisions to put the F-35 on  a more sound footing. More such decisions may lie ahead. For example,  if software testing continues to be delayed, if funding falls short of  expectations, or if unit cost targets cannot be met, DOD may have to  make decisions about whether to proceed with production as planned  with less capable aircraft or to alter the production rate. Also, if reliability  falls short of goals, DOD may have to make decisions about other ways  to reduce sustainment costs, such as reduced flying hours. Eventually,  DOD will have to make contingency plans for these and other issues. At  this point, we believe the most pressing issue is the effect software  testing delays are likely to have on the capabilities of the initial  operational aircraft that each military service will receive. In order to make  informed decisions about weapon system investments and future force  structure, it is important that Congress and the services have a clear  understanding of the capabilities that the initial operational F-35 aircraft  will possess.
section_title: Recommendation for Executive Action
paragraphs: Due to the uncertainty surrounding the delivery of F-35 software  capabilities, we recommend that the Secretary of Defense conduct an  assessment of the specific capabilities that realistically can be delivered  and those that will not likely be delivered to each of the services by their  established initial operational capability dates. The results of this  assessment should be shared with Congress and the military services as  soon as possible but no later than July 2015.
section_title: Agency Comments
paragraphs: DOD provided comments on a draft of this report, which are reprinted in  appendix lll.  DOD concurred with our recommendation.
We are sending copies of this report to appropriate congressional  committees; the Secretary of Defense; the Secretaries of the Air Force,  Army, and Navy; the Commandant of the Marine Corps; and the Director  of the Office of Management and Budget. The report is available at no  charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. Staff members making key contributions to  this report are listed in appendix IV.
section_title: Appendix I: Prior GAO Reports and DOD Responses
subsections: section_title: Est. dev. Costs dev. Length Aircraft unit cost $34.4 Billion 10 years $69 Million
paragraphs: Key program event   Start of system  development and  demonstration approved.
Primary GAO message   Critical technologies needed  for key aircraft performance  elements are not mature.  Program should delay start of  system development until  critical technologies are  mature to acceptable levels.
DOD response and actions   DOD did not delay start of system  development and demonstration  stating technologies were at  acceptable maturity levels and  will manage risks in development.
The program undergoes  re-plan to address higher  than expected design  weight, which added $7  billion and 18 months to  development schedule.
We recommend that the  program reduce risks and  establish executable  business case that is  knowledge-based with an  evolutionary acquisition  strategy.
DOD partially concurred but does  not adjust strategy, believing that  their approach is balanced  between cost, schedule and  technical risk.
Program sets in motion  plan to enter production  in 2007 shortly after first  flight of the non- production representative  aircraft.
The program plans to enter  production with less than 1  percent of testing complete.  We recommend program  delay investing in production  until flight testing shows that  JSF performs as expected.
DOD partially concurred but did  not delay start of production  because they believe the risk  level was appropriate.
Congress reduced  funding for first two low- rate production buys  thereby slowing the ramp  up of production.
Progress is being made but  concerns remained about  undue overlap in testing and  production. We recommend  limits to annual production  quantities to 24 a year until  flying quantities are  demonstrated.
DOD non-concurred and felt that  the program had an acceptable  level of concurrency and an  appropriate acquisition strategy.
DOD implemented a Mid- Course Risk Reduction  Plan to replenish  management reserves  from about $400 million  to about $1 billion by  reducing test resources.
We believe new plan  increased risks and DOD  should revise it to address  concerns about testing,  management reserves, and  manufacturing concerns. We  determined that the cost  estimate was not reliable and  that a new cost estimate and  schedule risk assessment is  needed.
DOD did not revise risk plan or  restore testing resources, stating  that they will monitor the new  plan and adjust it if necessary.  Consistent with a report  recommendation, a new cost  estimate was prepared, but DOD  did not conduct a risk and  uncertainty analysis.
section_title: Est. dev. Costs dev. Length Aircraft unit cost $44.4 Billion 13 years $104 Million
paragraphs: Key program event   The program increased  the cost estimate and  adds a year to  development but  accelerated the  production ramp up.  Independent DOD cost  estimate (JET I) projects  even higher costs and  further delays.
Primary GAO message   Moving forward with an  accelerated procurement  plan and use of cost  reimbursement contracts is  very risky. We recommended  the program report on the  risks and mitigation strategy  for this approach.
DOD response and actions   DOD agreed to report its  contracting strategy and plans to  Congress and conduct a  schedule risk analysis. The  program reported completing the  first schedule risk assessment  with plans to update semi- annually. The Department  announced a major program  reducing procurement and  moving to fixed-price contracts.
The program was  restructured to reflect  findings of recent  independent cost team  (JET II) and independent  manufacturing review  team. As a result,  development funds  increased, test aircraft  were added, the  schedule was extended,  and the early production  rate decreased.
Costs and schedule delays  inhibit the program’s ability to  meet needs on time. We  recommend the program  complete a full  comprehensive cost estimate  and assess warfighter and  IOC requirements. We  suggest that Congress  require DOD to tie annual  procurement requests to  demonstrated progress.
DOD continued restructuring,  increasing test resources and  lowering the production rate.  Independent review teams  evaluated aircraft and engine  manufacturing processes. Cost  increases later resulted in a  Nunn-McCurdy breach. Military  services are currently reviewing  capability requirements as we  recommended.
Restructuring continued  with additional  development cost  increases; schedule  growth; further reduction  in near-term procurement  quantities; and  decreased the rate for  future production.  The  Secretary of Defense  placed the STOVL  variant on a two-year  probation; decoupled  STOVL from the other  variants; and reduced  STOVL production plans  for fiscal years 2011 to  2013.
The restructuring actions are  positive and if implemented  properly, should lead to more  achievable and predictable  outcomes. Concurrency of  development, test, and  production is substantial and  provides risk to the program.  We recommended the  program maintain funding  levels as budgeted; establish  criteria for STOVL probation;  and conduct an independent  review of software  development, integration, and  test processes.
DOD concurred with all three of  the recommendations. DOD lifted  STOVL probation citing improved  performance. Subsequently, DOD  further reduced procurement  quantities, decreasing funding  requirements through 2016. The  initial independent software  assessment began and ongoing  reviews are planned to continue  through 2012.
section_title: Est. dev. Costs dev. Length Aircraft unit cost $55.2 Billion 18 years $137 Million
paragraphs: Key program event   The program established  a new acquisition  program baseline and  approved the  continuation of system  development, increasing  costs for development  and procurements and  extending the period of  planned procurements by  2 years.
Primary GAO message   Extensive restructuring  places the program on a  more achievable course.  Most of the program’s  instability continues to be  concurrency of development,  test, and production. We  recommend the Cost  Assessment Program  Evaluation office conduct an  analysis on the impact of  lower annual funding levels;  JSF program office conducts  an assessment of the supply  chain and transportation  network.   The program is moving in the  right direction but must fully  validate design and  operational performance and  at the same time make the  system affordable. We did  not make recommendations  to DOD.
DOD response and actions   DOD partially concurred with  conducting an analysis on the  impact of lower annual funding  levels and concurred with  assessing the supply chain and  transportation network.
DOD agreed with GAO’s  observations.
section_title: Appendix Il: Scope and Methodology
paragraphs: To assess the program’s ongoing development and testing we reviewed  the status of software development and integration and contractor  management improvement initiatives. We also interviewed officials from  the program office, prime contractor, and the Defense Contract  Management Agency (DCMA) to discuss current development status and  software releases. In addition, we compared management objectives to  progress made on these objectives during the year. We obtained and  analyzed data on flights and test points, both planned and accomplished  during 2013. We compared test progress against the total program plans  to complete. We also reviewed the Director, Operational Test and  Evaluation’s annual F-35 assessment. In addition, we interviewed officials  from the F-35 program office and aircraft prime contractor to discuss  development test plans and achievements. We also collected information  from the program office, prime contractor and Department of Defense  (DOD) test pilots regarding the program’s technical risks including the  helmet mounted display, autonomic logistics information system, carrier  arresting hook, and structural durability.
To assess the program’s funding and long-term affordability, we reviewed  financial management reports, annual Selected Acquisition Reports, and  monthly status reports available as of December 2013. In addition, we  reviewed total program funding requirements from the Defense  Acquisition Executive Summary. We used this data to project annual  funding requirements through the expected end of the F-35 acquisition in  2037. We also analyzed the fiscal year 2014 President’s Budget data to  identify the current status of unit costs for each variant, and the  differences in this cost since 2012. We reviewed the Office of the  Secretary of Defense’s F-35 Joint Strike Fighter Concurrency Quick Look  Review, and discussed and analyzed reported concurrency costs with the  prime contractor and program office. We obtained and discussed the life- cycle operating and support cost through the program’s Selected  Acquisition Report and projections made by the Cost Analysis and  Program Evaluation (CAPE) office. We identified changes in cost and  interviewed officials from the program office prime contractor, Naval Air  Systems Command, and the CAPE office regarding reasons for these  changes. We also discussed future plans of the DOD and prime  contractor to try and reduce life-cycle sustainment costs with officials from  the prime contractor, program office, and CAPE. We analyzed reliability  data and discussed these issues with program and prime contractor  officials.
To assess manufacturing progress we obtained and analyzed data  related to aircraft delivery rates and work performance data through the  end of calendar year 2013. This data was compared to program  objectives identified in these areas and used to identify trends. We  reviewed data and briefings provided by the program office, prime  contractor, and DCMA in order to identify issues in manufacturing  processes. We discussed reasons for delivery delays and plans for  improvement with the prime contractor. We also toured the prime  contractor’s manufacturing facility in Fort Worth, Texas and collected and  analyzed data related to aircraft quality through December 2013. We  reviewed and discussed information on the prime contractor’s global  supply chain including their management processes for oversight.
We assessed the reliability of DOD and contractor data by reviewing  existing information about the data, and interviewing agency officials  knowledgeable about the data. We determined that the data were  sufficiently reliable for the purposes of this report. We conducted this  performance audit from June 2013 to March 2014 in accordance with  generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on  our audit objectives.
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition the contact name above, the following staff members made  key contributions to this report: Travis Masters, Assistant Director; Marvin  Bonner; Peter Anderson, Megan Porter, Roxanna Sun and Abby Volk.
section_title: Related GAO Products
paragraphs: F-35 Joint Strike Fighter: Restructuring Has Improved the Program, but  Affordability Challenges and Other Risks Remain. GAO-13-690T.  Washington, D.C.: June 19, 2013.
F-35 Joint Strike Fighter: Program Has Improved in Some Areas, but  Affordability Challenges and Other Risks Remain. GAO-13-500T.  Washington, D.C.: April 17, 2013.
F-35 Joint Strike Fighter: Current Outlook Is Improved, but Long-Term  Affordability Is a Major Concern. GAO-13-309. Washington, D.C.: March  11, 2013.
Joint Strike Fighter: DOD Actions Needed to Further Enhance  Restructuring and Address Affordability Risks. GAO-12-437. Washington,  D.C.: June 14, 2012.
Joint Strike Fighter: Restructuring Added Resources and Reduced Risk,  but Concurrency Is Still a Major Concern. GAO-12-525T. Washington,  D.C.: March 20, 2012.
Joint Strike Fighter: Restructuring Places Program on Firmer Footing, but  Progress Is Still Lagging. GAO-11-677T. Washington, D.C.: May 19,  2011.
Joint Strike Fighter: Restructuring Places Program on Firmer Footing, but  Progress Still Lags. GAO-11-325. Washington, D.C.: April 7, 2011.
Joint Strike Fighter: Restructuring Should Improve Outcomes, but  Progress Is Still Lagging Overall. GAO-11-450T. Washington, D.C.:  March 15, 2011.
Tactical Aircraft: Air Force Fighter Force Structure Reports Generally  Addressed Congressional Mandates, but Reflected Dated Plans and  Guidance, and Limited Analyses. GAO-11-323R. Washington, D.C.:  February 24, 2011.
Joint Strike Fighter: Assessment of DOD’s Funding Projection for the  F136 Alternate Engine. GAO-10-1020R. Washington, D.C.: September  15, 2010.
Joint Strike Fighter: Accelerating Procurement before Completing  Development Increases the Government’s Financial Risk. GAO-09-303.  Washington D.C.: March 12, 2009.
Joint Strike Fighter: Recent Decisions by DOD Add to Program Risks.  GAO-08-388. Washington, D.C.: March 11, 2008.
Joint Strike Fighter: Progress Made and Challenges Remain.  GAO-07-360. Washington, D.C.: March 15, 2007.
Joint Strike Fighter: DOD Plans to Enter Production before Testing  Demonstrates Acceptable Performance. GAO-06-356. Washington, D.C.:  March 15, 2006.
Tactical Aircraft: Opportunity to Reduce Risks in the Joint Strike Fighter  Program with Different Acquisition Strategy. GAO-05-271. Washington,  D.C.: March 15, 2005.