F-35 Joint Strike Fighter: Problems Completing Software Testing May Hinder Delivery of Expected Warfighting Capabilities
Released: 2014-03-24T00:00:00

Letter

Background

program baseline was finalized in March 2012, and since that time costs have remained relatively stable. Table 1 notes the significant cost, quantity, and schedule changes from the original program baseline and the relative stability since the new baseline was established.

At the time the new F-35 acquisition program baseline was finalized, it did not identify new initial operational capability (IOC) dates for the three military services. The following year DOD issued a memorandum noting that Marine Corps and Air Force were planning to field initial operational capabilities in 2015 and 2016, respectively, and that the Navy planned to field its initial capability in 2018. The memorandum emphasized that the Marine Corps and Air Force initial operational capabilities would be achieved with aircraft that possess initial combat capabilities, and noted that those aircraft would need additional lethality and survivability enhancements to meet the full spectrum of warfighter requirements in the future. These new parameters represented a delay of 5 to 6 years from the program’s initial 2001 baseline and a reduction in the capabilities expected at IOC.

In March 2005 we recommended that DOD implement an evolutionary, incremental approach to developing and fielding the F-35—then known as the Joint Strike Fighter—to ensure that the warfighters would receive an initial combat capability that, at a minimum, would meet their most immediate needs. Again in March 2010, we recommended that DOD identify the absolute minimum combat capabilities that would be acceptable by each of the military services to field their initial operational capabilities and establish reasonable, realistic timeframes for achieving those requirements. In both instances, we noted that the military services should consider trading off desired capabilities in order to more rapidly field aircraft with an initial set of useable capabilities and that any capabilities not needed to meet immediate warfighting needs should be deferred to a future development increment. In both instances, DOD agreed with the intent of our recommendation, but believed that its existing program management practices were sufficient.

Testing of Critical Software Poses a Significant Challenge to the Delivery of F- 35 Warfighting Capabilities

Delays in the testing of critical mission systems software have put the delivery of expected warfighting capabilities to the Marine Corps at risk, and could affect the delivery of capabilities to the Air Force and Navy as well. F-35 developmental flight testing is separated into two key areas: mission systems and flight sciences. Mission systems testing is done to verify that the software and systems that provide critical warfighting capabilities function properly and meet requirements, while flight science testing is done to verify the aircraft’s basic flying capabilities. In a March 2013 report we found that development and testing of mission systems software was behind schedule, due largely to delayed software deliveries, limited capability in the software when delivered, and the need to fix problems and retest multiple software versions. These same challenges continued thorough 2013, and as a result progress in mission systems testing has been limited. The Director of Operational Test and Evaluation (DOT&E) predicts that the delivery of expected warfighting capabilities to the Marine Corps could be delayed by as much as 13 months. Delays of this magnitude could also increase the already significant concurrency between testing and aircraft procurement and result in additional cost growth. Although mission systems testing is behind, the F-35 program has been able to accomplish nearly all of its planned flight science testing. The program also continued to make progress in addressing key technical risks, although some of that progress has been limited.

Limited Progress in Mission Systems Software Testing May Cause Delays and Add Cost to F-35 Development

While the F-35 program was able to accomplish all of the mission system test flights it had planned in 2013, it did not accomplish all of the planned falling short by 11 percent. The F-35 program planned to fly test points,329 mission systems test flights and accomplish 2,817 test points in 2013. The program actually flew 352 test flights, exceeding the goal, but only accomplished 2,518 test points. According to program and contractor officials, slow progress in developing, delivering, and testing mission systems software continues to be the program’s most significant risk area. The F-35 program is developing and fielding mission systems software capabilities in blocks: (1) Block 1, (2) Block 2A, (3) Block 2B, (4) Block 3i, and (5) Block 3F. Each subsequent block builds on the capabilities provided in the preceding blocks. Blocks 1 and 2A provide training capabilities and are essentially complete, with some final development and testing still underway. Blocks 2B and 3i provide initial warfighting capabilities and are needed by the Marine Corps and Air Force, respectively, to achieve initial operational capability. Block 3F is expected to provide the full suite of warfighting capabilities, and is the block the Navy expects to have to achieve its initial operational capability.

Developmental testing of Block 2B software is behind schedule and will likely delay the delivery of expected warfighting capabilities. The delivery of this software capability is of high near-term importance because it provides initial warfighting capability for the overall F-35 program, and is needed by the Marine Corps to field its initial operational capability in July 2015. As of January 2014, the program planned to have verified the functionality of 27 percent of the software’s capability on-board the aircraft, but had only been able to verify 13 percent. This leaves a significant amount of work to be done before October 2014, which is when the program expects to complete developmental flight testing of this software block. According to DOT&E, Block 2B developmental testing will not be completed as scheduled and could be delayed by as much as 13 months, as the program has had to devote time and resources to address problems and completing development of prior software blocks. Delays of this magnitude would mean that the Marine Corps will not likely have all of the capabilities it expects in July 2015, its planned initial operational capability date. At this time it is not clear exactly which of the expected capabilities will be available as testing is still ongoing. The effects of these delays compound as they also put the timely delivery of Air Force and Navy initial operational capabilities at risk. The Air Force expects to field its initial operational capability in August 2016 with Block 3i aircraft that possess the same warfighting capabilities as the Marine Corps aircraft, but have upgraded computer hardware. The Navy plans to achieve its initial operational capability in August 2018 with Block 3F aircraft that possess the full suite of F-35 warfighting capabilities.

Program and contractor officials have stated that while they recognize that the program faces software risks, they still expect to deliver all of the planned F-35 software capabilities to the military services as currently scheduled. However, given the uncertainty in mission systems software testing and the significance of the F-35 to future force structure plans, it is important that the military services have a clear understanding of the specific capabilities that they can realistically expect to receive, and those capabilities that are not likely to be delivered by their initial operational capability dates—the first of which is scheduled for July 2015. In addition, because the F-35 is DOD’s most costly and ambitious acquisition program, it is important that DOD keep Congress informed about the status of the program. Without a clear understanding of the specific capabilities that will initially be delivered, Congress and the military services may not be able to make fully informed resource allocation decisions.

Delays in mission systems software testing could also increase costs. As currently planned, DOD expects to complete developmental flight testing in 2017. If the flight test schedule is extended, the program may have to retain testing and engineering personnel longer than currently expected, which would increase development cost. DOD currently expects to have invested $70.2 billion to procure 359 aircraft by 2017 when developmental flight testing is scheduled to end. Our past reports have concluded that purchasing aircraft while concurrently conducting developmental flight testing increases the risk that problems will be discovered late in testing and additional funding will be needed to rework aircraft that have already been purchased. If F-35 procurement plans remain unchanged and developmental testing continues into 2018, the cost risks associated with concurrency will likely increase as DOD expects to have invested $83.4 billion in 459 aircraft by that point in time.

The F-35 contractor recognizes that additional testing efficiencies are important in order to deliver capabilities on schedule and cost. One way it plans to gain efficiency is to use test results from one F-35 variant to close out test points for the other two variants in instances in which the variants have common functions. According to test officials, most mission systems testing can be accomplished on any variant and only a limited amount of variant-specific testing is required. Contractor officials pointed out that this type of efficiency would help mitigate some testing risk, but they also recognized that it will still be difficult to make up the lost time in the test program. They noted that delays in specific test events generally impact the entire test schedule because the ability to conduct future testing is often dependent on the completion of the earlier events.

Program Accomplished Most Flight Science Test Goals in 2013

The program accomplished nearly all of the flight sciences testing, including weapons testing, it had planned for 2013. As of December 2013, the program had achieved half of the total number of test points required to complete all flight science testing for the program. Figure 1 below shows the number of flight science test points planned and accomplished for each F-35 variant as of December 2013, and also identifies the total number of test points required for each variant to complete flight science testing.

The program made progress despite the fact that flight testing was halted twice at the beginning of the year to investigate and fix cracks in an engine fan blade and leaky fuel hoses. In addition, program and contractor officials emphasized that employee furloughs that occurred in 2013, due to mandatory sequestration, limited the amount of flight testing that could be done during that time as well.science and weapons testing accomplishments included: Some of the key flight

Conventional takeoff and landing variant – The program successfully demonstrated the variant’s ability to launch AIM-120 missiles from its internal weapons bay and to refuel while in flight. The program also continued testing the aircraft’s ability to function at high vertical flight angles, although program officials noted that the testing took longer than expected. As of December 2013, the program had accomplished 59 percent of its total expected flight science test points for this variant.

Short takeoff and vertical landing variant – The program successfully demonstrated the STOVL’s ability to takeoff vertically, launch weapons from its internal weapons bay, and dump fuel when needed. In addition, the program conducted some testing of the variant at sea on an amphibious assault ship—specifically the USS WASP. As of December 2013, the program had accomplished 49 percent of its total expected flight science test points for this variant.

Carrier-suitable variant – The program began testing the capability of the aircraft to function at high vertical flight angles. In addition, the program successfully demonstrated the aircraft’s ability to dump fuel when needed. Program and contractor officials noted that the program also made progress to begin testing to verify that the aircraft’s new arresting hook system could successfully catch a cable on a set of carrier arresting gear installed onshore at the Lakehurst facility. As of December 2013, the program had accomplished 43 percent of its total expected flight science test points for this variant.

Developmental testing is not the only testing that the program still has to complete. The F-35 program is also scheduled to begin operational testing in June 2015, to determine that the aircraft variants can effectively perform their intended missions in a realistic threat environment.

Program Progress in Addressing Key Technical Risks Has Been Mixed

While the F-35 program made progress addressing some key technical risks in 2013, it continued to encounter slower than expected progress in developing the Autonomic Logistics Information System (ALIS). Over time, we have reported on 4 areas of technical and structural risk that the program identified during flight, ground, and lab testing that if not addressed, could result in substantially degraded capabilities and mission effectiveness. In 2013, we found that the program made the following progress in each of those areas:

Helmet mounted display - provides flight data, targeting, and other sensor data to the pilot, and is integral to reducing pilot workload and achieving the F-35’s concept of operations. The original helmet mounted display encountered significant technical deficiencies, including display jitter, the undesired shaking of the visor display, and latency, the perceivable lag that occurs in transmitting sensor data, and did not meet warfighter requirements. The program made adjustments to the helmet design, including adding sensors to lessen the display jitter, and redesigning elements to minimize latency. The program tested these design changes in 2013 and found that most of the technical deficiencies had been adequately addressed, and that the helmet’s performance was sufficiently suitable to support Marine Corps initial operational capability in 2015. DOT&E and program test pilots noted that the current night vision camera continues to have problems. The program has identified a new camera that it believes will address those problems, but that camera has not been fully tested to verify its capabilities.

Arresting Hook System - allows the F-35 carrier-suitable variant to engage landing wires on aircraft carriers, was redesigned after the original hook system was found to be deficient. The program determined that the original hook assembly was not strong enough to reliably catch the wire and stop the airplane. As a result, the program modified the hook system’s hydraulic components, and made structural modifications to the plane. In March 2013, the program completed a critical design review of the hook system to verify that the new design is sound. Land testing of the redesigned system has been successful, and the program anticipates that it will be ready for carrier testing in October 2014.

Durability - structural and durability testing of the aircraft continued in 2013, and the program completed the first round of this testing on all three variants. The conventional takeoff and landing variant and the short takeoff and vertical landing variants have also started their second round of testing. During this second round of testing, the short takeoff and vertical landing test aircraft developed bulkhead cracks at the equivalent of 17 years of service life. Contractor officials noted that they were working to develop a solution to those cracks, but the total cost and schedule impacts of these bulkhead cracks are unknown at this time.

Autonomic Logistics Information System - an important tool to predict and diagnose maintenance and supply issues, automate logistics support processes and provide decision aids aimed at reducing life-cycle sustainment costs and improving force readiness. ALIS is being developed and fielded in increments. In 2013, the program had to release an update to the first increment because problems were discovered after the increment was released to the testing locations. The additional time to develop and field this update will likely delay the delivery of future increments. The program completed site activation of ALIS systems at some training and testing locations, and is in the process of adding capabilities and maturing ALIS in a second increment to support the Marine Corps’ initial operational capability. DOT&E notes that, although the second increment is scheduled to be delivered in time to support the Marine Corps’ initial operational capability, there is no margin for error in the development schedule. Testing of this ALIS increment is about two months behind largely due to a lack of test facilities. Program officials note that they are in the process of adding facilities. The third, and final, increment of ALIS that provides full capability is not expected to be released until 2016.

Program Funding Projections and Unit Cost Targets May Not Be Achievable

The F-35 program’s high projected annual acquisition funding levels continue to put the program’s long-term affordability at risk. Currently the acquisition program requires $12.6 billion per year through 2037, which does not appear to be achievable given the current fiscal environment. The program is reducing unit costs to meet targets, but a significant amount of additional cost reduction is needed if it expects to meet those targets before the beginning of full rate production—currently scheduled for 2019. Additionally, the most recent life-cycle sustainment cost estimate for the F-35 fleet is more than $1 trillion, which DOD officials have deemed unaffordable. The program’s long term sustainment estimates reflect assumptions about key cost drivers that the program does not control, including fuel costs, labor costs, and inflation rates. The program is also focusing on product reliability, which is something that the program can control, and something we have found in our prior best practices work to be a key to driving down sustainment costs. According to program reliability data, each F-35 variant was tracking closely to its reliability plan as of December 2013, although the program has a long way to go to achieve its reliability goals.

Long-term Acquisition Funding Projections Pose Affordability Challenges

The overall affordability of the F-35 acquisition program remains a significant concern. As of March 2013, the program office estimated that the total acquisition cost will be $390.4 billion. DOD’s estimated annual funding levels to finish development and procurement of the F-35 are shown in figure 2.

From fiscal years 2014 to 2018, DOD plans to increase development and procurement funding for the F-35 from around $8 billion to around $13 billion, an investment of more than $50 billion over that 5-year period. This build-up will occur during years of potential reductions in DOD’s budget as a result of sequestration. From fiscal year 2014 through fiscal year 2037, the program projects that it will require, on average, development and procurement funding of $12.6 billion per year, with several peak years at around $15 billion. Such a high average annual cost requirement poses affordability risks. At $12.6 billion a year, the F-35 acquisition program alone would consume around one-quarter of all of DOD’s annual major defense acquisition funding. Therefore, any change in F-35 funding is likely to affect DOD’s ability to fully fund its other major acquisition programs. In addition, maintaining this level of sustained funding will be difficult in a period of declining or flat defense budgets and competition with other large acquisition programs such as the KC-46 tanker and a new bomber. These costs do not include the costs to operate and maintain the F-35s as they are produced and fielded.

Recognizing the affordability challenges posed by the F-35 program, the Under Secretary of Defense for Acquisition, Technology, and Logistics established affordability unit cost targets for each F-35 variant to be met by the start of full rate production in 2019. The program is likely to be challenged to meet those targets, as the three variants still require anywhere from $41 million to $49 million in unit cost reductions (see table 2). In addition, the program’s current funding and quantity projections indicate that unit costs in 2019 could actually be higher than the targets. The Under Secretary issued a memorandum in April 2013 explaining that affordability constraints are intended to force prioritization of requirements, drive performance and cost trades, and ensure that unaffordable programs do not enter the acquisition process. The memorandum goes on to state that “if affordability caps are breached, costs must be reduced or else program cancelation can be expected.”

The F-35 program made progress this year in decreasing the unit costs of the conventional take-off and landing and carrier-suitable variants, but the unit cost of the short takeoff and vertical landing variant increased by nearly $10 million. According to program officials, the unit cost of the short takeoff and vertical landing variant increased because the program had to delay the procurement of a number of aircraft into the future, which reduced near-term quantities and made each individual unit more costly, and engine costs were higher than originally estimated. There is still uncertainty surrounding these estimates depending upon how DOD chooses to implement sequestration in future budgets.

Estimated Sustainment Costs Remain High but Emphasis on Improving Reliability Could Reduce Costs

In addition to the concerns about the affordability of the F-35 acquisition program, there are also significant concerns about the cost of operating and supporting the F-35 fleet over the coming decades. Currently, the Cost Assessment and Program Evaluation (CAPE) office, within the Office of the Secretary of Defense, estimates that the cost to operate and support the fleet over 30 years is likely to exceed $1 trillion, which is 3 times higher than what was projected when the development program began in 2001. CAPE’s estimates also indicate that F-35 operations and support costs could surpass the average cost of legacy aircraft by 40 percent or more, when original estimates indicated that the F-35 would cost less than the legacy aircraft. Program officials recently stated that their estimates indicate that operation and support costs are likely to be closer to $860 billion, and not the $1 trillion estimated by CAPE. According to CAPE, program, and contractor officials, F-35 sustainment cost estimates differ as the assumed future values for key cost drivers, like inflation rates and fuel costs, vary among cost estimators. CAPE officials emphasize that the difference between cost estimates is almost entirely attributable to the use of different inflation indices. Table 3 below lists the top cost drivers in the F-35 operation and sustainment estimates. While it is important for the program to consider potential reductions or increases in these variables listed below as it estimates the F-35’s long- term operation and sustainment costs, some of those variables can be directly controlled by the program office while others like inflation rates and fuel costs cannot.

The F-35 program office and prime contractor are working to make the long-term program more affordable. Starting in September 2013, they established a sustainment cost initiative team to meet regularly and discuss options for driving down sustainment costs. According to contracting officials, they also developed a management team dedicated to improving the aircraft’s prognostics and health management system. Additionally, the program is awaiting the results of a business case analysis of the costs and benefits of various sustainment options. The first phase of that analysis, completed in 2012, found that relying on government personnel for sustainment processes would be less costly than having contractors do the work. The second phase, which began in 2013, is examining multiple aspects of the F-35 sustainment strategy, including the program’s approach to maintenance management and supply chain support. The findings of this second phase are expected to identify affordability opportunities and areas for additional future analysis. Those findings are expected to be provided to the program by March 2014.

As the program faces key decisions about its F-35 operation and support strategy reliability is still a significant concern. Our past work has found that weapon system operating and support costs are directly correlated to weapon system reliability, which is something the program can affect. We found that lower reliability causes an imbalance in the relationship between readiness and operating costs, and lends toward the need for high costs to maintain readiness, as seen in figure 3 below. We also previously found that reliability problems identified in DOD weapon systems resulted in cost overruns and schedule delays.

DOD and the contractor use various measures to track and improve F-35 reliability, including average flying hours between failures, which is defined as the number of flying hours achieved divided by the number of failures incurred. As indicated in figure 4, the conventional takeoff and landing variant and the short takeoff and vertical landing variant were not meeting expected reliability as of September 2013, while the carrier- suitable variant was performing better than expected.

DOT&E’s recent report noted concerns about the program’s ability to achieve its reliability goals by the time each of the F-35 variants reaches maturity—defined as 75,000 flight hours for the CTOL and STOVL variants; and 50,000 flight hours for the CV. DOT&E also noted that the F-35 design is becoming more stable, and although the program still has a large number of flight hours to go until system maturity, additional reliability growth is not likely to occur without a focused, aggressive and well-resourced effort.

F-35 Aircraft Manufacturing Efficiency Continues to Improve

F-35 manufacturing has improved and the contractor’s management of its suppliers is evolving. As the number of aircraft in production has increased, learning has taken place and manufacturing efficiency has improved. For example, the prime contractor has seen reductions in overall labor hours needed to manufacture the aircraft. The number of F- 35 aircraft produced and delivered annually by the prime contractor has steadily increased since the first low rate production aircraft were delivered in 2011. In 2013, the contractor delivered 35 aircraft to the government, 5 more than it delivered in 2012 and 26 more than it delivered in 2011. The prime contractor has put in place a supplier management system to oversee key supplier performance allowing them to identify poor performers and take appropriate action to address issues such as part shortages and poor quality. According to contractor officials, actions taken as a result of this system contributed to improvements in supplier performance over the past year.

Program Manufacturing Processes Continue to Show Improvement

The prime contractor continues to gain efficiencies in the manufacturing process as it learns more about manufacturing the aircraft. Reductions in the amount of time spent on work completed out of its specific work station have contributed to overall labor hour reductions. Aircraft delivered in 2012 averaged about 93 hours of out of station work per aircraft, while in 2013 about 8 hours of out of station work were expended per aircraft on average. While these gains in efficiency have moved the program closer to meeting its established labor hour goals, there is still a long way to go. In 2013, the prime contractor was unable to reach labor hour goals for both the CTOL and STOVL variants. By the end of 2014, the prime contractor expects to significantly reduce the average labor hours to produce the CTOL and STOVL variants. However, in order to meet its goal, the program will have to reduce the average number of hours per aircraft for CTOL production by nearly 20,000 and the average number of hours per aircraft for STOVL production by more than 14,000 over the next year. While challenging, this goal appears to be achievable given that the program has been able to reduce the average labor hours for CTOL and STOVL production by more than 20,000 hours annually since 2011. Figure 5 identifies the prime contractor’s trend in reduction of labor hours since the beginning of low-rate initial production as well as the contractor’s plans for 2014.

As manufacturing efficiency has improved, the prime contractor has also been able to increase throughput, delivering more aircraft year over year—9 in 2011, 30 in 2012, and 35 in 2013. Over the past year, the prime contractor continued to deliver aircraft closer to contracted delivery dates. Last year we found that deliveries averaged 11 months late, but had improved considerably.deliveries in 2013 averaging about 5 months late. While deliveries in 2013 were later than planned, the trend continued to move in the right direction. Figure 6 tracks the actual delivery dates against the dates specified in the contract.

Prime Contractor Supplier Management System

The prime contractor is responsible for managing a complex supply chain made up of a large number of national and international suppliers. Currently, the prime contractor oversees about 1,500 domestic suppliers and 80 international suppliers spread across 11 countries. Figure 7 identifies those countries participating in F-35 production. The number of suppliers has grown significantly over the past three years. Since 2011, 153 new suppliers have been added to the supply chain, 67 of those were added between 2012 and 2013. The prime contractor expects more suppliers to be added as production increases and the program progresses.

The prime contractor’s insight into the performance of 52 of its key suppliers through its Supplier Integration Management system has led to actions that have improved supplier performance. The system tracks supplier performance data in 23 areas including, but not limited to, cost growth, parts shortage occurrences, and the number of corrective action reports filed. That data is reviewed and scored on a monthly basis, with each supplier receiving an overall score based on their performance. According to the prime contractor, a score of 80 or above, out of 100, is considered good performance. All 52 of the key suppliers tracked using this system were considered good performers as of December 2013. In addition, 15 showed improvement in performance over the last year. According to contractor officials, the system identifies poor performers who are counseled and corrective actions are identified and implemented. For example, according to officials from the prime contractor, one supplier was identified through this process as having a large number of parts that did not conform to specifications. The prime contractor held meetings with and provided direction to that supplier. As a result of the prime contractor’s actions, the supplier’s performance has improved over the last year. In addition, officials from the prime contractor have identified part shortages—parts that are late to production need—as a major concern. The number of part shortages has slightly increased over the last year. Although the root cause of the shortages is still being assessed, officials from the prime contractor stated they are currently working on ways to improve part availability.

Conclusions

Since the F-35 program restructuring was completed in March 2012, acquisition cost and schedule estimates have remained relatively stable, and the program has made progress in key areas. However, persistent software problems have slowed progress in mission systems flight testing, which is critical to delivering the warfighting capabilities expected by the military services. These persistent delays put the program’s development cost and schedule at risk. As a result, DOT&E now projects that the warfighting capabilities expected by the Marine Corps in July 2015, will not likely be delivered on time, and could be delayed as much as 13 months. This means that the Marine Corps may initially receive less capable aircraft than it expects, and if progress in mission systems software testing continues to be slower than planned, Air Force and Navy initial operational capabilities may also be affected. The program may also have to extend its overall developmental flight test schedule, which would increase concurrency between testing and production and could result in additional development cost growth. In addition to software concerns, the current funding plans may be unaffordable, given current budget constraints. This situation could worsen if unit cost targets are not met. Finally, the estimated cost of operating and supporting the fleet over its life-cycle continues to be high and could increase further if aircraft reliability goals are not met.

DOD has already made a number of difficult decisions to put the F-35 on a more sound footing. More such decisions may lie ahead. For example, if software testing continues to be delayed, if funding falls short of expectations, or if unit cost targets cannot be met, DOD may have to make decisions about whether to proceed with production as planned with less capable aircraft or to alter the production rate. Also, if reliability falls short of goals, DOD may have to make decisions about other ways to reduce sustainment costs, such as reduced flying hours. Eventually, DOD will have to make contingency plans for these and other issues. At this point, we believe the most pressing issue is the effect software testing delays are likely to have on the capabilities of the initial operational aircraft that each military service will receive. In order to make informed decisions about weapon system investments and future force structure, it is important that Congress and the services have a clear understanding of the capabilities that the initial operational F-35 aircraft will possess.

Recommendation for Executive Action

Due to the uncertainty surrounding the delivery of F-35 software capabilities, we recommend that the Secretary of Defense conduct an assessment of the specific capabilities that realistically can be delivered and those that will not likely be delivered to each of the services by their established initial operational capability dates. The results of this assessment should be shared with Congress and the military services as soon as possible but no later than July 2015.

Agency Comments

DOD provided comments on a draft of this report, which are reprinted in appendix lll. DOD concurred with our recommendation.

We are sending copies of this report to appropriate congressional committees; the Secretary of Defense; the Secretaries of the Air Force, Army, and Navy; the Commandant of the Marine Corps; and the Director of the Office of Management and Budget. The report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Staff members making key contributions to this report are listed in appendix IV.

Appendix I: Prior GAO Reports and DOD Responses

Est. dev. Costs dev. Length Aircraft unit cost $34.4 Billion 10 years $69 Million

Key program event Start of system development and demonstration approved.

Primary GAO message Critical technologies needed for key aircraft performance elements are not mature. Program should delay start of system development until critical technologies are mature to acceptable levels.

DOD response and actions DOD did not delay start of system development and demonstration stating technologies were at acceptable maturity levels and will manage risks in development.

The program undergoes re-plan to address higher than expected design weight, which added $7 billion and 18 months to development schedule.

We recommend that the program reduce risks and establish executable business case that is knowledge-based with an evolutionary acquisition strategy.

DOD partially concurred but does not adjust strategy, believing that their approach is balanced between cost, schedule and technical risk.

Program sets in motion plan to enter production in 2007 shortly after first flight of the non- production representative aircraft.

The program plans to enter production with less than 1 percent of testing complete. We recommend program delay investing in production until flight testing shows that JSF performs as expected.

DOD partially concurred but did not delay start of production because they believe the risk level was appropriate.

Congress reduced funding for first two low- rate production buys thereby slowing the ramp up of production.

Progress is being made but concerns remained about undue overlap in testing and production. We recommend limits to annual production quantities to 24 a year until flying quantities are demonstrated.

DOD non-concurred and felt that the program had an acceptable level of concurrency and an appropriate acquisition strategy.

DOD implemented a Mid- Course Risk Reduction Plan to replenish management reserves from about $400 million to about $1 billion by reducing test resources.

We believe new plan increased risks and DOD should revise it to address concerns about testing, management reserves, and manufacturing concerns. We determined that the cost estimate was not reliable and that a new cost estimate and schedule risk assessment is needed.

DOD did not revise risk plan or restore testing resources, stating that they will monitor the new plan and adjust it if necessary. Consistent with a report recommendation, a new cost estimate was prepared, but DOD did not conduct a risk and uncertainty analysis.

Est. dev. Costs dev. Length Aircraft unit cost $44.4 Billion 13 years $104 Million

Key program event The program increased the cost estimate and adds a year to development but accelerated the production ramp up. Independent DOD cost estimate (JET I) projects even higher costs and further delays.

Primary GAO message Moving forward with an accelerated procurement plan and use of cost reimbursement contracts is very risky. We recommended the program report on the risks and mitigation strategy for this approach.

DOD response and actions DOD agreed to report its contracting strategy and plans to Congress and conduct a schedule risk analysis. The program reported completing the first schedule risk assessment with plans to update semi- annually. The Department announced a major program reducing procurement and moving to fixed-price contracts.

The program was restructured to reflect findings of recent independent cost team (JET II) and independent manufacturing review team. As a result, development funds increased, test aircraft were added, the schedule was extended, and the early production rate decreased.

Costs and schedule delays inhibit the program’s ability to meet needs on time. We recommend the program complete a full comprehensive cost estimate and assess warfighter and IOC requirements. We suggest that Congress require DOD to tie annual procurement requests to demonstrated progress.

DOD continued restructuring, increasing test resources and lowering the production rate. Independent review teams evaluated aircraft and engine manufacturing processes. Cost increases later resulted in a Nunn-McCurdy breach. Military services are currently reviewing capability requirements as we recommended.

Restructuring continued with additional development cost increases; schedule growth; further reduction in near-term procurement quantities; and decreased the rate for future production. The Secretary of Defense placed the STOVL variant on a two-year probation; decoupled STOVL from the other variants; and reduced STOVL production plans for fiscal years 2011 to 2013.

The restructuring actions are positive and if implemented properly, should lead to more achievable and predictable outcomes. Concurrency of development, test, and production is substantial and provides risk to the program. We recommended the program maintain funding levels as budgeted; establish criteria for STOVL probation; and conduct an independent review of software development, integration, and test processes.

DOD concurred with all three of the recommendations. DOD lifted STOVL probation citing improved performance. Subsequently, DOD further reduced procurement quantities, decreasing funding requirements through 2016. The initial independent software assessment began and ongoing reviews are planned to continue through 2012.

Est. dev. Costs dev. Length Aircraft unit cost $55.2 Billion 18 years $137 Million

Key program event The program established a new acquisition program baseline and approved the continuation of system development, increasing costs for development and procurements and extending the period of planned procurements by 2 years.

Primary GAO message Extensive restructuring places the program on a more achievable course. Most of the program’s instability continues to be concurrency of development, test, and production. We recommend the Cost Assessment Program Evaluation office conduct an analysis on the impact of lower annual funding levels; JSF program office conducts an assessment of the supply chain and transportation network. The program is moving in the right direction but must fully validate design and operational performance and at the same time make the system affordable. We did not make recommendations to DOD.

DOD response and actions DOD partially concurred with conducting an analysis on the impact of lower annual funding levels and concurred with assessing the supply chain and transportation network.

DOD agreed with GAO’s observations.

Appendix Il: Scope and Methodology

To assess the program’s ongoing development and testing we reviewed the status of software development and integration and contractor management improvement initiatives. We also interviewed officials from the program office, prime contractor, and the Defense Contract Management Agency (DCMA) to discuss current development status and software releases. In addition, we compared management objectives to progress made on these objectives during the year. We obtained and analyzed data on flights and test points, both planned and accomplished during 2013. We compared test progress against the total program plans to complete. We also reviewed the Director, Operational Test and Evaluation’s annual F-35 assessment. In addition, we interviewed officials from the F-35 program office and aircraft prime contractor to discuss development test plans and achievements. We also collected information from the program office, prime contractor and Department of Defense (DOD) test pilots regarding the program’s technical risks including the helmet mounted display, autonomic logistics information system, carrier arresting hook, and structural durability.

To assess the program’s funding and long-term affordability, we reviewed financial management reports, annual Selected Acquisition Reports, and monthly status reports available as of December 2013. In addition, we reviewed total program funding requirements from the Defense Acquisition Executive Summary. We used this data to project annual funding requirements through the expected end of the F-35 acquisition in 2037. We also analyzed the fiscal year 2014 President’s Budget data to identify the current status of unit costs for each variant, and the differences in this cost since 2012. We reviewed the Office of the Secretary of Defense’s F-35 Joint Strike Fighter Concurrency Quick Look Review, and discussed and analyzed reported concurrency costs with the prime contractor and program office. We obtained and discussed the life- cycle operating and support cost through the program’s Selected Acquisition Report and projections made by the Cost Analysis and Program Evaluation (CAPE) office. We identified changes in cost and interviewed officials from the program office prime contractor, Naval Air Systems Command, and the CAPE office regarding reasons for these changes. We also discussed future plans of the DOD and prime contractor to try and reduce life-cycle sustainment costs with officials from the prime contractor, program office, and CAPE. We analyzed reliability data and discussed these issues with program and prime contractor officials.

To assess manufacturing progress we obtained and analyzed data related to aircraft delivery rates and work performance data through the end of calendar year 2013. This data was compared to program objectives identified in these areas and used to identify trends. We reviewed data and briefings provided by the program office, prime contractor, and DCMA in order to identify issues in manufacturing processes. We discussed reasons for delivery delays and plans for improvement with the prime contractor. We also toured the prime contractor’s manufacturing facility in Fort Worth, Texas and collected and analyzed data related to aircraft quality through December 2013. We reviewed and discussed information on the prime contractor’s global supply chain including their management processes for oversight.

We assessed the reliability of DOD and contractor data by reviewing existing information about the data, and interviewing agency officials knowledgeable about the data. We determined that the data were sufficiently reliable for the purposes of this report. We conducted this performance audit from June 2013 to March 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: Comments from the Department of Defense

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition the contact name above, the following staff members made key contributions to this report: Travis Masters, Assistant Director; Marvin Bonner; Peter Anderson, Megan Porter, Roxanna Sun and Abby Volk.

Related GAO Products

F-35 Joint Strike Fighter: Restructuring Has Improved the Program, but Affordability Challenges and Other Risks Remain. GAO-13-690T. Washington, D.C.: June 19, 2013.

F-35 Joint Strike Fighter: Program Has Improved in Some Areas, but Affordability Challenges and Other Risks Remain. GAO-13-500T. Washington, D.C.: April 17, 2013.

F-35 Joint Strike Fighter: Current Outlook Is Improved, but Long-Term Affordability Is a Major Concern. GAO-13-309. Washington, D.C.: March 11, 2013.

Joint Strike Fighter: DOD Actions Needed to Further Enhance Restructuring and Address Affordability Risks. GAO-12-437. Washington, D.C.: June 14, 2012.

Joint Strike Fighter: Restructuring Added Resources and Reduced Risk, but Concurrency Is Still a Major Concern. GAO-12-525T. Washington, D.C.: March 20, 2012.

Joint Strike Fighter: Restructuring Places Program on Firmer Footing, but Progress Is Still Lagging. GAO-11-677T. Washington, D.C.: May 19, 2011.

Joint Strike Fighter: Restructuring Places Program on Firmer Footing, but Progress Still Lags. GAO-11-325. Washington, D.C.: April 7, 2011.

Joint Strike Fighter: Restructuring Should Improve Outcomes, but Progress Is Still Lagging Overall. GAO-11-450T. Washington, D.C.: March 15, 2011.

Tactical Aircraft: Air Force Fighter Force Structure Reports Generally Addressed Congressional Mandates, but Reflected Dated Plans and Guidance, and Limited Analyses. GAO-11-323R. Washington, D.C.: February 24, 2011.

Joint Strike Fighter: Assessment of DOD’s Funding Projection for the F136 Alternate Engine. GAO-10-1020R. Washington, D.C.: September 15, 2010.

Joint Strike Fighter: Accelerating Procurement before Completing Development Increases the Government’s Financial Risk. GAO-09-303. Washington D.C.: March 12, 2009.

Joint Strike Fighter: Recent Decisions by DOD Add to Program Risks. GAO-08-388. Washington, D.C.: March 11, 2008.

Joint Strike Fighter: Progress Made and Challenges Remain. GAO-07-360. Washington, D.C.: March 15, 2007.

Joint Strike Fighter: DOD Plans to Enter Production before Testing Demonstrates Acceptable Performance. GAO-06-356. Washington, D.C.: March 15, 2006.

Tactical Aircraft: Opportunity to Reduce Risks in the Joint Strike Fighter Program with Different Acquisition Strategy. GAO-05-271. Washington, D.C.: March 15, 2005.