Defense Acquisitions: Observations on the F-35 and Air Force's Advanced Battle Management System
Released: 2019-05-02T00:00:00

Letter

F-35 Modernization, Reliability, and Sustainment and Supply Chain Efforts Face Risks and Challenges

The F-35 plays a key role in DOD’s modernization efforts. However, it faces concerns in several areas that will inform the program’s cost and performance in the future. These include the risk in its modernization efforts, its aircraft not meeting all reliability targets, and sustainment and supply chain challenges. Specifically, the F-35 program plans to award Block 4 development contracts before it has key business case documents that would normally inform this decision. Also, the program is not meeting all of its Reliability and Maintainability (R&M) targets. Finally, the F-35 program’s sustainment costs are rising as it also faces significant supply chain challenges.

The F-35 Program Will Start Block 4 Development without a Full Business Case

The F-35 baseline aircraft program completed development in April 2018. It started formal operational testing of the baseline aircraft in December 2018 after a 3-month delay. This testing was delayed for two main reasons: (1) to resolve critical deficiencies identified in developmental testing, and (2) to accommodate an unexpected grounding following the crash of an F-35B in September 2018. According to a test official, the program expects to complete testing in December 2019, about 3 months later than planned due to delays with the simulator that is used for more complex testing. Until that testing is complete, there is still a risk that additional deficiencies may be identified. With the program wrapping up development of the baseline program, it is transitioning to early development and testing activities for the Block 4 modernization efforts, which the F-35 Joint Program Office estimates will cost about $10.5 billion. With Block 4, DOD plans to add new capabilities and modernize the F-35 aircraft to address evolving threats.

In April 2019, we found that DOD will not have a complete business case for Block 4 before it plans to award development contracts in 2019. Section 224 of the National Defense Authorization Act for Fiscal Year 2017 required DOD to submit a report containing certain elements of an acquisition program baseline—in essence, a business case—to include cost, schedule, and performance information and independent estimates—for Block 4. In 2018, we found that DOD’s report to Congress was incomplete. In its report, DOD stated that the acquisition program baseline would continue to be refined over the next year. DOD officials stated that the updated F-35 program baseline, with the Block 4 efforts included, will be released in April 2019.

Over the past year, the program has already invested over $1.4 billion, in part to gain the knowledge it needs to develop that business case, such as a preliminary design review, as well as to establish Block 4 testing facilities and support early capabilities’ development. The program incorporated some Block 4 activities into its acquisition strategy, which was approved in October 2018. However, we found that three key Block 4 business case documents will not be ready before the program’s planned development contract awards in May 2019: Independent Technology Readiness Assessment: A Technology Readiness Assessment is a systematic, evidence-based process that evaluates the maturity of hardware and software technologies critical to the performance of a larger system or the fulfillment of the key objectives of an acquisition program. The program office plans to conduct a partial independent assessment of initial capabilities sometime between October and December 2019 with additional assessments to follow. A program official stated that technologies will not be integrated into the aircraft until they are adequately mature. However, without a complete independent Technology Readiness Assessment, the program will not have identified potential critical technology elements and, as a result, may be at risk of delaying the delivery of new capabilities.

Test and Evaluation Master Plan: Although the F-35 program has begun testing Block 4 capabilities, it does not have an approved Test and Evaluation Master Plan. The Test and Evaluation Master Plan documents the overall structure, strategy, and objectives, as well as the associated resources needed for execution. Developmental and operational test officials have expressed concerns about the lack of an approved test plan, uncertain funding, the number of test aircraft available, and the draft test schedule, among other things. An approved, properly resourced test plan is essential for planning and preparing for adequate testing of the Block 4 capabilities. According to these officials, without an approved plan, the F-35 program is providing the test authorities with capabilities to be tested without giving them the necessary direction on how to adequately prepare to conduct the tests, making it difficult to execute testing. While this is still a concern, F-35 program officials explained that over the past 3 months they have been providing the test authorities with the direction needed to conduct testing.

Independent Cost Estimate: The Block 4 Independent Cost Estimate, which details the program’s total estimated life cycle cost, is not complete. In August 2017, we reported that DOD estimated the development funding needed for the first phase of Block 4 was projected to be over $3.9 billion through 2022. Since then, the program incorporated more fidelity and specific Block 4 efforts that were not in the original estimate into its Block 4 cost estimate. Based on the program office’s latest estimate, the cost of Block 4 capabilities is expected to be $10.5 billion through 2024. According to OSD’s Cost Assessment and Program Evaluation office, it will provide the Independent Cost Estimate between October and December 2019 to support the F-35 program’s pending full-rate production decision, but this would occur several months after the program plans to award the Block 4 development contracts. According to the GAO Cost Guide, an Independent Cost Estimate is considered one of the best and most reliable estimate validation methods as it provides an independent view of expected program costs that tests the program office’s estimate for reasonableness. Without an Independent Cost Estimate, Congress does not have insight into the full potential cost of Block 4.

The expected completion dates for these documents are between October and December 2019, at the earliest. Figure 1 shows key Block 4 dates such as the Block 4 re-plan, which included revising the cost estimate for Block 4 that DOD established in 2017, the planned development contract awards, and planned completion dates for the three remaining critical business case documents.

As seen in figure 1, the program office plans to award Block 4 development contracts in May 2019, at least five months before any of the critical business case documents will be available. Based on best practices identified by GAO, without an independent Technology Readiness Assessment, Test and Evaluation Master Plan, or an Independent Cost Estimate, program officials cannot have a high level of confidence that the requirements are firm and that risk has been adequately reduced before beginning efforts estimated to cost $10.5 billion in funding to develop Block 4. If program officials move ahead with Block 4 contracts without gaining the knowledge that a full business case would provide, Block 4 modernization efforts will be at risk of experiencing the same kind of cost and schedule growth the baseline development program experienced.

To address this risk, in April 2019, we recommended to the DOD that it should ensure the F-35 program office complete its business case, to include the three documents discussed above, at least for the initial Block 4 capabilities under development before initiating additional development work. DOD did not concur with this recommendation. In its comments, DOD stated that the F-35 program office has adequate knowledge to begin Block 4 development. We maintain, however, that completing its business case before awarding its Block 4 development contracts would put DOD and the program in a better position to effectively and successfully develop Block 4 capabilities.

The F-35 Program Is Still Not Meeting All Reliability and Maintainability Targets

As we reported in April 2019, the program has made slow, consistent progress in improving the F-35’s R&M metrics’ performance but half of the metrics are not achieving targets. All F-35 variants are generally performing near or above targets for four of the eight R&M metrics, while still falling short for the other four. Each F-35 aircraft variant is measured against eight R&M metrics, four of which are in part of the contract. All eight R&M metrics are described in the program’s Operational Requirements Document (ORD)—the document that outlines the targeted performance levels for these metrics that DOD and the military services agreed the F-35 should meet in 2000. Based on our analysis, while the program is on track to meet half of the targets, the program office has not taken adequate steps to ensure the others will be met. Additionally, in December 2018, the Director, Operational Test & Evaluation reported that, although performance for the four under-performing metrics has shown slow growth over the years, none of these metrics are meeting interim goals needed to reach requirements at each variant’s maturity. Each F-35 variants’ R&M performance against these metrics is shown in table 1.

Since the program began tracking R&M performance in 2009, it has seen small, annual improvements. Over the past year, all variants showed a slight improvement in targeted performance levels for one metric, the mean flight hours between failure—design controlled, but saw little or no discernable improvement for the four metrics not meeting targets. However, based on current performance, the program does not expect to meet those targets by full aircraft maturity. According to F-35 program officials, the ORD R&M metrics should be re-evaluated to determine more realistic R&M performance metrics, but the program has not yet taken actions to do so. Until the program office does so, it remains accountable for ensuring those ORD R&M metrics are achieved.

In June 2018, we recommended that the F-35 program identify steps it needs to take to ensure the F-35 aircraft meet R&M requirements before each variant reaches maturity and update its R&M Improvement Program (RMIP)—DOD’s action plan for improving R&M—with these steps. DOD concurred with our recommendation but has yet to take substantive actions to address it. DOD did, however, complete 16 improvement projects since we last reported on this. Despite completing these projects, there were not significant gains in the R&M metrics not meeting targets. Program officials advised, however, that measurable improvements in R&M can take time to manifest. To speed this process, the program is accelerating planned upgrades to older aircraft where appropriate, which officials stated should translate to an overall improvement in the program’s R&M performance.

The F-35 Program Office’s Improvement Plan Does Not Address Under- Performing Targets

The F-35 program office has estimated that implementing all of the identified improvement projects currently contained in its RMIP could result in potential life cycle cost savings of over $9.2 billion by improving the F-35’s R&M. However, we found that, as of December 2018, the guidance the F-35 program office has used to implement the RMIP does not define specific, measurable objectives for what the desired goals for the F-35’s R&M performance should be or align improvement projects with R&M goals. Furthermore, the RMIP has not been a funding priority.

Federal internal control standards state that programs should define objectives when implementing programs such as the RMIP. Although the F-35 program RMIP’s guidance has a general goal of improving R&M, it does not identify achieving the eight R&M targets listed in the ORD as an objective. Program officials acknowledged that the RMIP’s guidance does not include such an objective. Instead, officials stated they are using the RMIP to prioritize and fund projects that will improve aircraft availability and mission capability—neither of which are included in the eight R&M metrics, but are necessary and important initiatives. The program is focusing on these two areas in part because a September 2018 memorandum from the Secretary of Defense to the Secretaries of the military departments included a goal for the F-35 fleet to attain a mission capable rate of 80 percent by the end of fiscal year 2019. According to program officials, improving these two areas will translate into improvements in the F-35 overall R&M. However, we found that the RMIP’s guidance does not discuss these priorities or align how any improvement projects would ensure targets under all eight R&M targets will be met.

In our prior work on weapon system acquisitions, we have identified a number of best practices for improving program outcomes if implemented, such as clearly establishing well-defined requirements and securing stable funding that matches resources to requirements. We found that the program office has not prioritized or dedicated funding in its budget to improve R&M, in part because program officials explained that they were focused on initiatives intended to lower the cost of the aircraft. In addition, any current funding for R&M improvement projects comes from the program’s operation and maintenance funds, which are only available for one fiscal year. Officials explained that, if the funding runs out or is used by the program for other efforts, then R&M projects will go unfunded or be suspended until new funding is available. In fiscal year 2018, for example, while some projects were completed, several other projects were suspended when that year’s funding ran out. As of December 2018, according to a contractor representative, all of the identified improvement projects currently unfunded in the program’s RMIP would cost about $30 million to implement, but were not funded.

Program officials also stated that they are in the process of revising the RMIP and have considered including more specific objectives in addition to improving aircraft availability and mission capability, such as more focus on improving R&M performance where ORD R&M targets are not currently being met. According to the program, any revisions to the RMIP and changes to how it will be funded, however, will not be complete until April 2019.

By not defining objectives in its RMIP guidance for meeting all eight R&M metrics, aligning which improvement projects will ensure those metrics are met, and prioritizing funding for those projects, the program is at risk of not fully meeting its R&M targets. As a result, the warfighter may accept aircraft that are less reliable than originally planned, and whose operation and sustainment costs may raise affordability questions. In addition, the military services recently identified the need to cut sustainment costs—by 43 percent in the case of the Air Force—to improve the F-35’s affordability in sustainment. Increasing costs from less reliable aircraft will add strain to an already unaffordable program.

To address these issues, in April 2019, we recommended to DOD that it should ensure that the F-35 program office 1. assess whether the ORD R&M targets are still feasible and revise the 2. as it revises its RMIP, identify specific and measurable R&M objectives in its RMIP guidance; 3. as it revises its RMIP, identify and document which RMIP projects will achieve the identified objectives of the RMIP guidance; and 4. prioritize funding for the RMIP.

DOD concurred with these recommendations and stated that it will take actions to address them.

Continued Concerns with F-35 Sustainment Costs and Supply Chain, and Logistics System

We have previously reported on the F-35 program’s rising estimated sustainment costs and challenges maintaining an expanding fleet. In October 2017, we reported that estimated F-35 life-cycle sustainment costs increased by 24 percent from fiscal years 2012 through 2016 due to an increase in projected flying hours and other factors. We also reported that sustainment costs were not fully transparent to the military services. For example, the Marine Corps received an initial funding requirement for fiscal year 2017 sustainment of $293 million, which then increased to $364 million in the execution without a full explanation from the program office. We recommended that DOD take steps to improve communication with the services and provide more information about how F-35 sustainment costs they are being charged relate to the capabilities received. DOD concurred with the recommendation and has begun taking actions to address it.

In addition, DOD faces substantial supply chain challenges that are lowering F-35 aircraft performance. In April 2019, we reported that F-35 aircraft performance was falling short of warfighter requirements—that is, aircraft could not perform as many missions or fly as often as required. Specifically, F-35A aircraft were mission capable only 52 percent of the time from May through November 2018—far short of the 80 percent target set by the former Secretary of Defense. This lower-than-desired aircraft performance is due largely to F-35 spare parts shortages and limited part repair capabilities. For example, during this time period, F-35 aircraft were unable to fly about 30 percent of the time due to spare parts shortages.

Additionally, DOD’s capabilities to repair F-35 spare parts at its depots are years behind schedule, which has resulted in an average of 188 days to repair an F-35 part and a backlog of about 4,300 spare parts awaiting repair at military depots or manufacturers. We also reported that DOD faces challenges managing, moving and maintaining accountability of F- 35 parts within the supply chain. We made eight recommendations to DOD, including that DOD determine what actions are needed to close the gap between warfighter requirements for aircraft performance and F-35 supply chain capabilities. DOD concurred with the recommendations and identified actions that it was taking or planned in response.

Finally, the F-35’s Autonomic Logistics Information System (ALIS) has the potential to lead to increased costs for the program if key issues are not addressed. ALIS is the F-35’s central logistics system intended to support operations, mission planning, supply-chain management, maintenance, and other processes. In April 2016, we identified several risks, including that ALIS (1) was not initially designed to be deployable, (2) lacked redundant infrastructure, (3) did not communicate well with legacy aircraft systems, (4) had data accuracy and accessibility issues, and (5) had security risks. In addition, DOD had not included certain analyses and information, such as historical cost data, to increase the credibility and accuracy of ALIS’s estimated costs. Further, a 2013 DOD-commissioned study found that schedule slippage and functionality problems with ALIS could lead to between $20 billion and $100 billion in additional costs.

We have made several recommendations to DOD to improve ALIS planning and cost estimates, and to develop a performance measurement process for ALIS to better address problems based on actual system performance and user requirements. DOD generally concurred with our recommendations and has taken some actions, including developing a plan that identifies and prioritizes key ALIS risks. However, more work remains. We are currently conducting a review examining DOD’s progress in implementing our ALIS-related recommendations, addressing concerns from ALIS users, identifying emergent financial and operational risks associated with ALIS, taking near-term actions to improve ALIS functionality, and assessing DOD’s actions regarding the long-term viability of ALIS to ensure capable sustainment of the F-35 fleet. We plan to issue a report based on our current work later in 2019.

Air Force’s Advanced Battle Management System Acquisition Strategy Is in the Early Planning Stages

Based on our ongoing work, ABMS is early in the acquisition process, as the specific capabilities and overarching acquisition strategy are still to be determined by the Air Force. As a result, the Air Force has not yet established a cost and technical baseline for ABMS. When ABMS planning began in 2017, program officials stated that the intent of the program was to replace and modernize the capabilities of the AWACS system—which provides the warfighter with the capability to detect, identify, and track airborne and maritime threats. But changes in Air Force expectations for how it would fight during future conflicts led the department to assess options for developing a more robust and survivable air, land, and sea battle management system that can operate in contested environments. In July 2018, the ABMS Initial Capabilities Document—which describes capability needs and associated gaps—was approved by the DOD Joint Requirements Oversight Council.

Our ongoing work also found that, in December 2018, the Air Force determined it would not continue its planned JSTARS Recapitalization program—which was intended to provide surveillance and information on moving ground targets—well into the future, as initially expected. As a result of a recent study, the Air Force has extended the estimated service life of the JSTARS fleet, and will incorporate its capabilities into the ABMS in the short term, and retire JSTARS in the 2030s.

Our preliminary observations indicate that the details about ABMS are still to be determined. The Air Force expects to fully define ABMS through an Analysis of Alternatives (AOA) that it plans to complete by the summer of 2019, as shown in figure 2.

The ABMS AOA, led by the Air Force’s Air Combat Command, will assess how ABMS will deliver air-centric capabilities, such as those currently provided by AWACS. Air Force officials explained that they plan to utilize an existing AOA completed for the JSTARS Recapitalization program, approved in May 2012, to identify and assess ABMS’s potential ground target tracking capabilities. Originally planned as a 9-month study, Air Force officials stated that the ABMS AOA was shortened to a 6-month effort. As a result, the Air Force received conditional approval to reduce the number of alternatives studied from five to three.

Our ongoing work indicates that the Air Force plans to develop ABMS over three phases. The first phase began in fiscal year 2018 and goes through 2023. In this phase, the Air Force plans to integrate existing sensors, improve battle management systems, and upgrade communication networks across 10 existing acquisition programs. Table 2 includes information on three existing programs the Air Force plans to enhance during the first phase of ABMS.

According to an Air Force acquisition official, the technologies associated with the first phase are considered to be mature but there may be risks as the Air Force integrates technologies.

Air Force officials explained that their approaches to the second and third phases of ABMS are not fully developed, but noted that the phases would be informed by the AOA results. That said, the Air Force expects to start phase 2 in 2024 by integrating advanced sensors and software into its existing battle management command and control platforms while at the same time retiring JSTARS. Air Force officials have reported that the third phase, planned for the mid-2030s, is expected to provide multi-sensor, resilient battle management command and control capability using multiple types of communications methods, with an initial operational capability planned for 2035. The Air Force estimates that ABMS’s acquisition cost through fiscal year 2024 will be $3.8 billion.

Because ABMS is composed of many different defense acquisition programs, the Air Force intends to manage it as a family of systems directed by a Chief Architect and not a traditional acquisition program manager. According to the Air Force, the ABMS Chief Architect is the first of its kind, and the Air Force believes the position will be instrumental in integrating the various programs and technologies into an overall system. Based on our preliminary analysis, the roles and responsibilities of the Chief Architect have not been fully defined. However, according to the Air Force, the Chief Architect is expected to be responsible for (1) leading a high-level analysis and determining the overall design of ABMS, (2) coordinating with the service-level commands and the acquisition programs involved to make sure they are aligned with the ABMS development, and (3) identifying the enabling technologies for integration into ABMS.

Chairman Norcross, Ranking Member Hartzler, and members of the Subcommittee, this completes my prepared statement. I would be pleased to respond to any questions you may have. We look forward to continuing to work with the Congress as we to continue to monitor and report on the progress of the F-35 program and the ABMS.

GAO Contact and Staff Acknowledgments

If you or your staff have any questions about this testimony, please contact Michael J. Sullivan at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals making key contributions to this statement are Justin Jaynes (Assistant Director), Diana Maurer, Jennifer Baker, Desirée E. Cunningham, Alissa Czyz, Stephanie Gustafson, Kasea Hamar, Jeff Hubbard, Jessica Karnis, Matt Metz, Robin Wilson, and Lauren Wright.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.