F-35 Aircraft Sustainment: DOD Needs to Address Challenges Affecting Readiness and Cost Transparency
Released: 2017-10-26T00:00:00

Letter

Background

The F-35 Lightning II program, also known as the Joint Strike Fighter program, is a joint, multinational acquisition intended to develop and field a family of next-generation strike fighter aircraft for the U.S. Air Force, Navy, and Marine Corps (hereinafter referred to as the services), eight international partners, and foreign military sales customers. There are three F-35 variants and each will be a multi-role, stealthy strike aircraft replacement for or complement to legacy fighter aircraft, as seen in figure 1.

F-35 Milestones and Stakeholders

DOD initiated the F-35 program in October 2001, and it is nearing the end of system development and preparing for operational testing. DOD has also been concurrently fielding and operating a growing fleet of aircraft as part of low-rate initial production. As of August 2017, 253 aircraft have been fielded and are flying from nine locations in the United States and three international locations. The Marine Corps and Air Force declared initial operational capability in 2015 and 2016, respectively, and the Navy is scheduled to declare initial operational capability in 2018. In 2019, DOD plans to begin full-rate production of the aircraft. See figure 2 for a timeline of major events and anticipated fleet growth in the F-35 program.

By full-rate production, DOD would generally be required to establish adequate sustainment and support systems for the F-35. Per DOD guidance for weapon system acquisitions, these sustainment and support systems should be defined in a support concept that is incorporated into a sustainment strategy. For the F-35, this concept should comprise the necessary plans to conduct operations, maintenance, and sustainment throughout the system’s life cycle, with the F-35 Life Cycle Sustainment Plan serving as the principal document governing F-35 sustainment. According to F-35 operational requirements, this concept must provide warfighting and peacetime capability with the lowest cost of ownership, and all variants must be able to deploy rapidly, sustain high mission reliability, and sustain a high sortie-generation rate.

Sustainment for the F-35 aircraft is a large and complex undertaking with many stakeholders. The F-35 Joint Program Office is responsible for managing and overseeing the support functions required to field and maintain the readiness and operational capability of the F-35 aircraft across the enterprise. The F-35 program currently relies heavily on contractors to provide sustainment support and has two product support integrators. As the product support integrator for the aircraft system, Lockheed Martin is charged with integrating sustainment support for the system, including that for the F-35 supply chain, depot maintenance, and pilot and maintainer training, as well as providing engineering and technical support. Currently, DOD is contracting for sustainment support with Lockheed Martin largely through annual contracts, and according to F-35 program officials, plans to transition to 5-year, fixed-price, performance-based sustainment contracts in 2020.

DOD has established a Hybrid Product Support Integrator organization— a collaboration of government and contractor organizations tasked with managing product support to meet the F-35 strategy and performance outcomes. This organization was initially established in 2016 as a part of the F-35 Joint Program Office, and is expected to be fully implemented by 2019. According to program officials, the establishment of the Hybrid Product Support Integrator is an acknowledgement that DOD needs to take a more significant role in providing sustainment support for the F-35. In addition, the U.S. Air Force, Navy, and Marine Corps have each established an F-35 integration office or cell focused on how the services will operate and afford the F-35, among other things.

The F-35 Global Support Solution

DOD is planning to meet the sustainment requirements of its F-35 customers by providing a common, global support solution. As part of this common solution, participants share critical aspects of sustainment support, some of which are discussed below, and which are in various stages of implementation to support the growing fleet.

Depot maintenance: The F-35 sustainment strategy has a two-level maintenance concept, consisting of organizational-level maintenance performed by squadron-level personnel, and depot-level maintenance. Depot-level maintenance includes structural repair, software upgrades, engine system overhaul and repair, component repair, and other activities that require specialized skills, facilities, or tooling to conduct the repairs. DOD is establishing modification and repair capabilities at six military service depots in the United States and additional repair facilities overseas.

Supply chain: All F-35 customers, including the U.S. military services and international partners, share a global pool of spare parts, which is managed by Lockheed Martin. According to program officials, these pooled assets are unique to the F-35 and include consumable and repairable spare parts for the airframe, support equipment, pilot flight equipment, and training devices. The services and international partners can also purchase packages of spare parts that are tailored to their individual deployment and shipboard operational requirements.

Training: Currently, the F-35 program is conducting pilot and maintainer training at Eglin Air Force Base, Luke Air Force Base, Marine Corps Air Station Beaufort, and Naval Air Station Lemoore. The F-35 program’s training system includes pilot and maintenance training devices and courseware that are tailored to multiple variants and services.

Infrastructure: F-35 customers are responsible for setting up their own F-35 facilities—hangars, training facilities, and depots, among other things—and the program office works with them in a supporting role. Sustainment infrastructure requirements to support the F-35 are defined in a series of facility requirement documents that are updated and provided to all customers annually.

F-35 Costs and Technical Characteristics

Many of the costs of F-35 sustainment—also known as operating and support costs—are allocated across the military services and international partners based upon a number of factors, including the number of aircraft that each customer owns and their operational requirements. Such operating and support costs consist of sustainment costs incurred from the initial system deployment through the end-of-system operations, and they include all costs of operating, maintaining, and supporting a fielded system. The Office of the Director for Cost Assessment and Program Evaluation develops independent cost estimates for F-35 operating and support costs, which are reported in DOD’s annual F-35 Selected Acquisition Report as the official operating and support cost estimates for the program. Additionally, the program office develops an annual estimate for the operating and support costs of maintaining and supporting the F- 35 over its 60-year life cycle, which can differ from the estimate conducted by the Office of the Director for Cost Assessment and Program Evaluation, due in part to differences in assumptions between the two estimates. Additionally, there are numerous factors that will affect life- cycle operating and support costs for the F-35, including aspects of the F- 35 program that are still maturing. These include the following:

Reliability and maintainability: Reliability and maintainability data measure aircraft performance to determine how often the aircraft experiences failures and how much time it takes to repair those failures. These data are monitored through a series of metrics that measure the intended performance of the aircraft in meeting its requirements as it progresses toward maturity at a cumulative 200,000 flight hours, with at least 75,000 flight hours each for the F- 35A and F-35B, and 50,000 flight hours for the F-35C. Reliability and maintainability drive sortie-generation rates and the size of the logistics footprint for the F-35, as well as inform program operating and support costs, which are tied to the performance of the system at maturity.

Technical data: Technical data for weapon systems include the details necessary to ensure the adequacy of performance, as well as instruction, maintenance, and other actions needed to support weapon systems. Technical data constitute an important part of a weapon system program, such as the F-35. Identifying technical data needs, costs, and ownership are essential for DOD to effectively consider and maximize competition for future product support of F-35 sustainment.

DOD Is Currently Sustaining More Than 250 F-35 Aircraft, but Insufficient Planning Has Led to Significant Challenges That Pose Risk to Its Growing Fleet

DOD Is Sustaining More Than 250 F-35 Aircraft, but Faces Significant Challenges That Are Affecting Readiness

DOD has currently fielded and is sustaining more than 250 F-35 aircraft, and the number is expected to triple by the end of 2021 and keep growing as the program moves into full-rate production. DOD has also supported significant F-35 milestones such as the initial operational capability declarations of the Marine Corps and Air Force in 2015 and 2016, respectively, and the transfer of an operational squadron to Japan in early 2017. As a fifth generation aircraft, the F-35 is intended to improve situational awareness through sensor fusion and will enhance the ability of legacy aircraft to conduct various missions while flying together with it. The F-35 was also designed with increased stealth capabilities, the capacity to carry weapons internally instead of externally to reduce drag and enable stealth, and advanced sensor systems. In particular, the aircraft is designed to execute missions in high-threat areas, requiring fewer support assets and possessing a greater survivability rate as compared with fourth generation aircraft such as the Air Force’s F-16s and the Navy’s and Marine Corps’ F/A-18s. Squadron officials at multiple F-35 locations that we visited expressed enthusiasm for the unique capabilities of the aircraft, such as the increased situational awareness that the F-35 provides pilots relative to legacy aircraft and the relative ease with which pilots are able to learn how to employ its tactical capabilities. They also noted improvement in the performance of the aircraft as it has been continuously developed.

However, DOD is facing several key sustainment challenges that pose risks to its ability to meet current and future warfighter readiness requirements, and these could limit the ability of the military services to fully leverage the capabilities of the aircraft. Table 1 summarizes these challenges, which are largely attributable to insufficient planning, as discussed in more detail below.

Repair capacity: DOD does not have enough capacity to repair F-35 aircraft parts because the establishment of repair capabilities at the military depots is 6 years behind schedule. There are many different components of the F-35 aircraft that DOD plans to repair at the six military depots within the United States, as documented in an F-35 Depot Implementation Plan. Repair capabilities at the military depots were originally planned to be completed by 2016, but program officials told us that some capabilities have now been delayed until 2022. Program officials in part attributed these delays to the military services not providing enough funding for depot requirements; however, service officials told us that the program office did not clearly identify some depot requirements in a timely manner necessary for the services to fund those requirements.

In addition, DOD did not plan for and fund the stocks of material needed to repair parts at the depots—referred to as “lay-in material.” Program officials said that they had incorrectly assumed that lay-in material would be included as part of the contracts for establishing repair capabilities at the military depots. As a result, DOD has had to fund and negotiate additional contracts with the prime contractor for the lay-in material. Currently, moreover, due in part to the late identification of requirements and funding, the lay-in material to support repairs for more than a dozen different aircraft components is not expected to be delivered to the depots until months—or in some cases, years—after the technical capabilities to conduct the repairs have been established. As seen in figure 3, for certain F-35 parts, these delays have resulted in repair times that are significantly longer than those the program had projected, leading to repair backlogs. According to prime contractor officials, because of these capacity shortfalls, DOD is currently relying on the original equipment manufacturers to repair parts, but the capacity of these manufacturers is already strained by requirements to produce the parts needed to support aircraft production.

Program officials said that establishing the depot repair capabilities is now the F-35 Joint Program Office Product Support Manager’s top priority. As such, the program is working to implement several different initiatives to accelerate the development of repair capabilities, including trying to better align lay-in material requirements with the activation of repair capabilities, prioritizing the establishment of certain repair capabilities to align with the readiness requirements of the fleet, and looking at options to decrease the amount of time that it takes to establish repair capabilities for each component line. However, program officials said that plans are still preliminary, and that they are unsure how much funding will be available to implement these initiatives.

Spare parts: DOD is experiencing shortages of spare parts in the F-35 supply chain, resulting in lower than expected readiness. From January through August 7, 2017, the prime contractor reported that the average percentage of time that F-35 aircraft were unable to fly because they were awaiting parts was about 22 percent—more than double that of DOD’s objective of 10 percent, as seen in figure 4.

According to program office and contractor officials, the shortages of spare parts are due in part to the delays in the establishment of depot repair capabilities, incomplete plans and funding that did not account for the long lead time for parts, insufficient amounts of service funding, and poor reliability of certain parts. For instance, 19 percent of F-35 parts have a lead time of more than 2 years. The 2 to 3 years that it takes to procure these parts includes both a lengthy period for contracting and a period for the production of the parts once contracts have been established. However, program office and military service officials told us that the timing of prior service funding authorizations and contract awards did not account for this long lead time to procure parts, resulting in parts that were late to meet the military services’ operational needs. According to DOD officials, the parts within the F-35 global pool of spare parts are unique to the F-35 system and generally cannot be obtained from other sources.

The program office and prime contractor have identified steps needed to increase the availability of spare parts to prevent these challenges from worsening as the number of aircraft in the fleet grows, such as improving the production and repair capacity of suppliers and aligning the timing of the military services’ funding authorizations with the required lead time for parts. However, according to DOD documentation, planned funding and contract awards for fiscal years 2018 and 2019 are still forecasted to be later than needed to meet demand for new parts, and the program’s ability to accelerate this timeline is uncertain. Thus, parts shortages are expected to continue for several years and may worsen if DOD and the contractor are not able to fully implement these actions.

DOD Has Not Fully Defined Future Technical Data Needs, and Some Technical Data Are Immature

DOD has not fully defined all of the technical data it needs from the prime contractor to maximize the potential for future competition of contracts among providers for sustainment requirements, nor does it know the associated costs of these data. In 2014, we recommended that the program office develop a long-term Intellectual Property Strategy to include the identification of all critical technical data needs and their associated costs. As of September 2017, the program has taken some steps to develop an Intellectual Property Strategy, but it has not identified all critical needs and their associated costs. Program officials said that they are currently working with the prime contractor to develop a list of technical data requirements. Program officials said that once this effort is complete, DOD will be in position to begin prioritizing and negotiating for specific data rights that the program needs to facilitate its sustainment plans. Officials acknowledged, however, that there is risk associated with efforts to obtain required technical data rights for F-35 sustainment to promote increased competition because the contractors may not be willing to provide these rights, or the costs may be too high. They also told us that the program office deals with such risks on a case-by-case basis, and that if a data right needed by the program office to implement the sustainment strategy cannot be obtained, then plans will have to be adjusted accordingly. Program officials said that, in some cases, they will likely have to make legal claims against the prime contractor’s technical data rights assertions, based on government funding of such products.

Moreover, the technical data needed to repair F-35 aircraft, such as maintenance instructions, are still not fully developed. According to contractor officials, the contractor and DOD have developed and verified more than 84 percent of the unit-level technical data needed to address known maintenance requirements, such as instructions for how to replace specific parts on the aircraft. However, according to program and contractor officials, the technical data needed for maintainers to troubleshoot issues with the aircraft are lagging behind planned development. Such data are intended to help maintainers when the source of a maintenance issue is unclear, by providing guidance on the actions needed to isolate the most likely problems. In the absence of troubleshooting instructions, maintainers sometimes incorrectly identify what needs to be fixed on the aircraft. For instance, officials from one squadron said that the troubleshooting data are sometimes insufficient to pinpoint the issue with the aircraft, which can lead the maintainer to remove a component, order a new part from the contractor, and subsequently find that the new part does not fix the issue—a scenario that is both inefficient and costly. According to program and contractor officials, the immaturity of technical data for troubleshooting maintenance issues could be contributing to the high rate of parts that the F-35 squadrons are sending to the depots for repair that do not actually need to be repaired, resulting in inefficiencies at the depots. For example, officials at one depot we visited said that 68 percent of the parts they receive from F-35 squadrons do not need to be repaired and that the process for testing such parts usually takes nearly 10 hours to complete, which is both inefficient and can add to repair backlogs.

DOD Has Not Developed a Plan for Intermediate-level Maintenance Capabilities

The Navy and Marine Corps require intermediate-level maintenance capabilities for shipboard deployments because it is more difficult and time-consuming to obtain spare parts, or to send parts to the depots for repair, when onboard a ship. DOD has been conducting analyses to support the requirement and has recently identified the initial intermediate-level repair capabilities that it plans to implement, including select avionics, support equipment, and hydraulic repairs. These decisions will trigger other requirements and related costs that must be planned for—such as for personnel, technical data, support equipment, and updates to policies governing the maintenance of spare parts— before the capability can be implemented. For example, program officials told us that once determinations are made about intermediate-level maintenance, the program will have to develop a plan that specifies what technical data rights are needed, and when, to facilitate intermediate-level maintenance, and will then have to negotiate with the contractor to obtain those technical data rights. In August 2017, the program office identified new funding requirements for DOD to implement initial intermediate-level maintenance capabilities for fiscal years 2019 through 2023. However, these requirements are not currently funded in DOD’s budget, leaving a projected shortfall of $267 million over this time period.

Because a funded plan for intermediate-level maintenance is not yet in place, the Marine Corps will not have the desired level of intermediate- level maintenance capabilities for its initial shipboard deployments planned for 2018. Accordingly, it will be highly reliant on the currently challenged F-35 supply chain and depot repair capabilities for support, and will likely experience degraded readiness. In addition, without such a plan, it is unclear whether such capabilities will be available to support the Navy’s first planned F-35 shipboard deployments in 2021.

DOD Faces Delays in Required ALIS Development, and Its Development Plan Is Not Fully Funded

Central to F-35 sustainment is the Autonomic Logistics Information System (ALIS)—a complex system supporting operations, mission planning, supply-chain management, maintenance, and other processes. However, ALIS is in continuous development, with planned updates that support required sustainment capabilities for years to come. For example, future versions of ALIS are intended to improve data collection and reporting, and to provide capabilities to support intermediate-level maintenance. Historically, ALIS has experienced delays. For instance, an ALIS version that was initially planned to be completed for testing in 2010, is now being tested in 2017. In 2016 we found that DOD did not have a plan to ensure that ALIS was fully functional as key program milestones approached, and we recommended that DOD develop a plan to prioritize and address ALIS risks. Since that time, the program office has implemented this recommendation through the development of an ALIS Technical Roadmap to plan for these requirements. However, emerging requirements, such as to address cyber security vulnerabilities and system obsolescence, will likely lead to changes in the Roadmap that could further delay the date when these sustainment capabilities are provided. Furthermore, the requirements and associated timelines for ALIS development that are identified in this plan may not be realistic because the requirements are not fully funded in upcoming service budgets, resulting in additional risks to the program’s plan.

DOD’s Sustainment Plans Do Not Fully Include Key Requirements, Associated Timelines, and Aligned Funding, but Some Initial Steps Are Being Taken

As discussed above, DOD’s challenges are due in large part to sustainment plans that do not fully include key requirements, associated timelines, and aligned (that is, timely and sufficient) funding to support those requirements. F-35 program stakeholders have long recognized the program’s need for more comprehensive and detailed planning documents to identify the key activities and decision points necessary to establish sustainment capabilities and guide the F-35 sustainment strategy. For instance, in 2009 an Independent Logistics Assessment team recommended, among other things, that DOD develop a program- wide integrated master schedule that includes key governmental activities and tasks necessary to establish F-35 logistics capabilities required through full-rate production, but the program did not develop such a tool. In 2014 the program office identified the need to establish a road map with clear decision points to prepare the F-35 enterprise for long-term sustainment. Finally, in December 2016 the Under Secretary of Defense for Acquisition, Technology, and Logistics directed the program office to submit an integrated master schedule for the deployment of global F-35 sustainment capabilities by January 2017, which is not yet completed.

Program officials said that they are now developing an integrated master schedule, and that this schedule will incorporate major sustainment milestones required to implement the program’s sustainment strategy. DOD is also updating sustainment strategy documents, including the F-35 Life Cycle Sustainment Plan and Acquisition Strategy, to include an Intellectual Property Strategy. However, the timeframes for completion of these documents are uncertain, in part due to ongoing DOD efforts to refine its follow-on modernization plans for the F-35, which will affect the sustainment plans. Thus, the scope and the degree to which these updates will address the challenges that DOD is facing are unclear. For instance, an Office of the Secretary of Defense official charged with reviewing these plans said that there is still significant work to be done by the military services and the program office to identify and align sustainment requirements with funding in order to support the fiscal year 2019 budget process, which will ultimately be necessary to inform these plans. Military service headquarters officials told us that, as customers of the program, they need to better understand from the program office when sustainment capabilities—such as military depots—will be established, and when associated funding is needed to support that schedule. In August 2017, the program office identified some specific funding requirements for the military services, beyond what they have already budgeted for F-35 sustainment, which are needed to address some of the sustainment challenges discussed above—including spare parts shortages, gaps in depot lay-in material, and ALIS development. While this is a positive step by the program office, it also demonstrates that DOD faces a funding shortage of approximately $1.5 billion between fiscal years 2018 and 2023 for F-35 sustainment, as well as significant readiness risks associated with this lack of alignment between requirements and funding.

The different elements of F-35 sustainment support are highly integrated, and challenges or delays in one area can significantly affect outcomes in other areas. For example, the delays in established repair capacity at the depots constitute one of the reasons why the program has an insufficient supply of spare parts. Procurement decisions can also significantly affect sustainment outcomes. The Air Force and Marine Corps are considering an acceleration of their purchases of F-35 aircraft, thus creating more demand on the already strained sustainment enterprise, for which DOD has not always provided timely funding (for example, funding for spare parts).

Our prior work on acquisition program management has identified a number of key program management practices that can improve program outcomes if implemented, such as clearly establishing well-defined requirements, developing realistic cost estimates and schedules, and securing stable funding that matches resources to requirements. As DOD prepares for the growth of the fleet and attempts to address existing sustainment challenges, its effort to develop an integrated master schedule is a positive step. Such a schedule, if comprehensive and realistic, could be a critical tool to guide the revision of DOD’s sustainment plans to better ensure that the plans that form the basis of its strategy are sufficient to meet warfighter requirements. Ultimately, however, without plans that include all key requirements and decision points with aligned funding, the F-35 program will likely face continual challenges in providing timely sustainment support to the warfighter, and may have difficulties in fully implementing its F-35 sustainment strategy in time to meet the needs of a growing fleet. Further, as the services consider accelerating their purchases of F-35 aircraft, DOD risks purchasing aircraft that the program and the services are not ready to sustain.

DOD Is Testing Agreements with the Contractor but May Not Be Well Positioned to Enter into Multi-year, Performance-based Sustainment Contracts by 2020

DOD Is Testing Performance-based Agreements to Incentivize the Prime Contractor

DOD is conducting pilot—or trial—performance-based agreements with the prime contractor as a part of its annual cost-reimbursable sustainment contracts, in order to test metrics and performance-management processes. According to F-35 program officials, DOD plans to transition to multi-year, fixed-price, performance-based contracts in fiscal year 2020. Performance-based logistics is a support strategy that emphasizes performance in contracts, rather than delivery of goods and services, and payment is related to the degree to which performance meets contracted standards. In 2012, the Under Secretary of Defense for Acquisition, Technology, and Logistics directed an increased use of performance- based logistics agreements, stating that such agreements can yield significant cost and performance benefits if effectively implemented.

DOD has developed a series of performance objectives to provide insight into the level of sustainment support that the prime contractor is providing to the military services. From these objectives, DOD has selected three system-level metrics, listed below, to incentivize the contractor under the pilot performance-based agreements:

Air Vehicle Availability (AVA): measures the percentage of total time during which aircraft are safe to fly, available for use, and able to perform at least one tasked mission;

Full Mission Capable (FMC): measures the percentage of time during which aircraft are fully capable of accomplishing all tasked missions;

Mission Effectiveness (ME): measures the extent to which the F-35 components and mission systems affected the successful completion of each assigned mission.

In these pilot agreements, DOD and the contractor together negotiated minimum and objective targets against which the performance of the aircraft—and the support provided by the contractor to enable that performance—is measured. For fiscal years 2016 and 2017, these agreements were 1-year, cost-reimbursable contracts with potential incentives for the contractor based on assessed performance of the aircraft across the three system-level metrics. According to F-35 program officials and documentation we reviewed, DOD plans to establish a 2-year contract for fiscal years 2018 through 2019, with select elements that are performance-based, in preparation to transition to a 5- year, fixed-price, performance-based contract for the 2020—2024 time period. Program officials said that this 5-year contract is planned to include 2 base years and 3 pre-negotiated option years.

DOD Has Not Achieved Most of Its Performance Targets for the Pilot Agreements and May Not Be Using the Appropriate Metrics to Achieve Desired Outcomes

DOD Has Not Achieved Most of Its Performance Targets

DOD did not achieve most of the performance targets that it set for the pilot performance-based agreements for the 2016 sustainment contract. Subsequently, DOD negotiated lower targets for some metrics in the 2017 sustainment contract. As of June 2017, DOD was meeting several of the minimum targets established in the 2017 sustainment contract, but none of the objective targets. According to program and contractor officials, the failure to meet these targets is largely due to the sustainment challenges that we discussed previously in this report. For example, the limited availability of spare parts within the F-35 supply chain is contributing to lower than expected AVA and FMC rates. Figure 5 below shows the actual fleet performance results for the 2016 and 2017 (through June 2017) pilot performance-based agreements. The 2016 pilot performance- based agreement began in March 2016 and spanned a 10-month period, through December 2016. The 2017 agreement began in March 2017, and program officials said that it is expected to continue through February 2018.

Furthermore, the performance targets established in the sustainment contracts for the pilot performance-based agreements are lower than the desired aircraft performance targets that the services have identified for their aircraft. As part of the pilot performance-based agreements, each of the military services has established individual agreements with the program office that identify their respective required levels of minimum and objective aircraft performance for their units, across key metrics. Program officials said that while they try to meet the services’ performance requirements when negotiating the contracts, the agreements with the services are not binding. The performance targets that have been negotiated on the sustainment contracts are generally lower than those required by the services. For instance, the Marine Corps established a minimum performance target for non-deployed units of 60 percent FMC aircraft for 2017, but the minimum target established in the contract for that same metric was 14 percent. Similarly, the Air Force identified a minimum performance target for non-deployed units of 65 percent AVA, but the minimum target established in the contract for that same metric was 52 percent. Program officials said that the costs of meeting the services’ performance requirements would be too high given the current supply chain challenges across the fleet. Figure 6 shows the differences between the performance targets required by the Marine Corps and those that DOD was able to negotiate under the pilot performance-based agreement in 2017.

DOD may not be using the appropriate metrics under the pilot performance-based agreements to achieve desired outcomes. DOD guidance states that optimal performance-based contracts use objective, measurable, and manageable metrics that accurately assess the support provider’s performance against the delivery of targeted warfighter outcomes. It also defines ideal metrics as those that are, among other things: (1) reflective of processes over which the contractor has control, and (2) able to motivate desired behavior. We found the following:

The contractor does not have full control over the performance outcomes for which it is paid: The system-level metrics that the prime contractor is being assessed against are not fully reflective of processes over which the contractor has control, because actions that the F-35 squadrons take when maintaining or operating the aircraft affect the metric outcomes being measured. For example, a contractor official at one site that we visited cited an instance when a military service maintainer towed an aircraft into a hangar and broke a surface panel, resulting in the aircraft not being able to fly for 60 days because there was no surface panel replacement available in the supply chain. Thus, the contractor could be held accountable for a lack of performance that the customer created. Conversely, to keep aircraft flying, military service maintainers have taken actions that mask contractor failures to provide support—for example, cannibalizing parts from other aircraft at rates significantly higher than DOD intends, based on data provided by the prime contractor and shown in figure 7. Because the contractor does not fully control the outcomes for which it is being assessed, prime contractor and military service officials said that contentious negotiations have occurred at times about how to assign responsibility for performance. This ultimately makes it difficult for DOD to hold the contractor accountable. Further, one of the three system-level metrics—Mission Effectiveness—is assessed by pilots subjectively after each flight. Some pilots and service officials whom we spoke to said that different pilots may make differing determinations about the effectiveness of the mission, which could affect the measured performance outcomes.

DOD has established performance review groups to review and reconcile data in instances where the contractor does not believe that it should be held responsible for certain metric outcomes, but this process requires both DOD and the contractor to make subjective determinations about the root causes of particular performance failures in order to determine whether the contractor or the military services are to blame. Figure 8 shows how this reconciliation process can result in adjustments to the measured performance data when assessing the level of support provided by the contractor. Under the pilot performance-based agreements, the reconciled data points serve as the basis for calculating contractor incentive fees.

Additionally, DOD is working to implement agreements that define lower- level metrics for which the military services will be held responsible, such as defining how long it should take for maintainers to conduct maintenance, but these agreements have not yet been fully implemented. Ultimately, service officials told us that the complexity of these adjudication efforts indicates that DOD may not be holding the contractor accountable for the appropriate metrics.

Current metrics may not motivate the desired behaviors from all stakeholders. The current metrics may not consistently motivate the necessary behaviors from all stakeholders to either achieve desired warfighter outcomes or meet the current metrics on contract. For example, DOD has established AVA as its primary metric, and it provides greater incentive fees to the contractor for meeting the AVA targets as compared with the other two metrics. However, Marine Corps and Navy officials told us that FMC aircraft are more important for operational deployments, as they represent aircraft that are ready for war. DOD’s performance-based logistics guidance states that it is important to exercise caution when selecting a combination of metrics, to ensure that they do not create undesirable conflicts. The achievement of the AVA and FMC metrics may at times be in conflict with one another. For instance, according to contractor and program officials, an aircraft is still considered to be available if its low observable—or stealth—systems are not working, but for it to be considered a fully-mission capable aircraft, a military service would have to ground the aircraft for several days to repair the low observable system. Contractor officials have also expressed concern that the metrics they are being paid for may not be as important to the services as other factors—such as achievement of flying hours or the ability to train pilots—and that this could affect whether the services will take all necessary actions to meet the targets for which the contractor is paid. Officials from a training unit we visited said that they were focused on training pilots, not on achieving the metric targets identified in the contract. This unit was able to exceed its required flight hours to support pilot training in April 2017, even though the performance of its aircraft fell well below desired Marine Corps performance levels for AVA and FMC.

Program office and contractor officials noted that pilot performance-based agreements were put in place to gather lessons learned and ensure that DOD has the appropriate metrics before entering into 5-year, fixed-price contracts. However, contractor officials said that the performance review process does not include a step to review how the metrics are driving behaviors or to determine whether DOD has the appropriate metrics in place, and they suggested that a more robust effort to consider lessons learned from the pilot agreements is needed. Service officials have suggested that incentivizing simpler metrics that focus on individual aspects of F-35 sustainment for which the contractor has more control— such as supply chain responsiveness or depot-level repair—instead of system-level performance metrics may be more appropriate. Without reexamining the metrics to ensure that they are objectively measurable, reflective of processes that the contractor can control, and able to motivate desired behaviors, DOD may not be well positioned to accurately assess contractor performance or achieve optimal outcomes across future performance-based sustainment contracts that will likely cost tens of billions of dollars.

DOD Does Not Yet Have Full Information on F-35 Sustainment Costs or Technical Aircraft Characteristics

DOD does not yet have full information on F-35 sustainment costs or technical characteristics such as reliability and maintainability, and this could pose risks to its ability to effectively negotiate 5-year, fixed-price performance-based contracts with the prime contractor by 2020. Although DOD has fielded more than 250 aircraft, the aircraft system remains immature. DOD has established a target for system maturity of 200,000 total flight hours, with minimum flight hours for each variant. DOD reached 100,000 total F-35 flight hours in July 2017, and it does not expect to reach its maturity targets for all variants until fiscal year 2024.

Specifically, we found that DOD does not have full visibility into the actual costs for some key sustainment requirements that are considered cost- drivers within the program, such as the actual costs of parts and repairs. Given the immaturity of the system, DOD has relied on projected parts reliability and pricing to formulate cost estimates, but officials said that actual costs are needed to improve both their confidence in the estimates and their understanding of how cost is related to performance. There is potential for the actual costs of sustainment requirements to change significantly from initial projections. For instance, the costs of initial spare parts over the life cycle increased by $447 million in the program’s estimate from the 2014 estimate to the 2015 estimate, due largely to increases in unit prices from those initially projected. According to program officials, their understanding of actual costs is limited in part because of the immaturity of the system. Program officials said that they are taking steps to obtain more actual cost information as the aircraft matures, and to determine how much repairs should cost, in order to better position themselves for contract negotiations. However, in addition to system immaturity, program officials said that they are experiencing challenges in obtaining important details about existing cost data needed to inform their cost models from the contractor, such as the costs of the individual parts and repairs that the contractor purchases from its suppliers.

Further, we found that there are a number of technical aspects of the aircraft that are immature or uncertain. While the F-35 is meeting expectations for some measurements of reliability and maintainability, other measurements are still lagging behind operational requirements. For example, aircraft are experiencing failures that result in the loss of a capability to perform a mission-essential function at more than twice the rate expected across all variants. Mean repair times for critical components that fail are also more than twice as long as the operational requirements dictate. Additionally, the significant software releases required to complete F-35 system development—referred to as Block 3F—are planned to be tested and released in 2017. However in April 2017 we reported that the program’s schedule for completion of Block 3F and associated testing would likely be delayed due in part to software issues and system instability. Additionally, as of June 2017, the DOD Office of the Director for Operational Test and Evaluation predicted that required initial operational test and evaluation for Block 3F would likely not begin until late 2018 or early 2019. According to operational testing officials, such software releases can lead to different reliability and maintainability issues than were previously known, as the aircraft becomes capable of flying at higher speeds and altitudes. According to these officials, there would be inherent risk in signing a fixed-price, performance-based contract before the reliability and maintainability data for Block 3F are more fully known, as those data will influence how much aircraft performance should cost at maturity.

DOD guidance states that in order for performance-based arrangements to be effective, the government must clearly understand program requirements, costs, and technical characteristics; and that systems should achieve a level of maturity and design stability. Program officials said they believe that DOD can gain sufficient knowledge of the costs and technical characteristics of the aircraft prior to 2020, and that they will seek to write options into the multi-year, performance-based contract if there are still risks that need to be mitigated. However, program officials said that the program office has not established criteria addressing the extent of the cost and technical data that it will require prior to entering into the planned agreements. While the program still has a few years until that date, program officials said that the process to develop this contract is expected to begin in late 2017. In April 2017 we reported on the risks of moving forward with additional F-35 program development before DOD has a full understanding of the aircraft’s baseline Block 3F capabilities, specifically citing difficulties in presenting a sound business case for soliciting contractor proposals without such knowledge. The program office could face similar challenges preparing for a fixed-price, performance-based sustainment contract amid existing uncertainty.

Without a full understanding of F-35 costs and technical characteristics at maturity, DOD may not be well positioned to accurately determine how much fleet performance should cost over a 5-year, fixed-price, performance-based contract, and thus may be at risk of overpaying the contractor while not receiving the expected level of sustainment support.

DOD Has Taken Some Actions Aimed at Reducing F-35 Sustainment Costs, but These Costs Continue to Rise and Are Not Fully Transparent to the Military Services

DOD Has Undertaken Some Initiatives to Reduce Rising F-35 Sustainment Costs but Has Not Established Affordability Constraints Based on the Military Services’ Budgets

DOD has taken some actions to try to reduce estimated sustainment costs for F-35 operating and support, which, according to the program office’s fiscal year 2016 cost estimate, are projected to cost $1.06 trillion in then-year dollars (see figure 9 below). For example, the program office has established a Cost War Room to identify and implement cost- reduction initiatives with the goal of reducing the program office’s 2012 operating and support cost estimate by 30 percent by 2022. These initiatives include updating assumptions about fuel usage, among others. According to program documentation, such efforts are projected to result in a cost avoidance of $60.7 billion. The program office also has an effort targeted at improving reliability and maintainability of F-35 components. As of May 2017, the program office had completed 38 improvement projects that are expected to result in $1.7 billion in operating and support cost avoidance. However, at the same time, the projected operating and support costs estimated by the program office have increased from fiscal year 2012 to fiscal year 2016, due to an increase in projected flying hours, an extension of the aircraft’s life cycle from 56 to 60 years, and refinements to the cost models, among other factors. Figure 9 shows the increase to the program office’s life cycle operating and support cost estimate since fiscal year 2012.

In addition, DOD has not established affordability constraints for the F-35 program that are linked to the military services’ budgets, as we recommended in September 2014. In our prior work, we found that the program’s affordability targets may not be reflective of what the services can actually afford because it did not use the military services’ budgets to establish the targets. At that time, the annual F-35 operating and support costs were estimated to be considerably higher than the combined annual costs of several legacy aircraft, and according to DOD officials, the sustainment strategy was not affordable. We recommended that DOD establish affordability targets linked to the services’ budgets, because without such targets DOD cannot be sure whether the cost savings they are pursuing will lead to an affordable sustainment strategy. The department concurred with this recommendation but has not taken specific action on it at the program level. We made this a priority recommendation for DOD in July 2017. The Senate Armed Services Committee also directed DOD to provide it with a plan for improving the transparency and affordability of the F-35 sustainment strategy, to include identifying affordability constraints linked to, and informed by, the military services’ budgets. The Marine Corps has recently taken steps to develop budget-based affordability targets for their portion of F-35 sustainment costs. The Marine Corps identified the need to reduce steady-state sustainment costs per aircraft by at least 20 percent through cost modeling efforts and budget analysis, and Marine Corps officials said they believe that such a reduction would make the program affordable for the Marine Corps. Marine Corps officials stated that to achieve such reductions, they are exploring options to reduce costs—such as transitioning maintenance tasks from depots to operational units, and revising sustainment support personnel requirements—in coordination with the program office and prime contractor. The program office could use this service target to inform the establishment of program-level affordability constraints.

As previously discussed, the program is experiencing sustainment challenges due in part to some requirements not being fully funded, and this could present a continued risk going forward if sustainment for the F- 35 is not affordable within the services’ budgets. Program officials also told us that if the services cannot fully fund sustainment requirements, DOD will have to prioritize funding and defer requirements to later years. However, given the F-35’s global sustainment strategy of providing support across the military services and the international partners through shared pools of funding, a single customer that cannot fully fund requirements may affect the ability of DOD and the contractor to provide adequate sustainment support across the global F-35 fleet.

Actual F-35 Sustainment Costs Are Not Fully Transparent to the Military Services

F-35 actual sustainment costs that are being charged by the program office to the military services, as well as the capabilities associated with those costs, are not fully transparent to the services. In addition to estimating projected costs for F-35 sustainment over the aircraft’s life cycle as described above, the program office also calculates the actual F- 35 sustainment costs that will be charged to the military services on an annual basis. To determine these actual sustainment costs, the military services first submit their F-35 sustainment capability requirements to the program office for approval. The program office approves requirements as a basis for its annual life-cycle operating and support cost estimate, which is used to provide each of the military services with an estimate for their respective portion of F-35 sustainment costs to support the services’ budget planning process. The program office then negotiates with the prime contractor the level of support the contractor will provide to meet service sustainment requirements. It is at this point that the program office informs the services of the actual costs that they will be charged for contracted sustainment. According to program officials, the contracted level of support may not include all the requirements initially submitted by the military services for a given contract period, and the associated costs of the contract services may not align with initial estimates given to the military services, because support is negotiated between the program office and the prime contractor.

Air Force, Navy, and Marine Corps officials told us that they do not fully understand how the actual costs that they are charged by the program office for F-35 sustainment are clearly linked to the capabilities that they are receiving. They cited issues related to unexplained cost increases, difficulty in tracking their requirements to the contracts, and concerns about how to track their dollars to shared pools of sustainment assets, as discussed in detail below.

Unexplained or unexpected growth in actual sustainment costs: Service headquarters officials cited concerns about unexplained or unexpected growth in sustainment costs, particularly between the cost estimate that they were quoted for budget planning purposes and what they are actually charged by the program office in the budget execution year. For example, according to program documentation, the Marine Corps was initially given a funding requirement for fiscal year 2017 sustainment support of $293 million, which then increased to $364 million in the execution year, largely due to increases in contractor personnel costs. Marine Corps officials said that the reasons behind this growth in personnel costs were not clearly substantiated for the Marine Corps by the program office. In order to afford these increased costs for sustainment support, Marine Corps officials said that the Marine Corps had to reduce its planned flying hours. In another instance, documents provided by the Navy show that the program office increased the cost of the Navy’s and Marine Corps’ combined spare parts requirements for fiscal year 2017 from an original estimate of $261 million to $402 million over the course of the execution year. In addition, service officials told us that they sometimes become aware of the growth in sustainment costs late in the services’ budgeting process, making it difficult for them to find additional funding for such changes.

Tracking requirements to negotiated contract services and costs: Officials from two of the services told us that they have had difficulty in tracking their respective services’ requirements to the costs being charged by the program and the capabilities that are negotiated on the contract. For instance, Air Force officials stated that the Air Force specified a desired performance level for AVA of 65 percent to the program office as a minimum target for its squadrons, but ultimately the program office contracted for a target of 52 percent. Air Force officials said they were not aware of this change until after the contract was negotiated. Similarly, Navy officials also told us that the program office does not notify the Navy of changes from the estimated costs to the actual contract costs or the requirements that are included during negotiations for sustainment contracts, even when the requirements differ from what the Navy intended. As a result, officials said that the services often have limited visibility into the support that the contractor will provide along with the actual costs for which the services are responsible, until after the contract is signed.

Shared pools of F-35 sustainment assets: These transparency concerns are complicated by the fact that the services are paying into shared pools for F-35 sustainment, and the costs they are being charged for some requirements—such as for spare parts—cannot be directly tracked to an item that the services own or support that is specifically provided to an individual service. Service officials said that the funds they have contributed to the shared sustainment support have not resulted in the expected sustainment support. Specifically, Air Force officials questioned why key performance points in the program—such as depot repair capabilities and supply availability— are lagging by several years in some instances, and said that they need better accounting from the program office on how the money the Air Force has contributed to the program has been spent, and why those funds have not resulted in improved performance. Furthermore, Air Force officials raised questions about whether all program participants are paying for their required shares of F-35 sustainment costs, and said that they have not been able to obtain such information from the program office.

This lack of transparency is due in part to insufficient communication between the program office and the services, particularly as requirements and costs change. Program officials have acknowledged that the program office has not always provided the services with the level of detail and clarity around costs that the services would like, but said that recently the program has been more focused on communicating with the military services. Program officials also told us that the services are free to contact the program office should they have any concerns regarding F-35 sustainment costs and how they are shared. However, given the consistent concerns expressed to us across the services, it appears that this level of dialogue has not been adequate to facilitate the services’ understanding of sustainment costs. Two of the services have requested organizations external to the F-35 program to conduct reviews of the program to better understand their respective portions of F-35 sustainment costs and, in some cases, identify potential opportunities for cost savings. While these studies will likely provide valuable information to the services and the program office, they also add costs to an already expensive weapon system. For example, according to program officials, the contract for the study requested by the Marine Corps has cost the program office at least $2.7 million. Further, reliance on one-time studies by external organizations to help program participants understand their F- 35 sustainment costs and associated capabilities is not a practical substitute for the effective communication needed in a program of this magnitude.

F-35 program guidance has emphasized the need to ensure that costs are transparent to stakeholders. Further, our prior work examining programs with multiple governmental customers found that when customers understand how costs and underlying assumptions are determined, they can better anticipate potential changes to those assumptions, identify their effects on costs, and incorporate that information into their budget plans. Without better communication on the relationship between the costs and the associated capabilities delivered for F-35 sustainment support, the military services may not be able to appropriately plan for sustainment costs over the life cycle of the F-35 or to make affordability trade-offs between requirements, as they try to prioritize funding within their budgets.

Conclusions

DOD’s F-35 program is at a critical juncture. With aircraft development nearing completion within the next few years, DOD must now shift its attention and resources to sustaining the growing F-35 fleet. While production accelerates, DOD’s reactive approach to planning for and funding the capabilities needed to sustain the F-35 has resulted in significant readiness challenges—including multi-year delays in establishing repair capabilities and spare parts shortages. There is little doubt that the F-35 brings unique capabilities to the U.S. military, but without revising sustainment plans to include the key requirements and decision points needed to fully implement the F-35 sustainment strategy, and without aligned funding plans to meet those requirements, DOD is at risk of being unable to leverage the capabilities of the aircraft it has recently purchased. Furthermore, until it improves its plans, DOD faces a larger uncertainty as to whether it can successfully sustain a rapidly expanding fleet.

DOD’s plan to enter into multi-year, performance-based contracts with the prime contractor has the potential to produce cost savings and other benefits. However, important lessons are emerging from its pilot agreements with the contractor that are intended to inform the upcoming multi-year contract negotiations. To date, DOD has not achieved the desired aircraft performance under the pilot agreements, but it continues to move quickly toward negotiating longer-term contracts—which are likely to cost tens of billions of dollars—by 2020. Without examining whether it has the appropriate metrics to incentivize the contractor or a sufficient understanding of the actual costs and technical characteristics of the aircraft before entering into multi-year, performance-based contracts, DOD could find itself overpaying for sustainment support that is not sufficient to meet warfighter requirements.

Finally, on a broader level, DOD’s projected costs to sustain the F-35 fleet over its life cycle have risen since 2012 despite the department’s concerted efforts to reduce costs. Already the most expensive weapon system in DOD’s history, these rising costs are particularly concerning because the military services do not fully understand what they are paying for. This puts them in a precarious position as they consider critical trade-offs that might make F-35 sustainment more affordable. Without improving communication with the services to help them better understand how the sustainment costs they are being charged relate to the capabilities that they receive, the services may not be able to effectively budget for the F-35 over the long term.

Recommendations for Executive Action

We are making the following four recommendations to DOD.

The Under Secretary of Defense for Acquisition, Technology, and Logistics, in coordination with the F-35 Program Executive Officer, should revise sustainment plans to ensure that they include the key requirements and decision points needed to fully implement the F-35 sustainment strategy and aligned funding plans to meet those requirements. (Recommendation 1)

The Under Secretary of Defense for Acquisition, Technology, and Logistics, in coordination with the F-35 Program Executive Officer, should re-examine the metrics that it will use to hold the contractor accountable under the fixed-price, performance-based contracts to ensure that such metrics are objectively measurable, are fully reflective of processes over which the contractor has control, and drive desired behaviors by all stakeholders. (Recommendation 2)

The Under Secretary of Defense for Acquisition, Technology, and Logistics, in coordination with the F-35 Program Executive Officer, should, prior to entering into multi-year, fixed-price, performance- based contracts, ensure that DOD has sufficient knowledge of the actual costs of sustainment and technical characteristics of the aircraft after baseline development is complete and the system reaches maturity. (Recommendation 3)

The Under Secretary of Defense for Acquisition, Technology, and Logistics, in coordination with the F-35 Program Executive Officer, should take steps to improve communication with the services and provide more information about how the F-35 sustainment costs they are being charged relate to the capabilities received. (Recommendation 4)

Agency Comments

We provided a draft of this report to DOD for review and comment. In its written comments, reproduced in appendix II, DOD concurred with our recommendations and identified actions that it would take in response.

We are sending copies of this report to appropriate congressional committees; the Secretary of Defense; the Under Secretary of Defense for Acquisition, Technology, and Logistics; the F-35 Program Executive Officer; the Secretaries of the Air Force and Navy; and the Commandant of the Marine Corps. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-5431 or russellc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Staff members making key contributions to this report are listed in appendix III.

Appendix I: Scope and Methodology

For each of our objectives, we reviewed relevant sustainment plans, guidance, and program documentation, and collected information by interviewing officials from the Office of the Assistant Secretary of Defense (Logistics and Materiel Readiness), the F-35 Joint Program Office, the U.S. Air Force, the U.S. Navy, the U.S. Marine Corps, and the prime contractor, Lockheed Martin. To interview officials and observe F-35 operations, maintenance, and training, we conducted visits to two F-35 operational locations—Hill Air Force Base, Utah, and Marine Corps Air Station Iwakuni, Japan; two F-35 training locations—Eglin Air Force Base, Florida, which also includes a Navy F-35 training squadron, and Marine Corps Air Station Beaufort, South Carolina; and two F-35 maintenance depots—Ogden Air Logistics Complex, Utah, and Fleet Readiness Center Southeast, Florida. A full listing of organizations with whom we met is provided later in this appendix.

We also gathered various data related to F-35 sustainment, such as supply chain and repair data and aircraft performance data. To determine the reliability of these data, we collected information on how the data were collected, managed, and used through a questionnaire and interviews with cognizant Department of Defense (DOD) officials and the prime contractor. In our assessment, we identified some limitations in the way that certain data are collected and reported, such as data related to aircraft performance (Air Vehicle Availability, Full Mission Capable, and Mission Effectiveness metrics), data related to aircraft that are not mission capable due to supply issues, and parts cannibalization rates that could potentially result in inaccuracies. However, these data come from the program’s data systems of record, and are the same data used by the program office and prime contractor to monitor the health of the supply chain and assess aircraft performance against contract requirements and program objectives. As such, they are the best source of data available to provide information on the progress and challenges within the program. We determined that these data presented in our findings are sufficiently reliable for the way in which we report them. Specifically, the parts cannibalization rates are consistent with the trends observed across other key data elements within the program, and with the testimonial evidence provided to us by the units with whom we met during our review, and are sufficiently reliable to report as a data trend relative to program objectives. All other performance data presented in our report are sufficiently reliable to present as specific data points, in order to describe the status of sustainment requirements and measured aircraft performance across key metrics as reported by the prime contractor and DOD.

To assess the status of DOD’s efforts to sustain the F-35 fleet and any challenges it has faced, we reviewed DOD and contractor plans, briefings, and schedules to determine the current status of key requirements and decision points necessary to establish F-35 sustainment capabilities, such as depot and other maintenance capabilities, the supply chain, technical data, and development of key software systems, among other things, and spoke with cognizant officials about these issues. We also compared actual data obtained about F-35 repair and supply chain capabilities with DOD’s objectives for these capabilities to identify areas of challenge for the program. Specifically, we obtained data on aircraft that were not mission capable due to supply issues from January 2017 through August 7, 2017 and average repair times as of May 2017, in order to provide the most recently available information about the health of the supply chain. As discussed above, we determined that these data are sufficiently reliable to present as specific data points. In addition, we identified key acquisition program management practices that can improve program outcomes if implemented—such as clearly establishing well-defined requirements, developing realistic cost estimates and schedules, and securing stable funding that matches resources to requirements—and assessed DOD’s sustainment planning efforts against these criteria.

To assess the extent to which DOD is positioned to enter into multi-year performance-based F-35 sustainment contracts, we reviewed documentation related to DOD’s pilot—or trial— performance-based agreements for F-35 sustainment, such as sustainment contracts, readiness data provided by the military services, metric taxonomies, and agreements between the program office and the military services that identify the services’ desired performance targets. We also obtained aircraft performance data from the Sustainment Performance Management System for the 2016 pilot performance-based agreement (March 2016 – December 2016) and the 2017 pilot performance-based agreement (March 2017 – June 2017) to the extent available at the time we completed our audit work. As discussed above, we determined that these data are sufficiently reliable to present as specific data points. These time periods are the only time periods for which the program office has assessed contractor performance under these pilot arrangements. We also reviewed performance-management guidance and processes and interviewed officials to determine how performance data are being collected and assessed. In addition, we reviewed aircraft maturity, reliability, and maintainability data, and documentation related to cost- visibility issues, and we spoke with cognizant officials about these issues to determine DOD’s level of understanding of the costs and technical characteristics that will affect future sustainment support. In addition, we obtained cannibalization data from March 2016 to March 2017 in order to review and report recent trends in cannibalization rates over a time in which the program has introduced a significant amount of aircraft to the fleet. As discussed above, we determined that these data are sufficiently reliable to present as trend data relative to the program objective. Further, we reviewed DOD guidance and best practices related to performance- based agreements to identify attributes of ideal performance metrics and effective performance-based agreements. We then compared these attributes with the information described above to determine whether DOD has the appropriate metrics to achieve desired outcomes and the necessary information to effectively negotiate multi-year, fixed-price, performance-based contracts with the prime contractor by 2020, as planned.

To assess the progress, if any, DOD has made toward reducing F-35 sustainment costs, and the extent to which costs are transparent to the military services, we reviewed F-35 Joint Program Office sustainment- cost estimates from fiscal year 2012 to fiscal year 2016 in order to identify changes to the estimate since the program’s sustainment cost baseline was established in 2012; documentation related to program office and service cost-reduction efforts; sustainment contracts; F-35 cost-sharing rules; and budget documentation from both the program office and the military services. The fiscal year 2016 sustainment-cost estimate is the most current cost estimate conducted by the program office. In addition, we interviewed cognizant officials from the F-35 Joint Program Office and military services to discuss how the program office informs the military services of F-35 sustainment costs, and the degree to which the services understand these costs and the sustainment capabilities provided for those costs. We compared this information with program guidance and with key operating principles for programs that involve multiple governmental customers identified in our prior work in order to assess the transparency of F-35 sustainment costs for the military services.

We conducted this performance audit from October 2016 to October 2017 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

We met with officials from the following Department of Defense (DOD) and contractor organizations during our review. We selected these organizations based on their oversight, planning, and execution roles in support of F-35 sustainment and operations.

DOD Organizations

Other Organizations

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Alissa Czyz (Assistant Director), Vincent Buquicchio, Kasea Hamar, Jeff Hubbard, Amie Lesser, Sean Manzano, Carol Petersen, Clarice Ransom, Michael Silver, Maria Staunton, Cheryl Weissman, and Delia Zee made key contributions to this report.

Related GAO Products

F-35 Joint Strike Fighter: DOD’s Proposed Follow-on Modernization Acquisition Strategy Reflects an Incremental Approach Although Plans Are Not Yet Finalized. GAO-17-690R. Washington, D.C.: August 8, 2017.

F-35 Joint Strike Fighter: DOD Needs to Complete Developmental Testing Before Making Significant New Investments. GAO-17-351. Washington, D.C.: April 24, 2017.

F-35 Joint Strike Fighter: Continued Oversight Needed as Program Plans to Begin Development of New Capabilities. GAO-16-390. Washington, D.C.: April 14, 2016.

F-35 Sustainment: DOD Needs a Plan to Address Risks Related to Its Central Logistics System. GAO-16-439. Washington, D.C.: April 14, 2016.

F-35 Joint Strike Fighter: Preliminary Observations on Program Progress. GAO-16-489T. Washington, D.C.: March 23, 2016.

F-35 Joint Strike Fighter: Assessment Needed to Address Affordability Challenges. GAO-15-364. Washington, D.C.: April 14, 2015.

F-35 Sustainment: Need for Affordable Strategy, Greater Attention to Risks, and Improved Cost Estimates. GAO-14-778. Washington, D.C.: September 23, 2014.

F-35 Joint Strike Fighter: Slower Than Expected Progress in Software Testing May Limit Initial Warfighting Capabilities. GAO-14-468T. Washington, D.C.: March 26, 2014.

F-35 Joint Strike Fighter: Problems Completing Software Testing May Hinder Delivery of Expected Warfighting Capabilities. GAO-14-322. Washington, D.C.: March 24, 2014.

F-35 Joint Strike Fighter: Restructuring Has Improved the Program, but Affordability Challenges and Other Risks Remain. GAO-13-690T. Washington, D.C.: June 19, 2013.

F-35 Joint Strike Fighter: Program Has Improved in Some Areas, but Affordability Challenges and Other Risks Remain. GAO-13-500T. Washington, D.C.: April 17, 2013.

F-35 Joint Strike Fighter: Current Outlook Is Improved, but Long-Term Affordability Is a Major Concern. GAO-13-309. Washington, D.C.: March 11, 2013.

Joint Strike Fighter: DOD Actions Needed to Further Enhance Restructuring and Address Affordability Risks. GAO-12-437. Washington, D.C.: June 14, 2012.

Joint Strike Fighter: Restructuring Added Resources and Reduced Risk, but Concurrency Is Still a Major Concern. GAO-12-525T. Washington, D.C.: March 20, 2012.

Joint Strike Fighter: Implications of Program Restructuring and Other Recent Developments on Key Aspects of DOD’s Prior Alternate Engine Analyses. GAO-11-903R. Washington, D.C.: September 14, 2011.

Joint Strike Fighter: Restructuring Places Program on Firmer Footing, but Progress Is Still Lagging. GAO-11-677T. Washington, D.C.: May 19, 2011.

Joint Strike Fighter: Restructuring Places Program on Firmer Footing, but Progress Still Lags. GAO-11-325. Washington, D.C.: April 7, 2011.

Joint Strike Fighter: Restructuring Should Improve Outcomes, but Progress Is Still Lagging Overall. GAO-11-450T. Washington, D.C.: March 15, 2011.