Tactical Aircraft: Status of the F/A-22 and Joint Strike Fighter Programs
Released: 2004-03-25T00:00:00

Letter

Background

The Air Force began the F/A-22 development program in 1986 and expected to complete development in 9 years for an estimated cost of $12.6 billion. Today, after being in development for almost two decades, the estimated development cost is $28.7 billion, a 127 percent increase. The average unit procurement cost to buy the F/A-22 has also increased 122 percent. The result of these changes has been a loss of buying power that has reduced the initial buy quantity from 750 to 277 aircraft. Table 1 shows the changes in the development program from 1986 to 2002.

We have reported in the past that the F/A-22 acquisition approach was a major contributor to the cost increases and delays in schedule that led to reduced buying power. In testimony last year, we identified lessons to be learned in the F/A-22 program, which did not follow a knowledge-based acquisition approach used by successful commercial firms. Leading commercial firms that we studied employ an acquisition approach that evolves a product to its ultimate capabilities on the basis of mature technologies and available resources. These firms then ensure that high levels of knowledge exist at three critical junctures in a development program. First, a match must be made between a customer’s needs and the available resources—technology, engineering knowledge, time, and funding—before a new development program is launched. Second, a product’s design must demonstrate its ability to meet performance requirements and be stable about midway through development. Third, the developer must show that the product can be manufactured within cost, schedule, and quality targets and is demonstrated to be reliable before production begins. DOD issued new acquisition policy in May 2003 that governs the development of major acquisition systems. This new policy embraces the best practice concepts of knowledge-based, evolutionary acquisition and represents a good first step toward achieving better outcomes from major acquisition programs.

The initial F-22 acquisition strategy did not employ an evolutionary approach. Instead, it sought to develop revolutionary capabilities from the outset of the program taking on significant risk and onerous technology challenges. Three critical technologies were immature at the start of the program—low-observable materials, propulsion, and integrated avionics. Integrated avionics has been a source of major schedule delays and cost increases in the F/A-22 program. Starting the program with these immature technologies prevented the program from knowing cost, schedule, and performance ramifications until late in the development program, after significant investments had already been made. Efforts to mature technology cascaded into development, delaying attainment of design and production maturity.

The JSF, which started in 1996, is not as far along in its development, but is experiencing problems that could similarly threaten DOD’s investment. It is at a critical crossroad, one that, based on our prior work, was approached and passed by several other DOD programs without capturing the appropriate knowledge for the sizable investment decisions being made. While the JSF program started with higher risks by failing to mature its technologies, it is considering a delay to its investment decision that determines the need to invest in tooling, labor, and facilities to manufacture aircraft until the airframe design has become more stable.

Significant Changes Require Additional Investments to Expand F/A-22 Capability

The basic mission of the F/A-22, initially focused on air-to-air dominance, has changed to include a significantly greater emphasis on attacking ground targets. To accomplish this expanded mission, the Air Force will need additional investments to develop and expand air-to-ground attack capabilities for the F/A-22. Moreover, the efforts to expand its capability will also add risks to an already challenged program. To accommodate planned changes will also require a new computer architecture and processor to replace the current less capable ones.

The expanded air-to-ground attack capability will allow the F/A-22 to engage a greater variety of ground targets, such as surface-to-air missile systems, that have posed a significant threat to U.S. aircraft in recent years. This was not previously considered a primary role for the aircraft as it was intended to be primarily an air-to-air fighter to replace the F-15. From the outset the F/A-22 was built to counter expected large numbers of new advanced Soviet fighter aircraft, but this expected threat never materialized.

The Air Force has a modernization program to improve the capabilities of the F/A-22 focused largely on a more robust air-to-ground capability. It intends to do so using five developmental spirals planned over more than a 10-year period, with the initial spiral started in 2003. In March 2003, the Office of Secretary of Defense’s Cost Analysis Improvement Group (CAIG) estimated that the Air Force would need $11.7 billion for the planned modernization program. The CAIG estimate included costs for development, production, and the retrofit of some aircraft. As of March 2003, the Air Force F/A-22 approved program baseline did not include estimated costs for the full modernization effort. Instead, the Air Force estimate included $3.5 billion for modernization efforts planned through fiscal year 2009. Table 2 shows each spiral as currently planned.

To complete the planned enhancements, the F/A-22 will also need a new computer architecture and avionics processors. Current architecture and processors will be upgraded to support enhancement through the Global Strike Enhanced development spiral. However, because the current architecture and processors are old and obsolete and do not have sufficient capacity to meet the increased processing demands required for planned new air-to-ground capabilities beyond the Global Strike Enhanced spiral, they must be replaced.

Rather than start a new development program, the F/A-22 program office plans to leverage two other ongoing Air Force development or modification programs for this new processing capability: the new architecture being developed for the JSF and the new commercial off-the-shelf general-purpose processors designed for newer versions of the F-16. According to F/A-22 program officials, they do not expect the new architecture to be fully developed and ready for installation in the F/A-22 for at least 5 to 6 years.

Additional risks are likely because the new processor and architecture are being developed by other major aircraft programs and will require extensive integration and operational testing to ensure that the F/A-22 program does not encounter similar problems that have delayed integration and testing of the F/A-22’s current avionics suite. F/A-22 program officials acknowledge that this mass changeover of the F/A-22 computer architecture and avionics processor will be a time-consuming and costly effort and will likely create additional program risks. Air Force cost estimates are not yet available, but program officials estimate the nonrecurring engineering costs alone could be at least $300 million. At the time of our review, the Air Force had not made a decision about retrofitting aircraft equipped with the old microprocessor.

Further Delays in Initial Operational Testing Could Impact Planned Full Rate Production Decision

The Air Force schedule includes plans to make the full rate production decision in December 2004, but initial operational test and evaluation (IOT&E) has not started. The Air Force’s efforts to stabilize avionics software and improve its performance have not been sufficiently demonstrated to start IOT&E, and the planned entrance criterion was changed. In addition, the F/A-22 program is not performing as expected in some other key performance areas like system reliability. These problems have contributed to the need for a new test schedule and an additional 7- month delay in the start IOT&E. Together these problems increase the potential for additional development costs and delays in the full rate production decision. Since our report in March 2003, the Air Force has corrected some key design problems identified at that time, but others remain.

Avionics Do Not Meet Criterion to Start Operational Testing

The stability and performance of F/A-22 avionics has been a major problem causing delays in the completion of developmental testing and the start of IOT&E. Because the F/A-22 avionics encountered frequent shutdowns over the last few years, many test flights were delayed. As a result, the Air Force Operational Test and Evaluation Center wanted assurances that the avionics would work before it was willing to start the IOT&E program. It established a requirement for a 20-hour performance metric that was to be demonstrated before IOT&E would begin. This metric was subsequently changed to a 5-hour metric that included additional types of failures, and it became the Defense Acquisition Board’s criterion to start IOT&E. In turn, Congress included the new metric, known as Mean Time Between Avionics Anomaly or MTBAA, in the National Defense Authorization Act for Fiscal Year 2004. As of January 2004, the Air Force had not been able to demonstrate that the avionics could meet either of these criteria.

Testing as of January 2004 showed the program had achieved 2.7 hours— 54 percent of the 5-hour stability requirement to begin IOT&E. While the Air Force has not been able to meet the new criteria, major failures, resulting in a complete shutdown of the avionics system, have significantly diminished. These failures are occurring only about once every 25 hours on average. This is the result of a substantial effort on the part of the Air Force and the contractor to identify and fix problems that led to the instability in the F/A-22 avionics software. However, less serious failures are still occurring frequently.

Reliability Requirements Not Being Met

The F/A-22 program is not meeting its requirements for a reliable aircraft, and it is not using a knowledge-based approach. The Air Force established reliability requirements to be achieved at the completion of development and at system maturity. As a measure of the system’s overall reliability, the Air Force established a requirement for 1.95-hours mean time between maintenance by the completion of development and 3-hours mean time between maintenance at system maturity. This measure of reliability represents the average flight time between maintenance actions. As of October 2003, the Air Force had only been able to demonstrate a reliability of about 0.5 flying hours between maintenance actions or about 26 percent of the development requirement and 17 percent of system maturity requirement. This has led to test aircraft spending more time than planned on the ground undergoing maintenance.

In addition to the high level of maintenance required, failures in F/A-22 parts and components also caused reliability problems. During 2003, the Air Force identified 68 parts that had a high rate of failure causing them to be removed or replaced and affecting the F/A-22 system reliability. The contractor has initiated programs to eliminate the high failure rates experienced by these parts. The canopy has also been experiencing failures during testing, allowing it to achieve only about 15 percent of its expected 1,600-hour life. The Air Force is considering using a second manufacturer for canopies, but until it has passed qualification testing, it cannot be used as an alternative source for the high-failing canopies.

The F/A-22 program began limited production before demonstrating reliability. Our work has shown that product development engineers from leading commercial firms expect to achieve reliability requirements before entering production. They told us reliability is attained through an iterative process of design, testing, analysis, and redesign. Commercial firms understand that once a system enters production, the costs to achieve reliability through this iterative design change process become significantly more expensive. The F/A-22 aircraft has been in production since fiscal year 1999, and the Air Force has on contract 52 production aircraft, and an additional 22 aircraft on long lead contracts representing 27 percent of the planned buy quantity. With 83 percent of the reliability requirement yet to be achieved through this iterative design change process, the Air Force can expect to incur additional development and design change costs. If the Air Force fails to improve the F/A-22’s reliability before fielding the aircraft, the high failure rates will result in higher operational and support costs to keep the aircraft available for training or combat use.

Operational Testing Delayed and Divided into Two Phases

Avionics and reliability problems were the major contributors to delays in F/A-22 flight-testing in 2003. As a result, the start of IOT&E was delayed an additional 7 months. Realizing the Air Force would not be ready to enter initial operational testing as previously planned, the Office of the Secretary of Defense requested the F/A-22 program to establish a new operational test plan that includes measures to ensure the aircraft and its avionics are ready before entering operational testing. In response, the Air Force put in place a two-phase operational test program.

Phase 1, also called an operational assessment, is not the official start of operational testing. It is intended to assess the F/A-22’s readiness for IOT&E. Started in October 2003, it calls for testing two F/A-22 aircraft.

Phase 2 testing is considered the actual start of IOT&E. To begin this phase, the Air Force must meet a number of criteria. Perhaps most importantly, it must demonstrate that the F/A-22’s integrated avionics will be able to operate for sufficient lengths of time, without shutting down. Figure 2 compares the changes in the planned test program since our March 2003 report.

According to Air Force test officials, results of some phase 1 tests could be used to satisfy IOT&E requirements if the aircraft and software configurations do not change for IOT&E testing. This could reduce the scope of the test effort planned during IOT&E. The Defense Acquisition Board is scheduled to review the F/A-22’s readiness for IOT&E later this month.

At the present time, the Air Force expects to complete IOT&E in October 2004, before the full rate production decision, now expected in December 2004. The time allotted to complete IOT&E under the new test plan, however, has been compressed by 4 months, assuming phase 1 testing results are not permitted to be used for IOT&E. This means the Air Force would have less time than previously planned to complete the same amount of testing. If the Air Force continues to experience delays in testing prior to IOT&E, then the full rate production decision would also have to be delayed until IOT&E is complete and the Beyond Low Rate Initial Production Report is delivered to Congress.

Past Design Problems Corrected

The Air Force has corrected design problems discussed in our March 2003 report. To correct the movement or buffeting of the vertical fins in the tail section of the aircraft, the Air Force designed and implemented modifications, which strengthen the fin and hinge assemblies. Because of this problem, the Air Force placed restrictions on flights below 10,000 feet. Testing was done above and below 10,000 feet, and the flight restrictions were removed. Likewise, the Air Force modified the aircraft to prevent overheating concerns in the rear portion of the aircraft by adding thermal protection and strengthened strategic areas in the aft tail sections. The Air Force also plans to modify later production aircraft using a new venting approach to resolve the heat problems. We reported that the Air Force had also experienced separations in the horizontal tail materials. After additional testing, the Air Force deemed that the original tails met requirements established for the life of the airframe. However, the Air Force redesigned the tail to reduce producibility costs. Tests will be performed on the redesigned tail in late 2004.

Business Case Information Did Not Justify Current Aircraft Quantities or Modernization Investment Plans

The business case made to justify the F/A-22 program at its outset is no longer valid. Since that time, program cost and schedule have grown substantially and affordable quantities have been reduced by 60 percent. The expected threat, for which this aircraft was originally designed, never materialized, and new, more demanding ground threats, like surface-to-air missile systems, have evolved, requiring expanded capabilities that will require significant new developmental investments. In addition, technical problems have not been resolved, and uncertainty about the outcome of operational testing could lead to additional development costs and further delays.

Today, the Air Force estimates the total F/A-22 acquisition program will cost about $72 billion, excluding about $8 billion estimated by the CAIG to complete modernization activities. Including these costs brings the estimated total investment for the F/A-22 program to about $80 billion. Through fiscal year 2004, about one-half of this investment has been funded, leaving key investment decisions in the near future on the remaining $40 billion for aircraft production and upgrades in capability.

Last year, in light of the changes in the program and investments that remained, the Subcommittee on National Security, Emerging Threats, and International Relations of the House Committee on Government Reform asked DOD to provide a new business case justifying the Air Force’s planned number of F/A-22s (276 at that time) as well as how many F/A-22s are affordable. In its response, DOD did not sufficiently address key business case questions such as how many F/A-22s are needed, how many are affordable, and if alternatives to planned investments increasing the F/A-22 air-to-ground capabilities exist.

Instead, DOD stated it planned to buy 277 F/A-22s based on a “buy to budget” concept that determines quantities on the availability and efficient use of funds by the F/A-22 program office. Furthermore, justification for expanding the capability to a more robust air-to-ground attack capability was not addressed in DOD’s response. While ground targets such as surface-to-air missile systems are acknowledged to be a significant threat today, the response did not establish a justification for this investment or state what alternatives were considered. For example, the JSF aircraft is also expected to have an air-to-ground role, as are planned future unmanned combat air vehicles. These could be viable alternatives to this additional investment in F/A-22 capability.

While the business case information submitted to the subcommittee called for 277 aircraft, DOD stated it could only afford to acquire between 216 and 218 aircraft within the congressionally imposed cap on production costs—currently at $36.8 billion. DOD expects improvements in manufacturing efficiencies and other areas will provide it with sufficient funds to buy additional F/A-22 aircraft. However, this seems to be an unlikely scenario given the program’s history. Previously, DOD, under its “buy to budget” approach, used $876 million mostly from production funds to cover increases in development costs, thus reducing aircraft quantities by 49. With testing still incomplete and many important performance areas not yet demonstrated, the possibility for additional increases in development costs is likely.

The analysis and conclusions in our recent report led us to recommend that DOD complete a new business case that justifies the need for the F/A-22 and that determines the quantities needed and affordable to carry out its air-to-air and air-to-ground mission. In preparing the business case, we also recommended DOD look at alternatives to the F/A-22 for dealing with the ground threats that were driving the need for an expanded air-to- ground capability. In response to a draft of that report, DOD partially concurred, stating that it evaluates the F/A-22 business case elements as part of the annual budget process. Additionally, DOD’s response acknowledged that this year the department is undertaking a broader set of reviews under the Joint Capabilities Review process and that the F/A-22 will be a part of that review. In our report, as part of the evaluation of DOD’s comments, we noted that an independent and in-depth study of the F/A-22 program has been requested by the Office of Management and Budget and that such a study provided an opportunity for completing a business case analysis.

JSF Joint Strike Fighter Program Is Approaching a Decision Point

The JSF acquisition program is approaching a key investment decision point in its development as it prepares to stabilize the design for its critical design reviews. The program has many demands and requirements to satisfy before it is completed. It is the most expensive acquisition program in DOD’s history with plans to buy almost 2,500 aircraft for an estimated acquisition cost of about $200 billion. The design plans are for three variants for the Air Force, Navy, and Marine Corps, with development partners and potential customers that span the globe. Upcoming investment decisions will be a prominent indicator of the risk program management and senior leadership will assume for this program. The program’s size—in terms of funding, number of aircraft, and program participants—will create challenges for decision makers over the next several years. They will face decisions that need to be guided by a sound business case and an evolutionary, knowledge-based acquisition process that will provide more predictable cost, schedule, and performance outcomes.

The JSF is a joint, multi-national acquisition program for the Air Force, Navy, Marine Corps, and eight cooperative international partners. The program’s objective is to develop and deploy an affordable weapon system that satisfies a variety of war fighters with different needs. The system is intended to consist of a family of highly common and affordable strike aircraft designed to meet an advanced threat and a logistics system to enable the JSF to be self-sufficient or part of a multisystem and multiservice operation. This family of strike aircraft will consist of three variants: conventional takeoff and landing, aircraft carrier suitable, and short take off and vertical landing. The JSF program began in November 1996. After a five-year competitive concept demonstration phase between Boeing and Lockheed-Martin, DOD awarded Lockheed-Martin a contract in October 2001 to begin system development and demonstration.

Current Status of the Joint Strike Fighter Program’s Cost, Schedule, and Funding

We are aware that program managers are contemplating changes to the program that could delay the schedule and increase costs, but confirmation and details are not yet available. Nonetheless, current program office estimates do provide some insights. Since the JSF acquisition program began in 1996, the cost of development has grown by about 80 percent. As shown in figure 3, the majority of this cost growth, from an estimated $24.8 to $34.4 billion, was recognized at the time the program transitioned from concept development to system development and demonstration in 2001. The program office cited schedule delays, implementation of a new block development approach that extended the program by 36 months, and a more mature cost estimate as the major causes for the increase. Since the start of system development and demonstration, the estimate has increased by an additional $10.3 billion because of continued efforts to achieve international commonality, optimize engine interchangeability, further refinements to the estimating methodology, and schedule delays for additional design work.

In both 2000 and 2001, when the program was making the critical decision to move into system development and demonstration, we reported and testified that technologies had not been sufficiently demonstrated to reduce risk to a level commensurate with a decision to commit major capital and time to product development. While some of these technologies continue to be troublesome, in March 2003, the program’s preliminary design review revealed significant issues related to aircraft design maturity.

Weight has become the most significant design risk for the program as it approaches its critical design review. The increased weight of each variant design could degrade aircraft range and maneuverability if not brought under control. According to the program office, the airframe design has matured more slowly than anticipated and software development and integration is posing a significant design challenge. Also, in a 2003 annual report, the Director of Operational Test & Evaluation stated that weight growth is a significant design risk for all the variants, that the development schedule is aggressive, and that efforts to reduce weight have eroded a large part of the schedule.

We also note that the program’s funding profile assumes almost $90 billion of funding over the next 10 years, an average of almost $9 billion a year. This will require the JSF program to compete with many other large programs for scarce funding during this same time frame. Sustaining this level of high funding for such a long period will be a challenge. The JSF program’s latest planned funding profile for development and procurement—as of December 2002—is shown in figure 4.

Knowledge-Based Decisions Are Needed for the Road Ahead

The JSF program faces critical decisions over the next 24 months. Decisions made today will greatly influence the efficiency of the rest of its funding—almost 90 percent of the total. As a result of current concerns over system integration risk, the program office is currently restructuring the development program, which will add significant cost and delay the development schedule. For example, it is considering delaying its critical design reviews, its first flights of development aircraft, and its limited rate production decision to allow more time to mitigate design risk and gather more knowledge before moving forward with continued major investments.

While no one wants to delay critical reviews, now is the time to get the design right rather than later. Going forward with an incomplete review may cause more problems later in the effort. Indeed, based on our past best practices work and lessons learned from the F/A-22 development effort, we have seen many examples where programs moved forward past their critical design review without gathering the knowledge needed to verify that their design was stable. This has led to poor cost and delivery outcomes for these programs. We have also seen the reverse, where programs have gathered appropriate knowledge before their critical design review. These programs had much more predictable cost and schedule outcomes.

The F/A-22 program held its critical design review in 1995 with only about 26 percent of its design drawings complete. Best practice criteria calls for 90 percent of drawings to be complete before a design can be considered stable enough to commit to additional significant investments of time, labor, material, and capital. Figure 5 shows the engineering drawing completion history of the F/A-22 along with changes to development cost estimates as the program progressed.

An incomplete F/A-22 critical design review contributed to several design and manufacturing problems that resulted in design changes, labor inefficiencies, cost increases, and schedule delays. Since the time of its critical design review, the F/A-22’s development costs have increased by about 50 percent. The JSF program has the opportunity to avoid a similar situation. We believe that, given the apparent design challenges at this point in the program, a delay to gather more knowledge before increasing the investment is warranted and may help to reduce turbulence later in development, before the program begins “bending metal” for development aircraft. The JSF program is at a pivotal point, one in which the effort will turn from a paper design to actually manufacturing a product, something that requires considerably more money. While we believe the program moved forward with too much technology risk up to this point, it has an opportunity now to achieve critical design knowledge by taking the time to develop a mature design before moving into manufacturing. The program can use lessons learned from the F/A-22 acquisition right now to keep on track and deliver an affordable, high quality weapon system sooner rather than later.

Additional Challenges Attendant with International Cooperation

The JSF program is based on a complex set of relationships among all three services and governments and industries from eight foreign partners. The program is expected to benefit the United States by reducing its share of development costs, increasing future aircraft sales, giving it access to foreign industrial capabilities, and improving interoperability among the services and allies. For their part, partner governments expect to benefit from relationships with U.S. aerospace companies, access to JSF program data, and influence over aircraft requirements. They will also benefit financially by obtaining waivers of nonrecurring aircraft costs on an aircraft they could otherwise not afford to develop on their own. The partners expect a return on their investment through JSF contract awards for their industries that will improve their defense industrial capability, a critical condition for their participation. They have agreed to contribute about $4.5 billion to the JSF development program and are expected to purchase several hundred aircraft once it enters production.

With these mutual benefits come challenges. Support for the program from our international partners hinges in large part on expectations for financial returns, technology transfer, and information sharing. If these expectations are not met, that support could deteriorate. In addition, a large number of export authorizations are needed to share information and execute contracts. These authorizations must be done in a timely manner to maintain schedule and ensure competition. Finally, transfer of sensitive U.S. military technologies needed to achieve commonality and interoperability goals will push the boundaries of U.S. disclosure policies.

Conclusions

DOD is not immune to efforts to address the fiscal imbalance confronting the nation and will continue to face challenges based on competing priorities, both within and external to its budget. This will require decisions based on a sound and sustainable business case for DOD’s acquisition programs based on clear priorities, comprehensive needs assessments, and a thorough analysis of available resources. In addition, it will require an acquisition process that provides for knowledge-based decisions at critical investment junctures in order to maximize available dollars. DOD has instituted a new acquisition policy that embraces evolutionary and knowledge-based acquisition concepts. However, policy alone will not solve the problems DOD faces. This will also require disciplined actions on the part of DOD’s leadership to employ the concepts established in its new policy.

While it is too late for the F/A-22 to go back and follow these concepts, there still is time to evaluate the need for additional aircraft; over fifty F/A-22’s are presently on contract. Because of the nation’s fiscal challenges, tough choices will need to be made regarding future spending priorities, including the remaining potential $40 billion investment in the F/A-22. In light of this substantial investment and the many changes that have occurred in the F/A-22 program, we believe decision makers would benefit from a new business case that justifies the need for the full air-to- air and air-to-ground capabilities and the quantities needed and affordable.

The JSF program has a greater opportunity to make critical investment decisions using a knowledge-based approach. While the program started off with a high-risk approach by not maturing technologies before starting system development, it has the opportunity to manage the system development phase and stabilize the design before committing to large investments in manufacturing capability—tooling, labor, and facilities—to build test aircraft. The JSF program is considering a delay in its critical design review to attain greater design stability in its airframe. In addition to seeking greater design stability, leadership in DOD can reap the benefits of its new acquisition policy by actively promoting and maintaining a disciplined approach throughout the remaining critical decision points over the next few years. With these activities in place, DOD will be in a better position to request continued JSF funding and support.

Mr. Chairman, this concludes my prepared statement. I would be happy to respond to any questions that you or other members of the Subcommittee may have. If you have future questions about our work on the F/A-22 or JSF, please call Allen Li at (202) 512-4841.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.