Tactical Aircraft: DOD Should Reconsider Decision to Increase F/A-22 Production Rates While Development Risks Continue
Released: 2003-03-14T00:00:00

Letter

Background

The F/A-22 is to be an air superiority and ground attack aircraft with advanced features to make it less detectable to adversaries (stealth characteristics) and capable of high speeds for long ranges. It is designed to have integrated avionics that greatly improve pilots’ awareness of the situation surrounding them. The objectives of the F/A-22 development program are to (1) design, fabricate, test, and deliver nine F/A-22 development test aircraft, two non-flying structural test aircraft, six production representative test aircraft, and 37 flight-qualified engines; (2) design, fabricate, integrate, and test the avionics; and (3) design, develop, and test the support and training systems. The F/A-22 is being developed under contracts with Lockheed Martin Corporation, the prime contractor (for the aircraft), and Pratt & Whitney Corporation (for the engine).

Following a history of increasing cost estimates to complete the development phase of the F/A-22 program, the National Defense Authorization Act for Fiscal Year 1998 established a cost limitation for both the development and production. Subsequently, the National Defense Authorization Act of 2002 eliminated the cost limitation for the development, but left the cost limit for production cost in place. The production program is now limited to $36.8 billion. The current cost estimate of the development program is $21.9 billion.

Currently, the F/A-22 program is in both development and production. Development is in its final stages, and low rate initial production has begun. Since fiscal year 1997, funds have been appropriated to acquire production aircraft, and the F/A-22 acquisition plan calls for steadily increasing annual production rates.

The aircraft’s development problems and schedule delays have caused congressional concerns, particularly in light of DOD’s planned increase in production rates. The National Defense Appropriations Act for Fiscal Year 2003 prohibited the obligation of funds for the acquisition of more than 16 production aircraft in fiscal year 2003, until the Under Secretary of Defense for Acquisition, Technology, and Logistics submits the following to the congressional defense committees: (1) a formal risk assessment that identifies and characterizes the potential cost, technical, schedule, or other significant risks resulting from increasing the F/A-22 production quantities prior to the Dedicated Initial Operational Test and Evaluation (DIOT&E) of the aircraft and (2) either a certification that increasing the F/A-22 production quantity for fiscal year 2003 beyond 16 aircraft involves lower risk and lower total program cost than staying at that quantity or implementing a revised production plan, funding, and test schedule. In December 2002, DOD submitted the risk assessment and certification to Congress.

F/A-22 Technical Problems Continue to Affect Performance

The F/A-22 developmental program did not meet key performance goals established for fiscal year 2002 and continues to confront numerous technical challenges. Major technical problems include instability of the avionics software, violent movement, or “buffeting,” of vertical fins, overheating in portions of the aircraft, weakening of materials in the horizontal tail, and the inability to meet airlift support and maintenance requirements. Modifications are being made to some test aircraft to address some of these problems in preparation for operational testing. Nevertheless, these problems continue to restrict the performance and testing of the F/A-22.

Avionics Instability

Software instability has hampered efforts to integrate advanced avionics capabilities into the F/A-22 system. Avionics control and integrated airborne electronics and sensors provide an increased awareness of the situation around the pilot. The Air Force told us that the avionics have failed or shut down during numerous tests of F/A-22 aircraft due to software problems. The shutdowns occur when the pilot attempts to use the radar, communication, navigation, identification, and electronic warfare systems concurrently. Although the plane can still be flown after the avionics have failed, the pilot is unable to successfully demonstrate the performance of the avionics. Therefore, the Air Force has had to extend the test program schedule.

The Air Force recognized that the avionics problems pose a high technical risk to the F/A-22 program, and in June 2002 the Air Force convened a special team to address the problem. According to the team, the unpredictable nature of the shutdowns was not surprising considering the complexity of the avionics system. The team recommended that the software be stabilized in the laboratory before releasing it to flight testing. The team further recommended conducting a stress test on the software system architecture to reduce problems and ensure that it is operating properly. The Air Force implemented these recommendations. Further, the Air Force extended the avionics schedule to accommodate avionics stability testing and now plans to complete avionics testing in the first quarter of 2005. However, Air Force officials stated that they do not yet understand the problems associated with the instability of the avionics software well enough to predict when they will be able to resolve this problem.

Vertical Fin Buffeting

Under some circumstances, the F/A-22 experiences violent movement, or buffeting, of the vertical fins in the tail section of the aircraft. This occurs as air, moving first over the body and the wings of the aircraft, places unequal pressures on the vertical fins and rudders. Unless the violent movement is resolved or the fins strengthened, the vertical fins will break over time because the pressures experienced exceed the strength limits of the fins. In addition, the buffeting problem has restricted the testing of aerial maneuvers of the aircraft.

Lockheed Martin has developed several modifications to strengthen the vertical fins and has performed an analysis to test the structural strength of the aircraft. It concluded that no flight restrictions above 10,000 feet are necessary as a result of buffeting. Currently, the Air Force has not begun testing to verify flight operations at or below 10,000 feet; operational limitations at altitudes below 10,000 feet remain in effect, with testing scheduled to begin in June 2003.

Overheating Concerns

Overheating in the rear portions of the aircraft has significantly restricted the duration of high-speed flight testing. As the F/A-22 flies, heat builds up inside several areas in of the rear of the aircraft. Continued exposure to high temperatures would weaken these parts of the aircraft. For example, a portion of the airframe that sits between the engines’ exhausts experiences the highest temperatures. This intense heat could weaken or damage the airframe. To prevent this heat buildup during flight testing, the aircraft is restricted to flying just over 500 miles per hour, about the same speed as a modern jet liner, and significantly below the supercruise requirement. Currently, the F/A-22 flies with temperature sensors in those areas of the aircraft, and it slows down whenever the temperature approaches a certain level. The Air Force may add copper sheets to the rear of the aircraft to alleviate the problem. The Air Force began these modifications in January 2003 and plans to complete them by July 2003.

Horizontal Tail Material Separations

F/A-22 aircraft have experienced separations of materials in the horizontal tail and the shaft, which allows the tail to pivot. Because the separations reduce tail strength, the Air Force restricted flight testing of some aircraft until it determined that this problem would not affect flight safety during testing. The Air Force and the contractor initially believed that improvements to the aircraft’s manufacturing process would solve this problem. However, the Air Force has determined that it could only solve this problem by redesigning the tail of the aircraft. The Air Force plans to conduct flight testing of the redesigned tail between February 2004 and April 2004.

Meeting Airlift Support Requirements

The Air Force estimates it will not meet the F/A-22 airlift support requirement despite last year’s estimate that it would meet all identified key performance parameters. (Appendix I contains a list of key performance parameters.) The airlift support requirement is that 8 C-141 aircraft or their equivalents would be sufficient to deploy a squadron of 24 F/A-22s for thirty days without resupply. Today the Air Force estimates that 8.8 C-141 equivalents will be necessary.

Impact of Maintenance Needs on Performance

The F/A-22’s performance may also be affected by maintenance needs that exceed established objectives. The Air Force estimates that the F/A-22 should, at this point in its development, be able to complete 1.67 flying hours between maintenance actions and 1.95 flying hours by the end of development. However, aircraft are requiring five times the maintenance actions expected at this point in development. As of November 2002, the development test aircraft have been completing only .29 flying hours between maintenance actions. Therefore, the development test aircraft are spending more time than planned on the ground undergoing maintenance.

In addition, the F/A-22 program has not completed the testing required to prove the aircraft can be maintained worldwide without unique support equipment. For example, the Air Force planned to fly the F/A-22 a minimum of 650 hours prior to the start of operational testing to establish that special support equipment is not necessary to maintain the materials on the exterior of the aircraft. These materials are critical to the aircraft’s low observable, or stealthy, nature. However, as of December 2002, the program has only accomplished 191.6 hours. According to the Air Force, the program will not complete testing for this requirement until the completion of the development program, currently planned for July 2004.

Modifications to Improve Performance

In 2002, the F/A-22 development program implemented several modifications to development aircraft to improve performance. The majority of modifications were related to installing the necessary upgrades to complete operational testing. The last three development test aircraft have required an average of 63 modifications. The first two production aircraft have required an average of 50 of these upgrades.

In addition, the program repaired problems in the aircraft’s arresting gear system that were discovered during development testing. Further, the Air Force has scheduled modifications to address the previously cited problems found with the vertical tail of the aircraft (fin-buffeting). The Air Force included these repairs in its 2002 modification schedule, but did not begin them in 2002. The modifications will begin during fiscal year 2003.

Flight Test Schedules Have Been Extended and May Slip Further

Progress in F/A-22 flight testing was slower than expected in 2002 in all test areas, according to Office of the Secretary of Defense (OSD) testing officials. Consequently, the Air Force extended flight test schedules and reduced the number of flight tests. Many tasks originally planned for 2002 were rescheduled for 2003. Further, the Air Force now plans to conduct more developmental flight testing concurrently with operational testing.

Continuing technical problems were the primary reasons for the delays in flight testing. In addition, late delivery of development aircraft to the flight test center was a contributing problem; three developmental aircraft were delivered from 9 to 12 months late. Late deliveries were due not only to technical problems, but also to continuing problems associated with the manufacture and assembly of development aircraft by the prime contractor.

With the new schedule, the Air Force delayed the beginning of operational testing for 4 months, until the portion of developmental testing required to begin operational testing could be completed. Operational testing is now planned to begin in August 2003. Figure 1 and table 1 show the changes in the FA/-22 flight test schedules.

However, according to OSD officials involved in operational testing, there is a high risk of not completing an adequate amount of development flight testing before operational testing is scheduled to begin. Indeed, we believe that it is unlikely that the Air Force will be able to complete all necessary avionics flight testing prior to the planned start of operational testing. Based on F/A-22 flight test accomplishment data and current flight test plans, we project that the start of operational testing might be delayed until January 2004. As a result, operational testing could be delayed by several months beyond the current planned date of August 2003.

Development and Modernization Costs Have Increased

In December 2002, the Air Force estimated that development costs had increased by $876 million, bringing total development costs to $21.9 billion. This increase was due to the technical problems and schedule delays discussed earlier.

In addition, since fiscal year 2001, there have been dramatic increases in planned funding for modernization upgrades that enhance the operational capabilities of the F/A-22, as shown in figure 2. Currently, the Air Force has almost $3.0 billion in funding for modernization projects, which it plans to spend through fiscal year 2009. Most of the recent increase in modernization funding is necessary to provide increased ground attack capability. Other modernization projects include upgrading avionics software, adding an improved short-range missile capability, upgrading instrumentation for testing, and incorporating a classified project.

In December 2002, in response to the increase in development costs, the Under Secretary of Defense, Comptroller, approved the restructuring of the F/A-22 program. According to the Comptroller, the cost increase will not require increased funds from Congress. Rather, the estimated $876 million increase for development will be met by a $763 million decrease in production funding and a transfer of $113 million from modernization funds. This restructure eliminates 27 aircraft from the current production program, reducing the total number of aircraft to be acquired from 303 to 276.

Risks in the F/A-22 Acquisition Plan

Despite continuing development problems and challenges, the Air Force plans to continue acquiring production aircraft at increasing annual rates. This is a very risky strategy, because, as we have previously reported, the Air Force may encounter higher production costs as a result of acquiring significant quantities of aircraft before adequate testing. Late testing could identify problems that require costly modifications in order to achieve satisfactory performance.

For example, as shown in figure 3, the Air Force plans to acquire 20 aircraft during 2003, rather than the maximum of 16 Congress allowed without DOD’s submittal of a risk assessment and certification. DOD justified this strategy in the December 2002 risk assessment and certification it submitted to Congress. In this document, DOD certified that acquiring more than 16 aircraft involved lower risk and lower total program cost than acquiring only 16. DOD identified the costs associated with acquiring more than 16 aircraft per year as between $7 million and $221 million, depending on the number of aircraft in excess of 16. DOD concluded that this additional cost would be less than the potential cost of modifying production aircraft once operational testing has been completed. Figure 3 shows the Air Force’s acquisition plan.

However, DOD’s risk assessment may be overly optimistic because it is grounded in the conclusion that there is a low risk that remaining development and operational testing will identify needs for expensive modifications. The performance capabilities of the F/A-22 and its schedule will remain uncertain until technical problems have been addressed, including testing of modifications or fixes necessary to potentially alleviate these problems. Furthermore, we believe that the amount of development and operational testing and the remaining uncertainties increase the possibility that modifications considered unlikely in DOD’s analysis will, indeed, need to be made. For example, the Air Force has still not completely defined the fin-buffet problem described earlier in this report. The remaining 15 percent of flight testing to help characterize the problem is not scheduled to begin until June 2003. Consequently, there is still the possibility that additional modifications and costs may be necessary to correct this problem on production aircraft. DOD’s risk assessment acknowledges that additional fin buffet testing is needed, but concludes that modifications are not expected.

The optimism of DOD’s risk assessment is reflected in the Air Force’s general acquisition strategy. As also shown by figure 3, the Air Force is currently committed to acquiring 73 production aircraft (26 percent) before operational and development testing is complete. We believe that— like the fiscal year 2003 decision to acquire more than 16 aircraft—this is an overly optimistic strategy given the remaining F/A-22 technical problems and the current status of testing. As we have noted, acquiring aircraft before completing adequate testing to resolve significant technical problems increases the risk of costly modifications later. If F/A-22 testing schedules slip further—as we believe is likely—even more aircraft will be acquired before development and operational testing is complete, and the risk of costly modifications will increase still more.

Conclusions

Continuing the acquisition of aircraft in increasing quantities when significant development testing and technical problems remain is an acquisition strategy that relies on overly optimistic assumptions regarding the outcome and timing of the remaining testing events. By employing such a strategy, major problems are more likely to be discovered after production has begun when it is either too late or very costly to correct them. At the very least, key decisions are being made without adequate information about the weapon system’s demonstrated operational test results. In its certification, DOD quantified the estimated costs associated with a higher production rate. However, the potential advantage was predicated on the assumption that the risks of modifications are low. As we stated last year, by limiting F/A-22 production quantities and completing development testing, the Air Force could gain information that would reduce uncertainties and the risks of increased costs and delays before committing to additional production aircraft. As we discussed earlier in this report, DOD recently decided to reduce production quantities as part of a program restructure to address F/A-22 development problems and associated cost increases. Based on uncertainties about the resolution of problems found in the past year, we continue to maintain the position that production quantities should be limited.

Recommendations for Executive Action

In light of continued uncertainties regarding the resolution of problems found in the past year and notwithstanding the December 2, 2002 certification provided by DOD, we recommend that the Secretary of Defense reconsider the Department’s decision to increase the annual production rate beyond 16 aircraft until greater knowledge on any need for modifications is established through completion of operational testing, and update the 2002 risk assessment and certification with sufficient detail to allow for verification of the conclusions following the completion of operational testing.

Agency Comments and Our Evaluation

In written comments on a draft of this report, DOD stated that it agreed, for the most part, with our description of the current state of the F/A-22 program’s content, schedule, and cost. However, DOD did not concur with our recommendation that it not increase its production rate beyond the maximum of 16 aircraft Congress allowed without DOD submitting a risk assessment and certification. DOD said that our recommendation does not sufficiently account for the costs of termination associated with the approval given to funding long-lead items, the manufacturing inefficiencies associated with a reduction in aircraft quantities, or the effects of inflation on the cost of acquiring aircraft at a lower rate. DOD also noted that we had not provided a quantitative assessment to justify limiting production, and it reiterated its reliance on the risk assessment and certification it submitted to Congress in December 2002. DOD also asserted, incorrectly, that our report concludes that minimal cost risk would be realized by slowing production.

Following review of DOD’s comments, we clarified the recommendation in our draft report by establishing two recommendations. These recommendations are based on the current state of the program— including the challenges and risks it faces—and on our examination of DOD’s risk assessment and certification. DOD acknowledges the challenges faced by the program but believes the risk of modification is low. As we discussed in this report, until testing has been completed and technical problems have been addressed, the performance capabilities of the F/A-22 and its schedule will remain uncertain; thus, it is not possible to predict that expensive modifications will not be required. For example, as we stated earlier in this report, DOD’s risk assessment concludes that significant costs associated with a more extensive modification to resolve the fin buffet problem may be required, but the probability is low. DOD arrives at this conclusion even though the last phase of testing to help characterize the fin buffet problem has not yet begun. Furthermore, we continue to believe there is still significant risk that the F/A-22 program will not be able to begin operational testing as scheduled in August 2003. Subsequent to our providing the draft of this report to DOD for comment, OSD’s operational test and evaluation office issued a report stating that F/A-22 technical and schedule risk are still high, as is the risk that operational testing will be further delayed.

While DOD’s December 2002 risk assessment and certification did provide an indication that manufacturing inefficiencies and inflation as a result of lower production rates would increase costs, sufficient detail was not provided in its risk assessment for us to verify DOD’s conclusion. We requested additional detailed information to help us evaluate and verify the conclusions. However, the information provided to us was not adequate to verify the conclusions contained in the risk assessment. Regardless, even with such verification, still needing to be resolved are the uncertainties to date regarding when development problems can be fixed and the possibility of finding additional problems prior to the completion of operational testing. As a result, we have little confidence that existing problems can be quickly resolved and will not result in further delays. Our work has shown that continuing the acquisition of aircraft in increasing quantities when significant development testing and technical problems remain is risky. By employing such a strategy, major problems are more likely to be discovered after the program has begun production when it is either too late or very costly to correct them.

DOD also provided various technical comments, which we have incorporated as appropriate. One of these comments related to the total number of production aircraft to be acquired. The projected number of production aircraft the Air Force plans to or can actually acquire has historically been fluid and elusive. For example, the President’s budget for fiscal year 2003 reflected plans to acquire 333 production aircraft, even though the approved program at the time called for acquiring 295 production aircraft. In its technical comments, DOD stated that the approved program plan is to acquire 295 aircraft. As a result of the recent F/A-22 restructuring to cover development cost increases, the Air Force says that it now plans to acquire 276 aircraft. However, DOD estimates that the cost of production to acquire these 276 aircraft will be $42.2 billion, which exceeds the current production cost limit by $5.4 billion. Consequently, unless the production cost limit is raised or substantial cost reduction plans are achieved, it appears that the number of aircraft that can actually be purchased will have to be lowered from the 276 planned. This is particularly true if production or development costs—or both—continue to rise and no additional funds are provided by the Congress.

Last month, we recommended in another report that DOD provide Congress with documentation reflecting the quantity of aircraft that DOD believes can be procured within the existing production cost limit. DOD’s explanation in its technical comments to a draft of this report identifies the likelihood that F/A-22 aircraft quantities will continue to fluctuate. This makes our recent recommendation that much more compelling.

Scope and Methodology

To determine whether the development program is likely to meet performance goals, we analyzed information on the status of key performance parameters. We compared performance goals established by the Under Secretary of Defense for Acquisition, Technology, and Logistics with the Air Force’s estimates of performance for completion of development made in December 2002.

To identify the status of F/A-22 modifications, we collected updated information on the status of existing aircraft structural problems that have required aircraft modifications. To determine whether the program is expected to meet schedule goals, we reviewed program and avionics schedules and discussed potential changes to these schedules with F/A-22 program officials. We tracked progress in the flight test program and evaluated schedule variances in the contractors’ performance management system and compared planned milestone accomplishment dates with actual dates. We tracked technical problems in manufacturing and assembling the development test aircraft.

To determine whether the program is likely to meet the cost goal, we examined (1) the extent to which the development program is likely to be completed within the current cost estimate, (2) the Air Force’s plans to fund the program for fiscal year 2003, and (3) the program’s funding plan compared to the current cost estimate.

In examining DOD’s risk assessment, we discussed the various DOD assumptions and approaches used in the assessment with a program official who conducted the assessment. We then analyzed the various DOD assumptions and approaches used to make the assessment conclusions.

In making these determinations, assessments, and identifications, we required access to current information about test results, performance estimates, schedule achievements and revisions, costs being incurred, aircraft modifications, and the program’s plans for continued development and initial production. The Air Force and contractors gave us access to sufficient information to make informed judgments on the matters covered in this report.

In performing our work, we obtained information or interviewed officials from the Office of the Secretary of Defense, Washington, D.C., and the F/A-22 System Program Office, Wright-Patterson Air Force Base, Ohio. We performed our work from September 2002 through December 2002 in accordance with generally accepted government auditing standards.

We are sending copies of this report to interested congressional committees; the Secretary of Defense; the Secretary of the Air Force; and the Director, Office of Management and Budget. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. Please contact me at (202) 512-4841 or Catherine Baltzell at (202) 512-8001 if you or your staff have any questions concerning this report. Major contributors to this report are listed in appendix III.

Appendix I: Estimates of Performance for Key Parameters

Combat radius sub + super Radar detection range Independent airlift support (C-141 equivalents) Sortie generation rate Average flight test hours between maintenance Interoperability

Margin Favorable 12% favorable 3% favorable 0 NA 15% favorable 5% favorable (0.8) unfavorable 0 0 0 The acceleration parameter is a measure of the time it takes the aircraft to increase speed to a certain level. If the aircraft is able to increase speed to a certain level in less time than expected, this is considered favorable. Therefore, a measure of less than 100 percent is favorable.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Staff Acknowledgments

Acknowledgments

Catherine Baltzell, Marvin E. Bonner, Edward Browning, Gary Middleton, Sameena Nooruddin, Madhav Panwar, Karen A. Richey, Don M. Springman, and Ralph White made key contributions to this report.

Related GAO Products

Tactical Aircraft: F-22 Delays Indicate Initial Production Rates Should Be Lower to Reduce Risks. GAO-02-298. Washington, D.C.: March 5, 2002.

Tactical Aircraft: Continuing Difficulty Keeping F-22 Production Costs Within the Congressional Limitation. GAO-01-782. Washington, D.C.: July 16, 2001.

Tactical Aircraft: F-22 Development and Testing Delays Indicate Need for Limit on Low-Rate Production. GAO-01-310. Washington, D.C.: March 15, 2001.

Defense Acquisitions: Recent F-22 Production Cost Estimates Exceeded Congressional Limitation. GAO/NSIAD-00-178. Washington, D.C.: August 15, 2000.

Defense Acquisitions: Use of Cost Reduction Plans in Estimating F-22 Total Production Costs. GAO/T-NSIAD-00-200. Washington, D.C.: June 15, 2000.

Budget Issues: Budgetary Implications of Selected GAO Work for Fiscal Year 2001. GAO/OCG-00-8. Washington, D.C.: March 31, 2000.

F-22 Aircraft: Development Cost Goal Achievable If Major Problems Are Avoided. GAO/NSIAD-00-68. Washington, D.C.: March 14, 2000.

Defense Acquisitions: Progress in Meeting F-22 Cost and Schedule Goals. GAO/T-NSIAD-00-58. Washington, D.C.: December 7, 1999.

Fiscal Year 2000 Budget: DOD’s Production and RDT&E Programs. GAO/NSIAD-99-233R. Washington, D.C.: September 23, 1999.

Budget Issues: Budgetary Implications of Selected GAO Work for Fiscal Year 2000. GAO/OCG-99-26. Washington, D.C.: April 16, 1999.

Defense Acquisitions: Progress of the F-22 and F/A-18E/F Engineering and Manufacturing Development Programs. GAO/T-NSIAD-99-113. Washington, D.C.: March 17, 1999.

F-22 Aircraft: Issues in Achieving Engineering and Manufacturing Development Goals. GAO/NSIAD-99-55. Washington, D.C.: March 15, 1999.

F-22 Aircraft: Progress of the Engineering and Manufacturing Development Program. GAO/T-NSIAD-98-137. Washington, D.C.: March 25, 1998.

F-22 Aircraft: Progress in Achieving Engineering and Manufacturing Development Goals. GAO/NSIAD-98-67. Washington, D.C.: March 10, 1998.

Tactical Aircraft: Restructuring of the Air Force F-22 Fighter Program. GAO/NSIAD-97-156. Washington, D.C.: June 4, 1997.

Defense Aircraft Investments: Major Program Commitments Based on Optimistic Budget Projections. GAO/T-NSIAD-97-103. Washington, D.C.: March 5, 1997.

F-22 Restructuring. GAO/NSIAD-97-100BR. Washington, D.C.: February 28, 1997.

Tactical Aircraft: Concurrency in Development and Production of F-22 Aircraft Should Be Reduced. GAO/NSIAD-95-59. Washington, D.C.: April 19, 1995.

Tactical Aircraft: F-15 Replacement Issues. GAO/T-NSIAD-94-176. Washington, D.C.: May 5, 1994.

Tactical Aircraft: F-15 Replacement Is Premature as Currently Planned. GAO/NSIAD-94-118. Washington, D.C.: March 25, 1994.