Tactical Aircraft: F-22A Modernization Program Faces Cost, Technical, and Sustainment Risks
Released: 2012-05-02T00:00:00

Letter

Background

The Air Force’s F-22A Raptor is the only operational fifth-generation tactical aircraft, incorporating a low observable (stealth) and highly maneuverable airframe, advanced integrated avionics, and a supercruise engine capable of sustained supersonic flight. The F-22A acquisition program began in 1991 with an intended development period of 12 years and a planned quantity of 648 aircraft. The system development and demonstration period eventually spanned 14 years, during which time threats, missions, and some requirements changed. In particular, the F-22A was originally designed to fly primarily air-to-air missions; however, since that time the Air Force has decided to add air-to-ground capabilities to the F-22A. Development costs substantially increased and total quantities were eventually decreased to 188 aircraft. When the final aircraft is delivered in May 2012, the F-22A acquisition program will be complete at a cost of $67.3 billion.

In 2003, the Air Force established a modernization program to develop and insert new and enhanced capabilities considered necessary to meet the threat. According to Air Force officials, modernization is defined as a process of upgrading and modifying aircraft with a focus on adding new capabilities. The modernization is now proceeding in four related increments, each with multiple projects: the initial phase of modernization, addressed some Increment 2,requirements deferred from the acquisition program and added some new ground attack capability. It has been fielded. Increment 3.1 began fielding in November 2011 and adds enhanced radar and enhanced air-to-ground attack capabilities. Increment 3.2A is a software upgrade to increase the F-22A’s electronic protection, combat identification, and Link-16 communications and data link capabilities. Increment 3.2B will increase the F-22A’s electronic protection, geo- location, and Intra Flight Data Link (IFDL) capabilities, and adds AIM- 9X and AIM-120D missiles.

In addition to these efforts, in 2006, the Air Force began a Reliability and Maintainability Maturation Program (RAMMP). Although the Air Force does not consider this part of the modernization program, it is integral to making the F-22A weapon system more available, reliable, and maintainable. Since the F-22A’s initial fielding in 2006, maintenance issues have prevented it from achieving reliability and availability requirements, and fleet operating and support (O&S) costs are much higher than projected earlier in the program.

F-22A Modernization Costs Have Increased and Deliveries of New Capabilities to the Warfighter Have Been Delayed

Total projected cost of the F-22A modernization program has more than doubled since it started. While the program has completed and fielded some of its planned capabilities, the overall schedule to complete integration and testing of planned capabilities and deliver them to the warfighter has slipped by nearly 7 years. The content, scope, and phasing of planned capabilities also shifted over time with changes in requirements, priorities, and annual funding decisions. Visibility and oversight of the program’s cost and schedule is hampered by a management structure that does not directly track and account for the full cost of specific capability increments. The Air Force plans to separately break out and manage the fourth increment as a major defense acquisition program, which should improve management and oversight.

F-22A Modernization Costs Have Risen Sharply Since the Program Began

The Air Force is now expected to spend around $11.7 billion to modernize and improve the reliability of the F-22A, compared with the $5.4 billion projected soon after the start of development. Officials underestimated the scope of the total program and the time and money that would eventually be needed to develop and field new capabilities. Contributing factors to this cost growth include (1) changed and added requirements; (2) unexpected expenses for building a support infrastructure; and (3) unplanned efforts to improve aircraft reliability and maintainability. Program officials also said that instability in modernization funding contributed to some of the cost growth by stretching the time required to complete projects. Figure 1 shows increased cost estimates over time for the modernization program and other related costs.

Modernization increments include development and procurement costs directly tied to one of the four increments for acquiring upgraded capabilities. These include infrastructure costs for lab support, test operations, program management, retrofit to bring all aircraft to a common configuration, and other efforts integral to supporting modernization increments. Other improvement costs principally include the RAMMP reliability and maintainability projects and making structural repairs needed for the aircraft to achieve its required 8,000 hour service life. At this point, an estimated $5.5 billion of the $11.7 billon has been spent. A future investment of around $6.2 billion remains: $1.3 billion for Increment 3.2B, $3.6 billion for other modernization and support activities, and $1.3 billion for completing the RAMMP and structural repairs.

Completion of F-22A Modernization Projects Has Been Significantly Delayed as Content, Scope, and Phasing of Capabilities Changed over Time

When the F-22A modernization development program began, the Air Force expected to have all current planned capabilities integrated and fielding started by 2010. Now, the final increment is not expected to begin fielding until 2017, 7 years later than initially planned. Air Force officials stated that they underestimated the sheer magnitude of the modernization effort, both in the amount of time required to develop and integrate the capability, and costs to complete the modernization. According to program officials, contributing factors to delays include (1) additional requirements, (2) unexpected problems and delays during testing, and (3) research, development, testing, and evaluation funding fluctuations. Figure 2 compares the initial and latest schedules.

According to Air Force officials, the program currently intends to upgrade 143 aircraft with the full complement of modernized capabilities by fiscal year 2020 and retain 36 aircraft with only Increment 2 capabilities to be used in training. Increment 3.1 is being fielded in fiscal years 2011 to 2016 and Increment 3.2A from fiscal years 2014 to 2016. Increment 3.2B, the last currently planned increment, is expected to field from fiscal years 2017 and 2020. Future capability enhancements are expected to follow the current modernization program, but have not been defined.

The content, scope, and phasing plan changed over time, contributing to cost and schedule problems. Figure 3 illustrates the changing nature of modernization projects, particularly in the later increments.

Some capabilities, such as the Multifunction Advanced Data Link, have been eliminated because of changes in requirements and immature technology. Some, like the AIM-9X missile, have been added to the program to meet emerging threats. Some required capabilities have been reduced, such as the Geolocate project, which will now field a less- capable version than initially planned.

Air Force officials stated that potential new capabilities are analyzed and vetted by evaluating technical maturity and applying cost as independent variable principles to determine which to include in the F-22A modernization program. As a result of this evaluation process, certain capabilities have been modified, deferred, added, or eliminated. Most changes affect the final two increments. For example, MADL, which was intended to provide communications interoperability with the F-35 Joint Strike Fighter, was removed from Increment 3.2B. MADL and other deferred efforts, such as the full Small Diameter Bomb capability, may eventually be delivered in future increments yet to be defined.

Visibility and Oversight of the Program’s Cost and Schedule Is Hampered by a Management Structure and Funding Mechanism

Tracking and accounting for the full and accurate cost of each modernization increment, and individual projects within each increment, are limited by the way the modernization program is structured, funded, and executed. As depicted in figure 4, only 26 percent of total projected costs can be traced directly to the four modernization increments. About 57 percent of total costs go to fund activities that support all the modernization efforts and the overall F-22A program but are not charged to specific increments. These activities include test operations, the building and use of government labs, management activities, retrofit efforts to bring the fleet to a common configuration, and other infrastructure accounts. The remaining 17 percent funds the RAMMP program and structural repairs. While Air Force officials do not consider these efforts as part of the funded modernization program, we note that these efforts are needed to improve fleet affordability and achieve desired aircraft life and thus integral to justify future modernization investments.

Program accountability and oversight have been hampered by how the modernization program was established, managed, and funded. As we reported in March 2005, the Air Force embarked on the modernization program without a knowledge-based business case to support the multibillion dollar investment to significantly change the aircraft’s capabilities and missions.should have been established as an entirely separate acquisition program with a new business case because of the magnitude of the proposed changes. A sound business case would have matched requirements with resources—proven technologies, sufficient engineering capabilities, time, We stated that the modernization program and funding—when undertaking new product development.information about the schedule and funding was not adequately known at the start of modernization. Rather than making the new business case to justify and manage the modernization program as a separate major defense acquisition, Air Force officials incorporated it within the existing F-22A acquisition program and comingled funds. Their rationale was their belief that breaking modernization efforts out as a separate program would have delayed the capability. As a result, development funding and infrastructure expenses were added to the existing acquisition program’s baseline.

The modernization program proceeded without establishing its own set of acquisition milestones and has not been subject to the same level of scrutiny by senior defense leaders or the performance reporting required of major defense acquisition programs as provided for in DOD acquisition policy. At their discretion, DOD chose to execute it within the baseline F-22A program.

In November 2004, defense leaders recognized that the size and importance of the modernization program warranted a higher level of scrutiny. The acting Under Secretary of Defense for Acquisition, Technology and Logistics directed the Air Force to hold separate milestone reviews for the future stages of the modernization program to be consistent with DOD acquisition policy. Under this Air Force direction, the current modernization projects would not require formal milestones, but Office of the Secretary of Defense (OSD) oversight would be provided by periodic reviews. In 2007, OSD directed the Air Force to update the F-22A Acquisition Program Baseline to reflect the approved Increments 3.1 and 3.2; however, the Air Force believed that an acquisition strategy report rather than a baseline would provide better insight into funding and schedule details for the modernization increments. Not separating the modernization program from the F-22A program baseline was consistent with how the Air Force had handled modernization programs for prior aircraft. However, had the Air Force initiated the program under existing guidelines established by DOD Instruction 5000.02 for managing and implementing major acquisition programs, oversight of the program would have benefitted. Under these guidelines, programs are required to have an approved minimum set of Key Performance Parameters, included in the Capability Development Document; an approved Acquisition Strategy; Acquisition Program Baseline; an Analysis of Alternatives; and an Independent Cost Estimate, for a Milestone B decision that would allow them to proceed into the Engineering and Manufacturing Development phase.

OSD recently reiterated its requirement for the F-22A to be consistent with DOD policy, and in December 2011, OSD directed the Air Force to establish increment 3.2B as a separate major defense acquisition program. According to the Air Force, this increment is expected to cost around $1.5 billion. Given the significant slips in schedule experienced by increments 3.1 and 3.2A, the decision to separately oversee increment 3.2B is a late but positive change. Increment 3.2B will be reported as its own major program with system development starting in fiscal year 2013. This should improve management, cost visibility, and program oversight. Air Force officials told us that they expect to manage and report all future F-22A modernization programs as separate acquisitions, starting with Increment 3.2B.

Performance Outcomes Have Been Judged Satisfactory, but Testing and Improving Reliability and Affordability of the Fleet Will be Challenging

Testing how well new capabilities perform is ongoing; results to date have been satisfactory but development and operational testing of the largest and most challenging sets of capabilities have not yet begun. Going forward, major challenges will be developing, integrating, and testing new hardware and software to counter emerging future threats. Other risks are associated with availability of unique test assets, greater reliance on laboratory ground tests, and relocation of a key F-22A lab that is needed to help support testing of software for the new capabilities. Parallel efforts to improve F-22A reliability and maintainability are critical to ensure life- cycle sustainment of the fleet is affordable and to justify future modernization investments.

Operational Testing Results on New Capabilities Have Been Mostly Positive to Date, but More Challenging Efforts Are Still Ahead

New F-22A capabilities delivered by the modernization program will be demonstrated through follow-on operational testing and evaluation to assess the upgraded F-22A’s effectiveness and suitability. Testing on the first two increments successfully demonstrated new air-to-ground capabilities. Testing of the third and fourth increments has not begun and several technical risks remain for these new capabilities. Successful mitigation of these risks is critical to keeping F-22A’s planned upgrades on schedule and within planned costs. Table 1 shows the current status of F-22A modernization operational testing for each increment.

Follow-on operational testing and evaluation for F-22A fighters incorporating Increment 2 capabilities, including assessments of expanded air-to-ground capability and improvements in system suitability, were successfully completed in August 2007. The F-22A’s configured with Increment 2 capabilities were found to be operationally effective in suppressing and destroying fixed enemy air defenses, and also demonstrated successful fixes of deficiencies and weapons integration problems that had caused problems in previous testing. Flight testing demonstrated the ability to employ the Joint Direct Attack Munition (JDAM) at supersonic speeds in a high- threat anti-access environment where stealth capabilities are needed. Without this capability, baseline aircraft were only able to launch JDAMs at fixed targets in lower threat environments and at slower speeds while using target coordinates from ground spotters.

Increment 3.1 further enhances F-22A’s air-to-ground capability by allowing the aircraft to find and locate ground targets with on-board systems, rather than relying on external personnel and platforms for targeting. Increment 3.1 completed follow-on operational testing in November 2011; a significant delay of 4 years from the original plan due to shortcomings identified with the baseline and upgraded aircraft. In 2009 and 2010, the Director of Operational Test and Evaluation (DOT&E) reported significant stealth-related maintenance issues that lowered operational availability and mission capability rates. F-22A program officials identified technical issues in upgrading radar, navigation, and software that needed to be addressed to meet operational testing requirements.

The Air Force began Increment 3.1 operational testing in January 2011, but soon encountered flight delays that persisted from March to September 2011. The entire F-22A fleet was ordered to stand-down due to potential problems with the aircraft’s oxygen generation system. Unavailability of the test range and technical problems with ground support equipment also contributed to the lengthy flight delay. The Air Force completed flight testing for Increment 3.1 in November 2011 and expects to release the operational test report in late March 2012. In its 2011 annual report, DOT&E did not identify any significant remaining issues since flights had resumed. DOT&E also approved reducing trials from 16 to 8 and decreasing simulator test trials from 96 to 64. According to program officials, hardware and software issues had been identified and fixed as testing progressed and test pilots provided very positive feedback on Increment 3.1’s enhancements.

Increment 3.2A development began in November 2011 after significant delays. This increment involves updating software to enhance electronic protection and combat identification capabilities, so that F-22A can handle new threats expected in the future. Developmental testing for this increment is expected to start in 2012 and be completed in late 2013. Operational testing and evaluation will follow and is planned to conclude in 2014.

Program officials assessed the Increment 3.2A schedule as having moderate risk. Test aircraft have been operating much longer than planned and were to be replaced by new production aircraft; however, this has not happened due to the substantial reduction in the size of the F-22A fleet. Other risks appear lower. For example, some software for electronic protection and combat identification capabilities has already been developed for the F-35 Joint Strike Fighter. Also, while the Link-16 upgrade will involve a significant amount of development work, program officials consider it to be moderate risk.

Increment 3.2B is scheduled to begin Engineering and Manufacturing Development in December 2012 and the decision to enter into production is scheduled for January 2016. Key capability upgrades include integrating the AIM-9X and AIM-120D missiles on the F-22A and upgrading geolocation and electronic protection subsystems. Early requirements analysis determined that AIM-9X integration may be more difficult and take longer than expected and officials have already begun risk reduction efforts. Overall, software integration is considered to have the highest risk for Increment 3.2B projects, while hardware development is rated as a moderate risk. Program officials believe that the full range of capabilities added in the modernization program can be accommodated within the weight and space limitations of the F-22A aircraft, but this will be a critical consideration in any future modernization plans.

The Air Force is seeking ways to reduce the costs of Increments 3.2A and 3.2B by streamlining program activities. Officials want to make more use of developmental tests to also satisfy operational test requirements, allowing the program to identify errors for correction earlier and reducing overall costs by eliminating redundant tests. The program also intends to increase its use of F-22A ground laboratories to substitute for more expensive flight tests. The F-22A lab infrastructure is an extensive, distributed system of dedicated labs that integrate and certify flight software releases to the field and support F-22A modernization, production and sustainment activities. However, there are technical risks if lab tests do not fully replicate the performance of actual F-22A aircraft in intended environments. Officials are also expecting to save money by relocating the Raptor Avionics Integration Lab—a critical work site that stimulates sensors for targeting—from Marietta, Georgia, to Ogden Air Logistics Center, Utah by the summer of 2012. Program officials acknowledge there are some risks in this. For example, unique equipment could be damaged during the move and experienced lab staff could decide to leave the F-22A program rather than relocate.

RAMMP Program Addresses Reliability and Maintainability Deficiencies to Improve Affordability and Justify Future Investments

In addition to capability upgrades, the F-22A budget also funds efforts to address reliability and maintainability deficiencies that have increased support costs and have prevented the F-22A from meeting a key performance requirement. RAMMP is to develop and implement enhancements to increase aircraft availability, make maintenance faster and less costly, and reduce total life-cycle operating and support costs and cost per flying hour. While RAMMP is expected to reduce life-cycle costs over the long term, up-front investments to help realize future cost reductions have increased. The program had planned to spend about $258 million between 2005 and 2011, but actual investments through 2011 were about $528 million. The total RAMMP funding requirement through the year 2023 is now estimated at almost $1.3 billion. Air Force officials attributed part of RAMMP’s increased costs to additional projects and increased labor hours to address corrosion.

Keeping the F-22A fleet affordable and meeting required performance measures is critical to sustaining fleet operations over the long term, and ensuring it is available in sufficient numbers for required missions. Projected operational and support costs are much higher than earlier estimates. For example, a 2007 independent estimate by the Air Force Cost Analysis Agency projected a $49,549 cost per flying hour in 2015 (by which time the F-22A was expected to reach full maturity), more than double the $23,282 cost per flight hour estimated in 2005.

Air Force officials gave various reasons for sustainment cost increases including (1) unrealized savings from the F-22A’s performance-based logistics contract (2) fixed costs that had to be spread over a smaller number of aircraft; and (3) higher than expected costs to refurbish or replace broken parts, including diminishing manufacturing sources. However, the one common contributing factor—and the most impactful— is the cost and complexity of maintaining stealth characteristics and restoring aircraft to the required stealth level after flight operations and maintenance. Our recent report found that the number of maintenance personnel required to maintain the F-22A’s specialized stealth exterior has increased, posing a continuing support challenge for this aircraft.

This has important implications for the affordability and life-cycle cost estimates for the F-35 Joint Strike Fighter.

When it started in 2006, a major goal of RAMMP was to improve F-22A reliability to meet its key performance requirement by the time the fleet reached maturity at 100,000 total flight hours. This performance indicator, known as mean time between maintenance (MTBM), required aircraft in the F-22A fleet to fly an average of 3 hours between maintenance events, excluding routine servicing and inspections. This performance standard was a key performance requirement in the F-22A acquisition contract, but the fleet has never been able to meet that requirement. Currently, the MTBM achieved by the operational test aircraft with improvements is 2.47 hours.

In April 2011, the Joint Requirements Oversight Council approved changing the main reliability metric from MTBM to another performance indicator, known as material availability. Officials believed the MTBM indicator was hard to define and measure, was unrealistic, and did not accurately reflect the fleet’s readiness to perform missions. Material availability is defined as the percentage of the fleet available to perform assigned missions at any given time. This standard calls for the F-22A fleet to achieve increasing levels of availability between 2011 and 2015 toward the final goal of 70.6 percent. Last year, the F-22A fleet achieved a 55.5 percent materiel availability rate. Stealth-related maintenance, system component reliability problems, and lack of spare engines were factors contributing to the fleet not achieving the goal. However, program officials expect the F-22A fleet to achieve the final availability goal by 2015 after the full fielding of reliability improvements. The Air Force reported that operational test on aircraft integrated with the current reliability improvements have achieved 78 percent availability; they anticipate significant gains by the overall fleet once reliability improvements are installed on all F-22A aircraft.

Conclusions

Keeping the F-22A as the world’s most advanced stealth fighter requires the Air Force to counter changing threats, as well as ensure the F-22A fleet is affordable, reliable, and sustainable. In response to changing threats, officials began a Modernization Program to add new missions and capabilities while fixing problems and deficiencies that were carried over from the original development program. However, the F-22A modernization program has not had the management rigor or oversight on par with the $11.7 billion investment it entails. The program was not well-defined when it began in 2003, has had fluid scope and cost, and has been challenging from an oversight perspective as it was blended into the baseline F-22A program rather than being managed separately. As early as 2004, OSD began discussing the need to manage future modernization increments as separate acquisition programs. While modernization has been underway, the Air Force has found it necessary to invest in improved reliability and availability of the F-22A through the RAMMP program. The original reliability requirement was not met and has since been changed to another indicator. Meanwhile, O&S costs have been significantly higher than planned, with maintenance of the aircraft’s stealth levels being particularly demanding. The lessons learned on the maintenance of the stealthy F-22A may have implications for the F-35 Joint Strike Fighter.

Splitting out increment 3.2B as a separate major acquisition defense program indicates that OSD is reasserting its role in the F-22A program. This is beneficial for oversight in light of the significant decisions and investments yet to come for the program. Increment 3.2B requires around $1.3 billion, while completing the RAMMP program, ongoing modernization projects, and other improvements will require an estimated $4.9 billion—a total future investment of around $6.2 billion. The program is highly dependent on a single contractor, whose responsibilities encompass managing the development and production of the F-22A; development, production, and retrofit of modernization; execution of the RAMMP program; and life-cycle support of the F-22A fleet, including supply and maintenance. Finally, the Air Force informed us that it expects to manage future modernization increments as separate acquisitions. However, given the approach the Air Force has taken to date on this and other modernization programs, there is little assurance that this will occur without specific OSD direction.

Recommendation for Executive Action

As new and enhanced capabilities are proposed and vetted beyond Increment 3.2B in the F-22A modernization program, we recommend that the Under Secretary for Acquisition, Technology and Logistics evaluate those capabilities in accordance with DOD policy and statutory criteria to determine if they should be established as separate major defense acquisition programs, each with its own milestones, business case, and cost baseline that includes all applicable direct and indirect support costs required to complete the program.

Agency Comments and Our Evaluation

DOD provided us written comments on a draft of this report. The comments appear in appendix II. DOD also provided technical comments that were incorporated as appropriate. During the agency comment period, DOD requested clarification regarding our recommendation. As a result, we revised the recommendation to more clearly state that the Under Secretary of Acquisition, Technology and Logistics will evaluate future planned F-22A modernization capabilities to determine if those meeting DOD policy and statutory criteria should be established as a separate major acquisition program.

DOD concurred with the revised recommendation.

We are sending copies of this report to interested congressional committees, the Secretary of Defense, the Secretary of the Air Force and the Under Secretary of Defense for Acquisition, Technology and Logistics. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have questions about this report, please contact me at (202) 512-4841 or SullivanM@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report.

Appendix I: Scope and Methodology

To determine the extent to which F-22A modernization met cost and schedule goals and operational requirements, we reviewed documentation of program plans and status, including cost estimates, briefings by program office officials to Department of Defense (DOD) and Air Force oversight officials, annual Selected Acquisition Reports, Defense Acquisition Executive Summary reports, Director, Operational Test and Evaluation (DOT&E) annual test result summaries, Defense Contract Management Agency (DCMA) program assessment reports, acquisition plans, operational requirements documentation, contract documentation, schedules and other data. We reviewed documentation of key decisions made on F-22A modernization, including acquisition decision memoranda and Joint Requirements Oversight Council memoranda. We reviewed F-22A cost performance report data, contract cost data, and budgetary documents. In assessing the achievement of cost goals by the F-22 modernization and other improvement efforts, we compared the program cost estimate from 2004, shortly before development began for Increment 2, with the latest available estimates. We determined what changes in planned capabilities occurred after modernization efforts began. In assessing the F-22A modernization’s achievement of schedule goals and delivery of planned capabilities, we identified progress made in delivering new capabilities in accordance with plans, and determined what factors contributed to schedule changes. We interviewed program office officials having knowledge of factors driving cost estimate and schedule changes over time. We also interviewed officials from the F-22A Program Office, DOD test organizations, and Air Combat Command to obtain their views on progress; ongoing concerns and actions taken to address them; and future plans to complete F-22A development procurement and operational testing. We used the latest cost data available during the period of our review; however the F-22A program office was preparing a new cost estimate for F-22A modernization and the estimated costs of increments beyond Increment 3.2B had not yet been determined or added to this estimate.

To determine what progress has been made in completing developmental and operational testing, and resolving system deficiencies, we reviewed DOT&E annual test report summaries and briefings to DOD oversight and requirements officials. We reviewed summaries of recent operational test results provided by Air Force test officials and program risk information related to developmental and operational testing for F-22A modernization. We reviewed documentation of program decisions, including acquisition decision memoranda. We reviewed data from prior GAO reviews on operations and support costs for F-22A and other stealth aircraft, Selected Acquisition Reports, Defense Acquisition Executive Summary reports, contract documents, and program cost estimates. In assessing progress made in operational testing, we compared initial and current operational test plans to determine if significant changes were made after testing began. We identified relevant factors contributing to testing delays. In assessing the resolution of system deficiencies, we identified the number of successful test points flown during operational testing and identified what changes were made in requirements after operational testing began. We determined what key risks and issues remain that could affect developmental and operational testing in the future. We identified issues contributing to increased operations and sustainment costs and to decreased aircraft availability, and actions taken by the F- 22A program to mitigate them. We interviewed officials from the F-22A Program Office, DOD test organizations, and Air Combat Command to obtain their views on progress, ongoing concerns and actions taken to address them, and future plans to complete developmental and operational testing. At the time of our review, the final follow-on operational test and evaluation results for Increment 3.1 were not yet available and other test information we had requested was not readily available within the reporting period for this report due to its high classification level. Accordingly, our analysis of actual results and data was somewhat constrained and our reporting limited to providing summary level observations due to the classification level of some of the data. Notwithstanding, DOD officials gave us access to sufficient information to make informed judgments on the matters covered in this report.

In performing our work, we obtained information and interviewed officials from the F-22A Program Office, Wright-Patterson Air Force Base, Ohio; Air Combat Command, Langley Air Force Base Virginia; Office of the Director, Operational Test & Evaluation, Office of the Secretary of Defense, Arlington, Virginia; and the Air Force Operational Test and Evaluation Center, Kirtland Air Force Base, New Mexico. We assessed the reliability of DOD and F-22A contractor data by (1) obtaining and reviewing related information from various sources, and (2) interviewing agency officials knowledgeable about the data. We determined that the data were sufficiently reliable for the purposes of this report. We conducted this performance audit from June 2011 to March 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Bruce Fairbairn, Assistant Director; Marvin Bonner; Sean Seales; Marie Ahearn; Ana Aviles; Laura Greifner; Travis Masters; and Roxanna Sun made key contributions to this report.

Related GAO Products

Tactical Aircraft: Comparison of F-22A and Legacy Fighter Modernization Programs. GAO-12-524. Washington, D.C.: April 26, 2012.

Joint Strike Fighter: Restructuring Places Program on Firmer Footing, but Progress Still Lags. GAO-11-325. Washington, D.C.: April 7, 2011.

Defense Acquisitions: Assessments of Selected Weapon Programs. GAO-11-233SP. Washington, D.C.: March 29, 2011.

Tactical Aircraft: DOD’s Ability to Meet Future Requirements Is Uncertain, with Key Analyses Needed to Inform Upcoming Investment Decisions. GAO-10-789. Washington, D.C.: July 29, 2010.

Defense Management: DOD Needs Better Information and Guidance to More Effectively Manage and Reduce Operating and Support Costs of Major Weapon Systems. GAO-10-717. Washington, D.C.: July 20, 2010.

Defense Contracting: DOD Has Enhanced Insight into Undefinitized Contract Action Use, but Management at Local Commands Needs Improvement. GAO-10-299. Washington, D.C.: January 28, 2010.

Defense Acquisitions: Measuring the Value of DOD’s Weapon Programs Requires Starting with Realistic Baselines. GAO-09-543T. Washington, D.C.: April 1, 2009.

GAO Cost Estimating and Assessment Guide. GAO-09-3SP. Washington, D.C.: March 2, 2009.

Defense Acquisitions: A Knowledge-Based Funding Approach Could Improve Major Weapon System Program Outcomes. GAO-08-619. Washington, D.C.: July 2, 2008.

Tactical Aircraft: DOD Needs a Joint and Integrated Investment Strategy. GAO-07-415. Washington, D.C.: April 2, 2007.

Tactical Aircraft: DOD Should Present a New F-22A Business Case before Making Further Investments. GAO-06-455R. Washington, D.C.: April 26, 2006.

Defense Acquisitions: Air Force Still Needs Business Case to Support F/A-22 Quantities and Increased Capabilities. GAO-05-304. Washington, D.C.: March 15, 2005.