F-22 Modernization: Cost and Schedule Transparency Is Improved, Further Visibility into Reliability Efforts Is Needed
Published: 2014-05-15T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Air Force's cost estimate for modernizing the F-22 aircraft's capabilities and making improvements that address the aircraft's reliability and structural problems is $11.3 billion, as of January 2014. In February 2013, the Air Force awarded a multibillion dollar contract for F-22 modernization efforts from 2013 through 2023. The National Defense Authorization Act for Fiscal Year 2013 directed the Air Force to report annually on improvement efforts.
GAO was asked to review F-22 modernization and improvement efforts. This report assesses (1) the Air Force's new approach to structuring its F-22 modernization efforts, (2) the Air Force's management of its F-22 reliability and structural improvement efforts, and (3) challenges facing efforts to modernize the F-22 and address reliability and structural problems moving forward. GAO analyzed program management, contracting, and other relevant documents; analyzed aircraft availability information; and spoke with Air Force officials.
section_title: What GAO Found
paragraphs: After a decade of managing its efforts to modernize the F-22 as part of the original F-22 program, the Air Force has now begun structuring these efforts as distinct incremental programs, and plans to continue doing so moving forward. For example, the Department of Defense (DOD) recently designated F-22 modernization Increment 3.2B—intended to integrate two types of missiles onto the F-22 and upgrade other subsystems—as a major defense acquisition program with its own baseline cost and schedule estimate. This approach is consistent with the approach mandated in the National Defense Authorization Act for Fiscal Year 2013 and recommended in a May 2012 GAO report, and should allow decision makers to see a more precise view of cost and schedule changes. Furthermore, the Air Force plans to use individual contractual agreements for each modernization effort, a contracting approach that should support informed management and oversight of each increment by increasing visibility into funds obligated to individual efforts.
In contrast to modernization, the larger of the Air Force's two primary F-22 improvement efforts—the Reliability and Maintainability Maturation Program (RAMMP)—is not managed with its own cost and schedule baseline. This approach limits transparency of cost and schedule progress. The National Defense Authorization Act for Fiscal Year 2013 directed the Air Force to submit an annual report to the congressional defense committees on RAMMP and on F-22 structural repair efforts, to include baseline cost and schedule estimates. While the first report submitted under this requirement in April 2013 provided some cost and schedule information on the structural repair efforts, which could be used as a baseline estimate going forward, it did not include such information for RAMMP. Without a comprehensive baseline cost and schedule estimate for reliability efforts that encompasses the life of the aircraft across all types of funding, it is difficult to consistently track cost and schedule progress on projects that, to date, have cost almost $1 billion.
The Air Force faces schedule and performance challenges in its management of F-22 modernization and improvement efforts as a result of continuing maintenance issues. Delays in completing depot-level maintenance have translated into delays in fielding modernization increments, which could also affect future increments. Further, the F-22 fleet will not achieve its availability requirement as scheduled, despite substantial investment in RAMMP projects. RAMMP is intended to increase the availability of the F-22 to perform its missions, primarily through improving reliability—the likelihood that the aircraft will operate without failure—and reducing the time required to maintain the aircraft. Air Combat Command officials stated that the current requirement is based on assumptions that may no longer be applicable, and that they expect to revisit this requirement at some future point, although they did not specify when that reassessment would occur. Thus, continued investment in some projects aimed at achieving a requirement that may no longer be valid may be unwarranted. Any delay in revisiting this requirement could limit funding DOD might be able to make available for other high priority activities in a time of austere federal budgets.
section_title: What GAO Recommends
paragraphs: To support program oversight and ensure efficient use of future funding, GAO recommends that the Air Force include comprehensive cost and schedule baselines for reliability projects in its annual report to Congress and expedite reassessment of the F-22 availability requirement. DOD disagreed with the first recommendation, stating that reliability programs cannot be baselined like major defense acquisition programs, and agreed with the second. GAO continues to believe the first recommendation is valid as discussed in the report.

section_title: Letter
paragraphs: Since 2003, the Air Force has undertaken a number of efforts to  modernize the F-22 and make improvements that address the aircraft’s  reliability and structural problems. The most recent combined cost  estimate for these efforts was approximately $11.3 billion, of which nearly  60 percent has already been invested. In February 2013, the Air Force  awarded a contract representing several billion dollars of that total to  continue to modernize the F-22 from 2013 through 2023. The Air Force  recently began development on the latest modernization effort, and  continues significant investment in improvements aimed at achieving the  F-22’s original reliability and service life requirements. The National  Defense Authorization Act for Fiscal Year 2013 (NDAA) established  managing and reporting requirements for new modernization efforts and  the aircraft’s reliability and structural improvement programs.
As billions of dollars continue to be invested in the F-22, you asked us to  review the acquisition and contracting strategies for the F-22  modernization and improvement efforts. This report assesses (1) the Air  Force’s new approach to structuring its F-22 modernization efforts, (2) the  Air Force’s management of its F-22 reliability and structural improvement  efforts, and (3) challenges facing efforts to modernize the F-22 and  address reliability and structural problems moving forward. To assess the  Air Force’s new approach to structuring modernization efforts, we  analyzed actual and projected program management data for fiscal years  2003 to 2023, from sources including acquisition decision memorandums,  selected acquisition reports, and independent and program cost  estimates. These sources were issued in 2012 or 2013. We also reviewed  contracting and budget documents relevant to F-22 modernization. To  assess the Air Force’s management of its F-22 reliability and structural  improvement efforts, we collected and analyzed Air Force reports and  briefing materials on aircraft availability and reliability and repair projects.  To assess challenges facing efforts to modernize the F-22 and address  reliability and structural problems moving forward, we analyzed program  schedules and actual and projected aircraft availability data for fiscal  years 2011 to 2018, current as of August 2013, and reviewed program  documents discussing maintenance issues affecting the aircraft. We also  discussed progress and plans for modernization and improvement efforts  with DOD, Air Force, and F-22 program officials. We assessed the  reliability of DOD data on selected acquisition reports, cost estimates, and  materiel availability rates and projections by reviewing documentation  related to the data and interviewing agency officials knowledgeable about  the data. We determined that the data were sufficiently reliable for the  purposes of this report. Appendix I contains a more detailed description of  our scope and methodology.
We conducted this performance audit from July 2013 to May 2014 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The F-22 Raptor is the first operational tactical aircraft fielded by the Air  Force that incorporates a low observable (stealth) and highly  maneuverable airframe, advanced integrated avionics, and an engine  capable of sustained supersonic flight. The F-22 acquisition program  began in 1991 with an intended development period of 12 years and a  planned quantity of 648 aircraft. However, F-22 development eventually  spanned 14 years, during which time costs increased substantially and  threats, missions, and some requirements changed. As a result, total  quantities were significantly decreased, with 195 of the 648 planned  aircraft being delivered to the Air Force. As of March 2014, the total  number of active F-22 aircraft was 182.delivered in 2012, the F-22 acquisition program was completed at a total  estimated cost of over $67 billion.
When the final aircraft was  In 2003, the Air Force established an F-22 modernization program  primarily to develop and insert new and enhanced ground attack  capabilities considered necessary to meet current and future threats. The  Air Force’s current total cost estimate for F-22 modernization is around  $9.4 billion, which includes costs directly attributable to modernization, as  well as costs to support testing and laboratory operations, provide  periodic software updates, and ensure compliance with military and  civilian standards for identifying other aircraft and avoiding collisions.  According to Air Force officials, modernization is defined as a process of  upgrading and modifying aircraft with a focus on adding new capabilities.  The Air Force has divided its modernization efforts into a series of  increments, each with multiple upgrades. aircraft have been upgraded with Increment 2, and more than 140 of  those aircraft are expected to be upgraded with subsequent  modernization increments. Aircraft that are primarily used for training are  not expected to receive upgrades beyond Increment 2. Table 1 lists the  current F-22 modernization increments in order and identifies the main  features and key events associated with each one.
The Air Force numbering scheme considers Increment 1 to be the baseline capabilities  delivered by the F-22 acquisition program.
Development work for Increment 3.2B began in 2008 but a formal decision to begin a system  development program was not made until 2013.
In addition to modernization efforts, the Air Force is also continuing to  pursue two primary improvement efforts. These fix identified issues  affecting the aircraft, rather than developing and adding new capabilities.  One of the efforts, the Structural Repair Program (SRP), started in 2006,  and involves installation of structural hardware that is needed for the  aircraft to achieve its originally expected 8,000 flight hour service life.SRP is expected to be completed in 2019. The other effort, known as the  Reliability and Maintainability Maturation Program (RAMMP), started in  2005, and consists of a large number of individual projects primarily  intended to increase the amount of time the aircraft are available for  operational use. According to program officials, as of January 2014 there  were over 100 RAMMP projects of varying scope and cost under way,  and over 200 projects had been completed. Program officials have noted  that new RAMMP projects can be initiated as maintenance issues are  identified. They also noted that RAMMP is expected to continue  throughout the life of the aircraft. These two primary improvement  programs, as well as efforts to enhance engine reliability, are currently  expected to cost a total of $1.9 billion.
Figure 1 illustrates how the current cost estimate of $11.3 billion is divided  between modernization and improvement efforts, as well as the amounts  invested through 2013 and still to be invested.
As illustrated in figure 1, more than half of the current estimated funding  for both the modernization and improvement efforts has already been  invested. Total spending to complete F-22 modernization and  improvement efforts is uncertain as capability needs evolve and reliability  improvement projects are identified. Program officials stated that they  intend to issue an updated cost estimate later in 2014, to include cost  projections through the life of the aircraft.
section_title: Air Force’s New Approach to Structuring F-22 Modernization Should Improve Oversight and Transparency
paragraphs: The Air Force’s decision to structure new F-22 modernization efforts as  independent programs should provide greater visibility into each  individual effort. We found in May 2012 that modernization efforts had  initially been managed and funded as part of the original F-22 program  and costs for specific increments were not individually tracked, which  limited transparency. However, beginning with Increment 3.2B, the Air  Force plans to manage each modernization effort as a distinct, stand- alone program with levels of oversight and approval set according to the  dollar value of the program. Increment 3.2B was designated as a major  defense acquisition program with an approved cost, schedule, and  performance baseline in June 2013. The Air Force is now funding the  program as a discrete element within its budget and is required to provide  periodic reports, known as selected acquisition reports, to Congress on its  cost and schedule status. Looking ahead, the Air Force is planning to  begin four smaller increments intended to ensure compliance with military  and civilian standards for identification of other aircraft and avoidance of  collisions. This incremental approach is consistent with the approach we  recommended in May 2012 and Congress mandated in the 2013 NDAA,  and should provide greater transparency into cost, schedule, and  performance progress.
F-22 modernization efforts are primarily performed under a contracting  vehicle known as an indefinite delivery / indefinite quantity contract  (ID/IQ).contract ceiling of $6.9 billion to cover F-22 modernization efforts from  2013 to 2023. Program officials stated that about $6.2 billion of the  funding under this contract will be used to continue work on  modernization efforts that began prior to the contract award as well as  other planned efforts. They noted that the remaining $700 million would  be used to cover costs associated with future requirements that have not  yet been defined. According to program officials, they plan to complete an  updated cost estimate later in 2014 that is expected to reflect all  modernization costs through the life of the aircraft. As of August 2013,  less than $500 million had been obligated under this contract, the majority  of which is for engineering and manufacturing development work on  Increment 3.2B. As a result, more than $6.4 billion remained available to  be obligated.
In February 2013, the Air Force awarded an ID/IQ contract with a  The Air Force plans to use individual contractual agreements, known as  delivery orders, which are issued under the ID/IQ contract, for each  modernization effort. The delivery orders establish, among other things,  the requirements, cost or price, and schedule for the work to be  performed under the order. Program officials have stated that they intend  to issue a separate delivery order for each future modernization effort.  Issuing delivery orders for each modernization effort should support  informed management and oversight of each increment by increasing  visibility into funds obligated to individual efforts.
section_title: Air Force’s Approach to Managing Reliability Improvement Efforts Limits Transparency of Cost and Schedule Progress
paragraphs: The larger of the Air Force’s two primary F-22 improvement efforts— RAMMP—is not managed with its own cost and schedule baseline. This  approach limits transparency of cost and schedule progress. The 2013  NDAA directed the Secretary of the Air Force to submit an annual report  to the congressional defense committees on the cost, schedule, and  performance of RAMMP and SRP. That direction required the Air Force  to identify cost and schedule baselines for these two programs, which  would provide Congress with visibility into the progress of these  improvement efforts over time. The Air Force submitted its first report in  April 2013, but did not explicitly identify baseline cost and schedule  estimates for RAMMP or SRP.
The Air Force’s 2013 report included a cost estimate through completion  for SRP, as well as some schedule information. Although not explicitly  identified as baselines, these cost and schedule estimates may provide  the information necessary for use as SRP baseline estimates going  forward, if the Air Force chooses to do so. However, the report lacked  information for RAMMP needed to serve as baseline cost and schedule  estimates going forward. According to DOD guidance, program cost  baselines should reflect realistic estimates that cover the entire program  and schedule baselines should include projected dates for major decision  points, among other things. For RAMMP, the report does not provide any  schedule information, and it identifies projected development and  procurement funding through fiscal year 2017. However, program officials  have stated that RAMMP projects will continue for the life of the F-22,  which is currently expected to retire around 2033. Additionally, RAMMP  has received operations and maintenance funding, but this funding is not  reflected in the report. Without a comprehensive baseline cost and  schedule estimate for reliability efforts that encompasses the life of the  aircraft across all types of funding, it is difficult to consistently track cost  and schedule progress on projects that to date have cost almost $1  billion. Air Force officials agreed that the 2013 report did not contain a  complete cost and schedule baseline for RAMMP.
In contrast to its approach of contracting for modernization efforts under  individual delivery orders, the Air Force’s approach to contracting for  RAMMP makes it difficult to track funding obligated to individual projects.  According to program officials, these projects are primarily funded  through an F-22 air vehicle sustainment contract. Under this contract,  work on individual projects can be split across multiple line items, making  it difficult to manage and oversee cost and schedule performance of  individual projects. These projects range in size and complexity from  smaller efforts that cost less than $1 million to a project to improve the  durability of stealth coatings that is expected to cost more than $50  million.
Unanticipated problems with the F-22 can generate additional RAMMP  projects, which makes it challenging to accurately estimate future  RAMMP costs. For example, to address concerns with the F-22 life  support system, an Air Force committee that advises on science issues  recommended that a back-up oxygen system be incorporated into the  aircraft. Development of this system has resulted in at least $20 million of  unplanned work being funded through RAMMP.
section_title: F-22 Modernization and Improvement Efforts Will Continue to Face Schedule and Performance Challenges
paragraphs: The Air Force faces schedule and performance challenges in its  management of F-22 modernization and improvement efforts as a result  of continuing maintenance issues. Delays in completing depot-level  maintenance have translated into delays in fielding modernization  increments, which could also affect future increments. Despite substantial  investments in improving F-22 reliability, the aircraft does not yet meet its  availability requirement, and the Air Force does not expect it to meet this  requirement.
subsections: section_title: Maintenance Delays Are Affecting Schedule for Fielding Modernization Efforts
paragraphs: Longer than expected time frames for completing depot-level  maintenance on F-22 aircraft extended the near-term F-22 modernization  schedule and could affect fielding of future modernization increments.  Depot-level maintenance refers to major maintenance and repairs, such  as overhauling, upgrading, or rebuilding parts, assemblies, or  subassemblies, which is usually performed at a facility known as a depot.  A contractor-run depot in Palmdale, California performing F-22  maintenance has returned multiple aircraft back to the fleet months later  than planned. For example, in 2013 this depot returned one aircraft back  to the fleet more than 10 months later than originally scheduled. Fielding  of modernization increments can take place at a depot, and program  officials noted that the delays at the Palmdale depot have affected the  schedule for fielding Increment 3.1. As of May 2012, the Air Force  expected Increment 3.1 to be fielded in fiscal year 2016. That increment  is now not expected to complete fielding until August 2017. In addition,  the SRP program, which was initially planned to complete in late 2017, is  now not expected to finish until 2019 as a result of the depot maintenance  delays.
Further delays may affect the time line for fielding future modernization  increments because the increments build on each other sequentially.  Increment 3.2A is a software upgrade that requires Increment 3.1  hardware, and delays in the Increment 3.1 schedule have caused the  expected fielding of Increment 3.2A to move from fiscal year 2016 to  fiscal year 2018. Similarly, any further delays in fielding these earlier  increments could affect fielding of Increment 3.2B.
According to an Air Force analysis of F-22 maintenance issues, work  related to maintaining the stealth features of the F-22 accounts for almost  half of the time that the aircraft are unavailable due to maintenance.  Program officials noted that after repairs or modifications that involve  removing a panel with stealth coatings, those coatings must be restored,  which can take several days. As a result, minor repairs or modifications  that would take a few hours on a non-stealth aircraft can require days of  maintenance on an F-22. Therefore, when an aircraft is scheduled to  enter a depot to receive modifications that need to be performed at the  depot, the Air Force often plans to perform other major work that the  aircraft is intended to receive at the same time, since the stealth coatings  will need to be restored anyway. For example, when an F-22 aircraft  enters a depot for SRP modifications, the Air Force may also schedule  Increment 3.1 modifications during that time. However, officials noted that  modernization efforts can be completed outside of the depot if needed.
One way the Air Force is attempting to address the depot maintenance  situation is by consolidating all F-22 depot maintenance at a government- operated facility in Ogden, Utah. According to an Air Force analysis of F- 22 depot operations, management turnover at the contractor-run depot in  Palmdale and additional workload to address aircraft corrosion issues  contributed to this depot’s delays in delivering aircraft back to the fleet.
The Palmdale facility has higher labor rates than the Ogden facility and  has charged more labor hours than the Ogden facility when performing  modifications. The Air Force analysis also found that consolidating all F- 22 depot maintenance at the Ogden facility should provide better  schedule performance while reducing costs for F-22 depot maintenance  by approximately $20 million per year. The Air Force plans to mitigate  schedule risks related to depot consolidation by retaining a residual  capacity at Palmdale into 2015.
section_title: Despite Efforts to Improve Reliability and Maintainability, the F-22 Fleet Will Not Meet Its Availability Requirement
paragraphs: The F-22 fleet will not achieve its availability requirement as scheduled  despite substantial investments in RAMMP projects. RAMMP is intended  to increase the availability of the F-22 to perform its missions, primarily  through improving reliability—the likelihood that the aircraft will operate  without failure—and reducing the time required to maintain the aircraft.  Cumulative spending on RAMMP projects totals about $900 million  through the end of fiscal year 2013, and has had some positive effect on  availability over time. However, the Air Force has never been able to  meet the F-22’s aircraft availability requirement and does not expect to  meet that requirement within the next 4 years. The original requirement  was expressed in average hours of flying time between maintenance  events for the F-22 fleet, excluding routine servicing and inspections, and  was set at an average of 3.0 hours. The program was not able to meet  that requirement, and it was officially changed to a requirement for overall  aircraft availability, set at 70.6 percent materiel availability. This is defined  as the percentage of the fleet operationally capable of performing an  assigned mission at any given time. The level of availability is required to  build over time, from 61.2 percent by the end of fiscal year 2011 to 70.6  percent by the end of fiscal year 2015. This requirement was established  to support readiness requirements for projected training and contingency  operations. As shown in figure 2, the Air Force has consistently fallen  short of its availability requirements. Program projections indicate that it  will not achieve 70.6 percent availability by fiscal year 2018.
Officials from Air Combat Command, which is involved in developing  requirements for the F-22, said that the current materiel availability  requirement is based on assumptions that may no longer be applicable.  According to Air Force guidance on calculating availability requirements,  these assumptions include maintenance needs and the frequency and  duration of flights. Command officials noted that they expect to revisit this  requirement at some future point with the DOD office that approves F-22  operational requirements to discuss changing the requirement to reflect  operational realities, but they did not tell us specifically when they plan to  do such a reassessment.projects aimed at achieving the current requirement may be unnecessary   Continued investment in some RAMMP  if it is no longer considered valid. Moreover, any delay in revisiting this  requirement could limit funding that DOD might be able to make available  for other high priority activities in a time of austere federal budgets.
section_title: Conclusions
paragraphs: The Air Force’s decision to structure F-22 modernization efforts as distinct  incremental programs should increase transparency and provide greater  visibility into cost, schedule and performance. It is consistent with both  GAO’s prior recommendation and statutory direction. However,  opportunities remain for the Air Force to achieve greater transparency. In  particular, the first annual report to Congress on F-22 improvement efforts  lacked full baseline cost and schedule estimates for RAMMP projects. In  addition, the report did not include operations and maintenance funding  associated with such projects. Rather, it provided development and  procurement funding projections through fiscal year 2017, although the  program is expected to continue through the life of the aircraft. A more  comprehensive baseline estimate for RAMMP would make it easier to  track changes in cost and schedule, supporting better program oversight.
Although the Air Force has made substantial investment in RAMMP  projects and seen some improvement in aircraft availability, the F-22  program has not been able to achieve its availability requirements. If, as  stated by Air Force officials, the current availability requirement is based  on assumptions that are no longer valid, then spending on some RAMMP  projects could be unnecessary. Promptly revisiting this requirement would  help minimize any such unnecessary investment of future funding.
section_title: Recommendations for Executive Action
paragraphs: To support program oversight and ensure efficient use of future funding,  we recommend that the Secretary of Defense direct the Secretary of the  Air Force to take the following two actions:  Incorporate into the Air Force’s annual report to Congress on F-22  improvement projects a comprehensive cost and schedule baseline  estimate for RAMMP that includes development, procurement, and  operations and maintenance costs through the expected life cycle of  the fleet.
Expedite reassessment of the F-22 materiel availability requirement  and determine the necessary changes, if any, to the number and  scope of RAMMP projects.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this product to DOD for comment. In its written  comments, reprinted in appendix II, DOD disagreed with one  recommendation and agreed with the other. DOD also provided technical  comments, which were incorporated as appropriate.
DOD did not agree with our first recommendation to incorporate a  comprehensive cost and schedule baseline estimate for RAMMP that  includes development, procurement, and operations and maintenance  costs through the expected life cycle of the fleet into the Air Force’s  annual report to Congress on F-22 improvement projects. DOD’s  response stated that reliability and maintainability programs cannot be  baselined like major defense acquisition programs because of cost  fluctuations based on life cycle issues that arise as the weapon system  ages. We recognize that RAMMP is not a major defense acquisition  program and did not intend to imply that it was, although we believe that  DOD guidance for program cost and schedule baselines provides  elements of sound baseline estimates that could be used in relation to  RAMMP. We acknowledge in our report the challenges of accurately  estimating future RAMMP costs. Nevertheless, the 2013 NDAA required  comparison of costs and schedules to an appropriate baseline, and the  Air Force’s April 2013 report did not identify a cost or schedule baseline  for RAMMP. Rather, it used the funding levels requested in the fiscal year  2013 President’s Budget submission. Continuing to use this approach will  not provide visibility into the extent to which RAMMP projects are being  executed efficiently. As noted in our report, program officials expect to  issue an updated cost estimate for F-22 modernization and improvement  efforts later in 2014. That updated estimate is expected to include cost  projections through the life of the aircraft, and to the extent that it  incorporates RAMMP costs, it could provide a starting point for a baseline  estimate to be included in future reports to Congress. The Air Force’s  April 2013 report states that the way it is structuring and managing  RAMMP helps the Air Force get the most value for its money while  making judicious use of its time. Inclusion of appropriate cost and  schedule baseline estimates would support both goals. Therefore, we  continue to believe the recommendation is valid.
DOD agreed with our second recommendation to expedite reassessment  of the F-22 materiel availability requirement and determine any necessary  changes to the number and scope of RAMMP projects. DOD’s response  stated that the Air Force assesses weapon system materiel availability  annually during formulation of flying hour programs to sustain required  levels of readiness, and added that the Air Force will continue to reassess  the F-22 materiel availability requirement and adjust RAMMP priorities  accordingly. While DOD’s commitment to regularly reassessing the F-22  materiel availability requirement is commendable, the intent of our  recommendation is that DOD should formally reassess the requirement  with the Joint Requirements Oversight Council—the DOD office that  approves key weapon system requirements. As we noted in our report,  Air Combat Command officials stated that they intend to do this at some  future point, and our recommendation is simply that they do so as  expeditiously as possible.
We are sending copies of this report to interested congressional  committees, the Secretary of Defense, and the Secretary of the Air Force.  In addition, the report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: We were asked to review and report on the F-22 modernization and  improvement efforts. This report assesses: (1) the Air Force’s new  approach to structuring its F-22 modernization efforts, (2) the Air Force’s  management of its F-22 reliability and structural improvement efforts, and  (3) challenges facing efforts to modernize the F-22 and address reliability  and structural problems moving forward.
To assess the Air Force’s new approach to structuring modernization  efforts, we analyzed actual and projected program management data for  fiscal years 2003 to 2023, from sources including acquisition decision  memorandums, selected acquisition reports, and independent and  program cost estimates. These sources were issued in 2012 or 2013. We  also reviewed contracting and budget documents relevant to F-22  modernization. To assess the Air Force’s management of its F-22  reliability and structural improvement efforts, we collected and analyzed  Air Force reports and briefing materials on aircraft availability and  reliability and repair projects. To assess challenges facing efforts to  modernize the F-22 and address reliability and structural problems  moving forward, we analyzed program schedules and actual and  projected aircraft availability data for fiscal years 2011 to 2018, current as  of August 2013, and reviewed program documents discussing  maintenance issues affecting the aircraft.
In performing our work, we obtained information and interviewed officials  from the F-22 System Program Office at Wright-Patterson Air Force Base,  Ohio, and Hill Air Force Base, Utah; Air Combat Command, Joint Base  Langley-Eustis, Virginia; Office of the Secretary of Defense, Arlington,  Virginia; and the Defense Contract Management Agency, Fort Worth,  Texas.
We assessed the reliability of DOD data on selected acquisition reports,  cost estimates, and materiel availability rates and projections by  reviewing documentation related to the data and interviewing agency  officials knowledgeable about the data. We determined that the data were  sufficiently reliable for the purposes of this report.
We conducted this performance audit from July 2013 to May 2014 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
paragraphs: Michael J. Sullivan, (202) 512-4841 or sullivanm@gao.gov.
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, key contributors to this report  were Travis J. Masters, Assistant Director; Marie P. Ahearn; Robert P.  Bullock; Laura Greifner; Kristine R. Hassinger; Roxanna T. Sun; and  Christopher D. Zbrozek.