Joint Strike Fighter: Accelerating Procurement before Completing Development Increases the Government's Financial Risk
Published: 2009-03-12T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Joint Strike Fighter (JSF) is the Department of Defense's (DOD) most complex and ambitious aircraft acquisition, seeking to simultaneously produce and field three different versions of the aircraft for the Air Force, Navy, Marine Corps, and eight international partners. The total investment required now exceeds $1 trillion--more than $300 billion to acquire 2,456 aircraft and $760 billion in life cycle operating and support costs, according to program estimates. The Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 requires GAO to review the JSF program annually for 5 years. This is the fifth and final report under the mandate in which GAO (1) determines the program's progress in meeting cost, schedule, and performance goals; (2) assesses manufacturing results and schedule risks; and (3) evaluates development test plans, progress, and risks. GAO's work included analyses of a wide range of program documents, cost data and interviews with defense and contractor officials.
section_title: What GAO Found
paragraphs: JSF development will cost more and take longer than reported to the Congress last year, and DOD wants to accelerate procurement. Two recent estimates project additional costs ranging from $2.4 billion to $7.4 billion and 1 to 3 more years to complete development. Despite cost and schedule troubles, DOD wants to accelerate JSF procurement by 169 aircraft from fiscal years 2010 through 2015; this could require up to $33.4 billion in additional procurement funding for those 6 years. DOD plans to procure hundreds of aircraft on cost-reimbursement contracts, magnifying the financial risk to the government. Ongoing manufacturing inefficiencies and parts problems have significantly delayed the delivery of test assets. The prime contractor has extended manufacturing schedules three times and delivered 2 of 13 test aircraft. The program is still recovering from earlier problems that resulted in design changes, late parts deliveries, and inefficient manufacturing. The contractor is taking positive steps to improve operations, the supplier base, and schedule management. Schedule risk analyses could further enhance management insight into problem areas and inform corrective actions. Officials expect to deliver all test aircraft and fix many problems by 2010. By then, DOD plans to have purchased 62 operational aircraft and will be ramping up procurement. Procuring large numbers of production jets while still working to deliver test jets and mature manufacturing processes does not seem prudent, and looming plans to accelerate procurement will be difficult to achieve cost effectively. DOD's revised test plan adds a year to the schedule, better aligns resources and availability dates, and lessens the overlap between development and operational testing, but it still allows little time for error discovery and rework. DOD's decision late in 2007 to reduce test aircraft and flight tests adds to risks while any additional delays in delivering test aircraft will further compress the schedule. The revised plan relies on state-of-the-art simulation labs, a flying test bed, and desk studies to verify nearly 83 percent of JSF capabilities. Only 17 percent is to be verified through flight testing. Despite advances, the ability to so extensively substitute for flight testing has not yet been demonstrated. Significant overlap of development, test, and procurement results in DOD making substantial investments before flight testing proves that the JSF will perform as expected. Under the accelerated procurement plan, DOD may procure 360 aircraft costing an estimated $57 billion before completing development flight testing.

section_title: Letter
paragraphs: The Joint Strike Fighter (JSF) is the Department of Defense’s (DOD) most  complex and ambitious aircraft acquisition, seeking to simultaneously  produce and field three aircraft variants for the Air Force, Navy, Marine  Corps, and eight international partners. The JSF is critical to our nation’s  plans for recapitalizing the tactical air forces and will require a long-term  commitment to very large annual funding outlays. The total expected  investment is now more than $1 trillion—more than $300 billion to acquire  2,456 aircraft and $760 billion in life cycle operation and support costs,  according to official program estimates.
The Ronald W. Reagan National Defense Authorization Act for Fiscal Year  2005 requires GAO to review the JSF program annually for 5 years.  Previous reports identified opportunities for the program to reduce risks  and improve chances for more successful outcomes. We have expressed  concern about the substantial overlap of development, test, and  production activities and recommended a more evolutionary and  knowledge-based acquisition strategy with limited investment in  production aircraft until each variant demonstrates required capabilities in  flight testing. Our most recent report questioned DOD’s decision to reduce  test aircraft and flight hours and recommended that a new comprehensive  cost estimate be prepared. The department has not implemented our  recommendations and cost and schedule increases have been the result.  This is the fifth and final report under the mandate in which we  (1) determine the JSF program’s progress in meeting cost, schedule, and  performance goals; (2) assess plans and risks in manufacturing and  capacity to accelerate production; and (3) evaluate plans, progress, and  risks with testing plans and risks in testing activities.
To conduct this work, we tracked and compared current cost and  schedule estimates with those of prior years, identified changes, and  determined causes. We obtained program status reports, manufacturing  data, and test planning documents. We assessed the program’s scheduling  estimates against best practices. We discussed results to date and future  plans to complete JSF development and accelerate procurement with  DOD, JSF, and contractor officials. Some of the cost data used in our  report are based on cost projections that were current at the time of our  review rather than the official program of record. Appendix I includes  additional information about our scope and methodology. We conducted  this performance audit from June 2008 to March 2009 in accordance with  generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on our  audit objectives.
subsections: section_title: Background
paragraphs: The JSF is a joint, multinational acquisition program for the Air Force,  Navy, and Marine Corps and eight international partners. The program  began in November 1996 with a 5-year competition between Lockheed  Martin and Boeing to determine the most capable and affordable  preliminary aircraft design. Lockheed Martin won the competition, and the  program entered system development and demonstration in October 2001.  Program goals are to develop and field an affordable, highly common  family of stealthy, next-generation strike fighter aircraft for the Navy, Air  Force, Marine Corps, and U.S. allies.
The JSF is a single-seat, single-engine aircraft, designed to rapidly  transition between air-to-ground and air-to-air missions while still  airborne. To achieve its mission, the JSF will incorporate low-observable  (stealth) technologies, defensive avionics, advanced onboard and offboard  sensor fusion, internal and external weapons, and advanced prognostic  maintenance capability. The JSF family consists of three variants. The  conventional takeoff and landing (CTOL) variant will primarily be an air- to-ground replacement for the Air Force’s F-16 Falcon and the A-10  Warthog aircraft, and will complement the F-22A Raptor. The short takeoff  and vertical landing (STOVL) variant will be a multi-role strike fighter to  replace the Marine Corps’ F/A-18C/D Hornet and AV-8B Harrier aircraft.  The carrier-suitable variant will provide the Navy a multi-role, stealthy  strike aircraft to complement the F/A-18E/F Super Hornet. DOD is  planning to buy a total of 2,456 JSFs and allies are expected to procure a  minimum of 730 CTOL and STOVL aircraft.
Because of the program’s sheer size and the numbers of aircraft it will  replace, the JSF is the linchpin of DOD’s long-term plan to modernize  tactical air forces. It is DOD’s largest acquisition program, with total cost  currently estimated at $300 billion, and the longest in planned duration,  with procurement projected through 2034. In addition, the JSF remains  DOD’s largest cooperative program. Our international partners are  providing about $4.8 billion toward development, and foreign firms are  part of the industrial base producing aircraft. DOD’s funding requirements  for the JSF assume economic benefits from these foreign purchases in  reducing unit costs for U.S aircraft.
Table 1 shows the evolution of DOD’s official estimated cost, quantity, and  deliveries from the initiation of system development in October 2001 to the  current official program of record dated December 2007 and submitted to  the Congress in April 2008. It depicts quantities reduced in the last major  program restructure in 2004, the impacts of increased costs on unit prices,  and the slip in delivering initial operating capability to the warfighter.
In our March 2008 report, we stated that JSF costs would likely grow  much higher than reported because the program of record at that time did  not include all acquisition costs (including the alternate engine program  directed by the Congress), made overoptimistic assumptions, and did not  fully reflect the mounting cost and schedule pressures from manufacturing  inefficiencies and compressed time frames for completing development.  We questioned the Mid-Course Risk Reduction Plan adopted by DOD in  September 2007 that cut two development test aircraft, reduced test  flights, and accelerated the reduction in the prime contractor’s  development workforce in order to replenish management reserves  depleted by design changes and manufacturing problems. We  recommended that DOD accomplish a full and comprehensive estimate of  the total program verified by an independent third party and revisit the  Mid-Course Risk Reduction Plan with an intensive analysis of the causes  of management reserve depletion, progress against the baseline  manufacturing schedule, and correction of deficiencies in the contractor’s  earned value management system. DOD agreed to make a comprehensive  independent cost estimate, but decided to go ahead as planned with the  Mid-Course Risk Reduction Plan, stating that it would monitor and  evaluate progress and revise the plan later if it failed to achieve  expectations.
section_title: More Money and Time Will Be Needed to Complete JSF Development, While DOD Plans to Accelerate Procurement
paragraphs: Two recent estimates indicate that JSF development will cost more and  take longer to complete than reported to the Congress in April 2008,  primarily because of contract cost overruns and extended time to  complete flight testing. DOD also plans to accelerate aircraft procurement  over the next 6 years—buying more aircraft sooner than planned last year.  This new plan will require significantly more procurement funding sooner,  but officials did not assess its net effect on total program costs through  completion of JSF acquisition.
subsections: section_title: New Estimates Project Rising Costs and Further Delays to Complete JSF Development
paragraphs: Development costs are projected to increase between $2.4 billion and   $7.4 billion and the schedule for completing system development extended  from 1 to 3 years, according to recent estimates—one by the JSF Program  Office and one by a joint team of Office of the Secretary of Defense (OSD),  Air Force, and Navy officials. Cost overruns on both the aircraft and  engine contracts, delays in manufacturing test aircraft, and a need for a  longer, more robust flight test program are the primary cost drivers. The  joint team’s estimate is higher than the program office’s because it  included costs for the alternate engine program directed by the Congress  and used more conservative assumptions based on current and legacy  aircraft experiences. Program officials contend that funding the program  to the higher cost estimate is premature and believe processes are in place  to substantially improve on the test experiences of past programs.  Regardless, both estimates agree that cost and time to complete  development have increased from the official program of record at the  time of our review. (See table 2.)
The program office’s revised development cost estimate projects an  additional $2.4 billion and a 1-year extension in the schedule compared to  the official program of record reported to the Congress. This would  increase the system development portion of the acquisition program to  $46.8 billion and delay its completion to October 2014. The cost increases  primarily resulted from the following factors.    $1.2 billion for aircraft development. Program officials declared a cost  overrun on the prime air system contract because of increased labor  hours, higher prices, and supply shortages. Included in this figure is  $200 million to be added to the contractor’s management reserve. Last  year, we reported that mounting cost and schedule pressures depleted  reserves much faster than anticipated. By summer 2007, the program  had spent about two-thirds of budgeted funds but had accomplished  only half the work required. Since then, DOD’s efforts to restore  reserves and fix manufacturing inefficiencies have not fully achieved  intended results, requiring another cash infusion.   $800 million for engine development. According to officials from the  Defense Contract Management Agency, the engine contractor  continued to face development problems, which resulted in a contract  cost overrun. Higher costs for labor and materials, supplier problems,  and the rework needed to correct deficiencies with the engine blade  design discovered during ground testing were major contributing  factors.   $300 million for flight test extension. The program extended system  development by 1 year to provide more time for development and  operational flight testing. In April 2008, an operational test review team  recommended extending the development contract by 1 year. The  review team considered but dismissed several other options to address  the schedule problem, including deferring requirements.
On the other hand, the joint estimating team estimates that it will cost  $14.8 billion to complete JSF development, $7.4 billion more than the  official program of record at the time of our review. This would increase  the total development costs to $51.8 billion from the $44.4 billion reported  to the Congress last year—a 17 percent increase. The joint team also  projected a 3-year program extension beyond the program of record in  order to complete system development, 2 years more than the new  program office estimate.
The joint team’s estimate was $5 billion more than the new program office  estimate. Several factors account for the difference between the two  estimates.
Alternate engine. The joint estimating team included $1.4 billion to  complete development of an alternate engine for the JSF; the program  office estimate did not include alternate engine costs. The Congress  has directed DOD to develop a second source for the JSF engine to  induce competition and to ensure that one engine’s failures would not  ground all JSFs, thereby reducing operational risks in the future. DOD  has not wanted to pursue this second engine source and twice removed  funding from the JSF program line.
Engineering staffing. The joint estimating team projected a need for  the contractor to retain considerably more engineering staff and for  longer periods of time than the program office estimate to complete  development, evaluate test results, and correct deficiencies. Releasing  engineering staff prematurely risks not discovering problems until late  in development or during fielding, when they would be more expensive  to address.
Software development. The joint estimating team believes that the  software productivity rate will be less than the program’s calculation  and anticipates much more growth in software requirements. The JSF  aircraft is expected to require 7.5 million lines of computer code—the  most by far of any aircraft. By comparison, the F/A-18E/F has only 1.1  million and the F-22A 2.2 million. Experiences on past acquisitions  have shown 30 to 100 percent growth in software requirements over  time, while the JSF Program Office estimate assumed no growth.
Flight testing. The joint estimating team projects that flight testing will  require more time and effort than the program office has built into its  current schedule. Continuing delays in delivering test aircraft are  expected to hamper and further compress test schedules. In particular,  the joint team projects that the two aircraft dedicated to carrier  suitability tests will be late off the production line, thereby delaying  test activities. It also projected that the JSF will require about 2,700  hours of flight testing for mission systems, significantly more than the  1,700 hours that the program office currently estimates.
Manufacturing production hours. The joint estimating team projects  that production span times for the JSF will be longer than the program  office estimates based on the program’s performance to date and  experience of recent programs, such as the F/A-18 E/F and the F-22A.  The span time is an indicator of how long the manufacturing effort  takes and when flight testing can begin. The program office assumes  that span times will decrease over the course of the development  contract. We note that span times typically increase during  development, as was the case for both the B-2 bomber and F-22A  programs.
Program officials believe that their estimate is more accurate and that  providing extra funding to address future risks is premature and does not  provide incentives for contractors to control costs. The program office  attributes its lower cost estimate to several factors. First, the quantity,  quality, and flexibility of the JSF laboratories should enable the program  to reduce more risks in a laboratory environment, rather than through  flight testing, which is considerably more expensive. In addition, the  program’s efforts to develop the final software system infrastructure early  should reduce significant software problems later in the program,  according to the program office. The program office also believes that  costs will be lower because progress in several key development areas is  either matching plans or is ahead of where legacy programs were at  similar points of time in their development. For example, the program is  currently reducing engineering staff as planned. The program is also  producing software at a rate significantly higher than that of the F-22A  program and is at least 18 months ahead of where the F-22A program was  at a similar point in developing mission systems, according to program  officials. Officials told us that they intend to fund the fiscal year 2010  development budget based on the joint team’s higher estimate. However, it  is not clear at this stage which estimate will serve as the basis for future  budget submissions.
section_title: Much Higher Annual Procurement Funding Required to Accelerate JSF Procurement
paragraphs: The program office and joint estimating team also projected procurement  funding requirements for the 6-year period fiscal years 2010-2015 based on  DOD plans to accelerate procurement of operational aircraft. Through this  effort, DOD wants to recapitalize tactical air forces sooner and mitigate  projected fighter shortfalls in the future. Compared to last year, this  accelerated plan would procure an additional 169 aircraft during these 6  years, moving aircraft that had been scheduled for procurement beyond  2015 to earlier years. According to the two estimates, this plan would  require from $21.8 billion to $33.4 billion more funding than the official  program of record, as shown in table 3.
The joint team’s estimate is higher than the program office’s, primarily for  these reasons:    The joint team projected slower gains in production efficiency than the  program office. Typically, production efficiency is improved and unit  costs are lowered over time as a workforce becomes more experienced  building a new product and manufacturing processes are honed.
The joint team also assumed fewer savings from commonality.
Commonality—a key selling point for the JSF program—refers to the  use of the same or similar parts, structures, and subsystems shared by  the three variants. Greater commonality can save money by decreasing  development times and facilitating economic order quantities.
The team projected higher labor and material costs and longer  production span times, based on JSF performance to date in  manufacturing development test aircraft.
Table 4 shows the additional aircraft and funding requirements for DOD’s  accelerated plan compared to the official program of record. These  quantities are for the United States only; during this same period, the  international partners are expected to buy 273 aircraft.
section_title: Procurement of Operational Aircraft under Cost Reimbursement Contracts to Continue; Increases the Government’s Exposure to Risks
paragraphs: The JSF program is procuring a substantial number of aircraft on cost  reimbursement contracts. Cost contracts place most of the risk on the  buyer—DOD in this case—who is liable to pay more than budgeted should  labor, material, or other incurred costs be more than expected when the  contract was signed. JSF officials plan to procure at least the first four  low-rate production lots under cost reimbursement contracts and to  transition to fixed-price instruments when appropriate, possibly between  lots 5 and 7 (fiscal years 2011 to 2013). It is unclear exactly how and when  this will happen, but the expectation is to transition to fixed pricing once  the air vehicle has a mature design, has been demonstrated in flight tests,  and is producible at established cost targets. To date, DOD has procured  the first three lots for a total of 30 aircraft and $7.4 billion on cost  reimbursement terms. Under the accelerated procurement plan, DOD  could procure as many as 360 aircraft costing about $57 billion through  fiscal year 2013 on cost reimbursement contracts, as illustrated in figure 1.
Cost reimbursement contracts provide for payment of allowable incurred  costs, to the extent prescribed in the contract. According to the Federal  Acquisition Regulation, cost reimbursement contracts are suitable for use  only when uncertainties involved in contract performance do not permit  costs to be estimated with sufficient accuracy to use any type of fixed- price contract. Cost reimbursement contracts for weapon production are  considered appropriate when the program lacks sufficient knowledge  about system design, manufacturing processes, and testing results to  establish firm prices and delivery dates. In contrast, a fixed-price contract  provides for a pre-established price, places more of the risk and  responsibility for costs on the contractor, and provides more incentive for  efficient and economical performance.
Procuring up to 360 production aircraft on cost reimbursement  contracts—-nearly 15 percent of the total DOD program—seems to be a  tacit acknowledgment by DOD and the contractor that knowledge on JSF  design, production processes, and costs for labor and material is not yet  sufficiently mature and that pricing information is not exact enough for  the contractor to assume the risk under a fixed-price contract. It also  seems to be a consequence of the substantial concurrency of development,  test, and production built into the JSF schedule. Significant overlap of  these activities means that DOD is procuring considerable quantities of  operational aircraft while development test aircraft are still on the  production line and much in advance of testing to prove aircraft  performance and suitability. Establishing a clear and accountable path to  ensure that the contractor assumes more of the risk is prudent. We note  that the significant ramp up in JSF production under the accelerated  profile starts with lot 5, the fiscal year 2011 procurement of 70 aircraft.
section_title: Continued Manufacturing Inefficiencies Will Make It Difficult for the Program to Meet Its Production Schedule
paragraphs: Manufacturing of development test aircraft is taking more time, money,  and effort than planned, but officials believe that they can work through  these problems and deliver the 11 remaining test aircraft by early 2010.  However, by that time, DOD may have procured as many as 62 production  aircraft, accumulating a backlog of aircraft to be produced. Manufacturing  inefficiencies and parts shortages continue to delay the completion and  delivery of development test aircraft needed for flight testing. The  contractor has not yet demonstrated mature manufacturing processes, or  an ability to produce at currently planned annual rates. It has taken steps  to improve manufacturing, the supplier base, and schedule management.  However, given the manufacturing challenges, we believe that DOD’s plan  to accelerate production in the near term adds considerable risk and will  be difficult to achieve in a cost-effective manner.
subsections: section_title: Time and Money Needed for Manufacturing Development Test Aircraft Continue to Increase
paragraphs: The prime contractor has restructured the JSF manufacturing schedule  three times, each time lengthening the time to deliver aircraft to the test  program. Delays and manufacturing inefficiencies are prime causes of  contract cost overruns. The contractor has produced two development  test aircraft—an original non–production representative model and the  first STOVL aircraft. It now projects delivering the remaining 11 aircraft in  2009 and early 2010. Problems and delays are largely the residual effects  from difficulties early in development. The effects of the late release of  engineering drawings, design changes, delays in establishing a supplier  base, and parts shortages continue to cause delays and force inefficient  production line work-arounds where unfinished work is completed out of  station. Data provided by the Defense Contract Management Agency and  the JSF Program Office show continuing critical parts shortages, out-of- station work, and quality issues. The total projected labor hours to  manufacture test aircraft increased by 40 percent just in the past year, as  illustrated in figure 2.
An efficient production line establishes an orderly flow of work as a product moves from  workstation to workstation and on to final assembly. Out-of-station work, sometimes  referred to as traveled work, refers to completing unfinished work on major components,  for example, the wings, after they have left the wing workstation and moved down the  production line to another station, such as mate and final assembly. decline in labor hours, indicating lesser gains in worker efficiency. As of  June 2008, the planned hours for these two major stations increased by  about 90 percent over the June 2007 schedule, which itself had shown an  increase from the 2006 schedule.
The overlap in the work schedule between manufacturing the wing and  mating (connecting) it to the aircraft fuselage has been a major concern  for several years because it causes inefficient out-of-station work. The  contractor continues to address this concern, but the new schedule  indicates that this problem will continue at least through 2009. One  indicator of its persistence is the projected hours for building the last test  aircraft. As figure 4 shows, estimated labor hours increased more than 80  percent from the June 2007 to June 2008 schedules, and the planned hours  for wing assembly and for the mate and delivery phases more than tripled.
section_title: Prime Contractor Actions to Improve Schedule Management, Manufacturing Efficiency, and Supplier Base
paragraphs: Our evaluation determined that the prime contractor now has good tools  and integrated processes in place that should improve its schedule  management activities and is also implementing actions to improve  manufacturing efficiency, the delivery of parts, and proactive oversight of  subcontractors. The effects from these recent actions are not yet fully  apparent, and the contractor has not yet accomplished its own schedule  risk assessment that could provide more insight into impacts from areas of  risk and uncertainty. The coming year will be critical for implementing  management improvements in order to accomplish a firm and effective  transition from manufacturing a few test aircraft to producing operational  aircraft at high annual rates.
subsections: section_title: Prime Contractor’s Scheduling Management Processes Meet Many Best Practices, but Program Risks Are Not Entirely Visible
paragraphs: The prime contractor demonstrated to us that its schedule management  processes largely meet established best practices criteria. With  improvements implemented in 2008, the contractor’s management systems  meet or partially meet eight of the nine established criteria. For example,  the master schedule can identify and track activities associated with over  600 projects. It also establishes the critical path between activities,  allowing the program to examine the impacts of schedule delays and  determine schedule flexibility. Appendix II discusses our examination of  the prime contractor’s schedule management process against best  practices criteria in more detail.
The one area not meeting best practices was related to performing a  schedule risk analysis, and as a result, the contractor has limited insight  into areas of risk and uncertainty in the schedule. The prime contractor  has not conducted its own risk assessment that would (1) determine the  level of confidence it has in meeting completion dates and (2) identify and  apportion reserve funds for contingencies. A thorough risk analysis could  improve management insight and subsequent corrective actions on two  recurring problem areas in particular: schedule slippage and inadequate  management reserve levels. Naval Air Systems Command officials did  accomplish an independent schedule risk analysis that indicated that the  program could slip more than 2 years based on the productivity risks  associated with software development and assembly of the various  airframes as well as the time needed to complete all flight testing. Both the  contractor and the JSF Program Office disputed the findings of the Naval  Air Systems Command schedule risk analysis primarily because the  analysis was done without direct involvement of program officials.
section_title: Improvements in Manufacturing and Supplier Base Are Ongoing
paragraphs: The prime contractor is implementing changes designed to address the  manufacturing inefficiencies and parts shortages discussed earlier. These  include (1) increasing oversight of key subcontractors that are having  problems, (2) securing long-term raw material purchase price agreements  for both the prime and key subcontractors, and (3) implementing better  manufacturing line processes. On this latter point, according to program  officials, the prime contractor has taken specific steps to improve wing  manufacturing performance—one of the most troublesome workstations.  Defense Contract Management Agency officials noted that the contractor  produced the second STOVL aircraft variant with less work performed out  of station than for the first STOVL aircraft. Also, program office and  contractor officials report some alleviation of parts shortages and  improvements in quality, but also believe that the effects from previous  design delays, parts shortages, and labor inefficiencies will continue to  persist over the near term.
The lag time between taking action and demonstrating improvement may  partly explain why some manufacturing performance metrics are not  demonstrating a clear continued rate of quality improvement, as would be  desirable and expected for a program ramping up annual production rates.  This lag time may be evident in two important metrics—scrap, rework,  and repair rates and manufacturing defect rates—both of which have  increased somewhat since 2006. Program and contracting officials point  out, however, that while this performance is not desirable, these and other  metrics compare very favorably with those of prior acquisitions at similar  stages of development, including the F-16 and F-22A.
Supplier costs are expected to make up an even more substantial share of  total expenses as the program moves further into production. According to  contractor officials, efforts are focused on maturing the supply base and  working more closely and directly with key suppliers to reduce costs,  alleviate parts shortages, and support higher production rates. Key  suppliers have struggled to develop critical and complex parts while  others have had problems with limited production capacity. For example,  the supplier responsible for the advanced electro-hydraulic actuation  system had delivered parts with missing subcomponents and parts that  were not built to specifications. The major “team mate” supplier  responsible for fuselage and tail assembly has experienced delays caused  by its limited machining capacity. Given these supplier issues,  manufacturing inefficiencies, and accumulating backlog in production, we  believe that the program’s plans to accelerate production in the near term  adds considerable risk and will be difficult to achieve in a cost-effective  manner.
section_title: The JSF’s Test Plan Is Improved but Faces Numerous Challenges to Complete Development on Time and on Budget
paragraphs: DOD will make significant investments—in both dollars and the number of  aircraft procured—before completing JSF flight testing. DOD’s proposal to  accelerate procurement further increases financial risks in a very  challenging test environment. DOD’s new test plan adds an extra year to  the schedule and better aligns resources, but is still aggressive with little  room for error, presenting a formidable challenge to complete system  development, support initial operational testing, and, eventually, a full rate  production decision. DOD decisions to reduce development test aircraft  and flight tests add to the risks, while any additional delays in  manufacturing test aircraft will further compress the schedule. The JSF  has just begun development flight testing with two test aircraft and has  already experienced some setbacks—normal in any program, but of  special concern when assets are minimal. Some in DOD forecast that  another 2 or more years beyond the 1-year extension just approved will  eventually be needed to successfully prove aircraft performance and  complete system development. The department has stated that the  contractor’s state-of-the-art ground test labs and a flying test bed will  mitigate risks in the flight regimen and their use will effectively substitute  for flight testing. This approach is promising, but not yet proven.
subsections: section_title: Significant Investments Made before Development Flight Tests Are Completed
paragraphs: DOD is investing heavily in procuring JSF aircraft before flight testing  proves that they will perform as expected. Procuring aircraft before  testing successfully demonstrates that the design is mature and that the  weapon system will work as intended increases the likelihood of design  and requirements changes resulting in subsequent cost growth, schedule  delays, and performance limitations. Also, systems already built and  fielded may later require substantial modifications, further adding to costs.  The uncertain environment as testing progresses is one reason why the  prime contractor and DOD are using cost reimbursable contracts until  rather late in procurement. Table 5 depicts planned investments—in both  dollars and aircraft—prior to the completion of development flight testing.  Under the accelerated production plan and using the lower procurement  cost estimate prepared by the program office, DOD may procure 360  aircraft at a total estimated cost of $57 billion before development flight  testing is completed. This overlap will be further exacerbated should the  joint estimating team’s predictions of higher cost and lengthier schedule  prove accurate.
section_title: Flight Testing Is Still in Its Infancy and Has Fallen Behind Schedule
paragraphs: The JSF program had completed about 2 percent of its development flight  testing as of November 2008. Figure 5 shows the expected ramp up in  flight testing with most effort occurring in fiscal years 2010 through 2012.  Past programs have shown that many problems are not discovered until  flight testing. As such, the program is likely to experience considerable  cost growth in the future as it steps up its flight testing, discovers new  problems, and makes the necessary technical and design corrections.
While the program has been able to demonstrate basic aircraft flying  capabilities, it has recently experienced testing delays and has fallen  behind the flight test plan established in 2007. At the time of our review,  the program had flown about half of its planned 155 flight tests for 2008.  The test program currently has two development test aircraft and an  integrated airborne test bed, with the following experiences to date:    Sixty-five test flights on the original, non–production representative  prototype contributed to discoveries in, among other things, landing  gear door fitting, aerial refueling operations, and weapons bay  functions. The prototype experienced a 3-month delay because of  engine bay nacelle vent fan malfunctions that were subsequently  resolved.
Initial testing of the first of 12 production representative prototypes  began in June 2008—a STOVL variant flown in conventional mode. By  the time of our review it accumulated 14 test flights demonstrating  important handling qualities and reducing risks associated with, among  other things, the landing gear, fuel system performance, and STOVL  doors operation. Engine problems delayed full STOVL testing by 6  months.
Thirty-seven flights on the cooperative avionics test bed tested mission  system software and demonstrated communication and navigation  capabilities.
Looking ahead, the program expects to take delivery of the remaining  development test aircraft during 2009 and early in fiscal year 2010. In the  same time frame, it also plans to begin flight testing 6 of its 12 production  representative prototype test aircraft (CTOL and STOVL aircraft),  including the first 2 aircraft dedicated to mission system testing. The first  carrier variant development test aircraft is expected to begin full flight  testing—including ship suitability testing—in 2010. A fully integrated,  mission capable aircraft is not expected to enter flight testing until 2012 by  which time DOD plans to have already purchased 241 aircraft for about   $43 billion under cost reimbursement contracts.
section_title: Program’s Test Plan Extends Development and Relies Heavily on Ground Testing and Simulations to Verify Aircraft Performance
paragraphs: The JSF Program Office developed a new and improved test plan in the  spring of 2008 that extended the development period by 1 year, better  aligned test resources and availability dates, and lessened the overlap  between development and operational testing. The new plan is still  aggressive, however, and has little room for error discovery, rework, and  recovery from downtime should test assets be grounded or otherwise  unavailable. The sheer complexity of the JSF program—with 7.5 million  lines of software code, three variants, and multi-mission development— suggests that the aircraft will encounter many unforeseen problems during  flight testing requiring additional schedule to rework. Our past work has  shown that programs that do not allow sufficient time to address the  inevitable problems discovered in testing, run a greater risk of significant  cost increases and schedule delays when problems do arise. The joint  estimating team noted that the program’s flight test plan assumed a higher  productivity than has been seen on recent flight test programs. As such,  the joint team believes that flight testing will require an additional 2 years  to complete beyond the 1 year already added to development and suggests  that more flight test hours will be necessary to test mission systems and  the carrier variant in particular.
The Mid-Course Risk Recovery Plan, approved in late 2007, cut two  development test aircraft, reduced test flights, and relies more on ground  laboratories and simulations to verify performance, adding substantial risk  to the program. Our 2008 report discussed the objections from several  prominent DOD offices to the Mid-Course Risk Recovery Plan. The  Director of Operational Test and Evaluation, for example, identified risks  in the revised verification strategy and cited inadequate capacity to handle  the pace of mission testing and ship suitability and signature testing. This  increases the likelihood of not finding and resolving critical design  deficiencies until operational testing, when it is more costly and disruptive  to do so.
The test plan relies heavily on a series of advanced and robust simulation  labs and a flying test bed to verify aircraft and subsystem performance.  Figure 6 shows that 83 percent of the aircraft’s capabilities are to be  verified through labs, the flying test bed, and subject matter analysis, while  only 17 percent of test points are to be verified through flight testing. The  JSF program spent $5 billion on its system of simulation labs and models.  Program officials argue that their heavy investment in simulation labs will  allow early risk reduction, thereby reducing the need for additional flight  testing, controlling costs, and meeting the key milestones of the program’s  aggressive test plan.
The JSF program’s simulation labs appear more prolific, integrated, and  capable than the labs used in past aircraft programs. The program utilizes  18 labs for development whereas only 9 were used for the F-22A, 7 for the  F-18, and 5 for the F-16. According to program officials, the greater  number of labs allows engineers to work simultaneously on different  development blocks, reducing bottlenecks that may occur in testing. In  contrast, engineers of the F-18 and F-22A programs had to interrupt or  shut down development on one development block while they were  making corrections to another. Also in contrast to past programs, key JSF  simulation labs are colocated at a Lockheed Martin plant in Fort Worth,  Texas. The F-22A program utilized three locations and two different  companies. According to the program office, the colocation of the key  testing labs facilitates more seamless integration of key aircraft  components. Program officials also noted that JSF labs use the actual  aircraft components to a greater extent than labs did in past programs and  also have greater software processing capacity. This allows for more  realistic data, which should reduce the need for additional flight testing.  Further, the JSF utilizes the first fully integrated airborne test bed for  mission system testing. According to program officials, the test bed’s  design is geospatially proportionate to an actual F-35 aircraft, enhancing  its ability to accurately verify aircraft performance.
While the labs appear more prolific, integrated, and complex than those  used in legacy programs, concerns about their extensive use in verifying  aircraft performance remain. The extent of the JSF program’s planned lab  use is unprecedented, but the ability to substitute for flight testing has not  yet been demonstrated. In addition, the labs have yet to be fully  accredited. Accreditation is required to ensure the adequate capability of  labs and models to perform verification tasks. It is critical that the models  behave like aircraft to ensure that the system’s performance requirements  are being verified accurately. The program office said that it is on track to  complete the accreditation of the labs in time to begin verifying system  performance. However, the Director of Operational Test and Evaluation  reports that the progress of the accreditation support packages is behind  schedule and suggests that more flight testing may be needed as the  accreditation process reveals the limitations of the models. Some DOD  officials are also concerned that the labs will be understaffed. The Director  of Operational Test and Evaluation and DOD’s joint estimating team both  reported that the program’s current resource plans reduce engineering  staff too rapidly. Engineering and test personnel are critical to analyzing  the data generated from the labs. Without adequate staff, there is a greater  risk that the labs will not be sufficiently utilized, which could, in turn,  result in schedule delays or cost increases.
section_title: Early Operational Assessment Identifies Several Challenges That May Have Operational Impact If Not Addressed
paragraphs: While the program is projecting that it will meet all key performance  parameters, most will not be verified through ground and flight testing  until fiscal years 2010 through 2013. In addition, a 2008 operational  assessment by the Air Force Operational Test and Evaluation Center  pointed out several technical challenges that it believes are likely to have a  severe operational impact if not adequately addressed. While some of the  report’s concerns are not specific requirements, some DOD officials  believe that the shortfalls may adversely affect the JSF’s ability to meet  warfighter needs. For example:    The current F-35 power system may cause excessive damage to runway  surfaces which could limit its ability to operate in certain locations.  The program is still evaluating the problem and plans to gather data  and conduct further studies when full-scale models or actual aircraft  are available. According to a program official, changes to the aircraft’s  design or to the current concept of operations may be needed. The  program has alerted the services and believes it will have a better  understanding of the problem sometime in 2009.
Thermal management challenges hamper the ability to conduct  missions in hot and cold environments. The Director of Operational  Test and Evaluation reported that an alternative main engine fuel pump  to remedy this problem is under development but will not be available  before the low-rate initial production Lot 3, which is likely to affect  operational testing. The test team aborted a test sortie because of high  fuel temperatures in June.
The JSF’s advanced integrated support system aims to improve and  streamline aircraft logistics and maintenance functions in order to  reduce life cycle costs. The current integrated support system for the  JSF prohibits operating two detachments from one squadron  simultaneously. This limitation will severely affect current operating  practices, which include dividing one squadron of aircraft into  subgroups to deploy and operate at different locations.
section_title: Conclusions
paragraphs: The JSF is DOD’s largest and most complex acquisition program and the  linchpin of efforts to recapitalize our tactical air forces. It is now in its  most challenging phase, at a crossroads of a sort. Challenges are many— continuing cost and schedule pressures; complex, extensive, and unproven  software requirements; and a nascent, very aggressive test program with  diminished flight test assets. Looking forward, the contractor plans to  complete work expeditiously to deliver the test assets, significantly step  up flight testing, begin verifying mission system capabilities, mature  manufacturing processes, and quickly ramp up production of operational  aircraft. As such, the credibility of the program’s test plans, manufacturing  capacity, and subsequent cost and schedule estimates should become  more apparent.
The program must move forward, but given all these challenges,  accelerating procurement in a cost-reimbursement contract  environment—where uncertainties in contract performance do not permit  costs to be estimated with sufficient accuracy to use any type of fixed- price contract—places very significant financial risk on the government.  Accelerating plans also does not equate to an ability to deliver to those  plans. Because the program’s manufacturing processes are still maturing  and flight testing is still in its infancy, incorporating an accelerated  production schedule introduces even more risk and uncertainty to the  program. Our past work has shown that programs that make production  decisions prior to fully proving a system’s design through testing and  demonstration of mature manufacturing processes have an increased risk  of design and production changes and retrofits of completed aircraft.  Until the contractor demonstrates that it can produce aircraft in a timely  and efficient manner, DOD cannot fully grasp future funding requirements.  DOD needs tangible assurance from the prime contractor that it can meet  expected development and production expectations. By accelerating   low-rate production quantities before manufacturing and testing processes  are mature, DOD accepts most of the contractors’ production and  manufacturing inefficiencies. At minimum, the contractor needs to  develop a detailed plan demonstrating how it can successfully meet  program development and production goals in the near future within cost  and schedule parameters. With an improved contracting framework and a  more reasoned look to the future, the JSF program can more effectively  meet DOD and warfighter needs in a constrained budget environment.
section_title: Recommendations for Executive Action
paragraphs: Given the program’s ongoing manufacturing problems and nascent flight  test program, we believe that moving forward with an accelerated  procurement plan is very risky. This risk is reflected by the extended use  of cost reimbursement contracts for low-rate production quantities—a  contract mechanism that places most of the cost risk on the government.
As such, to enhance congressional oversight, increase the likelihood of  more successful program outcomes, and maintain confidence that the  program is on track to meet planned cost, schedule, and performance  goals, we recommend that the Secretary of Defense take the following two  actions:  1.  Direct the Under Secretary of Defense for Acquisition, Technology and  Logistics to report to the congressional defense committees by  October 1, 2009.  This report should include, at minimum,    an explanation of the cost and other risks associated with a cost-  reimbursable contract as compared to a fixed-price contract for  JSF’s future low-rate production quantities, the program’s strategy for managing and mitigating risks associated  with the use of cost contracts, and   plans for transitioning to fixed-price contracts for production to  include time frames and criteria.  2.  Direct the JSF Program Office to ensure that the prime contractor  performs periodic schedule risk analyses for the JSF program to  provide better insight into management reserve, production  efficiencies, and schedule completion to allow for corrections as early  as possible.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided us with written comments on a draft of this report. The  comments are reprinted in appendix III.
DOD substantively agreed with our first recommendation regarding a  report to the Congress on contracting strategy, but believed that the Under  Secretary of Defense for Acquisition, Technology and Logistics should be  responsible for the report, not the JSF Program Office as stated in our  draft. As the milestone decision authority, the Under Secretary is  responsible for approving the contracting strategy, contract awards, and  the transition to full rate production. We agree with DOD and revised the  recommendation accordingly.
DOD also agreed with the second recommendation and will direct that the  prime contractor perform periodic schedule risk analysis. In coordination  with the JSF Program Office, the department intends to determine an  optimum schedule for the contractor that will provide insight into JSF cost  and schedule to influence key milestones and decision making.
We are sending copies of this report to the Secretary of Defense; the  Secretaries of the Air Force, Army, and Navy; and the Director of the  Office of Management and Budget. The report also is available at no  charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on the  last page of this report. Staff members making key contributions to this  report are listed in appendix IV.
section_title: Appendix I: Scope and Methodology
paragraphs: To determine the Joint Strike Fighter (JSF) program’s progress in meeting  cost, schedule, and performance goals, we received briefings by program  and contractor officials and reviewed financial management reports,  budget documents, annual selected acquisition reports, monthly status  reports, performance indicators, and other data. We compared reported  progress with prior years’ data, identified changes in cost and schedule,  and obtained officials’ reasons for these changes. We interviewed  Department of Defense (DOD), JSF Program Office, and contractor  officials to obtain their views on progress, ongoing concerns and actions  taken to address them, and future plans to complete JSF development and  accelerate procurement. This review was the fifth and final effort under  the mandate of the Ronald W. Reagan National Defense Authorization Act  for Fiscal Year 2005. We were provided sufficient information to make the  assessments contained in this report.
In assessing program cost estimates, we compared the official program  cost estimate in the December 31, 2007, selected acquisition report to  estimates developed by the JSF program and an independent joint  estimating team for fiscal years 2010 through 2015. Because the fiscal year  2010 budget had not been submitted to the Congress at the time of the  draft report, some of the report’s findings are based on preliminary cost  projections that existed at the time of our review rather than the official  program of record. As such, the cost projections in this report may be  different than the final fiscal year 2010 program of record. We interviewed  members of the joint estimating team to obtain a detailed understanding of  the methodology, data, and approach used in developing their cost  estimate and schedule risk analysis of the JSF program. We also  interviewed JSF program officials to understand the program’s cost  estimate methodology, assumptions, and results and to obtain their  response to the joint estimating team’s analysis. Based on this analysis, we  were able to identify significant differences in the cost estimating  methodologies and assumptions used by the joint estimating team and the  program office and to determine major risk drivers in the program.
To assess the program’s plans and risks in manufacturing and its capacity  to accelerate production from fiscal years 2010 through 2015, we analyzed  manufacturing cost and work performance data to assess progress against  plans. We compared budgeted program labor hours to actual labor hours,  identified growth trends, and noted differences between future labor  requirements and current plans to release engineering staff. We reviewed  data and briefings provided by the program and the Office of the Secretary  of Defense (OSD) to assess supplier performance and ability to support  accelerated production from fiscal years 2010 through 2015. We also  determined reasons for manufacturing delays, discussed program and  contractor plans to improve, and projected expected impacts on  development and operational tests. We also reviewed the prime  contractor’s schedule estimates and compared them with relevant best  practices to determine the extent to which they reflect key estimating  practices that are fundamental to having a reliable schedule. In doing so,  we interviewed cognizant program officials to discuss their use of best  practices in creating the program’s current schedule and interviewed  officials from Naval Air Systems Command to understand their approach  and to obtain results of their independent schedule risk analysis.
To assess plans, progress, and risks in test activities, we examined  program documents and interviewed DOD, program office, and contractor  officials about current test plans and progress. To assess progress toward  test plans, we compared the number of flight tests conducted as of  October 2008 to those in the original test plan established in 2007. We also  reviewed documents and interviewed prime contractor officials regarding  flight testing, the integrated airborne test bed, and ground testing. To  further assess the ground labs and test bed, we interviewed DOD and  program officials and toured the testing labs and aircraft at the Lockheed  Martin plant in Fort Worth, Texas.
In performing our work, we obtained information and interviewed officials  from the JSF Joint Program Office, Arlington, Virginia; Lockheed Martin  Aeronautics, Fort Worth, Texas; Defense Contract Management Agency,  Fort Worth, Texas; Defense Contract Management Agency, East Hartford,  Connecticut; Naval Air Systems Command, Patuxent River, Maryland; Air  Force Operational Test and Evaluation Center, Kirtland Air Force Base,  New Mexico; Air Force Cost Analysis Agency, Arlington, Virginia; and OSD  offices of the Under Secretary of Defense for Acquisition, Technology and  Logistics, the Director of Program Analysis and Evaluation and its Cost  Analysis Improvement Group, and the Director of Operational Test and  Evaluation in Washington, D.C.
We conducted this performance audit from June 2008 to March 2009 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix II: GAO Assessment of Prime Contractor Schedule Management Processes
subsections: section_title: Prime Contractor’s Scheduling Management Processes Meet Many Best Practices, but Program Risks Are Not Entirely Visible
paragraphs: The success of any program depends in part on having a reliable schedule  of when the program’s set of work activities will occur, how long they will  take, and how they are related to one another. As such, the schedule not  only provides a road map for systematic execution of a program, but also  provides the means by which to gauge progress, identify and address  potential problems, and promote accountability.
In general, best practices and related federal guidance call for a program  schedule to identify, sequence, integrate, and resource all key activities to  be performed, and to understand and proactively address activities that  pose critical risks. More specifically, our research has identified nine  practices associated with effective schedule estimating. These practices  are (1) capturing all activities, (2) sequencing all activities, (3) assigning  resources to all activities, (4) establishing the duration of all activities,   (5) integrating schedule activities horizontally and vertically,   (6) establishing the critical path for all activities, (7) identifying float  between activities, (8) conducting a schedule risk analysis, and   (9) updating the schedule using logic and durations to determine dates.
Of these nine practices, the JSF program either met or partially met eight  with only one not being met. The area that did not meet the practices  related to performing a schedule risk analysis. Specifically, the JSF  program has not conducted its own schedule risk analysis that would  determine the level of confidence it has in meeting completion dates.  Further, an assessment is also critical to identifying and apportioning  reserves for contingencies. Since the JSF program has not conducted its  own schedule risk analysis, it has limited insight into areas of risk and  uncertainty in the schedule. Naval Air Systems Command officials did  accomplish an independent schedule risk analysis that indicated that the  program could slip more than 2 years based on the productivity risks  associated with software development and assembly of the various  airframes as well as the time needed to complete all flight testing.
In addition to a schedule risk analysis not being performed, we found  several other schedule management concerns that further reduce the  visibility of manufacturing risks. First, the use of best scheduling practices  at the subcontractor level is still being developed, potentially affecting the  integration of subcontractor schedules into the integrated master  schedule. Integrating prime and subcontractor schedules is critical to  meeting program schedules and cost expectations. The prime contractor is  working with subcontractors to increase their level of schedule maturity.  Another area of concern is that out-of-station work made it difficult to  identify specific span times for individual manufacturing tasks. As a  result, the detailed information related to the manufacturing work was not  visible in the master schedule. Furthermore, because of the program’s  enormous size and complexity, the schedule has been difficult to maintain,  requiring manual validation processes to ensure its integrity and validity.  Ongoing JSF schedule validity will be an area that needs careful attention  as it represents a potential weak point in the overall implementation of the  integrated master schedule.
Despite this shortcoming, it is also important to recognize the significant  progress that the JSF program team has made in the area of schedule  management. Since the previous Defense Contract Management Agency  schedule review, both the schedule and the processes to manage it have  greatly improved. For example, the schedule can track and verify activities  associated with over 600 projects. It also successfully captures and  sequences key activities and establishes the critical path between key  activities, allowing the program to examine the impacts of schedule delays  and determine schedule flexibility.
subsections: section_title: The Sheer Size and Complexity of the JSF Schedule Have Created Major Challenges to Ensuring Schedule Integrity and Validity
paragraphs: The JSF schedule is maintained in Microsoft Project and consists of over  600 individual projects. Because the size and complexity of the schedule is  so immense, it has been difficult to maintain. As such, a number of manual  validation processes are required to ensure its integrity and validity. To its  credit, the contractor has developed custom processes and tools to help  manage the program schedule. However, because of its enormous size and  complexity, the JSF’s ongoing schedule validity will be an area that needs  careful attention as it represents a potential weak point in the overall  implementation of the integrated master schedule.
Because the schedule was so large, we reviewed a subset of it, focusing on  the delivery of one airframe for each variant of the F-35 being produced  (i.e., BF4, AF1, and AF3). This subset schedule covered a time span from  August 2006 through September 2014, and we analyzed it against our best  practices for effective schedule estimating. See table 6 for the results of  our analyses relative to each of the nine practices.
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact named above, the following staff members made  key contributions to this report: Bruce Fairbairn, Assistant Director; Ridge  Bowman; Charlie Shivers; Georgeann Higgins; Matt Lea; Karen Richey;  Tim Boatwright; and Greg Campbell.
section_title: Related GAO Products
paragraphs: Defense Acquisitions: Better Weapon Program Outcomes Require  Discipline, Accountability, and Fundamental Changes in the  Acquisition Environment. GAO-08-782T. Washington, D.C.: June 3, 2008.
Defense Acquisitions: Assessments of Selected Weapon Programs.  GAO-08-467SP. Washington, D.C.: March 31, 2008.
Joint Strike Fighter: Impact of Recent Decisions on Program Risks.  GAO-08-569T. Washington, D.C.: March 11, 2008.
Joint Strike Fighter: Recent Decisions by DOD Add to Program Risks.  GAO-08-388. Washington, D.C.: March 11, 2008.
Tactical Aircraft: DOD Needs a Joint and Integrated Investment  Strategy. GAO-07-415. Washington, D.C.: April 2, 2007.
Defense Acquisitions: Assessments of Selected Weapon Programs.  GAO-07-406SP. Washington, D.C.: March 30, 2007.
Defense Acquisitions: Analysis of Costs for the Joint Strike Fighter  Engine Program. GAO-07-656T. Washington, D.C.: March 22, 2007.
Joint Strike Fighter: Progress Made and Challenges Remain. GAO-07-360.  Washington, D.C.: March 15, 2007.
Tactical Aircraft: DOD’s Cancellation of the Joint Strike Fighter  Alternate Engine Program Was Not Based on a Comprehensive Analysis.   GAO-06-717R. Washington, D.C.: May 22, 2006.
Defense Acquisitions: Major Weapon Systems Continue to Experience  Cost and Schedule Problems under DOD’s Revised Policy. GAO-06-368.  Washington, D.C.: April 13, 2006.
Defense Acquisitions: Actions Needed to Get Better Results on Weapons  Systems Investments. GAO-06-585T. Washington, D.C.: April 5, 2006.
Tactical Aircraft: Recapitalization Goals Are Not Supported by  Knowledge-Based F-22A and JSF Business Cases. GAO-06-487T.  Washington, D.C.: March 16, 2006.
Joint Strike Fighter: DOD Plans to Enter Production before Testing  Demonstrates Acceptable Performance. GAO-06-356. Washington, D.C.:  March 15, 2006.
Joint Strike Fighter: Management of the Technology Transfer Process.  GAO-06-364. Washington, D.C.: March 14, 2006.
Tactical Aircraft: F/A-22 and JSF Acquisition Plans and Implications  for Tactical Aircraft Modernization. GAO-05-519T. Washington, D.C:  April 6, 2005.
Tactical Aircraft: Opportunity to Reduce Risks in the Joint Strike  Fighter Program with Different Acquisition Strategy. GAO-05-271.  Washington, D.C.: March 15, 2005.