Tactical Aircraft: Opportunity to Reduce Risks in the Joint Strike Fighter Program with Different Acquisition Strategy
Published: 2005-03-15T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Under the Ronald W. Reagan National Defense Authorization Act of 2005, GAO is required to to review the Joint Strike Fighter (JSF) program annually for the next 5 years. This is the first GAO report, and it (1) analyzes the JSF program's business case for delivering new capabilities to the warfighter and (2) determines whether the JSF program's acquisition strategy follows an evolutionary, knowledge-based approach. Also, the act requires GAO to certify whether we had access to sufficient information to make informed judgments on the matters contained in our report.
section_title: What GAO Found
paragraphs: GAO found that the original business case for the JSF program has proven to be unexecutable. DOD now plans to buy 535 fewer aircraft than originally planned. Due to increases in total program costs and program acquisition unit costs, the DOD has reduced buying power and is now buying fewer JSF's at a higher investment than originally planned. The first delivery of initial operational capabilities to the warfighter have been delayed 2 years so far. The program's current acquisition strategy does not fully follow the intent of DOD's evolutionary, knowledge-based acquisition policy that is based on best practices. An evolutionary, knowledge-based strategy will be necessary to successfully execute a new business case in the future. Instead, the program plans to concurrently develop the JSF technologies, integrate and demonstrate the expected product design, and produce deliverable fighters, which is a risky approach. Finally, as a result of a lengthy program replanning effort that had been in process during most of 2004, GAO did not have access to the cost estimate expected to be contained in the JSF's Selected Acquisition Report, to be delivered by Congress in the spring of 2005. At the time of GAO's review, JSF program officials were still collecting the necessary information to develop and complete the estimate. Therefore, GAO's review was limited to the estimated program costs contained in the December 31, 2003, Selected Acquisition Report.

section_title: Letter
paragraphs: The Joint Strike Fighter (JSF) program is the Department of Defense’s  (DOD) most costly aircraft acquisition. The program’s goals are to develop  and field more than 2,400 stealthy strike fighter aircraft for the Navy, Air  Force, and Marine Corps and potentially several hundred more aircraft for  U.S. allies. The JSF is intended to provide greater capability and to replace  DOD’s aging fighter and attack aircraft. DOD estimates that the total cost  to develop and procure its fleet of aircraft will reach $245 billion, with  total costs to maintain and operate the JSF adding another $344 billion  over its life cycle. This expense must be measured against other DOD and  national priorities as the government moves into lean budget years. The  JSF will be competing for a decreasing share of the federal budget  available for “discretionary” spending. This includes defense spending and  is in contrast to “mandatory” spending, such as Social Security and  Medicare/Medicaid. In fiscal year 2004, discretionary spending accounted  for about 39 percent of the federal budget. The Congressional Budget  Office projects that discretionary spending, as a percentage of the overall  budget, is likely to decrease in the future.
Since the program began, in November 1996, it has experienced technical  challenges that have resulted in significant cost increases and schedule  overruns. During most of 2004, program officials worked to understand  and define current development risks in order to prepare more accurate  cost and delivery estimates. The upcoming investment decisions to begin  manufacturing development prototypes and to begin long-lead funding for  production aircraft in 2006 will be prominent indicators of the risk DOD is  willing to accept as the JSF program moves forward and annual outlays  needed to support the program significantly increase.
The Ronald W. Reagan National Defense Authorization Act for Fiscal Year  2005 (P.L. 108-375) requires us to review the JSF program annually for the  next 5 years. This is our first report, and it (1) analyzes the JSF program’s  business case for delivering new capabilities to the warfighter and   (2) determines whether the JSF program’s acquisition strategy follows an  evolutionary, knowledge-based approach. The best practice is to establish  an incremental—or evolutionary—approach to meet these needs by  delivering increasingly better performance over time as funding and  technologies permit and provide specific knowledge about the system at  key decision points in the acquisition process.
The act also requires us to certify whether we had access to sufficient  information to make informed judgments on the matters contained in our  report. As a result of a lengthy program replanning effort that had been in  process during most of 2004, we did not have access to the cost estimate  expected to be contained in JSF’s Selected Acquisition Report, to be  delivered to Congress in the spring of 2005. At the time of our review, JSF  program officials were still collecting the necessary information to  develop and complete this estimate. Therefore, our review was limited to  the estimated program costs contained in the December 31, 2003, Selected  Acquisition Report. We did, however, have access to top-level program and  preliminary schedule information reflecting the status of the replanning  effort. Recognizing this limitation in scope, we did have access to  sufficient information to make informed judgments on the matters covered  in this report. We performed our work from June 2004 through March 2005  in accordance with generally accepted government auditing standards. For  more on our scope and methodology, see appendix II.
subsections: section_title: Background
paragraphs: JSF is a joint, multinational acquisition program for the Air Force, Navy,  Marine Corps, and eight cooperative international partners. The program  began in November 1996 with a 5-year competition between Lockheed  Martin and Boeing to determine the most capable and affordable  preliminary aircraft design. Lockheed Martin won the competition, and the  program entered system development and demonstration in October 2001.
The program’s objective is to develop and deploy a technically superior  and affordable fleet of aircraft that support the warfighter in performing a  wide range of missions in a variety of theaters. The single-seat, single- engine aircraft is being designed to be self-sufficient or part of a  multisystem and multiservice operation, and to rapidly transition between  air-to-surface and air-to-air missions while still airborne. To achieve its  mission, the JSF will incorporate low observable technologies, defensive  avionics, advanced onboard and offboard sensor fusion, and internal and  external weapons. The JSF aircraft design has three variants: conventional  takeoff and landing variant for the Air Force, aircraft carrier-suitable  variant for the Navy, and short takeoff and vertical landing variant for the  Marine Corps, the United Kingdom, and the Air Force. These aircraft are  intended to replace aging fighter and attack aircraft currently in the  inventory (see table 1).
In 2004, DOD extended the JSF program schedule to address problems  discovered during systems integration and the preliminary design review.  Design efforts revealed significant airframe weight problems that affected  the aircraft’s ability to meet key performance requirements. Software  development and integration also posed a significant development  challenge. Program officials delayed the critical design reviews, first  flights of development aircraft, and the low-rate initial production decision  to allow more time to mitigate design risk and gather more knowledge  before continuing to make major investments. As a result, the initial  operational capability date was delayed. DOD is in the process of  reestablishing resource levels needed to deliver capabilities, given current  and expected future conditions. The new business case will be presented  to the Office of the Secretary of Defense (OSD) decision makers this  spring.
A key to successful product development is the formulation of a business  case that matches requirements with resources—proven technologies,  sufficient engineering capabilities, time, and funding—-when undertaking  a new product development. First, the user’s needs must be accurately  defined, alternative approaches to satisfying these needs properly  analyzed, and quantities needed for the chosen system must be well  understood. The developed product must be producible at a cost that  matches the users’ expectations and budgetary resources. Finally, the  developer must have the resources to design and deliver the product with  the features that the customer wants and to deliver it when it is needed. If  the financial, material, and intellectual resources to develop the product  are not available, development does not go forward. If the business case  measures up, the organization commits to the development of the product,  including the financial investment. This calls for a realistic assessment of  risks and costs; doing otherwise undermines the intent of the business  case and invites failure. Program managers in organizations employing  best practices are incentivized to identify risk early, be intolerant of  unknowns, and be conservative in their estimates. Ultimately, preserving  the business case strengthens the ability of managers to say no to  pressures to accept high risks or unknowns.
section_title: More Resources Are Now Needed to Deliver Planned Capabilities
paragraphs: A key objective of the JSF acquisition program is to develop and produce  fighter aircraft with greater capabilities and lower acquisition and  ownership costs than previous fighter aircraft and to deliver the aircraft in  time to replace DOD’s aging fleet. However, since the program began in  1996, several program decisions have resulted in increased program costs,  reduced procurement quantities, and delayed delivery dates—making the  original business case unexecutable. Continued program uncertainties  about the aircraft redesign, software development, flight test program, and  procurement quantities make it difficult to estimate the total amount of  resources needed. Given the uncertainties, the program needs more time  to gain knowledge before committing to a new, more accurate business  case. The current pause to replan JSF development and production  provides the program this opportunity. Finally, frequent changes in JSF  program management, if continued, will compromise efforts to execute  the business case agreements.
subsections: section_title: JSF’s Original Business Case Is Unexecutable
paragraphs: Several significant changes to the JSF acquisition program have made  DOD’s original business case unexecutable. Purchase quantities have been  reduced by more than 500 aircraft, total program costs have increased by  about $12 billion, and delivery of the aircraft has been delayed by about 2  years (see table 2 and app. IV for more details). These changes have  effectively reduced DOD’s buying power for its investment, as it now plans  to buy fewer aircraft with a greater financial investment.
The JSF acquisition program’s estimated development and procurement  costs have increased. In addition, the number of aircraft it plans to deliver  has been reduced. As a result, unit costs for the JSF aircraft have  increased substantially, thereby reducing the program’s buying power. The  most significant quantity reduction occurred after system development  began in 2001, when the program reduced the number of aircraft it plans  to procure from 2,852 to 2,443, or by 14 percent. The Navy—concerned  that it could not afford the number of tactical aircraft it planned to  purchase—reduced the number of JSF aircraft for joint Navy and Marine  Corps operations from 1,089 to 680 by reducing the number of backup  aircraft needed. However, the Navy has not indicated to the developer the  exact mix of the carrier and short takeoff and vertical landing variants it  intends to purchase.
The cost estimate to fully develop the JSF has increased by over 80  percent. DOD expected that by using a joint development program for the  three variants instead of three separate programs, JSF development costs  could be cut by about 40 percent. However, cost increases have nearly  eroded all of the estimated savings. Development costs were originally  estimated at $24.8 billion. By the 2001 system development decision, these  costs had increased by $9.6 billion largely because of a 36-month schedule  extension to allow more time to mature the mission systems and a more  mature cost estimate.
By 2004, costs increased an additional $10.4 billion to $44.8 billion. The  program office cited several reasons, including efforts to achieve greater  international commonality, optimize engine interchangeability, refine the  estimating methodology, and extend the schedule for unexpected design  work. Almost half of this increase, $4.9 billion, was a result of an  approximately 18-month delay for unexpected design work caused by  increased aircraft weight that degraded the aircraft’s key performance  capabilities. Figure 1 compares the original and latest development cost  estimates.
Current estimates for the program acquisition unit cost are about $100  million, and the total estimated cost to own an aircraft over its life cycle is  $240 million—an increase of 23 percent and 11 percent, respectively. In  1996, the program established unit flyaway cost goals for each variant,  expecting the variants to have a high degree of commonality and to be  built on a common production line. However, commonality among the  variants has decreased, and the cost to produce the aircraft has increased  (see table 3). The unit flyaway cost for the conventional takeoff and  landing variant has increased by 42 percent; the cost for the short takeoff  and vertical landing variant has increased by a range of 37 to 55 percent;  and the cost for the carrier variant has increased by a range of 29 to 43  percent. According to program data, a large part of the cost increase since  the start of development can be attributed to labor costs for building the  airframe and to the costs for producing the complex mission systems.
With reduced quantities and increased program costs, the JSF program is  now buying fewer aircraft at a higher cost, thereby reducing the program’s  buying power. How effectively DOD manages its JSF funds will determine  whether it receives a good return on its investment. A sound and  executable business case is needed to effectively do this. Our reviews over  the past 20 years have consistently found that DOD’s weapon system  acquisitions take much longer and cost more than originally planned,  causing disruptions and increasing pressures to make unplanned trade- offs to accommodate the resulting budget needs.
subsections: section_title: Operational Capabilities Have Been Delayed
paragraphs: The timely delivery of the JSF to replace aging legacy aircraft was cited as  a critical need by the warfighter at the program start. When the program  was initiated, in 1996, it planned to deliver initial operational capabilities  to the warfighter in 2010. However, largely because of technical  challenges, the program has delayed the delivery of operational aircraft,  and current estimates put delivery at 2012 to 2013. Because of these  delays, the services may have to operate legacy aircraft longer than  expected. These challenges have also delayed interim milestones such as  the start of system development, design reviews, and production  decisions. Figure 2 illustrates changes to the overall program schedule  since it began in 1996 through 2004.
section_title: Program Uncertainties Make It Difficult to Estimate Resources
paragraphs: The full impact on costs, schedules, and aircraft performance brought  about by recent design changes and aggressive software development and  flight test programs add risks that may not be fully understood for some  time. Continuing uncertainties about total quantities and types of the three  JSF variants that the services and the international partners expect to  purchase in the future also make it difficult to accurately estimate costs  and schedules.
subsections: section_title: Aircraft Design, Software Development, and Flight Test Program Not Fully Understood
paragraphs: In December 2003, DOD estimated program costs based on a notional idea  of a restructured program. The cost estimates not only lacked detail but  were based on a different aircraft design, development schedule, and  procurement plan than what is now being considered. Over the past year,  DOD has been working to restructure the JSF program to accommodate  changes in the aircraft’s design; until this restructuring is completed, it will  be difficult to accurately estimate program costs. The need for design  changes largely resulted from the increased weight of the short takeoff  and vertical landing variant and the impact it was having on key  performance parameters. The other JSF variants’ designs were affected as  well. The program plans to have a more comprehensive cost estimate in  the spring of 2005. However, a detailed assessment has not been  conducted to determine the exact impact that the restructured program  will have on meeting performance specifications. Until the detailed design  efforts are complete—after the critical design review in February 2006— the program will have difficulty assessing the impact of the design changes  on performance. While the program office anticipates that recent design  changes will allow the aircraft to meet key performance parameters,  preliminary program data indicate that the design is still not meeting  several speed, maneuverability, and radar cross section specifications. In  addition, program officials noted that they will not know with certainty if  the weight problems have been resolved until after the plane is  manufactured and weighed in mid-2007.
Program officials recognize that JSF’s development schedule is aggressive  and are examining ways to reduce program requirements while keeping  costs and schedules constant. Design and software teams have found  greater complexity and less efficiency as they develop the 17 million lines  of software needed for the system. Program analysis also indicated that  some aircraft capabilities will have to be deferred to stay within cost and  schedule constraints. As a result, the program office is working with the  warfighters to determine what capabilities could be deferred to later in the  development program or to follow on development efforts while still  meeting the warfighter’s basic needs. Many of these capabilities are  related to the software-intensive mission systems suite. They are also  examining the content and schedule of the planned 7-year, 10,000-hour  flight test program. According to the program office, the test program was  already considered aggressive, and recent program changes have only  increased the risks of completing it on time.
section_title: Quantities of Variants Still Unknown
paragraphs: Continued uncertainty about the number and mix of variants the services  plan to purchase also affects JSF’s acquisition plans. While the Air Force  has announced its intention to acquire the short takeoff and vertical  landing variant, it has yet to announce when or how many it expects to  buy or how this purchase will affect the quantity of the conventional  takeoff and landing variant it plans to buy. DOD’s 2003 acquisition report  states that the annual total quantity and mix of JSF variants and their  related procurement costs for Navy and Marine Corps JSF purchases  remains to be determined. Foreign partners have expressed intent to buy  about 700 aircraft between 2012 and 2015, but no formal agreements have  been signed at this time.
The upcoming 2005 Quadrennial Defense Review—an examination of U.S.  defense needs conducted every 4 years—could also affect the  procurement quantities and schedule.
section_title: Frequently Changing Program Managers Reduces Accountability
paragraphs: Since the JSF program began, a little over 8 years ago, the program has  had five program managers—a new program manager assigned about  every 2 years. The development program is estimated to last another 9  years, and it is likely that the program manager currently involved in  decisions about key program elements such as design, cost, and schedule  will not be responsible for seeing JSF through its completion. In other  words, plans accepted now will likely become the responsibility of future  program managers.
Leading commercial firms limit product development cycle times, thereby  increasing the possibility that program managers will remain on programs  until they are complete.  Holding one program manager accountable for  the content of the program when key decisions are made encourages that  person to raise issues and problems early and realistically estimate the  resources needed to deliver the program. This puts the manager in a good  position to deliver a high-quality product on time and within budget. We  note that the law governing the defense acquisition workforce recognizes  the need for long-term assignments in the performance of the program  manager function. Specifically, the assignment period for program  managers is required to be at least until completion of the major milestone  that occurs closest in time to the date on which the manager has served in  the position for 4 years.
section_title: JSF’s Current Acquisition Strategy May Not Provide for Successful Program Execution
paragraphs: The JSF program does not have an evolutionary, knowledge-based  acquisition strategy that fully follows the intent of DOD’s acquisition  policy. This type of strategy is necessary for having an executable business  case in the future. The current strategy includes plans to make large  production commitments well before system development and testing  have been completed, significantly increasing the risk of further delays  and cost increases due to design changes and manufacturing  inefficiencies. It is also dependent on an aggressive test aircraft delivery  schedule and an optimistic funding profile that assumes an unprecedented  $225 billion over the next 22 years, or an average of $10 billion a year.  DOD plans to bear the financial risk of concurrently developing and  initially producing the JSF on a cost reimbursement basis with the prime  contractor, an uncommon practice for such a large number of units, until  the design and manufacturing processes are mature. Program officials  currently have an opportunity to change the acquisition strategy. DOD  policy and best practices call for programs to use an acquisition strategy  that reflects an evolutionary, knowledge-based approach—that is, one that  ensures appropriate technology, design, and manufacturing knowledge are  captured at key milestones before committing to increased investments.  Our past work has shown that when programs demonstrate a high level of  knowledge before making significant commitments, they are able to  deliver products within identified resources.
subsections: section_title: JSF Program Not Fully Employing an Evolutionary, Knowledge- Based Approach
paragraphs: In recent years, DOD has revised its acquisition policy to support an  evolutionary, knowledge-based approach for acquiring major weapon  systems based on best practices. JSF’s acquisition strategy does not fully  follow the intent of this policy. Instead, it strives to achieve the ultimate  JSF capability within a single product development increment. While the  acquisition strategy calls for delivering a small number of aircraft with  limited capabilities, the program has committed to deliver the full  capability by the end of system development and demonstration in 2013  within an established cost and schedule, contrary to an evolutionary  approach. The JSF program bypassed early opportunities to trade or defer  to later increments those features and capabilities that could not be  readily met. The planned approach will not capture adequate knowledge  about technologies, design, and manufacturing processes for investment  decisions at key investment junctures. Figure 3 shows a comparison of an  evolutionary, knowledge-based process based on best practices and JSF’s  more concurrent approach.
Successful commercial companies use an evolutionary acquisition  approach where new products are developed in increments based on  available resources. Companies have found that trying to capture the  knowledge required to stabilize the design of a product that requires  significant amounts of new content is an unmanageable task if the goal is  to reduce cycle times and get the product to the customer as quickly as  possible. With an evolutionary acquisition approach, design elements that  are not currently achievable are planned for and managed as increments in  future generations of the product, and each increment is managed as a  separate knowledge-based acquisition, with separate milestones, costs,  and schedules.
Programs that attain the right knowledge at the right time reduce the risk  of incurring design, development, and manufacturing problems that result  in cost and schedule overruns. Our past work has shown that to ensure  successful program outcomes, a high level of demonstrated knowledge  must be attained at three key junctures for each increment in the program.
At knowledge point 1, the customer’s needs should match the developer’s  available resources—mature technologies, engineering knowledge, time,  and funding—before system development starts. This is indicated by a  demonstration that technologies needed to meet essential product  requirements work in their intended environment and the producer has  completed a preliminary design of the product that shows that the design  is feasible.
At knowledge point 2, the product’s design is stable and has demonstrated  that it is capable of meeting performance requirements before  transitioning from system integration to system demonstration. This is  best indicated by a prototype demonstration of the design and release of  90 percent of the engineering drawings to manufacturing organizations.
At knowledge point 3, the product must be producible within cost,  schedule, and quality targets and demonstrated to be reliable and work as  intended before production begins. This is indicated by a demonstration of  an integrated product in its intended environment and by bringing critical  manufacturing processes under statistical control.
subsections: section_title: JSF Began System Development Well before Knowledge Point 1
paragraphs: The start of the JSF system development was approved in 2001—well  before a match was made between the customer’s requirements and the  resources needed to meet those requirements. Many of the technologies  needed for the product’s full capabilities were demonstrated only in a lab  environment or ground testing and not in the form, fit, or functionality  needed for the intended product design. Also, while the program had a  proposed technical solution to meet the warfighter’s requirements, it did  not deliver a preliminary design based on sound systems engineering  principles. At the JSF preliminary design review, held about 1½ years after  development started, significant design issues surfaced, potentially  affecting the critical performance capabilities of the aircraft. The program  has worked to find solutions to design problems, but at a substantial cost.  The detailed design work has fallen behind schedule, delaying the critical  design reviews for 16 to 22 months. Table 4 compares the product  knowledge available at the JSF system development start and the  knowledge expected to be available to support future decision points  based on the current acquisition plan.
Knowing that a product’s design is stable before system demonstration  reduces the risk of costly design changes occurring during the  manufacturing of production representative prototypes—when  investments in acquisitions become even more significant. The JSF  program expects to have all critical drawings and a small number of other  drawings completed by the planned February 2006 critical design review— the milestone at which design stability is determined. However, these  drawings represent only about 35 percent of the total drawings needed to  complete the JSF design. While program officials believe that having 35  percent of the total drawings will allow them to track JSF’s design  stability, we have found that programs that moved forward with less than  90 percent of the total drawings at the start of the product demonstration  phase were challenged to stabilize the design at the same time they were  trying to build and test the product. This overlap frequently results in  costly design changes and parts shortages during manufacturing, which, in  turn, result in labor inefficiencies, schedule delays, and quality problems.  The F/A-22 and PAC-3 missile are prime examples of programs that failed  to complete 90 percent of their drawings by the critical design review and  suffered substantial cost increases and schedule delays.
Using prototypes to demonstrate the design is a best practice that provides  additional evidence of design stability. JSF will not have this type of  demonstration before the critical design review. Prototype testing allows  the design to be demonstrated before making costly investments in  materials, manufacturing equipment, and personnel to begin building  production representative prototypes for the system demonstration phase.  The JSF program is building an early prototype of the conventional takeoff  and landing variant and plans to use this prototype to validate  performance predictions, manufacturing processes, and reliability and  maintainability models. According to the current schedule, however, the  first demonstrations will occur after the critical design review, after most  of the design drawings have been released, and after manufacturing has  begun for many of the remaining test aircraft. Any significant design  problems found during the prototype demonstrations would likely require  more time and money for redesign efforts and retrofitting of test aircraft  already in the manufacturing process.
section_title: JSF Program Plans to Enter Production before Knowledge Point 3
paragraphs: In addition to lacking mature technologies and design stability, the JSF  program will lack critical production knowledge when it plans to enter  low-rate initial production in 2007. Between 2007 and 2013, when the  program is scheduled to move to full-rate production, it expects to buy  nearly 500 JSF aircraft—20 percent of its planned total buys—at a cost of  roughly $50 billion. Under the program’s preliminary plan, it expects to  increase low-rate production from 5 aircraft a year to 143 aircraft a year,  significantly increasing the financial investment after production begins.  Between 2007 and 2009, the program plans to increase low-rate production  spending from about $100 million a month to over $500 million a month,  and before development has ended and an integrated aircraft has  undergone operational evaluations, DOD expects to spend nearly $1  billion a month. To achieve its production rate, the program will invest  significantly in tooling, facilities, and personnel. According to contractor  officials, an additional $1.2 billion in tooling alone would be needed to  ramp up the production rate to 143 aircraft a year. Over half of this  increase would be needed by 2009—more than 2 years before operational  flight testing begins.
Despite this substantial investment, the key event to support the decision  to enter low-rate production in 2007 is the JSF’s first flight. Significant  commitments will thus be made to JSF production before requisite  knowledge is available. This is a much lower standard than called for by  best practices. The following are examples of technology, design, and  production knowledge that will not have been achieved at the time JSF  enters low-rate initial production.
Technology: According to information provided by the program  office, only one of JSF’s eight critical technologies is expected to  be demonstrated in an operational environment by the 2007 low-  rate production decision. The remaining seven technologies, which  include the complex mission systems and prognostics and health  maintenance systems, are not expected to be mature prior to  entering production. (See app. III for program office’s projected  time frames for demonstrating the eight critical technologies.)
Design: Low levels of design knowledge will continue beyond the  production decision. Only about 40 percent of the 17 million lines  of code needed for the system’s software will have been released.  The complex software needed to integrate the advanced mission  systems is not scheduled for release until about 2010—3 years after  JSF is scheduled to enter production. In addition, most structural  fatigue testing and radar cross section testing of full-up test  articles—needed to verify the stability of the aircraft’s structural  design—are not planned to be completed until 2010.
Production: The program will not demonstrate that critical  manufacturing processes are in statistical control when it enters  production. At that time, only one test aircraft will be completed  and delivered. According to the contractor, manufacturing  processes will not be under statistical control until after all of the  system development and demonstration aircraft have been built.  Also, flight testing of a fully configured and integrated JSF (with  critical mission systems and prognostics technologies) is not  scheduled until 2011. Operational testing to evaluate the  effectiveness and suitability of the integrated system will continue  until the full-rate production decision in 2013.
The JSF, like many past DOD weapons programs, is very susceptible to  discovering costly problems late in development when the more complex  software and advanced capabilities are tested. In the case of the JSF,  several hundred aircraft costing several billions of dollars may already be  on order or delivered, making any changes that result from testing costly  to incorporate. Figure 4 shows the proposed low-rate initial production  plan and how it overlaps with development and test activities.
section_title: Failing to Meet Aggressive Delivery Schedule Could Delay Flight Testing
paragraphs: If the JSF program cannot meet aggressive delivery schedules for test  aircraft, flight testing will be delayed. Flight testing provides key  knowledge about JSF performance needed to make investment decisions  for production. The JSF program is attempting to develop three different  aircraft, for three different services. All want to fly at supersonic speeds,  shoot air-to-air missiles, and drop bombs on a target, but they all have  vastly different operational concepts. While each of the variants may look  similar externally, subtle design differences provide many needed  capabilities that are unique to each service. As a result, the program will  attempt to design, build, and test simultaneously three distinct aircraft  designs. This difficult task is further complicated by plans to manufacture  and deliver in a 5-year period, 15 flight test aircraft and 8 ground test  articles. When compared with schedules of other programs with fewer  variables, JSF’s schedule is aggressive. For example, the F/A-22 program  took almost 8 years to manufacture and deliver nine flight test aircraft and  two ground test articles of a single aircraft design.
While the first aircraft had only been in assembly for about 8 months, it  was already behind schedule as of January 2005. According to the Defense  Contract Management Agency, based on the manufacturing status of the  center fuselage, wing, forward fuselage, and software development, the  first flight, scheduled for August 2006, could be delayed from 2 to 6  months. Late engineering releases to the manufacturing floor have resulted  in parts shortages and manufacturing inefficiencies. According to  contractor data, as of January 2005, it had taken about 50 percent more  labor hours than planned to complete manufacturing efforts.
section_title: Program Funding Level Assumptions May Be Difficult to Achieve
paragraphs: To execute its current acquisition strategy, the JSF program must obtain  on average over $10 billion annually in acquisition funds over the next 2  decades. Regardless of likely increases in program costs, the sizable  continued investment in JSF—estimated at roughly $225 billion over 22  years—must be viewed within the context of the fiscal imbalance facing  the nation within the next 10 years. The JSF program will have to compete  with many other large defense programs, such as the Army’s Future  Combat System and the Missile Defense Agency’s ballistic missile defense  system, for funding during this same time frame. There are also important  competing priorities external to DOD’s budget. Fully funding specific  programs or activities will undoubtedly create shortfalls in others.
Funding challenges will be even greater if the program fails to translate  current cost estimates into actual costs. For example, we estimate that  another 1-year delay in JSF development would cost $4 billion to $5 billion  based on current and expected development spending rates. A 10 percent  increase in production costs would amount to $20 billion. The JSF  program’s latest planned funding profile for development and  procurement—as of December 2003—is shown in figure 5.
section_title: Current Strategy Requires Prolonged Reliance on Cost Reimbursement Contract
paragraphs: The program’s acquisition strategy is to concurrently develop, test, and  produce the JSF aircraft, creating a risky approach. Because of this risk,  the program office plans to place initial production orders on a cost  reimbursement basis. According to program officials, a cost reimbursable  contract is necessary during the initial production phase because of the  uncertainties inherent in concurrent development and production  programs that prevent the pricing of initial production orders on a fixed- price basis. Cost reimbursement contracts provide for payment of  allowable incurred costs, to the extent prescribed in the contract. They are  used when uncertainties involved in contract performance do not permit  costs to be estimated with sufficient accuracy to use any type of fixed- price contract. Cost reimbursement contracts require only the contractor’s  “best efforts,” thus placing a greater cost risk on the buyer—in this case,  DOD. In contrast, a fixed-price contract provides for a pre-established  price and places more risk and responsibility for costs and resulting profit  or loss on the contractor and provides more incentive for efficient and  economical performance. However, to negotiate a fixed-price contract  requires certainty about the item to be purchased, which in the case of the  JSF will not be possible until late in the development program.
The program plans to transition to a fixed-price contract once the air  vehicle has a mature design, has been demonstrated in flight test, and is  producible at established cost targets. According to program officials, this  transition will occur sometime before full-rate production begins in 2013.  The program office believes the combination of the early concept  development work, the block development approach, and what it  characterizes as the relatively small numbers of aircraft in the initial  production buys allow decisions to be made earlier than normal with an  acceptable level of risk.
section_title: Conclusions
paragraphs: The JSF program is at a crossroads. DOD has not been able to deliver on  its initial promises, and the sizable investment DOD plans to make over  the next few years greatly raises the stakes to meet future promises. Given  the many uncertainties surrounding JSF’s development, program officials  need more time to gain knowledge before committing to a business case.  JSF’s failure to adequately match requirements and resources has already  resulted in increases in cost, schedule, and performance estimates, and a  reduction in DOD’s buying power. The new business case must also be  accompanied by an acquisition strategy that adopts an evolutionary  approach to product development—one that enables knowledge-based  investment decisions to maximize remaining program dollars. While the  warfighter may not receive the ultimate capability initially, an evolutionary  approach provides a useful product sooner and in sufficient quantities to  start replacing the rapidly aging legacy fighter and attack force. The  decisions DOD makes now and over the next 2 years will greatly influence  the efficiency of its remaining funding—over 90 percent of the $245 billion  estimated total program costs. Chief among these are the investments  needed to increase production to 143 aircraft a year, increasing production  expenditures from $100 million a month to $1 billion a month by 2013.
While delays are never welcomed, time taken by DOD now to gain more  knowledge and reduce risk before increasing its investment may well save  time and money later in development and production. Now is the time to  get the strategy right for delivering on the remainder of the investment.  With an evolutionary, knowledge-based plan in place, DOD managers will  be in a better position to succeed in delivering the warfighter needed  capabilities within budgeted resources.
section_title: Recommendations for Executive Actions
paragraphs: Given that DOD has invested only about 10 percent of the estimated cost  to develop and produce the JSF aircraft, and that significant investments  are planned in the next few years that can lock the program into a higher- risk acquisition, we recommend the Secretary of Defense take the  following two actions to increase the likelihood of having a successful  program outcome by delivering capabilities to the warfighter when needed  and within available resources:  (1) Establish an executable program consistent with best practices  and DOD policy regarding evolutionary acquisitions. DOD officials  should define an affordable first increment, with its own business  case that clearly defines the warfighter’s most immediate needs  and accurately identifies the resources required to deliver on this  needed capability. The business case should be established with a  high degree of confidence based on known constraints about  technology, engineering knowledge, time, and money. For those  warfighter needs that cannot be accommodated within this first  increment, the program should outline a strategy to meet these  needs through subsequent increments, each dependent on having  sufficient product knowledge to start system development and  demonstration. Each increment should be managed as a distinct  acquisition with its own business case for supporting the  investment.  (2) Develop and implement a knowledge-based acquisition approach,  as called for by best practices and DOD’s acquisition policy, an  approach that ensures attainment and use of demonstrated product  knowledge before making future investments for each product  increment. Before increasing the investment in production  resources (tooling, materials, and personnel) greater than  investments already in place to support the manufacturing of  development test aircraft, the Secretary should ensure knowledge  consistent with best practices is captured. This should help  minimize the number of low-rate initial production aircraft DOD  procures on a cost reimbursement basis, reducing the potential  financial risk to the government.
section_title: Agency Comments and Our Evaluation
paragraphs: The Office of the Under Secretary of Defense (Acquisition, Technology,  and Logistics), provided us with written comments on a draft of this  report. The comments appear in appendix I.
DOD partially concurred with our recommendation that the Secretary  establish an executable program that includes an affordable first  increment with its own business case that clearly defines the warfighter’s  most immediate needs and accurately identifies the resources required to  deliver on this capability.  DOD stated that the JSF program acquisition  strategy is based on an appropriate balance of technical, cost, and  schedule risk considerations to achieve program objectives.  Warfighter  representatives are involved in determining the content for each block  capability, and technology maturity is factored into the decision plan that  has been endorsed by DOD leadership.  DOD stated its JSF management  practices achieve the objectives of the GAO recommendation.
We believe DOD’s acquisition strategy will not provide the full benefits of  an evolutionary approach as suggested by DOD’s policy and best practices.   DOD has not structured the JSF development program into increments  managed as separate acquisitions with their own cost, schedule, and  decision milestones, making the likelihood of successful program  outcomes low.  The JSF strategy resembles other past major acquisition  programs that have attempted to achieve the ultimate capability in a single  development increment. DOD has allowed technology development to spill  over into product development, weakening any foundation for program  cost or schedule estimates.  This has led to poor outcomes for other  programs, such as the F/A-22 and Comanche, where lengthy and costly  development efforts resulted in either program cancellation or a  significant reduction in the number of systems to be acquired, a real loss in  DOD buying power.  Without a true evolutionary approach supported by a  business case for each increment, it will be difficult for the JSF program to  meet product requirements within current estimates of time and money.
DOD also partially concurred with our recommendation to develop and  implement a knowledge-based acquisition approach, which ensures  attainment and use of demonstrated product knowledge before making  future investments for each product increment.  The department agrees  that a knowledge-based approach is critical to making prudent acquisition  decisions and stated that its current JSF acquisition strategy incorporates  this type of approach.  The department admits it has accepted some  concurrency between development and production to reduce schedule  and cost, but it will consider the production readiness of the JSF design at  the low-rate and full-rate production decision milestones.  It states that the  new program plan includes clear entry and exit criteria for critical  milestones to ensure technologies are mature and required incremental  objectives are achieved before obligating funds.  DOD stated that it  conducts regular program reviews, and the Defense Acquisition Board will  review program readiness prior to making any milestone decision.  The  frequent rotation of program leadership ensures ongoing cooperative  oversight of emerging challenges and program decisions, and ensures  accountability for the implementation of those decisions.  Finally, DOD  states that the acquisition strategy is consistent with acquisition directives  and ensures the department commits resources only after determining that  specific developmental or knowledge-based criteria are achieved.
We believe the JSF’s acquisition strategy will not capture the right  knowledge at the right time for informed decisions on future  investments—over $200 billion dollars.  The program does not have the  practices in place to capture knowledge at key junctures.  DOD will not  have captured knowledge before production starts that ensures the design  is mature, reliable, and works or that manufacturing processes are in  control---keys to successful outcomes in the production phase.  Further,  the large investments planned in production capability for the JSF over the  next few years are vulnerable to costly changes as the aircraft is still being  designed and tested.  DOD has historically developed new weapon  systems in a highly concurrent environment that usually forces acquisition  programs to manage technology, design, and manufacturing risk at the  same time.  While DOD believes it can manage the risk of concurrent  development and production by holding regular program reviews and with  entrance and exit criteria for decisions, DOD’s own experience has shown  this approach to be risky and often not totally effective.  This has been  DOD’s traditional approach to weapons acquisition, the same approach  that has led to programs costing significantly more than planned and  taking much longer to develop.  This environment has made it difficult to  make informed decisions because appropriate knowledge has not been  available at key decision points.  If decisions are tied to the availability of  critical knowledge, program managers can be held accountable for the  timely capture of that knowledge instead of less precise or ill-defined  criteria included in risk reduction plans.  DOD’s practice of frequently  changing program managers also decreases accountability because  commitments made today will likely not be carried through by the same  managers who made the commitments.
We are sending copies of this report to the Secretary of Defense; the  Secretaries of the Air Force, Army, and Navy; and the Director of the  Office of Management and Budget. We will also provide copies to others  on request. In addition, the report will be available at no charge on the  GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841 or Michael Hazard at (937) 258-7917. Other  staff making key contributions to this report were Marvin Bonner,  Matthew Drerup, Matthew Lea, David Schilling, Karen Sloan, and Adam  Vodraska.
section_title: Appendix I: Comments from the Department of Defense
section_title: Appendix II: Scope and Methodology
paragraphs: To determine the status of the Joint Strike Fighter (JSF) business case for  delivering new capabilities to the warfighter, we compared the original  program estimates with current estimates. For development, we used the  program estimates that justified the program when it started in 1996. This  was the point at which JSF transitioned from a technology development  environment to an acquisition program environment, with the commitment  to delivery a family of strike aircraft that meet the Air Force, Navy, and  Marine Corps needs. At that time, total production, acquisition, and  ownership costs had not been estimated.  However, the program had  estimated the unit flyaway costs for each variant. The total production,  acquisition, and ownership estimates were first established to support the  decision to enter the system development and demonstration phase in  2001. We used these estimates as the baseline for these costs. We  identified changes in costs, quantities, and schedules as well as the causes  for the changes. We also identified program conditions that may affect  these estimates in the future. To accomplish this, we reviewed  management plans, cost reports, progress briefings, program baselines,  risk reports, and independent program assessments. We also interviewed  officials from the Department of Defense’s (DOD) acquisition program  management office and prime contractor.
To evaluate whether the current acquisition plan follows an evolutionary,  knowledge-based approach to meeting business case goals in the future,  we applied GAO’s methodology for assessing risks in major weapon  systems. This methodology is derived from best practices and experiences  of leading commercial firms and successful defense acquisition programs.  We reviewed Office of the Secretary of Defense (OSD), program office,  and prime contractor processes and management actions. We compared  the program’s plans and results to date against best practice standards in  capturing product knowledge in terms of technology, design, and  production maturity information and in applying knowledge to support  major program investment decisions. We reviewed management plans,  acquisition strategies, test plans, risk assessments, and program status  briefings. We identified gaps in product knowledge, reasons for these gaps,  and the risks associated with moving forward with inadequate knowledge  at future decision points. We also reviewed DOD’s acquisition policy to  determine whether JSF’s approach met its intent.
In performing our work, we obtained information and interviewed officials  from the JSF Joint Program Office, Arlington, Virginia; Lockheed Martin  Aeronautical Systems, Fort Worth, Texas; Defense Contract Management  Agency, Fort Worth, Texas; Institute for Defense Analyses, Alexandria,  Virginia; and offices of the Director, Operational Test and Evaluation, and  Acquisition, Technology and Logistics, which are part of the Office of  Secretary of Defense in Washington, D.C.
section_title: Appendix III: Projected Time Frames for Demonstration of Critical Technologies
subsections: section_title: Technology description
paragraphs: Includes integration of propulsion, vehicle management  system, and other subsystems as they affect aircraft  stability, control, and flying qualities (especially short  takeoff and vertical landing). Aircraft improvements are  to reduce pilot workload and increase flight safety.
Involves the ability to detect and isolate the cause of  aircraft problems and then predict when maintenance  activity will have to occur on systems with pending  failures. Life-cycle cost savings are dependent on  prognostics and health management through improved  sortie generation rate, reduced logistics and manpower  requirements, and more efficient inventory control.
Involves designing an integrated support concept that  includes an aircraft with supportable stealth  characteristics and improved logistics and maintenance  functions. Life-cycle cost savings are expected from  improved logistics and maintenance functions. Life-cycle  cost savings are expected from low observable  maintenance techniques and streamlined logistics and  inventory systems.
Includes areas of electrical power, electrical wiring,  environmental control systems, fire protection, fuel  systems, hydraulics, landing gear systems, mechanisms  and secondary power. Important for reducing aircraft  weight, decreasing maintenance cost, and improving  reliability.
Includes the ability to use commercial-based processors  in an open architecture design to provide processing  capability for radar, information management,  communications, etc. Use of commercial processors  reduces development and production costs, and an  open architecture design reduces future development  and upgrade costs.
Includes advanced integration with communication,  navigation, and identification functions and electronic  warfare functions through improved apertures,  antennas, modules, radomes, etc. Important for  reducing avionics cost and weight, and decreasing  maintenance cost through improved reliability.
Involves decreasing pilot workload by providing  information for targeting, situational awareness, and  survivability through fusion of radar, electronic warfare,  and communication, navigation, and identification data.  Improvements are achieved through highly integrated  concept of shared and managed resources, which  reduce production costs, aircraft weight, and volume  requirements, in addition to providing improved  reliability.
Involves lean, automated, highly efficient aircraft  fabrication and assembly techniques. Manufacturing  costs should be less through improved flow time, lower  manpower requirements, and reduced tooling cost.
section_title: Appendix IV: Measures of JSF Program Cost and Schedule Changes