Joint Strike Fighter: Strong Risk Management Essential as Program Enters Most Challenging Phase
Released: 2009-05-20T00:00:00

Letter

More Money and Time Will Be Needed to Complete JSF Development, While DOD Plans to Accelerate Procurement

JSF development will cost more and take longer to complete than reported to the Congress in April 2008, primarily because of contract cost overruns and extended time needed to complete flight testing. DOD is also significantly increasing annual procurement rates and plans to buy some aircraft sooner than reported last year. The new plan will require increased annual procurement funding over the next 6 years, but officials did not assess its net effect on total program costs through completion of JSF acquisition.

Total development costs are projected to increase between $2.4 billion and $7.4 billion and the schedule for completing system development to be extended from 1 to 3 years, according to estimates made in late 2008—one by the JSF Program Office and one by a joint team of Office of the Secretary of Defense (OSD), Air Force, and Navy officials. Cost overruns on both the aircraft and engine contracts, delays in manufacturing test aircraft, and a need for a longer, more robust flight test program were the primary cost drivers. The joint team’s estimate is higher than the program office’s because it included costs for the alternate engine program directed by the Congress and used more conservative assumptions based on current and legacy aircraft experiences. Table 1 compares these two estimates with the official program of record which was reported to the Congress in April 2008.

Although annual budgets and procurement quantities for fiscal year 2011 and out are still being reviewed by defense officials and are not available to us, we expect the JSF program to continue its rapid increase in annual procurement quantities and to buy some aircraft sooner than reported to the Congress in April 2008. At that time, DOD planned to ramp up procurement to reach a maximum of 130 aircraft per year by fiscal year 2015 (U.S. quantities only) and sustain this rate for 8 years. Procurement budget requirements for that plan were projected to be over $12 billion per year during peak production. The new fiscal year 2010 procurement budget requests funding of $6.8 billion for 30 JSF aircraft, a unit cost of $227 million. This budget is substantially lower than both the program office’s and the joint team’s estimates for 2010, in terms of unit costs and overall procurement funding.

Last month, the Secretary of Defense announced plans to procure 513 JSF aircraft during the 6-year period, fiscal years 2010 through 2015. This total includes procuring 28 more aircraft during this period than previously planned. This plan does not increase the total aircraft to be procured through completion of the JSF program but would buy these 28 aircraft in earlier years than previously scheduled. By accelerating procurement, DOD hopes to recapitalize tactical air forces sooner and mitigate projected future fighter shortfalls. The additional aircraft represent a scaling back of the proposed JSF procurement plans that we reported on in March 2009.

The joint team’s estimate included $420 million for the alternate engine program. DOD’s 2010 budget request did not include this funding.

At that time, DOD was proposing to accelerate procurement by 169 aircraft during these same years. That proposal would have required from $22 billion to $33 billion more in total procurement funding over that period, according to the respective estimates of the program office and joint estimating team. We have not yet been provided budgets and annual procurement quantities for fiscal years 2011 and out under the Secretary’s revised plan that would establish the increased funding requirements for the new accelerated plan compared to annual procurement funding requirements under the April 2008 program of record. Appendixes 1 and 2 provide an historical track of cost and schedule estimates.

DOD’s Proposal to Cancel the Alternate Engine Program May Bypass Long-term Merits

DOD and the Congress have had a continuing debate for several years on the merits of an alternate engine program to provide a second source and competition for engine procurement and life cycle support. The alternate engine program was part of the original JSF acquisition strategy. The department first proposed canceling the alternate engine program in the 2007 budget and has not asked for funding in the budgets since then. The administration does not believe an alternate engine is needed as a hedge against the failure of the main engine program and believes savings from competition would be small. The Congress has added funding each year since 2007 to sustain the alternate engine development, including $465 million for fiscal year 2009. To date, the two contractors have spent over $8 billion on engines development—over $6 billion with the main engine contractor and over $2 billion with the second source contractor.

The way forward for the JSF engine acquisition strategy entails one of many critical choices facing DOD today, and underscores the importance of decisions facing the program. As we noted in past testimonies before this committee, the acquisition strategy for the JSF engine must weigh expected costs against potential rewards. In each of the past 2 years we have testified before this committee on the merits of a competitive engine program for the Joint Strike Fighter. While we did not update our analysis we believe it is still relevant and the same conclusions can be drawn. We reported in 2008 that to continue the JSF alternate engine program, an additional investment of about $3.5 billion to $4.5 billion in development and production-related costs, may be required to ensure competition. Our earlier cost analysis suggests that a savings of 9 to 11 percent would recoup that investment. As we reported last year, a competitive strategy has the potential for savings equal to or exceeding that amount across the life cycle of the engine. Prior experience indicates that it is reasonable to assume that competition on the JSF engine program could yield savings of at least that much. As a result, we remain confident that competitive pressures could yield enough savings to offset the costs of competition over the JSF program’s life. However, we recognize that this ultimately will depend on the final approach for the competition, the number of aircraft actually purchased, and the ratio of engines awarded to each contractor.

Results from past competitions provide evidence of potential financial and nonfinancial savings that can be derived from engine programs. One relevant case study to consider is the “Great Engine War” of the 1980s— the competition between Pratt & Whitney and General Electric to supply military engines for the F-16 and other fighter aircraft programs. At that time, all engines for the F-14 and F-15 aircraft were being produced on a sole-source basis by Pratt & Whitney, which was criticized for increased procurement and maintenance costs, along with a general lack of responsiveness to government concerns about those programs. For example, safety issues with the single-engine F-16 aircraft were seen as having greater consequences than safety issues with the twin-engine F-14 or F-15 aircraft. To address concerns, the Air Force began to fund the development and testing of an alternate engine to be produced by General Electric; the Air Force also supported the advent of an improved derivative of the Pratt & Whitney engine. Beginning in 1983, the Air Force initiated a competition that Air Force documentation suggests resulted in significant cost savings in the program. In the first 4 years of the competition, when actual costs are compared to the program’s baseline estimate, results included (1) nearly 30 percent cumulative savings for acquisition costs, (2) roughly 16 percent cumulative savings for operations and support costs; and (3) total savings of about 21 percent in overall life cycle costs.

The Great Engine War was able to generate significant benefits because competition incentivized contractors to improve designs and reduce costs during production and sustainment. Competitive pressure continues today as the F-15 and F-16 aircraft are still being sold internationally. While other defense competitions resulted in some level of benefits, especially with regard to contractor responsiveness, they did not see the same levels of success absent continued competitive pressures.

Similar competition for the JSF engines may also provide benefits that do not result in immediate financial savings, but could result in reduced costs or other positive outcomes over time. Our prior work, along with studies by DOD and others, indicate there are a number of nonfinancial benefits that may result from competition, including better performance, increased reliability, and improved contractor responsiveness. In addition, the long-term effects of the JSF engine program on the global industrial base go far beyond the two competing contractors.

DOD and others have performed studies and have widespread concurrence as to these other benefits, including better engine performance, increased reliability, and improved contractor responsiveness. In fact, in 1998 and 2002, DOD program management advisory groups assessed the JSF alternate engine program and found the potential for significant benefits in these and other areas. Table 2 summarizes the benefits determined by those groups.

While the benefits highlighted may be more difficult to quantify, they are no less important, and ultimately were strongly considered in recommending continuation of the alternate engine program. These studies concluded that the program would maintain the industrial base for fighter engine technology, enhance readiness, instill contractor incentives for better performance, ensure an operational alternative if the current engine developed problems, and enhance international participation.

Another potential benefit of having an alternate engine program, and one also supported by the program advisory groups, is to reduce the risk that a single point systemic failure in the engine design could substantially affect the fighter aircraft fleet. This point is underscored by recent failures of the Pratt & Whitney test program. In August 2007, an engine running at a test facility experienced failures in the low pressure turbine blade and bearing, which resulted in a suspension of all engine test activity. In February 2008, during follow-on testing to prove the root cause of these failures, a blade failure occurred in another engine, resulting in delays to both the Air Force and Marine Corps variant flight test programs.

Continued Manufacturing Inefficiencies Will Make it Difficult for the Program to Meet Its Production Schedule

Manufacturing of JSF development test aircraft is taking more time, money, and effort than planned. Officials believe that they can work through these problems and deliver the 9 remaining test aircraft by early 2010; however, by that time, DOD may have already ordered as many as 58 production aircraft. Manufacturing inefficiencies and parts shortages continue to delay the completion and delivery of development test aircraft needed for flight testing. The contractor has not yet demonstrated mature manufacturing processes, or an ability to produce aircraft consistently at currently planned annual rates. It has taken steps to improve manufacturing processes, the supplier base, and schedule management; however, given the manufacturing challenges, we believe that DOD’s plan to accelerate procurement in the near term adds considerable risk and will be difficult to achieve.

The prime contractor has restructured the JSF manufacturing schedule several times, each time lengthening the schedule to deliver aircraft to the test program. Delays and manufacturing inefficiencies are prime causes of contract cost overruns. The contractor has delivered four development flight test aircraft and projects delivering the remaining nine aircraft in 2009 and early 2010. Problems and delays are largely the residual effects from the late release of engineering drawings, design changes, delays in establishing a supplier base, and parts shortages, which continue to cause delays and force inefficient production line work-arounds where unfinished work is completed out of station. Data provided by the Defense Contract Management Agency and the JSF Program Office show continuing critical parts shortages, out-of-station work, and quality issues. The total projected labor hours to manufacture test aircraft increased by 40 percent just in the past year, as illustrated in figure 1.

Performance data for two major cost areas—wing assembly and mate and delivery—indicate even more substantial growth. Figure 2 compares the increased budgeted hours in the 2008 schedule to 2007 estimates. The 2007 schedule assumed a steeper drop in labor hours as more units are produced and manufacturing and worker knowledge increases. The new schedule, based upon actual performance, projects a less steep decline in labor hours, indicating slower learning and lesser gains in worker efficiency. As of June 2008, the planned hours for these two major stations increased by about 90 percent over the June 2007 schedule, which itself had shown an increase from the 2006 schedule. The overlap in the work schedule between manufacturing the wing and mating (connecting) it to the aircraft fuselage has been a major concern for several years because it causes inefficient out-of-station work. The contractor continues to address this concern, but the new schedule indicates that this problem will continue at least through 2009.

The prime contractor has taken significant steps to improve schedule management, manufacturing efficiency, and supplier base. Our review found that the prime contractor has good schedule management tools and integrated processes in place. The one area not meeting commercial best practices was the absence of schedule risk analysis that would provide better insight into areas of risk and uncertainty in the schedule. DOD agreed with our March 2009 recommendation and will direct the contractor to perform periodic schedule risk analyses. The prime contractor is also implementing changes designed to address the manufacturing inefficiencies and parts shortages discussed earlier. These include (1) increasing oversight of key subcontractors that are having problems, (2) securing long-term raw material purchase price agreements for both the prime and key subcontractors, and (3) implementing better manufacturing line processes. On this latter point, according to program officials, the prime contractor has taken specific steps to improve wing manufacturing performance—noted above as one of the most troublesome workstations. Defense Contract Management Agency officials noted that the contractor produced the second short take off and landing aircraft variant with less work performed out of station than for the first such aircraft. Also, program office and contractor officials report some alleviation of parts shortages and improvements in quality, but also believe that the effects from previous design delays, parts shortages, and labor inefficiencies will continue to persist over the near term.

Use of Cost Contracts for Production Aircraft Elevates the Government’s Financial Risk

DOD is procuring a substantial number of JSF aircraft using cost reimbursement contracts. Cost reimbursement contracts place most of the program’s financial risk on the buyer—DOD in this case—who is liable to pay more than budgeted should labor, material, or other incurred costs be more than expected when the contract was signed. Subsequent cost increases, such as the growth in manufacturing labor hours discussed above, are mostly passed on to the Government. Thus far, DOD has procured the first three production lots using cost reimbursement contracts—a total of 28 aircraft and an estimated $6.7 billion to date. JSF officials expect to also procure the fourth lot using cost reimbursement and to transition to fixed-price contracts when appropriate, possibly between lots 5 and 7 (fiscal years 2011 to 2013). It is unclear exactly how and when this will happen, but the expectation is to transition to fixed pricing once the air vehicle has a mature design, has been demonstrated in flight tests, and is producible at established cost targets. Under the April 2008 program of record, DOD was planning to procure as many as 275 aircraft costing an estimated $41.6 billion through fiscal year 2013 using cost reimbursement contracts. The plan to accelerate procurement of 28 aircraft would likely add to the quantities purchased on such contracts.

Cost reimbursement contracts provide for payment of allowable incurred costs, to the extent prescribed in the contract. According to the Federal Acquisition Regulation, cost reimbursement contracts are suitable for use only when uncertainties involved in contract performance do not permit costs to be estimated with sufficient accuracy to use any type of fixed- price contract. Cost reimbursement contracts for weapon production are considered appropriate when the program lacks sufficient knowledge about system design, manufacturing processes, and testing results to establish firm prices and delivery dates. In contrast, a fixed-price contract provides for a pre-established price, places more of the risk and responsibility for costs on the contractor, and provides more incentive for efficient and economical performance.

Procuring large numbers of production aircraft using cost reimbursement contracts reflects that the JSF design, production processes, and costs for labor and material is not yet sufficiently mature and that pricing information is not exact enough for the contractor to assume the risk under a fixed-price contract. We see it as a consequence of the substantial concurrency of development, test, and production built into the JSF schedule. Significant overlap of these activities means that DOD is procuring considerable quantities of operational aircraft while development test aircraft are still on the manufacturing line and where much testing remains to prove aircraft performance and suitability. Establishing a clear and accountable path to ensure that the contractor assumes more of the risk is prudent. Accordingly, we recommended in March 2009 that DOD report to the congressional defense committees by October 2009 explaining costs and risks associated with cost reimbursement contracts for production, the strategy for managing and mitigating risks, and plans for transitioning to fixed price contracts for production. DOD concurred.

The former Assistant Secretary of the Air Force for Acquisition agreed with our concerns about significant concurrency and the need to transition to a fixed price environment. In an April 2009 memo, as the Assistant Secretary of the Air Force for Acquisition, she discussed her views on the concurrency of production and development testing as driving risks to the development program. She recommended that the JSF joint program office closely examine manufacturing processes and work to convert cost reimbursement contracts to fixed-price as soon as practical.

JSF’s Test Plan Is Improved but Flight Test Program Is Still in Its Infancy

After reducing test resources and activities to save money in 2007, the JSF Program Office developed a new test plan in the spring of 2008 that extended the development period by 1 year, better aligned test resources and availability dates, and lessened the overlap between development and operational testing. While improved, the new plan is still aggressive and has little room for error discovery, rework, and recovery from downtime should test assets be grounded or otherwise unavailable. The sheer complexity of the JSF program—with 22.9 million lines of software code, three variants, and multi-mission development— suggests that the aircraft will encounter many unforeseen problems during flight testing requiring additional time in the schedule for rework. Given the complexity of the program, the joint estimating team noted that an additional 2 years beyond the recent 1 year extension may be needed to complete development.

The test plan relies heavily on a series of advanced and robust simulation labs and a flying test bed to verify aircraft and subsystem performance. Figure 3 shows that 83 percent of the aircraft’s capabilities are to be verified through labs, the flying test bed, and subject-matter analysis, while only 17 percent of test points are to be verified through flight testing. Program officials argue that their heavy investment in simulation labs will allow early risk reduction, thereby reducing the need for additional flight testing, controlling costs, and meeting the key milestones of the program’s aggressive test plan. However, while the JSF program’s simulation labs appear more prolific, integrated, and capable than the labs used in past aircraft programs, their ability to substitute for flight testing has not yet been demonstrated.

Despite an improved test plan, JSF flight testing is still in its infancy. Only about 2 percent of its development flight testing had been completed as of November 2008. Figure 4 shows the expected ramp up in flight testing with most effort occurring in fiscal years 2010 through 2012. Past programs have shown that many problems are not discovered until flight testing. As such, the program is likely to experience considerable cost growth in the future as it steps up its flight testing, discovers new problems, and makes the necessary technical and design corrections.

While the program has been able to complete key ground tests and demonstrate basic aircraft flying capabilities, it continues to experience flight testing delays. Most notably, flight testing of full short takeoff and vertical landing capabilities has further been delayed. Flight testing of the carrier variant has also been delayed. Program officials do not believe either of the delays will affect planned initial operational capability dates. In 2009 and early fiscal year 2010, the program plans to begin flight testing 6 development test aircraft, including the first 2 aircraft dedicated to mission system testing. A fully integrated, mission-capable aircraft is not expected to enter flight testing until 2012.

Despite the nascency of the flight test program and subsequent flight testing delays, DOD is investing heavily in procuring JSF aircraft. Procuring aircraft before testing successfully demonstrates that the design is mature and that the weapon system will work as intended increases the likelihood of expensive design changes becoming necessary when production is underway. Also, systems already built and fielded may later require substantial modifications, further adding to costs. The uncertain environment as testing progresses is one reason why the prime contractor and DOD are using cost-reimbursable contracts until rather late in procurement. Table 3 depicts planned investments—in both dollars and aircraft—prior to the completion of development flight testing. DOD may procure 273 aircraft at a total estimated cost of $41.9 billion before development flight testing is completed. Table 3 also shows the expected contract types.

Concluding Remarks

The JSF program is entering its most challenging phase, a crossroads of a sort. Looking forward, the contractor plans to complete work expeditiously to deliver the test assets, significantly step up flight testing, begin verifying mission system capabilities, mature manufacturing processes, and quickly ramp up production of operational aircraft. Challenges are many— continuing cost and schedule pressures; complex, extensive, and unproven software requirements; and a nascent, very aggressive test program with diminished flight test assets.

While the program must move forward, we continue to believe that the program’s concurrent development and production of the aircraft is extremely risky. By committing to procure large quantities of the aircraft before testing is complete and manufacturing processes are mature, DOD has significantly increased the risk of further compromising its return on investment—as well as delaying the delivery of critical capabilities to the warfighter. Furthermore, the program’s plan to procure large quantities of the aircraft using cost-reimbursement contracts—where uncertainties in contract performance do not permit costs to be estimated with sufficient accuracy to use a fixed-price contract—places additional financial risk on the government. Until the contractor demonstrates that it can produce aircraft in a timely and efficient manner, DOD cannot fully understand future funding requirements. DOD needs to ensure that the prime contractor can meet expected development and production expectations. At a minimum, the contractor needs to develop a detailed plan demonstrating how it can successfully meet program development and production goals in the near future within cost and schedule parameters. As such, in our March 2009 report, we recommended that Secretary of Defense direct the Under Secretary of Defense for Acquisition, Technology and Logistics to report to congressional defense committees explaining the risks associated with using cost-reimbursable contracts as compared to fixed price contracts for JSF’s production quantities, the program’s strategy for managing those risks, and plans for transitioning to fixed-price contracts for production. DOD agreed with our recommendation. With an improved contracting framework and a more reasoned look to the future, the JSF program can more effectively meet DOD and warfighter needs in a constrained budget environment.

Mr. Chairman, this concludes my prepared statement. I would be happy to answer any questions you may have at this time.

For further information about this statement, please contact Michael J. Sullivan at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals who made key contributions to this statement are Ridge Bowman, Bruce Fairbairn, Matt Lea, and Charlie Shivers.

Appendix I: Changes in JSF Cost, Quantity, and Delivery Estimates

development start) (2004 Replan)

Cost Estimates (then-year dollars in billions)

Unit Cost Estimates (then-year dollars in millions)

Military construction costs have not been fully established and the reporting basis changed over time in these DOD reports.

Appendix II: F-35 Joint Strike Fighter Schedule

Short Takeoff and Vertical Landing Short Takeoff and Vertical Landing * Aircraft flown in conventional mode. The first test to demonstrated full short takeoff and vertical landing capabilities is scheduled for September 2009.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.