Defense Acquisitions: Timely and Accurate Estimates of Costs and Requirements Are Needed to Define Optimal Future Strategic Airlift Mix
Released: 2008-11-21T00:00:00

Letter

Background

The Air Force’s C-5 and C-17 strategic airlifters both possess intercontinental range with aerial refueling and can carry weapons and equipment too large for any other DOD aircraft. Each also has some complementary characteristics that favor a mixed fleet. The larger C-5 can carry more cargo than the C-17 and is the only aircraft capable of handling some equipment, such as the Army’s 74-ton mobile scissors bridge. The C- 17 is more modern, has a higher mission capable rate, and is more flexible in that it also provides tactical (intratheater) airlift to austere, forward- deployed bases. Figure 1 compares the two strategic airlifters.

The Air Force acquired the C-5 fleet in two production batches. Aircraft designated C-5A were built between 1969 and 1974 and given new wings in the 1980s. Aircraft designated C-5B were built in a second production run in the 1980s. In 1999, the Air Force began modernizing its C-5 aircraft to improve fleet reliability and mission capable rates. The modernization is being done in two phases.

The Avionics Modernization Program (AMP) upgrades capabilities, including Global Air Traffic Management, navigation and safety equipment, modern digital equipment, and an all-weather flight control system.

The Reliability Enhancement and Reengining Program (RERP) replaces engines and modifies electrical, fuel, and other subsystems.

Together, these two upgrades were expected to improve the fleet’s mission capable rate to at least 75 percent, thereby increasing payload capability and transportation throughput, and to reduce total ownership costs over the life cycle by about $14 billion in 2008 dollars.

DOD initially expected to spend about $12 billion on the C-5 AMP and RERP efforts. However, both modernization efforts experienced cost problems. AMP development costs increased by approximately 20 percent and would have been higher had the Air Force not reduced requirements and deferred some development activities to other programs. Officials waived 14 operational requirements and deferred the correction of 250 deficiencies identified during testing, many of which will be addressed and funded in RERP or future efforts. In 2007, DOD reported that RERP average procurement unit costs grew more than 50 percent from the original baseline estimate.

C-17 procurement began in 1988 and the Air Force’s current plan is to acquire a total of 205 C-17s for $66 billion. The first production C-17 aircraft was delivered to the Air Force in June 1993 and the service has accepted delivery of 178 aircraft through October 2008. Delivery of the 205th aircraft is projected in August 2010. One aircraft is dedicated to provide airlift capabilities to the North Atlantic Treaty Organization, effectively setting the Air Force’s operational force at 204.

DOD periodically assesses global threats, the national military strategy, and its force structure to determine future airlift requirements and to judge the sufficiency of its acquisition and modernization plans. The analytical basis for DOD’s current airlift requirements is the mobility capabilities study completed in December 2005. Officials used the study results to report in the 2006 Quadrennial Defense Review that 180 C-17s and 112 fully modernized C-5s—i.e., those receiving both the AMP and RERP modification—would be sufficient to meet national military strategy with acceptable risk. The Air Force is now engaged in a new mobility capabilities study, the results of which will be briefed in May 2009. A final written report is expected to be issued in November 2009. According to Air Force officials, the new study will take into account a variety of changes that have occurred since the last mobility study was completed in 2005, including the following: Addition of over 92,000 Marines and Army soldiers and their equipment that will need to be transported to locations across the United States and around the world.

Establishment of a new African Command that will require the movement of troops and equipment to a variety of locations around the second largest continent in the world. Introduction of Mine Resistant Ambush Protected vehicles, which are being used in Iraq to provide enhanced protection for U.S. troops. Increase in weight of the Army’s Future Combat System vehicles, which makes it no longer possible to transport some vehicles with C- 130 aircraft (DOD’s primary tactical airlifter).

In addition to the new mobility capabilities study, Congress in 2008 directed the Secretary of Defense to conduct a requirements-based study on alternatives for the proper size and mix of fixed-wing strategic and tactical airlift to meet the national military strategy for the 2012, 2018, and 2024 time frames. The study, accomplished by the Institute for Defense Analysis, is due in January 2009.

Mix of C-5s and C-17s Needed to Meet DOD’s Strategic Airlift Requirement Continues to Evolve

The numbers and specific mix of strategic airlifters continues to be in a state of flux as DOD struggles to define and control costs and to establish firm requirements. Over the past several years, DOD has made changes in the number of C-5s and C-17s it says it needs to meet its strategic airlift requirement. C-5 modernization cost increases prompted DOD to reduce the number of C-5s it plans to fully modernize. Subsequently, Congress provided additional funding that the Air Force is using to procure more C- 17s, which would offset the loss in capability of modernizing fewer C-5s. While officials believe this mix will allow DOD to meet its strategic airlift requirement, the number of C-5s DOD modernizes and C-17s it procures may change again, pending the results of ongoing mobility studies, potential C-5 retirements, and the eventual cost estimates of C-5 modernization. Past efforts to identify the appropriate mix have been hampered by the lack of sufficient metrics, such as a million ton-mile metric, that quantify specific C-5 and C-17 needs. At this time, DOD officials have not determined what metrics it will use to make strategic airlift decisions.

Fewer C-5s Will Be Fully Modernized Due to Significant RERP Cost Increases

C-5 modernization cost increases caused DOD to change its approach for meeting its strategic airlift requirements. DOD had planned to meet the requirements with 112 fully modernized C-5s—i.e., those receiving both the AMP and RERP modifications—and 180 C-17 aircraft. The cost for the C-5 modernization efforts was estimated to be approximately $12 billion— about $900 million for the AMP program and $11.1 billion for the RERP program.

However, just prior to the RERP production decision in February 2007, the prime contractor, Lockheed Martin, indicated that RERP costs related to labor and supplier parts had significantly increased, prompting new cost estimates. The Air Force’s estimate of $17.5 billion was $4.2 billion more than Lockheed Martin’s estimate of $13.3 billion at that time. The new estimate increased projected average procurement unit costs by more than 50 percent compared to the original baseline and triggered a statutory requirement for review and certification of the program.

Following notification to Congress of the cost increase, the Under Secretary of Defense for Acquisition, Technology and Logistics requested that the CAIG estimate the cost of various options for DOD to meet its strategic airlift mission. The CAIG analyzed 14 options covering a range of scenarios for the RERP program in three broad categories: modifying all C-5 aircraft, partially modifying the C-5 fleet, and canceling the C-5 RERP program. Each option also assumed that the department would have at least 203 C-17 aircraft, 14 more than the program planned to acquire at that time. The CAIG estimated the cost of providing the RERP modification to all 111 aircraft to be $15.4 billion, halfway between the contractor’s and the Air Force’s estimates. Based on this analysis, the Under Secretary of Defense for Acquisition, Technology and Logistics concluded that the cost to RERP all C-5 aircraft was unaffordable and opted to limit full modification to 52 aircraft—47 C-5 Bs, both C-5 Cs, and 3 system development and demonstration aircraft. While the Air Force is expected to spend $3.4 billion less under the restructured program, ultimately less than one-half of the 111 aircraft will be modernized and at a much higher unit cost than originally estimated (see table 1).

Additional C-17s Offset Capability Loss from Modifying Fewer C-5s

As part of the C-5 RERP restructuring, U.S. Transportation Command identified a need for 205 C-17s, 25 more than were authorized at the time the 2005 mobility capabilities study was completed. Subsequent to the study, Congress provided additional funding that the Air Force used to procure 10 more C-17s in 2007 and 15 more in 2008. The following table shows the changes in the strategic airlift mix over the past 3 years and the impact the mixes have had on DOD’s ability to meet its strategic airlift mission (see table 2). While DOD did not use a million ton-mile per day metric to determine how many C-5s and C-17s it needed in its December 2005 mobility study, officials were able to quantify for us the million ton- mile per day capabilities of the scenarios identified below.

The fiscal year 2009 National Defense Authorization Act authorizes procurement of 6 more C-17s, which will bring the total number to 211 aircraft.

Forthcoming Decisions Could Lead to a Change in the Strategic Airlift Mix in the Near Future

The number and mix of aircraft needed to support DOD’s strategic airlift mission could change again based on the results of ongoing mobility studies, possible C-5 retirements, and the eventual cost of C-5 modernization efforts. However, DOD’s ability to make sound strategic airlift portfolio decisions, including the number of C-5s to retire and the number of additional C-17s that should be procured, may be hampered if appropriate metrics are not included in the study efforts.

Mobility Requirements Studies

DOD is currently studying its future mobility requirements. The congressionally directed requirements study by the Institute of Defense Analysis is due in January 2009, followed by the Air Force’s mobility capabilities study in May 2009. Some expect the studies will identify increased demands on airlift, particularly for the C-17 since it can perform both a strategic and tactical role. As Army equipment becomes heavier and/or bulkier, the C-17 may be the only aircraft capable of delivering major weapon systems to the front lines and to more austere bases in the theater of combat. The results of both studies, if done accurately and comprehensively, should provide the analytical foundation for the future airlift force structure.

We previously reported on shortcomings in the Institute for Defense Analysis’ study plan that could make it difficult for decision makers to know how much strategic airlift is needed. For example, the study plan did not provide details on assumptions and the measures of effectiveness, or metrics, the command officials would be using in their evaluation. Measures of effectiveness are considered to be especially important when evaluating alternatives, such as comparing the results of two analyses that measure different airlift force mixes. We recommended in April 2008 that DOD take action to ensure that the final study plan included sufficient details to address all the elements specified in the law and needed to inform decision makers on airlift issues. DOD concurred with our recommendation.

We also identified shortcomings in DOD’s 2005 mobility capabilities study approach that, if not addressed, could be repeated again in the current study. Unlike prior studies, the 2005 study did not recommend a specific airlift requirement expressed in million ton-miles per day—a common metric integral to prior capability studies that defines and quantifies airlift requirements as a basis for computing the size and optimal mix of airlift forces. Instead, DOD officials stated that it expressed its airlift requirement in terms of specific numbers and types of aircraft needed to meet the national defense strategy to take into account real-world operating parameters that may cause aircraft payloads to vary significantly from standard planning factors. Later, in response to congressional direction, DOD translated the requirements into a million ton-mile requirement. We also found the study did not identify the operational impact of increased or decreased strategic airlift on achieving warfighting objectives that would be associated with different mixes of C-5 and C-17 aircraft. As a result, we could not determine how the study concluded that the mix of C-5s and C-17s at that time was adequate for meeting mobility requirements and for supporting strategic airlift portfolio investment decisions. In 2006, we recommended that DOD include mobility metrics, along with warfighting metrics to determine air superiority, when completing future mobility capabilities studies. DOD concurred with this recommendation.

Although DOD concurred with the recommendation, a Transportation Command official stated that a decision has not yet been made on what specific metrics will be used to determine the number and mix of strategic airlifters in the current mobility capabilities study. At the time of this writing, the study plan had not been finalized and it is unclear whether a million ton-miles metric will be used, though it is being considered. DOD often uses the million ton-mile metric as an easy way to compare the capacity of different fleet mixes. For example, according to a DOD official, since C-130s, C-130Js, C-17s, C-5As, C-5Bs, and C-5Ms all have different capabilities when it comes to payload and range, it is difficult to compare different mixes of them without using this metric.

Potential C-5 Retirements

Congressional legislation would allow the Air Force to begin to retire some C-5s , if appropriate, beginning October 1, 2008, as long as the Air Force maintains a strategic airlift fleet of 299 aircraft. Prior to retiring any aircraft, DOD’s Director of the Operational Test and Evaluation must complete its assessment of a C-5A that had received RERP modification. According to an Air Force official, testing of a C-5A will begin in August 2009 and the final report will be released in February 2010.

Air Mobility Command officials told us that while they are concerned about the emerging global threats and requirements, fiscal and personnel demands require that the command limit overall fleet size once warfighting risk is reduced to a reasonable level. Therefore, the Air Mobility Command will consider retiring C-5s, as the law and requirements allow, on a one- for-one basis after 205 C-17s have been procured to ensure the right combination of aircraft and capability is balanced against cost and risk. A decision on whether and when to retire C-5s will not likely be made until after the mobility study has been completed.

Relative Capability Increases from Modernized C-5s and New C-17 Aircraft

Finally, if the cost for C-5 modernization continues to increase, Air Force officials may have to reconsider the mix within its airlift portfolio or request additional funding. Additional investments in C-17 aircraft may become more attractive. Currently, a new C-17 would cost about $276 million compared to $132 million to fully modernize a C-5. Each new C-17 potentially adds 100 percent of its cargo capacity toward meeting the total airlift requirement. Because the C-5s are already part of the operational force, each aircraft’s current capacity is already counted toward the total requirement. Consequently, according to DOD data, the C-5 modernization programs only provide a marginal increase of 14 percent in capability over nonmodernized aircraft. Using DOD’s million ton-mile per day planning factors, we, working in collaboration with DOD, calculated that DOD would need to fully modernize 7 C-5s to attain the equivalent capability achieved from acquiring 1 additional C-17 and the costs would be over 3 times more (see table 3).

The analysis does not include the life-cycle costs of adding more C-17s to DOD’s airlift portfolio. However, previous DOD analysis indicated that the life-cycle costs would be approximately the same if DOD replaced 30 C-5s with 30 C-17s.

C-5 Modernization Costs Have Not Been Fully Identified

The Air Force has not fully identified the funding needed to modernize the C-5 aircraft, and costs are likely to increase. The current cost estimate is $9.1 billion to AMP the entire fleet of 111 aircraft and RERP 52 aircraft. However, we believe this is understated. The current budget does not fully fund the revised RERP program and the CAIG’s cost estimate does not adequately address risk and uncertainty. Further, the cost estimate does not include the costs for a new modernization upgrade program slated to begin in fiscal year 2010 that would fix AMP deficiencies and add new capabilities. Alternatively, some future modification costs may be avoided should the Air Force justify retirement of some older C-5s.

Current Estimate for C-5 Modernization Costs Is Questionable

The current budget does not sufficiently fund the revised RERP program. According to the CAIG’s analysis, the C-5 RERP is underfunded by about $294 million across the Future Years Defense Plan for fiscal years 2009- 2013. Approximately $250 million less is needed in fiscal years 2009 through 2011, and $544 million more is needed in fiscal years 2012 and 2013. According to program officials, the Air Force is committed to fully funding the CAIG RERP cost estimate in the fiscal year 2010 President’s budget yet to be submitted. However, program officials could not identify sources for the additional funding needed in fiscal years 2012 and 2013. Figure 2 provides the funding estimate developed by the CAIG to support the restructured RERP program compared to the fiscal year 2009 budget.

While our review of the CAIG’s cost-estimating methodology found it generally well documented, comprehensive, and accurate, we found some weaknesses that impair the credibility and overall reliability of the C-5 cost estimate. Specifically, the CAIG did not take risk or uncertainty into account for some major cost drivers, in particular the propulsion system and labor. Because cost estimates predict future program costs, uncertainty is always associated with them. For example, there is always a chance that the actual cost will differ from the estimate because of a lack of knowledge about the future as well as errors resulting from historical data inconsistencies, assumptions, cost-estimating equations, and factors that are typically used to develop an estimate. Quantifying that risk and uncertainty is considered to be a cost estimating best practice because it captures the cumulative effect of risks and recognizes the potential for error.

In a memo documenting its independent cost estimate, the CAIG stated that the biggest risk to the cost estimate was the purchase agreement between Lockheed Martin and General Electric for the propulsion system that is conditioned on specific annual procurement quantities. The CAIG had estimated that the Air Force could save 18 percent by meeting the quantity and schedule identified in the revised RERP. However, CAIG officials stated that if the budget is not sufficient to meet these agreed-to quantities, then anticipated price breaks would not occur, resulting in increased costs of the C-5 RERP to the government. Despite this significant risk, the CAIG did not perform a risk/uncertainty analysis to determine the extent to which costs would increase should the buy quantity be cut. CAIG officials stated that they believe propulsion system procurement risk has been mitigated because they have identified the quantities necessary to meet the conditions of the purchase agreement and the Air Force plans to fully fund to this estimate. Despite these assurances, however, we have found that DOD often changes procurement quantities and there is a risk that quantities for the C-5 RERP program may change. For example, DOD’s Selected Acquisition Report summary shows that of the 56 programs currently in production, 38 (or 68 percent), have experienced a quantity change since their production decisions.

In addition, the CAIG did not quantify or address uncertainty with its $2.1 billion labor cost estimate associated with the installation of the RERP on C-5 production aircraft. The RERP program experienced a 29-month break in production between the last system development and demonstration unit and the first production unit. As such, the CAIG had to estimate inefficiencies due to loss of learning and how it would affect the costs of future production. The CAIG’s assumptions differed from those used by the Air Force and Lockheed Martin, which caused the CAIG estimate to be about $200 million more than Lockheed Martin’s estimate and about $400 million less than the Air Force’s labor estimate. As a result of the weaknesses discussed above, the Air Force’s basis for making strategic airlift portfolio investment decisions is impaired, and the RERP program is at increased risk of experiencing cost overruns.

A New Modernization Program for the C-5 Will Add Further Costs

Additional modernization efforts not yet budgeted will add to future C-5 costs. Air Force officials stated that a new C-5 upgrade program is slated to begin in fiscal year 2010. The initial funding requirement is $65 million— $40 million in research, development, test, and evaluation funds and $25 million in procurement funds—to migrate all C-5s toward a standard software configuration, based on changes made in the AMP and RERP programs. Requirements previous waived on the AMP may also be addressed in the initial block of this program. Additional funding will be requested in 2012 and beyond to provide additional capabilities. According to a program official, the total requirements and funding needs for this modernization program have not been finalized. However, at this time it is not expected to be as costly as the C-5 AMP or RERP.

The eventual costs for modernizing C-5 aircraft hinge upon the decisions DOD officials make about the number and mix of strategic airlifters DOD needs in the future. If additional C-5 capability is needed, more C-5 aircraft may need to receive the RERP modification and costs will increase. On the other hand, if decision makers believe additional C-17 capability is needed in lieu of the C-5, the Air Force may be able to reduce the number of aircraft that need the AMP modification and additional modifications slated to begin in fiscal year 2010.

The Air Force Must Make a Decision Soon Regarding C-17 Acquisition and Eventual Shutdown of the Production Line

Results from the two mobility studies, potential C-5 retirements, and future modernization cost increases could lead to a decision to extend C- 17 production beyond the 205 now authorized. The production line is currently scheduled to close in September 2010 with the supplier base and portions of the line closing sooner. A well-reasoned near-term decision on the final force size could avoid substantial future costs from ending C-17 production prematurely and later restarting production. Eventually, the Air Force is responsible for providing the substantial funding for shutdown expenses. Contractor and Air Force cost estimates vary widely, ranging up to $1 billion and more, and need to be reconciled. Only $37 million has been budgeted thus far.

Careful Planning Needed to Avoid Shutting Down and Restarting the Production Line

Two alternatives to completely shutting down the C-17 production line would be to slow down the rate of production or to close out current production while preserving some ability to restart the line should conditions change and additional C-17s are needed. Both options are not attractive. Slowing down production raises unit costs due to smaller annual quantities. Analysis indicates that, once closed, it would not be feasible or cost effective to restart production due to costs for hiring and training a new workforce, reinstalling tooling, and reestablishing the supply base. Therefore, careful planning to firmly and timely establish future C-17 fleet requirements is important to avoid unnecessary costs associated with a slowdown or a premature shutdown.

Our review of contractor and Air Force plans identified the following major challenges and substantial costs for restarting production: A workforce of nearly 3,100 people would have to be hired and trained following a complete shutdown. Boeing officials believe that the vast majority of its current workforce will not be available for reemployment 1 year after the shutdown. A union representative stated that workers, whose average age is around 57 years, will either retire or find other jobs during the shutdown period. Boeing does not have the ability to move its workers to other production lines at its Long Beach, California, plant because the C-17 is the last aircraft being built at that facility.

Production tooling would need to be dismantled, stored, and maintained for the entire length of the shutdown. Tooling would then have to be reinstalled and restored to its original working condition, which officials believe would be no small feat considering the significant amount of automation involved in the production process. In addition, new tooling may need to be procured depending on design changes that may occur during the shutdown period.

Suppliers would have to be identified and their processes and parts would have to be qualified. Boeing officials stated that some suppliers it deals with now may not be available and many other suppliers may not participate in the C-17 production program because they may develop alternative business ventures following the shutdown. Suppliers of certain specialty parts are of utmost concern.

Cost estimates to shut down and restart the C-17 production line are very substantial. In 2006, when various shutdown options were being evaluated, Boeing estimated that the cost to shut down and restart would be about $918 million; the Air Force estimated $2 billion. Table 4 shows the main assumptions used to develop these estimates.

Boeing and the Air Force used different cost-estimating assumptions when completing their production shutdown and restart estimates, and neither estimate reflects all potential costs. For example, Boeing’s estimate only addresses mothballing the production facility in anticipation of restarting production, whereas the Air Force’s estimate includes the costs to stop and restart production as well as a final shutdown that would occur after a second production run is completed. Neither Boeing nor the Air Force included costs to hire and train approximately 3,100 workers or to make configuration changes that may have occurred to the aircraft during the time period when the line was shut down. According to Boeing and Air Force officials, these two cost categories could add millions of dollars to their production shutdown and restart estimates.

Air Force Has Not Budgeted Sufficient Funds for the C-17 Production Shutdown

At this writing, the Air Force has allocated $37 million in its outyears budget for C-17 production shutdown, far less than what is believed to be needed. As above, cost estimates vary and were not consistently derived. Identifying and reconciling the full costs of shutdown is needed. The Air Force plans to negotiate and issue a postproduction contract to Boeing to identify the disposition of tooling and other production assets and to ensure that a viable sustainment supplier base has been established when production has ended.

In 2006, Boeing and the Air Force developed considerably different estimates of about $1 billion and $465 million, respectively, for the cost of shutting down production completely. Table 5 below compares the two cost estimates developed in 2006 and the underlying assumptions that contributed most significantly to the differences.

Differences between the two estimates can largely be attributed to the underlying assumptions used. For example, in assessing alternatives for shutting down the C-17 production facility at Long Beach, California, Boeing considered three options for the site: redeploy the property to Boeing for reuse, treat the property as an operating asset, or sell the property. Boeing officials concluded that the current production facility would not be used for future business and should be sold. Therefore, they included costs to demolish facilities and for environmental remediation in their estimate. Air Force officials believe demolition and environmental remediation costs should not be part of the shutdown costs and did not include them.

Since 2006, Congress has provided additional funding that the Air Force is using to procure additional C-17s. Thus, DOD officials stated that the requirement to shut down the production has been delayed 1 year and now estimate the shutdown costs to be $505 million, due to inflation. Boeing has not updated its estimate.

Conclusions

Strategic airlift is an essential, key enabler of U.S. military capability to rapidly project forces worldwide. The C-5 and C-17 provide complementary capabilities. However, DOD continues to struggle with identifying the specific quantities and determining the optimal mix of aircraft needed. Clarity is needed before committing additional billions of dollars to C-5 modernization programs, establishing C-5 retirement schedules, and/or acquiring additional C-17 aircraft. Careful planning is also important to avoid the costs of shutting down the C-17 line prematurely and later deciding to restart the production. The new mobility studies, if done correctly, could bring clarity to strategic airlift capabilities needed to support the future force and changed threats, as well as inform future tactical airlift requirements because of the C-17’s dual role. Important metrics left out of the 2005 capabilities study—such as specific ton-mile mobility requirements and relative reliability rates—are considered critical factors in quantifying and analyzing cost-effective force mixes. DOD concurred with our prior recommendation to use mobility metrics to inform future mobility capabilities studies. However, at this writing, it is unclear whether DOD will use a million ton-mile metric in its current analysis to determine the cost-effective mix of aircraft and guide important investment decisions related to the expenditure of billions of dollars. Until comprehensive requirements—supported by appropriate, quantifiable metrics—and the full costs for alternate courses of action are identified, DOD decision making on the future size and mix of strategic airlift is hampered, thus increasing the risk of incurring unnecessary costs and establishing a less than optimal mix of strategic and tactical airlift forces.

Recommendations for Executive Action

To better inform decision makers and provide improved and quantifiable projections of airlift mobility requirements and investment needs, we recommend that the Secretary of Defense direct the Office of the Secretary of Defense (Program Analysis and Evaluation) and U.S. Transportation Command to take the following action: Ensure that the new mobility capabilities study specifies the ton-mile per day metric and other relevant metrics to support sound strategic airlift decisions, including (1) the cost-effective mix of C-5 and C-17 aircraft consistent with national security strategy; (2) the number of C-5s required to airlift equipment that can only be carried by that aircraft; (3) C-5A retirement schedules, if warranted by analysis; (4) the number of C-17s needed to accomplish both its strategic and tactical roles; and (5) future procurement and modernization needs.

We also recommend that the Secretary of Defense direct the Secretary of the Air Force to take the following two actions: Prepare updated, reliable, and comprehensive cost estimates for C-5 modernization to include risk and sensitivity analyses on major cost drivers for the RERP, the costs of a new modernization program, and potential savings should some C-5A retirements be warranted.

Identify and budget for the full costs to shut down the C-17 production line in the time frame consistent with final decisions on the future size of the C-17 fleet.

Agency Comments and Our Evaluation

DOD provided us with written comments on a draft of this report. DOD concurred with one recommendation, partially concurred with another, and did not concur with one recommendation. DOD’s comments appear in appendix II. DOD also provided technical comments on our draft report which we incorporated as appropriate in the report.

In its comments, DOD partially concurred with the draft recommendation to ensure that the current mobility capabilities study specifies a ton-mile per day metric and other relevant metrics to support sound strategic airlift decisions. The department stated that the Office of the Secretary of Defense (Program Analysis and Evaluation) and U.S. Transportation Command, offices that are jointly working on the mobility capabilities study, will specify metrics, including a ton-miles per day metric, for the study. Accordingly, we redirected our recommendation to these offices instead of the Air Force.

DOD did not concur with the recommendation to prepare updated C-5 modernization cost estimates. DOD stated that the modernization is being accomplished in two phases—the AMP and RERP—and that the third modernization effort we identified that is slated to begin in fiscal year 2010 is a sustainment program that would fall below the ACAT II threshold. Further, department officials believe that the most recent cost estimates for the AMP and RERP are sufficient and areas of concern and mitigating factors related to the RERP estimate were identified during the Nunn- McCurdy process.

We continue to believe updated estimates are warranted. We are not suggesting that the department revisit its RERP Nunn-McCurdy decision. For the most part, we believe the CAIG did an adequate job identifying the costs associated with 14 different RERP options, especially given the short time frame in which the estimates had to be completed. However, the CAIG did not perform risk or uncertainty analysis, which is considered to be a cost-estimating best practice. Since the program is not fully funded and the Air Force has not yet completed its analysis of mobility requirements, there remains significant risk that funding will not be available in outyears or that the quantities of C-5s and C-17s could change. Therefore, we believe it is important that DOD identify a range of potential costs based on different quantities, so that decision makers will have adequate information for future budget decisions related to the number and mix of strategic airlifters.

DOD concurred with our recommendation to identify and budget for the full costs to shut down the C-17 production line consistent with final decisions on the future size of the C-17 fleet.

We are sending copies of this report to the Secretary of Defense and interested congressional committees. The report is also available at no charge on the GAO Web site at http://www.gao.gov.

If you have any questions about this report or need additional information, please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report were Bruce Fairbairn, Assistant Director; Cheryl Andrew; Marvin Bonner; John Crawford; Karen Sloan; Karen Richey; and Marie Ahearn.

Appendix I: Objectives, Scope, and Methodology

This report examines the Department of Defense’s (DOD) efforts to meet its strategic airlift requirements. Specifically, we (1) identify the impact C- 5 modernization cost increases have had on the mix of aircraft DOD needs to meet its strategic airlift requirement, (2) assess the current C-5 modernization cost estimate, and (3) identify C-17 production plans and issues related to a production line shutdown.

To determine the impact C-5 modernization cost increases have on the mix of aircraft DOD needs to meet its strategic airlift requirement, we compared the mix of aircraft DOD planned to use to meet the strategic airlift requirements identified in a 2005 mobility study to the mix of aircraft DOD currently plans to use. We collected information on the 14 options DOD considered to meet its strategic airlift requirements following the C-5 Reliability Enhancement and Reengining Program (RERP) cost breach notification to Congress. We also obtained cost data from the Cost Analysis Improvement Group (CAIG) related to the 14 options. We met with appropriate officials from the Office of the Secretary of Defense, the Office of the Secretary of the Air Force, and Air Mobility Command to obtain their opinions on the mix of C-5 and C-17 aircraft needed to meet the current mobility requirement. When available, we obtained analytical documentation supporting the views of these officials.

We also considered the impact decisions related to the current, ongoing mobility studies, pending congressional legislation, and further C-5 cost increases could have on the future mix of C-5 and C-17s. We identified issues related to DOD’s 2005 mobility study and discussed DOD’s efforts to address these same issues in the current study with DOD officials.

To determine the current cost estimates for the C-5 modernization efforts, we collected cost estimates for the C-5 Avionics Modernization Program (AMP) and C-5 RERP. We also gathered preliminary cost data for the modernization program slated to begin in fiscal year 2010 from program officials. We used C-5 AMP cost data from the December 2007 Selected Acquisition Report and C-5 RERP cost data from the June 2008 Selected Acquisition Report. The CAIG identified funding shortfalls for the C-5 RERP program.

During our examination of C-5 RERP costs, we collected various cost estimates and supporting documentation developed by Lockheed Martin, the DOD CAIG, and the Air Force. We met with appropriate officials from each of these organizations to determine the underlying assumptions of their respective estimates and to understand key differences between the estimates. Since the CAIG’s estimate serves as DOD’s official estimate for the restructured C-5 RERP program, we compared how that estimate was developed to GAO’s guide for estimating and managing program costs to determine the reliability of the DOD’s cost estimate. We also interviewed officials from the Institute for Defense Analysis to discuss their analysis and findings related to the various C-5 RERP cost estimates.

To identify C-17 production plans and issues related to the C-17 production line shutdown, we collected information on the number of C- 17s DOD is currently authorized to procure and discussed production shutdown time frames with Boeing, the prime contractor. We collected cost estimates and underlying assumptions Boeing and the Air Force made in 2006 for shutting down the production line completely and shutting down and restarting the production line at a later time. Additionally, we obtained the opinions of Boeing and Air Force program officials about the costs and challenges associated with shutting down and restarting the production line. Our discussions with both of these organizations focused on potential personnel, supplier, and tooling issues. We also discussed this topic with Boeing union officials who have dealt with downsizing efforts at Boeing’s Long Beach, California, facility and Lockheed Martin officials who were involved with shutting down and restarting the C-5 production line during the 1970s and 1980s at the Marietta, Georgia, plant. Finally, we reviewed DOD budget documents to determine how much had been budgeted for the C-17 production line shutdown.

We conducted this performance audit from February 2008 to November 2008 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense