Defense Acquisitions: Sound Business Case Needed to Implement Missile Defense Agency's Targets Program
Released: 2008-09-26T00:00:00

Letter

Background

MDA is the agency responsible for designing, developing, producing, and procuring targets for testing the nation’s ballistic missile defense system. In 2001, MDA initiated a Targets and Countermeasures Program Office. The MDA’s Targets and Countermeasures program oversees the design, development, fabrication, and production of ballistic missile targets and countermeasures to be used in the MDA flight test program. These targets must represent the full spectrum of threat missile capabilities (separating and nonseparating reentry vehicles, varying radar cross sections, countermeasures, etc.) and ranges. The appropriate targets are engaged by both strategic and tactical missile defense systems developed by MDA, the Army, and the Navy.

MDA originally used numerous contractors to design and build the various targets; however in December 2003 Lockheed Martin was awarded a contract to be the lead systems integrator for most of the existing targets. The lead systems integrator was chosen to provide system-level management, implement a single organization and management structure, and to manage all processes. Additionally, MDA contracted with Lockheed Martin to plan and develop a new family of targets called the Flexible Target Family (FTF) because existing targets were deemed no longer able to meet test needs due to aging components and an evolving threat. MDA’s original goals for FTF included commonality of components across the family of short, medium and long-range targets, the purchase of inventory, reduced cycle time in the building of individual targets, increased complexity, cost savings, and ground, air, and sea launch capabilities.

Although the development of FTF began in 2005, the first FTF target has yet to be delivered.

While MDA manages the targets program, the Missile Defense Executive Board (MDEB), which was created in March 2007, provides advice and recommendations on the missile defense program. Specifically, the MDEB reviews the implementation of strategic policies and plans, program priorities, and investment options for protecting the United States and its allies from missile attacks. The MDEB is chaired by the Under Secretary of Defense for Acquisition, Technology and Logistics, who is authorized to make recommendations to the Deputy Secretary of Defense on missile defense issues. According to its charter, the board will provide to the Under Secretary a recommended strategic program plan and a feasible funding strategy for approval. Recommendations will be based on a business case analysis that considers the best approach to field integrated defense capabilities in support of joint objectives. A consistent business case analysis approach will facilitate the delivery of capabilities to the war fighter in a timely, efficient, cost-effective manner. The board will review the missile defense program and recommend changes in the missile defense strategic plan. In addition, Congress has mandated that DOT&E annually assess the adequacy and sufficiency of the BMDS test program. Congress has also mandated that DOT&E perform other BMDS program related test and evaluation responsibilities.

Increased Target Failures and Anomalies Have Made It Difficult to Supply Reliable Targets On Time

The number of target failures and anomalies during flight tests has increased over time. From 2002 through 2005, 7 percent (3 of 42 launches) of the targets launched had in-flight target failures and anomalies, while from 2006 through 2007, 16 percent (6 of 38 launches) of the targets launched had in-flight failures or anomalies. Though problems providing reliable and timely targets have negatively affected many of the missile defense elements, THAAD and GMD missile defense elements have been affected the most. GMD experienced a target failure in 2007 in which the primary objectives failed to be achieved. THAAD experienced one failure and two anomalies between 2006 and 2007.

The immediate effect of a target delay or problem is on an individual flight test. For example, unavailable targets have delayed tests such as FTT-08 and FTM-15. There are longer term effects of these delays on subsequent tests and demonstration of capability in the flight test program. Specifically, MDA’s flight test program has been restructured to reflect target deficiencies and availability. For example, validating GMD capability has been delayed due to target availability and a failure. The THAAD flight test program has decreased the amount of flight tests due to reduced availability of targets. Additionally, target failures and anomalies caused THAAD to replan key objectives in their flight test plan due to the inability to collect necessary information during flight tests.

Since we briefed your staff in July 2008, MDA has experienced two additional target anomalies. The target flown in a recent sensor test (FTX- 03) under-flew the planned trajectory. Test reports indicate that target trajectory fell short of what was planned and thus did not reach the anticipated intercept area. While the anomaly did not prevent MDA from achieving the sensor test objectives, had this been an intercept attempt, the target anomaly would have precluded the launch of the interceptor due to safety restrictions on the test range. In September 2008, the THAAD element was unable to complete a planned flight test because the target missile malfunctioned and did not have enough momentum to reach the intercept area. Therefore, the two THAAD interceptors taking part in the flight test were not launched.

At the same time, the unit cost of targets has grown exponentially, from $4.5 to $8.5 million from 2002 to 2006 to current estimates of $32 to $65 million for the targets planned from 2008 to 2010. Many factors contribute to this cost growth, including increased complexity of targets to better reflect an evolving threat and late changes to target requirements on contract. In addition, the shift to a lead systems integrator resulted in an increase in composite hourly labor rates. MDA has also encountered development problems with the FTF program and the total cost of FTF remains unclear. According to information provided by MDA in July 2008, nonrecurring engineering costs plus the cost to purchase some test assets is at least $1 billion, of which about $553 million has already been spent.

Several Factors Contribute to Problems with Targets

Difficulties with target availability are traceable to (1) supplier base and problems incorporating requirements into the contracts and establishing program baselines for existing targets and (2) the lack of a sound business case for FTF development. Regarding existing targets, inventory is aging for key components. For example, motors used in some targets are now over 40 years old. In addition, vendors who previously supplied targets to MDA are leaving the market due to lack of business. MDA has also experienced program and requirements issues with the existing targets on contract. According to MDA officials, several target contracts did not capture all of the needed test requirements. Furthermore, target technical, cost, and schedule baselines were not always established in a timely manner. MDA officials acknowledged that they have only recently established these baselines for all missions planned to take place in the next 2 years.

MDA began the FTF without establishing a sound business case. We have previously reported on the importance of using a solid, executable business case before committing resources to a new product development effort. A sound business case demonstrates that (1) the identified needs are real and necessary and are best met with the chosen concept and (2) the chosen concept can be developed and produced with existing resources—such as technical knowledge, funding, time and management capacity. According to a July 2006 MDA briefing, the decision to commit to a lead systems integrator and FTF was not based on a formal cost or business case analysis. The briefing further indicated that the FTF cost estimates considered at the time only included integration costs and did not address nonrecurring engineering costs, unit costs for FTF configured targets, or the costs to purchase existing target systems through the prime contract. Nor was there any indication that FTF costs had been reviewed by independent cost organizations within the Department of Defense. It is also unclear whether MDA evaluated all alternatives before committing to a lead systems integrator and the FTF. MDA did perform a study to identify requirements design attributes and candidate configurations for a family of targets by reviewing existing systems. However, in a July 2006 assessment, MDA stated that other alternatives for improving legacy targets were not considered at that time.

We also could not identify an original approved FTF acquisition strategy. Originally, the lead systems integrator proposed using existing launch vehicles for short- and medium-range missions, and developing a new long-range launch vehicle family while maximizing the reuse of existing target system designs. However, MDA committed to a riskier strategy that required developing a single common architecture across three target system families. Early in the development of FTF, MDA underestimated the technical and design challenges involved in the development of a new target family. By May of 2006, MDA recognized that the funding set aside for FTF development was no longer adequate and made a decision to focus developmental efforts on the ground-launched 72-inch long-range target needed for near-term missions and defer other key capabilities.

Despite the focus on the 72-inch ground-launched target, development problems have delayed the first launch from July 2008 to April 2009. The 72-inch target qualification process has been more difficult and costly than expected and development is still not complete. Of the 69 components in development, 24 had not passed the qualification process as of August 2008. In addition, according to the targets program director, the qualification process completion date has moved from early October to November 2008. Delays have also occurred because development of the 72-inch target began without a stable design. Eleven of the remaining unqualified components are considered critical technologies and are currently being redesigned. MDA acknowledged both underestimating the difficulty in using components never qualified for this shock and vibration environment and underestimating the complexity of combining multiple launch vehicle and mission requirements. Similarly, the 52-inch target’s first flight will occur no earlier than fiscal year 2012, representing a 3 year slip from the original expectations of first flight in fiscal year 2009. This slip has resulted in the need to initiate an interim target development effort to satisfy near-term target requirements of the Aegis flight test program.

MDA Is Undertaking A Series of Initiatives To Address Problems with Targets Development, but Challenges Remain

MDA recently began implementing a series of management initiatives to improve the targets program, such as establishing technical, cost, and schedule baselines for all missions in the 2-year integrated master test plan. The new program director and staff have also begun drafting long- term target capability requirements, and developing and implementing a new cost model for targets. In addition, plans have been made to improve risk management by (1) considering program-wide issues, and (2) including programmatic risks, cost and schedule in the risk assessments.

Overall, the success of the flight test program depends on the success of FTF. However, the current FTF acquisition strategy will yield fewer targets at higher costs and the entire FTF might not be fully pursued. For example, key capabilities such as the air-launched targets have been deferred and MDA is considering whether to continue with the 52-inch target development. Further, the initial plan for 10 to 12 72-inch target flights per year has been reduced to 4 to 5 flights per year. In addition, the development and production of the first four 72-inch targets cost has grown 63 percent, from $248 million to at least $405 million, and the cost to complete development of the entire FTF is not known. Given that the GMD and THAAD flight test programs have already been negatively affected by target difficulties, further changes to the flight test program could become necessary if the FTF program does not deliver its capabilities as scheduled.

Conclusions

Targets are sophisticated and expensive missiles that are essential to flight testing the ballistic missile defense system. As such, management of the targets program must be both effective in supplying enough reliable targets and efficient as to their cost. Increasing problems with target failures and anomalies have caused degradations in the missile defense flight test program. The strategy of supplanting the use of existing targets, which contain aging components, with a new family of targets, has not gone as planned. The new FTF has been delayed, costs have increased, and the program’s scope is being reduced. The difficulties with executing FTF are due, at least in part, to the fact that a sound business case was not established before proceeding with the program and the attendant contracting strategy. MDA has recently initiated a series of actions to improve the targets program in general, and the FTF program specifically. Even so, the availability of targets for flight tests continues to be problematic, and as a result the scope of the flight test program has been reduced to better match available targets. The availability of targets is also dependent on the ultimate success of the FTF program.

Recommendations For Executive Action

We recommend that the Secretary of Defense take the following two actions: Require the Director, MDA to establish a revised business case for providing reliable and timely targets for a robust flight test program. Such a business case should, among other things, determine the best approach for providing an ample supply of quality targets to support the missile defense flight test program, including consideration of approaches other than the FTF. demonstrate that the chosen approach—including the attendant acquisition and contracting strategy–can be executed with available technologies, funding, time, and management capacity. establish separate cost, schedule, and performance baselines for each class of target under development against which progress can be measured. reflect input from key organizations, such as the Missile Defense Executive Board, independent cost estimators, and the Director, Operational Test and Evaluation.

Align MDA’s plans and resources with the targets program approach resulting from the above business case.

Matters For Congressional Consideration

The Congress has repeatedly stressed the need for robust testing of the Ballistic Missile Defense System and has become concerned with the health of the MDA targets program. The Congress has also expressed concern that the FTF program is proceeding at a slower pace and greater cost than expected. Therefore, Congress may wish to consider whether to require the Secretary of Defense to report the departmentally approved missile defense target development and procurement strategy, business case, and baselines to the Congressional Defense Committees.

Agency Comments and Our Evaluation

DOD provided written comments on a draft of this report. These comments are reprinted in appendix II. DOD also provided technical comments, which we incorporated as appropriate.

In our first recommendation, we outlined four components that the Secretary of Defense should require in a revised business case for the missile defense targets program. DOD partially concurred with this recommendation. In responding to the first component of the business case, which is to determine the best approach for providing an ample supply of quality targets, DOD stated that MDA is reviewing the acquisition strategy for targets. It is important that the review be a fresh look at the business case for targets, which is why we recommended that approaches other than the FTF be considered as well. DOD’s response failed to make it clear whether such approaches will be included in the MDA review.

In responding to the second component of the business case, to demonstrate that the chosen approach could be executed with available technologies, funding, time, and management capacity, DOD explained that the review underway will result in an approach that will be technologically achievable, executable within budget, and will support the flight test schedule. DOD noted that the delivery of the first four 72-inch FTF targets will inform the review in the area of cost. While we agree that such cost information will be important, we also note that the 72-inch targets have already experienced cost overruns, qualification problems, and delivery delays. As such, they remain unproven, which underscores the need for the business case review to remain open to other approaches to supplying targets.

DOD also partially concurred with the third component of the business case, to establish cost, schedule, and performance baselines for each class of target. DOD indicated that baselines are established for each individual target procured and that a baseline for each class of target would be redundant. While individual baselines are appropriate for legacy systems in production, the key issue is establishing baselines for new classes of targets under development, like the FTF. In the course of our review, MDA could not provide a full cost estimate or a baseline for the FTF developmental effort. We have modified the recommendation to deal specifically with establishing baselines for new classes of targets under development, like the FTF.

DOD partially concurred with the last component of the first recommendation, which recommends that the business case reflect input from key organizations, such as the MDEB, independent cost estimators, and DOT&E. DOD acknowledged that the MDEB and DOT&E inputs have been extremely valuable but that the decision regarding the targets business case should be reserved for the Director of MDA. Our recommendation calls for the input of key organizations but does not suggest that their approval is required. Thus, it does not pose a challenge to the MDA Director’s decision making authority. MDA has not always obtained such input on key decisions. For example, DOT&E was not consulted for the recent decision to cancel an important GMD flight test. Also, estimates of MDA costs have typically not been obtained from independent cost estimators.

Regarding our second recommendation to align MDA’s plans and resources with the targets program, DOD partially concurred, stating that revisions to the acquisition strategy will be incorporated into the Program Budget development.

Finally, DOD did not concur with a third recommendation, subsequently removed from the final report, to report the departmentally approved missile defense target development and procurement strategy, business case, and baselines to the Congressional Defense Committees. DOD preferred to rely on briefings to Congressional staff to convey this information. MDA began developing the new target family without a complete acquisition strategy, baselines, or a business case and the program has already experienced qualification problems, schedule delays and cost overruns. Because of the importance of an ample supply of reliable targets to the success of the Ballistic Missile Defense System, we continue to believe that the Secretary of Defense needs to ensure that MDA takes a fresh look at alternatives when it establishes a revised acquisition strategy and business case. The departmentally approved approach should then be reported to the Congressional Defense Committees because MDA’s targets program and strategy are of interest to the Congress, as evidenced by the language that requested our assessment. Accordingly, we have elevated the recommendation for executive action to a matter for congressional consideration.

We will send copies of this report to the Secretary of Defense, the Director, MDA, appropriate congressional committees, and other interested parties. This report is also available at no charge on GAO’s Web site at http://www.gao.gov. If you or your staff have any questions concerning this report and its appendixes, please contact me at (202) 512- 4841 or FrancisP@gao.gov. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report. The major contributors are listed in appendix III.

Appendix I: Briefing to Defense Committees on Review of the Missile Defense Agency’s Targets Program

Briefing Overview

Acronyms

Introduction

GAO undertook this work in response to a mandate contained in the Conference Report accompanying the National Defense Authorization Act for Fiscal Year 2008 (H.R. Rep. No. 110-477).

We focused our review on the following three objectives: (1) Is the Missile Defense Agency (MDA) meeting the need for reliable and timely targets for the flight test program? (2) What are the causes of any deficiencies or delays? (3) What are the prospects for resolution of any problems identified?

We will follow up on this briefing by September 30, 2008, with a report.

Technical comments from the agency were incorporated as appropriate. Our scope & methodology is described on slide 29.

Background

The Ballistic Missile Defense System (BMDS) and its elements verify design performance and assess capabilities in a variety of ways, including flight testing BMDS missiles to track or intercept live targets that represent real-world missile threats.

MDA’s Targets and Countermeasures program oversees the design, development, fabrication, and production of ballistic missile targets and countermeasures to be used in the BMDS test program. Target systems are ballistic missiles and are available in numerous configurations designed to test the performance of the system’s sensors and weapons.

The target systems include launch vehicles, payloads, instrumentation, and flight controls.

Existing Targets

Flexible Target Family

Lead Systems Integrator

In December 2003, MDA awarded Lockheed Martin a contract to be the Lead Systems Integrator (LSI) for most of the existing targets and FTF target development.

Lockheed Martin now subcontracts with vendors who previously supplied targets directly to MDA.

MDA has Difficulty Supplying Reliable

In-flight target failures and anomalies All targets used to date were predecessors to FTF.

FTF has not delivered a new development target yet.

Existing targets include off-the-shelf components with some modifications, new productions of existing designs, and modifications.

Effect of target problems on elements is greater than in-flight anomalies and failures. THAAD and GMD affected most by target problems.

Target Delays and Performance Issues Since 2005

FTF Development Delays

First FTF launch was scheduled for July 2008, now April 2009 Deferrals of subsequent developmental work LV-2 air launch, all sea launch, LV-3, Common-liquid Rocket Boosters, and the 52” configured FTF have been deferred. 52” configured FTF was assumed to be available for first use in FY 2009.

Recurring Target Unit Costs Growing

Factors Affecting Target Cost Growth

increasing complexity of targets (including existing targets) late changes to target requirements on contract schedule delays shift to Lead Systems Integrator – higher labor rates, additional oversight of subcontractors Cost increases associated with recent target failures and anomalies are only partially estimated.~$87 M cost increase associated with FTG-03 failure ~$80 M associated with target availability and affordability Unknown cost implications for Radar Data Collection “anomalies”

Cost of Flexible Target Family Unclear; Cost

MDA has estimated some cost for FTF, but total nonrecurring development remains unclear because some cost figures include the purchase of test assets.

According to information MDA provided July 15, 2008, nonrecurring cost plus the cost to purchase some test assets are estimated at about $1.2 billion.

Sunk costs are about $553 million.

Objective 2: Causes of Problems

Supplier base problems with existing targets: Aging inventory/diminishing sources Motors used in some targets are over 40 years old. Vendors are leaving the market due to lack of business.

Target contracts have not captured all element testing requirements. Target technical, cost, and schedule baselines were not always established before targets were placed on contract. MDA acknowledges they have recently established these baselines for all missions in the next 2 years.

Flat charges assessed to each element do not represent actual use. Flight test costs.

Objective 2: Causes of Problems

According to MDA, contracting with an LSI increased costs; the magnitude of the increase exceeded expectations.

For example, composite hourly labor rates were 32% higher in 2004 than prior to the contract with the LSI.

Effective cost of subcontractor labor is higher on prime contract due to management oversight of subcontractors.

First FTF target delayed from July 2008 to April 2009 because the ground launched (72”) target encountered environmental qualification problems.

Heritage components put in unverified environment, failures due to high vibration and shock levels.11 of 69 components are considered critical.

As of April 2008, all 11 were in redesign or failure analysis. 5 of these critical components were not projected to complete qualification testing until September at the earliest.

As of April 2008, 35 components had not completed qualification testing. By June 2008, 28 components remained unqualified.

MDA acknowledged (1) underestimating the difficulty in using components never qualified for this shock and vibration environment and (2) underestimating the complexity of combining multiple launch vehicle and mission requirements.

Design Challenges Contributed to FTF Delay

According to MDA, they did perform a trade study to

identify requirements, design attributes, and candidate configurations for a family of targets by reviewing existing systems.

In a July 2006 assessment, MDA stated that other alternatives for improving legacy targets were not considered at that time.

Objective 2: Causes of Problems

Objective 2: Causes of Problems

According to a July 2006 MDA briefing, the decision to adopt the FTF and a LSI was not based on a formal cost or business case analysis.

FTF original cost estimates only included the integration costs and did not address: nonrecurring engineering costs, unit costs for FTF configured targets, or the costs to purchase existing target systems through the prime contract.

A formal analysis of the relative merits and costs of each alternative could not be performed when the commitments to the LSI and the FTF were made because cost estimates were not available.

Objective 2: Causes of Problems

We could not identify an original approved FTF acquisition strategy.

MDA did, however, provide a May 2006 FTF briefing that stated: At that time, the majority of the investment required for the FTF LV (LV-2 and LV-3) family design effort had been expended.

LV-2 development cost estimated at $248 million (FY06 dollars). Completion of entire FTF LV family is required to realize substantial yearly recurring cost reductions. Assumed 10 to 12 LV missions per year.

Completion of the entire FTF family is required to support the increased pace and complexity of BMDS testing 2-4 years in the future.

Objective 2: Causes of Problems

Design and cost implications of the FTF requirements were underestimated.

In May 2006, MDA acknowledged that significant technology and design challenges were inherent in FTF.

Initial FTF requirements “levied functional capabilities and operational environments far above existing targets’ current qualifications.”

Choice of nonproprietary, single common architecture required new “clean-sheet-of-paper” designs. Only capable of marginal reuse of existing designs.

Funding set aside was no longer adequate to resource the development. Decision made to focus on ground-launched LV-2 for near- term missions, and delay other key capabilities.

Objective 3: Prospects for Resolution

Full FTF LV family may not be fully pursued.

The LV-3 and the LV-2 air-launched capabilities are deferred; LV-2 air launch considered a key capability which provides greater test flexibility.

Assumption of 10–12 LV-2 flights per year, including 4 GMD tests, is no longer valid, expect only 4–5 per year. The development and production costs of the first 4 LV-2s have grown from $248 M to $405 M (a 63 percent increase).

MDA is considering whether to continue with 52” development First 52” flight will occur no earlier than FY12 (delayed from FY09).

The cost to complete development of the FTF is not known.

Objective 3: Prospects for Resolution

MDA recently began implementing a series of management initiatives for BMDS target acquisition. Establishing technical, cost, schedule baselines for all missions in the 2-year integrated master test plan.

Implementing a requirements stabilization process requiring higher-level approval for target changes.

Drafting long-term target capability requirements Drafting initial comprehensive plan to validate system-level models and simulations.

Improving risk management by (1) considering program-wide issues, (2) including programmatic risks, cost & schedule.

Developing and implementing a new cost model for targets. New program director and personnel.

Objective 3: Prospects for Resolution

BMDS flight test program has changed to reflect target availability and deficiencies.

THAAD flight test program extended. GMD flight test objectives delayed.

Success of flight test program assumes success of FTF.

If everything goes as planned, the targets program may be able to meet the future needs of the BMDS test plan. MDA’s Integrated Master Test Plan is modified as often as every quarter.

Key deferred work not in funding plan–including air- launched LV-2 capability.

Conclusions

The decision to pursue the FTF and the LSI was made

without demonstrating a sound business case. Specifically, MDA did not show that the FTF could be developed and produced within existing resources.

Assumptions underlying the FTF development have changed, including whether the concept of a “family” of targets can be accomplished.

MDA has taken actions to improve target acquisition deficiencies, but these actions may not be enough to provide an adequate supply of targets.

Scope, Methodology, and Auditing Standards

To assess the ability of MDA’s targets program to provide support for the BMDS test efforts, we analyzed test plans and schedules, flight test reports, budget documents, and program execution reviews. We also analyzed program directives and acquisition policies and procedures. We interviewed MDA officials from GMD, THAAD, Aegis BMD, and the Targets and Countermeasures program offices as well as officials from MDA’s Business, Test, and Engineering directorates. We also interviewed officials from DOD’s Office of the Director, Test and Evaluation, and the Space and Missile Defense Command. Our analysis covered data ranging from December 2003 through July 2008.

We conducted this performance audit from February 2008 through July 2008, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Acronyms

BMDS CDR DOT&E FTF GMD GSE LV LSI MBRV MDA RV RDC STSS THAAD

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

Major contributors to this report were David B. Best, Ivy G. Hübler, Steven B. Stern, Robert S. Swierczek, Letisha T. Watson, and Alyssa B. Weir.