Defense Acquisitions: Missile Defense Transition Provides Opportunity to Strengthen Acquisition Approach
Released: 2010-02-25T00:00:00

Letter

Background

MDA’s BMDS is being designed to counter ballistic missiles of all rang short, medium, intermediate, and long. Since ballistic missiles have different ranges, speeds, sizes, and performance characteristics, MDA is employing an integrated and layered architecture to provide multiple opportunities to destroy ballistic missiles before they can reach their targets. The system’s architecture includes networked space-based sensors as well as ground- and sea-based radars, ground- and sea-based interceptor missiles, and a command and control, battle management, and communications network providing the warfighter with the necessary communication links to the sensors and interceptor missiles.

A possible engagement scenario to defend against an intercontinental ballistic missile would occur as follows: Infrared sensors aboard early-warning satellites detect the hot plume of a missile launch and alert the command authority of a possible attack.

Upon receiving the alert, land- or sea-based radars are directed to track the various objects released from the missile and, if so designed, to identify the warhead from among spent rocket motors, countermeasures, and debris.

When the trajectory of the missile’s warhead has been adequately established, an interceptor—consisting of a kill vehicle mounted atop a booster—is launched to engage the threat. The interceptor boosts itself toward a predicted intercept point and releases the kill vehicle.

The kill vehicle uses its onboard sensors and divert thrusters to detect ead. With a combined closing identify, and steer itself into the warh speed of approximately 10 kilometers per second (22,000 miles per hour), the warhead is destroyed above the atmosphere through a to kill” collision with the kill vehicle.

Some interceptors use sensors to steer themselves into the inbound ballistic missile. Inside the atmosphere, weapon systems kill the ballistic missile using a range of mechanisms, such as direct collision between the interceptor missile and the inbound ballistic missile, or using the combined effects of a blast fragmentation warhead (heat, pressure, and shrapnel) in cases where a direct hit does not occur.

In the August 2009 BMDS Accountability Report, MDA presents the BMD performance from the perspectives of homeland defense and regional/theater capabilities. Homeland defense uses the capabilities of s Ground-based Interceptors (GBI), Aegis BMD assets, and BMDS radar against the threat from intercontinental and intermediate-range ballist ic missiles, while regional and theater defense use Aegis BMD Standard Missile-3 (SM-3) and THAAD interceptors with mobile radars against threats from medium-range missiles and short-range ballistic missiles.

Table 1 provides a brief description of eight BMDS elements that are currently under development by MDA.

Significant Adjustments Made to Missile Defense in Fiscal Year 2009

The new administration proposed significant changes to the BMDS program in 2009 including program terminations and changes to some of the BMDS elements we reported on in the past, as well as changes to plans for missile defense in Europe. Administration proposals culminated in reductions of approximately $1 billion from MDA’s budget request for fiscal year 2010. In the spring of 2009, the Secretary of Defense recommended termination of the Multiple Kill Vehicle element. Originally designed as an optional warhead for all midcourse interceptors, MDA terminated the Multiple Kill Vehicle element because of feasibility issues raised about this technology, which was still in its early stages of development, as well as a decision to refocus MDA’s resources on new technologies aimed at early intercept of ballistic missiles. MDA also terminated its Kinetic Energy Interceptor element because of technical issues, its incompatibility with operational infrastructures, and delays during development. It was originally designed as a mobile land-based missile defense system to destroy medium, intermediate, and intercontinental ballistic missiles during the boost and midcourse phases of their flight. The ABL program was also significantly affected by the Secretary of Defense’s proposal to designate it as a technology program and cancel the plans for the purchase of a second aircraft that would have provided an operational capability. In addition, MDA requested increased funding for the Aegis BMD and THAAD programs for fiscal year 2010 following administration recommendations. MDA plans to use these funds to move both elements toward meeting full funding policies, to increase production for Aegis BMD and THAAD interceptors, to increase the interceptor production rate and number of THAAD batteries, and to increase the number of Aegis BMD ships.

MDA is also responding to the new administration’s shift in its approach to European missile defense. In September 2009, DOD altered its approach to European defense, which originally focused on GBIs from the GMD element and a large fixed radar as well as transportable X-Band radars, and is now focusing on providing defenses against long-range threats to the United States and short-, medium-, and intermediate-range Iranian threats to Europe. This new “Phased, Adaptive Approach” consists primarily of Aegis BMD sea-based and land-based systems and interceptors, as well as various sensors to be deployed over time as the various capabilities are matured. According to DOD, this new approach offers a number of improvements over the previous architecture, such as providing missile defenses sooner with greater flexibility to meet evolving threats, providing more opportunities to involve close allies, and delivering greater capability to defend against a large number of threat missiles.

In addition, during fiscal year 2009, MDA transitioned to a new Director and the agency’s development effort was rebalanced to focus more on regional/theater missile defense. This rebalancing included shifting technology development efforts from boost-phase intercept technologies to early intercept technologies (or ascent phase). MDA officials state that because early intercept technology initiates intercept as early as possible to execute a shoot-look-shoot tactic and defeat a threat before countermeasures are deployed, it will ultimately reduce the number of interceptors required to defeat a raid of threat missiles and save on the costs of maintaining a significant number of expensive interceptors to destroy advanced countermeasures in a later phase of a threat missile’s flight. According to the MDA Director, this technology will force the deployment of countermeasures early in flight where they are less effective.

In June 2009, MDA also began to change its acquisition management strategy. From its inception in 2002 to December 2007, MDA managed the acquisition of missile defense capabilities by organizing the development effort into 2-year increments known as blocks. Each block was intended to provide the BMDS with capabilities that enhanced the development and overall performance of the system. The first 2-year block—Block 2004— fielded a limited initial capability that included early versions of the GMD, Aegis BMD, Patriot Advanced Capability-3, and C2BMC elements as well as various sensors. The agency’s second 2-year block—Block 2006— culminated on December 31, 2007, and fielded additional BMDS assets. On December 7, 2007, according to MDA in response to recommendations from GAO, MDA’s Director announced a new acquisition management strategy to better communicate its plans and goals to Congress. The agency’s new approach was based on fielding capabilities that address particular threats as opposed to a biennial time period. This approach divided fielding capabilities into five blocks.

The capabilities-based five-block approach included several positive changes, including the commitment by DOD to establish total acquisition costs and unit cost for selected block assets, including in a block only those elements or components that will be fielded during the block, and abandoning the practice of deferring work from one block to another. MDA was still transitioning to this new capabilities-based block approach when the MDA Director terminated it in June 2009—a year and a half after it was created. According to MDA, the agency terminated the capability- based block structure to address the explanatory statement accompanying the Department of Defense Appropriations Act, 2009, which stated that MDA’s “justification materials should no longer be presented in the Block format, but rather by fiscal year for each activity within the program element.” The agency has decided that it will manage the BMDS as a single integrated program and is in the process of determining how it will implement changes to its acquisition management strategy.

Progress Made in Fiscal Year 2009

In fiscal year 2009, MDA achieved several noteworthy accomplishments. For example, MDA revised its testing approach to better align tests with modeling and simulation needs and is undertaking a new targets development effort to resolve long-standing problems supplying sufficient and reliable targets. The agency also demonstrated increased levels of performance for some of its BMDS elements through flight and ground testing. MDA testing achievements during the year indicate an increased level of interoperability among multiple elements, improving both system- level performance and advancing the validation of BMDS models and simulations needed to predict performance. In addition, the agency delivered most of the assets as planned by the end of fiscal year 2009.

MDA Revised Its Testing Approach

In fiscal year 2009, MDA revised its testing approach in response to GAO and DOD concerns. In March 2009 we reported that MDA’s Integrated Master Test Plan—its test baseline—was not effective for management and oversight because it was revised frequently, only extended through the following fiscal year and was not well integrated with other key aspects of testing such as target acquisitions. Most of the annual revisions to the test baseline were occurring either because MDA changed the substance of tests, changed the timing of tests, or added tests to the baseline. In other instances, MDA canceled planned tests which also affected the test baseline. In addition, the BMDS Operational Test Agency identified several limitations in the previous BMDS test program, including unaccredited models and simulations, flight test artificialities, and inadequate modeling of some environmental conditions.

Members of Congress also expressed concern with MDA’s test approach. For example, in the fiscal year 2008 National Defense Authorization Act conference report, conferees noted that MDA failed to ensure an adequate testing program and that its test and targets program needed to be managed in a way that fully supported high-priority near-term programs.

MDA extensively revised its test plan in fiscal year 2009 to address many of these concerns. For example, the new Integrated Master Test Plan bases test scenarios on modeling and simulation needs and extends the test baseline through 2015, which allows for better estimation of target needs, range requirements, and test assets. As part of the revised test plan, MDA scheduled dedicated periods of developmental and operational testing, during which the system configuration will remain fixed to allow the warfighter to carry out training, tactics, techniques, and procedures for developmental and operational evaluation. Additionally, the new test plan will provide sufficient time after test events to conduct a full post-test analysis. These improvements are important because BMDS performance cannot be fully assessed until models and simulations are accredited and validated and the test program cannot be executed without meeting its targets needs.

MDA Began New Target Development Effort

In July 2009, MDA also initiated a new target acquisition strategy to address recurring target performance issues and increases in target costs. According to the Director of MDA, this new target approach is based on streamlining a set of classes of targets to increase quality control of an inventory of identical targets that represent general threat characteristics to account for intelligence uncertainties. He further stated that a goal of the new target acquisition strategy is to minimize the number of targets needed to emulate specific threats and establish backup targets, which will be available in 2012.

Targets have been a recurring cause of flight test delays, cancellations, and failures since 2006. In the past, we reported that the THAAD program was unable to achieve its first intercept attempt in 2006 because the target malfunctioned. The program also experienced target anomalies in 2007 that precluded the completion of two radar characterization tests. During the same year, the GMD program experienced long-term effects on its flight test schedule when it was unable to achieve all primary test objectives because of a target failure. We also reported in March 2009 that the Aegis BMD program was unable to conduct an intercept because the target was not available. In addition, in its January 2009 report to the defense committees, MDA acknowledged target availability and reliability problems and reported its plan for a new target acquisition strategy to address these issues and improve costs, quality, and reliability.

In revising its target acquisition strategy, MDA solicited input from industry in an effort to better understand possible new target solutions that might be available to improve cost, quality, and performance. To leverage industry capability and promote a more competitive contract environment, MDA decided to use multiple contractors with multiple contracts instead of a single prime contract, increasing its flexibility to respond to changing program requirements. The agency plans to award a new contract for each class of target needed to execute the BMDS test plan. MDA will begin making decisions on contract awards and new target designs over the next year. According to program officials, MDA originally planned to issue five requests for proposals for new contracts in fiscal year 2010 and one additional request in fiscal year 2011. However, to reflect changes in the test plan and subsequent changes to the acquisition strategy, the program now plans to issue two requests for proposals in fiscal year 2010 and one in fiscal year 2011. The Targets and Countermeasures program anticipates that the first targets will be delivered under the new strategy in fiscal year 2012, and the first intercontinental ballistic missile target is expected to be delivered in fiscal year 2013.

MDA also made progress in several ongoing target development efforts that could enhance the ability to test the BMDS. During fiscal year 2009, the Targets and Countermeasures program made progress in developing four new targets—the LV-2 target, Aegis Readiness Assessment Vehicle-C target, a new medium range target, and the Extended-Long Range Air Launched Target. Each target adds a new capability to MDA’s target portfolio. For example, the LV-2 target provides the potential for significantly expanding the intermediate range payload and range performance over current inventory capabilities. The Aegis Readiness Assessment Vehicle-C target provides a new, low-cost capability as it is designed to contribute additional separating and maneuvering capabilities in short- and medium-range targets. MDA’s new Medium Range Target provides improved kill assessment capability at this range. In addition, the new Extended-Long Range Air Launched Target is a medium-range target that provides a greater range capability than previous air-launched targets and adds the ability to deploy associated objects—a capability not currently available in other similar target types. MDA expects each of these targets to be ready for use in flight tests in fiscal year 2010.

MDA Demonstrated Some Improved Performance through Testing

In fiscal year 2009, MDA conducted several ground tests and flight tests demonstrating improved performance in several areas of the BMDS including element-level functionality, theater and regional performance, and interoperability. Table 2 identifies key test events achieved in fiscal year 2009 for each element.

In June 2009, the ABL program successfully completed its first two tracking tests against boosting missile targets. These tests marked the first time ABL demonstrated a complete low-power engagement sequence against a boosting target. In addition, the ABL was able to demonstrate its ability to fire its high energy laser in an airborne environment during a flight test in August 2009. During this test, the laser was fired into a calorimeter on board the aircraft to capture the laser’s energy and measure performance characteristics of the laser’s beam.

The Aegis BMD program also demonstrated increased levels of element performance through Navy fleet exercises and developmental tests. For example, Aegis BMD demonstrated, for the first time, its capability to destroy a ballistic missile in the terminal phase of flight using Standard Missile-2 Block IV missiles while simultaneously conducting a mission using the Standard Missile-2 Block IIIA missile against a cruise missile target. The program also conducted successful developmental component tests for the next generation of the Aegis BMD interceptor—the SM-3 Block IB. Developmental testing will continue into 2010. In addition, the program successfully demonstrated that the latest software release of the Aegis BMD system had the capability to support the program’s next generation interceptor during simulated SM-3 Block IB engagements.

The C2BMC program also satisfied multiple test objectives and increased its capability in fiscal year 2009. The program participated in many system- level tests during the year that enabled it to demonstrate multiple capabilities, including improved situational awareness and sensor management. During testing, C2BMC used multi-sensor correlation and provided integrated situational awareness for weapons release decisions.

GMD, for the first time, used information from multiple sensors to develop and successfully conduct an intercept of a live target during a flight test. In December 2008, target information from four different sensors and satellite data were input into the GMD fire control system to develop an intercept plan. The involvement of multiple sensors provides better information to develop an engagement. In addition, GMD made progress in addressing BMDS Operational Test Agency concerns regarding the formatting, tracking, and accounting of messages from GMD sensors. For example, MDA added test instrumentation to collect data for regional/theater tests communications. However, the agency still faces ongoing challenges assessing timeliness with the exchange of messages at the strategic level. According to BMDS Operational Test Agency officials, they continue to work with MDA to resolve this issue. Key to the integration and functionality of the BMDS is communications and message traffic. The timely reception of messages from sensors to weapon systems is key to support decisions and achieve effective intercepts. In March 2009, we reported that these data management problems prevented the analysis of the timeliness of message data, according to BMDS Operational Test Agency officials.

The STSS program successfully completed the ground testing and integration of components to support the launch of its two demonstration satellites in September 2009. These satellites will use onboard infrared sensors to detect, track, and discriminate ballistic missiles throughout their trajectories.

THAAD also demonstrated improved element-level functionality when it successfully launched a salvo of two THAAD interceptors to intercept a separating target inside the earth’s atmosphere. The primary interceptor hit the target and the second interceptor hit the largest remaining piece of target debris seconds later.

BMDS Demonstrated Improved Theater and Regional Performance and an Increased Level of Interoperability

Regional and theater BMDS assets—Aegis BMD and THAAD—succeeded in demonstrating improved interoperability in fiscal year 2009. For example, during a THAAD intercept test, Aegis BMD tracked a target and provided the information to THAAD’s fire control. As a result, the missile was successfully engaged by THAAD. Additionally, during this test, the forward-based radar supporting THAAD was also able to discriminate the threat reentry vehicle from other objects and provide the information to support the engagement. According to program officials, the THAAD element reported that C2BMC provided accurate and timely status information for the BMDS as well as situational awareness of the test to the warfighter.

MDA also demonstrated interoperability for BMDS elements during several ground tests in fiscal year 2009. For example, during one ground test—GTD-03—MDA successfully demonstrated simultaneous theater and regional capabilities using operational BMDS hardware and actual communications between them. In addition, MDA demonstrated simultaneous BMDS capabilities to conduct training while the BMDS network remained operational during this test. This capability allows MDA to conduct development activities while maintaining readiness to engage in missile defense operations. This ground test also allowed several BMDS elements to demonstrate that they could successfully exchange data with other elements. Additionally, in December 2008 numerous elements worked together to support system-level post-flight reconstruction needed to validate BMDS models and simulations. This system-level post-flight reconstruction for flight test FTX-03 was the first ever and was highly successful because different MDA groups achieved the same results, according to MDA officials.

MDA Made Progress Developing and Integrating Models and Simulations

MDA took significant steps forward in fiscal year 2009 in developing the modeling and simulation tools necessary to understand BMDS performance against strategic and theater/regional threats. Because the potential combinations of BMDS configurations, intercept scenarios, and missile threats are too numerous for ground and flight testing, assessing overall BMDS performance depends upon the use of models and simulations to understand the capabilities and limitations of the system. Such an end-to-end system-level simulation brings together the capabilities of various element models in order to analyze how the BMDS integrated and fielded radars, communication networks, and interceptors perform during scenarios. However, to work effectively these models and simulations need to be anchored to data from ground and flight tests and validated by independent evaluators—the BMDS Operational Test Agency—in order to have confidence in their results. Moreover, the system-level simulation itself is expected to change over time as additional models become available to represent the evolving BMDS configuration.

In March 2009 we reported that MDA experienced several problems in its overall modeling and simulation program, which negatively affected the 2007 performance assessment and led to the cancellation of the 2008 performance assessment. Performance Assessment 2007 was unsuccessful primarily because of inadequate flight and ground test data for verification and validation to support accreditation and a lack of common threat and environment input data among element models. MDA officials canceled their 2008 performance assessment efforts in April 2008 because of developmental risks associated with modeling and simulations, focusing instead on testing and models for Performance Assessment 2009.

In fiscal year 2009, MDA made some progress integrating the individual element models and simulations for Performance Assessment 2009. A leading accomplishment was the development of a system-level simulation for regional and theater scenarios in addition to existing strategic scenarios for a more complete analysis of BMDS performance. Performance Assessment 2007 only included homeland defense scenarios against strategic threats. One of MDA’s goals for the performance assessment is the integration of models that communicate like the networked BMDS. As of October 2009, Performance Assessment 2009 achieved interactive communications among the element models and simulations. In addition, MDA achieved consistency in representing the threat missile and post-intercept data among all models and scenarios, which was also a weakness of Performance Assessment 2007. Finally, the BMDS Operational Test Agency observed that conducting Performance Assessment 2009 is helping to build confidence in BMDS-level simulation capability for the subsequent Performance Assessment 2010.

MDA Delivered Many Assets as Planned in Fiscal Year 2009

In fiscal year 2009, MDA met many of its delivery goals. Four MDA elements—Aegis BMD, GMD, Sensors, and C2BMC—were scheduled to deliver a total of 41 assets and capabilities in fiscal year 2009. MDA delivered 34 of these assets or 83 percent. Table 3 outlines BMDS asset deliveries in fiscal year 2009.

Aegis BMD planned to install the Aegis Weapons System 3.6.1 software on 20 ships and deliver 10 SM-3 missiles in fiscal year 2009. The program met its goal to deliver the 10 missiles and began to deliver additional rounds, initially designated for 2010, ahead of schedule. However, the program fell behind on its goal of installing the 3.6.1 software on 20 ships, delivering 18 by the end of the fiscal year 2009. Aegis BMD officials pointed out that all ship sets were available but because of real-world national security situations, these ships were not available for installations in fiscal year 2009. Nonetheless, one of the remaining ships was completed in December 2009 and another will be completed by March 2010. In fiscal year 2009, Aegis BMD also delivered an additional ship set with the next generation Aegis BMD Weapon System, 4.0.1, for a total of 19 ship deliveries.

The GMD program also partially met its delivery goals in fiscal year 2009. The program delivered an additional silo at Vandenberg Air Force Base as planned, but lagged in its GBI deliveries. For example, in fiscal year 2009, GMD emplaced three interceptors that were initially planned for fiscal year 2008 and only one of the three interceptors planned for fiscal year 2009.

The Sensors program met most of its delivery goals, successfully fielding a new near-term discrimination algorithm, activating an additional AN/TPY-2 radar site, and delivering an additional AN/TPY-2 radar. However, it fell short of meeting all of its delivery goals for the fiscal year. Although the program completed the construction for the Thule radar site ahead of schedule in fiscal year 2008, it was unable to deliver Thule radar communications and upgrades as planned in fiscal year 2009. These activities have been delayed until fiscal year 2010.

Finally, C2BMC delivered four additional C2BMC Web browsers, five work stations, and an additional combatant command suite. Additionally, the program office rolled out the Global Engagement Manager suite and added four work stations that support it. However, it was unable to meet its schedule baseline goal of an additional fielding and site activation to declare its next spiral operational. This was due to major program restructures needed to accelerate C2BMC capabilities for other BMDS elements as well as programmatic changes to fulfill warfighter requests and meet new administration direction.

Significant Challenges Remain in Developing the BMDS

While there was progress in addressing concerns about test planning and target development as well as in delivering assets, all BMDS elements experienced delays in conducting tests, were unable to accomplish all planned objectives, and experienced performance challenges. Poor target performance continued to be a problem causing several test delays and leaving several test objectives unfulfilled. The test problems also precluded the agency from gathering key knowledge through tests specified by the MDA Director that were originally planned to be completed in fiscal year 2008. MDA’s efforts to develop advanced algorithms and its efforts to demonstrate homeland defense were also affected by target issues. These shortfalls in testing continued to delay validation of the models and simulations used to assess the overall performance of the BMDS. Consequently, comprehensive assessments of the capabilities and limitations of the BMDS are still not possible. MDA also redefined its schedule baseline, eliminating goals for delivering integrated capabilities so we were not able to assess MDA’s progress in this key area.

MDA Experienced Testing Delays and Shortfalls Across the Board in Fiscal Year 2009

During fiscal year 2009, although several tests showed progress in individual elements and some system-level capabilities, all BMDS elements experienced test delays and shortfalls in part because of problems with the availability and performance of target missiles. None of the elements conducted all planned tests as scheduled and none achieved all planned objectives. Table 4 outlines BMDS test and target issues in fiscal year 2009.

Two BMDS elements—ABL and C2BMC—experienced delays in achieving fiscal year 2009 test events. For example, ABL experienced delays in development and ground testing that resulted in the delay of its first full flight test demonstration until fiscal year 2010. Additionally, C2BMC was unable to conduct testing needed to further develop its next spiral capability because of BMDS-level delays in developing the models and simulations needed to conduct this testing. Major program restructures needed to accelerate C2BMC capabilities for other BMDS elements and programmatic changes to fulfill warfighter requests and meet new administration direction also contributed to C2BMC’s inability to conduct planned fiscal year 2009 testing.

As noted in table 4, targets affected the BMDS test program for four elements in fiscal year 2009. The Aegis BMD, GMD, Sensors, and THAAD test program were affected by either target availability or target reliability and performance issues. In fiscal year 2009, targets contributed to a test cancellation and test delays and prevented elements from completing tests or achieving all test objectives. One test for Aegis BMD—FTM-15—was originally projected to use the new Flexible Target Family’s LV-2 target in fiscal year 2008, but because of qualification difficulties, the target was unavailable and the test was not conducted. This test was planned as the first Aegis BMD SM-3 engagement against an intermediate-range target. It was also expected to verify interoperability of Aegis BMD, a Sensors radar, and C2BMC. As of December 2009, MDA had canceled the test and planned to combine several of the FTM-15 objectives with those in a future flight test in 2013—FTM-23. However, as of February 2010, the Director of MDA stated that the test is being rescheduled for 2011. Test documentation was not provided for our review so it remains unclear whether the test will include the original test objectives, target, and BMDS hardware and software configurations.

The GMD and Sensors programs were also unable to complete all planned objectives because of a target failure during an intercept test. During a December 2008 flight test—FTG-05—the target failed to release planned countermeasures. A similar target failure was experienced in a prior 2008 test—FTX-03—and MDA’s risk assessments leading up to the FTG-05 test could not determine the root cause of the failure. These risk assessments determined that a similar failure would be “likely” and the consequences “severe” if MDA proceeded with the test in December 2008, even after taking mitigation steps. According to the Defense Contract Management Agency, the cost to execute FTG-05 exceeded $210 million. This was the last planned flight test using this type of target. As a result of the target failure, GMD was unable to assess the Capability Enhancement-I kill vehicle against countermeasures. According to the July 2009 Integrated Master Test Plan, this test is now planned to be conducted in the third quarter of fiscal year 2011—nearly 4 years after this configuration completed fielding.

The GMD program had to delay its second planned fiscal year 2009 intercept test—FTG-06—to fiscal year 2010 because pretest analysis raised concerns that the target may not perform as required. This test was important because it was planned as the first test of GMD’s enhanced version of the kill vehicle called the Capability Enhancement II exoatmospheric kill vehicle. This test was also designed to demonstrate a long-flight time for the GBI and GMD’s capability against countermeasures. In early 2009, MDA altered the target to present a more representative threat. Since MDA did not have modeling data to represent the new characteristics of the target, MDA officials were concerned about the target’s expected performance and decided to delay the test. In January 2010, MDA conducted FTG-06. However, all test objectives were not met as the GBI failed to intercept the target as planned. According to an MDA official, a Failure Review Board was convened to investigate the test results, but its investigation is expected to take months to complete.

As we reported in March 2009, THAAD program officials had to reschedule the planned fiscal year 2008 BMDS-level event, FTT-10, into fiscal year 2009 because of a target malfunction. THAAD successfully completed this test event in fiscal year 2009. In addition, a Short Range Air Launch Target planned for use in a third quarter fiscal year 2009 THAAD flight test FTT-11, had a component failure and subsequently needed to be requalified. This failure caused the THAAD program to modify its plannedflight test objectives and move the test into fiscal year 2010, also resulting in delays to a subsequent test—FTT-12. FTT-11 was conducted in December 2009 but could not be completed due to failure of the target missile. The air-launched target was successfully deployed from a transport aircraft, but the target’s rocket motor did not ignite. The THAAD interceptor was not launched and test objectives were not achieved. According to the Director of MDA, the Failure Review Board was concluding its investigation of the root cause of this failure. The board’s report was not available during our audit.

Target reliability and failures in fiscal year 2009 also prevented several elements from achieving all planned objectives. In March 2009, Aegis BMD experienced target difficulties when two refurbished lower-cost Army targets for a short-range mission fell short of their expected trajectory. One target was outside the intercept control area and Aegis BMD was not able to fire the interceptor because of safety limitations. In the second test, the target, while short of its expected trajectory, fell in the intercept control area and was successfully intercepted.

It Will Be Several Years Before New Target Acquisition Approach Addresses Problems

It will be several years before MDA’s new approach to target development and acquisitions will be fully implemented because most targets needed through fiscal year 2011 are already under contract and will not be affected by the new strategy. The activities under existing contracts will not be complete until 2013. Moreover, MDA’s implementation of a new acquisition management strategy does not necessarily mean that any particular target currently being used, such as the LV-2, will be phased out of the test program. MDA could decide to continue to use an existing target under the new strategy, and as a result, some existing target missiles could continue to be procured under new contracts.

MDA has not presented a complete business case for proceeding with a new target acquisition management strategy. A complete business case includes establishing top-level cost, schedule, and performance baselines available internally and externally for oversight. It is the essential first step in any acquisition program because it sets the stage for acquisition and execution. Program officials told us that they would have cost, schedule, and performance baselines finalized and documented as part of the decision to proceed with new contract awards. These baselines, however, will be very detailed and spread across multiple documents and therefore are unsuitable for internal and external oversight. The officials further stated that they do not intend to establish top-level cost, schedule, and performance baseline measures similar to approved program baselines that are established for DOD’s major defense acquisitions to provide accountability. In September 2008, we reported that MDA had difficulty in developing and supplying new targets in part because a sound business case was not developed before significant decisions were made. In that report we recommended that MDA develop cost, schedule and performance baselines as part of an effort to establish a sound business case for each new class of target under development.

As part of the new target development efforts, MDA also developed a new cost model. However, because the cost model and test baseline are continually updated, the Targets and Countermeasure program continues to lack solid cost baselines against which progress can be measured. According to the Director of MDA, the agency will continue to update its cost model as the Integrated Master Test Plan changes, noting that where the technical content of the test plan remains constant, cost, schedule, and performance baselines can be measured from year to year. However, as we reported in March 2009, the Integrated Master Test Plan changes frequently. In fact, the latest approved version is dated July 2009, and according to MDA’s Director, a revised version of the Integrated Master Test Plan is expected in March 2010, which limits the baseline’s stability to approximately 8 months and limits our ability to measure MDA’s progress against a cost baseline.

MDA’s ability to develop an accurate cost baseline is also affected by the lack of historical data available for targets or for other similar missiles. Program officials said that they are now collecting more useful cost data for new contracts by requiring more detailed cost reporting from their contractors. This approach will allow program officials to gather more complete and accurate data over time to make the new cost model a more powerful cost estimating tool.

Testing and Targets Problems Affected Planned Progress of Overall BMDS Development

The inability of MDA to successfully conduct its test plan precluded the agency from collecting critical information needed for key decisions and significantly affected development of advanced algorithms and homeland defense capabilities.

Key Director’s Knowledge Points Delayed Again

In fiscal year 2009, MDA was unable to accomplish any of the Director’s knowledge points that were to be achieved through tests. Several of these tests were originally planned for fiscal year 2008, but were delayed into 2009 and then again delayed into fiscal years 2010 and 2011. Table 5 shows the original test date and MDA’s current estimate for obtaining the necessary knowledge.

Target issues continued to affect MDA’s ability to fully develop algorithms needed for discrimination capability. In March 2009, we reported that multiple elements experienced test failures which caused delays in collecting data needed to develop discrimination capability. For example, in 2007, two THAAD radar characterization tests were unsuccessful because of target anomalies. These tests were designed with characteristics needed for radar observation in support of advanced discrimination algorithm development. However, target problems prevented an opportunity for the radar to exercise all of the planned algorithms, causing a loss of expected data. Similarly, in a 2008 sensor characterization test, the target failed to release its countermeasures, which prevented the sensors from collecting expected data. Consequently, MDA was unable to fully develop discrimination algorithms as planned.

In fiscal year 2009, MDA continued to be unable to develop its advanced algorithms as planned as key tests that were designed to reduce the maturation risk were affected by targets. For example, the Sensors and GMD elements were unable to collect data to develop their advanced algorithms when the target failed to release countermeasures and present the expected scene complexity during FTG-05. The subsequent delay to the next intercept test—FTG-06—until January 2010 has also reduced the data MDA had expected in fiscal year 2009 for the development of discrimination capability. Additionally, target unavailability caused MDA to delay a THAAD test—FTT-11—from fiscal year 2009. This test was designed to provide data for the development of advanced algorithms for the THAAD radars. The test was conducted in fiscal year 2010 but could not be completed because the target malfunctioned during deployment. According to the Director of MDA, the Failure Review Board was concluding its investigation of the root cause of this failure. The board’s report was not available during our audit.

Likewise, GMD continues to experience delays demonstrating increased interceptor performance for homeland defense as the two aforementioned tests—FTG-05 and FTG-06—were not conducted as planned. As we testified in February 2009, MDA had expected to conduct seven GMD interceptor flight tests from the start of fiscal year 2007 through the first quarter of fiscal year 2009. However, MDA was able to conduct only two, which, according to the Director of Operational Test and Evaluation, has limited the complete sets of information necessary for validating ground- based interceptor models. MDA also delayed the other planned flight test, FTG-06, beyond fiscal year 2009 because of target issues and an anomaly with a component of the Sea-Based X-band radar. As of June 2009, MDA estimated this test to cost over $236 million while the Defense Contract Management Agency estimated the cost to exceed $310 million. These costs are likely understated because they do not include all of the cost increases of delaying the test first to September 2009, nor do they include any cost increases of further delaying the test until the second quarter of fiscal year 2010.

Although the Aegis BMD missile—SM-3 Block IA—capability against an intermediate range ballistic missile is not a requirement, MDA has planned for years and invested millions of dollars in a plan to test the Aegis BMD system and SM-3 Block IA interceptor against this type of threat. At the start of fiscal year 2009, Aegis BMD officials intended to conduct this test in the third quarter of fiscal year 2009. However, as of December 2009, MDA had canceled the test and planned to combine several objectives with those in a future flight test in 2013. As of February 2010, the Director of MDA stated that the test is being rescheduled for 2011. Test documentation was not provided for our review, so it remains unclear whether the test will include the original test objectives, target, and BMDS hardware and software configurations.

New Test Plan Not Yet Complete, Not Fully Tied to Resources, and Not Synchronized with Other Management Baselines

MDA’s new July 2009 test plan was intended to provide stability; however, program officials already anticipate major revisions and alterations. According to MDA officials, budget decisions and the presidential decision to implement a European phased, adaptive approach, drove changes to the test and targets program. For example, the new strategy for European missile defense will primarily utilize Aegis BMD interceptors as opposed to GMD interceptors. Tests in support of developing this capability have not yet been added to the test plan. The Director of MDA stated that his agency is coordinating with the Office of the Director, Operational Test and Evaluation and with the BMDS Operational Test Agency to address these changes. According to the Director of MDA, flight and ground testing to support phases one through four of the Phased Adaptive Approach will be baselined in the March 2010 Integrated Master Test Plan, but the test plan was not available for our review during our audit.

One way MDA’s new testing approach was intended to provide stability is that it was structured to slow the spiral development fielding process, allowing the warfighter to gain confidence in the BMDS before fielding decisions are made. However, BMDS Operational Test Agency officials told us that changes to hardware and software configurations need to follow the process jointly agreed to with MDA, noting that changes to the operational baseline should not occur until the appropriate developmental tests and operational tests have been completed. After the adoption of the new test plan through October 2009, MDA continued to incorporate software changes as updates to the operational baseline. According to Operational Test Agency officials, most of the proposed and approved software changes had not been through system-level testing and immediately made future test configurations in the Integrated Master Test Plan invalid. Changes made without full system-level testing, could result in possible adverse effects to the BMDS and the warfighter’s ability to use the system effectively. The BMDS Operational test Agency continues to work with MDA on these issues. BMDS Operational Test Officials told us that they have seen improvements since October 2009, noting that there has been an increase in early coordination and presentation of data to support interim releases of software and hardware. According to these officials, these improvements coupled with the new warfighter and MDA- accepted approach for testing—allowing developmental testing to occur before operational testing and before new capabilities are delivered to the Warfighter—will likely resolve issues encountered with frequent changes to software and hardware.

We testified in February 2009 that the success of MDA’s new approach to testing hinges on providing sufficient resources, among other factors. However, these resource challenges continue to affect the test plan because MDA’s new test plan was not fully resourced when it was approved in July 2009. In addition, BMDS Operational Test Agency officials also raised concerns that the Integrated Master Test Plan is not currently resourced to support the necessary personnel to analyze the tests or the performance assessment.

Until the new development efforts are fully reflected in the test plan, MDA will also not be able to fully integrate that plan with other key aspects of testing and development, such as the acquisition of targets. The test plan is one of six management baselines MDA uses to track program progress. However, MDA determined that these baselines consist of a disparate set of non-integrated business processes. More importantly, MDA acknowledged that there is inconsistent management, configuration control, integration, and synchronization of existing manual processes. MDA is developing new business tools to automate the integration of these baselines and projects. While it will take several years for the agency to integrate these baselines using those tools and synchronize them with other key testing and development efforts, the initial capability to automatically integrate cost, schedule, and performance baselines will be available in early fiscal year 2011.

Overall BMDS Performance Cannot Be Assessed Because MDA Models Are Not Sufficiently Mature

MDA models and simulations have not matured sufficiently to assess overall BMDS performance and may not fully mature until 2016, instead of 2011as we reported last year. According to the BMDS Operational Test Agency, it could not project which models and simulations could be accredited for Performance Assessment 2009. It expects to make its determination in July 2010 at the earliest. Further, functionality shortfalls diminished the usable scope and integration issues have delayed the execution of Performance Assessment 2009 by at least 6 months. As a result, the BMDS Operational Test Agency did not use the Performance Assessment 2009 data in its 2009 annual operational assessment as it had once intended. According to these officials, because of the known limitations and the changes to the BMDS operational configuration that will occur in 2010, the BMDS Operational Test Agency also will not be able to use the results as part of its 2010 annual operational assessment.

MDA officials acknowledged that their primary challenge for the next several years will be obtaining enough flight test data to anchor and accredit the models. Moreover, the BMDS Operational Test Agency is still concerned about the effect on the validation of models due to artificialities in flight tests, particularly for GMD. The BMDS Operational Test Agency believes that the validation of models will improve as artificialities in flight tests are reduced. Another unresolved modeling and simulation weakness in the testing program has been addressing different weather conditions. MDA, in concert with the BMDS Operational Test Agency, is addressing modeling deficiencies with respect to weather conditions, but specific plans to resolve this weakness were not available during our audit. Finally, the BMDS Operational Test Agency anticipates that deficiencies in modeling the BMDS communications system at the regional and theater levels that exist in Performance Assessment 2009 will improve in the subsequent Performance Assessment 2010.

Progress in Delivering Integrated Capabilities Could Not Be Assessed

In 2008, we assessed MDA’s capability delivery progress against its integrated capability schedule goals and found that many slipped to 2009. We are no longer able to assess MDA’s progress in delivering integrated capabilities because, in fiscal year 2009, the agency eliminated integrated capability delivery goals from its schedule baseline. In its most recent BMDS Accountability Report, MDA redefined its schedule baseline to consist solely of hardware and software deliveries spread across fiscal years. MDA assigned schedule metrics to asset deliveries on an element level only and removed key schedule measures—engagement sequence groups—from its August 2009 BMDS Accountability Report that tracked integrated block capability deliveries and provided a means for assessing the readiness of BMDS capabilities, integration, and functionality. Thus, MDA provided no information about its progress and plans to deliver integrated BMDS capabilities.

MDA previously identified its capability delivery schedule goals and baselines within the block structure, in terms of assets and engagement sequence groups made available for fielding in a particular timeframe. Under this capabilities-based five-block acquisition management strategy, some blocks contained schedule baselines for deliveries of significant increments of capabilities against particular threats, culminating in the full capability declaration at a projected date. According to MDA, engagement sequence groups created manageable combinations of system configurations and provided a structure to assess BMDS performance. Because MDA presented early, partial and full capability delivery dates for individual engagement sequence groups, engagement sequence groups served as baseline to measure the schedule of integrated capability deliveries.

MDA officials told us that the agency eliminated engagement sequence groups as measures of integrated capability deliveries to address warfighter concerns. According to MDA officials, the warfighter did not assess engagement sequence groups since they were organized in a way that did not align with warfighter operations, tactics, and procedures. During our audit, MDA had not replaced these previously reported integrated capability delivery baselines with new metrics. However, according to the Director of MDA, the agency is working to develop new baselines and schedules from which progress can be measured. In addition, agency officials told us that MDA is transitioning to an incremental BMDS capability delivery concept. However, MDA did not provide a definition of incremental BMDS capability deliveries or define them as schedule goals in the August 2009 BMDS Accountability Report. MDA also did not identify anticipated delivery dates for its performance metrics; however, the Director of MDA stated that developmental baselines are anticipated to be developed, reviewed and approved by the third quarter of fiscal year 2010. Furthermore, major MDA documents designed to communicate MDA’s BMDS schedule are not synchronized. Although MDA officials told us that they have recently synchronized the Integrated Master Schedule with the Integrated Master Test Plan, the two documents’ schedule still does not correspond to the BMDS Master Plan. The Integrated Master Test Plan will be revised in February 2010, rendering all three documents again unsynchronized with MDA’s acquisition strategy and programmatic decisions.

While it has eliminated its externally reported integrated capability declaration goals, MDA continues to internally track capability declarations for at least two of its assets—the Sea-based X-band radar and the Shariki AN/TPY-2 radar—whose capability declarations slipped again in fiscal year 2009. The Sea-based X-band radar partial capability declaration appears to have slipped from fiscal year 2009 to fiscal year 2010, while full capability will be declared with less knowledge than initially planned. According to MDA officials, the agency was planning for a partial capability declaration in June 2009, following successful execution of four test events—GTI-03, FTX-03, FTG-05, and GTD-03—and analysis. However, these events slipped over the course of the year, and according to MDA, the partial capability declaration was delayed to fiscal year 2010. According to the Director of MDA, the capability declaration is currently planned to occur after analysis can include both FTG-06 and a test—CD-03—planned for September 2010. It remains unclear what effect the problems encountered in FTG-06 will have on the declaration decision.

The Shariki radar was designated by MDA to reach a full capability declaration by December 2008, but that was subsequently delayed to July 2009. The radar was to undergo the military mission capability assessment, in which the warfighter verifies the radar’s readiness for full operational use by the services in the context of the present BMDS architecture. To date, the full capability declaration has not been made. Consequently, the date for the full mission capability has not been determined. Furthermore, as with the Sea-based X-band radar, the decision has not been made as to whether the Shariki radar capability declaration process will continue under the original plan or migrate to the new approach.

BMDS Assets Still Being Delivered Despite Developmental Problems and Test Delays

Despite testing delays, developmental problems, and the continued inability to complete the Director’s test-related knowledge points, MDA proceeded with manufacturing, production, and fielding of BMDS assets prior to operational testing and evaluation.

The Aegis BMD program intends to execute a contract modification in the second quarter of 2010 to acquire 18 operationally configured SM-3 Block IB missiles, used for testing and fielding. These 18 SM-3 Block IB missiles were originally justified in the fiscal year 2010 budget request as needed for flight testing and for delivery to the fleet as operational assets. According to MDA’s September 2009 SM-3 Block IB utilization plan, 2 missiles are to be used for flight tests, 10 are to be used for fleet deployment, and 6 are to be used for either fleet proficiency training or fleet deployment. However, MDA is proceeding with the contract modification even though flight testing of a fully integrated prototype for this missile type in an operational environment will not have occurred. The first flight test—FTM-16—that could demonstrate some performance of the missile is currently scheduled for the third quarter of fiscal year 2011. In addition, the program is still maturing several critical technologies, such as the throttleable divert and attitude control system, and developmental testing of these technologies will not be complete until after the manufacturing decision for these 18 missiles. The manufacturing decision is also scheduled to occur almost a year before the manufacturing readiness review—currently scheduled for the second quarter of fiscal year 2011. Consequently, approval for production of this missile is scheduled before the results of developmental testing to demonstrate that the technologies and design are fully mature, before the first flight test demonstrates the system functions as intended, and before the readiness to begin manufacturing has been assessed—all of which increases the risk of costly design changes while production is underway. The Director of MDA and the Assistant Secretary of the Navy for Research, Development and Acquisition approved a developmental baseline in January 2010 that set production criteria and projected an initial production decision for 74 SM-3 Block IB missiles in the third quarter of fiscal year 2011.

GMD continues to manufacture and field the Capability Enhancement II exoatmospheric kill vehicle prior to having it verified through operationally realistic flight testing. In March 2009, we reported that MDA had planned to conduct an intercept test to assess Capability Enhancement II exoatmospheric kill vehicle in the first quarter of fiscal year 2008—months before emplacing any interceptors with this configuration. However, developmental problems with the new configuration’s inertial measurement unit and problems with the target delayed the first flight test with the Capability Enhancement II configuration—FTG-06—until the fourth quarter of fiscal year 2009. This test was again delayed because of modeling uncertainties with the target and failures experienced with the Sea-Based X-Band radar during testing. GMD officials stated that they do not plan to adjust deliveries of the Capability Enhancement II exoatmospheric kill vehicle because of the test delay. However, MDA officials told us that they will not add Capability Enhancement II to the operational baseline until after FTG-06 has been conducted. As previously noted, FTG-06 was conducted in January 2010 but was unsuccessful. According to the July 2009 revised Integrated Master Test Plan, the next planned intercept test with a similar configuration as FTG-06—a three-stage booster and a Capability Enhancement II exoatmospheric kill vehicle—is not scheduled to take place until at least fourth quarter fiscal year 2012. If MDA delivers Capability Enhancement II exoatmospheric kill vehicle units as currently scheduled, it will have delivered all of the Capability Enhancement II exoatmospheric kill vehicles that are currently under contract before the test is conducted.

MDA’s concurrent approach to developing and fielding assets has led to concerns about the performance of some fielded assets. In March 2009, we reported that MDA had initiated a refurbishment program in 2007 to replace questionable parts and that some improvements had already been introduced into the manufacturing flow. However, according to program officials, they discovered additional problems during early refurbishments causing the program to expand its effort. Additionally, as MDA continues to manufacture ground based interceptors, it is discovering additional process and design issues, and the corrective actions are being incorporated into the refurbishment program. The program has three categories for refurbishments—minimal, moderate, and extensive—with the cost of each vehicle going through refurbishment varying from vehicle to vehicle. MDA originally estimated that the cost for extensive refurbishment of an individual interceptor could reach as high as $24 million.

MDA Lacks Controls and Mechanisms Needed to Establish Transparency, Accountability, and Oversight

MDA continues to face challenges with transparency, accountability, and oversight controls and mechanisms. In establishing MDA in 2002, the Secretary of Defense directed the agency to develop the BMDS as a single program using a capabilities-based, spiral upgrade approach to quickly deliver a set of integrated defensive capabilities. To accomplish this mission, MDA was granted exceptional flexibility in setting requirements and managing the acquisition. This flexibility allowed MDA to begin delivering an initial defensive capability in 2004, but at the expense of transparency and accountability.

Since our first MDA report in 2004, we have repeatedly found that MDA’s approach for building its cost, schedule, and performance goals hindered transparency and limited accountability of the BMDS development effort. Specifically in April 2004, we reported that MDA’s goals did not provide a reliable and complete baseline for accountability purposes and decision making because these goals varied year to year, did not include all associated costs, and were based on assumptions about performance that were not explicitly stated. These conclusions still hold true for several aspects of the BMDS acquisition strategy. For example, MDA’s goals change continuously, cost baselines have yet to be established, and some details regarding performance goals are still not explicitly stated. Since 2004, we have also made recommendations to develop baselines and report variances to those baselines to promote a higher level of transparency and accountability for the agency; to adjust its block strategy to ensure that it was knowledge-based and aligned with agency goals; and to strengthen oversight by, for example, having the Missile Defense Executive Board (MDEB) consider the extent to which MDA could adapt and adopt aspects of DOD’s standard acquisition policies to enhance oversight.

Members of Congress have also expressed concerns regarding the block strategy, acquisition management strategy, accountability, and oversight of MDA. For example, in 2007, the House Appropriations Committee directed MDA to “develop a system-wide plan to report according to the spirit of existing acquisition laws to improve accountability and transparency of its program.” More recently, in the National Defense Authorization Act for Fiscal Year 2008, Congress required MDA to establish acquisition cost, schedule, and performance baselines for each system element that has entered the equivalent of the systems development and demonstration phase of acquisition or is being produced or acquired for operational fielding. MDA is not yet fully compliant with this requirement. However, officials indicated that they are working toward fulfilling this requirement, but the expected date for full compliance was unknown at the time of our audit.

While MDA has committed to take actions to address concerns about accountability and transparency, it has made limited progress in implementation, as shown in table 6.

MDA’s termination of its capabilities-based block approach in June 2009 marked the third acquisition management strategy for the BMDS in the last 3 years and effectively reduced transparency and accountability for the agency. As previously noted, MDA has organized the development of the BMDS using two different block approaches in the past—(1) sequential 2- year blocks of BMDS-wide integrated capabilities and (2) five capabilities- based blocks of different MDA elements against particular threats. Changing the block structure is problematic because each time the block structure is changed, the connection is obscured between the old block structure’s scope and resources and the new block structure’s rearranged scope and resources. This makes it difficult for decision makers to hold MDA accountable for expected outcomes and clouds transparency of the agency’s efforts.

In March 2008, we reported that the agency’s capabilities-based block approach had begun to provide improvements to transparency and accountability, but as we recommended, transparency and accountability could have been further improved with MDA’s development and reporting of full acquisition cost estimates as well as independent verification of those costs.

New MDA Initiatives Provide Opportunity to Increase Transparency and Accountability

Although key controls and mechanisms needed to establish a sound acquisition process for MDA are still lacking, MDA has initiatives underway that could improve the transparency, accountability, and oversight of the acquisition of the BMDS. In June 2009, the MDA Director testified before the Senate Armed Services Committee that MDA is responding to the Weapon System Acquisition Reform Act of 2009 through the establishment of acquisition milestone decisions. These decisions are designed to ensure appropriate competitive acquisition strategies. He further noted that as the Acquisition Executive for the initial phases of missile defense, he is implementing milestone review and baseline reporting processes that are closely aligned with the principles of DOD’s acquisition policies, commonly referred to as the DOD 5000 series. He further noted that he recognized the need to incorporate the tenets of the DOD 5000 series to ensure that programs are affordable, are justified by the warfighter, and demonstrate acceptable risk through a milestone review process overseen by the MDEB. He also stated that MDA intends to separate the management of its technology and development programs. The Director testified that under his authority, potential programs that may provide technological or material solutions for MDA will undergo a Milestone “A” decision to determine if they should become programs. These technology-based programs will be managed by knowledge points and incubated until maturity, at which time MDA along with the service acquisition executive will be able to make a Milestone “B” decision as to whether the program should be converted to a development program. He explained that the Under Secretary of Defense for Acquisition, Technology and Logistics will make Milestone “C” production decisions regarding the programs.

We were only able to obtain limited insight into these initiatives because the agency only determined how they will be implemented at the end of our audit and was just beginning to implement them. In regards to the milestone decisions, the Director of MDA indicated that the agency is undertaking a baseline phase review process. The agency is transitioning to managing the six developmental baselines at the project element level. These baselines will be approved in developmental baseline reviews and managed through quarterly performance element reviews. MDA has identified three phases of development where baselines are approved— technology development, product development, and initial production phases—which may ensure that the appropriate level of knowledge is obtained before acquisitions move from one phase to the next. Approval of the product development and initial production baselines will be jointly reviewed by the Director of MDA and the respective service acquisition executive. In addition, while our draft was being reviewed by MDA, the Director of MDA provided us with initial information regarding the definition of these new phases and the process for establishing cost, schedule, or performance baselines. Based on our initial briefing on MDA’s new process, it may include many of the necessary elements of a sound business case—such as establishing top-level cost, schedule, and performance measures that are available internally and externally for oversight.

Although we were unable to fully evaluate MDA’s new initiatives, these initiatives do offer an opportunity for the agency to increase transparency and accountability if they are implemented in accordance with knowledge- based acquisition principles, leading to the establishment of sound business cases and realistic baselines.

Over the past 10 years, we have conducted extensive research on successful programs and have found that successful defense programs ensure that their acquisitions begin with realistic plans and baselines prior to the start of development. We have previously reported that the key cause of poor weapon system outcomes, at the program level, is the consistent lack of disciplined analysis that would provide an understanding of what it would take to field a weapon system before system development begins. We have reported that there is a clear set of prerequisites that must be met by each program’s acquisition strategy to realize successful outcomes. These prerequisites include the following: Establishing a clear, knowledge-based, executable business case for the product. An executable business case is one that provides demonstrated evidence that (1) the identified needs are real and necessary and can best be met with the chosen concept and (2) the chosen concept can be developed and produced within existing resources—including technologies, funding, time, and management capacity. Knowledge-based acquisition principles and business cases combined are necessary to establish realistic cost, schedule and performance baselines. Without documented realistic baselines there is no foundation to accurately measure program progress.

Separating technology development activities from product development activities. As noted earlier, the Director of MDA plans to separate technology development and product development for the BMDS. The process of developing technology culminates in discovery—the gathering of knowledge—and must, by its nature, allow room for unexpected results and delays. When immature technologies are brought onto the critical path of product development programs too early, they often cause long delays in an environment where large workforces must be employed; complex tools, plants, and facilities must be operated; long and expensive supplier networks must be paid; and the product itself must sometimes be redesigned once the final form of the technologies is known. Ensuring that only mature technologies are brought into product development is a key step for successful programs.

Employing early systems engineering discipline in order to develop realistic cost and schedule estimates prior to development start. Early systems engineering provides the knowledge a product developer needs to identify and resolve performance and resource gaps before product development begins, either by reducing requirements, deferring them to the future, or increasing the estimated cost for the weapon system’s development. Requirements that are too risky given the state of technology and design should not be allowed into this expensive environment. MDA’s Director noted that he has taken steps to enhance systems engineering by designating a senior executive position to establish engineering policy, ensure the disciplined practice of systems engineering fundamentals, and develop the systems engineering competencies of the missile defense workforce; creating knowledge centers; and increasing the number of recent engineering school graduates. While these initiatives hold promise for the future, they could provide further enhancements if they are used as the foundation to develop realistic cost and schedule estimates for the BMDS.

These practices could address MDA’s past problems of initiating programs and beginning system development based on limited systems engineering knowledge. These programs depended on critical technologies that were immature and not ready for product development or production. The Director of MDA acknowledged the importance of changing MDA’s acquisition approach to adopt knowledge-based acquisition processes.

Conclusions

In order to respond to a presidential directive to deliver a missile defense capability in a rapid manner, MDA has been given unprecedented funding and decision-making flexibility. This flexibility has allowed concurrent development, testing, manufacturing and fielding and enabled MDA to quickly develop and field the first increment of capability in 2005. However, while this approach has expedited the fielding of assets, it also resulted in less transparency and accountability than is normally present in a major weapon program. Since the program’s inception, MDA’s lack of baselines and its management of the BMDS with high levels of uncertainty about requirements and program cost estimates effectively set the missile defense program on a path to an undefined destination at an unknown cost. Across the agency, these practices left programs with limited knowledge and few opportunities for crucial management oversight and decision making concerning the agency’s investment and the warfighter’s continuing needs. At the program level, these practices contributed to quality problems affecting targets acquisitions, which in turn, hampered MDA’s ability to conduct tests as planned.

As MDA transitions to new leadership, a new acquisition strategy, a new test strategy, and a shift in emphasis toward early intercept capabilities, the agency has an opportunity to chart a course that enables transparency and accountability as well as flexibility, and it appears committed to doing so. Importantly, the Director of MDA has begun new initiatives in accordance with guiding principles of DOD’s acquisition policies, which already embrace knowledge-based practices and sound management controls. The Director of MDA intends to apply these new policies to each element or appropriate portions of the elements, as is currently done across DOD, in order to provide a foundation for the Congress and others to assess progress and hold senior leadership accountable for outcomes.

These initial steps are promising, but it will take time to fully implement them and once implemented they will need to be sustained and the tools consistently used in order to establish accountability. If this is done effectively, with baselines set at a program level, MDA can respond to strategic changes affecting the overall configuration of the system without losing basic knowledge about cost, schedule, and performance. Such actions do not have to result in a slower or more burdensome acquisition process. In the past, weapon programs often rushed into systems development before they were ready, in part because DOD’s acquisition process did not require early formal milestone reviews and programs would rarely be terminated once underway. Over time, in fact, these changes could help programs replace risk with knowledge, thereby increasing the chances of developing weapon systems within cost and schedule targets while meeting user needs.

Recommendations for Executive Action

As MDA implements its initiatives to improve transparency, accountability, and oversight, and begins efforts to manage and oversee MDA at the element level, we recommend that the Secretary of Defense direct MDA to take the following eight actions: Establish cost, schedule, and performance baselines for the acquisition of each new class of target when it is approved by the Director prior to proceeding with acquisition and report those baselines to Congress.

Obtain independent Cost Assessment and Program Evaluation cost estimates in support of these cost baselines.

Ensure that program acquisition unit costs for BMDS assets are reported in the BMDS Accountability Report, to provide Congress with more complete and comprehensive information by including development costs.

Update DOD’s Plan to Enhance the Accountability and Transparency of the Ballistic Missile Defense Program to reflect MDA’s current initiatives and include dates for fulfilling each commitment.

Report top-level test goals for each element, or appropriate portions thereof, to Congress in the next BMDS Accountability Report.

Develop and report to Congress in the annual BMDS Accountability Report a measure for schedule baseline goals that incorporates delivering integrated capabilities to the warfighter.

Develop and report to Congress in the annual BMDS Accountability Report the dates at which performance baselines will be achieved.

Report to Congress variances against all established baselines.

Several of these actions, such as establishing cost, schedule, and performance baselines, have been recommended in prior GAO reports or addressed in legislation. This report, however, restates these recommendations in the context of changes made to the missile defense program, for example, the deletion of the block structure and increased focus on elements.

We further recommend that the Secretary of Defense direct MDA to take the following two actions: Delay the manufacturing decision for SM-3 Block IB missiles intended for delivery to the fleet as operational assets until after (1) the critical technologies have completed developmental testing, (2) a successful first flight test demonstrates that the system functions as intended, and (3) the successful conclusion of the manufacturing readiness review.

Ensure that developmental hardware and software changes are not made to the operational baseline that disrupt the assessments needed to understand the capabilities and limitations of new BMDS developments.

Agency Comments and Our Evaluation

DOD provided written comments on a draft of this report. These comments are reprinted in appendix I. DOD also provided technical comments, which were incorporated as appropriate.

DOD fully concurred with 9 of our 10 recommendations, including our recommendation to establish cost, schedule, and performance baselines for the acquisition of each new class of target when it is approved by the MDA Director prior to proceeding with acquisition and report those baselines to Congress. In response to our recommendation, DOD commented that MDA has already established and the Director has approved cost, schedule, and performance baselines for the acquisition of each new class of target. The department noted that these baselines are contained in multiple documents and will be brought together in a Target Program Baseline prior to contract award. However, MDA should ensure that the Target Program Baseline establishes top-level cost, schedule, and performance baseline measures similar to approved program baselines that are established for DOD’s major defense acquisitions and available for internal and external oversight. It is unclear whether MDA will make its Target Program Baseline available internally for oversight and report it to Congress as we recommended.

DOD partially concurred with our recommendation that the Secretary of Defense direct MDA to delay the manufacturing decision for SM-3 Block IB missiles intended for delivery to the fleet as operational assets until after (1) the critical technologies have completed developmental testing, (2) a successful first flight test demonstrates that the system functions as intended, and (3) the successful conclusion of the manufacturing readiness review. In response to this recommendation, DOD stated that manufacturing of SM-3 Block IB missiles to support testing is under way, but the production decision for SM-3 Block IB missiles used for fleet operation is planned to occur after criteria listed in our recommendation have been met. However, during our review, we found that the 18 SM-3 Block IB missiles in question were originally justified in the fiscal year 2010 budget request as needed for “flight testing and for delivery to the fleet as operational assets.” In addition, Aegis BMD Program Office responses related to this matter indicate that these missiles will be used operationally if a security situation requires it. Furthermore, according to MDA’s September 2009 SM-3 Block IB utilization plan briefed and approved by the MDA Acquisition Strategy Board, only 2 of these missiles are specifically designated for flight tests, while 10 are to be used for fleet deployment and 6 are to be used for either fleet proficiency training or fleet deployment. Based on this information, the contract modification to acquire these 18 SM-3 Block IB missiles will take place before the critical technologies are fully matured at the conclusion of FTM-16—the first SM-3 Block IB end-to-end flight test of a fully integrated, production- representative prototype. Thus, we maintain that approval for manufacturing of these 18 SM-3 Block IB missiles—the majority of which will be deployed to the fleet—is scheduled to occur before the results of developmental testing to demonstrate that the technologies and design are fully mature, before the first flight test demonstrates the system functions as intended, and before the readiness to begin manufacturing has been assessed—all of which increase the risk of costly design changes and retrofit.

We are sending copies of this report to the Secretary of Defense and to the Director of MDA. The report also is available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Comments from the Department of Defense

Appendix II: Scope and Methodology

To examine the progress that eight Missile Defense Agency (MDA) elements have made during fiscal year 2009 toward schedule, testing, and performance baselines, we developed data collection instruments that were completed by each element’s program office. These instruments collected detailed information on prime contracts, design reviews, test schedules and results, element performance, noteworthy progress, lessons learned, and challenges facing the elements during the fiscal year. In addition, we reviewed individual element Program Execution Reviews, test plans and reports, production plans, staffer day briefings, and other requirements documents. We held interviews with officials in each element’s program office and followed up on the information we received with MDA’s Agency Operations Office; the Department of Defense’s (DOD) Office of the Director, Operational Test and Evaluation; and MDA’s Ballistic Missile Defense System (BMDS) Operational Test Agency.

To further review individual element and BMDS-level performance progress during the fiscal year, we met with officials in MDA’s Modeling and Simulation Directorate at the Missile Defense Integration and Operations Center, individual element program offices, and MDA’s BMDS Operational Test Agency to discuss modeling and simulations plans and procedures as well as other performance metrics. We also reviewed DOD and MDA policies, memos, and flight test plans related to modeling and simulations. In addition, we reviewed various elements’ verification, validation, and accreditation plans, MDA performance briefings, and validation, verification, and accreditation plans for MDA’s BMDS Performance Assessment 2009.

We assessed MDA’s testing and target development progress by reviewing MDA’s Integrated Master Test Plans, Integrated Master Schedule, target acquisition plan, and target business case analysis. In addition, we met with officials in the Targets and Countermeasures Program Office to obtain information on MDA’s acquisition management strategy including plans for cost, schedule, and testing. We also met with MDA’s testing directorate, MDA’s BMDS Operational Test Agency, and DOD’s Office of the Director of Test and Evaluation to discuss the progress, challenges, and lessons learned during fiscal year 2009 testing.

To analyze MDA’s changing acquisition approach and the agency’s progress in addressing issues related to transparency, accountability, and oversight, we interviewed officials from the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics; MDA’s Agency Operations Directorate; MDA’s Advanced Technology Directorate; and MDA’s Office of Quality, Safety, and Mission Assurance Directorate. We also reviewed various MDA statements and documents related to MDA’s block structure. We reviewed DOD acquisition system policy and various DOD directives to gain insight into other DOD systems’ accountability and oversight mechanisms. We also analyzed MDA’s acquisition directives and Missile Defense Executive Board briefings to examine MDA’s current level of oversight. In addition, we reviewed MDA budget estimate submission justifications, Integrated Master Test Plans, the Ballistic Missile Defense Master Plan, the BMDS Accountability Report, and prior reports that outlined the agency’s baselines and goals.

Our work was performed both at MDA headquarters in Arlington, Virginia and at various program offices located in Huntsville, Alabama. In Arlington we met with officials from the Aegis Ballistic Missile Defense Program Office; Airborne Laser Program Office; Command, Control, Battle Management, and Communications (C2BMC) Program Office; MDA’s Agency Operations Office; DOD’s Office of the Director, Operational Test and Evaluation; and the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics. In Huntsville, Alabama we interviewed officials from the Ground-based Midcourse Defense (GMD) Program Office, the Sensors Program Office, the Terminal High Altitude Area Defense Project Office, the Targets and Countermeasures Program Office, the Advanced Technology Directorate, and the Office of the Director for BMDS Tests. We met with officials from the Missile Defense Integration and Operations Center at Schriever Air Force Base in Colorado Springs, Colorado, to discuss the C2BMC and Space Tracking and Surveillance System elements as well as to receive further information on MDA’s models and simulations. Additionally, we interviewed Raytheon officials in Tucson, Arizona, to discuss the Kinetic Energy Interceptor, GMD, and Aegis BMD elements’ status.

In December 2007, the conference report accompanying the National Defense Authorization Act for Fiscal Year 2008 noted the importance of DOD and MDA providing information to GAO in a timely and responsive manner to facilitate the review of ballistic missile defense programs. During the course this audit, we experienced significant delays in obtaining information from MDA. During the audit, MDA did not always provide GAO staff with expeditious access to requested documents and articles of information, which delayed some audit analysis and contributed to extra staff hours. Of the documents and information we requested, we received approximately 24 percent within the 10 to15 business day protocols that were agreed upon with MDA. Pre-existing documentation took MDA on average about 28 business days to provide and many pre- existing documents took 40 business days or more to be provided to GAO. Notwithstanding these delays, we were able to obtain the information needed to satisfy our objectives in accordance with generally accepted government auditing standards.

We conducted this performance audit from April 2009 to February 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the contact named above, David Best, Assistant Director; LaTonya Miller; Ivy Hübler; Tom Mahalek; Steven Stern; Meredith Allen Kimmett; Wiktor Niewiadomski; Kenneth E. Patton; Karen Richey; Robert Swierczek; and Alyssa Weir made key contributions to this report.