Missile Defense: Mixed Progress in Achieving Acquisition Goals and Improving Accountability
Released: 2014-04-02T00:00:00

Letter

Background

MDA’s BMDS is being designed to counter ballistic missiles of all ranges—short, medium, intermediate, and intercontinental. Because ballistic missiles have different ranges, speeds, sizes, and performance characteristics, MDA is developing multiple systems that, when integrated, provide multiple opportunities to destroy ballistic missiles in flight for the strategic defense of the United States and regional defense of its deployed forces and allies. The BMDS architecture includes space- based sensors, ground- and sea-based radars, ground- and sea-based interceptor missiles, and a command and control, battle management, and communications system to provide the warfighter with the necessary communication links to the sensors and interceptor missiles.

Table 1 provides a brief description of some of the BMDS systems, which MDA refers to as elements, and programs included in this year’s assessment. More details can be found in our report.

When MDA was established in 2002, the Secretary of Defense granted it exceptional flexibility to set requirements and manage the acquisition of the BMDS in order to quickly deliver protection against ballistic missiles. This decision enabled MDA to rapidly deliver assets, but we have reported that it has come at the expense of transparency and accountability. Examples of key problems we have cited in reports in recent years and which continue to affect MDA’s acquisitions are highlighted below.

MDA’s highly concurrent acquisition approach has led to significant cost growth, schedule delays, and in some cases, performance shortfalls. Concurrency is broadly defined as the overlap between technology development and product development or between product development and production. While some concurrency is understandable, committing to product development before requirements are understood and technologies are mature or committing to production and fielding before development is complete is a high-risk strategy that often results in performance shortfalls, unexpected cost increases, schedule delays, and test problems. At the very least, a highly concurrent strategy forces decision makers to make key decisions without adequate information about the weapon’s demonstrated operational effectiveness, reliability, and readiness for production. According to MDA officials, they have taken some steps to identify and track concurrency in their programs. However, high levels of concurrency adopted earlier for some programs persist today.

Testing disruptions have reduced the knowledge planned to be available to inform acquisition decisions and understand performance. For example, flight test failures disrupted MDA’s acquisitions of several components and forced MDA to suspend or slow production of three out of four interceptors, including the GMD interceptor and the Aegis BMD Standard Missile-3 Block IB (SM-3 Bock IB). In the Ground-based Midcourse Defense (GMD) case, because MDA moved forward years ago with CE-I and CE-II interceptor production before completing its flight testing program, test failures have exacerbated disruptions to the program. Specifically, because the program has delivered approximately three-fourths of the interceptors for fielding, it faces difficult and costly decisions on how it will implement corrections from prior test failures. Additionally, after fielding these assets, the program has had to add tests that were previously not planned, in order to assess the extent to which prior issues were resolved. It also had to delay tests that were needed to understand the system’s capabilities and limitations.

MDA has been challenged to meet some of its goals for the European Phased Adaptive Approach (EPAA). During the past several years, MDA has been responding to a mandate from the President to develop and deploy new missile defense systems in Europe. This four-phase effort was designed to rely on increasingly capable missiles, sensors, and command and control systems to defend Europe and the United States. Each successive phase is expected to defend larger areas against more numerous and more capable threat missiles. DOD delivered the first phase, for short and medium range defense of Europe, in December 2011, and has been making progress in developing some systems to support future phases. However, in March 2013, the Secretary of Defense canceled two programs, planned for the fourth phase, thus eliminating the fourth phase, which was intended to provide additional layer for defense of the United States against intercontinental ballistic missiles. The cancelations were driven in part by affordability concerns, schedule delays and technical risks associated with these programs. Our previous work found similar issues with other EPAA efforts. We also found that MDA has lacked a comprehensive management approach to synchronize key EPAA activities.

Finally, MDA’s acquisition baseline reporting has provided limited insight into the cost and schedule progress of the BMDS. Due to the acquisition flexibilities it has been granted, BMDS’s entrance into DOD’s acquisition process is deferred, and laws and policies that generally require major defense acquisition programs to take certain steps at certain phases in the acquisition process will not apply until the program enters this process. For example, major defense acquisition programs are generally required to document key performance, cost, and schedule goals in an acquisition baseline at certain phases in the acquisition process; because BMDS has not progressed through threshold phases of the DOD acquisition process, this requirement is not yet applicable. To improve the transparency and accountability of BMDS development efforts, Congress has enacted legislation requiring MDA to establish some baselines. MDA reported baselines for several BMDS programs to Congress for the first time in its June 2010 BMDS Accountability Report (BAR). Specifically, MDA’s baselines, including resource and schedule baselines, are reported in the BAR and are updated annually. Since 2011, although progress has been made to improve the reporting, we have found issues affecting the usefulness of MDA’s acquisition baselines for oversight due to (1) a lack of clarity, consistency, and completeness; (2) a lack of high-quality supporting cost estimates and schedules; and (3) instability in the content of the baselines.

Our work has recommended a number of actions that can be taken to address the problems we identified. Generally, we have recommended that DOD reduce concurrency and more closely follow knowledge based acquisition practices. We also made recommendations designed to reduce testing risk, and to improve schedule and cost reporting. DOD has generally concurred with our recommendations, and has undertaken some actions to reduce acquisition risk, and improve accountability and transparency.

MDA Made Progress on Testing and Further Improved Some Management Practices

This year we found that MDA gained important knowledge about the BMDS system-level performance and individual elements by successfully executing several flight tests. We also found that MDA further improved some of its acquisition practices for managing the European Phased Adaptive Approach (EPAA) and improved the clarity of its resource and schedule baselines.

Progress: MDA Demonstrates BMDS Capability Through Testing

In April 2014, we reported that MDA made progress in demonstrating the systems’ capabilities by conducting the first system-level operational flight test in September 2013. This is a significant achievement because it is the first time that MDA conducted an operational flight test that involved multiple elements working simultaneously. The test involved warfighters from several combatant commands, and according to independent testing officials, recreated a potentially realistic scenario. During this test, MDA launched two medium-range ballistic missile targets, including its newly developed air-launched extended-medium range ballistic missile (eMRBM). Both the Aegis SM-3 Block IA and THAAD successfully intercepted their targets, demonstrating progress towards achieving an integrated BMDS. In addition, the Aegis BMD SM-3 Block IB and GMD programs successfully conducted developmental flight tests in 2013 that demonstrated key capabilities and modifications made to resolve prior issues. Specifically, the Aegis BMD SM-3 Block IB intercepted all targets in its last three flight tests. GMD also successfully conducted a non- intercept flight test of its CE-II interceptor, demonstrating the performance of a guidance component that MDA redesigned in response to a December 2010 flight test failure.

Progress: EPAA Acquisition Management Improves

We also found that DOD improved the acquisition management of EPAA. In our first report on the subject in 2010, we assessed progress of EPAA acquisition planning against six key acquisition principles that synchronize acquisition activities and ensure accountability.that DOD has established testing and acquisition plans for technology We found development and engineering, and had begun work on identifying key stakeholders. This year, we found improvements in these areas. For example, DOD completed identifying EPAA stakeholders and in 2012 issued a directive updating the warfighter role in testing and capability acceptance.

Lastly, in April 2014, we found that MDA continued to improve the clarity of its resource and schedule baselines, which are reported to Congress in its annual acquisition report called the BAR. In its 2013 BAR, MDA continued to incorporate useful changes it made last year, and took some additional actions to improve the completeness and clarity of the BAR baselines by: identifying the date of the initial baseline and, if applicable, the date when the initial baseline was most recently revised; explaining most of the significant cost and schedule changes from the current baseline estimates against both the estimates reported in the prior year’s BAR and the latest initial baseline; and making the baselines easier to read by removing cluttered formatting such as strikethroughs and highlights that made some of the events listed in past BARs unreadable.

MDA Continues to Face Testing and Acquisition Challenges

Although MDA has taken some steps to improve its acquisitions, the agency continues to face several challenges that we have found in previous reviews. Specifically, it faces challenges stemming from high- risk acquisition practices, as well as challenges in BMDS testing, managing the development of EPAA capabilities, and reporting resource and schedule baselines that support oversight. Until MDA addresses these challenges, the agency and decision makers may not obtain the information needed to assess the capabilities of the BMDS or make informed acquisition and investment decisions.

Challenge: Implementing Higher Risk Acquisition Programs

While MDA has gained important insights through testing and taken some steps to improve management and increase transparency, it still faces challenges stemming from higher-risk acquisition strategies that overlap production activities with development activities. While some concurrency is understandable, committing to production and fielding before development is complete often results in performance shortfalls, unexpected cost increases, schedule delays, and test problems. It can also create pressure to keep producing to avoid work stoppages. Our April 2014 report found that Aegis BMD SM-3 Block IB and GMD, which have already produced some of their assets before completing testing, discovered issues during testing that could affect or have affected production. Although both programs demonstrated progress in resolving previous issues, some of which stemmed from their concurrent acquisition strategies, testing revealed new issues. Specifically:

An interceptor failure during a September 2013 test of Aegis BMD SM-3 Block IB means that a key component, common to the deployed SM-3 Block IA, may need to be redesigned and flight tested. While the failure review is not yet complete, if a redesign is necessary, interceptors that were already produced may require retrofits. MDA continues to procure new SM-3 Block IBs while it investigates the cause of the failure.

A GMD CE-I interceptor failure in a July 2013 flight means that MDA did not demonstrate the interceptor could perform under more challenging conditions than previously tested, further delaying knowledge of the interceptors performance capability. Additionally, the failure precluded confirmation that previous design changes improved performance, and delayed the upcoming test needed to resume production of CE-II interceptors. According to program officials, the failure review is not complete, but the failure could have been caused by a component common to both the CE-I and CE-II interceptors. It is still unclear what, if any, corrective action will be needed. The GMD program has had many years of significant and costly disruptions caused by production getting well ahead of testing and then discovering issues during testing. Consequently, even though some assets have already been produced, MDA has had to add tests that were previously not planned and delay tests that are necessary to understand the system’s capabilities and limitations. Additionally, since it has delivered approximately three-fourths of its interceptors, MDA faces difficult and costly decisions on how it will implement corrections from prior test failures. As a result of these development challenges, the GMD program will likely continue to experience delays, disruptions, and cost growth.

We made recommendations to address the ongoing issues with both systems in our April 2014 report. First, we recommended that the Secretary of Defense direct MDA’s Director to flight test any modifications that may be required to the Aegis SM-3 Block IB, before the Under Secretary of Defense, Acquisitions, Technology, & Logistics approves full production allowing the program to manufacture the remaining interceptors. Second, we also recommended testing the fielded GMD CE- I interceptor in order to complete the original purpose of the failed test to (1) demonstrate the CE-I’s effectiveness against a longer range threat in more challenging conditions, and (2) confirm the effectiveness of previous upgrades as well as (3) confirm any new modifications to address the failure work as intended. DOD partially concurred with the recommendation on the Aegis SM-3 Block IB, stating that MDA will verify the efficacy of any modifications by testing and that the full production decision will be vetted through the DOD process. DOD did not agree with the recommendation on GMD, stating that the decision to flight test the interceptor will be made by the Director, MDA, based on the judgment of other stakeholders.

Challenge: Gaining Expected Knowledge From Testing

In this year’s reports, we found that testing has provided less knowledge than initially planned.experienced testing shortfalls, including failures of Aegis and GMD interceptors I mentioned above. The agency also combined, delayed, and deleted some tests, and eliminated test objectives in others. These changes reduced the knowledge expected to be available to understand the capabilities and limitations of the BMDS. Examples of key testing problems we cited in this year’s reports are: While MDA accomplished some testing goals, it

Operational Integration—Although the September 2013 operational flight test demonstrated layered defense between Aegis BMD and THAAD, the Director, Operational Test and Evaluation concluded that the test did not achieve true integration. Specifically there were system network issues, interoperability limitations, and component failures. For example, the test uncovered several issues with communication networks that are needed for interoperability between the elements. Interoperability is important because it can improve missile defense effectiveness and mitigate some limitations of the systems working alone.

Test plan revisions continue to reduce the knowledge planned to be available to understand BMDS performance and inform acquisition decisions. In our March 2014 and April 2014 reports, we found that MDA combined, delayed, and deleted some tests, and eliminated test objectives in others. For example, MDA had to make some adjustments to its September 2013 operational flight test, reducing the number of targets from five to two and removing the participation of more mature elements. The agency also reduced the number of ground tests, which are used to assess performance and interoperability. While MDA added other ground tests to mitigate some effects of this reduction, they are smaller in scope and may not provide the same amount of data about how the systems work together.

Previously GAO has made recommendations to improve MDA’s ability to gather expected knowledge from testing. For example, we recommended that MDA add non-intercept tests for new targets and ensure that its test plan can absorb unforeseen events, like failures, in order to minimize disruptions to the test schedule. We also recommended that MDA synchronize its testing with development and delivery schedules for its MDA generally concurred with our recommendations, but has assets.not fully implemented them.

Challenge: Managing Development and Deployment of U.S. Missile Defense in Europe

In March 2014, we found that while MDA made further improvements to the way it manages EPAA, it has yet to develop or implement a complete Specifically, MDA management strategy for synchronizing these efforts.has not established an integrated schedule and has yet to completely define EPAA requirements. As a result, it remains unclear how different EPAA efforts are aligned together and what constitutes success in delivering EPAA capabilities. Considering that defensive capability planned for EPAA increasingly depends on integrated performance of the participating systems, an acquisition approach that identifies and synchronizes all needed activities becomes increasingly important. While flexibility is a hallmark of the EPAA policy, it also increases the risk of delivering less capability than expected without demonstrating the actual performance of what is delivered.

In fact, our March 2014 report found concurrency, fragmentation of development activities, and delays for some originally planned capabilities. For example, we found that some systems may be delivered later than originally anticipated for integration activities. This reduces the time to discover and correct issues. We also found schedule delays that reduced both the capability MDA plans to deliver and the understanding of how that capability will perform. For example, although MDA delivered the first set of capability in December 2011, an upgrade originally planned for 2014, is now expected in 2015. Additionally, we found that MDA split the delivery of capability it initially planned to deliver in 2015 into two segments. It now plans to deliver what it calls “basic” or “core” capability in 2015 and the remainder in 2017. Similarly, MDA also realigned its plans for the capability it initially planned for 2018 into two segments— designating a subset of originally planned capability to be delivered in 2018, with the remainder in 2020 or later. Finally, MDA postponed its plans to conduct a formal system-level end-to-end assessment of EPAA capabilities because of concerns with data reliability associated with such tests. MDA is currently making investments to develop the tools it needs to improve the reliability of their system-level assessments, but they are expected to be ready after two-thirds of EPAA capabilities have been delivered.

We have previously made recommendations to improve management of EPAA, which are highlighted in this year’s report. Although DOD generally concurred with these recommendations, it has not yet fully implemented them.

Challenge: Reporting Resource and Schedule Baselines that Support Oversight

Although we found in March 2014 that MDA took some additional steps to improve the clarity of its resource and schedule baselines, this was the fourth year that we have found MDA’s resource baselines are not sufficiently reliable to support oversight. Additionally, issues with the content and presentation of the schedule baselines continue to limit the usefulness of the information for decision makers. According to agency officials, MDA is taking steps to improve the reliability of their resource baselines, however, until MDA completes these efforts, its baselines will not be useful for decision makers to gauge progress.

Since MDA first reported baselines in June 2010, we have found that the underlying information supporting its resource baselines does not meet best practice standards for high-quality cost estimates. baselines reported in its 2013 BAR remain unreliable because the agency is still in the process of improving the quality of the cost estimates that support its baselines. For example,

MDA has not fully implemented its cost estimating handbook. In April 2013, we reported that, in June 2012, MDA completed an internal Cost Estimating Handbook, largely based on GAO’s Cost Estimating and Assessment Guide which, if implemented, could help address nearly all the shortfalls we identified. According to MDA officials, the agency is still in the process of applying that handbook to its cost estimates and therefore revised estimates for BMDS elements included in the 2013 BAR were not ready for our review.

MDA has not obtained independent cost estimates of the reported baselines. Officials from DOD’s Office of the Director for Cost Assessment and Program Evaluation told us that although they examined costs for some BMDS elements over the last two years, they have not completed a formal independent cost estimate for a BMDS element since 2010.

GAO, GAO Cost Estimating and Assessment Guide, GAO-09-3SP (Washington, D.C.: March 2009).

MDA’s cost estimates reported in the 2013 BAR do not include operation and support costs funded by individual military services. In April 2013, we found that MDA was not reporting the operation and support costs borne by other military services and concluded that as a result MDA’s reported costs may significantly understate the full costs for some BMDS elements. We recommended MDA include these costs in its resource baselines reported in the BAR.full costs of DOD programs, but the department stated that the BAR should only include content for which MDA is responsible. However, limiting the baseline reporting to only MDA costs precludes decision makers from having insight into all the costs associated with MDA’s weapons systems. We continue to believe that reporting these costs would aid both departmental and congressional decision makers as they make difficult choices of where to invest limited resources.DOD does not currently report the full costs for MDA’s missile defense acquisitions.

DOD agreed that decision makers should have insight into the In the National Defense Authorization Act for Fiscal Year 2014, Congress took steps to address concerns over MDA’s cost estimates. As a result, we did not make any new recommendations regarding cost this year. However, we plan to continue to monitor MDA’s progress because establishing high-quality cost estimates that are accurate, credible, and complete is fundamental to creating realistic resource baselines.

In April 2014, we also found that assessing MDA’s progress in achieving its schedule goals is difficult because MDA’s 2013 schedule baselines are not presented in a way that allows decision makers to understand or easily monitor progress.identify numerous events, but provide little information on the events and why they are important. In addition, MDA’s schedule baselines do not present any comparisons of event dates. Because MDA’s schedule baselines only present current event dates, decision makers do not have the ability to see if and how these dates have changed.

For instance, MDA’s schedule baselines We recommended that the Secretary of Defense direct the MDA Director to improve the content of the schedule baselines by highlighting critical events, explaining what these events entail and why they are important, and by presenting information in a format that allows identification of changes from the previous BAR as well as from the initial baseline. DOD concurred with our recommendation.

This concludes my statement, I am happy to answer any questions you have.

GAO Contact and Staff Acknowledgments

For future questions about this statement, please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals making key contributions to the work this statement is based on include David B. Best and Patricia Lentini, Assistant Directors; Susan C. Ditto; Aryn Ehlow; Wiktor Niewiadomski; John H. Pendleton; Karen Richey; Brian T. Smith; Jennifer Spence; Steven Stern; Robert Swierczek; Jay Tallon; Brian Tittle; and Hai V. Tran; Alyssa Weir; and Gwyneth B. Woolwine.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.