Missile Defense: Actions Needed to Improve Transparency and Accountability
Released: 2011-04-13T00:00:00

Letter

Acquisition Flexibility Given to MDA has Downsides for Oversight and Accountability

MDA is a unique agency with extraordinary acquisition flexibility and a challenging mission, however while that flexibility has helped it to rapidly field systems, it has also hampered oversight and accountability.

Over the years, Congress has created a framework of laws that makes major defense acquisition programs accountable for their planned outcomes and cost, gives decision makers a means to conduct oversight, and ensures some level of independent program review. Application of many of these laws is triggered by the phases of the Department of Defense’s acquisition cycle, such as entry into engineering and manufacturing development. Specifically, major defense acquisition programs are generally required by law and policy to do the following:  Document program parameters in an acquisition program baseline that, as implemented by DOD, has been approved by the Milestone Decision Authority, a higher-level DOD official prior to the program’s entry into the engineering and manufacturing development phase. The baseline provides decision makers with the program’s best estimate of the program’s total cost for an increment of work, average unit costs for assets to be delivered, the date that an operational capability will be fielded, and the weapon’s intended performance parameters.  Once approved, measure the program against the baseline, which is the program’s initial business case, or obtain the approval of a higher-level acquisition executive before making changes.  Obtain an independent life-cycle cost estimate prior to beginning engineering and manufacturing development, and/or production and deployment. Independent life-cycle cost estimates provide confidence that a program is executable within estimated cost.  Regularly provide detailed program status information to Congress, including information on cost, in Selected Acquisition Reports.  Report certain increases in unit cost measured from the original or current program baseline.  Covered major defense acquisition programs and subprograms are required to complete initial operation test and evaluation before proceeding beyond low-rate initial production. After testing is completed, the Director for Operational Test and Evaluation assesses whether the results of the test confirm that the system or components are effective and suitable for combat.

When MDA was established in 2002, it was granted exceptional flexibility in setting requirements and managing the acquisition, in order that its BMDS be developed as a single program, using a capabilities-based, spiral upgrade approach to quickly deliver a set of integrated defensive capabilities. This decision deferred application of DOD acquisition policy to BMDS until a mature capability is ready to be handed over to a military service for production and operation. Because the BMDS program has not formally entered the DOD acquisition cycle, application of laws that are designed to facilitate oversight and accountability of DOD acquisition programs and that are triggered by phases of this cycle, such as the engineering and manufacturing development phase, has also effectively been deferred. This gives MDA unique latitude to manage the BMDS and it enabled MDA to begin delivering an initial defensive capability in 2004. However, the flexibility also came at the expense of transparency and accountability.

Specifically, a BMDS cost, schedule, and performance baseline does not have to be established or approved by anyone outside MDA. Recent laws have created some baseline-related requirements for parts of the BMDS. In addition, while most major defense acquisition programs are required by statute to obtain an independent verification of cost estimates, MDA has only recently developed cost estimates for selected assets and plans to work with the DOD Office of the Director for Cost Assessment and Program Evaluation to develop independent cost estimates for more MDA elements. Further, assessments of a system’s suitability and effectiveness in combat have only been accomplished, with limitations, for the currently deployed Aegis BMD weapon system. The limited amount of testing completed, which has been primarily developmental in nature, and the lack of verified, validated, and accredited models and simulations prevent the Director of Operational Test and Evaluation from fully assessing the effectiveness, suitability, and survivability of the BMDS in annual assessments. MDA has agreed to conduct an operational flight test in 2012.

As we concluded in a prior report, having less transparency and accountability than is normally present in a major weapon program has had consequences. The lack of baselines for the BMDS along with high levels of uncertainty about requirements and program cost estimates effectively set the missile defense program on a path to an undefined destination at an unknown cost. Across the agency, these practices left programs with limited knowledge and few opportunities for crucial management oversight and decision making concerning the agency’s investment and the warfighter’s continuing needs. At the program level, these practices contributed to quality problems affecting targets acquisitions, which in turn, hampered MDA’s ability to conduct tests as planned.

Numerous Strategy Changes Have Exacerbated Transparency and Accountability Challenges

MDA has employed at least three strategies to acquire and deploy missile defense systems, which has exacerbated transparency and accountability challenges. From its inception in 2002 through 2007, MDA developed missile defense capability in 2-year increments, known as blocks, each built on preceding blocks intended to enhance the development and capability of the BMDS. However, there was little visibility into baseline costs and schedules associated with the systems that comprised the blocks or how the blocks addressed particular threats.

In response to our recommendations, in December 2007, MDA announced a new capabilities-based block structure intended to improve the program’s transparency, accountability, and oversight. Instead of being based on 2-year time periods, the new blocks focused on fielding capabilities that addressed particular threats. Because the new block structure was not aligned to regular time periods, multiple blocks were under way concurrently. This approach included several positive changes, including a DOD commitment to establish total acquisition costs and unit costs for selected block assets, including only those elements or components of elements in a block that would be fielded during the block and abandoning deferrals of work from one block to another.

MDA was still transitioning to this new capabilities-based block approach when the Director, MDA terminated it in June 2009. According to MDA, this was done in order to address congressional concerns regarding how to structure MDA’s budget justification materials. This termination marked the third acquisition management strategy for the BMDS in the prior 3 years and effectively reduced transparency and accountability for the agency. The agency then began to manage BMDS as a single integrated program but planned to report on cost, schedule, and performance issues by each element within the program.

Changing the acquisition strategy is problematic because each time it is changed, the connection is obscured between the old strategies’ scope and resources and the new strategy’s rearranged scope and resources. This makes it difficult for decision makers to hold MDA accountable for expected outcomes and clouds transparency of the agency’s efforts.

We also reported in December 2010 that the adoption of the European Phase Adaptive Approach (PAA) for deploying missile defense assets has limitations in transparency and accountability. Specifically, we reported that DOD made progress in acquisition planning for technology development and systems engineering and testing and partial progress in defining requirements and identifying stakeholders but had not yet developed a European PAA acquisition decision schedule or an overall European PAA investment cost. We found that the limited visibility into the costs and schedule for the European PAA and the lack of some key acquisition management processes reflect the oversight challenges with the acquisition of missile defense capabilities that we have previously reported. We concluded that for the European PAA, the flexibility desired by DOD is not incompatible with appropriate visibility into key aspects of acquisition management. Moreover, as DOD proceeds with the European PAA acquisition activities, it is important for Congress and the President to have assurance that the European PAA policy is working as intended and that acquisition activities are cost-effective. We made recommendations also in January 2011 regarding the development of life-cycle cost estimates and an integrated schedule for the acquisition, infrastructure and personnel activities to help identify European PAA implementation risks. DOD partially concurred with the first recommendation and fully concurred with the second.

Prior GAO Recommendations and Congressional Actions to Improve Transparency and Accountability

Congress has taken action to address concerns regarding the acquisition management strategy, accountability, and oversight of MDA. For example, in the National Defense Authorization Act for Fiscal Year 2008, Congress required MDA to establish acquisition cost, schedule, and performance baselines for each system element that has entered the equivalent of the engineering and manufacturing development phase of acquisition or is being produced or acquired for operational fielding. Most recently, the Ike Skelton National Defense Authorization Act for Fiscal Year 2011 requires the Secretary of Defense to ensure that MDA establishes and maintains an acquisition baseline for each program element of the BMDS.

Since our first MDA report in 2004, we have made a series of recommendations to improve transparency and accountability, many of which are designed to adapt the key transparency and accountability features already embedded in the DOD acquisition regulation and apply them to MDA. Some of our key recommendations include: Establishing and reporting to Congress costs and unit costs, including development costs in unit costs, including sunk costs in cost estimates, reporting top-level test goals, obtaining independent cost and taking steps to ensure the underlying cost estimates are high quality, reliable, and documented reporting variances.

Improving transparency by requesting and using procur instead of research, development, testing and evaluation funds to acquire fielded assets.

Strengthening the test program by establishing baselines for each new class of target in development, including sufficient schedule and resource margin, including spare test assets and targets, and strengthening the role of the Director, Operational Test and Evaluation in assessing missile defense progress.

Implementing a knowledge-based acquisition strategy consistent with DOD acquisition regulations, and ensure that items are not manufactured for fielding before their performance has been validated through testing.

DOD has committed to take action on many of these recommenda While agreeing with our recommendations to enhance baseline reporting, there are differences in MDA’s perspectives on such issues as sunk costs and changes in unit cost. tions.

MDA Has Recently Made Significant Progress in Increasing Transparency and Accountability

In 2010, MDA made significant progress in implementing some of these recommendations by finalizing a new baseline phase review process in which the agency set detailed baselines for several BMDS elements, or portions of elements, for the first time. Specifically, MDA established resource, schedule, test, operational capacity, technical, and contract baselines for several BMDS components. It reported these to Congress in its June 2010 BMDS Accountability Report.

MDA also identified three phases of development where baselines are approved—technology development, product development, and initial production phases—and specified the key knowledge that is needed at each phase. MDA officials stated that they expect that aligning the development efforts with the phases will help to ensure that the appropriate level of knowledge is obtained before the acquisitions move from one phase to the next.

In another key step, approval of the product development and initial production baselines will be jointly reviewed by the Director of MDA and the respective service acquisition executive, as a number of missile defense systems are expected to eventually transition to the military services for operation. In addition, in regard to these new phases, the agency established a process for approving baselines. As a result of MDA’s new baseline phase review process, its 2010 BMDS Accountability Report is more comprehensive than its 2009 report. program and that its test and targets program needed to be managed way that fully supported high-priority near-term programs.

We reported last year that MDA extensively revised the test plan to address these concerns. MDA’s new approach now bases test scenari os r on modeling and simulation needs and extends the test baseline to cove the Future Years Defense Program which allows for better estimation of art of its new target needs, range requirements, and test assets. Also, as p test plan, MDA scheduled dedicated periods of developmental and operational testing, during which the system configuration will rema fixed to allow the warfighter to carry out training, tactics, techniques, and procedures for developmental and operational evaluation. Additionally, the new test plan is expected to provide sufficient time after test events to conduct a full post-test analysis. As we reported last year, these improvements are important because BMDS performance cannot be assessed until models and simulations are accredited and validated and the test program cannot be executed without meeting its target needs.

Our assessment of the schedule baselines determined that we could not compare the asset delivery schedule to the prior year’s baseline because MDA has stopped reporting a comprehensive list of planned asset deliveries.

Finally, we found the test baseline to be well documented. However, because it is success oriented, any problems encountered in executin the plan can cause ripple effects throughout remaining test events. Th frequent changes that continue to occur undermine the value of the test baseline as an oversight tool.

Rapid Pace of Fielding Assets Makes Transparency and Accountability Even More Important

Ove suc ens to t c consistent lack of disciplined analysis that would provide an understanding of what it would take to field a weapon system before system development begins. We have reported that there is a clear set of prerequisites that must be met by each program’s acquisition strategy t o realize successful outcomes. These prerequisites include establishin g a clear, knowledge-based, executable business case for the product. An executable business case is one that provides demonstrated evidence (1) the identified needs are real and necessary and can best be met with the chosen concept and (2) the chosen concept can be developed and produced within existing resources—including technologies, funding, time, and management capacity. Knowledge-based acquisition principle and business cases combined are necessary to establish realistic cost, schedule and performance baselines. Without documented realistic baselines there is no foundation to accurately measure program progre Our work has shown that when agencies do not follow a knowledge-ba approach to acquisition, high levels of uncertainty about requirements, technologies, and design often exist at the start of development program As a result, cost estimates and related funding needs are often understated. r the past 10 years, we have conducted extensive research on cessful programs and have found that successful defense programs ure that their acquisitions begin with realistic plans and baselines pri he start of development. We have previously reported that ause of poor weapon system outcomes, at the program level, is the s.

Aegis Ashore program and the Ground-based Midcourse Defense (GMD) program.  Testing and Targets: As in previous years, failures and delays in testing have continued to delay the validation of models and simulations used to assess BMDS performance. Target availability was a significant, though not the only, driver to the test plan delays. Since 2006, we have reported that target availability has delayed and prompted modifications to planned test objective s. This trend continued in 2010. We reported this year that five tests scheduled for fiscal year 2010 were canceled because of a moratorium on air launches of targets. The moratorium was imposed following the failure of an air launched target participating in MDA’s December 2009 Theater High Altitude Area Defense (THAAD) flight test. A failure review board investi identified the rigging of cables to the missile in the aircraft as immediate cause of the failure and shortcomings in inte processes at the contractor as the underlying cause. Additionally, target shortfalls contributed to delays in flight tests, reduced the number of flight tests, and altered flight test objectives.

December 2009 THAAD flight test failure. The extended use of undefinitized contract actions has previously been identified by GAO and others as risky to the government. Because co officers normally reimburse contractors for all allowable co incur before definitization, contractors bear less risk and have l incentive to control costs during this period. The government als risks incurring unnecessary costs as requirements may change before the contract is definitized.

Aegis Ashore: the ship-based Aegis BMD. It is expected to track and intercept ballistic missiles in their midcourse phase of flight using Standard Missile-3 (SM-3) interceptor variants as they become available. However, while Aegis BMD has demonstrated performance at sea, these demonstrations used the currently fielded 3.6.1 version of Aegis BMD with the SM-3 IA interceptor, not the newer variant of re the Aegis operating system and new interceptor that Aegis Asho will use. Aegis Ashore is dependent on next-generation versions of -3 Aegis systems—Aegis 4.0.1 and Aegis 5.0—as well as the new SM IB interceptor, all of which are currently under development. Moreover, a series of changes are required to further modify these new variants of Aegis BMD for use on land with Aegis Ashore. These modifications include changes to the Vertical Launching System; suppression or disabling of certain features used at se design, integration, and fabrication of a new deckhouse enclosure for the radar, and potential changes to the SM-3 IB interceptor. Changes to those existing Aegis BMD components that will be reused for Aegis Ashore may reduce their maturity in the context of the new Aegis Ashore program, and new features will require Aegis Ashore is MDA’s future land-based variant of testing and assessment to demonstrate their performance. MDA is plans to make production decisions for the first operational Aeg Ashore before conducting both ground and flight tests. We concluded in this year’s report that it is a highly concurrent effor with significant cost, schedule and performance risk.  Ground-based Midcourse Defense: GMD is a ground-based d system designed to provide combatant commanders the capability to defend the homeland against a limited attack from intermediate, and intercontinental-range ballistic missiles during the midcourse phase of flight. The GMD consists of a ground-based interceptor—a booster with an Exoatmospheric Kill Vehicle on top—and a fire control system that receives target information from sensors in order to formulate a battle plan. GMD continues to deliver assets before testing has fully determined their capabilities and limitations. The Director, MDA testified on March 31, 2011 that he considers the GMD interceptors essentially prototypes. In the urgency to deploy assets to meet the Presidential directive to field an initial capability by 2004, assets were built and deployed before developmental testing was completed. During the ongoing developmental testing, issues were found that led to a need for retrofits. GMD intercept tests conducted to date have already led to major hardware or software changes to the interceptors—not all of which have been verified through flight testing. In addition, manufacturing of a new variant called the Capability Enhancement II is well underway and more than half of those variants have already been delivered although their capability has not been validated through developmental flight tests. To date, the two f tests utilizing this variant have both failed to intercept the target. According to MDA, as a result of the most recent failure in December 2010, deliveries of this variant have been halted. Again, because of the urgency to deploy some capability, limited work was undertaken on long-term sustainment for the system which is critical to ensure the system remains effective through 2032. In September 2010, MDA finalized the GMD Stockpile Reliability Program Plan, a key step in developing the knowledge needed to determine the sustainment needs of the GMD system.

Concluding Observations

Chairman Nelson, Ranking Member Sessions, and Members of the Subcommittee, this completes my prepared statement. I would be respond to any questions you may have at this time.

Contact and Staff Acknowledgments

For questions about this statement, please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals making key contributions to this statement include David Best, Assistant Director; LaTonya Miller; Steven Stern; Meredith Allen Kimmett; Letisha Antone; Gwyneth Woolwine; Teague Lyons; Kenneth E. Patton; Robert Swierczek; and Alyssa Weir.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.