Defense Acquisitions: Assessment of DOD Efforts to Enhance Missile Defense Capabilities and Oversight
Released: 2008-02-26T00:00:00

Letter

Background

Funded at $8 billion to nearly $10 billion annually, MDA’s BMDS is the largest research development program in the Department of Defense’s budget. Since the 1980s, DOD has spent more that $100 billion on the development and early fielding of this system and it estimates that continued development will require an additional $50 billion between fiscal years 2008 and 2013.

Since 2002, MDA has worked to fulfill its mission through its development and fielding of a diverse collection of land-, air-, sea-, and space-based assets. These assets are developed and fielded through nine BMDS elements and include the Airborne Laser (ABL); Aegis Ballistic Missile Defense (Aegis BMD); BMDS Sensors; Command, Control, Battle Management, and Communications (C2BMC); Ground-based Midcourse Defense (GMD); Kinetic Energy Interceptors (KEI); Multiple Kill Vehicles (MKV); Space Tracking and Surveillance System (STSS); and Terminal High Altitude Area Defense (THAAD).

To develop a system capable of carrying out its mission, MDA, until December 2007, executed an acquisition strategy in which the development of missile defense capabilities was organized in 2-year increments known as blocks. Each block was intended to provide the BMDS with capabilities that enhanced the development and overall performance of the system. The first 2-year block, known as Block 2004, fielded a limited initial capability that included early versions of the GMD, Aegis BMD, Patriot Advanced Capability-3, and C2BMC elements. The agency’s second 2-year block– Block 2006– culminated on December 31, 2007 and fielded additional BMDS assets. This block also provided improved GMD interceptors, enhanced Aegis BMD missiles, upgraded Aegis BMD ships, a Forward-Based X-Band-Transportable radar, and enhancements to C2BMC software. On December 7, 2007, MDA’s Director approved a new block construct that will be the basis for all future development and fielding, which I will discuss in more detail shortly.

To assess progress during Block 2006, we examined the accomplishments of nine BMDS elements that MDA is developing and fielding. Our work included examining documents such as Program Execution Reviews, test plans and reports, production plans, and Contract Performance Reports. We also interviewed officials within each element program office and within MDA functional directorates. In addition, we discussed each element’s test program and its results with DOD’s Office of the Director, Operational Test and Evaluation. In following up on transparency, accountability, and oversight issues raised in our March 2007 report, we held discussions with officials in MDA’s Directorate of Business Operations to determine whether its new block structure improved accountability and transparency of the BMDS. In addition, we reviewed pertinent sections of the U.S. Code to compare MDA’s current level of accountability with federal acquisition laws. We also interviewed officials from the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics and DOD’s Joint Staff to discuss the oversight role of the new Missile Defense Executive Board (MDEB). Additionally, we reviewed the MDEB charter to identify the oversight responsibility of the board.

Fielded Capability Increased, but Less than Planned at Higher Cost

MDA made progress in developing and fielding the BMDS during 2007. Additional assets were fielded and/or upgraded, several tests met planned objectives, and other development activities were conducted. On the other hand, fewer assets were fielded than originally planned, some tests were delayed, and the cost of the block increased by approximately $1 billion. To stay within the revised budget despite increasing contractor costs, MDA deferred some budgeted work to future blocks. Such deferrals, coupled with a planning methodology used by some contractors that could obscure cost reporting, prevent us from determining the full cost of Block 2006. MDA was able to meet most test objectives despite delays in several elements’ test schedules. Neither we nor DOD could evaluate the aggregate performance of fielded assets because flight testing to date has not generated sufficient data. An evaluation of aggregate performance would also have to consider that (1) some parts in fielded interceptors identified as potentially problematic have not been replaced yet, and (2) tests done to date have been developmental in nature and do not provide sufficient realism for DOD to determine if the BMDS is suitable and effective for battle.

Fielding of Assets and Cost

During Block 2006, MDA increased its inventory of BMDS assets while enhancing the system’s performance. It fielded 14 additional Ground-based interceptors, 12 Aegis BMD missiles designed to engage more advanced threats, 4 new Aegis BMD destroyers, 1 new Aegis BMD cruiser, and 8 Web browsers and 1 software suite for C2BMC. In addition, MDA upgraded half of its Aegis BMD ship fleet, successfully conducted four Aegis BMD and two GMD intercept tests, and completed a number of ground tests to demonstrate the capability of BMDS components.

Although MDA fielded an increased capability, it was unable to deliver all assets originally planned for Block 2006. The Sensors element was the only Block 2006 element to meet all of its original goals set in March 2005 while the remaining elements––GMD, Aegis BMD, C2BMC––were unable to meet all of their original quantity goals. Sensors delivered a second FBX-T in January 2007 while the GMD element fielded 14 of 15 Ground- Based interceptors originally planned during Block 2006. Last year, we reported that MDA delayed the partial upgrade of the Thule early warning radar—one of GMD’s original goals-- until a full upgrade could be accomplished. Additionally, the Aegis BMD element delivered 4 additional Destroyers and 1 new Cruiser as originally planned, but did not meet its original goal for missile deliveries––delivering 12 of 19 SM-3 missiles planned for the block. C2BMC also did not deliver two of the three software suites originally planned for Block 2006.

MDA’s Block 2006 program of work culminated with higher than anticipated costs. In March 2007, we reported that MDA’s cost goal for Block 2006 increased by approximately $1 billion because of greater than expected GMD operations and sustainment costs and technical problems. If the contractors continue to perform as they did in fiscal year 2007, we estimate that at completion, the cumulative overrun in the contracts could be between about $1.9 billion and $2.8 billion. To stay within its revised budget, MDA deferred some work it expected to accomplish during the block. When work is deferred, its costs are no longer accounted for in the original block. In other words, if work planned and budgeted for Block 2006 was deferred to Block 2008, that work would be counted as a Block 2008 cost. Because MDA did not track the cost of the deferred work, the agency could not make an adjustment that would have matched the cost with the correct block. Consequently, we were unable to determine the full cost of Block 2006.

Another reason why it is difficult to determine the actual cost of Block 2006 is a planning methodology employed by MDA prime contractors that can obscure the full cost of work. Contractors typically divide the total work of a contract into small efforts in order to define them more clearly and to ensure proper oversight. Work is planned into types of work packages including: (1) level of effort– work that contains tasks of a general or supportive nature and does not produce a definite end product and (2) discrete—work that has a definable end product or event. When work is discrete, delivery of the end product provides a sound basis for determining actual contractor performance. When discrete work is instead planned as level of effort, the contractor’s performance becomes less transparent because work is considered complete when the time planned for it has expired, whether or not the intended product has been completed. Earned value management does not recognize such variances in completing scheduled work and to the extent more work has to be done to complete the product, additional costs could be incurred that are not yet recognized. Many of MDA’s prime contractors plan a large percentage of their work as level of effort. MDA officials agree that its contractors have improperly planned discrete work as level of effort, and are taking steps to remedy the situation.

We also observed that while several contractors had difficulty with controlling costs, during fiscal year 2007, MDA awarded approximately 90 percent or $579 million of available award fee to its prime contractors. In particular, contractors developing the ABL and Aegis BMD Weapon System were rated as performing very well in the cost and/or program management elements and received commensurate fees, even though earned value management data showed that their cost and schedule performance was declining. Although DOD guidance discourages the use of earned value performance metrics in award fee criteria, MDA includes this––one of many factors for consideration in rating contractors’ performance––in several of its award fee plans. The agency recognizes that there is not always a good link between its intentions for award fees and the amount of fee being earned by its contractors. In an effort to rectify this problem, the agency has begun to revise its award fee policy to align agency practices more closely with DOD’s current policy that better links performance with award fees.

Testing and Performance of Fielded Capability

Most test objectives were achieved during 2007, although several BMDS programs experienced setbacks in their test schedules. The MKV, KEI, and Sensors elements were able to execute all scheduled activities as planned. The Aegis BMD, THAAD, ABL, STSS, and C2BMC elements experienced test delays, but all were able to achieve their primary test objectives. GMD successfully completed an intercept with an operationally representative interceptor and a radar characterization test. A second intercept test employing the SBX radar has been delayed because a target malfunction delayed the execution of the first intercept test. The SBX capability is important as it is a primary sensor to be used to engage ballistic missiles in the midcourse phase of flight. As of yet, this capability has not been verified through flight testing.

As we reported in March 2007, MDA altered its original Block 2006 performance goals commensurate with the agency’s reductions in the delivery of fielded assets. For several reasons, information is not sufficient to assess whether MDA achieved its revised performance goals. First, MDA uses a combination of simulations and flight tests to determine whether performance goals are met. However, too few flight tests have been completed to ensure the accuracy of the models and simulations predictions. Second, confidence in the performance of the BMDS is reduced because of unresolved technical and quality issues in the GMD element. For example, the GMD element has experienced the same anomaly during each of its flight tests since 2001. This anomaly has not yet prevented the program from achieving any of its primary test objectives, but to date neither its source nor solution has been clearly identified. Program officials plan to continue their assessment of test data to determine the anomaly’s root cause. The performance of some fielded GMD assets is also questionable because they contain parts identified by auditors in MDA’s Office of Quality, Safety, and Mission Assurance as less reliable or inappropriate for use in space that have not yet been replaced. MDA has begun to replace the questionable parts in the manufacturing process and to purchase the parts for retrofit into fielded interceptors. However, it will not complete the retrofit effort until 2012.

Finally, tests of the GMD element have been of a developmental nature, and have not included operationally representative test geometries in which GMD will perform its mission. MDA has added operational test objectives to its developmental test program, but the objectives are mostly aimed at proving that military personnel can operate the equipment. The lack of data has limited the operational test and evaluation Director’s annual BMDS assessment to commenting on aspects of tests that were operationally realistic and thus has prevented the Director from determining whether the system is suitable and effective for the battlefield.

Key Steps Taken to Enhance BMDS Oversight, But More Can Be Done

Since its initiation in 2002, MDA has been given a significant amount of flexibility. While this flexibility allows agile decision making, it lessens the transparency of MDA’s acquisition processes, making it difficult to conduct oversight and hold the agency accountable for its planned outcomes and costs. As we reported in March 2007, MDA operates with considerable autonomy to change goals and plans, which makes it difficult to reconcile outcomes with original expectations and to determine the actual cost of each block and of individual operational assets. In the past year, MDA has begun implementing two initiatives—a new block construct and a new executive board—to improve transparency, accountability, and oversight. These initiatives represent improvements over current practices, although we see additional improvements MDA can make. In addition, Congress has directed that MDA begin buying certain assets with procurement funds like other programs, which should promote accountability for and transparency of the BMDS.

New Block Structure Offers Improvements, but Does Not Resolve All Issues

In 2007, MDA redefined its block construct to better communicate its plans and goals to Congress. The agency’s new construct is based on fielding capabilities that address particular threats as opposed to the previous biennial time periods. MDA’s new block construct makes many positive changes. These include establishing unit cost for selected block assets, incorporating into a block only those elements or components that will be fielded during the block, and abandoning the practice of deferring work from block to block.

These changes should improve the transparency of the BMDS program and make MDA more accountable for the investment being made in missile defense. For example, the actual cost of each block can be tracked because MDA will no longer defer work planned for one block, along with its cost, to a future block. In addition, MDA plans to develop unit cost for selected BMDS assets–– such as THAAD interceptors–– so that cost growth of those assets can be monitored. In addition, the agency plans to request an independent verification of these unit costs and report significant cost growth to Congress. However, MDA has not yet determined all of the assets that will report a unit cost or how much a unit cost must increase before it is reported to Congress.

Although improvements are inherent in MDA’s proposed block construct, the new construct does not resolve all transparency and accountability issues. For example, MDA officials told us that the agency does not plan to estimate the full cost of a block. Instead, the cost baseline reported to Congress will include all prior costs of the block and the expected budget for the block for the 6 years included in DOD’s Future Years Defense Plan. Costs beyond the 6th year of the plan will not be estimated. Once baselined, if the budget for a block changes, MDA plans to report and explain those variations to Congress. Because the full cost of each block will not be known, it will be difficult for decision makers to compare the value of investing in each block to the value of investing in other DOD programs or to determine whether a block is affordable over the long term. Other DOD programs are required to provide the full cost estimate of developing and producing their weapon system, even if the costs extend beyond the Future Years Defense Plan.

Another issue yet to be addressed is whether the concurrent development and fielding of BMDS assets will continue. Fully developing an asset and demonstrating its capability prior to production increases the likelihood that the product will perform as designed and can be produced at the cost estimated. To field an initial capability quickly, MDA accepted the risk of concurrent development and fielding during Block 2004. It continued to do so during Block 2006 as it fielded assets before they were fully tested. For example, by the end of Block 2004, the agency realized that the performance of some ground-based interceptors could be degraded because the interceptors included inappropriate or potentially unreliable parts. As noted earlier, MDA has begun the process of retrofitting these interceptors, but work will not be completed until 2012. Meanwhile, there is a risk that some interceptors might not perform as designed. MDA has not addressed whether it will accept similar performance risks under its new block construct or whether it will fully develop and demonstrate all elements/components prior to fielding.

MDA has not addressed whether it will transfer assets produced during a block to a military service for production and operation at the block’s completion. Officials representing multiple DOD organizations recognize that transfer criteria are neither complete nor clear given the BMDS’s complexity. Without clear transfer criteria, MDA has transferred the management of only one element—the Patriot Advanced Capability-3—to the military for production and operation. For other elements, MDA and the military services have been negotiating the transition of responsibilities for the sustainment of fielded elements—a task that has proven to be time consuming. Although MDA documents show that under its new block construct the agency should be ready to deliver BMDS components that are fully mission-capable, MDA officials could not tell us whether at the end of a block MDA’s Director will recommend when management of components, including production responsibilities, will be transferred to the military.

New Executive Board Offers Improved, but Not Full, Oversight

Oversight improvement initiatives are also underway for MDA. In March 2007, the Deputy Secretary of Defense established a Missile Defense Executive Board (MDEB) to recommend and oversee implementation of strategic policies and plans, program priorities, and investment options for protecting the United States and its allies from missile attacks. The MDEB is also to replace existing groups and structures, such as the Missile Defense Support Group.

The MDEB appears to be vested with more authority than the Missile Defense Support Group. When the Support Group was chartered in 2002, it was to provide constructive advice to MDA’s Director. However, the Director was not required to follow the advice of the group. According to a DOD official, although the Support Group met many times initially, it did not meet after June 2005. This led to the formation of the MDEB. Its mission is to review and make recommendations on MDA’s comprehensive acquisition strategy to the Deputy Secretary of Defense. It is also to provide the Under Secretary of Defense for Acquisition, Technology and Logistics, with a recommended strategic program plan and a feasible funding strategy based on business case analysis that considers the best approach to fielding integrated missile defense capabilities in support of joint MDA and warfighter objectives. The MDEB will be assisted by four standing committees. These committees, who are chaired by senior-level officials from the Office of the Secretary of Defense and the Joint Staff, could play an important oversight role as they are expected to make recommendations to the MDEB, which in turn, will recommend courses of action to the Under Secretary of Defense and the Director, MDA as appropriate.

Although the MDEB is expected to exercise some oversight of MDA, it will not have access to all the information normally available to DOD oversight bodies. For other major defense acquisition programs, the Defense Acquisition Board has access to critical information because before a program can enter the System Development and Demonstration phase of the acquisition cycle, statute requires that certain information be developed. However, in 2002, the Secretary of Defense deferred application of DOD policy that, among other things, require major defense programs to obtain approval before advancing from one phase of the acquisition cycle to another. Because MDA does not yet follow this cycle, and has not yet entered System Development and Demonstration, it has not triggered certain statutes requiring the development of information that the Defense Acquisition Board uses to inform its decisions. For example, most major defense acquisition programs are required by statute to obtain an independent verification of life-cycle cost estimates prior to beginning system development and demonstration, and/or production and deployment. Independent life-cycle cost estimates provide confidence that a program is executable within estimated cost. Although MDA plans to develop unit cost for selected block assets and to request that DOD’s Cost Analysis Improvement Group verify the unit costs, the agency does not yet plan to do so for a block cost estimate.

Statute also requires an independent verification of a system’s suitability for and effectiveness on the battlefield through operational testing before a program can proceed beyond low-rate initial production. After testing is completed, the Director for Operational Test and Evaluation assesses whether the test was adequate to support an evaluation of the system’s suitability and effectiveness for the battlefield, whether the test showed the system to be acceptable, and whether any limitations in suitability and effectiveness were noted. However, a comparable assessment of the BMDS assets being fielded will not be available to the MDEB as MDA conducts primarily developmental tests of its assets with some operational test objectives. As noted earlier, developmental tests do not provide sufficient data for operational test officials to make such an assessment of BMDS.

MDA will also make some decisions without needing approval from the MDEB or any higher level official. Although the charter of the MDEB includes the mission to make recommendations to MDA and the Under Secretary of Defense for Acquisition, Technology and Logistics on investment options, program priorities, and MDA’s strategy for developing and fielding an operational missile defense capability, the MDEB will not necessarily have the opportunity to review and recommend changes to BMDS blocks. MDA documents show that the agency plans to continue to define each block of development without requiring input from the MDEB. According to a briefing on the business rules and processes for MDA’s new block structure, the decision to initiate a new block of BMDS capability will be made by MDA’s Director. Also cost, schedule, and performance parameters will be established by MDA when technologies that the block depends upon are mature, a credible cost estimate can be developed, funding is available, and the threat is both imminent and severe. The Director will inform the MDEB as well as Congress when a new block is initiated, but he will not seek the approval of either.

Finally, there will be parts of the BMDS program that the MDEB will have difficulty overseeing because of the nature of the work being performed. MDA plans to place any program that is developing technology in a category known as Capability Development. These programs, such as ABL, KEI, and MKV, will not have a firm cost, schedule, or performance baseline. This is generally true for technology development programs in DOD because they are in a period of discovery, which makes schedule and cost difficult to estimate. On the other hand, the scale of the technology development in BMDS is unusually large, ranging from $2 billion to about $5 billion dollars a year—eventually comprising nearly half of MDA’s budget by fiscal year 2012. The MDEB will have access to the budgets planned for these programs over the next five or six years, each program’s focus, and whether the technology is meeting short term key events or knowledge points. But without some kind of baseline for gauging progress in these programs, the MDEB will not know how much more time or money will be needed to complete technology maturation. MDA’s experience with the ABL program provides a good example of the difficulty in estimating the cost and schedule of technology development. In 1996, the ABL program believed that all ABL technology could be demonstrated by 2001 at a cost of about $1 billion. However, MDA now projects that this technology will not be demonstrated until 2009 and its cost has grown to over $5 billion.

MDA Directed to Use Procurement Funding

In an effort to further improve the transparency of MDA’s acquisition processes, Congress has directed that MDA’s budget materials delineate between funds needed for research, development, test and evaluation; procurement; operations and maintenance; and military construction. Congress gave MDA the flexibility to field certain assets using research, development, test and evaluation funding which allowed MDA to fund the purchase of assets over multiple years. Congress recently restricted MDA’s authority and required MDA to purchase certain assets with procurement funds. Using procurement funds will mean that MDA will be required to ensure that assets are fully funded in the year of their purchase, rather than incrementally funded over several years. Additionally, our analysis of MDA data shows that incremental funding is usually more expensive than full-funding, in part, because inflation decreases the buying power of the dollar each year. For example, after reviewing MDA’s incremental funding plan for THAAD fire units and Aegis BMD missiles, we analyzed the effect of fully funding these assets and found that the agency could save about $125 million by fully funding their purchase and purchasing them in an economical manner.

Upcoming Report

Our annual report on missile defense is in draft and with DOD for comment. It will be issued in final by March 15, 2008. In that report, we are recommending additional steps that could build on efforts to further improve the transparency, accountability, and oversight of the missile defense program. Our recommendations include actions needed to improve cost reporting as well as testing and evaluation. DOD is in the process of preparing a formal response to the report and its recommendations.

Mr. Chairman, this concludes my statement. I would be pleased to respond to any questions you or members of the subcommittee may have.

Contacts and Staff Acknowledgments

For questions about this statement, please contact me at (202) 512-4841 or Francisp@gao.gov. Individuals making key contributions to this statement include David Best, Assistant Director; LaTonya D. Miller; Steven B. Stern; Meredith Allen Kimmett; Kenneth E. Patton; and Alyssa Weir.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.