Defense Acquisitions: Missile Defense Acquisition Strategy Generates Results but Delivers Less at a Higher Cost
Released: 2007-03-15T00:00:00

Letter

Background

MDA’s mission is to develop and field an integrated and layered Ballistic Missile Defense System to defend the United States, its deployed forces, allies, and friends against all ranges of enemy ballistic missiles in all phases of flight. This is challenging, requiring a complex combination of defensive components—space-based sensors, surveillance and tracking radars, advanced interceptors, command and control, and reliable communications—that work together as an integrated system.

A typical hit-to-kill engagement scenario for an intercontinental ballistic missile (ICBM) would unfold as follows: Infrared sensors aboard early-warning satellites detect the hot plume of a missile launch and alert the command authority of a possible attack.

Upon receiving the alert, land- or sea-based radars are directed to track the various objects released from the missile and, if so designed, to identify the warhead from among spent rocket motors, decoys, and debris.

When the trajectory of the missile’s warhead has been adequately established, an interceptor—consisting of a “kill vehicle” mounted atop a booster—is launched to engage the threat. The interceptor boosts itself toward a predicted intercept point and releases the kill vehicle.

The kill vehicle uses its onboard sensors and divert thrusters to detect, identify, and steer itself into the warhead. With a combined closing speed on the order of 10 kilometers per second (22,000 miles per hour), the warhead is destroyed through a “hit-to-kill” collision with the kill vehicle above the atmosphere.

To develop a system capable of carrying out such an engagement, MDA is executing an evolutionary acquisition strategy in which the fielding of missile defense capabilities is organized in 2-year increments known as blocks. Each block is intended to provide the Ballistic Missile Defense System with capabilities that will enhance the development and overall performance of the system. The first block—Block 2004—ended on December 31, 2005, and fielded a limited initial capability that included early versions of the Ground-based Midcourse Defense; Aegis Ballistic Missile Defense; Patriot Advanced Capability-3; and the Command, Control, Battle Management, and Communications element.

During calendar year 2006 and 2007, MDA is focusing its program of work on the enhancement of four fielded BMDS elements—GMD, Aegis BMD, Sensors, and C2BMC. The primary contribution of Block 2006 is that it fields additional assets and continues the evolution of Block 2004 by providing improved GMD interceptors, enhanced Aegis BMD missiles, upgraded Aegis BMD ships, a Forward-Based X-Band—Transportable radar, and enhancements to the C2BMC software.

MDA divides each year’s budget request into a request for the current block and requests for future blocks that have not yet formally begun. For example, in fiscal year 2006, MDA requested funds for Block 2006 and for blocks that begin in 2008, 2010, 2012, and 2014. When MDA submitted its Block 2006 budget to Congress in February 2005, the agency requested funding for not only the four elements fielding assets during Block 2006, but also for the continued development of three elements—ABL, STSS, and THAAD—that will not field assets for operational use until future blocks. According to MDA officials, these elements—which are primarily developmental elements—were included in the block because the agency believed that during the block time frame the elements offered some emergency capability. MDA also requested fiscal year 2006 funds for two other developmental elements, MKV and KEI. However, MDA did not include funding for these elements in its Block 2006 budget request because they provided no capability during the block. Instead, MDA requested funding for MKV in its fiscal year 2006 request for Advanced Component Development and Prototypes—a program element that is not tied to any block—and for KEI in the agency’s fiscal year 2006 request for Block 2014. Table 1 provides a brief description of all elements being developed by MDA.

As part of MDA’s planning process, the agency defines overarching goals for the development and fielding of the current BMDS block. These goals identify the composition of the block (the elements in development and those planned for fielding), the type and quantity of assets to be fielded, the cost associated with element development and fielding (including operation and sustainment activites), and the performance expected of fielded assets. For example, in March 2005, MDA told Congress that its Block 2006 program of work would include seven elements—ABL, Aegis BMD, C2BMC, GMD, Sensors, STSS, and THAAD. Further, MDA identified the cumulative number of assets that Aegis BMD, C2BMC, GMD, and the Sensors elements would field by the end of the block, and the performance that those assets would deliver in terms of probability of engagement success, the land area from which a ballistic missile launch could be denied, and the land area that could be protected from a ballistic missile launch. Finally, MDA told Congress that it would try to complete all Block 2006 work for $20.458 billion.

To enable MDA to meet its overarching goals, each element’s program office establishes its own plan for fielding and/or developmental activities. For example, each program office develops a delivery plan and a test schedule that contributes to MDA’s performance and fielding goals. The programs also work with their prime contractor to plan the block of work so that it can be completed within the program’s share of MDA’s budget.

Since 2002, missile defense has been seen as a national priority and has been funded at nearly requested levels. However, DOD’s Program Budget Decision in December 2004 called for MDA to plan for a $5 billion reduction in funding over fiscal years 2006-2011. Future MDA budgets could be affected by cost growth in federal entitlement programs that are likely to decrease discretionary spending and by increased DOD expenditures, such as expenses created by the Iraq conflict.

Last year we reported that MDA strayed from the knowledge-based acquisition strategy that allows successful developers to deliver, within budget, a product whose performance has been demonstrated. In doing so, MDA fielded assets before their capability was known and the full cost of the capability was not transparent to decision makers. We noted that it was possible for MDA to return to a knowledge-based approach to development while still fielding capability in blocks, but that corrective action was needed to put all BMDS elements on a knowledge-based approach. That is, instead of concurrently developing, testing and fielding the BMDS, MDA would need to adopt knowledge points at which the program would determine if it was ready to begin new acquisition activities. These knowledge points would be consistent with those called out in DOD’s acquisition system policy. To provide a basis for holding MDA accountable for delivering within estimated resources and to ensure the success of future MDA development efforts, we recommended that the Secretary of Defense implement a knowledge-based acquisition strategy for all the BMDS elements, assess whether the current 2-year block strategy was compatible with the knowledge-based development strategy, and adopt more transparent criteria for reporting each element’s quantities, cost, and performance. DOD has not taken any action on the first two recommendations because it considers MDA’s acquisition strategy as knowledge-based and because MDA’s block strategy is compatible with the strategy MDA is implementing. Neither did DOD agree to take action on our third recommendation to adopt more transparent criteria for identifying and reporting program changes. In its comments, DOD responded that MDA is required by statute to report significant variances in each block’s baseline and that these reports along with quarterly DOD reviews provide an adequate level of program oversight.

BMDS Elements Made Progress, but It Was Less Than Expected and It Cost More than Planned

MDA made progress during fiscal year 2006 in carrying out planned accomplishments for the block elements, but it will not deliver the value originally planned for Block 2006. Costs have increased, while the scope of work has decreased. It is also likely that in addition to fielding fewer assets other Block 2006 work will be deferred to offset growing contractor costs. Actual costs cannot be reconciled with original goals because the goals have been changed, work travels to and from other blocks, and individual program elements do not account for costs consistently. In addition, although element program offices achieved most of their 2006 test objectives, the performance of the BMDS cannot yet be fully assessed because there have been too few flight tests conducted to anchor the models and simulations that predict overall system performance. Several elements continue to experience technical problems which pose questions about the performance of the fielded system and could delay the enhancement of future blocks.

Block 2006 Costs Have Increased, but a Full Accounting Is Not Possible

Block 2006 costs have increased because of technical problems and greater than expected GMD operations and sustainment costs. In March 2006, shortly after the formal initiation of Block 2006, increasing costs and other events prompted MDA to reduce the quantity of assets it intended to field during the block. Although the agency reduced the scope of Block 2006, most of the elements’ prime contractors reported that work completed during fiscal year 2006 cost more than planned. Consequently, MDA officials told us it is likely that other work planned for Block 2006 will be deferred until Block 2008 to cover fiscal year 2006 overruns. Furthermore, changing goals, inconsistent reporting of costs by the individual elements, and MDA’s past practice of accounting for the cost of deferred work prevents a determination of the actual cost of Block 2006.

Cost Increases Lead to Revised Block 2006 Cost Goal

MDA’s cost goal for Block 2006 has increased by approximately $1 billion. In March 2005, MDA established a goal of $20.458 billion for the development, fielding, and sustainment of all Block 2006 components. However by March 2006, it had grown by about $1 billion. Cost increases were caused by the: addition of previously unknown operations and sustainment requirements, realignment of the GMD program to support a successful return to flight, realignment of the Aegis BMD program to address technical challenges and invest in upgrades to keep pace with the near term threat, and preparations for round-the-clock operation of the BMDS when the system was put on alert.

In an effort to keep costs within the goal, MDA shifted THAAD’s future development costs of $1.13 billion to another block. That is, the agency moved the cost associated with THAAD’s development in fiscal years 2006 through 2011—which in March 2005 was considered a Block 2006 cost—to Block 2008. This accounting change accommodated the cost increase. According to MDA’s November 2006 Report to Congress, THAAD costs will be reported as part of Block 2008 costs to better align the agency’s resources with the planned delivery of THAAD fire units in 2008. Tables 2 and 3 compare the Block 2006 cost goal established for the BMDS in March 2005 and March 2006.

For the purposes of this report, we have adjusted the March 2005 cost goal to reflect the deletion of future THAAD cost from Block 2006. This enables the revised cost goal that excludes THAAD to be compared with the original cost goal. Had THAAD’s cost been removed from MDA’s March 2005 cost goal, Block 2006 would have actually totaled about $19.3 billion. Comparing this with the March 2006 revised goal of approximately $20.3 billion reveals the $1 billion increase in estimated Block 2006 costs.

Individual Elements Account for Block 2006 Costs Differently

The 2-year block structure established by MDA has proven to be a complicated concept for its BMDS elements to implement. According to officials, MDA defines its block structure in two types of capabilities: Early Capability–A capability that has completed sufficient testing to provide confidence that the capability will perform as designed. In addition, operator training is complete and logistical support is ready. So far, Aegis BMD, C2BMC, and GMD are the only elements that have met these criteria.

Full Capability–These capabilities have completed all system-level testing and have shown that they meet expectations. At this stage, all doctrine, organization, training, material, leadership, personnel, and facilities are in place.

According to MDA officials, the early capability is typically fielded during one block and the full capability is usually attained during the next or a subsequent block. However, not all elements account for Block 2006 costs in the same manner. For example, table 4 below shows that some elements included costs that will be incurred to reach full capability—costs that will be recognized in fiscal year 2009 through 2011—while other elements have not.

According to agency officials, the cost of all activities needed to validate the performance of Block 2006 Fielded Configuration elements should be included as part of the BMDS Block 2006 costs even though these activities may occur during future blocks. According to officials from MDA’s Systems Engineering and Integration Directorate, the C2BMC and Aegis BMD programs’ cost accounting for Block 2006 are the most accurate because the programs included the costs to conduct follow-on testing in subsequent years. Additionally, the officials said that other elements of the BMDS will conduct similar tests in the years following the actual delivery of their Block 2006 capabilities; however, the costs were not included as Block 2006 costs. If each BMDS element were to consistently report block costs, the planned costs for Block 2006 would be higher than MDA’s current reported costs of $20.34 billion.

Increasing Costs Are Causing MDA to Reduce Scope of Block 2006

MDA is making some progress toward achieving its revised Block 2006 goals, but the number of fielded assets and their overall performance will be less than planned when MDA submitted its Block 2006 goals to Congress in March 2005. MDA notified Congress that it was revising its Block 2006 Fielded Configuration Baseline in March 2006, shortly after submitting its fiscal year 2007 budget.

When MDA provided Congress with its quantity goals in March 2005, it stated those goals cumulatively. That is, MDA added the number of Block 2004 assets that it planned to field by December 31, 2005, to the number of assets planned for Block 2006. However, in the case of GMD interceptors, MDA was unable to meet its Block 2004 quantity goals, which, in effect, caused MDA’s Block 2006 goal for interceptors to increase. For example, MDA planned to field 18 GMD interceptors by December 31, 2005, and to field an additional 7 interceptors during Block 2006, for a total of 25 interceptors by the end of Block 2006. But, because it did not meet its Block 2004 fielding goal—fielding only 10 of the 18 planned interceptors— MDA could not meet its Block 2006 cumulative goal of 25 without increasing its Block 2006 deliveries. For purposes of this report, we determined the number of assets that MDA would have to produce to meet its Block 2006 cumulative quantity goal. Table 5 depicts only those quantities and shows how they have changed over time.

According to MDA, it reduced the number of GMD interceptors in March 2006 for four primary reasons: delays in interceptor deliveries caused by an explosion at a a halt in production after several flight test failures and pending Mission Readiness Task Force (MRTF) reviews, a MRTF review that redirected some interceptors from fielding to testing, and the temporary suspension of fielding interceptors due to manufacturing and quality issues associated with the exoatomospheric kill vehicle (EKV).

MDA also delayed a partial upgrade to the Thule early warning radar until a full upgrade can be accomplished. According to a July 11, 2005, DOD memorandum, the full upgrade of Thule is the most economical option and it meets DOD’s desire to retain a single configuration of upgraded early warning radars.

Additionally, the delivery of Aegis BMD Standard Missile -3 (SM-3) was reduced as technical challenges associated with the Divert Attitude Control System were addressed and as investments in upgrades were made to keep pace with emerging ballistic missile threats. According to Aegis BMD officials, the program also revised the upgrade schedule for engagement destroyers because other priorities prevent the Navy from making one ship available before the end of the block.

Budget cuts to the C2BMC program also caused MDA to defer the installation of C2BMC suites at three sites. MDA had planned to install the suites at U.S. Central Command, European Command, and another site that was to be determined before the end of the block. However, MDA now plans to place less expensive Web browsers at these sites.

MDA made progress in fielding additional BMDS assets in 2006 and is generally on track to meet most of its revised block goals. MDA’s delivery schedules and System Element Review reports show that MDA planned to accomplish these goals by making the following progress by December 31, 2006: adding 4 Aegis BMD missiles to inventory, adding 2 new Aegis BMD destroyers for long-range surveillance and tracking, upgrading 2 Aegis BMD destroyers and 2 Aegis BMD cruisers to perform both engagement and long-range surveillance and tracking, adding 1 new Aegis BMD destroyer and 1 new cruiser with both engagement and long-range surveillance and tracking capability, completing a number of activities prior to delivering the FBX-T radar, delivering the hardware for the 3 Web browsers, and emplacing 8 GMD interceptors. With the exception of the GMD interceptors, MDA completed all work as planned. The GMD program was only able to emplace four interceptors by December 2006, rather than the eight planned. However, program officials told us that the contractor has increased the number of shifts that it is working and the program believes that with this change the contractor can accelerate deliveries and emplace as many as 24 interceptors by the end of the block. However, to do so, the GMD program will have to more than double its 2007 interceptor emplacement rate.

MDA Likely to Defer Additional Work into Future Blocks to Offset Rising Costs

Even though MDA reduced the quantity of assets it planned to deliver during Block 2006 to free up funds, most of the MDA’s prime contractors overran their fiscal year 2006 budgets. Collectively, the prime contractors developing elements included in Block 2006 exceeded their budgets by approximately $478 million, with GMD accounting for about 72 percent of the overrun. Table 6 contains our analysis of prime contractors’ cost and schedule performance in fiscal year 2006 and the potential overrun or underrun of each contract at completion. All estimates of the contracts’ cost at completion are based on the contractors’ performance through fiscal year 2006. Appendix II provides further details regarding the cost and schedule performance of the prime contractors for the seven elements shown in table 6.

As shown in table 6, the Sensors element is the only Block 2006 element that according to our analysis performed within its fiscal year 2006 budget. The ABL, Aegis BMD, GMD, and STSS programs overran their fiscal year budgets as a result of technical problems and integration issues encountered during the year. We could not assess the C2BMC contractor’s cost and schedule performance because MDA suspended Earned Value reporting during the year as the contractor replanned its Block 2006 program of work.

In addition to analyzing the fiscal year 2006 cost and schedule performance of elements included in Block 2006, we also analyzed the performance of elements included in other blocks. Of the elements reporting Earned Value data, only KEI performed within its budget. THAAD’s integration problems once again caused it to exceed its budget. We were unable to determine whether the work accomplished by the MKV contractor cost more than originally planned because Contract Performance Reports were suspended in February 2006 as the program transitioned from an advanced technology development program to a system development program. This transition prompted MKV to establish a new baseline for the program, which the contractor will not report against until early in fiscal year 2007.

MDA officials told us that MDA is likely to defer some Block 2006 work activities (other than the delivery of assets) into future blocks in an effort to operate within the funds programmed for the block. If the agency reports the cost of deferred work as it has in the past, the cost of Block 2006 will not include all work that benefits the block and the cost of the future block will be overstated.

The deferral of work, while necessary to offset increased costs, complicates making a comparison of a block’s actual costs with its original estimate. According to the Statement of Federal Financial Accounting Standards Number 4, a federal program should report the full cost of its outputs, which is defined as the total amount of resources used to produce the output. In March 2006, we reported that the cost of MDA’s Block 2004 program of work was understated because the reported costs for the block did not include the cost of Block 2004 activities that were deferred until Block 2006. Conversely, the cost of Block 2006 is overstated because the deferred activities from Block 2004 do not directly contribute to the output of Block 2006. Similarly, if MDA decides to defer Block 2006 activities until Block 2008 as officials in MDA’s Office of Agency Operations told us is likely, the cost of those activities will likely be captured as part of Block 2008 costs.

MDA Able to Achieve Most Test Objectives Despite Some Delays in Test Schedule

Most BMDS elements achieved their primary calendar year 2006 test objectives and conducted test activities on schedule. By December 2006– the midpoint of Block 2006–three of the six Block 2006 elements and all elements considered part of future blocks—met their 2006 primary test objectives. Only the ABL, Aegis BMD, and STSS elements were unable to achieve these objectives.

Progress Made on Block 2006 Elements

Although the elements encountered test delays, some were able to achieve noteworthy accomplishments. For example, in its third flight test, the GMD program exceeded its test objectives by intercepting a target. This intercept was particularly noteworthy because it was the first successful intercept attempt for the program since 2002. Also, although the test was for only one engagement scenario, it was notable because it was GMD’s first end-to-end test.

The GMD program originally planned to conduct four major flight tests, during fiscal year 2006, two using operational interceptors. However, the program was only able to conduct three flight tests during the fiscal year. In one, an operational interceptor was launched against a simulated target; in a second test, a simulated target was launched to demonstrate the ability of the Beale radar to provide a weapon system task plan; and in the other, an interceptor was launched against an actual target. It was in the third test that—for one end-to-end scenario—the program exceeded test objectives by destroying a target representative of a real world threat. The objectives of the fourth test were to be similar to those of the third test— an interceptor flying-by a target with no expectation of a hit. However, program officials told us that the success of the earlier tests caused them to accelerate the objectives of the fourth test by making it an intercept attempt. The fourth test has not yet taken place because a delay in the third test caused a similar delay in the fourth test and because components of the test interceptor are being changed to ensure that they will function reliably. This test is currently scheduled no earlier than the third quarter of fiscal year 2007.

Both the C2BMC and Sensors elements conducted all planned test activities on schedule and were able to meet their 2006 objectives. The C2BMC software, which enables the system to display real-time target information collected by BMDS sensors, was tested in several flight tests with the Aegis BMD and GMD programs and was generally successful. The Sensors element was also able to complete all tests planned to ensure that the Forward-Based X-Band— Transportable (FBX-T) radar will be ready for operations. The warfighter will determine when the FBX-T will become operational, but MDA officials told us that this may not occur until the United States is able to provide the radar’s data to Japan.

MDA was unable to successfully execute the 2006 test objectives for the STSS program. Thermal vacuum testing that was to be conducted after the first payload was integrated with space vehicle 1 was delayed as a result of integration problems. According to program officials, testing began in January 2007 and it was expected to be completed in late February 2007.

Although the Aegis BMD program conducted its planned test activities on schedule, it was unable to achieve all of its test objectives for 2006. Since the beginning of Block 2006, the program has conducted one successful intercept, which tested the new Standard Missile-3 design that is being fielded for the first time during Block 2006. This new missile design provides a capability against more difficult threats and has a longer service life than the missile produced in Block 2004. In December 2006, a second intercept attempt failed because the weapon system component was incorrectly configured and did not classify the target as a threat, which prevented the interceptor from launching. Had this test been successful, it would have been the first time that the pulse mode of the missile’s Solid Divert and Attitude Control System would have been partially flight tested.

A sixth BMDS element–ABL–experienced delays in its testing schedule and was also unable to achieve its fiscal year 2006 test objectives. ABL is an important element because if it works as desired, it will defeat enemy missiles soon after launch, before decoys are released to confuse other BMDS elements. Development of the element began in 1996, but MDA has not yet demonstrated that all of ABL’s leading-edge technologies will work. The ABL program plans to prove critical technologies during a lethality demonstration. This demonstration is a key knowledge point for ABL because it is the point at which MDA will decide the program’s future.

However, technical problems encountered with the element’s Beam Control/Fire Control component caused the program to experience over a 3-month delay in its ground test program, which has delayed the planned lethality demonstration until 2009. In addition, all software problems have not been completely resolved and, according to ABL’s Program Manager, will have to be corrected before flight testing can begin, which could further delay the lethality demonstration.

Test Results for Future Block Elements Are Positive

The KEI element also has a key decision point—a booster flight test— within the next few years. In preparation for this test, the program successfully conducted static fire tests and wind tunnel tests in fiscal year 2006 to better assess booster performance. Upon completion of KEI’s 2008 flight test and ABL’s 2009 lethalithy demonstration, MDA will compare the progress of the two programs and decide their futures.

In January 2005, MDA established ABL as the primary boost phase defense element. At the same time, MDA restructured the KEI program to develop an upgraded long-range midcourse interceptor and reduced KEI’s role in the boost phase to that of risk mitigation. A KEI official told us that a proposal is being developed that suggests MDA approach the 2009 decision as a down select or source selection that would decide whether ABL or KEI would be the BMDS boost phase capability.

The MKV program accomplished all of its planned activities as scheduled during fiscal year 2006, which included several successful propulsion tests. In November 2005, the program tested a preliminary design of MKV’s liquid propellant divert and attitude control system–the steering mechanism for the carrier and kill vehicles. This test was a precursor to a successful July 2006 test of the liquid divert and attitude control system’s divert thruster, which was conducted under more realistic conditions. The program also executed a solid propellant divert and attitude control system test in December 2005. Results of the December test, combined with a technology assessment, led program officials to pursue a low-risk, high-performance liquid fueled divert and attitude control system. The MKV program will continue to explore other divert and attitude control system technologies for future use.

The THAAD program achieved its primary fiscal year 2006 test objectives, although it did experience test delays. The program planned to conduct five flight tests during fiscal year 2006, but was only able to execute four. During the program’s first two flight tests, program officials demonstrated the missile’s performance, including the operation of the missile’s divert and attitude control system and the control of its kill vehicle. The third flight test conducted in July 2006 demonstrated THAAD’s ability to successfully locate and intercept a target, a primary 2006 test objective. The fourth THAAD flight test was declared a “no-test” after the target malfunctioned shortly after its launch, forcing program officials to terminate the test. THAAD officials told us that the aborted test will be deleted from the test schedule and any objectives of the test that have not been satisfied will be rolled-up into future tests. The program planned to conduct its fifth (missile only) flight test–to demonstrate the missile’s performance in the low atmosphere–in December 2006. However, due to reprioritization in test flights, the fifth flight test is now scheduled for the second quarter of fiscal year 2007. Flight test 6, the next scheduled flight test, was successfully conducted at the end of January 2007. It was the first flight test performed at the Pacific Missile Range.

Overall BMDS Performance Remains Unverified

In March 2005, MDA set performance goals for Block 2006 that included a numerical goal for the probability of a successful BMDS engagement, a defined area from which the BMDS would prevent an enemy from launching a ballistic missile, and a defined area that the BMDS would protect from ballistic missile attacks. In March 2006, MDA altered its Block 2006 performance goals commensurate with reductions in Block 2006 fielded assets. Although MDA revised its goal downward, insufficient data exists to assess whether MDA is on track to meet its new goal.

MDA uses the WILMA model to predict overall BMDS performance even though this model has not been validated or verified by DOD’s Operational Test Agency. According to Operational Test Agency officials, WILMA is a legacy model that does not have sufficient fidelity for BMDS performance analysis. MDA officials told us the agency is working to develop an improved model that can be matured as the system matures.

In addition, the GMD program has not completed sufficient flight testing to provide a high level of confidence that the BMDS can reliably intercept ICBMs. In September 2006, the GMD program completed an end-to-end test of one engagement sequence that the GMD element might carry out. While this test provided some assurance that the element will work as intended, the program must test other engagement sequences, which would include other GMD assets that have not yet participated in an end- to-end flight test. Additionally, independent test agencies told us that additional flight tests are needed to have a high level of confidence that GMD can repeatedly intercept incoming ICBMs. Additional tests are also needed to demonstrate that the GMD element can use long-range surveillance and tracking data developed by the Aegis BMD element. In March 2006, we reported that Aegis BMD was unable to participate in a GMD flight test, which prevented MDA from exercising Aegis BMD’s long- range surveillance and tracking capability in a manner consistent with an actual defensive mission. The program office told us that the Aegis BMD is capable of performing this function and has demonstrated its ability to surveil and track ICBMs in several exercises. Additionally, Aegis BMD has shown that it can communicate this data to GMD in real time. However, because of other testing priorities, GMD has not used this data to prepare a weapon system task plan in real time. Rather GMD developed the plan in post-test activities. Officials in the Office of the Director for Operational Test and Evaluation told us that having GMD prepare the test plan in real time would provide the data needed to more accurately gauge BMDS performance.

Delayed testing and technical problems may also impact the performance of the system and the timeliness of future enhancements to the fielded system. For example, the performance of the new configuration of the Aegis BMD SM-3 missile is unproven because design changes in the missile’s solid divert and attitude control system and one burn pattern of the third stage rocket motor, according to program officials, were not flight tested before they were cut into the production line. MDA is considering a full flight test of the pulsed solid divert and attitude control system during the third quarter of fiscal year 2007. The solid divert and attitude control system is needed to increase the missile’s ability to divert into its designated target and counter more complex threats. The zero pulse-mode of the missile’s third stage rocket motor, which is expected to provide a capability against a limited set of threat scenarios, will not be fully tested until fiscal year 2009.

Confidence in the performance of the BMDS is also reduced because the GMD element continues to struggle with technical issues affecting the reliability of some GMD interceptors. For example, GMD officials told us that the element has experienced one anomaly during each of its flight tests since its first flight test conducted in 1999. This anomaly has not yet prevented the program from achieving any of its primary test objectives; but, to date, neither its source nor solution has been clearly identified or defined. Program officials plan to continue their assessment of current and future test data to identify the root cause of the problem.

The reliability of emplaced GMD interceptors also remains uncertain because inadequate mission assurance/quality control procedures may have allowed less reliable or inappropriate parts to be incorporated into the manufacturing process. Program officials plan to replace these parts in the manufacturing process, but not until interceptor 18. The program plans to begin retrofitting the previous 17 interceptors in fiscal year 2009. According to GMD officials, the cost of retrofitting the interceptors will be at least $65.5 million and could be more if replacement of some parts proves more difficult than initially expected.

The ABL program also experienced a number of technical problems during fiscal year 2006 that delayed future decisions for the BMDS program. As previously noted, the program’s 2008 lethality demonstration will be delayed until 2009. The delay is caused by Beam Control/Fire Control (BC/FC) software, integration, and testing difficulties and unexpected hardware failures. According to contractor reports, additional software tests were needed because changes were made to the tested versions, the software included basic logic errors, and unanticipated problems were caused by differences in the software development laboratory and ABL aircraft environments.

MDA’S Flexibility Comes at the Cost of Program Transparency

MDA enjoys a significant amount of flexibility in developing the BMDS, but it comes at the cost of transparency and accountability. Because the BMDS program has not formally entered the system development and demonstration phase of the acquisition cycle, it is not yet required to apply several important oversight mechanisms contained in certain acquisition laws that, among other things, provide transparency into program progress and decisions. This has enabled MDA to be agile in decision making and to field an initial BMDS capability quickly. On the other hand, MDA operates with considerable autonomy to change goals and plans, making it difficult to reconcile outcomes with original expectations and to determine the actual cost of each block and of individual operational assets.

Acquisition Laws Promote Program Transparency

Past Congresses have established a framework of laws that make major defense acquisition programs accountable for their planned outcomes and cost, give decision makers a means to conduct oversight, and ensure some level of independent program review. The threshold application of these acquisition laws is typically triggered by a program’s entry into system development and demonstration—a phase during which the weapon system is designed and then demonstrated in tests. The BMDS has not entered into system development and demonstration because it is being developed outside DOD’s normal acquisition cycle.

To provide accountability, major defense acquisition programs are required by statute to document program goals in an acquisition program baseline that, as implemented by DOD, has been approved by a higher- level DOD official prior to the program’s initiation. The baseline, derived from the users’ best estimates of cost, schedule, and performance requirements, provides decision makers with the program’s total cost for an increment of work, average unit costs for assets to be delivered, the date that an initial operational capability will be fielded, and the weapon’s intended performance parameters. The baseline is considered the program’s initial business case–evidence that the concept of the program can be developed and produced within existing resources. Once approved, major acquisition programs are required to measure their program against the baseline or to obtain approval from a higher-level acquisition executive before making significant changes. Programs are also required to regularly provide detailed program status information to Congress, including information on program cost, in Selected Acquisition Reports. In addition, Congress has established a cost monitoring mechanism that requires programs to report significant increases in unit cost measured from the program baseline.

Other statutes ensure that DOD provides some independent program verification external to the program. Title 10, United States Code (U.S.C.), section 2434 prohibits the Secretary of Defense from approving system development and demonstration, or production and deployment, of a major defense acquisition program unless an independent estimate of the program’s full life-cycle cost has been considered by the Secretary. The independent verification of a program’s cost estimate allows decision makers to gauge whether the program is executable given other budget demands and it increases the likelihood that a program can execute its plan within estimated costs. In addition, 10 U.S.C. § 2399 requires completion of initial operational test and evaluation of a weapon system before a program can begin full-rate production. The Director of Operational Test and Evaluation, a DOD office independent of the acquisition program, not only approves the adequacy of the test plan and its subsequent evaluation, but also reports to the Secretary of Defense whether the test and evaluation were adequate and whether the test’s results confirm that the items are effective and suitable for combat.

By law, appropriations are to be applied only to the objects for which the appropriations were made except as otherwise provided by law. Research and development appropriations are typically specified by Congress to be used to pay the expenses of basic and applied scientific research, development, test, and evaluation. On the other hand, procurement appropriations are, in general, specified by Congress to be used for the purchase of weapon systems and equipment, that is, production or manufacturing. In the 1950s, Congress established a policy that items being purchased with procurement funds be fully funded in the year that the item is procured. This policy is meant to prevent a program from incrementally funding the purchase of operational systems.

According to the Congressional Research Service, “incremental funding fell out of favor because opponents believed it could make the total procurement costs of weapons and equipment more difficult for Congress to understand and track, create a potential for DOD to start procurement of an item without necessarily stating its total cost to Congress, permit one Congress to ‘tie the hands’ of future Congresses, and increase weapon procurement costs by exposing weapons under construction to uneconomic start-up and stop costs.”

Congress continues to enact legislation that improves program transparency. In 2006, Congress added 10 U.S.C. § 2366a, which prohibits programs from entering system development and demonstration until certain certifications are made. For example, the decision authority for the program must certify that the program has a high likelihood of accomplishing its intended mission and that the program is affordable considering unit cost, total acquisition cost, and the resources available during the year’s covered by DOD’s future years defense program.

Similar to other government programs, one of the laws affecting MDA decisions is the Antideficiency Act. The fundamental concept of the Antideficiency Act is to ensure that spending does not exceed appropriated funds. The act is one of the major laws in which Congress exercises its constitutional control of the public purse. The fiscal principles underlying the Antideficiency Act are quite simple. Government officials may not make payments, or commit the United States to make payments at some future time, for goods or services unless the available appropriation is sufficient to cover the cost in full. To ensure that it is always in compliance with this law, MDA adjusts its goals and defers work as needed to execute the BMDS within its available budget.

Benefits of MDA’s Flexibility

In 2001, DOD conducted extensive missile defense reviews to decide how best to defend the United States, deployed troops, friends, and allies from ballistic missile attacks. The studies determined that DOD needed to find new approaches to acquire and deploy missile defenses. Flexibility was one of the hallmarks of the new approach that DOD chose to implement. One flexibility accorded MDA was the authority to develop the BMDS outside of DOD’s normal acquisition cycle, by not formally entering the system development and demonstration phase. This effectively enabled MDA to defer application of certain acquisition laws until the agency transfers a fully developed capability to a military service for production, operation, and sustainment—the point at which DOD directed that the BMDS program reenter the acquisition cycle. At that point, basic development and initial fielding would generally be complete.

Because MDA currently does not have to apply many of the oversight requirements for major defense acquisition programs directed by acquisition laws, the BMDS program operates with unusual autonomy. In 2002, the Under Secretary of Defense for Acquisition, Technology, and Logistics delegated to MDA the authority to establish its own baseline and make changes to that baseline without approval outside of MDA. Because it has not formally entered system development and demonstration, MDA can also initiate a block of capability and move forward with its fielding without an independent cost estimate or an independent test of the effectiveness and suitability of assets intended for operational use. The ability to make decisions on its own and proceed without independent verifications reduces decision timelines, making the BMDS program more agile than other DOD programs.

MDA’s ability to quickly field a missile defense capability is also enhanced by its ability to field the BMDS before all testing is complete. MDA considers the assets it has fielded to be developmental assets and not the result of the production phase of the acquisition cycle. Because MDA has not advanced the BMDS or its elements into the acquisition cycle, it is continuing to produce and field assets without completing the operational test and evaluation normally required by 10 U.S.C. § 2399 before full-rate production. For example, MDA has acquired and emplaced 14 ground- based interceptors for operational use before both developmental and operational testing is completed. The agency’s strategy is to continue developmental testing while fielding assets and to also incorporate operational realism into these tests so that the Director of Operational Test and Evaluation can make an operational assessment of the fielded assets’ capability.

Because all of MDA’s funding comes from the Research, Development, Test, and Evaluation appropriation account, MDA enjoys greater flexibility in how it can use funds compared to a traditional DOD acquisition program where funding is typically divided into research, development, and evaluation, procurement, and operations and maintenance. This is particularly true of an element. For example, a Block 2006 element like GMD covers a wide range of activities, from research and development on future enhancements to the fabrication of interceptors for operations. If the GMD program runs into problems with one activity, it can defer work on another to cover the cost of the problems. MDA’s flexibility to change goals for each element complements the flexibility in how it uses its funds.

Comparing Actual Work to Budgeted Work Is Difficult

After a new block of the BMDS has been presented in the budget, MDA can change the outcomes–in terms of planned delivery of assets and other work activities–that are expected of the block. While this freedom enables MDA to operate within its budget, it decouples the activities actually completed from the activities that were budgeted, making it difficult to assess the value of what is actually accomplished. For example, between 2003 and mid-2005, MDA changed its Block 2004 delivery goals three times, progressively decreasing the number of assets planned for the block when it was initially approved for funding. This trend has continued into Block 2006, with the agency changing its delivery plans once since it presented its initial Block 2006 goals to Congress. MDA is required to report such changes only if MDA’s Director considers the changes significant.

In addition to deferring the delivery of assets from one block to another, MDA also has the flexibility to defer other work activities from a current to a future block. This creates a rolling scope, making it difficult to keep track of what an individual block is responsible for delivering. For example, during Block 2004, MDA deferred some planned development, deployment, characterization, and verification activities until Block 2006 so that it could cover contractor budget overruns. MDA is unable to determine exactly how much work was deferred. However, according to a November 2006 report to Congress, MDA found it necessary to defer the work until Block 2006 to make Block 2004 funding available to implement a new GMD test strategy following two GMD flight test failures, resolve quality issues associated with GMD interceptors and its exoatmospheric kill vehicle, and add an FBX-T radar to the initial deployed capability. Agency officials are already anticipating the deferral of work from Block 2006 into Block 2008. In fiscal year 2006, the work of five of the six contractors responsible for elements included in Block 2006 cost more than expected. Given program funding limits, MDA officials told us that they will either have to defer work or request additional funds from Congress during the remaining years of the block. MDA did not increase its fiscal year 2007 budget request; therefore, it is likely that the agency will once again have to defer some planned work into the next block.

Not only do changes in a block’s work plan make it difficult to know what outcomes the program expects to achieve, the changes also have the potential to impact the BMDS’ performance. For example, by decreasing the number of fielded interceptors, MDA decreases the likelihood that it can defeat enemy missiles if multiple threats are prevalent because the number of available interceptors will be limited. In addition, if activities, such as testing and validation, are not complete when assets are fielded, the assets may not perform as expected and changes may be needed. This effect of early fielding was seen in Block 2004 when GMD interceptors were fielded before testing was complete. Later tests showed that the interceptors may contain unreliable parts, some of which MDA now plans to replace.

Lack of Independent Reviews Contributes to Changing Goals

Although acquisition laws governing major defense acquisition programs as well as DOD acquisition policy recognize the need for independent program reviews, few such reviews are part of the BMDS program. This has contributed to the difficulty in assessing MDA’s progress toward expected outcomes. As described above, major programs are required by law to have an independent cost estimate (performed by the DOD Cost Analysis Improvement Group) for entry into system development and demonstration, as well as production and deployment. According to MDA officials, MDA has so far obtained an independent assessment of only one BMDS element’s life-cycle cost estimate—Aegis BMD’s estimate for Block 2004. In our opinion, without a full independent cost estimate, MDA has established optimistic block goals that could not be met. This is supported by an MDA spokesman’s statement that the agency’s optimism in establishing Block 2004 cost and quantity goals contributed to several goal changes. According to MDA officials, unlike its action on its Block 2004 cost goal, MDA did not request an assessment of MDA’s Block 2006 goal.

Further, DOD policy calls for a milestone decision authority with overall responsibility for the program that is independent of the program. Although the Director reports to the Under Secretary of Defense for Acquisition, Technology, and Logistics and keeps the Under Secretary and congressional defense committees informed of MDA decisions, MDA’s Director is authorized to make most program decisions without prior approval from a higher-level authority. The Under Secretary of Defense delegated this authority to the Director in a February 2002 memorandum. The Secretary of Defense also appointed MDA’s Director as both the BMDS Program Manager and its Acquisition Executive (including the authority to serve as milestone decision authority until an element is transferred out of MDA). As the Acquisition Executive, the Director was given responsibility for establishing programmatic policy and conducting all research and development of the BMDS. This delegation included responsibility for formulating BMDS acquisition strategy, making program commitments and terminations, deciding on affordability trade-offs, and baselining the capability and configuration of blocks and elements.

Block Costs Cannot Be Tracked

Because MDA can redefine outcomes, the actual cost of a block cannot be compared with the cost originally estimated. MDA considers the cost of deferred work—which may be the delayed delivery of assets or other work activities—as a cost of the block in which the work is performed, even though the work benefits and was planned for a prior block. Further, MDA does not track the cost of deferred work from one block to the next and, therefore, cannot make adjustments that would match the cost with the block it benefits. For example, in March 2006, we reported that MDA deferred some Block 2004 work until Block 2006 so that it could use the funds appropriated for that work to cover unexpected cost increases caused by such problems as poor quality control procedures and technical problems during development, testing, and production. MDA officials told us that additional funds have been, or will be, requested during Block 2006 to carry out the work. However, the officials could not tell us how much of the Block 2006 budget is attributable to the deferred work. These actions caused Block 2004 cost to be understated and Block 2006 cost to be overstated. In addition, if MDA delays some Block 2006 work until Block 2008, as expected, Block 2006 cost will become more difficult to compare with its original estimate as the cost of the deferred work will no longer count against the block. The Director, MDA, determines whether he reports the cost of work being deferred to future blocks and, so far, has not done so.

The planned and actual unit costs of assets being acquired for operational use are equally hard to determine. Because the BMDS and its elements are a single major defense acquisition program that has not officially entered into system development and demonstration, it is not required to provide the detailed reports to Congress directed by statute. While it is possible to reconstruct planned unit costs from budget documents, the planned unit cost of some assets—for example, GMD interceptors—is not easy to determine because the research and development funds used to buy the interceptors are spread across 3 to 5 budget years. Also, because MDA is not required to report significant increases in unit cost, it is not easy to determine whether an asset’s actual cost has increased significantly from its expected cost. For example, we were unable to compare the actual and planned cost of a GMD interceptor. By comparison, the Navy provides more transparency in reporting on the cost of ships, some of which are incrementally funded with procurement funds. When a Navy ship program overruns the cost estimate used to justify the budget, the Navy identifies the additional funding needed to cover the overruns separately from other shipbuilding programs.

Using research and development funds to purchase fielded assets further reduces cost transparency because these dollars are not covered by the full-funding policy for procurement. Therefore, when the program for a 2- year block is first presented in the budget, Congress is not necessarily fully aware of the dimensions and cost of that block. Although a particular block may call for the delivery of a specific number of interceptors, the full cost of those interceptors may not be contained in that block. In addition, incremental funding has the potential to “tie the hands” of future Congresses to finish funding for assets started in prior years. Otherwise, Congress could run the risk of a production stoppage and the increased costs associated with restarting the production line.

MDA Makes Significant Strides with Quality Improvement Processes

During Block 2004, poor quality control procedures that MDA officials attribute to acquisition streamlining and schedule pressures caused the missile defense program to experience test failures and slowed production. MDA has initiated a number of actions to correct its quality control weaknesses and those actions have been largely successful. Although MDA continues to identify quality control procedures that need improvement, the number of deficiencies has declined and contractors are responding to MDA’s improvement efforts. These efforts include a teaming approach designed to restore the reliability of MDA’s suppliers, regular quality inspections to quickly identify and find resolutions for quality problems, and award fees with an increased emphasis on quality assurance. In addition, MDA’s attempts to improve quality assurance have attracted the interest of other government agencies and contractors. MDA is leading quality improvement conferences and co-sponsoring a Space Quality Improvement Council.

Officials in MDA’s Office of Quality, Safety, and Mission Assurance and in GMD’s Program Office attribute the weaknesses in MDA’s quality control processes to acquisition streamlining and schedule pressures. According to a former DOD Director of Operational Test and Evaluation, during the early 1990’s there was a common goal for DOD management to streamline the acquisition process to reduce burgeoning costs of new weapons. By streamlining the process, DOD commissions and task forces hoped to drastically cut system development and production time and reduce costs by eliminating management layers, eliminating certain reporting requirements, using more commercial-off-the-shelf systems and subsystems, reducing oversight from within as well as from outside DOD, and by eliminating perceived duplication of testing. In addition to acquisition streamlining, schedule pressures caused MDA to be less attentive to quality assurance issues. This was particularly true for the GMD element that was tasked with completing development and producing assets for operational use within 2 years of a Presidential directive to begin fielding an initial missile defense capability. While the GMD program had realized for some time that its quality controls needed to be strengthened, the program’s accelerated schedule left little time to address quality problems.

MDA has initiated a number of mechanisms to rectify the quality control weaknesses identified in the BMDS program. For example, as early as 2003, MDA, in concert with industry partners, Boeing, Lockheed Martin, Raytheon, and Orbital Sciences began a teaming approach to restore reliability in a key supplier. In exchange for allowing the supplier to report to a single customer—MDA—the supplier gave MDA’s Office of Quality, Safety, and Mission Assurance authority to make a critical assessment of the supplier’s processes. This assessment determined that the supplier’s manufacturing processes lacked discipline, its corrective action procedures were ineffective, its technical data package was inadequate, and personnel were not properly trained. The supplier responded by hiring a Quality Assurance Director, five quality assurance professionals, a training manager, and a scheduler. In addition, the supplier installed an electronic problem reporting database, formed new boards—such as a failure review board—established a new configuration management system, and ensured that manufacturing activity was consistent with contract requirements. According to MDA, by 2005, these changes began to produce results. Between March 2004 and September 2005, test failures declined by 43 percent. In addition, open quality control issues decreased by 64 percent between September 2005 and August 2006 and on-time deliveries increased by 9 percent between March 2005 and August 2006. MDA’s teaming approach was expanded in 2006 to another problem supplier and many systemic solutions are already underway.

MDA also continues to carry-out regular contractor quality inspections. For example, during fiscal year 2006, MDA completed quality audits of 6 contractors and identified a total of 372 deficiencies and observations. As of December 2006, the contractors had closed 157 or 42 percent of all audit findings. These audits are also producing other signs of quality assurance improvements. For example, after an August 2006 review of Raytheon’s production of the last five GMD exoatmospheric kill vehicles, MDA auditors reported less variability in Raytheon’s production processes, increasing stability in its statistical process control data, fewer test problem reports and product waivers, compliance with manufacturing “clean room” requirements, and a sustained improvement in product quality. Because of the emphasis placed on the recognition of quality problems, Raytheon is conducting regular inspections independently of MDA to identify problems.

Over the course of 2006, MDA also continued to incorporate MDA Assurance Provisions (MAP) into its prime contracts. The MAP provides MDA methods to measure, verify, and validate mission success through the collection of metrics, risk assessment, technical evaluations, independent assessments, and reviews. Four BMDS elements–BMDS Sensors, C2BMC, KEI, and THAAD–modified their contracts during 2006 to incorporate the MAP. The remaining five BMDS elements have not yet included the plan on their contracts because the contract is mostly in compliance with the MAP or because of the timing and additional costs of adding the requirements.

MDA also encourages better quality assurance programs and contractors’ implementation of best practices through award fee plans. In 2003, three BMDS elements–BMDS Sensors, KEI, and THAAD–revised their contracts to include 25 MAP criteria in their award fee plans. For example, the BMDS Sensors element included system quality, reliability, and configuration control of data products as part of its award fee criteria for its FBX-T contract. Contractors are also bringing their best practices to the table. For example, in an effort to prevent foreign object debris in components under assembly, Raytheon and Orbital Sciences have placed all tools in special tool boxes known as shadow boxes. Raytheon has also incorporated equipment into the production process that handles critical components, removing the possibility that the components will be dropped or mishandled by production personnel.

Because of its quality assurance efforts, contractors and other government agencies have called on MDA to lead quality conferences and sponsor an improvement council. MDA’s Office of Quality, Safety, and Mission Assurance was co-sponsor of a conference on quality in the space and defense industry and the office’s Director has also served as panel discussion chair at numerous other conferences. The conferences focus on the safety, reliability, and quality aspects of all industries and agencies involved in defense and space exploration. MDA is also a co-sponsor of the Space Quality Improvement Council, a council established to cooperatively address critical issues in the development, acquisition, and deployment of national security space systems. Contractors are also adopting some MDA methods for improving quality assurance. For example, Raytheon Integrated Defense Systems has adopted the MAP as a performance standard for all of its defense programs.

Conclusions

In a general sense, our assessment of MDA’s progress on missile defense is similar to that of previous years: accomplishments have been made and capability has been increased, but costs have grown and the scope of planned work has been reduced. The fielding of additional assets, the ability to put BMDS on alert status, and the first end-to-end test of GMD were notable accomplishments during fiscal year 2006. On the other hand, it is not easy to answer the question of how well BMDS is progressing relative to the funds it has received and goals it has set for those funds.

As with previous years, we have found it difficult to reconcile the progress made in Block 2006 with the original cost and scope of the program. The block concept, while a useful construct for harvesting and fielding capability incrementally, is a muddy construct for accountability. Although BMDS is managed within a relatively level budget of about $10 billion a year, the scope of planned work is altered several times each year. Consequently, work travels from one block to another, weakening the connection between the actual cost and scope of work done and the estimated cost and scope of work used to justify budget requests. Block 2006 is a case in point. Compared with its original budget justification, it now contains unanticipated work from Block 2004 but has deferred some of its own planned work to future blocks. Costs for the THAAD element are no longer being counted in Block 2006 although they were last year. Some developmental elements that will be fielded in later blocks, such as KEI and MKV, are not considered part of Block 2006, while ABL, which is also a developmental element to be fielded in later blocks, is considered part of Block 2006. Establishing planned and actual costs for individual assets is also elusive because MDA’s development of the BMDS outside of DOD’s acquisition cycle blurs the audit trail. Using research and development funds—funds that are not covered by the full-funding policy—contributes to the difficulty in determining some assets’ cost.

None of the foregoing is to suggest that MDA has acted inconsistently with the authorities it has been granted. Indeed, by virtue of its not having formally begun system development and demonstration, coupled with its authority to use research and development funds to manufacture and field assets, MDA has the sanctioned flexibility to manage exactly as it has. It could be argued that without this latitude, the initial capability fielded last year and put on alert this year would not have been possible. Yet, the question remains as to whether this degree of flexibility should be retained on a program that will spend about $10 billion a year for the foreseeable future. It does not seem unreasonable to expect a program of this magnitude to be held to a higher standard of accountability than delivering some capability within budgeted funds. In fact, the program is likely to undergo greater scrutiny as DOD faces increasing pressure to make funding trade-offs between its investment portfolios, ongoing military operations, and recapitalization of its current weapon systems.

Within BMDS, key decisions lie ahead for DOD. Perhaps the most significant decision in the next 2 years will be to determine what investments should be made in the two boost phase elements—ABL and KEI—under development. This decision would benefit greatly from good data on actual versus expected performance, actual versus expected cost, and independent assessments of both cost and performance.

The recommendations that follow build upon those we made in last year’s report on missile defense. In general, those recommendations called for the Secretary of Defense to align individual BMDS elements around a knowledge-based strategy and to determine whether a block approach to fielding was compatible with such a strategy.

Recommendations for Executive Action

To increase transparency in the missile defense program, we recommend that the Secretary of Defense: Develop a firm cost, schedule, and performance baseline for those elements considered far enough along to be in system development and demonstration, and report against that baseline.

Propose an approach for those same elements that provides information consistent with the acquisition laws that govern baselines and unit cost reporting, independent cost estimates, and operational test and evaluation for major DOD programs. Such an approach could provide necessary information while preserving the MDA Director’s flexibility to make decisions. Include in blocks only those elements that will field capabilities during the block period and develop a firm cost, schedule, and performance baseline for that block capability including the unit cost of its assets.

Request and use procurement funds, rather than research, development, test, and evaluation funds, to acquire fielded assets.

Conduct an independent evaluation of ABL and KEI after key demonstrations, now scheduled for 2008 and 2009, to inform decisions on the future of the two programs.

Agency Comments and Our Evaluation

DOD’s comments on our draft report are reprinted in appendix I. DOD partially concurred with our first three recommendations and non- concurred with the last two.

In partially concurring with the first recommendation, DOD recognized the need for greater program transparency, but objected to implementing an element-centric approach to reporting, believing that this would detract from managing the BMDS as a single integrated system. We agree that management of the BMDS as a single, integrated program should be preserved. However, since DOD already requests funding and awards contracts by the individual elements that compose the BMDS, we believe that establishing a baseline for those elements far enough along to be considered in system development and demonstration provides the best basis for transparency of actual performance. This would not change DOD’s approach to managing the BMDS, because merely reporting the cost and performance of individual elements would not cause each element to become a major defense acquisition program. DOD stated that MDA intends to modify its current biennial block approach that is used to define reporting baselines. In making this change, MDA states that it intends to work with both Congress and GAO to ensure that its new approach provides useful information for accountability purposes. At this point, we believe that the information needed to define a reporting baseline for a block would best be derived from individual elements. That having been said, a discourse can be had on whether elements are the only way to achieve the needed transparency and we welcome the opportunity to work toward constructive changes.

DOD also partially concurred with our second recommendation that BMDS elements effectively in system development and demonstration provide information consistent with the acquisition laws that govern baselines and unit cost reporting, independent cost estimates, and operational test and evaluation for major programs. DOD did commit to providing additional information to Congress to promote accountability, consistency, and transparency. Nonetheless, DOD remains concerned that having elements, rather than the BMD system, report according to these laws will have a fragmenting effect on the development of an integrated system and put more emphasis on individual programs as though each is a major defense acquisition program. We believe that greater transparency into the BMDS program depends on DOD reporting in the same manner that it requests program funding. This ensures that decision makers can reconcile the expected cost and performance of assets DOD plans to acquire with actual cost and performance. We recognize that MDA does provide Congress with information on cost and testing, but this information is not of the caliber or consistency called for by acquisition laws.

DOD stated that our third recommendation on reporting at the BMDS-level appears to be inconsistent with our recommendations on reporting at the element level. The basis for our third recommendation is that a block, which is a construct to describe and manage a defined BMDS-wide capability, must be derived from the capabilities that individual elements can yield. Except for activities like integrated tests that involve multiple elements, the cost, schedule, and performance of the individual assets to be delivered in a block come from the elements. Further, those elements that are not far enough along to deliver assets or capabilities within a particular block should not be considered part of that block. We believe that as MDA works to modify its current biennial block approach, it needs to be clearer and more consistent about what is and is not included in a block and that the cost, schedule, and performance of the specific assets in the block should be derived from the information already generated by the elements.

DOD did not concur with our recommendation that it request and use procurement funds to acquire fielded assets. It noted that the flexibility provided by Research, Development, Test, and Evaluation funding is necessary to develop and acquire new capabilities quickly that can respond to new and unexpected ballistic missile threats. We recognize the need to be able to respond to such threats. However, other DOD programs are also faced with unexpected threats that must be addressed quickly and have found ways to do so while acquiring operational assets with procurements funds. If MDA requires more flexibility than other programs, there should be a reasonable budgetary accommodation available other than funding the entire budget with Research, Development, Test, and Evaluation funds. More needs to be done to get a better balance between flexibility and transparency. Thus, we continue to believe that decision makers should be informed of the full cost of assets at the time DOD is asking for approval to acquire them and that procurement funds are the best way to provide that transparency.

DOD also did not concur with our fifth recommendation to conduct an independent evaluation of ABL and KEI to inform the upcoming decisions on these programs. It believes that MDA’s current integrated development and decision-making approach should continue as planned. We continue to believe that MDA would benefit from an independent evaluation of both ABL and KEI. However, we do believe such an evaluation should be based on the results of the key demonstrations planned for the elements in 2008 and 2009. We have modified our recommendation accordingly.

We are sending copies of this report to the Secretary of Defense and to the Director, MDA. We will make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you, or your staff, have any questions concerning this report, please contact me at (202) 512-4841. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report. The major contributors are listed in appendix IV.

Appendix I: Comments from the Department of Defense

Appendix II: MDA Contracts

Like other government agencies, MDA acquires the supplies and services needed to fulfill its mission by awarding contracts. Two types of contracts are prevalent at MDA—contracts for support services and contracts for hardware. The contractors that support MDA’s mission are commonly known as support contractors, while the contractors that are responsible for developing elements of the Ballistic Missile Defense System (BMDS) are called prime contractors.

Support Contractors Are Key to BMDS Development

According to MDA’s manpower database, about 8,186 personnel positions—not counting prime contractors—currently support the missile defense program. These positions are filled by government civilian and military employees, contract support employees, employees of federally funded research and development centers (FFRDC), researchers in university and affiliated research centers, and a small number of executives on loan from other organizations. At least 94 percent of the 8,186 positions are paid by MDA through its research and development appropriation. Of this 94 percent, only about 33 percent, or 2,578 positions, are set aside for government civilian personnel. Another 57 percent, or 4,368 positions, are support contractors that are supplied by 44 different defense companies. The remaining 10 percent are positions either being filled, or expected to be filled, by employees of FFRDCs and university and affiliated research centers that are on contract or under other types of agreements to perform missile defense tasks. Table 7 illustrates the job functions that contract employees carry out.

MDA officials explained that the utilization of support contractors is key to its operation of the BMDS because it allows the agency to obtain necessary personnel and develop weapon systems more quickly. Additionally, the officials told us that its approach is consistent with federal government policy on the use of contractors. MDA officials estimate that while the average cost of the agency’s government employee is about $140,000 per year, a contract employee costs about $175,000 per year. Table 8 highlights the staffing levels for each BMDS element.

Most Prime Contractors Did Not Execute All Planned Work within Fiscal Year 2006 Cost and Schedule Budgets

Prime contractors developing elements of the Ballistic Missile Defense System (BMDS) typically receive most of the funds MDA requests from Congress each fiscal year. The efforts of prime contractors may be obtained through a wide range of contract types. Because MDA is requiring its prime contractors to perform work that includes enough uncertainty that the cost of the work cannot be accurately estimated, all of the agency’s prime contracts are cost reimbursement arrangements. Under a cost reimbursement contract, a contractor is paid for reasonable, allowable, and allocable costs incurred in performing the work directed by the government to the extent provided in the contract. The contract includes an estimate of the work’s total cost for the purpose of obligating funds and establishes a ceiling cost that the contractor may not exceed without the approval of the contracting officer.

Many of the cost reimbursement contracts awarded by MDA include an award fee. Cost plus award fee contracts provide for a fee consisting of a base amount, which may be zero, that is fixed at the inception of the contract and an award amount, based upon a subjective evaluation by the government, that is meant to encourage exceptional performance. The amount of the award fee is determined by the government’s assessment of the contractor’s performance compared to criteria stated in the contract. This evaluation is conducted at stated intervals during performance, so that the contractor can be periodically informed of the quality of its performance and, if necessary, areas in which improvement are required.

Two of the cost reimbursement contracts shown in table 9—MKV and C2BMC—differ somewhat from other elements’ cost reimbursement contracts. The MKV prime contract is an indefinite delivery/indefinite quantity cost-reimbursement arrangement. This type of contract allows the government to direct work through a series of task orders. Such a contract does not procure or specify a firm quantity of services (other than a minimum or maximum quantity). This contracting approach permits MDA to order services as they are needed after requirements materialize and provides the government with flexibility because the tasks can be aligned commensurate with available funding. Since the MKV element is relatively new to the BMDS, its funding is less predictable than other elements’ and the ability to decrease or increase funding on the contract each year is important to effectively manage the program.

The C2BMC element operates under an Other Transaction Agreement that is not subject to many procurement laws and regulations. However, even though an Other Transaction Agreement is not required to include all of the standard terms and conditions meant to safeguard the government, the C2BMC agreement was written to include similar clauses and provisions. We found no evidence at this time that the C2BMC agreement does not adequately protect MDA’s interests. MDA chose the Other Transaction Agreement to facilitate a collaborative relationship between industry, government, federally funded research and development centers, and university research centers. Contract officials told us that a contract awarded under the Federal Acquisition Regulation is normally regarded as an arms-length transaction in which the government gives the contractor a task that the contractor performs autonomously. While an important purpose of an Other Transaction Agreement is to broaden DOD’s technology and industrial base by allowing the development and use of instruments that reduce barriers to participation in defense research by commercial firms that traditionally have not done business with the government, the agreements’ value in encouraging more collaborative environments is also recognized. Table 9 outlines the contractual instruments that MDA uses to procure the services of its prime contractors.

Most Prime Contractors Exceed Their Fiscal Year 2006 Budgets

Excluding the C2BMC and MKV elements, MDA budgeted approximately $3 billion for its prime contractors to execute planned work during fiscal year 2006. To determine if these contractors are executing the work planned within the funds and time budgeted, each BMDS program office requires its prime contractor to provide monthly reports detailing cost and schedule performance. In these reports, which are known as Contract Performance Reports, the prime contractor makes comparisons that inform the program as to whether the contractor is completing work at the cost budgeted and whether the work scheduled is being completed on time. If the contractor does not use all funds budgeted or completes more work than planned, the report shows positive cost and/or schedule variances. Similarly, if the contractor uses more money than planned or cannot complete all of the work scheduled, the report shows negative cost and/or schedule variances. A contractor can also have mixed performance. That is, the contractor may spend more money than planned (a negative cost variance) but complete more work than scheduled (a positive schedule variance). Using data from Contract Performance Reports, a program manager can assess trends in cost and schedule performance, information that is useful because trends tend to persist. Studies have shown that once a contract is 15 percent complete, performance metrics are indicative of the contract’s final outcome.

We used contract performance report data to assess the fiscal year 2006 cost and schedule performance of prime contractors for seven of the nine BMDS elements being developed by MDA. When possible, we also predicted the likely cost of each prime contract at completion. Our predictions of final contract cost are based on the assumption that the contractor will continue to perform in the future as it has in the past. An assessment of each element is provided below.

Aegis BMD Contractors End Fiscal Year 2006 Mostly within Cost and on Schedule

The Aegis BMD program has awarded a prime contract for each of its two major components—the Aegis BMD Weapon System and the Standard Missile-3. During fiscal year 2006, the work of both prime contractors cost a little more than expected, but only the weapon system contractor was slightly behind schedule.

Even though the weapon system contractor was unable to perform fiscal year 2006 work at the planned cost, its cumulative cost performance remains positive because of good performance in prior years. At year’s end, the weapon system contract had a cumulative favorable cost variance of $0.1 million, but an unfavorable cumulative schedule variance of $0.8 million. As shown in figure 1, the contractor’s cost and schedule performance fluctuated significantly throughout the year.

The decline in the Aegis BMD Weapon System contractor’s cost performance began shortly after the contractor adjusted its cost and schedule baseline in September 2005. At that time, the contractor corrected its baseline to account for a December 2004 DOD budget cut.

However, it did not make adjustments to the baseline to incorporate new work that the government directed. This caused the contractor’s cost performance to decline significantly because although the cost of the new effort was being reported, the baseline included no budget for the work. Recognizing that the contract baseline still needed to be replanned, the Director issued approval to restructure the program and rebaseline the contract in December 2005. To accommodate the work added to the contract, MDA and the contractor realigned software deliveries for Block 2006. The contractor completed the rebaselining effort in April 2006, and since then the contractor has performed within budgeted cost and schedule. Based on the contractor’s fiscal year 2006 cost performance, we estimate that at completion the contract may cost from $0.1 to $4.7 million more than anticipated.

Aegis BMD SM-3 Contractor Overruns Cost Budget, but Is Ahead of Schedule

For fiscal year 2006, the Standard Missile-3 contractor incurred an unfavorable cost variance of $7.8 million and a favorable schedule variance of $0.7 million. Even though the contractor was unable to complete fiscal year 2006 work within the funds budgeted, it ended the year with a cumulative positive cost variance of $3.1 million. The cumulative positive cost variance was the result of the contractor performing 2005 work at $10.9 million less than budgeted. In addition, although the contractor performed work ahead of schedule in fiscal year 2006, it was unable to overcome a negative schedule variance of $9.6 million created in 2005 caused by delayed hardware deliveries and delayed test events. The contractor ended fiscal year 2006 with a cumulative $8.9 million negative schedule variance. Figure 2 shows cumulative variances at the beginning of fiscal year 2006 year along with a depiction of the contractor’s cost and schedule performance throughout the fiscal year.

The unfavorable cost variance for fiscal year 2006 was caused by performance issues associated with the third stage rocket motor, the kinetic warhead and the missile’s guidance system. In addition, production costs associated with the Solid Divert and Attitude Control System were higher than anticipated. If the contractor continues to perform as it did in fiscal year 2006, we estimate that at completion the contract could cost from $1.9 million less than expected to $2.7 million more than expected.

ABL Continues to Experience Cost and Schedule Growth

Our analysis of ABL’s Contract Performance Reports indicates that the prime contractor’s cost and schedule performance continued to decline during fiscal year 2006. The contractor overran its fiscal year 2006 budget by $54.8 million and did not perform $26.4 million of work on schedule. By September 2006, this resulted in an unfavorable cumulative cost variance of $77.9 million and an unfavorable cumulative schedule variance of $50 million. Figure 3 shows the decline in cost and schedule performance for the ABL prime contractor throughout fiscal year 2006.

During the fiscal year, the ABL contractor needed additional time and money to solve technical challenges associated with the element’s Beam Control/Fire Control component. Software, integration, and testing difficulties caused significant delays with the component. Software problems were caused by the incorporation of numerous changes, basic logic errors, and differences between the environment of the software development laboratory and the environment aboard the aircraft. Integration and testing of the complex system and hardware failures also contributed to the delays. Together, according to ABL’s program manager, these problems caused the contractor to experience about a 3 1/2 month schedule delay that in turn delays the program’s lethality demonstration from 2008 to 2009. Also, if the contractor’s cost performance continues to decline as it did in fiscal year 2006, we estimate that at completion the contract could overrun its budget by about $112.1 million to $248.3 million.

Limited Contractor Data Prevented Analysis of C2BMC Contractor Performance

We were unable to fully evaluate the contractor’s performance for the C2BMC program because the contractor did not report all data required to conduct earned value analysis for 7 months of the fiscal year. During fiscal year 2006, the C2MBC contractor ended the Block 2004 increment or Part 3 of its Other Transaction Agreement and began work on its Block 2006 program of work, referred to as Part 4 of the agreement. The contractor completed its Block 2004 program of work (Part 3) in December 2005 and was awarded the Block 2006 increment (Part 4) on December 28, 2005. However, budget cuts prompted the program to reduce the C2BMC enhancements planned for Block 2006 and revise its agreement with the contractor. Shortly after, the program received additional funds which led to a re-negotiation of the Part 4 agreement. The new scope of work included enhancements that could not be completed within available funding. In March 2006, the program began to replan its Block 2006 increment of work (Part 4) and suspended earned value management reporting. During the replan, which occurred throughout most of fiscal year 2006, the contractor reported only actual cost data in lieu of comparing actual costs to budgeted cost. The cost of the revised agreement on the Block 2006 increment of work was negotiated in October 2006.

GMD Contractor Continues to Spend More Money and Time than Budgeted

The GMD prime contractor’s cost performance continued to decline during fiscal year 2006, but its fiscal year schedule performance improved. By September 2006, the cumulative cost of all work completed was $1.06 billion more than expected and in fiscal year 2006 alone, work cost about $347 million more than budgeted. The contractor was able to complete $90.2 million of fiscal year 2006 work ahead of schedule; but the cumulative schedule variance continued to be negative at $137.8 million. Figure 4 depicts the cost and schedule performance for the GMD contractor during fiscal year 2006. Based on its fiscal year 2006 performance, the GMD contractor could overrun the total budgeted cost of the contract by about $1.5 to $1.9 billion.

The GMD program recently finished rebaselining its contract to reflect a significant program realignment to reduce program risk and to execute the program within available funding. While the new baseline was being implemented, earned value metrics, according to program officials, were significantly distorted because progress was measured against a plan of work that the program was no longer following. The contractor is in the process of developing a new contract baseline that incorporates the program’s new scope, schedule, and budget. By the end of September 2006, phase one of the new baseline covering fiscal year 2006-2007 efforts had been implemented and validated through Integrated Baseline Reviews of the prime contractor and its major subcontractors. Implementation of the phase 2 baseline covering the remaining contract effort was completed in October 2006 with the final integrated baseline reviews of the prime and major subcontractors completed by mid-December 2006.

Based on the data provided by the contractor during fiscal year 2006, technical and quality issues with the exoatmospheric kill vehicle (EKV) are the leading contributors to cost overruns and schedule slips for the GMD program. In fiscal year 2006, EKV related work cost $135.2 million more than budgeted. Quality problems identified after faulty parts had been incorporated into components required rework and forced the subcontractor to increase screening tests to identify defective parts.

Development issues with two boosters being developed to carry the exoatmospheric kill vehicles into space also increased costs during fiscal year 2006. The element’s Orbital Boost Vehicle experienced cost growth totaling $15.0 million while the Boost Vehicle+ booster experienced growth of $74.1 million. The Orbital Boost Vehicle’s cost grew as the need for more program management, systems engineering, and production support was required to work an extended delivery schedule. The Boost Vehicle+ contractor incurred additional costs as a result of its efforts to redesign the booster’s motors. For example, the contractor spent additional time preparing drawings and providing technical oversight of suppliers.

The contractor also experienced cost growth as it readied the Sea-based X-Band radar for deployment. Maintenance, repair, and certification problems cost more than expected. In addition to making changes that an independent review team suggested were needed before the radar was made operational, the contractor had to repair an unexpected ballast leak requiring the installation of hydraulic valves and other engineering changes.

GMD’s cumulative negative schedule variance is primarily caused by a subcontractor needing more time than planned to manufacture exoatmospheric kill vehicles. In addition, the prime contractor delayed planned tests because test interceptors were being produced at a slower rate. According to program officials, variances improved during fiscal year 2006 as the subcontractor delivered components on schedule.

KEI Contractor Makes Progress during Fiscal Year 2006

In July 2005, the KEI program modified its prime contract to require that the KEI element be capable of intercepting enemy missiles in the midcourse of their flight. Consequently, the program is rebaselining its prime contract to better align its cost and schedule objectives with the new work content. During fiscal year 2006, the contractor’s work cost approximately $0.6 million less than expected and the contractor completed about $0.6 million of work ahead of schedule. Cumulatively, the contractor’s cost performance has been positive, with all work to date being performed for $3.6 million less than budgeted. However, by year’s end, the cumulative schedule variance was a negative $5.3 million. We cannot estimate whether the total contract can be completed within budgeted cost because the contract is only 6 percent complete and trends cannot be developed until at least 15 percent of the contract is completed. Figure 5 highlights the contractor’s performance during fiscal year 2006.

The KEI prime contractor was able to perform within its budgeted costs during fiscal year 2006 as a result of its efficient use of test resources. Although the contractor improved its negative schedule variance over the course of the year, its cumulative schedule variance remains unfavorable because requirements changes have delayed the development of the element’s design and of manufacturing processes. Schedule delays caused the program to postpone its element-level System Design Review originally scheduled for July 2007. However, the contractor asserts that there is no impact to the booster flight test currently scheduled for fiscal year 2008.

Lack of Reporting Limits Knowledge of MKV Contractor’s Performance

Our analysis of the performance of the contractor developing the MKV element was limited because MDA suspended contract performance reporting in February 2006 as the program transitioned from an advanced technology development program to a system development program. The transition prompted MKV to establish a new contract baseline. Although the contractor could begin reporting after the baseline is in place, it is not issuing Contract Performance Reports until an Integrated Baseline Review is completed. Until that time, the contractor is measuring its progress against an integrated master schedule.

Sensors’ FBX-T Contractor Meets Fiscal Year Cost and Schedule Objectives

As of September 2006, the Sensor’s contractor had underrun its fiscal year 2006 budget by $3.8 million and it was ahead in completing $5.4 million of scheduled work. Considering prior years performance, the contractor is performing under budget with a favorable cumulative cost variance of $20.2 million and ahead of schedule with a favorable cumulative schedule variance of $26.6 million. Judging from the contractor’s cost and schedule performance in fiscal year 2006, we estimate that at the contract’s completion, the contractor will underrun the budgeted cost of the contract by between $26.3 million and $44.9 million. Figure 6 shows the favorable trend in FBX-T 2006 performance.

According to program officials, the cumulative favorable cost variance is driven by reduced cost in radar hardware and manufacturing created by machine process improvements and staffing efficiencies. The favorable cumulative schedule variance primarily results from a positive $17 million cumulative schedule variance brought forward from fiscal year 2005 that was created when the contractor began manufacturing radars 2 through 4 ahead of schedule.

STSS Contractor Performance Declines during the Year

The STSS contractor’s cost and schedule performance continued to degrade during fiscal year 2006. During the fiscal year, the contractor overran budgeted costs by about $66.8 million and was unable to complete $84.1 million of work as scheduled. Combining the contractor’s performance during fiscal year 2006 with its performance in prior years, the contract has a cumulative unfavorable cost variance of approximately $163.7 million and a cumulative negative schedule variance of $104.4 million. If the contractor’s performance continues to decline, the contract could exceed its budgeted cost at completion by $567.3 million to $1.4 billion. Figure 7 depicts the cumulative cost and schedule performance of the STSS prime contractor.

Quality issues at the payload subcontractor and technical difficulties encountered by the prime contractor during payload integration and testing contributed to the STSS element’s cumulative unfavorable cost and schedule variances. The first satellite’s payload experienced hardware failures when tested in a vacuum and at cold temperatures, slowing integration with the first satellite. Integration issues were also discovered as the payload was tested at successively higher levels of integration. According to program officials, the prime contractor tightened its inspection and oversight of the subcontractor responsible for integrating and testing the satellite payloads. Also, a re-education effort was undertaken to ensure that all personnel on the program knew and understood program instructions. Although the prime contractor continued to experience negative variances during the fiscal year, it should be noted that the subcontractor’s performance with respect to the second payload improved as the result of these added steps. However, the degradation of the prime contractor’s performance offset the improved performance of the subcontractor.

THAAD Contractor’s Performance Erodes in Fiscal Year 2006

During fiscal year 2006, the THAAD contractor expended more money and time than budgeted to accomplish planned work. During fiscal year 2006, the contractor incurred a negative cost variance of $87.9 million, which boosted the cumulative negative cost variance to $104.2 million. Similarly, the contractor did not complete $37.9 million of work scheduled for fiscal year 2006 on time. However, because the contractor completed prior years’ work ahead of schedule, the cumulative negative schedule variance was $28 million. Based on fiscal year performance, we estimate that at completion the contract could exceed its budgeted cost by between $134.7 million and $320.2 million.

The THAAD prime contractor’s negative cost variance for the fiscal year can be attributed to the increased cost of missile manufacturing, re- designs, and rework, as well as launcher hardware design, integration difficulties, and software problems. However, the contractor is performing well in regard to the radar portion of the contract, which is offsetting a portion of the negative cost variance.

The program’s negative schedule variance is largely driven by the missile, the launcher, and systems tests. The negative missile variance is mainly caused by problems with the Divert Attitude Control System and delays in activation of a test facility.

Appendix III: Scope and Methodology

To examine the progress MDA made in fiscal year 2006 toward its Block 2006 goals, we examined the efforts of individual programs, such as the GMD program, that are developing BMDS elements under the management of MDA. The elements included in our review collectively accounted for 72 percent of MDA’s fiscal year 2006 research and development budget request. We evaluated each element’s progress in fiscal year 2006 toward Block 2006 schedule, testing, performance, and cost goals. In making this comparison, we examined System Element Reviews, test and production schedules, test reports, and MDA briefing charts. We developed data collection instruments, which were submitted to MDA and each element program office, to gather detailed information on completed program activities including tests, prime contracts, and estimates of element performance. In addition, we visited an operational site at Vandenberg Air Force Base, California; and we visited MDA contractor facilities including Orbital Sciences Corporation in Chandler, Arizona; Raytheon in Tucson, Arizona; and Lockheed Martin in Sunnyvale, California. To understand performance issues, we talked with officials from MDA’s System’s Engineering and Integration Directorate. We also discussed fiscal year 2006 progress and performance with officials in MDA’s Agency Operations Office, each element program office, as well as the office of DOD’s Director, Operational Test and Evaluation, DOD’s office of Program Analysis and Evaluation, and DOD’s Operational Test Agency. To assess each element’s progress toward its cost goals, we reviewed Contract Performance Reports and, when available, the Defense Contract Management Agency’s analyses of these reports. We also interviewed officials from the Defense Contract Management Agency. We applied established earned value management techniques to data captured in Contract Performance Reports to determine trends and used established earned value management formulas to project the likely costs of prime contracts at completion. We reviewed each element’s prime contract and also examined fiscal year 2006 award fee plans and award fee letters.

In assessing MDA’s flexibility, transparency, and accountability, we interviewed officials from the Office of the Under Secretary of Defense’s Office for Acquisition, Technology, and Logistics. We also examined Government Auditing Standards, a Congressional Research Service report, U.S. Code Title 10, DOD acquisition system policy, and the Statement of Federal Financial Accounting Standards Number 4.

To determine the progress MDA has made in ensuring quality, we talked with officials from MDA’s Office of Safety, Quality, and Mission Assurance. We also held discussions with MDA’s Office of Agency Operations, and discussed quality issues at contractor facilities including Orbital Sciences Corporation in Chandler, Arizona; Raytheon in Tucson, Arizona; and Lockheed Martin in Sunnyvale, California.

To ensure that MDA-generated data used in our assessment are reliable, we evaluated the agency’s management control processes. We discussed these processes with MDA senior management. In addition, we confirmed the accuracy of MDA-generated data with multiple sources within MDA and, when possible, with independent experts. To assess the validity and reliability of prime contractors’ earned value management systems and reports, we interviewed officials and analyzed audit reports prepared by the Defense Contract Audit Agency. Finally, we assessed MDA’s internal accounting and administrative management controls by reviewing MDA’s Federal Manager’s Financial Integrity Report for Fiscal Years 2003, 2004, 2005, and 2006.

Our work was performed primarily at MDA headquarters in Arlington, Virginia. At this location, we met with officials from the Aegis Ballistic Missile Defense Program Office; Airborne Laser Program Office; Command, Control, Battle Management, and Communications Program Office; Multiple Kill Vehicle Program Office; MDA’s Agency Operations Office; MDA’s Office of Quality, Safety, and Mission Assurance; DOD’s office of the Director, Operational Test and Evaluation; DOD’s office of Program Analysis and Evaluation; and the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics. We held a teleconference with officials from DOD’s Operational Test Agency, also in Arlington, Virginia. In addition, we met with officials in Huntsville, Alabama, including officials from the Ground-based Midcourse Defense Program Office, the Terminal High Altitude Area Defense Project Office, the Kinetic Energy Interceptors Program Office, and the Defense Contract Management Agency.

We conducted our review from June 2006 through March 2007 in accordance with generally accepted government auditing standards.

Appendix IV: GAO Contact and Staff Acknowledgments

Acknowledgments

In addition to the individual named above, Barbara Haynes, Assistant Director; LaTonya Miller; Ivy Hubler; Steven Stern; Meredith Allen; Sigrid McGinty; Tony Beckham; and Adam Vodraska made key contributions to this report.