Defense Acquisitions: Progress Made in Fielding Missile Defense, but Program Is Short of Meeting Goals
Released: 2008-03-14T00:00:00

Letter

Background

The Missile Defense Agency’s mission is to develop an integrated and layered BMDS to defend the United States, its deployed forces, allies, and friends. The BMDS is expected to be capable of engaging all ranges of enemy ballistic missiles in all phases of flight. This is a challenging expectation, requiring a complex combination of defensive components— space-based sensors, surveillance and tracking radars, advanced interceptors, and a battle management, command, control, and communications component—that work together as an integrated system.

A typical scenario to engage an intercontinental ballistic missile (ICBM) would unfold as follows: Infrared sensors aboard early-warning satellites detect the hot plume of a missile launch and alert the command authority of a possible attack.

Upon receiving the alert, land- or sea-based radars are directed to track the various objects released from the missile and, if so designed, to identify the warhead from among spent rocket motors, decoys, and debris.

When the trajectory of the missile’s warhead has been adequately established, an interceptor—consisting of a kill vehicle mounted atop a booster—is launched to engage the threat. The interceptor boosts itself toward a predicted intercept point and releases the kill vehicle.

The kill vehicle uses its onboard sensors and divert thrusters to detect, identify, and steer itself into the warhead. With a combined closing speed on the order of 10 kilometers per second (22,000 miles per hour), the warhead is destroyed above the atmosphere through a “hit to kill” collision with the kill vehicle.

To develop a system capable of carrying out such an engagement, MDA, until December 2007, executed an acquisition strategy in which the development of missile defense capabilities was organized in 2-year increments known as blocks. Each block was intended to provide the BMDS with capabilities that enhanced the development and overall performance of the system. The first 2-year block—Block 2004—fielded a limited initial capability that included early versions of the GMD, Aegis BMD, Patriot Advanced Capability-3, and C2BMC elements. The agency’s second 2-year block—Block 2006—culminated on December 31, 2007, and fielded additional BMDS assets. Block 2006 also continued the evolution of Block 2004 by providing improved GMD interceptors, enhanced Aegis BMD missiles, upgraded Aegis BMD ships, a Forward-Based X-Band- Transportable radar, and enhancements to C2BMC software. On December 7, 2007, MDA’s Director approved a new block construct that will be the basis for all future development and fielding. Table 1 provides a brief description of all elements currently being developed by MDA.

More Capability Fielded but Less than Planned and at Higher Cost

MDA made progress in developing and fielding the BMDS during 2007. Additional assets were fielded and/or upgraded, several tests met planned objectives, and other development activities were conducted. On the other hand, fewer assets were fielded than originally planned, the cost of the block increased, some flight tests were deferred, and the performance of fielded assets could not be fully evaluated.

Block 2006 Improves BMDS Capability

During Block 2006, MDA increased its inventory of BMDS assets while enhancing the system’s performance. The agency fielded 14 additional Ground-based interceptors, 12 Aegis BMD missiles designed to engage more advanced threats, 4 new Aegis BMD destroyers, 1 new Aegis BMD cruiser, as well as 8 C2BMC Web browsers and 1 C2BMC suite. In addition, MDA upgraded half of its Aegis BMD ship fleet, successfully conducted four Aegis BMD and two GMD intercept tests, and completed a number of ground tests to demonstrate the capability of BMDS components. Considering assets fielded during Blocks 2004 and 2006, MDA, by December 31, 2007, had cumulatively fielded a total of 24 Ground-based interceptors, 2 upgraded early-warning radars, an upgraded Cobra Dane surveillance radar, 1 Sea-based X-band radar, 2 Forward-Based X-Band Transportable radars, 21 Aegis BMD missiles, 14 Aegis BMD destroyers, and 3 Aegis BMD cruisers. In addition, MDA had fielded 6 C2BMC suites; 46 warfighter enterprise workstations with situational awareness; BMDS planner and sensor management capabilities; 31 C2BMC Web browsers, 13 with laptop planners; and redundant communications node equipment to connect BMDS elements worldwide.

MDA Does Not Meet Original Goals, but Meets or Exceeds Most Revised Goals

In March 2005, MDA submitted to Congress the number of assets it planned to field during Block 2006. However, increasing costs, technical challenges, and schedule delays prompted the agency to reduce the quantity of planned assets. Consequently, in March 2006, shortly after submitting its fiscal year 2007 budget, MDA notified Congress that it was revising its Block 2006 Fielded Configuration Baseline. Although MDA did not meet its original block fielding goals, it was able in nearly all instances to meet or exceed its revised goals. Of the four elements delivering assets during Block 2006, one—Sensors—was able to meet its original goal. However, two elements—GMD and C2BMC—were able to exceed their revised fielding goals. Table 2 depicts the goals and the number of assets fielded.

Although GMD did not meet its original goal of fielding up to 15 interceptors and partially upgrading the Thule early warning radar, the element was able to surpass its revised goal of fielding 12 interceptors. By December 31, 2007, the GMD element fielded 14 interceptors—2 more than planned. To achieve its revised goal, the element’s prime contractor added a manufacturing shift during 2007 and extended the number of hours that certain shifts’ personnel worked. These actions allowed the contractor to more than double its interceptor emplacement rate.

Last year, we reported that MDA delayed the partial upgrade of the Thule early-warning radar—one of GMD’s original goals—until a full upgrade could be accomplished. According to DOD, the full upgrade of Thule is the most economical option and it meets DOD’s desire to retain a single configuration of upgraded early warning radars. The Thule early warning radar upgrade is being accomplished by two separate contract awards. Raytheon was awarded a contract in April 2006 to develop and install prime mission equipment; while Boeing was expected to receive a contract in January 2008 to integrate the equipment into the BMDS ground communication network.

In March 2005, MDA included three C2BMC suites as part of its fielding goal for Block 2006. These suites were to be fielded at U.S. European Command, U.S. Central Command, and another location that was to be identified later. Faced with a $30 million reduction in C2BMC’s fiscal year 2006 budget, MDA in March 2006 revised this goal to replace the 3 suites with 3 less expensive Web browsers. However, by the end of Block 2006, MDA found an innovative way to increase combatant commands’ situational awareness and planning capability. In 2005, the C2BMC program conducted a network load analysis and concluded that situational awareness and planning capability—equivalent to that provided by a suite- -could be gained by combining Web browsers and planners. To prove that this approach would work, MDA fielded 4 Web browsers and one planner at the U.S. European Command. MDA learned that this combination of hardware, fielded in the quantities needed to meet a command’s needs and connected to an existing server, provided the situational awareness and planning capability of a suite at less cost. MDA extended this approach by fielding one Web browser and one planner at four other locations—U.S. Forces Japan; U.S. Forces Korea; the Commander of U.S. Strategic Command; and the Commander of the Space and Missile Defense Command. In addition, MDA fielded one suite at U.S. Pacific Command.

The Aegis BMD element was able to meet its revised block goals for only one of its two components. The program upgraded all planned ships, but fielded three fewer Aegis BMD Standard Missile-3s (SM-3) than planned. The program did not meet its revised missile goal because three U.S missiles were delayed into 2008 to accommodate an unanticipated requirement to deliver three missiles to Japan. Figure 1 below depicts the location of current BMDS assets.

Block Costs Have Increased, but Full Cost Is Not Known

MDA’s Block 2006 program of work culminated with higher than anticipated costs. In March 2007, we reported that MDA’s cost goal for Block 2006 increased by approximately $1 billion because of greater than expected GMD operations and sustainment costs and technical problems. During fiscal year 2007, some prime contractors performing work for the BMDS overran their budgeted costs. To stay within its revised budget, MDA was forced to reduce the amount of work it expected to accomplish during the block. The full cost of the block cannot be determined because of the deferral of work from one block to another. In addition, some MDA prime contractors too often employ a planning methodology that has the potential to obscure the time and money that will be needed to produce the outcomes intended. If the work does not yield the intended results, MDA could incur additional future costs. While MDA struggled to contain costs during Block 2006, the agency awarded two contractors a large percentage of available fee for performance in cost and/or program management although the contractor-reported data showed declining cost and schedule performance. Both award fee plans for these contractors direct that cost and schedule performance be considered as factors in making the evaluation. While these factors are important, MDA’s award fee plans provide for the consideration of many other factors in making award fee determinations.

Five Contractors Overran 2007 Budgets

To determine if contractors are executing the work planned within the funds and time budgeted, each BMDS program office requires its prime contractor to provide monthly Earned Value Management reports detailing cost and schedule performance. If more work was completed than scheduled and the cost of the work performed was less than budgeted, the contractor reports a positive schedule and cost variance. However, if the contractor was unable to complete all of the work scheduled and needed more funds to complete the work than budgeted, the contractor reports a negative schedule and cost variance. Of course, the results can be mixed.

That is, the contractor may have completed more work than scheduled but at a cost that exceeded the budget.

As shown in table 3 below, the contractors for the nine BMDS elements collectively overran their fiscal year 2007 budgets by approximately $166 million. We estimate that at completion, the cumulative overrun in the contracts could be between about $1.3 billion and $1.9 billion. Our predictions of final contract costs were developed using formulas accepted within the cost community and were based on the assumption that the contractor will continue to perform in the future as it has in the past. It should also be noted that some contracts include more than Block 2006 work. For example, the STSS contract includes work being accomplished in anticipation of future blocks. Our analysis is presented in table 3 below. Appendix II provides further details on the cost and schedule performance of the contractors outlined in the table.

Technical problems and software issues caused several BMDS elements to overrun their fiscal year 2007 budgeted costs. In addition, 4 of the 10 contracts we reviewed contained some kind of replanning activity during fiscal year 2007 and the ABL contract was partially rebaselined. Contractors may replan when they conclude that the current plan for completing the effort remaining on the contract is unrealistic. A replan can include reallocating the remaining budget over the rest of the work, realigning the schedule within the contractually defined milestones, and setting either cost or schedule variances to zero or setting both to zero. A rebaseline is similar, but it may also add additional time and/or funding for the remaining work.

The ABL contractor was overrunning both its fiscal year 2007 budget and schedule early in the year. Although by year’s end it appears that the contractor recovered, the contractor would have continued to overrun both its budget and its schedule if most of the contract had not been rebaselined. The contractor realized cost and schedule growth as it worked to solve software integration problems in the Beam Control/Fire Control component and dealt with a low-power laser needed for flight tests that was not putting enough energy on the target. After encountering these problems, the ABL contractor did not have sufficient schedule or budget to complete the remaining contract work. Therefore, in May 2007, the program allowed the contractor to rebaseline all of the remaining work devoted to developing, integrating, flight testing, and delivering the ABL prototype. The rebaselining effort added about $253 million to the contract and extended the contract’s period of performance by more than a year.

The THAAD prime contractor’s cost overrun of $91.1 million was primarily caused by technical problems related to the element’s missile, launcher, radar, and test components. Missile component cost overruns were caused by higher than anticipated costs in hardware fabrication, assembly, and support touch labor for structures, propulsion, and other subassembly components. Additionally, design issues with the launcher’s missile round pallet and the electronics assembly that controls the launcher caused the contractor to experience higher than anticipated labor and material costs. The radar component ended the fiscal year with a negative cost variance as more staff was required than planned to resolve hardware design issues in the radar’s prime power unit. The contractor also experienced negative cost variances with the system test component because the Launch and Test Support Equipment required additional set-up time at the flight test range.

The STSS contractor’s $67.7 million fiscal year 2007 cost variance is primarily attributed to problems that occurred during thermal vacuum testing of the first satellite. Since the satellites are legacy hardware built under a former program, there are no spares available for testing. As a result, the contractor needed to handle the parts carefully to avoid damage to the hardware, increasing the time devoted to the test. Further test delays occurred when a number of interface issues surfaced during testing and when the cause of component problems could not be easily traced to their source. The program office believes that the cost variance would have been less if design engineers had been available during testing. Because engineers were not present to quickly identify the cause of component problems, a time-consuming analysis of each problem was needed.

Total Block 2006 Costs Cannot be Determined

In March 2007, we reported that a full accounting of Block 2006 costs was not possible because MDA has the flexibility to redefine block outcomes. That is, MDA can delay the delivery of assets or other work activities from block to block and count the work as a cost of the block during which the work is performed, even though the work does not benefit that block. For example, MDA deferred some Block 2004 work until Block 2006 so that it could use the funds appropriated for that work to cover unexpected cost increases caused by technical problems recognized during development, testing, and production. With the deferral of the work, its cost was no longer counted as a Block 2004 cost, but as a Block 2006 cost. As a result, Block 2004’s cost was understated and Block 2006’s cost is overstated. Because MDA did not track the cost of the deferred work, the agency could not make an adjustment that would have matched the cost with the correct block.

The cost of Block 2006 was further blurred as MDA found it necessary to defer some Block 2006 work until a future block. For example, when the STSS contractor overran its fiscal year 2007 budget because of testing problems, the program did not have sufficient funds to launch the demonstration satellites in 2007 as planned. The work is now scheduled for 2008. The consequence of deferring Block 2004 work to Block 2006 and Block 2006 to 2008 is that the full cost of Block 2006 cannot be determined.

Contractors’ Planning Methodology Has the Potential to Obscure Block Cost

Some MDA prime contractors too often employ a planning methodology that has the potential to obscure the time and money that will be needed to produce the outcomes intended. Contractors typically divide the total work of a contract into small efforts in order to define them more clearly and to ensure proper oversight. Work may be planned in categories including (1) level of effort (LOE) —work that contains tasks of a general or supportive nature and do not produce a definite end product—or (2) discrete work—work that has a definable end product or event.

Level of effort work assumes that if the staff assigned to the effort spend the planned length of time, they will attain the outcome expected. According to earned value experts and the National Defense Industrial Association, while it is appropriate to plan such tasks as supervision or contract administration as LOE, it is not appropriate to plan tasks that are intended to result in a product, such as a study or a software build, as LOE because contractors do not report schedule variances for LOE work. Therefore, when contractors incorrectly plan discrete work as LOE, reports that are meant to allow the government to assess contractor cost and schedule performance may be positive, but the government may not have full insight into the contractor’s progress. The greater the percentage of LOE, the weaker the link between inputs (time and money) and outcomes (end products), which is the essence of earned value analysis.

Essentially, depending on the magnitude of LOE, schedule variances at the bottom line can be understated. The significant amount of BMDS work being tracked by LOE may have limited our assessment of the contractors’ performance. That is, the contractor’s performance may appear to be more positive than it would be if work had been correctly planned. In such cases, the government may have to expend additional time and money to achieve the outcomes desired. MDA Earned Value Management officials agreed that some BMDS prime contractors incorrectly planned discrete work as LOE, but the agency is taking steps to remedy this situation so that they can better monitor the contractors’ performance.

While it is not possible to state with certainty how much work a contractor should plan as LOE, experts within the government cost community, such as Defense Contract Management Agency officials, agree that LOE levels over 20 percent warrant investigation. According to MDA, many of its prime contractors plan a much larger percentage than 20 percent of their work as LOE. Table 4 presents the percentage of work in each BMDS prime contract that is categorized as LOE.

The Aegis BMD SM-3, MKV, ABL, and C2BMC contractors planned more than half of certain work as LOE. In several instances, MDA Earned Value Management officials and program office reviewers agreed that some of the LOE work could be redefined into discrete work packages. For example, from January through December 2007, the C2BMC contractor planned 73 percent of its work as LOE. This included activities such as software development and integration and test activities that result in two definable products—software packages and tests. At the direction of the C2BMC Program Office, the C2BMC contractor redefined some contract work, including software development and integration and test activities, as discrete, reducing the amount of LOE on the contract to 52 percent.

The Aegis BMD element also reported a high percentage of LOE for its Standard Missile-3 contract, particularly considering that its products— individual missiles—are quite discrete. In August 2007, the element reported that the contractor had planned 73 percent of the contract work as LOE. The portion of the work that contained this amount of LOE was completed in March 2007 with an underrun of $7.2 million. Although the contractor reported an underrun for this work upon its completion, the high percentage of LOE may have, over the contract period, distorted the contractor’s actual cost and schedule performance. Similarly, it is important to note that the amount of LOE for the SM-3 work that is currently ongoing is considerably less. Program officials told us that prior to the commencement of this segment of work, the MDA Earned Value Management Group and program officials recommended that the program minimize the amount of LOE on its contracts. Currently, only 18 percent of the SM-3 contract is considered LOE.

Two Contractors Awarded Fee That Does Not Reflect Earned Value Performance

MDA uses award fees to encourage its contractors to perform in an innovative, efficient, and effective way in areas considered important to the development of the BMDS. Because award fees are intended to motivate contractor performance for work that is neither feasible nor effective to measure objectively, award fee criteria and evaluations tend to be subjective. Each element’s contract has an award fee plan that identifies the performance areas to be evaluated and the methodology by which those areas will be assessed. An award fee evaluation board—made up of MDA personnel, program officials, and officials from key organizations knowledgeable about the award fee evaluation areas–– judges the contractor’s performance against specified criteria in the award fee plan. The board then recommends to a fee determining official the amount of fee to be paid. MDA’s Director is the fee-determining official for all BMDS prime contracts that we assessed.

During fiscal year 2007, MDA awarded approximately 95 percent, or $606 million, of available award fee to its prime contractors. While the cost, schedule, and technical performance of several contractors appeared to be aligned with their award fee, two contractors were rated as performing very well in the cost and/or program management elements and received commensurate fees even though earned value management data showed that their cost and schedule performance was declining. On the other hand, MDA did not award any fee to the THAAD contractor for its management of contract cost during a time when earned value data showed steadily increasing costs. Although DOD guidance discourages the use of earned value performance metrics in award fee criteria, MDA includes this as a factor in several of its award fee plans. The agency considers many factors in rating contractors’ performance and making award fee determinations, including consideration of earned value data that shows cost, schedule, and technical trends. In addition, MDA has begun to revise its award fee policy to align agency practices more closely with DOD’s current policy that better links performance with award fees.

The ABL and Aegis BMD weapon system contractors received a large percentage of the 2007 award fee available to them for the cost and/or program management element. MDA rated the ABL contractor’s performance in cost and program management elements as “very good,” awarding the contractor 88 percent of the fee available in these performance areas. According to the award fee plan, one of several factors that is considered in rating the contractor’s performance as very good is whether earned value data indicates that there are few unfavorable cost, schedule, and/or technical variances or trends. During the February 2006 to January 2007 award fee period, earned value data shows that the contractor overran its budget by more than $57 million and did not complete $11 million of planned work. Similarly, the Aegis BMD weapon system contractor was to be rated as to how effectively it managed its contract’s cost. The award fee plan for this contractor also directs that earned value be one of the factors considered in making such an evaluation. During the fee period that ran from October 2006 through March 2007, MDA rated the contractor’s cost management performance as outstanding and awarded 100 percent of the available fee. Earned value data during this time period indicates that the contractor overran its budget by more than $6 million. MDA did not provide us with more detailed information as to other factors that may have influenced its decision as to the amount of fee awarded to the ABL and Aegis BMD Weapon System contractors.

MDA recognizes that there is not always a good link between the agency’s intentions for award fees and the amount of fee being earned by its contractors. In an effort to rectify this problem, the agency released a revised award fee policy in February 2007 to ensure its compliance with recent DOD policies that are intended to address award fee issues throughout the Department. Specifically, MDA’s policy directs that every contract’s award fee plan include: Criteria for each element of the award fee that is specific enough to enable the agency to evaluate contractor performance and to determine how much fee the contractor can earn for that element. The criteria is to clearly define the performance that the government expects from the contractor for the applicable award fee period and the criteria for any one element must be distinguishable from criteria for other elements of the award fee; An emphasis on rewarding results rather than effort or activity; and An incentive to meet or exceed agency requirements.

Additionally, MDA’s policy calls for using the Award Fee Advisory Board to not only make award fee recommendations to the fee determining official, but to also biannually report to MDA’s Director as to whether award fee recommendations are consistent with DOD’s Contractor Performance Assessment Report—a report that provides a record, both positive and negative, on a given contract for a specific period of time. Appendix II of this report provides additional information on BMDS prime contracts and award fees.

Most Test Objectives Achieved Despite Schedule Delays

During 2007, several BMDS programs experienced setbacks in their test schedules. The Aegis BMD, THAAD, ABL, STSS, and C2BMC elements experienced test delays, but all were able to achieve their primary test objectives. GMD, on the other hand, experienced a schedule delay caused by an in-flight target anomaly that prevented full accomplishment of one major 2007 test objective. The remaining three elements—MKV, KEI, and Sensors—were able to execute all scheduled activities as planned.

Several Elements Met Key Test Objectives

The Aegis BMD, THAAD, C2BMC, ABL, and STSS elements continued to achieve important test objectives in 2007, although some tests were delayed. Aegis BMD proved its capability against more advanced threats, while THAAD proved that it could intercept both inside and outside of the atmosphere. C2BMC completed a number of software and system-level tests. The ABL and STSS programs saw delays in important ground tests, but ABL was able to begin flight testing its beam control/fire control component using a low-power laser in 2007 and STSS completed thermal vacuum testing of both satellites by the end of the year. However, the delays in the ABL and STSS programs may hold up their incorporation into the BMDS during future blocks.

Although the Aegis BMD program encountered some test delays, it was able to achieve all fiscal year 2007 test objectives. In December 2006, the program stopped a test after a crew member changed the ship’s doctrine parameters just prior to target launch, preventing the ship’s fire control system from conducting the planned engagement. During this test event, the weapon system failed to recognize the test target as a threat, which prevented the SM-3 missile from launching. Also, according to program officials, the system did not provide a warning message which contributed to the mission being aborted prematurely and prevented the Aegis BMD program from meeting its test objectives. However, 4 months later, the same flight test event was successfully completed and all test objectives were met. During that event, the program was able to demonstrate that the Aegis BMD could simultaneously track and intercept a ballistic missile and an anti-ship cruise missile. In June 2007, the program successfully completed its first flight test utilizing an Aegis BMD destroyer to intercept a separating target, and in November, the program conducted its first test that engaged two ballistic missile targets simultaneously. During the last test, Aegis missiles onboard an Aegis BMD cruiser successfully intercepted two short-range non-separating targets and achieved all primary test objectives outlined for this event.

The THAAD program expected to complete four flight tests prior to the end of fiscal year 2007 but was only able to complete three. Two tests successfully resulted in intercepts of short-range ballistic missiles at different levels of the atmosphere. The third test successfully demonstrated component capability in a high-pressure environment and was the lowest altitude interceptor verification test to date. However, the fourth test was delayed, initially due to target availability driven by late modifications to the target hardware configuration. Additionally, during pre-flight testing, the contractor found debris in the interceptor. This caused the interceptor to be returned to the factory for problem investigation. While the problem was corrected and the interceptor was returned to the test range in only 7 days, the test was rescheduled because the test range was not available before the end of fiscal year 2007.

During fiscal year 2007, the C2BMC program completed BMDS-level ground and flight tests, successfully achieving its test objectives of verifying the capabilities and readiness of a new software configuration. The software is designed to provide the BMDS with improved defense planning capability, including better accuracy and speed; a new operational network; and additional user displays. Because of the integral nature of the C2BMC product, problems encountered in some elements’ test schedules have a cascading effect on C2BMC’s test schedule. Even though this limited C2BMC testing, a review of the integrated and distributed ground test data resulted in the decision to field the software in December 2007.

ABL achieved most of its test objectives during fiscal year 2007, but experienced delays during Block 2006 that deferred future BMDS program decisions. The program experienced a number of technical problems during fiscal year 2006 that pushed some planned activities into fiscal year 2007. One such activity was the execution of the program’s first of four key knowledge points—a ground test to demonstrate ABL’s ability to acquire and track a target while performing atmospheric compensation. The test was conducted in December 2006, 3 ½ months later than planned. At the culmination of the test, program officials noted two problems. First, the system’s beam control/fire control software was not integrated as anticipated. In addition, the energy that the low-power laser placed on the target during the test was not optimal. According to program officials, both of these issues were resolved before the system began flight testing the full beam control/fire control component in February 2007. However, the delays caused the program to further postpone a key lethality demonstration—a demonstration in which the ABL will attempt to shoot down a short-range ballistic missile—until last quarter of fiscal year 2009. This demonstration is important to the program because it is the point at which MDA will decide the program’s future.

Although the ABL program experienced some setbacks with its first key knowledge point, it was able to meet all objectives for each subsequent knowledge point. In addition to the first knowledge point, the program planned to demonstrate three additional knowledge points during fiscal year 2007. The second knowledge point was contingent upon completion of the first. To demonstrate the achievement of the two knowledge points, the contractor performed a flight test that showed the low-power laser was integrated and the beam control/fire control functioned sufficiently to perform target tracking and atmospheric compensation against an airborne target board. The third knowledge point was completed three months ahead of the planned 2007 schedule and demonstrated that ABL’s optical subsystem was adequate to support its high-power laser system. The fourth knowledge point–the completion of a series of flight tests to demonstrate the performance of the low-power laser system in flight—was completed in August 2007.

Delays in the STSS test program, along with funding shortages, postponed the planned 2007 launch of the program’s demonstration satellites. The STSS program is integrating two demonstration satellites with sensor payloads from legacy hardware developed under a former program. The use of legacy hardware has complicated the test program because spares needed for testing are not available. In order to preserve the condition of the legacy components, the program must exercise caution in handling the components to prevent damage, which has caused delays in testing. Additionally, a thermal vacuum test on the first space vehicle, to assess the ability of the satellite to operate in the cold vacuum of space, took twice as long as scheduled, due to a number of interface issues. Although the program was able to complete the integration and test of both demonstration satellites in 2007—major objectives for the program—funds were not available to launch the satellites as planned. Program officials believe that the satellites could be launched as early as April 2008 and as late as July 2008, 1 year later than originally scheduled. According to the program office, there is no margin in the 2008 budget, so any unexpected issues could put the 2008 launch date at risk. The delays in launching the STSS demonstration satellites do not impact MDA’s Block 2006 fielding plans as the satellites are intended to demonstrate a surveillance and tracking capability and do not provide any operational capability during the block. However, the delay in launching the demonstration satellites is causing a delay in MDA’s ability to initiate development of an operational constellation, which may delay a BMDS global midcourse tracking capability.

Despite delays in hardware and software testing and integration, other parts of the STSS program have proceeded according to schedule. Lessons learned from the thermal vacuum test for the first satellite’s sensor payload facilitated the completion of thermal vacuum testing of the second satellite’s payload in November 2007. Additionally, command and control capabilities of the ground segment were demonstrated and the second part of the acceptance test of STSS ground components was completed in September 2007.

Test Delays Prevent GMD from Achieving All 2007 Objectives

A target anomaly prevented the GMD element from achieving all 2007 objectives. The GMD program planned to conduct three flight tests–two intercept attempts and one radar characterization test—but was only able to conduct the radar test and one intercept test. The radar characterization test was conducted in March 2007. The target was launched from Vandenberg Air Force Base and was successfully tracked by the SBX radar and the radar of two Aegis BMD ships. During the test, officials indicated the SBX exhibited some anomalous behavior, yet was able to collect target tracking data and successfully transmit the information to the C2BMC element and the GMD fire control system at DOD’s Missile Defense Integration and Operations Center. No live interceptor was launched.

However, an intercept solution was generated and simulated interceptor missiles were “launched” from Fort Greely, Alaska. To address anomalous behavior, MDA adjusted software and performance parameters of the SBX radar.

In May 2007, the program attempted an intercept test, but a key component of the target malfunctioned. For that reason, the weapon system did not release the Ground-based interceptor and program officials declared the flight test a “no test” event. To date, program officials have not determined the root cause of the malfunction. In September 2007, the program successfully conducted a re-test and achieved an intercept of the target using target tracking data provided by the Beale upgraded early warning radar. MDA test officials told us that aging target inventory could have contributed to the target anomaly. The officials explained that some targets in MDA’s inventory are more than 40 years old and their reliability is relatively low. Target officials told us that they are taking preventive actions to avoid similar anomalies in the future.

The time needed to complete the first 2007 intercept delayed GMD’s second planned intercept attempt until at least the second quarter of fiscal year 2008. The delayed test was to have determined whether the SBX radar could provide data in “real time” that could be used by the GMD fire control component to develop a weapon task plan. Although the weapon task plan was not developed in real time during 2007, GMD was able to demonstrate that the SBX radar could plan an engagement when the target was live but the interceptor was simulated.

Three BMDS Elements Completed Activities as Planned

During 2007, the KEI program redefined its development efforts and focused on near-term objectives. Also, the MKV program redefined its strategy to acquire multiple kill capability. Once redefined, these programs conducted all planned activities as scheduled and each was able to meet all planned objectives. In addition, the Sensors program successfully completed all planned tests.

In June 2007, MDA directed the KEI program to focus on two near-term objectives—the development of its booster and its 2008 booster flight test. Some work, such as development of the fire control and communications and mobile launcher, was deferred into the future. During fiscal year 2007, the KEI program conducted all planned test activities, including booster static fire tests that demonstrated the rocket motor’s performance in induced environments and wind tunnel tests that gathered data to validate aerodynamic models for the booster flight controls.

MKV officials redefined their acquisition strategy by employing a parallel path to develop multiple kill vehicles for the GMD and KEI interceptors and the Aegis BMD SM-3 missile. MDA initiated the MKV program in 2004 with Lockheed Martin. In 2007, the MKV program added Raytheon as a second payload provider. According to program officials, the two payload providers may use different technologies and design approaches, but both adhere to the agency’s goal of delivering common, modular MKV payloads for integration with all BMDS midcourse interceptors. In fiscal year 2007, Lockheed Martin successfully conducted static fire tests of its Divert Attitude Control System as planned. Additionally, Raytheon, funded with excess KEI funds made available when that program was replanned, began concept development. Raytheon did not have any major test activities scheduled for the fiscal year.

During 2007, the Sensors program focused on testing FBX-T radars that were permanently emplaced and newly produced. After the first FBX-T was moved from its temporary location in Japan to its permanent location in Shariki, Japan, various ground tests and simulations were conducted to ensure its interoperability with the BMDS. The program also delivered a second FBX-T to Vandenberg Air Force Base, where its tracking capability is being tested against targets of opportunity. According to program officials, a decision has not been made as to where the second FBX-T radar will be permanently located.

BMDS Achievement of Performance Goals Remains Uncertain

As we reported in March 2007, MDA altered its original Block 2006 performance goals commensurate with the agency’s reductions in the delivery of fielded assets. However, insufficient data exists to fully assess whether MDA achieved its revised performance goals. The performance of some fielded assets is also questionable because parts have not yet been replaced that were identified by auditors in MDA’s Office of Quality, Safety, and Mission Assurance as less reliable or inappropriate for use in space. In addition, tests of the GMD element have not included target suite dynamic features and intercept geometries representative of the operational environment in which GMD will perform its mission and BMDS tests only allow a partial assessment of the system’s effectiveness, suitability, and survivability.

MDA uses a combination of simulations and flight tests to determine whether performance goals are met. Models and simulations are needed to predict performance because the cost of tests prevents the agency from conducting sufficient testing to compute statistical probabilities of performance. The models and simulations that project BMDS capability against intercontinental ballistic missiles present several problems. First, the models and simulations that predict performance of the GMD element have not been accredited by an independent agency. According to the Office of the Director, Operational Test and Evaluation without accredited models, GMD’s performance cannot be predicted with respect to (1) variations in threat parameters that lie within the bounds of intelligence estimates, (2) stressing ground-based interceptor fly-outs and exoatmospheric kill vehicle engagements, and (3) variations in natural environments that lie within meteorological norms. Second, too few flight tests have been completed to ensure the accuracy of the models’ and simulations’ predictions. Since 2002, MDA has only completed two end-to- end tests of engagement sequences that the GMD element might carry out. While these tests provide some evidence that the element can work as intended, MDA must test other engagement sequences, which would include other GMD assets that have not yet participated in an end-to-end flight test. For example, MDA has not yet used the Sea-based X-band radar as the primary sensor in an end-to-end test. Additionally, officials in the Office of the Director, Operational Test and Evaluation told us that MDA needs more flight tests to have a high level of confidence that GMD can repeatedly intercept incoming ICBMs. Further testing is also needed to demonstrate that Aegis BMD can provide real-time, long-range surveillance and tracking data for the GMD element. In March 2006, we reported that the cancellation of a GMD flight test prevented MDA from exercising Aegis BMD’s long-range surveillance and tracking capability in a manner consistent with an actual defensive mission. Program officials informed us that the Aegis BMD is capable of performing this function and has demonstrated its ability to surveil and track ICBMs in several exercises. However, MDA has not yet shown that Aegis BMD can communicate this data to GMD during a live intercept engagement and that GMD can use the data to prepare a weapon task plan for actual— rather than simulated––interceptors. Officials in the Office of the Director for Operational Test and Evaluation told us that having Aegis BMD perform long-range surveillance and tracking during a live engagement would provide the data needed to more accurately gauge performance.

Similarly, MDA has not yet proved that the FBX-T radar can provide real- time, long-range surveillance and tracking data for the GMD element. On several occasions, MDA has shown that the FBX-T can acquire and track targets of opportunity, but the radar’s data has not yet been used to develop a weapon system task plan for a GMD intercept engagement. Because the radar’s permanent location in Japan does not allow MDA to conduct tests in which the FBX-T is GMD’s primary fire control radar, the Director, Operational Test and Evaluation, in 2006 recommended that prior to emplacing a second FBX-T at its permanent location that MDA test the radar’s capability to act as GMD’s primary sensor in an intercept test.

Confidence in the performance of the BMDS is also reduced because of unresolved GMD technical and quality issues. The GMD element has experienced the same anomaly during each of its flight tests since 2001. This anomaly has not yet prevented the program from achieving any of its primary test objectives, but to date neither its source nor solution has been clearly identified or defined. Program officials plan to continue their assessment of test data to identify the anomaly’s root cause and have implemented design changes to mitigate the effects and reduce risks associated with the anomaly. The reliability of emplaced GMD interceptors raises further questions about the performance of the BMDS. Quality issues discovered by auditors in MDA’s Office of Quality, Safety, and Mission Assurance nearly 3 years ago have not yet been rectified in all fielded interceptors. According to the auditors, inadequate mission assurance and quality control procedures may have allowed less reliable parts or parts inappropriate for use in space to be incorporated into the manufacturing process, thereby limiting the reliability and performance of some fielded assets. The program has strengthened its quality control processes and is taking several steps to mitigate similar risks in the future. These steps include component analysis of failed items, implementing corrective action with vendors, and analyzing system operational data to determine which parts are affecting weapon system availability. MDA has begun to replace the questionable parts in the manufacturing process and to purchase the parts that it plans to replace in fielded interceptors. However, it will not complete the retrofit effort until 2012.

Additionally, test officials told us that although the end-to-end GMD test conducted during 2007 demonstrated that for a single engagement sequence military operators could successfully engage a target, the target represented a relatively unsophisticated threat because it lacked specific target suite dynamic features and intercept geometry. Other aspects of the test were more realistic—such as closing velocity and fly-out range—but these were relatively unchallenging. While the test parameters may be acceptable in a developmental test, they are not fully representative of an operational environment and do not provide high confidence that GMD will perform well operationally.

Finally, because BMDS assets are being fielded based on developmental tests, which are not always representative of the operational environment, operational test officials have limited test data to determine whether all BMDS elements/components being fielded are effective and suitable for and survivable on the battlefield. MDA has added operational test objectives to its developmental test program, but many of the objectives are aimed at proving that military personnel can operate the equipment. In addition, limited flight test data is available for characterizing the BMDS’ capability against intercontinental ballistic missiles. Up until 2007, the overall lack of data limited the Office of the Director of Operational Test and Evaluation, in annual assessments, to commenting on the operational realism of tests and recommending other tests needed to characterize system effectiveness and suitability. In 2007, tests provided sufficient information to partially quantify the effectiveness and suitability of the BMDS' midcourse capability (Aegis BMD and GMD) and to fully characterize a limited portion of the BMDS' terminal capability (PAC-3). However, according to the Office of the Director of Operational Test and Evaluation, further testing that incorporates realistic operational objectives and verification, validation, and accreditation of models and simulations will be needed before the performance, suitability, and survivability of the BMDS can be fully characterized.

Efforts Underway to Improve BMDS Management and Oversight

Since its initiation in 2002, MDA has been given a significant amount of flexibility in executing the development of the BMDS. While the flexibility has enabled MDA to be agile in decision making and to field an initial capability relatively quickly, it has diluted transparency into MDA’s acquisition processes, making it difficult to conduct oversight and hold the agency accountable for its planned outcomes and costs. As we reported in 2007, MDA operates with considerable autonomy to change goals and plans, which makes it difficult to reconcile outcomes with original expectations and to determine the actual cost of each block and of individual operational assets. In the past year, MDA has begun implementing two initiatives—a new block construct and a new executive board–to improve transparency, accountability, and oversight. These initiatives represent improvements over current practices, although they provide for less oversight than statutes provide for other major defense acquisition programs. In addition, Congress has directed that MDA’s budget materials, after 2009, request funds using the appropriation categories of research, development, and evaluation, procurement, operations and maintenance, and military construction, which should promote accountability for and transparency of the BMDS.

New Block Construct Offers Improvements, but Does Not Address All Issues

In 2007, MDA redefined its block construct to better communicate its plans and goals to Congress. The agency’s new construct is based on fielding capabilities that address particular threats as opposed to the biennial time periods that were the agency’s past approach to development and fielding. MDA’s new block construct makes many positive changes. These include establishing unit cost for selected block assets, including in a block only those elements or components that will be fielded during the block, and abandoning the practice of deferring work from block to block.

Features of New Block Construct

Table 5 illustrates MDA’s new block construct for fielding the BMDS.

MDA’s new block construct provides a means for comparing the expected and actual unit cost of assets included in a block. As we noted in our fiscal year 2006 report, MDA’s past block structure did not estimate unit costs for assets considered part of a given block or categorize block costs in a manner that allowed calculations of expected or actual unit costs. For example, the expected cost of Block 2006 GMD interceptors emplaced for operational use was not separated from other GMD costs. Even if MDA had categorized the interceptors’ cost, it would have been difficult to determine the exact cost of these interceptors because MDA acquires and assembles components into interceptors over several blocks and it has been difficult to track the cost of components to a specific group of interceptors. Under the new block construct, MDA expects to develop unit costs for selected block assets—such as THAAD interceptors—and request an independent verification of that unit cost from DOD’s Cost Analysis Improvement Group. MDA will also track the actual unit cost of the assets and report significant cost growth to Congress. However, MDA has not yet determined for which assets a unit cost will be developed and how much a unit cost must increase before that increase is reported to Congress.

The new construct also makes it clearer as to which assets should be included in a block. Under the agency’s prior block construct, assets included in a given block were sometimes not planned for delivery until a later block. For example, as we reported in March 2007, MDA included costs for ABL and STSS as part of its Block 2006 cost goal although those elements did not field or plan to field assets during Block 2006. Agency officials told us those elements were included in the block because they believed the elements could offer some emergency capability during the block timeframe.

Finally, the new block construct should improve the transparency of each block’s actual cost. Under its prior construct, MDA deferred work from one block to another; but it did not track the cost of the deferred work so that it could be attributed to the block that it benefited. For example, MDA deferred some work needed to characterize and verify the Block 2004 capability until Block 2006 and counted the cost of those activities as a cost of Block 2006. By doing so, it understated the cost of Block 2004 and overstated the cost of Block 2006. Because MDA did not track the cost of the deferred work, the agency was unable to adjust the cost of either block to accurately capture the cost of each. MDA officials told us that under its new block construct, MDA will no longer transfer work, along with its cost, to a future block. Rather, a block of work will not be considered complete until all work that benefits a block has been completed and its cost has been properly attributed to that block.

Block Construct Does Not Resolve All Issues

Although improvements are inherent in MDA’s new block construct, the new construct will not dispel all transparency and accountability concerns. MDA has not yet estimated the full cost of a block. Also, MDA has not addressed whether it will transfer assets produced during a block to a military service for production and operation at the block’s completion, or whether MDA will continue its practice of concurrently developing and fielding BMDS elements and components.

According to its fiscal year 2009 budget submission, MDA does not plan to initially develop a full cost estimate for any BMDS block. Instead, when a firm commitment can be made to Congress for a block of capability, MDA will develop a budget baseline for the block. This budget will include anticipated funding for each block activity that is planned for the 6 years included in DOD’s Future Years Defense Plan. MDA officials told us that if the budget for a baselined block changes, MDA plans to report and explain those variations to Congress. At some future date, MDA does expect to develop a full cost estimate for each committed block and is in discussions with DOD’s Cost Analysis Improvement Group on having the group verify each estimate; but documents do not yet include a timeline for estimating block cost or having that estimate verified. For accountability, other DOD programs are required to provide the full cost of developing and producing their weapon system before system development and demonstration can begin. Until the cost of a block of BMDS capability is fully known, it will be difficult for decision makers to compare the value of investing in a block of BMDS capability to the value of investing in other DOD programs or to determine whether the block of capability that is being initiated will be affordable over the long term.

The new block construct does not address whether the assets included in a block will be transferred at the block’s completion to a military service for production and operation. Officials representing multiple DOD organizations recognize that the transfer criteria established in 2002 are neither complete nor clear given the BMDS’s complexity. Without clear transfer criteria, MDA has transferred the management of only one element—the Patriot Advanced Capability-3—to the military for production and operation. Joint Staff officials told us that for all other elements, MDA and the military services have been negotiating the transition of responsibilities for the sustainment of fielded elements—a task that has proven arduous and time consuming. Although MDA documents show that under its new block construct the agency should be ready at the end of each block to deliver BMDS components that are fully mission-capable, MDA officials could not tell us when MDA’s Director will recommend that management of components, including production responsibilities, be transferred to the military. MDA officials maintain that even though a particular configuration of a weapon could be fully mission- capable, that configuration may never be produced because it could be replaced by a new configuration. Yet, by the block’s end, a transfer plan for the fully mission-capable configuration will have been drafted, developmental ground and flight tests will be complete, elements and components will be certified for operations, and doctrine, organization, training, material, leadership, personnel, and facilities are expected to be in place.

Another issue not addressed under MDA’s new block construct is whether the concurrent development and fielding of BMDS elements and/or components will continue. Fully developing a component or element and demonstrating its capability prior to production increases the likelihood that the product will perform as designed and can be produced at the cost estimated. To field an initial capability quickly, MDA accepted the risk of concurrent development and fielding during Block 2004. For example, by the end of Block 2004, the agency realized that the performance of some Ground-based interceptors could be degraded because the interceptors included inappropriate or potentially unreliable parts. MDA has begun the process of retrofitting these interceptors, but work will not be completed until 2012. Meanwhile there is a risk that some interceptors might not perform as designed. MDA also continued to accept this risk during Block 2006 as it fielded assets before they were fully tested. MDA has not addressed whether it will accept similar performance risks under its new block construct or whether it will fully develop and demonstrate all elements/components prior to fielding.

New Executive Board Offers Improved, but Not Full, Oversight

In March 2007, the Deputy Secretary of Defense established a Missile Defense Executive Board (MDEB) to recommend and oversee implementation of strategic policies and plans, program priorities, and investment options for protecting the United States and its allies from missile attacks. The MDEB was also to replace existing groups and structures, such as the Missile Defense Support Group (MDSG). However, while it has some oversight responsibilities, the MDEB was not established to provide full oversight of the BMDS program and it would likely be unable to carry out this mission even if tasked to do so. The MDEB will not receive some information that the Defense Acquisition Board relies upon to make program recommendations, and in other cases, MDA does not plan to seek the MDEB’s approval before deciding on a course of action. In addition, there are parts of the BMDS program for which there will be no baseline against which progress can be measured, which makes oversight difficult.

Functions of the New Board

According to its charter, the MDEB is vested with more responsibility than its predecessor, the MDSG. When the MDSG was chartered in 2002, it was to provide constructive advice to MDA’s Director. However, the Director was not required to follow the advice of the group. According to a DOD official, although the MDSG met many times initially, it did not meet after June 2005. This led, in 2007, to the formation of the MDEB. This board’s mission is to review and make recommendations on MDA’s comprehensive acquisition strategy to the Deputy Secretary of Defense. It is also to provide the Under Secretary of Defense, Acquisition, Technology and Logistics, with a recommended strategic program plan and a feasible funding strategy based on “business case” analysis that considers the best approach to fielding integrated missile defense capabilities in support of joint MDA and warfighter objectives The MDEB will be assisted by four standing committees. These committees, which are chaired by senior-level officials from the Office of the Secretary of Defense and the Joint Staff, could play an important oversight role as they are expected to make recommendations to the MDEB, which in turn will recommend courses of action to the Under Secretary of Defense for Acquisition, Technology, and Logistics (USD AT&L) and the Director, MDA, as appropriate. The following table identifies the chair of each standing committee as well as key committee functions.

The MDEB will not have access to all information normally available to oversight bodies. For other major defense acquisition programs, the Defense Acquisition Board must approve the program’s progress through the acquisition cycle. Further, before a program can enter the System Development and Demonstration phase of the cycle, statute requires that certain information be developed. This information is then provided to the Defense Acquisition Board. However, in 2002, the Secretary of Defense allowed MDA to defer application of the defense acquisition system that among other things require programs to follow a defined acquisition cycle and obtain approval before advancing from one phase of the cycle to another. Because MDA does not follow this cycle, it does not enter System Development and Demonstration and it does not trigger the statutes requiring the development of information that the Defense Acquisition Board uses to inform its decisions. For example, most major defense acquisition programs are required by statute to obtain an independent verification of program cost prior to beginning system development and demonstration, and/or production and deployment. Independent life- cycle cost estimates provide confidence that a program is executable within estimated cost and along with other DOD-wide budget demands. Although MDA plans to develop unit cost for selected block assets and request that DOD’s Cost Analysis Improvement Group verify the unit costs, the agency does not initially plan to develop a block cost estimate and, therefore, cannot seek an independent verification of that cost. In addition, even when MDA estimates block costs, the agency will not be required to obtain an independent verification of that cost, because, as noted earlier, the BMDS program operates outside of DOD’s acquisition cycle. Although not required, MDA officials told us that they have initiated discussions with the Cost Analysis Improvement Group on independent verifications of block cost estimates.

Statute also requires an independent verification of a system’s suitability for and effectiveness on the battlefield before a program can proceed beyond low-rate initial production. After the test is completed, the Director for Operational Test and Evaluation assesses whether the test was adequate to support an evaluation of the system’s suitability and effectiveness for the battlefield, whether the test showed the system to be acceptable, and whether any limitations in suitability and effectiveness were noted. However, a comparable assessment of the BMDS assets being produced for fielding will not be available to the MDEB. As noted earlier, the limited amount of testing completed, which has been primarily developmental in nature, and the lack of verified, validated, and accredited models and simulations prevent the Director of Operational Test and Evaluation from fully assessing the effectiveness, suitability, and survivability of the BMDS in annual assessments.

MDA will also make some decisions without approval from the MDEB or any higher level DOD official. Although the charter of the MDEB includes the mission to make recommendations to MDA and the Under Secretary of Defense for AT&L on investment options, program priorities, and MDA’s strategy for developing and fielding an operational missile defense capability, the MDEB will not have the opportunity to review and recommend changes to BMDS blocks. According to a briefing on the business rules and processes for MDA’s new block structure, the decision to initiate a new block of BMDS capability will be made by MDA’s Director. Also cost, schedule, and performance parameters will be established by MDA when technologies that the block depends upon are mature, a credible cost estimate can be developed, funding is available, and the threat is both imminent and severe. The Director will inform the MDEB as well as Congress when a new block is initiated, but he will not seek the approval of either.

Finally, there will be parts of the BMDS program that will be difficult for the MDEB to oversee because of the nature of the work being performed. MDA plans to place any program that is developing technology in a category known as Capability Development. These programs, such as ABL, KEI, and MKV, will not have a firm cost, schedule, or performance baseline. This is generally true for technology development programs in DOD because they are in a period of discovery, which makes schedule and cost difficult to estimate. On the other hand, the scale of the technology development in BMDS is unusually large, ranging from $2 billion to about $5 billion dollars a year—eventually comprising nearly half of MDA’s budget by fiscal year 2012. The MDEB will have access to the budgets planned for these programs over the next 5 or 6 years, each program’s focus, and whether the technology is meeting short-term key events or knowledge points. But without some kind of baseline for matching progress with cost, the MDEB will not know how much more time or money will be needed to complete technology maturation. MDA’s experience with the ABL program provides a good example of the difficulty in estimating the cost and schedule of technology development.

In 1996, the ABL program believed that all ABL technology could be demonstrated by 2001 at a cost of about $1 billion. However, MDA now projects that this technology will not be demonstrated until 2009 and its cost has grown to over $5 billion. While the uncertainties of technology development must be recognized, some organizations suggest ways to establish a baseline appropriate for such efforts. For example, the Air Force Research Laboratory suggested a methodology to estimate a technology’s cost once analytical and laboratory studies physically validate analytical predictions of separate elements of the technology.

MDEB Could Play Key Role In Proposal to Return BMDS to Standard Requirements Process

In an effort to further improve oversight, the Joint Requirements Oversight Council proposed a plan to transition the BMDS into standard DOD processes. In August 2007, the Vice Chairman of the Joint Chiefs of Staff and Joint Requirements Oversight Council Chairman requested the Deputy Secretary of Defense approve a proposal to return MDA to the Joint Capabilities Integration and Development System process and direct the Joint Requirements Oversight Council to validate BMDS capabilities. The Vice Chairman believed that the council should exercise oversight of MDA in order to improve Department-wide capability integration. More specifically, he noted that: In 2002, the Secretary of Defense exempted the BMDS program from the traditional requirements generation process to expedite fielding the system as soon as practicable.

Now that an initial capability for homeland defense has been deployed, there is no longer the same need for flexibility provided by the requirements exemption.

The current process, with MDA exempted, does not allow the Joint Requirements Oversight Council to provide appropriate military advice or to validate missile defense capabilities. Without this change, there is increasing potential that MDA-fielded systems will not be synchronized with other air and missile defense capabilities being developed.

The current process hinders the military departments’ ability to plan and program resources for fielding and sustainment of MDA-developed systems.

In responding to the proposal, the Acting Under Secretary of Defense for AT&L recommended that the Deputy Secretary of Defense delay his approval of the Joint Staff’s proposal until the MDEB could review the proposal and provide a recommendation. However, he agreed that more Joint Requirements Oversight Council involvement was necessary for the BMDS, although he was not sure that returning BMDS to standard DOD processes was the appropriate solution to the agency’s oversight issues. Instead, he noted that the Deputy Secretary of Defense recently established the MDEB to recommend and oversee the implementation of strategic policies and plans, program priorities, and investment options for the BMDS. He stated that since the MDEB is tasked with determining the best means of managing the BMDS throughout its life cycle, it should consider the Joint Staff’s proposal.

MDA Directed to Use Procurement Funding for the First Time

In an effort to improve the transparency of MDA’s acquisition processes, Congress has directed that MDA’s budget materials delineate between funds needed for research, development, and evaluation, procurement, operations and maintenance, and military construction. Using procurement funds will mean that MDA generally will be required to adhere to congressional policy that assets be fully funded in the year of their purchase, rather than incrementally funded over several years. The Congressional Research Service reported in 2006 that “incremental funding fell out of favor because opponents believed it could make the total procurement costs of weapons and equipment more difficult for Congress to understand and track, create a potential for DOD to start procurement of an item without necessarily stating its total cost to Congress, permit one Congress to ‘tie the hands’ of future Congresses, and increase weapon procurement costs by exposing weapons under construction to uneconomic start-up and stop costs.” Our analysis of MDA developed costs, which are presented in table 7, also shows that incremental funding is usually more expensive than full funding, in part, because inflation decreases the buying power of the dollar each year.

The National Defense Authorization Act for Fiscal Year 2008 directed MDA to submit a plan to transition from using research and development funds exclusively to using procurement, operations and maintenance, military construction, and research and development funds by March 1, 2008. However, it allowed MDA to continue to use research and development funds in fiscal year 2009 to incrementally fund previously approved missile defense assets. The act also directed that beginning in fiscal year 2009, the MDA budget request include, in addition to RDT&E funds, military construction funds and procurement funds for some long lead items such as those required for the third and fourth THAAD fire units and Aegis BMD SM-3 Block 1A missiles. MDA did not request long lead funding for either THAAD or SM-3 missiles in its fiscal year 2009 budget because MDA has slipped the schedule for procuring fire units 3 and 4 by one year, and the National Defense Authorization Act for Fiscal Year 2008 was not signed in time to allow MDA to adjust its budget request for SM-3 missiles.

Congress also provided MDA with the authority to use procurement funds for fiscal years 2009 and 2010 to field its BMDS capabilities on an incremental funding basis, without any requirement for full funding. Congress has granted similar authority to other DOD programs. In the conference report accompanying the Fiscal Year 2008 National Defense Authorization Act, the conferees indicated that if MDA wishes to use incremental funding after fiscal year 2010, DOD must request additional authority for a specific program or capability. Conferees cautioned DOD that additional authority will be considered on a limited case-by-case basis and that future missile defense programs will be funded in a manner more consistent with other DOD acquisition programs.

Since 2002, MDA has been granted the flexibility to incrementally fund the fielding of its operational assets with research and development funds. In some cases, the agency spreads the cost of assets across 5 to 7 budget years. After reviewing the agency’s incremental funding plan for future procurements of THAAD fire units and Aegis BMD missiles, we analyzed the effect of fully funding these assets using present value techniques and found that the agency could save about $125 million by fully funding their purchase and purchasing them in an economical manner. Our analysis is provided in table 7. In addition, more detailed analysis is available in appendix III.

According to our analysis, fully funding the THAAD and Aegis BMD assets will, in all instances, save MDA money. For example, full funding would save the THAAD program approximately $104 million and the Aegis BMD program nearly $22 million. In addition, by providing funds upfront, the contractors should be able to arrange production in the most efficient manner.

Conclusions

By the end of Block 2006, MDA posted a number of accomplishments for the BMDS, including fielding more assets, conducting several successful tests, and progressing with developmental efforts. As a result, fielded capability has increased. On the other hand, some problems continue that make it difficult to assess how well the BMDS is progressing relative to the funds it has received and the goals it has set for those funds. First, under the proposed block construct, MDA plans to develop a firm baseline for each block and have it independently reviewed. However, MDA has not yet developed estimates for full block costs, so the initial baseline incorporates the budget for each block only through DOD’s Future Years Defense Plan. Second, while MDA expects to estimate unit costs and track increases, it is unclear as to what criteria will be used for reporting variances to Congress. Third, while MDA has gotten some contractors to lower the portion of work planned as level of effort, a substantial amount of work remains so planned. Fourth, while it may not be reasonable to expect the same level of accountability for technology development efforts as it is for development and production of systems, the high level of investment—up to half of its budget—MDA plans to make in technology development warrants some mechanism for reconciling the cost of these efforts with their progress. Finally, MDA fields assets before development testing is complete and without conducting operational testing. We have previously recommended that MDA return to its original non-concurrent, knowledge-based approach to developing, testing, and fielding assets. Short of that, the developmental testing that is done provides the primary basis for the Director of Operational Test and Evaluation to assess whether a block of BMDS capability is suitable and effective for the battlefield. So far, BMDS testing has not yielded sufficient data to make a full assessment.

Recommendations for Executive Action

To build on efforts to improve the transparency, accountability, and oversight of the missile defense program, we recommend that the Secretary of Defense direct: MDA to develop a full cost for each block and request an independent verification of that cost; MDA to clarify the criteria that it will use for reporting unit cost MDA to examine a contractor’s planning efforts when 20 percent or more of a contract’s work is proposed as level of effort; MDA to investigate ways of developing a baseline or some other standard against which the progress of technology programs may be assessed; and MDA and the Director of Operational Test and Evaluation to agree on criteria and incorporate corresponding scope into developmental tests that will allow a determination of whether a block of BMDS capability is suitable and effective for fielding.

Agency Comments and Our Evaluation

DOD provided written comments on a draft of this report. These comments are reprinted in appendix I. DOD also provided technical comments, which we incorporated as appropriate.

DOD concurred with three of our five recommendations—developing a full cost estimate for each block and requesting an independent verification of that cost, clarifying criteria for reporting unit cost variances to Congress, and examining contractors’ planning efforts when 20 percent or more of a contract’s work is proposed as level of effort. The Department indicated that MDA has already taken steps to develop new cost models aligned with its new block structure and met with DOD’s Cost Analysis Improvement Group to initiate the planning process for the independent verifications of MDA’s cost estimates. The cost estimates will extend until block completion and will not be limited by a 6-year Future Years Defense Plan window. MDA is also working to establish criteria for reporting unit cost variances and to incorporate them into an MDA directive. Finally, MDA has made a review of prime contractors’ work planning efforts part of the Integrated Baseline Review process and the Defense Contract Management Agency has agreed to continuously validate the appropriateness of each contractor’s planning methodology as part of its ongoing contract surveillance.

DOD partially concurred with our recommendation that MDA investigate ways of developing a baseline or some other standard against which the progress of technology programs may be assessed. DOD observed that MDA uses knowledge points, technology readiness levels, and engineering and manufacturing readiness levels in assessing the progress of its technology programs and that it will continue to investigate other methods of making such assessments. While we recognize their value, these methods typically assess progress in the short term and do not provide an estimate of the remaining cost and time needed to complete a technology program. Because MDA must balance its efforts to improve the existing BMDS while developing new capability, DOD and MDA need to ensure that only the most beneficial technology programs in terms of performance, cost, and schedule are pursued. This will require an understanding of not only the benefit to be derived from the technology, but also an understanding of the cost and time needed to bring the technology to fruition.

DOD also partially concurred with our last recommendation that MDA and the Director of Operational Test and Evaluation (DOT&E) agree on criteria and additional scope for developmental tests that will allow a full determination of the effectiveness and suitability of a BMDS block for fielding. DOD noted that it is MDA’s mission to work with the warfighter, rather than DOT&E, to determine that the BMDS is ready for fielding, but that MDA will work closely with DOT&E to strengthen the testing of BMDS suitability and effectiveness. We agree that DOT&E is not responsible for fielding decisions, but its mission is to ensure that weapon systems are realistically and adequately tested and that accurate evaluations of operational effectiveness, suitability, and survivability are available for production decisions. MDA improved the operational realism of testing in 2007 and for the first time DOT&E considered tests at least partially adequate to make an assessment of the BMDS. However, a full assessment is not yet possible and we continue to recommend that MDA and DOT&E take steps to make as full a BMDS evaluation as possible. In doing so, MDA and DOT&E can work cooperatively to reduce the number of unknowns that will confront the warfighter when the system is required operationally and improve the likelihood that the BMDS will perform as needed in the field.

We are sending copies of this report to the Secretary of Defense and to the Director, MDA. We will make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you, or your staff have any questions concerning this report, please contact me at (202) 512-4841. Contact Points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report. The major contributors are listed in appendix V.

Appendix I: Comments from the Department of Defense

Appendix II: MDA Contracts

Overview of MDA Prime and Support Contracts

The Missile Defense Agency (MDA) employs prime contractors and support contractors to accomplish different tasks that are needed to develop and field the ballistic missile defense system. Prime contractors receive the bulk of funds MDA requests each year and work to provide the hardware and software for elements of the Ballistic Missile Defense System (BMDS). Support contractors provide a wide variety of useful services, such as special knowledge and skills not available in the government and the capability to provide temporary or intermittent services.

Prime Contracts

MDA has prime contracts with four defense companies—Boeing, Raytheon, Lockheed Martin, and Northrop Grumman—to develop elements of the BMDS. All current contracts and agreements are cost reimbursement type that provide for payment of reasonable, allowable, and allocable incurred costs to the extent provided in the contract or agreement. The contracts also provide fee for the contractor performing the work, but the amount earned depends on many variables, including the type of cost contract, contractor performance, technical risk, and complexity of the requirement. All of the cost reimbursement contracts used for the BMDS elements include cost plus award fee aspects. Cost plus award fee contracts provide for a fee consisting of a base fee—fixed at the inception of the contract that may be zero—and an award amount based upon a subjective evaluation by the government, meant to encourage exceptional performance. It should be noted that some award fee arrangements include objective criteria such as Key Performance events.

The Multiple Kill Vehicle (MKV) contract and Command, Control, Battle Management and Communications (C2BMC) Other Transaction Agreement differ somewhat from the other elements' contracts. The MKV prime contractor was awarded an indefinite delivery/indefinite quantity cost reimbursement contract. This type of contract allows MDA to order services as they are needed through a series of task orders. Without having to specify a firm quantity of services (other than a minimum or maximum quantity), the government has greater flexibility to align the tasks with available funding. The C2BMC element operates under an Other Transaction Agreement with cost reimbursement aspects. These types of agreements are not always subject to procurement laws and regulations meant to safeguard the government. MDA chose the Other Transaction Agreement to facilitate a collaborative relationship between industry, government, federally funded research and development centers, and university research centers.

DOD requires that all contractors awarded cost reimbursement contracts or other agreements of $20 million or greater implement an Earned Value Management System (EVMS) to integrate the planning of work scope, schedule, and resources, and to provide insight into their cost and schedule performance. To implement this system, contractors examine the totality of the work directed by the contract and break it into executable work packages. Each work package is assigned a schedule and a budget that is expected to enable the work’s completion. On a monthly basis, the contractor examines initiated work packages to determine whether the work scheduled for the month was performed on time and within budget. If more work was completed than scheduled and the cost of the work performed was less than budgeted, the contractor reports a positive schedule and cost variance. However, if the contractor was unable to complete all of the work scheduled and needed more funds to complete the work than budgeted, the contractor reports a negative schedule and cost variance. Of course, the results can be mixed. That is, the contractor may have completed more work than scheduled but at a cost that exceeded the budget. The contractor details its performance to MDA each month in Contract Performance Reports. These reports also identify the reasons that negative or positive variances are occurring. Used properly, the earned value concept allows program managers to identify problems early so that steps can be taken before the problems increase the contract’s overall cost and/or schedule.

In the course of subdividing the total work of the contract into smaller efforts, contractors plan work according to its type. Included in these classifications are discrete work—work that is expected to produce a product, such as a study, lines of software code, or a test—and work considered to be level of effort (LOE). LOE is work that does not result in a product, but is of a general or supportive nature. Supervision and contract administration are examples of work that do not produce definable end products and are appropriately planned as LOE.

Some BMDS Contracts Have a High Percentage of Level of Effort Work

Several contracts for BMDS systems have relatively high proportions of work planned as LOE. When work is incorrectly planned as LOE, the contractor’s performance becomes less transparent because earned value does not recognize schedule variances for such work. Rather, it is assumed that the time budgeted for an LOE effort will produce the intended result. Although an LOE work package will report cost variances, those variances will only be measured against how much the program intended to spend at certain time intervals. If LOE were to be used on activities that could otherwise be measured discretely, the project performance data could be favorably distorted and contractors and program managers might not be able to discern the value gained for the time spent on the task. Specifically, the program’s Contract Performance Reports would not indicate whether or not the work performed produced the product expected. By losing early insight into performance, the program could potentially need to spend more time and money to complete the task.

Since earned value management is less suited for work that is not intended to produce a specific product, or work that is termed LOE, the Standard for Earned Value Management Systems Intent Guide instructs that although some amount of LOE activity may be necessary, it must be held to the lowest practical level. In addition, earned value experts such as Defense Contract Management Agency officials agree that if a contractor plans more than 20 percent of the total contract work as LOE, the work plan should be examined to determine if work is being properly planned.

Although the amount of LOE should be minimized, some BMDS prime contracts have a relatively high percentage of LOE. As figure 2 illustrates, the MKV contractor planned much of the work for task orders open during fiscal year 2007 as LOE. Contractors for Aegis BMD SM-3 and C2BMC also planned a high percentage of their work as LOE. Both MDA’s Earned Value Management Group and program office reviewers encouraged the SM-3 and C2BMC contractors to reduce their LOE percentages. By the end of the fiscal year, the SM-3 and C2BMC contractors had reduced the amount of work planned as LOE.

In December 2006, the Aegis BMD SM-3 contractor completed work to develop and produce initial Block 1A missiles with 73 percent of this work categorized as LOE—well above the 15 percent that the Aegis BMD SM-3 program reports as its industry standard. Although we have reported that the contractor completed this segment of work ahead of cost but slightly behind schedule, it is difficult to assess whether this represents the contractor’s actual performance. The high percentage of LOE associated with this work may have limited our assessment and distorted whether the work completed was in all respects the work planned. Subsequently, the contractor initiated procurement of long lead materials to produce an additional 20 Block 1A missiles before work packages were developed. Once work packages were developed, only 18 percent of the work was planned as LOE.

The C2BMC program was able to reduce the percentage of work planned as LOE, but the program continues to encourage further reductions. During fiscal year 2007, the C2BMC contractor replanned its work and reduced the amount of work planned as LOE from 73 to 52 percent. This change was implemented after two closely related reviews suggested the percentage of LOE work was too high. Both the program office and its contractor acknowledge the high level of LOE and have made plans to limit it in future work.

As noted in figure 2, the MKV contractor considered all work being completed under two task orders—Task Orders 4 and 5—as LOE. The primary objective of Task Order 4 is to update the program plan and complete the systems engineering effort necessary to integrate the MKV warhead into the BMDS to the extent required for the systems requirements review. Both the system concept review, completed in July 2006, and the system requirements review, scheduled for December 2008, are major milestones. However, the contractor did not plan these milestone reviews as products. According to program officials, Task Order 4 will be reevaluated in February 2008 to reduce the amount of LOE and recognize more work as discrete. The MKV program also planned 100 percent of Task Order 5 work as LOE. Under this task order, the contractor was to design a prototype propulsion system, assemble and integrate the hardware for the prototype, and perform a static hot fire test of the integrated system. This effort culminates in hardware—a tangible end product—that is expected to exhibit certain performance characteristics during the static hot fire test. The contractor could have categorized this task order, at least in some part, as discrete work since the work was expected to deliver a defined product with a schedule that could slip or vary. Because the contractor categorized all of this task order as LOE, the program lost its ability to gauge performance and to make adjustments that might prevent contract cost growth.

Prime Contractors Exceed Fiscal Year 2007 Budgeted Costs but Make Gains on Schedule

We analyzed Fiscal Year 2007 Contract Performance Reports for MDA’s 10 prime contracts and determined that collectively the contractors overran budgeted costs by nearly $170 million but were ahead of schedule by nearly $200 million. However, the percentage of work planned as LOE should be scrutinized before accepting this as the contractors’ actual performance because a high percentage of LOE, as noted above, can potentially distort the contractors’ cost and schedule performance. The cumulative performance of one contractor is also distorted because it rebaselined part of its work. Rebaselining is an accepted EVM procedure that allows a contractor to reorganize all or part of its remaining contract work, add additional time or budget for the remaining effort, and, under some circumstances, set affected cost and/or schedule variances to zero. When variances are set to zero, the cumulative performance of the contractor appears more positive than it is.

Four of the 10 contracts we reviewed also contained some kind of replanning activity during fiscal year 2007. Contractors may replan when they conclude that the current plan for completing the effort remaining on the contract is unrealistic. A replan can consist of any of the following: reallocating the budget for the remaining effort within the existing constraints of the contract, realigning the schedule within the contractually defined milestones, and setting cost and/or schedule variances to zero. During the course of replanning a contract, the contractor must provide traceability to previous baselines as well as ensure that available funding is not exceeded.

Aegis BMD Contractors Perform within Budgeted Costs

The Aegis BMD program awarded two prime contracts for its major components, the Aegis BMD Weapon System and the Standard Missile-3. During the fiscal year, the contractors completed all work at less cost than budgeted. Both contractors ended the year with positive cumulative cost variances, but negative cumulative schedule variances. Based on our analysis, we project that if the contractors continue to perform at the same level, the weapon system contractor could underrun its budget by between $8.8 million and $17.7 million, while the SM-3 contractor could complete its work on 20 Block 1A missiles for $7.4 million to $11.1 million less than budgeted.

The weapon system contractor’s fiscal year 2007 cost performance resulted in a positive cost variance of $7.7 million. The positive variance was realized as two software packages required less effort than anticipated and were completed earlier than expected. Combined with its performance from earlier periods, the contractor finished the year with a cumulative positive cost variance of $7 million. This upward trend is depicted in figure 3.

The contractor produced a $3.8 million unfavorable schedule variance in fiscal year 2007. The contractor reported that the unfavorable cumulative variance was caused in part by a delay in receiving component materials for the radar’s processor.

During fiscal year 2007, the Aegis SM-3 contractor closed out work related to missile development and initial production of Block 1A missiles and began new work in February 2007 to manufacture an additional 20 Block 1A missiles. In performing the new work, the contractor underran its cost budget by $6.2 million, but failed to complete $4.0 million of planned work. The Aegis BMD SM-3 contractor’s cumulative cost and schedule variances are highlighted in figure 4.

The positive cost variance can be attributed to several factors including cost efficiencies realized from streamlining system engineering resources and lower than planned hardware costs. Our analysis predicts that if the SM-3 contractor continues to perform as it did through September 2007, it will underrun its budgeted costs for the 20 Block 1A missiles by between $7.4 million and $11.1 million. The contractor’s negative cumulative schedule variance of $4 million for the 20 missiles was primarily caused by delayed qualification testing and integration of hardware components.

Despite Rebaselining, ABL’s Cumulative Cost and Schedule Performance Remain Negative

In May 2007, MDA allowed ABL’s contractor to rebaseline one part of its contract after the work associated with a key knowledge point could not be completed on schedule. Because the contractor did not achieve this knowledge point as planned, the program was forced to postpone its lethality demonstration until August 2009. Technical issues including weapon system integration, beam control/fire control software modifications, and flight testing discoveries, all contributed to the delay in completing the knowledge point for the program. To provide funds and time to support the delay in the lethality demonstration, the program extended the contract’s period of performance by approximately 1 year and increased the contract’s ceiling cost by $253 million. Once the new baseline was incorporated, the contractor was able to complete fiscal year 2007 with positive cost and schedule variances of $3.7 million and $24.2 million, respectively. Figure 5 depicts the contractor’s cumulative cost and schedule performance.

As shown in figure 5 above, the ABL contractor was not able to overcome the negative cost and schedule variances of prior years and ended the fiscal year with an unfavorable cumulative cost variance of $74.2 million and an unfavorable cumulative schedule variance of $25.8 million. We estimate that, at completion, the contract could overrun its budget by between $95.4 million and $202.5 million.

C2BMC Program Incurs Negative Cumulative and Fiscal Year Variances

During fiscal year 2006, the C2BMC contractor did not report earned value because it was working on a replan of its Block 2006 increment of work (known as Part 4). Following the definitization of the Part 4 replan in November 2006, the C2BMC contractor resumed full EVM reporting with the first submittal covering February 2007 data. As part of the replan, the contractor adjusted a portion of its Part 4 work and set cost and schedule variances to zero in an effort to establish a baseline commensurate with the contractor’s replanning efforts. However, even with the adjustment, the C2BMC program ended fiscal year 2007 with negative fiscal year cost and schedule variances of $11.1 million and $1.5 million, respectively. Figure 6 shows the contractor’s cumulative performance in fiscal year 2007.

The unfavorable fiscal year cost variance was largely due to adding staff to support a software release; while the unfavorable fiscal year schedule variance was attributable to delays in hardware delivery, initiation of a new training system, and completing training material for the new system. Added to prior year negative variances, the C2BMC contractor reported cumulative negative cost and schedule variances of $14.5 million and $3.5 million, respectively. The contractor completed Part 4 work in December 2007 and reported an overrun of $9.9 million.

GMD Contractor Maintained Negative Cost and Schedule Variances Throughout the Fiscal Year

The GMD prime contractor’s cost performance improved significantly in fiscal year 2007. The contractor experienced a budget overrun of $22.1 million for the fiscal year following budget overruns in both fiscal years 2005 and 2006 that exceeded $300 million. Program officials attribute this turnaround in performance to several factors, including rigorous management of the contract’s estimate at completion, quality initiatives, and joint efforts by the contractor and program office to define scope, schedule, and price of change orders. The cumulative cost variance at the end of fiscal year 2007 was over $1 billion. We estimate that at completion the contract, with a target price of $15.54 billion, could exceed its budgeted cost by between $1.06 billion and $1.4 billion.

The contractor was able to complete $84.9 million more work than scheduled for fiscal year 2007, but could not overcome poor performance in earlier years and ended the year with a negative cumulative schedule variance of $52.9 million. Figure 7 illustrates both cost and schedule trends in GMD fiscal year 2007 performance.

The unfavorable fiscal year cost variance is primarily attributable to the EKV. During fiscal year 2007, the EKV contractor experienced negative cost variances as it incurred additional labor costs to recover delivery schedules, manufacturing schedule delays, hardware manufacturing problems, and embedded software development and system integration problems. With 18 percent of the EKV work remaining, the negative trends on this component could continue.

As we reported last year, the contractor was in the process of developing a new contract baseline to incorporate the updated scope, schedule, and budget that the contractor was working toward. In September 2006, phase one of the new baseline, covering fiscal year 2006-2007 efforts, had been implemented and validated through the Integrated Baseline Review of the prime contractor and its major subcontractors. Phase two of the review was completed in December 2006. Subsequent to the reviews, fiscal year 2007 ground and flight tests were replanned to reflect a contract change that added additional risk mitigation effort to one planned flight test and added a radar characterization system test.

KEI Replan Significantly Affects Cost and Schedule Variances

The KEI contractor replanned its work in April 2007 when MDA directed the program to focus, in the near term, on two main objectives: booster vehicle development and the 2008 booster flight developmental test. Prior to the replan, the KEI program was developing a land-mobile capability with fire control and communications and mobile launcher components. Although the contractor’s primary objectives are now focused around the booster segment of work, it is still performing some activities related to the fire control and communications component. During fiscal year 2007, the contractor incurred a positive cost variance of $2.1 million and a negative schedule variance of $7.5 million. Combined with variances from earlier fiscal years, the cumulative cost variance is a positive $5.7 million and the cumulative schedule variance is a negative $12.8 million. Figure 8 illustrates KEI’s cumulative performance over the course of the fiscal year.

KEI’s fiscal year favorable cost variance primarily results from completing work on the fire control and communications component, as well as systems engineering and integration with fewer staff than planned. We were unable to estimate whether the total contract is likely to be completed within budgeted cost since the contract is only 10 percent complete and trends cannot be developed until at least 15 percent of the contract is completed.

Work related to the interceptor’s booster and systems engineering and integration contributed to KEI’s cumulative negative fiscal year schedule variance of $7.5 million. The contractor reports that the booster work was understaffed, which caused delays in finalizing designs that, in turn, delayed procurement of subcomponents and materials and delayed analysis and tests. While the reduction in staff for systems engineering and integration work reduced costs for the contractor, it also delayed completion of the weapon system’s scheduled engineering, flight, and performance analysis products.

Information Not Available to Assess All MKV Work

We could evaluate only two of five MKV task orders open during fiscal year 2007 because the contractor did not report sufficient earned value data to make an assessment of the other three meaningful. MDA awarded the MKV contract in January 2004 and has since initiated eight task orders through its indefinite delivery/indefinite quantity contract. During fiscal year 2007, the program worked on five of these task orders—Task Orders 4, 5, 6, 7, and 8. We evaluated the contractor’s cost and schedule performance for Task Orders 5 and 6 only. Of the three task orders that we did not evaluate, the contractor began reporting full earned value on two so late in the fiscal year that little data was available for analysis. In the third case, the contractor’s reports did not include all data needed to make a cost and schedule assessment.

In June 2006, MDA issued Task Order 5 which directed the design, assembly, and integration of the hardware for a prototype propulsion system, and a static hot fire test of the integrated prototype. Because the contractor planned all activities for this task order as level of effort, the contractor reported zero schedule variance. Contract Performance Reports show that in preparation for the hot fire test in August 2007, the program discovered anomalies indicative of propellant contamination in the prototype’s propulsion system. These anomalies led to multiple unplanned propellant tank anomaly investigations, which contributed to the unfavorable $2.3 million cost variance for the fiscal year. Additionally, during the hot fire test, one of the thrusters in the propulsion system’s divert and attitude control component experienced anomalies due to foreign object contamination. This anomaly led to unplanned investigations which also contributed to increased costs. Figure 9 below depicts the unfavorable cumulative cost variance of $2.7 million and cumulative schedule variance of zero reported for Task Order 5. Based on our analysis, we predict the contractor will overrun its contract costs by between $2.6 million and $2.9 million.

MKV’s objective for Task Order 6 is to manufacture a prototype seeker capable of acquiring, tracking, and discriminating objects in space. The program plans to demonstrate the prototype seeker, which is a component of a carrier vehicle, through testing in 2009. In contrast to Task Order 5, the contractor correctly planned the bulk of Task Order 6 as discrete work and has been reporting the work’s cost and schedule status since March 2007. During this time, the contractor has completed 37 percent of the work directed by the task order at $0.3 million less than budgeted. The contractor was also able to complete $0.9 million more work than planned. See Figure 10 for an illustration of cumulative cost and schedule variances for this task order.

The program attributes its favorable fiscal year cost and schedule variances for Task Order 6 to the early progress made on interface requirements, hardware procurements, component drawings, and the prototype seeker’s architecture. Because detailed designs for the seeker are derived from models, the program is anticipating some rework will be needed as the designs are developed, processed, and released. Although program officials are expecting some degradation in cumulative cost and schedule variances to occur, the program does not expect an overrun of the contract’s budgeted cost at completion. Based on the contractor’s performance to date, we predict, at contract completion, the contractor will underrun costs by between $0.8 million and $2.5 million.

Sensors’ FBX-T Contractor Remains Ahead of Cost and on Schedule

The Sensors contractor’s performance during fiscal year 2007 resulted in a positive cost variance of $3.9 million and an unfavorable schedule variance of $8.8 million. Added to variances from prior years, the contractor is reporting cumulative positive cost and schedule variances of $24.1 million and $17.8 million, respectively. The contractor’s performance in 2007, suggests that at completion the contract will cost from $22.0 million to $46.8 million less than budgeted. The variances, depicted below in figure 11 represent the Sensors contractor’s cumulative cost and schedule performance over fiscal year 2007.

The contractor has reported favorable schedule and cost variances since the contract’s inception because the program was able to leverage the hardware design of the THAAD radar to reduce development timelines and it implemented manufacturing efficiencies to reduce manufacturing costs. However, during fiscal year 2007, the contractor experienced a negative schedule variance as it struggled to upgrade software expected to provide an increased capability for the FBX-T radar.

STSS Contractor’s Cumulative Cost and Schedule Variances Remain Unfavorable

After replanning a portion of its work in October 2006, the STSS contractor in fiscal year 2007 experienced an unfavorable cost variance of $67.7 million and a favorable schedule variance of $84.7 million. Combined with performance from earlier periods, the contractor is reporting cumulative negative cost and schedule variances of $231.4 million and $19.7 million, respectively. Figure 12 shows both cost and schedule trends during fiscal year 2007.

During the fiscal year, the contractor was able to accomplish a significant amount of work ahead of schedule after a replan added additional time for planned work efforts. However, the contractor was unable to overcome the negative schedule variances incurred in prior years.

Delays in hardware and software testing as well as integration issues contributed to fiscal year 2007’s negative cost variance. We did not estimate the cost of the STSS contract at completion. The contract includes not only the effort to develop and launch two demonstration satellites (the Block 2006 capability) but also effort that will benefit future blocks. Block 2006 work is about 86 percent complete, while work on future blocks is about 16 percent complete.

THAAD Contractor Improves Schedule, but Costs Continue to Be a Problem

The THAAD contractor overran its fiscal year 2007 budgeted costs by $91.1 million but accomplished $19.0 million more work than scheduled. Cumulatively, the contractor ended the year with an unfavorable cost variance of $195.2 million and a negative schedule variance of $9.1 million, as shown by figure 13.

The THAAD prime contractor’s cost overrun of $91.1 million was primarily caused by technical problems related to the element’s missile, launcher, radar, and test components. Missile component cost overruns were caused by higher than anticipated costs in hardware fabrication, assembly, and support touch labor as well as subcontractor material costs for structures, propulsion, and other sub-assembly components. Additionally, design issues with the launcher’s missile round pallet and the electronics assembly that controls the launcher caused the contractor to experience higher than anticipated labor and material costs. More staff than planned was required to resolve hardware design issues in the radar’s prime power unit, causing the radar component to end the fiscal year with a negative cost variance. The contractor also experienced negative cost variances with the system test component because the Launch and Test Support Equipment required additional set-up time at the flight test range.

THAAD’s prime contractor fared better in performing scheduled work. It was able to reduce its negative cumulative schedule variance over the course of the fiscal year because subcontracted missile items were delivered early and three flight tests were removed from the test program to accommodate target availability and budget constraints, allowing staff more time to work on current efforts.

The contractor projects an overrun of $174 million at contract completion, while we estimate that the overrun could range from $227.2 million to $325.8 million. To achieve its projection, the contractor needs to complete $1.04 worth of work for every dollar spent. In contrast, during fiscal year 2007, the contractor achieved an average of $0.82 worth of work for each dollar spent. Therefore, it seems unlikely that the contractor will be able to achieve its estimate at completion.

MDA Relies on Award Fees to Motivate Contractor Performance

Like other DOD programs, MDA has not always effectively used award fees to encourage contractors toward exceptional performance but it is making efforts to revise its award fee policy to do so. Over the course of fiscal year 2007, the agency sometimes rolled over large percentages of award fee—in most cases for work that was moved to later periods but also for one contractor that exhibited poor performance. In addition, some award fee plans allow fee to be awarded to contractors for merely meeting the requirements of their contract. For two contractors, MDA awarded fee amounts that were linked to very good or outstanding work in the cost and/or program management performance elements. During their award fee periods, the contractors’ earned value data showed declines in cost and/or schedule variances, although there are several other factors considered when rating contract performance. However, in June 2007, MDA issued a revised draft of its award fee guide in an effort to more closely link the amount of award fees earned with the level of contractor performance.

In an effort to encourage its defense contractors to perform in an innovative, efficient, and effective way in areas considered important to the development of the BMDS, MDA offers its contractors the opportunity to collectively earn billions of dollars through monetary incentives known as award fees. Award fees are intended to motivate exceptional performance in subjective areas such as technical ingenuity, cost, and schedule. Award fees are appropriate when contracting and program officials cannot devise predetermined objective targets applicable to cost, technical performance, or schedule.

Currently, all 10 of the contracts we assessed for BMDS elements utilize award fees in some manner to incentivize their contractor’s performance. Each element’s contract has an award fee plan that identifies the performance areas to be evaluated and the methodology by which those areas will be assessed. At the end of each period, the award fee evaluation board, made up of MDA personnel, program officials, and officials from key organizations knowledgeable about the award fee evaluation areas, begins its process. The board judges the contractor’s performance and recommends to a fee determining official the amount of fee to be paid. For all BMDS prime contracts we assessed, the fee determining official is the MDA Director. Table 1 provides a summary of the award fee process.

DOD Has Not Linked Award Fees to Acquisition Outcomes

GAO has found in the past that DOD has not always structured and implemented award fees in a way that effectively motivates contractors to improve performance and achieve acquisition outcomes. Specifically, GAO cited four issues with DOD’s award fee processes. GAO reported that in many evaluation periods when rollover—the process of moving unearned available award fee from one evaluation period to the next—was allowed, the contractor had the chance to earn almost the entire unearned fee, even in instances when the program was experiencing problems. Additionally, DOD guidance and federal acquisition regulations state that award fees should be used to motivate excellent contractor performance in key areas. However, GAO found that most DOD award fee contracts were paying a significant portion of the available fee from one evaluation period to the next for what award fee plans describe as “acceptable, average, expected, good, or satisfactory” performance. Furthermore, DOD paid billions of dollars in award fees to programs whose costs continued to grow and schedules increased by many months or years without delivering promised capabilities to the warfighter. GAO also found that some award fee criteria for DOD programs were focused on broad areas— such as how well the contractor was managing the program—instead of criteria directly linked with acquisition outcomes—such as meeting cost and schedule goals, and delivering desired capabilities. All of these DOD practices contribute to the difficulty in linking elements of contractor performance considered in award fee criteria to overall acquisition outcomes and may lessen the motivation for the contractor to strive for excellent performance.

Similar Problems Found in MDA Award Fee Practices

We assessed all award fee plans for the BMDS elements and fiscal year 2007 award fee letters for 9 of the 10 contractors. Our review revealed that during 2007 MDA experienced some of the same award fee problems that were prevalent in other DOD programs. MDA did not roll fee forward often, but when it did the contractor was, in one case, able to earn 100 percent of that fee. Also, MDA allowed another contractor to earn the unearned portion of fiscal year 2007 award fee in the same period through a separate pool composed of the unearned fee but tied to other performance areas. In two other instances, MDA awarded fee amounts that were linked to very good or outstanding work in the cost and/or program management performance element. However, during the award fee periods, earned value data indicates that these two contractors’ cost and/or schedule performance continued to decline. Although DOD guidance discourages use of earned value performance metrics in award fee criteria, MDA includes this as a factor in several of its award fee plans. MDA considers many factors in rating contractors’ performance and making award fee determinations, including considerations of earned value data that shows cost, schedule, and technical trends. Table 9 provides the award fee MDA made available to its contractors, as well as the fee earned during fiscal year 2007.

MDA is awarding some BMDS contractors a large percentage of the fees rolled over from a prior period. The agency’s award fee plans allow the fee determining official, at his discretion, to rollover all fee that is not awarded during one period to a future period. For example, in accordance with MDA’s award fee policy, the fee determining official may consider award fee rollover when a slipped schedule moves an award fee event to another period, it is the desire of the fee determining official to add greater incentive to an upcoming period, and when the contractor improves performance to such a great extent that it makes up for previous shortfalls. During fiscal year 2007, MDA rolled fee forward for 3 of the 8 contractors for which award fee letters were available. Table 10 presents a synopsis of this data.

As noted in table 10, MDA rolled over a large percentage of the fee that was not earned by the THAAD contractor during fiscal year 2007. During its last award fee period in fiscal year 2007, the THAAD contractor did not earn any of the fee associated with cost management. The award fee letter cited unfavorable cost variances and a growing variance projected at completion of the contract as the reasons for not awarding any of the fee for cost management. However, the fee determining official decided to roll 100 percent of that portion of the unearned fee to a rollover pool tied to minimizing cost overruns. Fee will be awarded from this pool at the end of the contract. By rolling the fee forward, MDA provided the contractor an additional opportunity to earn fee from prior periods. Rolling over fee in this instance may have failed to motivate the contractor to meet or exceed expectations.

The award fee plan for the GMD contract allowed the contractor to not only rollover fee, but to earn all unearned fee in the same period. During the fiscal year, the GMD contractor earned 97.7 percent of the $330 million dollars in award fees tied to performance areas outlined in the award fee plan. However, the award fee plan made provisions for the contractor to earn the unearned $7.5 million by creating a separate pool funded solely from this unearned portion and awarding the fee for performance in other areas. In this instance, the contractor did not have to wait to earn rolled over fees in later award fee periods—it was able to receive the unearned portion in the same period despite not meeting all of the criteria for its original objectives. GMD officials told us that this fee incentivized the contractor to achieve added objectives.

In contrast, the fee determining official handled rollover of fee on the ABL contract in accordance with DOD’s new policy. According to ABL’s award fee plan, MDA was to base its 2007 award fee decision primarily on the outcome of three knowledge points. During this period, the contractor completed two of the knowledge points, but could not complete a third. To encourage the contractor to complete the remaining knowledge point in a timely manner, the fee determining official rolled over only 35 percent of the fee available for the event.

MDA Award Fee Plans Allow Contractors to Earn Fee for Satisfactory or Lower Performance

All of the award fee plans we assessed allowed MDA to award fees for satisfactory ratings—that is, work considered to meet most of the requirements of the contract. Some award fee plans even allow fee for marginal performance or performance considered to meet some of the requirements of the contract. By paying for performance at the minimum standards or requirements of the contract, the intent of award fees to provide motivation for excellence above and beyond basic contract requirements is lost. While the definitions of satisfactory or marginal differed from element to element, the award fee plans allotted roughly more than 50 percent award fee to contractors performing at these levels. According to the award fee plans, MDA allows between 51 and 65 percent of available fee for work rated as marginal for the C2BMC and KEI contractor and no less than 66 percent of available fee for satisfactory performance by the ABL contractor. MDA’s practice of allowing more than 50 percent of available fee for satisfactory or, even, marginal performance illustrates why DOD in April 2007 directed that no more than 50 percent of available fee be given for satisfactory performance on all contract solicitations commencing after August 1, 2007.

Not All Award Fees Reflect Performance as Reported by the Earned Value Management System

Earned value is one of several factors that according to the award fee plans for the ABL and Aegis BMD Weapon System contractors will be considered in rating the contractors’ cost and program management performance. During a good part of fiscal year 2007, earned value data for both contractors showed that they were overrunning their fiscal year cost budgets. In addition, the ABL contractor was not completing all scheduled work. Even considering these variances, MDA presented the contractors with a significant portion of the award fee specifically tied to cost and/or program management. In contrast, the THAAD contractor also experienced downward trends in its cost variance during its last award fee period in fiscal year 2007, but was not paid any of the award fee tied to cost management.

The ABL and Aegis BMD Weapon System contractors received a large percentage of the 2007 award fee available to them for the cost and/or program management element. According to ABL’s award fee plan, one of several factors that is considered in rating the contractor’s performance as “very good” is whether earned value data indicates that there are few unfavorable cost, schedule, and/or technical variances or trends. During the award fee period that ran from February 2006 to January 2007, MDA rated the contractor’s cost and program management performance as very good and awarded 88 percent of the fee available for these areas of performance. Yet, earned value data indicates that the contractor overran its budget by more than $57 million and did not complete $11 million of planned work. Similarly, the Aegis BMD weapon system contractor was to be rated in one element of its award fee pool as to how effectively it managed its contract’s cost. Similar to ABL’s award fee plan, the weapon system contractor’s award fee plan directs that earned value data be one of the factors considered in evaluating cost management. During the fee period that ran from October 2006 through March 2007, MDA rated the contractor’s performance in this area as outstanding and awarded the contractor 100 percent of the fee tied to cost management. Earned value data for this time period indicates that the contractor overran its budget by more than $6 million. MDA did not provide us with more detailed information as to other factors that may have influenced its decision as to the amount of fee awarded to the ABL and Aegis BMD contractors.

In another instance, MDA more closely linked earned award fee to contractor performance. The THAAD contractor continued to overrun its 2007 cost budget, and was not awarded any fee tied to the cost management element during its last award fee period in fiscal year 2007. The award fee decision letter cites several examples of the contractor’s poor cost performance including cost overruns and an increased projected cost variance at contract completion. These and other cost management issues led the fee determining official to withhold the $9.8 million to be awarded on the basis of cost management.

MDA Is Making Changes to Its Award Fee Policy

MDA has made efforts to comply with DOD policy regarding some of GAO’s recommendations and responded to the DOD issued guidance by releasing its own revised award fee policy in February 2007. According to the policy, every contract’s award fee plan is directed to include: a focus on developing specific award fee criteria for each element of an emphasis on rewarding results rather than effort or activity, and an incentive to meet or exceed MDA requirements.

Additionally, the directive calls for using the Award Fee Advisory Board, established to make award fee recommendations to the fee determining official, to biannually review and report to the Director on the consistency between MDA’s award fees and DOD’s Contractor Performance Assessment Report—which provides a record, both positive and negative, on a given contract for a specific period of time. MDA’s directive also requires program managers to implement MDA’s new award fee policy at the earliest logical point, which is normally the beginning of the next award fee period.

MDA is currently constructing a revised draft of its award fee guide that addresses the rollover and rating scale issues from DOD’s March 2006 and April 2007 memorandums. In the latest draft, MDA limits rollover to exceptional cases and adopts the Under Secretary’s limitation of making only a portion of award fee available for rollover. MDA’s latest draft of the guide also makes use of the latest ratings scale, referencing the Under Secretary’s April 2007 direction, and applies the usage of the new scales to contract solicitations beginning after July 31, 2007.

MDA Tightens Controls on Unpriced Changes and Task Orders

MDA sometimes finds that events such as funding changes, technology advances, and concurrent development and deployment of the BMDS arise that make changes to the contract’s provisions or terms necessary. MDA describes contract changes that are within the scope of the contract but whose final price, or cost and fee, the agency and its contractor have not agreed upon as unpriced changes. MDA has followed the FAR in determining how quickly the agency should reach agreement on such unpriced changes’ price, or cost and fee. According to the FAR, an agreement should be reached before work begins if it can be done without adversely affecting the interest of the government. If a significant cost increase could result from the unpriced change, and time doesn’t permit negotiation of a price, the FAR requires the negotiation of a maximum price unless it is impractical to do so. In 2007, MDA began applying tighter limits on definitization of price.

MDA also issues unpriced task orders. MDA uses this term to describe task orders issued under established contract ordering provisions, such as an indefinite delivery/indefinite quantity contract, for which a definitive order price has not yet been agreed upon. MDA has followed the FAR requirements that task orders placed under an indefinite delivery/indefinite quantity contract must contain, at least, an estimated cost or fee.

During Block 2006—January 1, 2006 through December 31, 2007—MDA authorized 137 unpriced changes and task orders with a value of more than $6 billion. Consistent with the FAR requirements noted above, of the total 137 unpriced changes and unpriced task orders, 61 percent of these— totaling $5.9 billion—were not priced for more than 180 days. Agreement on the price of several was not reached for more than a year, and agreement on the price of one was not reached for more than two and a half years. Table 11 below shows the value of unpriced changes and task orders issued on behalf of each BMDS element and the number of days after the contractor was authorized to proceed with the work before MDA and its contractor agreed to a price, or cost and fee, for the work.

Realizing that unpriced changes and unpriced task orders may greatly reduce the government’s negotiation leverage and typically result in higher cost and fee for the overall effort, MDA, in February 2007, issued new contract guidance that required tighter limits on the timeframes for reaching agreement on price, or cost and fee. The agency now applies some of the Defense Federal Acquisition Regulation Supplement guidelines established for undefinitized contract actions to unpriced changes and unpriced task orders. Undefinitized contract actions are different from MDA’s unpriced changes or unpriced task orders in that they are contract actions on which performance is begun before agreement on all contract terms, including price, or cost and fee, is reached. A contract modification or change will not be considered an undefinitized contract action if it is within the scope and under the terms of the contract. MDA has elected to follow some of the stricter undefinitized contract action guidelines because the agency believes the guidelines will lead to better cost results. Similar to the undefinitized contract action guidelines, the agency’s new guidelines require that MDA’s unpriced changes and unpriced task orders be definitized within 180 days, that the contractor be given a dollar value that it cannot exceed until price agreement is reached, and that approval for the unpriced change or task order be obtained in advance. MDA’s new policy also, to the maximum extent practicable, limits the amount of funds that a contractor may be given approval to spend on the work before agreement is reached on price to less than 50 percent of the work’s expected price.

Support Contracts

MDA officials maintain that support contracts provide necessary personnel and are instrumental in developing the BMDS quickly. The agency contracts with 45 different companies that provide the majority of the personnel who perform a variety of tasks. Table 12 illustrates the broad categories of job functions that MDA support contractors carry out.

Last year we reported that MDA had 8,186 approved personnel positions. This number has not changed appreciably in the last year. According to MDA’s manpower database, about 8,748 personnel positions—not counting prime contractors—currently support the missile defense program. These positions are filled by government civilian and military employees, contract support employees, employees of federally funded research and development centers (FFRDC), researchers in university and affiliated research centers, as well as a small number of executives on loan from other organizations. MDA funds around 95 percent of the total 8,748 positions through its research and development appropriation. Of this 95 percent, 2,450, or about 29 percent, of the positions are set aside for government civilian personnel. Another 60 percent, or 5,005 positions, are allotted for support contractors. The remaining 11 percent are positions either being filled, or expected to be filled, by employees of FFRDCs and university and affiliated research centers that are on contract or under other types of agreements to perform missile defense tasks. MDA officials noted that nearly 500 of the 8,748 personnel positions available were currently vacant. Table 13 shows the staffing levels within the BMDS elements.

Support contractors in MDA program and functional offices may perform tasks that closely support those tasks described in the FAR as inherently governmental. According to the FAR, tasks such as determining agency policy and approving requirements for prime contracts should only be performed by government personnel. Contract personnel that, for example, develop statements of work, support acquisition planning, or assist in budget preparation are carrying out tasks that may closely support tasks meeting this definition. Having support contractors perform these tasks may create a potential risk that the contractors may influence the government’s control over and accountability for decisions. MDA officials told us that when support contractors perform tasks that closely support those reserved for government employees the agency mitigates its risk by having knowledgeable government personnel provide regular oversight or final approval of the work to ensure that the data being generated is reasonable.

Appendix III: Incremental Funding

Incremental Versus Full-funding of BMDS Assets

In the tables below we provide more information comparing the cost of purchasing THAAD and Aegis BMD assets incrementally versus fully- funding the assets. Table 14 presents MDA’s incremental funding plans for THAAD fire units 3 and 4, 48 Aegis BMD (SM-3) missiles to be produced during Blocks 2012 and 2014, and 19 shipsets intended to improve the performance of Aegis BMD ships. Tables 15 through 17 present our analysis of the cost of purchasing these same assets with procurement funds and following Congress’ full-funding policy.

Appendix IV: Scope and Methodology

To examine the progress MDA made in fiscal year 2007 toward its Block 2006 goals, we examined the accomplishments of nine BMDS elements. The elements included in our review collectively accounted for 77 percent of MDA’s fiscal year 2007 research and development budget request. We evaluated each element’s progress in fiscal year 2007 toward Block 2006 schedule, testing, performance, and cost goals. In assessing each element we examined Program Execution Reviews, test plans and reports, production plans, Contract Performance Reports, and MDA briefing charts. We developed data collection instruments that were completed by MDA and each element program office. The instruments gathered detailed information on completed program activities including tests, prime contracts, and estimates of element performance. To understand performance issues, we talked with officials from MDA’s Deputy for Engineering and Program Director for Targets and Countermeasures, each element program office, as well as the office of DOD’s Director, Operational Test and Evaluation. To assess each element’s progress toward its cost goals, we reviewed Contract Performance Reports and, when available, the Defense Contract Management Agency’s analyses of these reports. We applied established earned value management techniques to data captured in Contract Performance Reports to determine trends and used established earned value management formulas to project the likely costs of prime contracts at completion. We also interviewed MDA officials within the Deputy for Acquisition Management office to gather detailed information regarding BMDS prime contracts. We reviewed 10 prime contracts for the 9 BMDS elements and also examined fiscal year 2007 award fee plans, award fee letters, and gathered data on the number of and policy for unpriced changes and unpriced task orders. We became familiar with sections of the Federal Acquisition Regulation and Defense Federal Acquisition Regulation Supplement dealing with contract type, contract award fees, and undefinitized contract actions. To develop data on support contractors, we held discussions with officials in MDA’s Office of Business Operations. We also collected data from MDA’s Pride database on the numbers and types of employees supporting MDA operations.

In assessing MDA’s accountability, transparency, and oversight, we interviewed officials from the Office of the Under Secretary of Defense’s Office for Acquisition, Technology, and Logistics and Joint Staff officials. We also examined a Congressional Research Service report, U.S. Code, DOD acquisition system policy, the MDEB Charter, and various MDA documents related to the agency’s new block structure.

In determining whether MDA would save money if it fully funded THAAD and Aegis BMD assets rather than funding them incrementally, we used present value techniques to restate dollars that MDA planned to expend over a number of years to the equivalent number of dollars that would be needed if MDA fully funded the assets in the fiscal year that incremental funding was to begin. We also considered whether MDA would need to acquire long lead items for the assets and stated those dollars in the base year that their purchase would be required. We then compared the total cost of incrementally funding the assets, as shown in MDA’s funding plans, to the fully funded cost that our methodology produced.

To ensure that MDA-generated data used in our assessment are reliable, we evaluated the agency’s management control processes. We discussed these processes with MDA senior management. In addition, we confirmed the accuracy of MDA-generated data with multiple sources within MDA and, when possible, with independent experts. To assess the validity and reliability of prime contractors’ earned value management systems and reports, we interviewed officials and analyzed audit reports prepared by the Defense Contract Audit Agency. Finally, we assessed MDA’s internal accounting and administrative management controls by reviewing MDA’s Federal Manager’s Financial Integrity Report for Fiscal Years 2003, 2004, 2005, 2006, and 2007.

Our work was performed primarily at MDA headquarters in Arlington, Virginia. At this location, we met with officials from the Aegis Ballistic Missile Defense Program Office; Airborne Laser Program Office; Command, Control, Battle Management, and Communications Program Office; BMDS Targets Office, and MDA’s Agency Operations Office. We also met with DOD’s Office of the Director, Operational Test and Evaluation and the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics in Washington, DC. In addition, in Huntsville, Alabama, we met with officials from the Ground-based Midcourse Defense Program Office, the Terminal High Altitude Area Defense Project Office, the Kinetic Energy Interceptors Program Office, the Multiple Kill Vehicle Program Office, and BMDS Tests Office. We also met with Space Tracking and Surveillance System officials in Los Angeles, California.

We conducted this performance audit from May 2007 to March 2008 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix V: GAO Contact and Staff Acknowledgments

Acknowledgments

In addition to the individual named above, Barbara Haynes, Assistant Director; LaTonya Miller; Sigrid McGinty; Michele R. Williamson; Michael Hesse; Steven Stern; Meredith Allen Kimmett; Kenneth E. Patton; and Alyssa Weir made key contributions to this report.