Missile Defense: Actions Being Taken to Address Testing Recommendations, but Updated Assessment Needed
Released: 2004-03-11T00:00:00

Letter

Missile Defense Agency Actions Taken or Planned to Address DOT&E Recommendations

The August 2000 DOT&E report summarized the progress, up to that date, of the National Missile Defense program and the adequacy of testing in the context of a deployment decision. At the time, the development program revolved around a series of ground and flight tests and was to have culminated in an initial operational capability by the end of fiscal year 2005. Formal test documentation called for a total of 16 integrated flight tests (system-level intercept attempts) through 2004 with three additional flight tests during Initial Operational Test and Evaluation during the 2005 fiscal year. DOT&E’s principal finding was that ground and flight tests completed up to that time did not provide results of sufficient fidelity to support a deployment decision. Indeed, when the deployment readiness review was held, there had been two failed intercepts out of three attempts. Furthermore, as stated in the DOT&E report, ground testing was not adequate to yield credible estimates of GMD system performance. DOT&E indicated that the current test program required augmentation and probably significant funding increases to demonstrate an operationally effective system for deployment. Accordingly, the report included a list of detailed recommendations for enhancing the test program.

DOT&E made 50 specific, interrelated recommendations, which we organized into the following four overarching categories: Flight Testing, Ground Testing, Target Discrimination, and Programmatics. Although DOT&E categorized discrimination-related recommendations under the flight-testing and ground-testing categories, we created a separate category because discrimination was of principal concern to DOT&E at the time. DOD classified the full text of the recommendations. A detailed assessment indicating whether actions have been initiated by MDA and what their timing is relative to the September 2004 initial defensive capability date can be found in our June 2003 classified report on this subject. A summary of MDA actions to address the DOT&E recommendations is provided below.

DOT&E Recommendations on Flight Testing

Integrated flight tests of the GMD element are demonstrations of system performance during which an interceptor is launched to engage and intercept a target reentry vehicle (mock warhead) above the atmosphere.

Many recommendations (20 of 50) in the DOT&E report pertain to aspects of integrated flight testing, such as deficiencies in flight-test complexity, operational realism, and artificialities. DOT&E’s concerns with the composition of target suites in flight tests for testing discrimination are discussed separately in the discrimination section of this report. DOT&E reported that increasing the scope of flight testing was essential to stress the limits of system design and to keep pace with system development.

MDA is taking actions that address many of the shortcomings in flight testing DOT&E identified in its August 2000 report. Indeed, the development of the BMDS Test Bed—the agency’s key instrument for enhancing the existing test infrastructure to provide more realistic testing—should go far in addressing these DOT&E recommendations over the long term. Currently, flight tests are limited to target launches out of Vandenberg Air Force Base, California, and interceptor launches out of Kwajalein Missile Range in the western Pacific. For enhancing the capabilities of integrated flight testing, the test bed adds an interceptor launch site at Vandenberg Air Force Base; target launch facilities at Kodiak Launch Complex, Alaska; a GMD fire control node at Fort Greely, Alaska; an upgraded early warning radar at Beale Air Force Base, California; upgraded communication links among test bed components; and test infrastructure to support five additional intercept regions. The ship-based Aegis AN/SPY-1 radar is also available as a forward-deployed asset for early target tracking. In addition, the design and construction of a sea-based X-band radar, which would be positioned on a mobile platform in the Pacific, has been funded by MDA and is scheduled to be available for test bed utilization in late 2005. Other components of the BMDS Test Bed such as the Cobra Dane radar in Shemya, Alaska, and interceptors at Fort Greely will not actively participate in integrated flight tests at least through September 2007.

Flight Test Complexity

Several August 2000 DOT&E recommendations call for integrated flight testing with Category B engagements and scenarios with multiple threatening reentry vehicles, both of which are expected to be common during operational missions. In addition, the recommendations call for integrated flight testing to be performed under increasingly difficult conditions and to be made more challenging through, for example, testing under various solar and weather conditions. Our analysis of the GMD test program as it pertains to flight test complexity, based on the March 2003 Developmental Master Test Plan for the GMD element and related program documentation, is summarized below.

Flight Test Complexity—Actions Taken or Planned. The GMD test plan calls for Category B engagements beginning with Integrated Flight Test 15 (IFT-15), scheduled for the fourth quarter of fiscal year 2004. Furthermore, it indicates that Category B engagements would be a common occurrence of flight testing, because the weapon task plan would be generated from Beale or Aegis radar data. According to MDA officials, however, the decision to conduct future flight tests under Category B engagements is currently under review; the resolution will depend on the individual flight test scenario and the maturity of battle management assets. The GMD Developmental Master Test Plan also shows that an integrated flight test (designated IFT-22/23) in which two interceptors are launched against two attacking reentry vehicles (multiple simultaneous engagements) will be carried out in fiscal year 2007.

Flight Test Complexity—Actions Not Taken or Planned. Although previous flight tests have been conducted under limited adverse conditions (light rain), flight tests to assess the actual effects of severe weather on system performance are not currently planned. According to the program office, the verification of system performance in adverse weather will be achieved through modeling and simulation grounded in technical measurements and flight test data. Furthermore, a nighttime engagement was attempted during IFT-10 (December 2002), but the failure of the kill vehicle to separate from the surrogate booster precluded collection of any applicable data.

Operational Realism

The recommendations on operational realism reflect limitations of the current test range. Currently, intercept tests are constrained to a single corridor and intercept region—target launches out of Vandenberg Air Force Base and interceptor launches out of the Reagan Test Site. As a result, flight-test engagement conditions are limited to those with low closing velocities and short interceptor fly-out ranges. DOT&E called for an expansion of engagement conditions and suggested adding more intercept regions and launch locations to achieve new intercept geometries, higher closing velocities, and longer ranges flown by the interceptor during flight testing.

Operational Realism—Actions Taken or Planned. The expansion of the test range in the Pacific with the development of the BMDS Test Bed will have a significant impact on achieving operational realism in integrated flight tests. The Block 2004 Test Bed adds five intercept regions, target launches out of Kodiak Launch Complex, and interceptor launches out of Vandenberg Air Force Base. The combination allows for flight tests with new intercept geometries, additional crossing angles, higher closing velocities, and longer ranges flown by the interceptor. For example, IFT-15 (fourth quarter of fiscal year 2004) will be conducted with a target launch out of Kodiak, and IFT-17 (fourth quarter of fiscal year 2005) will be the first test with an interceptor launched from Vandenberg.

Operational Realism—Caveats. The principal caveat to the associated MDA actions addressing operational realism is timing. By September 2004, one of the five new intercept regions, north of Reagan Test Site, will have been exercised. The remaining new intercept regions will not be exercised until after September 2004. For example, the two intercept regions off the west coast of the United States will be used in IFT-17 (fourth quarter of fiscal year 2005) and IFT-18 (fourth quarter of fiscal year 2005), respectively. A fourth intercept point will be exercised in IFT-21 (third quarter of fiscal year 2006). Finally, the fifth intercept point will be exercised as part of the multiple simultaneous engagement to be conducted in fiscal year 2007.

Artificialities

The DOT&E recommendations on flight test artificialities—such as the removal of surrogates (test range assets emulating operational assets)— also reflect limitations of the current test range. The most artificial surrogate noted in the August 2000 DOT&E Report was the placement of a C-band transponder on the target reentry vehicle. The transponder was essential for the execution of flight tests, because in conjunction with the test range radar (designated FPQ-14), there were no other non-artificial options available to track the reentry vehicle with sufficient accuracy for executing the mission. DOT&E argued that this artificiality be phased out and, in general, recommended the system utilized in integrated flight tests be as functional and representative as possible.

Artificialities—Actions Taken or Planned. Use of the transponder/FPQ-14 radar combination as a surrogate radar for midcourse tracking is planned to be phased out. Indeed, IFT-15 (fourth quarter of fiscal year 2004) would be the first test that does not use this surrogate for mission execution. Rather, in integrated flight tests IFT-15 and beyond, midcourse tracking of the target suite would be achieved through the use of the Beale upgraded early warning radar or, pending ongoing analysis by GMD, the Aegis SPY-1 radar. The eventual use of the sea-based X-band radar beginning in late 2005 can also be used for midcourse tracking. The removal of other surrogates is under way. For example, the short-range surrogate interceptor booster, which has been used in all flight tests to date, is scheduled to be replaced with two more operationally representative boosters beginning with IFT-14 (third quarter of fiscal year 2004).

Artificialities—Actions Not Taken or Planned. The MDA is not currently considering conducting flight tests under unrehearsed and unscripted conditions.

Elimination of Flight Tests

Overall, the current DOT&E has looked favorably on MDA’s actions that address its recommendations, because the GMD test infrastructure is being significantly enhanced to allow for more flight test complexity, operational realism, and artificialities. We noted, however, that since DOT&E’s August 2000 assessment, MDA has reduced the extent of the flight test program, as follows: Integrated Flight Tests—Number of Cancellations. During the initial planning phases of the revised test program, MDA considered conducting four intercept attempts per year. But after considerable planning and contract evaluations, MDA limited the flight test program to no more than three intercept attempts per year because of overlapping test objectives and funding constraints. Significantly, the previous GMD test program at the time of the deployment readiness review called for a total of 19 integrated flight tests to be carried out through fiscal year 2005. The current test program, however, now has a total of 12 integrated flight tests through fiscal year 2005—because of the cancellation of IFT-11, 12, and 16, and the conversion of IFT-13 to booster tests (IFT-13A and 13B). To date, 8 of the 12 have been completed under largely the same test conditions that were critically assessed by DOT&E. In short, only two flight tests under improved test conditions with more representative hardware are planned to be conducted before September 2004, the time at which the initial defensive capability is scheduled to become available.

Operational Testing—No Longer Required. The previous GMD test program also called for operational testing—Initial Operational Test and Evaluation—by the military services. Operational testing is a statutory requirement for DOT&E to independently determine operational effectiveness and suitability of a deployed system for use by the warfighter. MDA does not plan to operationally test the Block 2004 GMD element before it is available for initial defensive operations. The September 2004 fielding is not connected with a full-rate production decision that would clearly trigger statutory operational testing requirements. Nonetheless, the Combined Test Force, a group of users and developers, plans tests to incorporate both developmental and operational test requirements in the test program.

DOT&E’S Recommendations on Ground Testing

The 13 ground testing recommendations formulated by DOT&E in its August 2000 report are focused concerns encompassing four areas: (1) realistic testing of kill vehicle functions in a Hardware-in-the-Loop (HWIL) facility, (2) ground-based lethality testing, (3) development of the system-level simulation known as the Lead System Integrator Integration Distributed Simulation (LIDS), and (4) Operations in a Nuclear Environment (OPINE) testing of kill vehicle components. In general, DOT&E’s recommendations on ground testing are not being addressed.

Hardware-in-the-Loop Testing

A number of the August 2000 DOT&E ground testing recommendations pertain to the hardware-in-the-loop testing of the kill vehicle built by Raytheon. For example, a test article is placed in an evacuated chamber to simulate an exoatmospheric environment, and infrared radiation of a simulated target scene is projected onto the kill vehicle’s sensors. DOT&E recommended “that an innovative new approach needs to be taken towards hardware-in-the-loop testing of the kill vehicle, so that potential design problems or discrimination challenges can be wrung out on the ground in lieu of expensive flight tests.” DOT&E stated that, in order to verify kill vehicle performance, kill vehicle testing should be executed using actual unit hardware in a hardware-in-the-loop facility capable of providing a realistic space environment and threat scene. MDA had taken steps to proceed with the design and construction of a hardware-in-the-loop laboratory at the Arnold Engineering Development Center, Tullahoma, Tennessee. Although an initial test capability had been planned for the 2004 time frame, testing at the Arnold Engineering facility has been deferred beyond Block 2004 based on Test Bed funding constraints. In response to a draft of this report, MDA stated that future investments and test events at this facility are subject to MDA internal management trade-offs among the numerous priorities associated with the whole missile defense program portfolio.

Lethality Testing

DOT&E made recommendations in its August 2000 report for improving GMD lethality testing—testing aimed at assessing a kill vehicle’s effectiveness in destroying a reentry vehicle. Current test plans call for an approach whereby ground-based experiments are conducted to collect data to anchor simulations, which in turn are used to assess lethality performance. Indeed, GMD expects to anchor such simulations from data derived from improved “sled testing,” which uses full-scale targets in the newly developed Holloman Air Force Base Hypersonic Upgrade Program facility. However, there are no plans to conduct intercept flight tests of the interceptor’s ability to destroy threat representative targets that fulfill the Live Fire Test and Evaluation requirements. Rather, hit point information is collected from actual intercept tests, which, in turn, is used as input to simulations to determine whether the impact was lethal.

Another area of ground testing recommendations identified in the August 2000 DOT&E report concerned the development and use of system-level digital simulations. During the time of the deployment readiness review, the prime contractor’s principal tool for assessing system performance over a broad range of scenarios was the end-to-end digital simulation known as LIDS. Because the development of the simulation was behind schedule and unavailable to support analyses of overall system performance, DOT&E reported that results obtained from it should not be used in making a deployment decision. DOT&E recommended that LIDS capability be “evolved to a fully validated, high-fidelity simulation.” In addition, DOT&E recommended that LIDS be made flexible enough to permit independent use by test agencies. MDA disagrees with the recommendations pertaining to LIDS. MDA views LIDS as one of many tools to analyze performance aspects of the GMD element and does not believe that LIDS needs to be developed to the level expected by DOT&E. According to the agency, a baseline of models and simulations are available that are intended to collectively support the entire range of analysis required to verify the capabilities of the GMD elements. Furthermore, MDA asserts the evolution of LIDS from Software Build 4 to its current Software Build 6.1.0 has improved the flexibility of the system to allow for sensitivity analyses by government users. According to MDA, extensive analysis using LIDS has been conducted at the Joint National Integration Center at Shriever Air Force Base, Colorado.

OPINE Testing

Finally, the remaining ground testing recommendations identified in the August 2000 DOT&E report focus on OPINE testing, which refers to the operation of individual GMD components in environments induced by nuclear explosions. Details can be found in the classified version of this report.

DOT&E Recommendations on Target Discrimination

Target discrimination is a critical function of a missile defense engagement that requires the successful execution of a sequence of functions, including target detection, target tracking, estimations of physical characteristics of tracked objects, and data fusion. DOT&E had two overarching concerns with the operational testing of the discrimination function: Capability against diverse threats. Fundamentally, successful target discrimination requires that the defense be able to anticipate many characteristics of the threat. DOT&E, therefore, was concerned that discrimination algorithms may not be sufficiently robust to handle unanticipated threat scenes.

The quality and quantity of information known prior to testing.

DOT&E was concerned that every physical property of target objects is known with unrealistic accuracy in advance of flight tests.

Twelve of 50 recommendations in the August 2000 DOT&E report pertain to the testing of the discrimination function. Specifically, DOT&E recommended adding challenging yet unsophisticated countermeasures to the target suites of integrated flight tests. DOT&E also recommended integrating countermeasures developed by the Countermeasures Hands- On Program (CHOP) into target suites of integrated flight tests. Finally, DOT&E recommended executing flight test events—either intercept attempts or risk reduction flights—that have a “pop quiz” component with respect to radar discrimination. Operationally, this type of flight test is more representative of a true tactical engagement, because the exact composition and type of countermeasures flown in an actual engagement are generally unknown. Details can be found in the classified version of this report.

Relative to the previous test program, MDA has substantially increased the scope of work being done in discrimination. MDA is pursuing a block approach that incrementally builds to a system-level discrimination architecture that incorporates a network of sensors. The idea is to observe the target suite throughout its trajectory using an array of ground- and space-based sensors and to combine individual observations to formulate a “discrimination solution.” MDA is also investing resources to study the discrimination problem and, for example, is moving forward with flight test events focused on radar discrimination and large analysis programs.

MDA has plans to conduct four Radar Certification Flights through fiscal year 2006. These are non-intercept flight tests for comprehensively characterizing the discrimination capability of the X-band radar and to support the development of upgraded early warning radars. Furthermore, these tests are expected to have a “pop quiz” component to examine radar discrimination. MDA has not yet scheduled “pop quiz” testing in relation to kill vehicle’s capability to perform target discrimination.

MDA initiated and continues to fund analysis programs for investigating promising technical concepts to improve its capabilities against enemy countermeasures. For example, one such program, Project Hercules, is focused on the development and testing of discrimination algorithms and draws on academic, government, and industry expertise. Details can be found in the classified version of this report.

Despite MDA’s increased scope of work in the discrimination area, as described above, the agency’s specific actions pertaining to integrated flight testing only partially address the August 2000 DOT&E recommendations. No intercept flight tests of the current test plan, which goes through IFT-26 (fiscal year 2007), are planned to address the challenge posed by an enemy’s use of unsophisticated but more challenging countermeasures. Rather, agency officials told us that the technical challenges posed by such countermeasures are being analyzed and may be inserted into the flight test program at a later time.

DOT&E Recommendations on Programmatics

The remaining five recommendations from the August 2000 DOT&E report pertain to concerns on programmatic issues, namely, adequacy of spares in flight testing, and performance requirements. MDA has not provided for adequate target or interceptor backups (hot spares) during flight tests. MDA officials stated that additional target and interceptor spares can be costly, but they are considering the issue. Even if implemented, MDA’s actions that address the recommendations on spares would not have a significant impact on the actual conduct of flight tests but would reduce schedule risk.

When DOT&E made its recommendations in August 2000, the GMD element was being developed according to operational requirements. However, MDA is now following a fundamentally new acquisition strategy—one that is capability-based with no formal operational requirements developed by the services. Hence, MDA has no plans to reexamine the reliability requirements. Nonetheless, the current test program is addressing certain performance issues raised by DOT&E. For example, the GMD program office is tracking the prime contractor’s progress in meeting target discrimination goals.

Characterization of Initial Defensive Capability

Under the new acquisition strategy outlined by the Secretary of Defense in his January 2002 memorandum, the ballistic missile defense program has been refocused into a broad-based research and development effort managed by MDA. The new program aims at developing layered defenses to intercept missiles in all phases of flight and, if directed, to use developmental prototypes and test assets to provide an early operational capability. And, as stated above, system development is not subject to the formal operational requirements developed by the Services.

On December 16, 2002, the President directed DOD to begin fielding the first increment of the multi-element ballistic missile defense system in 2004. The Secretary of Defense stated the next day that “…it would be a very preliminary, modest capability.” The initial defensive capability for defending the United States against long-range missiles would be based on the GMD element of the Test Bed and augmented with more interceptors and external sensors, as follows: GMD Element as part of the BMDS. The principal components of the GMD element for defensive operations include a total of up to 10 interceptors sited at Fort Greely (6) and Vandenberg Air Force Base (4); GMD fire control nodes at Fort Greely and Schriever Air Force Base for battle management and execution; an upgraded Cobra Dane radar at Eareckson Air Station; and an upgraded early warning radar at Beale Air Force Base.

External Sensors. Existing sensors external to the GMD element would also be available for defensive operations, including Defense Support Program satellites for early warning of missile launches, and three forward-deployed Aegis AN/SPY-1 radars on existing Navy destroyers for early midcourse tracking.

The above assets comprise the initial configuration, which is scheduled for fielding by the end of September 2004. The agency’s near-term intention is to grow this capability by adding 10 interceptors at Fort Greely, a sea-based X-band radar, and an upgraded early warning radar at Fylingdales, England, by the end of 2005.

Uncertainties of the Initial Defensive Capability

MDA is moving forward, as directed by the President, with the fielding of an initial defensive capability by the end of the 2004 fiscal year to protect the United States from long-range missiles. MDA cannot at this time formulate a credible assessment of system-level effectiveness, because critical components like the Cobra Dane radar and interceptor boosters have yet to be developed and tested in a flight test environment, and no initial defensive capability is available for a system-level demonstration and evaluation.

Cobra Dane Radar. The capabilities of the Cobra Dane radar will not be demonstrated in flight testing before September 2004. It is an L-band phased array radar located at Eareckson Air Station in Shemya, Alaska, at the western end of the Aleutian chain. Its close proximity to Russia allows it to perform its primary mission of collecting data on intercontinental ballistic missile and submarine launched ballistic missile test launches to the Kamchatka impact area. Since the Cobra Dane radar is currently being used in a surveillance mode, it does not require real time communications and data processing capabilities. After planned software and hardware upgrades to be completed in fiscal year 2004, it will have the additional mission to perform real-time acquisition and tracking, functions critical for ballistic missile defense.

Interceptor Boosters. In July 1998, the GMD prime contractor (Boeing) began developing a new three-stage booster for its ground-based interceptor from commercial off-the-shelf components. The contractor encountered difficulty, and by the time the booster was flight tested in August 2001, it was already about 18 months behind schedule. Subsequently, to reduce risk, MDA altered its strategy for acquiring a new booster for the GMD interceptor. Development of the original booster was transferred to Lockheed Martin, and MDA authorized the GMD prime contractor to develop a second source for the booster by awarding a subcontract to Orbital Sciences Corporation. Both contractors are developing boosters for use in the September 2004 initial defensive capability. The first demonstration of an operational booster in an attempted intercept is scheduled for the third quarter of fiscal year 2004.

System-Level Testing. A system-level demonstration of the initial defensive capability will not be conducted prior to September 2004. To date, integrated flight tests have demonstrated basic functionality of a representative ballistic missile defense system using surrogate and prototype components, and have shown success in intercepting a mock reentry vehicle in a developmental test environment. The first flight test consisting of components closest to the configuration of the September 2004 initial defensive capability is IFT-14, which is currently scheduled for the third quarter of fiscal year 2004. The test will incorporate Block 2004 prototypes of the interceptor booster and kill vehicle of the configuration intended for operational use beginning in September 2004. In addition, the first tactical build of the battle management software will be utilized in IFT-14. However, interceptors will not be launched out of Fort Greeley in IFT-14 and IFT-15 (the remaining integrated flight tests to be conducted before September 2004).

In commenting on a draft of this report, MDA stated that while it cannot address all technical concerns for the initial fielding, it has added the following activities: Enhanced producibility, quality, and reliability efforts.

Increased operational focus in the developmental program, e.g., military utility and effectiveness assessments.

Expanded command and control, battle management, and operator integration in BMDS testing to support fielding of initial defensive capabilities in 2004.

MDA also stated that the results of these program decisions are intended to provide a comprehensive program that demonstrates operational effectiveness and military utility against credible threats in an operational environment.

Effectiveness of September 2004 and December 2005 Initial Defensive Capability

System effectiveness is characterized in terms of the following four performance metrics: (1) defended area, (2) launch area denied, (3) probability of engagement success, and (4) raid size breakpoint. Defended area is the portion of the United States protected against long-range missile attacks and, as a metric, is usually reported relative to a single threat country or region; launch area denied simply refers to the collection of threat countries from which the United States is protected. The probability of engagement success is the probability that all attacking warheads are destroyed, derived from the probabilities associated with missile defense functions like detection, discrimination, and hit-to-kill. Finally, raid size breakpoint is the maximum number of warheads the system can realistically defeat in a single engagement. This metric is highly dependent on interceptor inventory.

A detailed discussion of GMD’s expected effectiveness is presented in the classified June 2003 version of this report.

Other Factors Affecting System Performance

A notable limitation of the effectiveness of the September 2004 initial defensive capability—and possibly the December 2005 capability— pertains to the inability of system radars to perform target discrimination. Neither the Cobra Dane radar nor the upgraded early warning radar at Beale is capable of performing rigorous discrimination, a function achievable only by the X-band radar. Rather, both radars will utilize common “target classification” software that enables them to classify objects as threatening or non-threatening. For example, debris would be classified as non-threatening, but objects like deployment buses and decoy replicas would be classified as threatening. Accordingly, the system would have to rely solely on the kill vehicle for a final target selection.

The assessment of kill vehicle discrimination is, therefore, critical for understanding the capability of the deployed system, a point made in the DOT&E report. Appropriately, the GMD prime contractor tracks the discrimination capability of the kill vehicle as a technical performance measure. The prime contractor’s December 2002 assessment rated the kill vehicle discrimination performance as meeting expectations based on analysis and simulation.

Lastly, measures of system suitability like availability and vulnerability— which complement system effectiveness—are important for characterizing the initial defensive capability as a whole. MDA is aiming for full-time operations but faces risks in achieving this goal. Details on system availability and vulnerability are provided in our June 2003 classified report.

Conclusions

Since DOT&E issued its August 2000 report, DOD has altered its approach to the acquisition of missile defense systems to one that follows a “capability-based” strategy. The new approach allows MDA to evolve and demonstrate additional improvements in missile defense systems before committing to procurement and operations. MDA’s test program for all missile defense elements, such as GMD, was also reoriented to focus on the development and use of the BMDS Test Bed. Over time, the Test Bed should facilitate testing that address many of DOT&E’s recommendations, especially those pertaining to flight test realism, complexity, and artificialities. However, most of the agency’s actions with respect to DOT&E’s ground testing recommendations, namely, those pertaining to comprehensive hardware-in-the-loop testing of the kill vehicle have been deferred. In addition, MDA is proceeding slowly with the flight testing against certain countermeasures, which DOT&E noted are simple for an enemy to implement. These unresolved concerns in the test program warrant attention by DOT&E and the test community in general. Given the importance of ground testing and discrimination testing for understanding system effectiveness, decision makers in the Congress and Office of the Secretary of Defense would benefit from having information on the agency’s progress in these matters as they consider investments in developing the ballistic missile defense system. As an independent office that reviews DOD’s weapon system testing and the office that made the recommendations discussed in this report, DOT&E would be in a good position to provide such information to decision makers.

Recommendations for Executive Action

As a means of providing decision makers with critical information when investments in missile defense are considered, we recommend that DOT&E report periodically, as it deems appropriate, on the status of MDA’s actions taken or planned in response to the August 2000 recommendations. In its review, DOT&E should include information and recommendations, as warranted, on MDA’s progress and planning (1) to improve hardware-in-the-loop testing of the kill vehicle, (2) to test kill vehicle components in nuclear environments, and (3) to test the GMD element’s capability to defeat likely and simple near-term countermeasures during integrated flight tests. In the report, DOT&E can advise the Director, MDA, on how the test program could be modified to accommodate DOT&E’s long-standing concerns.

Agency Comments and Our Evaluation

In commenting on a classified draft version of this report, DOD agreed with our recommendations. (See app. II for a reprinted version of DOD’s comments.) However, DOD conveyed the following concerns: The GMD test program as described in this report is no longer current.

It is difficult to reconcile the dated terms of reference of the original DOT&E recommendations with the current program strategy and structure.

The inherent robustness of the envisioned layered BMD System relative to midcourse countermeasures is overlooked.

While the GMD test program has, indeed, been in a constant state of flux, thus complicating our analysis, our report presents the latest, approved test program information provided to us by MDA.

Despite alterations to the acquisition strategy and structure of the ballistic missile defense system and its constituent elements, like GMD, we believe most of the DOT&E recommendations are still relevant because the technical challenges and uncertainty with developing, testing, and fielding effective defensive capabilities, as identified in the August 2000 DOT&E report, remain significant. For example, the DOT&E report issued in February 2003, FY02 Assessment of the Missile Defense Agency Ballistic Missile Defense System, continued to highlight the need for a comprehensive hardware-in-the-loop capability to test the kill vehicle under the stress of real physical phenomena and to test the kill vehicle’s discrimination capability. We do recognize that a number of recommendations for which no actions are currently planned, such as those recommendations dealing with flight testing during Initial Operational Test and Evaluation, are a direct result of MDA’s new acquisition approach.

The department is correct in stating that we did not address the capability of the envisioned ballistic missile defense system as a whole in defeating midcourse countermeasures. However, we do note that a system-level discrimination architecture would use a network of ground- and space- based sensors to formulate a “discrimination solution.” Also, given the early stages of development of the envisioned layered system, including boost-phase intercept, the value of this strategy has not been demonstrated.

Although the department agreed that DOT&E should report periodically on the status of MDA’s actions to address the August 2000 DOT&E recommendations, it did not believe additional reporting is required to track their resolution. The department pointed out that our recommendation grants DOT&E discretionary reporting authority where mandatory reporting already exists. We believe, however, the recommendation is worded appropriately. Existing statutory reporting requirements for DOT&E on the adequacy and sufficiency of the missile defense test program do not require that the August 2000 DOT&E recommendations be specifically addressed. We worded the recommendation to highlight the areas we believe DOT&E should address—hardware-in-the-loop testing of the kill vehicle, testing of kill vehicle components in nuclear environments, and testing the GMD element’s capability to defeat likely and simple near-term countermeasures—and to give DOT&E the discretion to address our recommendation in the manner it deems appropriate. To present its assessment, DOT&E could use existing or new reporting vehicles.

Finally, department comments pertaining to MDA actions on ground testing are addressed in the body of this report.

As arranged with your staff, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time, we plan to provide copies of this report to interested congressional committees; the Secretary of Defense; and the Director, Missile Defense Agency. We will make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staffs have any questions concerning this report, please contact me at (202) 512-4841. The major contributors to this report were Randy Zounes, Stan Lipscomb, Tana Davis, and Bill Graveline.

Appendix I: Scope and Methodology

In examining the actions taken or planned by the MDA in response to the DOT&E recommendations, we analyzed pertinent test documents, studies, and reports. These include the (1) GMD Element Developmental Master Test Plan (March 2003); (2) GMD System Element Reviews; (3) MDA “immersion day” briefing; (4) MDA written responses to our questions about MDA actions in response to the DOT&E recommendations; (5) Secretary of Defense January 2002 Memorandum on Missile Defense Program Direction; and (6) Independent Review Team (Welch panel) Reports. In addition, MDA officials briefed us on GMD’s program status and efforts to defeat enemy countermeasures. We also reviewed available documentation on the schedule and purpose of the Test Bed. These documents included studies on the enhanced test program restructure, fiscal year 2003 budget justifications, and the request for the contract proposal for the Block 2004 Test Bed.

To assess the effectiveness and limitations of the initial defensive capability, we relied on the following MDA documentation: (1) GMD System Element Review (January 2003); (2) BMDS Block 2004 Statement of Goals; and (3) National Security Presidential Directive (NSPD-23), the President’s directive to begin fielding an initial capability. We also identified uncertainties—based on the level of testing achieved to date—of the potential capabilities of individual elements of the initial defensive capability, such as the radars and interceptor boosters, as well as radar capabilities to perform the discrimination function.

We conducted our work primarily at the MDA, located in Arlington, Virginia, and the GMD Joint Program Office, located in Arlington, Virginia, and Huntsville, Alabama.

We conducted our audit work for the June 2003 classified report, upon which this unclassified version is based, from October 2001 to March 2003 in accordance with generally accepted government auditing standards. However, reported dates of GMD flight test events given in this unclassified version have been updated with the latest (December 2003) GMD test schedules.

Appendix II: Comments from the Department of Defense