Report to Congressional Requesters

United States General Accounting Office 

GAO 

April 2003 MISSILE DEFENSE 

Knowledge-Based 

Practices Are Being 

Adopted, but Risks 

Remain 

GAO-03-441 
Our work has shown that programs are most successful when they evolve 

products over time rather than try to make big leaps in capability and when 

the programs adopt knowledge-based acquisition processes. Similarly, MDA 

is taking an evolutionary approach to developing the missile defense system 

by developing capabilities in spirals or “blocks” rather than attempting to 

deliver all desired capabilities at one time. The agency intends to facilitate 

this approach by keeping requirements flexible before beginning activities to 

integrate technologies into a planned block, following a knowledge-based 

development plan, and demonstrating that technologies work as intended 

before beginning system integration of a block. In addition, the agency is 

seeking to involve stakeholders—such as the military services and 

operational testers—early in the development effort. 

However, MDA has not adopted some knowledge-based practices regarding 

long-term investment decision making and, as a result, the missile defense 

program’s success could be hampered. First, MDA is not making an early 

determination of the full cost of a capability. Such an estimate would help 

decision makers more effectively evaluate which technologies to include 

because they offer the best capability for the funds invested. Second, DOD is 

not allocating a “wedge” of funds in its Future Years Defense Plan for system 

production and operations. Without this wedge, DOD may not have the funds 

needed to procure and maintain the missile defense system. 

In addition, the President’s directive to begin fielding a missile defense 

capability by 2004 places MDA in danger of getting off track early and 

impairing the effort over the long term. This danger is highlighted by MDA’s 

decision to not follow some of its knowledge-based practices as it develops 

the first block of the system. For example, MDA is beginning system 

integration of its first block with immature technology and limited testing. 

While doing so may help MDA meet the President’s deadline, it also 

increases the potential that some elements may not work as intended. 

Examples of Missile Defense Elements 

The Department of Defense (DOD) 

would like to build a capable 

missile defense system that paces 

an ever-evolving threat. This is an 

expensive and risky endeavor 

because it requires a diverse set of 

technologies to be quickly 

developed, integrated, and 

deployed across an array of 

platforms. DOD estimates that it 

will need $50 billion for missile 

defense research and development 

over the next 6 years and likely 

additional funds in subsequent 

years. GAO was asked to review 

the Missile Defense Agency’s 

(MDA) strategy for this investment 

and determine what knowledgebased practices characteristic of 

successful programs are being 

adopted by MDA; what significant 

practices are not being adopted; 

and whether MDA is following the 

practices that it has adopted. 

GAO is recommending that DOD 

prepare life cycle cost estimates for 

missile defense elements before 

beginning integration activities and 

explore the option of setting aside 

funds to produce and operate the 

missile defense system over the 

long term. 

www.gao.gov/cgi-bin/getrpt?GAO-03-441.

To view the full report, including the scope 

and methodology, click on the link above. 

For more information, contact Robert Levin at 

(202) 512-4841 or levinr@gao.gov. 

Highlights of GAO-03-441, a report to 

Congressional Requesters 

April 2003

MISSILE DEFENSE 

Knowledge-Based Practices Are Being 

Adopted, but Risks Remain 
Page i GAO-03-441 Missile Defense 

Letter 1

Results in Brief 3

Background 4

Acquisition Strategy Adopts Many Knowledge-Based Practices 8

Two Knowledge-Based Practices Have Not Been Adopted 15

MDA Is Not Following Some Knowledge-Based Practices in 

Developing 2004 Capability 17

Conclusion 20

Recommendations for Executive Action 21

Agency Comments and Our Evaluation 22

Scope and Methodology 22

Appendix I Comments from the Department of Defense 24 

Table 

Table 1: Events and Accomplishments within MDA’s Integrated 

Master Plan 12

Figures 

Figure 1: Examples of Missile Defense Programs Transferred from 

Services into MDA 5

Figure 2: Notional Architecture of Future Ballistic Missile Defense 

System 7

Abbreviations 

BMDS Ballistic Missile Defense System 

DOD Department of Defense 

MDA Missile Defense Agency 

Contents 

This is a work of the U.S. Government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. It may contain copyrighted graphics, images or other materials. 

Permission from the copyright holder may be necessary should you wish to reproduce 

copyrighted materials separately from GAO’s product. 
Page 1 GAO-03-441 Missile Defense 

April 30, 2003 

The Honorable Carl Levin 

Ranking Member 

Committee on Armed Services 

United States Senate 

The Honorable Jack Reed 

United States Senate 

The threat from foreign ballistic missiles has grown steadily since the end 

of the Cold War. At least 25 countries now have or are in the process of 

acquiring missiles capable of delivering nuclear, biological, or chemical 

weapons. Countering this threat demands not only that the Department of 

Defense (DOD) develop cutting-edge technology, but also that it acquire 

and deploy complex systems more rapidly and effectively. 

DOD faces significant technical challenges in building a missile defense 

program. So far, DOD has had mixed results in achieving a “hit-to-kill” 

capability to destroy enemy warheads reliably, and DOD is still completing 

development of the components needed to detect and track a missile in all 

phases of its flight—from the boost phase, through the midcourse, and 

into the terminal phase. As it works to develop and prove these 

capabilities, DOD must also make sure that all missile defense elements 

can work together as an integrated system—a complex task in itself, given 

the number and diversity of elements involved in missile defense. Once 

these basic hurdles have been overcome, DOD still faces a far greater 

technical challenge in achieving target discrimination—that is, the 

capability to distinguish real warheads from decoys—to defeat more 

sophisticated threats. 

DOD also faces the challenge of delivering a weapon system with the 

capability promised and within the time and cost promised. For example, 

previous efforts to develop an airborne laser system and a space-based 

tracking and surveillance system encountered substantial delays, cost 

overruns, and other difficulties because DOD undertook these efforts 

United States General Accounting Office

Washington, DC 20548 
Page 2 GAO-03-441 Missile Defense 

without fully understanding the effort that would be needed to mature 

technologies critical to developing the systems’ required capabilities.1

Lastly, the development of missile defense weapon systems will also 

be costly. DOD estimates that it will need about $50 billion for missile 

defense research and development between fiscal years 2004 and 2009, 

or an average of over $8 billion per year. This investment does not include 

funding after 2009 to complete development of those systems, let alone the 

funding needed before and after 2009 to produce and operate the systems. 

In January 2002, facing these challenges, the Secretary of Defense directed 

the Missile Defense Agency (MDA) to orchestrate the integration of DOD’s 

diverse missile defense elements into one layered “system of systems” and 

to change its acquisition strategy as needed in doing so. In response, MDA 

has sought to implement practices that have proven successful in other 

DOD and commercial-sector programs. Taken together, these practices 

demand a high level of knowledge about a product at key junctures during 

development and that this knowledge be used to make prudent investment 

decisions. For example, developers should know that a technology has 

been demonstrated to work as intended before it is integrated into 

a product. 

The importance of making the best decisions possible is underscored by 

the time-critical nature of MDA’s efforts. In December 2002, the President 

directed DOD to begin fielding an initial capability in 2004 to protect the 

United States against missile attacks. To help ensure that MDA is using 

its resources wisely to develop the best possible system, you asked 

us to determine the extent to which the agency’s acquisition strategy 

incorporates the knowledge-based practices characteristic of successful 

development programs. To better answer your question, we separated it 

into three parts: (1) What knowledge-based practices are being adopted by 

MDA? (2) What significant practices are not being adopted? and (3) Is 

MDA following the knowledge-based practices that it has adopted? 

 1

 U.S. General Accounting Office, Missile Defense: Knowledge-Based Decision Making 

Needed to Reduce Risks in Developing Airborne Laser, GAO-02-631 (Washington, D.C.: 

July 12, 2002). U.S. General Accounting Office, Defense Acquisitions: Space-Based 

Infrared System-low at Risk of Missing Initial Deployment Date, GAO-01-6

(Washington, D.C.: Feb. 28, 2001). 
Page 3 GAO-03-441 Missile Defense 

MDA has adopted practices that offer the best opportunity to develop 

a complex weapon system successfully. Similar to the successful product 

development practices of leading commercial organizations, MDA is 

developing the missile defense system in “blocks,” rather than trying to 

make a big leap in capability. The agency intends to facilitate this 

evolutionary approach by keeping the system’s requirements flexible 

before beginning activities to integrate technologies into a planned 

block of the missile defense system, following a knowledge-based 

development plan, and maturing technology before beginning system 

integration of a block. In addition, the agency is seeking to involve 

stakeholders—such as the military services and the operational testers—

early in the development effort. 

However, MDA has not adopted two significant practices regarding 

long-term investment decision making, and, as a result, the program’s 

success could be hampered. First, MDA is not making an early 

determination of the full cost of a capability. Specifically, before 

beginning system integration, MDA does not estimate the total costs for 

development, production, operations, and sustainment of that block. Such 

an estimate would help decision makers in evaluating which technologies 

to include because they offer the best capability for the funds invested. 

MDA officials told us that they are considering steps to provide such 

estimates earlier. Second, DOD is not allocating a “wedge” of funds in its 

Future Years Defense Plan (fiscal years 2004 through 2009) for system 

production and operations. MDA officials told us that such a wedge has 

not been set aside because MDA’s acquisition strategy does not presume 

that a decision will be made to produce and operate the system. DOD 

risks, however, that when it is ready to procure and maintain the missile 

defense system, it will not have the funds to do so unless it reduces or 

eliminates its investment in other important weapon systems. This 

approach brings little transparency to future investment choices and may 

constrain options for decision makers. 

The President’s directive to begin fielding an initial defensive capability in 

2004 also places MDA in danger of getting off track early and introducing 

more risk into the missile defense effort over the long term. This danger is 

highlighted by MDA’s decision to not follow some of its knowledge-based 

practices as it develops the first block of the missile defense system, 

which will provide the initial capability. Because of time pressures, MDA 

must include components that have not been demonstrated as mature and 

ready for system integration into a particular element, let alone the block 

overall. For example, MDA has encountered considerable difficulty in 

developing a new three-stage booster and has yet to flight test interceptor 

Results in Brief 
Page 4 GAO-03-441 Missile Defense 

boosters in configurations planned for fielding in September 2004. Also, 

MDA’s test program has been limited to date and is under considerable 

schedule pressures. A knowledge-based approach to testing validates 

whether components (1) work individually, (2) work together as a system 

in a controlled setting, and (3) work together as a full system in a realistic 

setting. MDA’s integrated flight tests to date have used surrogate and 

prototype components and have been executed under non-stressing 

conditions. As a result, testing to date has provided only limited data for 

determining whether the system will work as intended in 2004. Also, MDA 

has no plans to demonstrate through flight testing the upgraded primary 

radar in Alaska that will be used to detect and track enemy missiles. 

We are making recommendations to DOD for providing decision makers 

with more timely information on the cost and funding needs of missile 

defense. In commenting on a draft of this report, DOD concurred with 

our recommendations. 

In January 2002, the Secretary of Defense refocused the ballistic missile 

defense program. The Secretary delegated to MDA authority to manage all 

ballistic missile defense systems under development and shifted such 

programs controlled by the military services—such as the Army’s Theater 

High Altitude Area Defense program—to the agency. Other programs 

moved to MDA are highlighted in figure 1. These programs, previously 

recognized by DOD as major defense acquisition programs, are now 

considered “elements” and have been consolidated into one overall major 

program called the Ballistic Missile Defense System (BMDS). Future 

architectures of the BMDS may also include “stand-alone” components 

(primarily sensors) that would operate in concert with the various missile 

defense elements. 

Background 
Page 5 GAO-03-441 Missile Defense 

Figure 1: Examples of Missile Defense Programs Transferred from Services 

into MDA 

Source: DOD. 

In December 2002, President Bush directed the Department of Defense to 

begin fielding the first block of the ballistic missile defense system for 

operational use in 2004. That is, in addition to focusing resources on the 

development of a testbed for developmental testing of missile defense 

elements, he instructed MDA to build in an operational capability that 

would protect the United States against missile attacks. The fielding of 

such capabilities is referred to as an “initial defensive operations” 

capability, and, in a statement by the Secretary of Defense, “...would be a 

very preliminary, modest capability.” The initial capability will be based on 

the testbed and augmented with additional developmental assets. 

When fully deployed, the BMDS will include (1) space- and ground-based 

sensors to provide early warning and tracking of missile launches; 

(2) ground-based radars to identify and refine the tracks of threatening 

reentry vehicles and associated objects; (3) ground- and sea-based 

interceptors to destroy enemy missiles through “hit-to-kill” impacts; and 

(4) fire control nodes for battle management and execution of the 

ballistic missile defense mission. A notional architecture of future BMDS 

blocks is illustrated in figure 2. For example, the initial capability for 

defense of the United States against long-range missiles would come 
Page 6 GAO-03-441 Missile Defense 

from the Ground-Based Midcourse Defense element and BMDS sensors, 

as follows: 

• Ground-Based Midcourse Defense Element. The principal components 

of the Ground-Based Midcourse Defense element for defensive operations 

include interceptors sited at Fort Greely, Alaska, and Vandenberg Air 

Force Base, California; a fire control node for battle management and 

execution located at Schriever Air Force Base, Colorado, with a backup 

node at Fort Greely; an upgraded Cobra Dane radar at Eareckson Air 

Station in Shemya, Alaska; and an upgraded early warning radar at Beale 

Air Force Base, California. 

• BMDS Sensors. Sensors external to the Ground-Based Midcourse 

Defense element and available for defensive operations include Defense 

Support Program satellites for missile warning and forward-deployed 

Aegis AN/SPY-1 radars on existing Navy cruisers. 

The above assets comprise the initial configuration, which is scheduled for 

fielding at the end of September 2004. The agency’s near-term intention is 

to expand this capability by adding more interceptors at Fort Greely, 

Alaska; a sea-based X-band radar deployed in the Pacific for use in flight 

testing; and an upgraded early warning radar at Fylingdales, England, by 

the end of 2005. 
Page 7 GAO-03-441 Missile Defense 

Figure 2: Notional Architecture of Future Ballistic Missile Defense System 
Page 8 GAO-03-441 Missile Defense 

To ensure the delivery of high-quality products on time and within budget, 

successful developers have adopted acquisition strategies that are 

anchored in knowledge. Specifically, they establish decision points for 

moving forward from technology development to product development 

and on to production.2

 At each decision point, decision makers ask 

themselves whether they have gained the knowledge they need to proceed 

into the next acquisition phase. For example, they determine whether the 

work can be completed with the money and time available and whether 

the product will be worth the required investment. If any of these 

questions are answered negatively, the program does not go forward. 

Other practices that facilitate successful outcomes include developing 

systems in stages rather than attempting to deliver all desired capabilities 

at one time, keeping requirements flexible so that the system can be 

produced within available resources, making sure technology is proven 

before incorporating it into a development program, and involving the 

right people at the right time in decision making. Commercial and DOD 

programs that have successfully implemented these practices have found 

that they help curb the incentive to rely on immature technologies and to 

over-promise the capability that can be delivered. Moreover, these 

programs found that keeping stakeholders involved in decision making 

ensured that the developed product better met the customers’ needs. 

MDA realizes the value of these practices and is seeking to incorporate 

them into its acquisition strategy. Specifically, as discussed below, MDA 

plans to evolve the missile defense system over time, rather than trying to 

make a big leap in its capability. MDA is also planning to keep the system’s 

requirements flexible before beginning system integration and to follow a 

knowledge-based development plan. In addition, MDA is seeking to 

involve stakeholders—such as the military services and the operational 

testers—early in the development effort. 

 2

 The three acquisition phases are distinguished by the activities that occur during each 

of the three phases. During technology development, scientists apply scientific knowledge 

to a practical engineering problem and demonstrate that components with desired 

capabilities can be developed. Product development includes integrating those components 

into a stable system design and demonstrating that the design will result in a product that 

meets the customer’s needs and can be produced with the time and money available. 

Production is the manufacturing of the product. 

Acquisition Strategy 

Adopts Many 

Knowledge-Based 

Practices 
Page 9 GAO-03-441 Missile Defense 

Historically, many new development programs in DOD have sought to 

achieve a great leap ahead in capabilities. Because the technology was 

often not available to make such leaps, programs were often in 

development for years while engineers tried to develop and mature the 

needed technologies. As the time required to develop a system increased, 

so did the cost of the system. 

In contrast, development programs are most successful when they take an 

evolutionary, or phased, approach. In doing so, they establish time-phased 

plans to develop new products in increments. The first increment often 

has a limited capability because it incorporates technology that is already 

mature or can be matured quickly. As new technology is developed, it is 

incorporated into subsequent increments so that the product’s capability 

evolves over time. This approach reduces risks by introducing less new 

content and technology into a program’s design and development effort. 

An evolutionary strategy also enables developers to deliver a series of 

interim capabilities to the customer more quickly. Recognizing the benefits 

of evolving systems, DOD recently revised its acquisition system policy to 

encourage evolutionary development. 

The Missile Defense Agency’s new strategy for developing the ballistic 

missile defense system adopts evolutionary development. At the beginning 

of each block, the agency intends to predict the capability that can be 

developed given a “tool box”3

 of currently available technology and then to 

design that system. As additional technology matures, the agency can 

incorporate it into the next block being developed. 

Two key practices adopted by MDA can be expected to help the agency in 

taking an evolutionary approach. They include (1) keeping requirements 

flexible and (2) following a knowledge-based development plan with 

specific decision points and criteria for moving forward. 

Customers generally want new products that are high performance at 

low cost, delivered as soon as possible. But developing and producing 

such a product may exceed the developer’s technology or engineering 

expertise, or may be too costly and time-consuming for the customer to 

accept. Therefore, what a customer needs in a product and what a 

developer can produce given available resources must be matched to 

 3

 All possible elements/components and interfaces that could be used in a ballistic missile 

defense system. 

Evolutionary Development 

Flexible requirements 
Page 10 GAO-03-441 Missile Defense 

form an achievable set of product requirements before development 

begins. To make this match, both the customer and the developer must be 

flexible so that potential gaps between needs and resources can be closed. 

Flexibility represents the customer’s ability and willingness to lower 

product expectations, coupled with the product developer’s willingness 

and ability to invest more resources to reduce technical risks before 

beginning system integration. Without flexibility, resources and needs can 

still be matched, but the options for closing the gaps between the two are 

limited to additional investments on the developer’s part. In fact, our past 

reviews have found that many traditional DOD acquisition programs 

incurred substantial cost increases and schedule delays because 

requirements by the military services were unrealistic and inflexible. 

Because of its flexibility, MDA’s new “capabilities-based approach” for the 

development of missile defense elements provides a greater opportunity 

to resolve this potential gap between resources and needs.4

 Instead of 

proceeding with the development of a solution based on firm operational 

requirements set by the services, the agency, in coordination with the 

warfighter, considers a number of system architectural options that can 

be developed given the mature technologies that are available. Decision 

makers evaluate the alternative architectures according to the potential 

military utility that each offers, given the constraints of cost and schedule. 

For example, decision makers would compare the regions of the world 

that each architecture could defend from missile launches (defended area) 

or each architecture’s probability of engagement success. This acquisition 

approach eliminates any possible gap between resources and needs; the 

customer (warfighter) accepts the best capability that the developer can 

deliver given available resources and assumes enhanced capabilities will 

be built into future blocks. 

In the early stages of a major defense acquisition program, DOD 

establishes a master schedule for moving through development and into 

production. Historically, once these milestones have been established, 

they have often been an impetus for moving forward even if requisite 

knowledge had not been attained—an action that invariably caused 

schedules to slip and costs to rise. By contrast, we have found that 

successful developers place more importance on capturing specific 

 4

 At the beginning of systems integration, MDA plans to establish system capability 

specifications. As in any organization, these specifications may change if testing shows 

that they are unattainable or that meeting them will be too costly or take too much time. 

Knowledge-based 

development plan 
Page 11 GAO-03-441 Missile Defense 

technology, design, and manufacturing knowledge than meeting 

milestones and they use this knowledge to make investment decisions. 

Moreover, these developers identify and use specific markers or criteria—

such as technology readiness levels, percentage of engineering drawings 

released to the manufacturer, or the percentage of manufacturing 

processes under statistical control—to ensure that the program has 

sufficient knowledge to move forward. 

MDA has similarly adopted a structured plan, called the Integrated Master 

Plan, for moving forward with requisite knowledge. Every block would 

move through eight formal “events,” each of which would include an 

identified set of accomplishments that should be completed before the 

program moves on to the next event. (See table 1 for a list of events and 

their associated accomplishments.) As a block moves through the events, 

MDA plans to use quantitative criteria whenever possible to enhance 

decisions on whether to continue developing the block as it is or to make 

changes. At the end of a block’s development, MDA expects to recommend 

one of four alternative actions to decision makers. Officials could 

recommend that the elements be (1) transferred to the services to be 

produced and fielded in its current configuration, (2) further developed 

in a subsequent block, (3) retained as a test asset in the missile defense 

testbed, with some capability available for operational use, or 

(4) terminated. 

One such quantitative criterion adopted by MDA is technology readiness 

levels. Our reviews have found that successful developers often use 

technology readiness levels as an analytical tool to assess the maturity of 

technology being considered for inclusion in a product. There are nine 

levels of maturity. The level increases as the technology becomes closer in 

form, fit, and function to the actual system and is demonstrated in more 

realistic environments. For example, technology is least mature, or least 

ready for inclusion in a product, when it is an idea being explored in paper 

studies. Conversely, technology is most mature when it has been 

incorporated into the intended product and that product has been 

demonstrated in its intended environment. The lower the level of 

technology readiness, the more ground that must be covered to bring the 

technology to the point at which it can meet the intended product’s cost, 

schedule, and performance requirements with little risk. We found that 

most successful developers insert new technology into a product only 

when the technology has been incorporated into prototype hardware and 

that hardware has been demonstrated to work in the environment in 

which it is expected to be used. 
Page 12 GAO-03-441 Missile Defense 

MDA’s knowledge-base development plan incorporates the use of 

technology readiness levels at Event 1 to assess the maturity of technology 

proposed for a block configuration. The strategy calls for including new 

technology at system integration (Event 4) if that technology has been 

proven in prototype hardware that works in the environment in which it 

is expected to be used. While the incorporation of mature technology at 

system integration is MDA’s preferred approach, the strategy retains the 

flexibility to include less mature technology if it offers a significant benefit 

and the risk of including it is acceptable. In such instances, MDA expects 

to develop a plan for reducing the risk of moving forward with immature 

technology and to remove the technology from the block if the risk has not 

been reduced at subsequent decision points. 

Table 1: Events and Accomplishments within MDA’s Integrated Master Plan 

Event 0–Block Capability Alternatives 

Block planning process completed 

Long lead targets, tests, and exercises identified 

Affordability Analysis completed 

Preliminary block plan approved 

Event 1–Preliminary Configuration Definition 

Preliminary block description approved 

Technology readiness levels assessed 

Performance assessments updated 

Preliminary concept of operations and operational architecture drafted 

Risks assessed and mitigation programs established 

Detailed cost estimates for elements/components available 

Cost/benefit analysis updated 

Integration/test objectives defined 

Element/component preliminary design reviews completed 

Required funding identified 

Integrated master schedule completed 

Event 2–Configuration Definition 

Critical design reviews for all element/component/targets programs completed 

Performance/cost assessments updated 

Risks assessed and mitigation programs updated 

Military utility characterized and concept of operations refined 

Preliminary integration test plan available 

Funding available and resources allocated 

Block definition updated 

Integrated master schedule updated 

Event 3–First Development Article 

First development article built and initial tests completed 

Targets built and initial tests completed 
Page 13 GAO-03-441 Missile Defense 

Test range and support planning completed 

Concept of operations defined and operational architecture available 

Funding and Estimate at Completion assessed 

Event 4–Integrated Test Readiness Review 

Block integration test planning completed 

Element/component test and checkout completed 

Target test and checkout completed 

Ballistic missile defense system tactics, techniques, and procedures for designated 

user defined 

Funding and Estimate at Completion updated 

Operational characterization of each element completed 

Operational certification of element completed 

Event 5–Interim Test and Progress Review 

50 percent of system test objectives accomplished 

Support systems defined 

Training systems defined 

Funding updated and Estimate at Completion verified 

Initial transition plans completed 

Initial operational characterization completed 

Event 6–Element/component transition decision points 

System/element/component testing completed 

Operational characterization completed 

Support systems planned and budgeted 

Training systems planned and budgeted 

Transition plans completed and funded 

Production plans available 

Updated block definition available 

Element/component certification of military utility completed 

Service total obligation authority available 

Event 7–Block Certification of military utility 

Military utility assessed and system element/component offered for transition 

Ballistic missile defense system capability demonstrated 

Life cycle cost estimate indicates long-term affordability 

Reliability, maintainability, and availability, and support requirements characterized 

Block certification of military utility completed 

Integration of declared block capability of ballistic missile defense system 

Source: DOD. 

Note: The events and accomplishments in MDA’s Integrated Master Plan are being revised. This set 

was in effect as of February 2003. 

Developers have found that if they are to be successful, all groups that 

have a stake in the product should be involved at all appropriate stages 

in the development effort. For example, cost analysts are needed to 

accurately estimate the cost of the product, experts in test and evaluation 

are needed to objectively assess the performance of product prototypes, 

Involvement of 

Stakeholders 
Page 14 GAO-03-441 Missile Defense 

and others are needed to enhance understanding of the customer’s 

needs. By involving these groups from the time a product design is 

created and keeping them involved throughout the product integration 

and demonstration phases, a program can ensure that it has a complete 

perspective. 

A key forum for stakeholder input is the Missile Defense Support Group, 

which includes representatives from the Joint Air and Missile Defense 

Organization; the Comptroller’s Office; the Director, Operational Test and 

Evaluation; and other units across DOD. The support group provides 

advice on such subjects as policy, operations, testing, acquisition, and 

resources to the Director of the Missile Defense Agency and the Under 

Secretary of Defense for Acquisition, Technology and Logistics, and 

supports the Senior Executive Council5

 in decision making. In addition, 

an analytical working group remains in close contact with MDA 

management so that it can collect information for the Missile Defense 

Support Group as well as conduct independent analyses of the missile 

defense program’s work. 

Initially, not all members of the Missile Defense Support Group and its 

working group believed that MDA’s approach to stakeholder involvement 

would be successful. Soon after the support group was established, 

members voiced concern that they were not getting sufficient access to 

agency information. Members said that communication with MDA was 

poor and that access to knowledgeable MDA individuals was limited, all 

of which made it difficult to provide timely advice. For example, in 

April 2002, the agency presented options to the support group to address 

capability shortfalls in sea-based terminal defense caused by the loss of 

the Navy Area missile defense program.6

 Members of the group questioned 

why a successor to the Navy Area program was needed. MDA planned 

additional briefings pertaining to this issue; however, the group was never 

briefed and MDA, without additional group input, went directly to the 

Under Secretary of Defense for Acquisition, Technology and Logistics to 

obtain approval for a particular approach to sea-based terminal missile 

 5

 The Senior Executive Council is led by the Deputy Secretary of Defense, and its members 

are the Under Secretary of Defense for Acquisition, Technology and Logistics, and the 

Secretaries of the Army, Navy, and Air Force. The Council provides oversight of MDA’s 

activities and is responsible for making program adjustments and deciding to transition or 

transfer a capability to the services. Furthermore, the Council approves MDA’s investment 

strategy and decisions. 

6

 DOD cancelled the Navy Area program in 2001 due to cost overruns. 
Page 15 GAO-03-441 Missile Defense 

defense. DOD officials told us that they did not seek further input from the 

group because there was insufficient time for it to fully understand and 

evaluate the issue before a decision had to be made. 

Shortly after this, support group members provided comments and 

suggestions to the Under Secretary and the MDA director on increasing 

communication, obtaining access to MDA personnel, and receiving timely 

information. Subsequently, MDA made progress in addressing the support 

group’s suggestions and concerns relating to these issues. As a result, 

according to support group members, the level of interaction, 

communication, and involvement has improved. In particular, support 

group members attend weekly system and element review meetings and 

have regular interaction with agency personnel outside of the Missile 

Defense Support Group forum. 

While MDA has adopted many of the practices of successful acquisition 

programs, it has not incorporated two particularly significant ones. 

Specifically, before beginning system integration, successful developers 

reduce their investment risk by estimating total costs and determining that 

funding is available for developing, producing, and operating the system. 

(We recently reported on the significance of DOD’s costs for operating its 

weapon systems and keeping them ready for action over many years.7

) 

In MDA’s case, decision makers would benefit from having this knowledge 

available before MDA begins system integration because decision makers 

would be better positioned to consider whether to delay until subsequent 

blocks those elements that currently have unaffordable production, 

operation, or maintenance costs and whether costs might be lowered in 

the future by inserting new technology or implementing better engineering 

solutions. The information would also help decision makers to compare 

all elements’ costs and decide which elements should be included in a 

planned block of the missile defense system because they offer the best 

capability for the funds invested. 

As of February 2003, MDA’s draft Integrated Master Plan did not call for 

an element’s life cycle cost to be estimated at the beginning of system 

 7

 U.S. General Accounting Office, Best Practices: Setting Requirements Differently 

Could Reduce Weapon Systems’ Total Ownership Costs, GAO-03-57 (Washington, D.C.: 

Feb. 11, 2003). 

Two KnowledgeBased Practices Have 

Not Been Adopted 
Page 16 GAO-03-441 Missile Defense 

integration (Event 4) but rather at the point when that element is 

considered for transfer to a military service for production, operation, and 

maintenance (Event 7). Moreover, MDA may never estimate the full cost of 

some elements because, according to officials, some elements may never 

be transferred to a military service for production and operation. For 

example, MDA plans to continually upgrade elements such as the 

Ground-Based Midcourse Defense element in the missile defense test bed 

even though it would be available for combat use. In such cases, MDA 

plans to estimate only the element’s development costs—not its 

production, operation, and maintenance costs. 

Also, DOD has not allocated funds in its Future Years Defense Plan 

(fiscal years 2004 through 2009) for the production, operation, and 

maintenance of any elements that might be transferred in the future to the 

military services. MDA has established optional decision points called 

“off ramps” where elements such as Theater High Altitude Area Defense 

could be transferred to the military services, but DOD has not set aside a 

“wedge” of funding for the element’s production and operating costs. 

MDA officials told us that the agency is considering revisions to its 

Integrated Master Plan so that it can provide decision makers with 

complete life cycle cost information on each element prior to beginning 

system integration activities for each block of the missile defense system. 

For example, MDA anticipates defining each element’s training and 

support systems before it begins system integration activities. The officials 

said that that they are still, however, in the process of determining the cost 

information that needs to be collected and the timing of its collection. In 

terms of setting aside a wedge of funding in the Future Years Defense Plan 

for production and operations costs, MDA officials told us that no such 

action is planned at this time because MDA’s acquisition strategy does 

not presume that a decision will be made to transfer the element to the 

military service. They told us that they expect funding would be 

made available. 

However, we believe that unplanned operation and maintenance costs 

could be a particular problem for DOD because its budget for these 

expenses is already stressed by the rising cost to operate and maintain 

many aging weapon systems. Also, when DOD is at the point of deciding 

whether to transfer elements of the missile defense system to a military 

service for production and operation, DOD could find that it does not have 

the funds available for missile defense without reducing or eliminating 

funding for other important weapon systems. 
Page 17 GAO-03-441 Missile Defense 

Because the President directed that a missile defense capability be fielded 

beginning in 2004, MDA will not be able to follow some knowledge-based 

practices in developing the initial capability in this brief time. As noted 

earlier, MDA’s draft Integrated Master Plan recommends that when a 

block enters system integration it include mature technology but the plan 

allows for the inclusion of less mature technology if the benefits are 

significant and if risks can be mitigated. Given the Presidential direction, 

MDA must include components in the block that have not been 

demonstrated as mature and ready for integration into a particular 

element, let alone the block overall. MDA’s plan also calls for rigorous 

testing before the agency recommends that the system or its elements be 

available for fielding. However, MDA’s test program has been limited to 

date and is under considerable time pressures. 

Our past reviews of DOD and commercial product development programs 

have shown that programs are in a much better position to succeed if 

components that incorporate new technologies are matured to a high level 

before being integrated into a product. Conversely, developers that 

initiated product developments with immature technology increased the 

risk that their products would fail tests and that some aspects of the 

products’ design would have to be reworked because components did not 

perform as predicted. The overall impact of such problems was often that 

products did not deliver the promised capability or the developers had to 

spend additional time and money to develop that capability. 

While its draft Integrated Master Plan recommends that system integration 

begin with mature technologies (Event 4), MDA has begun including 

components into the Block 2004 configuration that are not yet mature. 

Two examples are the Cobra Dane radar and the boosters for the 

Ground-Based Midcourse Defense interceptors. 

• The Cobra Dane radar is located at Eareckson Air Station in the western 

end of the Aleutian Islands chain in Alaska. Planned hardware and 

software upgrades intended to provide the radar with real-time acquisition 

and tracking capabilities are expected to be completed in fiscal year 2004. 

MDA has no plans, however, to demonstrate the expected functionality of 

the radar through integrated flight tests. 

• MDA has encountered considerable difficulty in developing a three-stage 

booster for the Ground-Based Midcourse Defense element and has yet to 

flight test interceptor boosters in configurations planned for fielding in 

September 2004. By the time the new booster was flight tested in August 

2001, it was already about 18 months behind schedule. The first booster 

MDA Is Not 

Following Some 

Knowledge-Based 

Practices in 

Developing 

2004 Capability 

System Integration 

of Block 2004 Begins 

with Some Immature 

Technologies 
Page 18 GAO-03-441 Missile Defense 

flight test was successful, but the second booster drifted off course and 

had to be destroyed 30 seconds after launch. Subsequently, the agency 

authorized two new contracts for developing boosters for use in the Block

2004 capability. While this strategy should reduce risk in the program, the 

first demonstrations of these boosters will occur in the flight tests 

scheduled later this year. 

MDA officials told us that they could not deploy an initial capability in the 

timeframe directed by the President if they did not continue to develop the 

technology while designing the system. MDA officials told us that they 

expect to follow their knowledge-based development plan as they develop 

the next block (Block 2006) of the missile defense system. 

The fundamental purpose of testing is to gauge the progress being made 

when an idea or concept is translated into an actual product and, 

ultimately, to make sure the product works as intended. Leading 

commercial firms conduct testing to discover potential developmental 

problems early. The firms focus on validating that their products have 

reached increasing levels of product maturity at given points in time. The 

firms’ products have three maturity levels in common: components work 

individually, components work together as a system in a controlled 

setting, and components work together as a full system in a realistic 

setting. Testing in this systematic manner helps ensure that problems 

are identified and corrected early, when the cost of solving problems is 

lower and more options are available. Over time, disciplined testing 

helps confirm that the product eventually produced will meet the 

customer’s needs. 

In the past, when DOD programs have been schedule—rather than event—

driven, program managers have found it difficult to slow the program if 

problems were identified during testing. MDA has been placed in a similar, 

pressured position as it prepares to field an initial capability by September 

2004. Also, only limited test data is available for determining whether a 

credible capability will be available at that date. 

The capability that MDA expects to deploy is essentially a collection of 

elements that are connected by battle management software. Initially, the 

mission of the software will be to hand off data from the radars that detect 

and track enemy missiles to the shooters that launch interceptors to kill 

the missiles. For example, the battle management software could 

communicate to the Ground-Based Midcourse element data on the 

position of an intercontinental ballistic missile being tracked by the Aegis 

Testing under Pressure 

and Limited to Date 

One system-level test is 

planned prior to fielding 
Page 19 GAO-03-441 Missile Defense 

Ballistic Missile Defense radar. MDA has begun the development of battle 

management hardware/software and has completed some ground tests of 

its capability. However, MDA plans to deploy the block although the battle 

management software’s ability to interoperate with the elements as an 

integrated missile defense system will not be flight tested until the Spring 

of 2004. 

Integrated flight tests to date have demonstrated that the Ground-Based 

Midcourse Defense and Aegis Ballistic Missile Defense elements can 

defeat a mock warhead in a test environment. However, the tested 

elements did not include all of the same components that will be part of 

the elements deployed in 2004. Instead these elements were tested using 

some surrogate and prototype components. For example, all tests of the 

Ground-Based Midcourse element have included a surrogate booster and 

a prototype kill vehicle. In addition, tests of this element have not included 

the Cobra Dane radar that will be used in September 2004 to detect 

and track intercontinental ballistic missiles. The Cobra Dane radar will 

not actively participate in integrated flight tests at least through 

September 2007. 

Element flight tests have also been executed under nonstressing 

conditions that are not fully representative of the environments that the 

elements would experience in combat. All flight tests completed to date 

have been limited to a single corridor and intercept region, that is all 

targets have been launched from Vandenberg Air Force Base, California, 

and interceptors have been launched from the Reagan Test Site in the 

Marshall Islands. As a result, flight-test engagement conditions are limited 

to those with slower closing velocities and shorter intercept ranges. 

Testing under conditions such as these significantly limit the data MDA 

can collect on system effectiveness and readiness. 

An operational test assesses the effectiveness of the system against the 

known threat and its suitability for combat use. U.S. law requires that such 

tests be carried out on major defense acquisition programs and assessed 

by DOD’s Director, Operational Test and Evaluation, before a full-rate 

production decision is made. The purpose of the Director’s assessment is 

to advise the Secretary of Defense and Congress on the effectiveness of 

the system against the known threat and its suitability for combat use. 

MDA does not plan to operationally test the Block 2004 Ground-Based 

Missile Defense element before it is available for initial defensive 

operations. The September 2004 fielding is not connected with a full-rate 

production decision that would clearly trigger statutory operational testing 

Element flight-test data 

is limited 

Operational test data is limited 
Page 20 GAO-03-441 Missile Defense 

requirements. Nonetheless, according to DOD officials, MDA plans to 

incorporate both developmental and some operational test requirements 

in integrated flight tests. 

The Director, Operational Test and Evaluation, will provide comments on 

an element’s operational effectiveness and suitability as demonstrated in 

these tests.8

 However, Operational Test and Evaluation officials said that 

because developmental tests are scripted, planned events, they do not 

provide the opportunity to assess how the equipment and its operators will 

function under unforeseen conditions. 

MDA is attempting to build a ballistic missile defense capability that paces 

an ever-evolving threat. This is an expensive and risky endeavor, because 

it requires a diverse set of technologies that must be quickly developed, 

integrated, and deployed across an array of land-, air-, sea-, and spacebased platforms. Whether MDA can successfully meet the challenge of 

quickly developing an effective and suitable missile defense system 

depends in large part on its willingness to adopt practices that have made 

other developers successful and to implement those practices as it 

develops each block. 

Certainly, the presidential directive has already caused MDA to not follow 

some of the knowledge-based practices that it had adopted as it develops 

Block 2004. Giving up this approach opens the door to greater cost and 

performance risks. Beginning system integration of Block 2004 with 

immature technology increases the potential that some element may not 

work as intended. If this happens, MDA will be faced either with fielding a 

less than credible system or likely spending more money in an attempt to 

develop the desired capability within the time allowed. In addition to the 

challenge it faces in Block 2004, MDA faces the challenge of getting its 

acquisition program back on track. Because the ballistic missile threat is 

 8

 The National Defense Authorization Act for Fiscal Year 2002 (P.L. 107-107) requires the 

Director of Operational Test and Evaluation to (1) annually assess, and report to Congress 

on, the adequacy and sufficiency of MDA’s test program during the preceding fiscal year, 

(2) monitor the development of MDA’s plan for ensuring that each critical technology for a 

missile defense program is successfully demonstrated in an appropriate environment 

before that technology enters into operational service (and provide the Director of MDA 

with appropriate comments), and (3) review, on an ongoing basis, the development of 

MDA’s annual program goals (including testing goals) and annual program plan (including 

schedules for flight tests and other significant testing activities) and provide any resulting 

comments on the plans to the Secretary of Defense and the Director of MDA. 

Conclusion 
Page 21 GAO-03-441 Missile Defense 

rapidly increasing, MDA could always believe it is operating in an 

emergency environment. Yet, it has never been proven that it takes longer 

to acquire a weapon system if a knowledge-based acquisition plan is 

followed. Instead, the opposite should be true, because such a plan 

decreases the likelihood that deadlines will be missed because critical 

elements do not work as intended. 

MDA and DOD also need to address the long-term implications of their 

investment strategy. Both are assuming increased investment risk by not 

having the right information available for decision makers at the right 

time. The level of anticipated spending magnifies this risk. MDA officials 

told us they are considering changes to MDA’s Integrated Master Plan to 

identify life cycle costs at the beginning of system integration activities so 

that tradeoff decisions can be made in a more timely manner. However, 

because DOD has not yet set aside funds to cover its long-term costs, the 

department could find that it cannot afford to procure and maintain that 

system unless it reduces or eliminates its investment in other important 

weapon systems. By setting aside funds in the Future Years Defense Plan, 

we believe DOD would bring needed visibility to the impending trade-offs 

between missile defense and other weapon system spending for 

procurement and operations. 

To assist MDA and DOD decision makers in determining which elements 

or components should be included in each new block of the Ballistic 

Missile Defense System, we recommend that the Missile Defense Agency, 

before beginning integration activities, prepare a life cycle cost estimate 

for configuring the element or component that the agency is considering 

including in the block. 

To help ensure that funds are available to produce and operate the 

elements of the missile defense system when a decision is made to transfer 

elements to the military services, we recommend that the Secretary of 

Defense explore the option of requiring the services to set aside funds for 

this purpose in the Future Years Defense Plan. 

Recommendations for 

Executive Action 
Page 22 GAO-03-441 Missile Defense 

In commenting on a draft of this report, the DOD concurred with 

our recommendations. 

Regarding our recommendation that MDA prepare life cycle cost 

estimates before beginning integration activities, DOD said that MDA 

will prepare its best estimate of life cycle costs based upon projected 

hardware life, operational cost drivers, and initial capability quantities 

prior to integration activities for each block. 

Regarding our recommendation that DOD set aside funds in its Future 

Years Defense Plan in anticipation of the transfer of missile defense 

system elements to the military services, DOD said that there is benefit in 

budgeting funds when such a transfer is anticipated. Doing so would 

promote budget stability and improve the likelihood that an element or 

component would actually be fielded. 

DOD also suggested technical changes, which we incorporated as 

appropriate. DOD’s comments are reprinted in appendix I. 

To address our objectives, we analyzed documents that detailed the 

Missile Defense Agency’s new acquisition practices and compared the 

practices to those of successful development programs. We also obtained 

detailed briefings from Missile Defense Agency officials regarding the 

agency’s plan for the implementation of these practices and contrasted 

that plan to the implementation plan of successful programs. In addition, 

we discussed the challenges and risks that the agency faces as it 

implements its new plan with the Institute of Defense Analyses, 

Alexandria, Virginia. We also discussed these issues with all members of 

the Missile Defense Support Group, including the Office of the 

Undersecretary for Acquisition, Technology and Logistics; Office of the 

Undersecretary for Policy; Office of the Undersecretary (Comptroller); 

General Counsel; Office of the Assistant Secretary (Command, Control, 

Communications, and Intelligence); Office of the Director, Operational 

Test and Evaluation; Office of the Director, Program Analysis and 

Evaluation; Office of the Director, Cost Analysis Improvement Group, 

in Washington, D.C.; and the Joint Staff; Department of the Army; 

Department of the Air Force; Department of the Navy; and the Missile 

Defense Agency in Arlington, Virginia. 

Agency Comments 

and Our Evaluation 

Scope and 

Methodology 
Page 23 GAO-03-441 Missile Defense 

We conducted our review from March 2002 to March 2003 in accordance 

with generally accepted government auditing standards. 

As arranged with your staff, unless you publicly announce its contents 

earlier, we plan no further distribution of this report until 30 days from its 

issue date. At that time, we plan to provide copies of this report to the 

Chairmen and Ranking Minority Members of the Senate Committee on 

Armed Services; the Senate Committee on Appropriations, Subcommittee 

on Defense; the House Committee on Armed Services; the House 

Committee on Appropriations, Subcommittee on Defense; the Secretary of 

Defense; and the Director, Missile Defense Agency. We will make copies 

available to others upon request. In addition, the report will be available at 

no charge on the GAO Web site at http://www.gao.gov/. 

If you or your staff have any questions concerning this report, please 

contact me on (202) 512-4841. Major contributors to this report were 

Katherine Schinasi, Barbara Haynes, Cristina Chaplain, David Hand, 

Alan Frazier, Randy Zounes, Adam Vodraska, Jose Ramos, and 

Greg Lagana. 

R. E. Levin 

Director 

Acquisition and Sourcing Management 
Appendix I: Comments from the Department 

of Defense 

Page 24 GAO-03-441 Missile Defense 

Appendix I: Comments from the Department 

of Defense 
Appendix I: Comments from the Department 

of Defense 

Page 25 GAO-03-441 Missile Defense (120135) 
The General Accounting Office, the audit, evaluation and investigative arm of 

Congress, exists to support Congress in meeting its constitutional responsibilities 

and to help improve the performance and accountability of the federal 

government for the American people. GAO examines the use of public funds; 

evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older 

products. The Web site features a search engine to help you locate documents 

using key words and phrases. You can print these documents in their entirety, 

including charts and other graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site 

daily. The list contains links to the full-text document files. To have GAO e-mail 

this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily 

E-mail alert for newly released products” under the GAO Reports heading. 

The first copy of each printed report is free. Additional copies are $2 each. A 

check or money order should be made out to the Superintendent of Documents. 

GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a 

single address are discounted 25 percent. Orders should be sent to: 

U.S. General Accounting Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

GAO’s Mission 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Mail or Phone 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Public Affairs