GAO-04-379R DOD Space Acquisition Policy 

United States General Accounting Office 

Washington, DC 20548 

January 29, 2004 

The Honorable Wayne Allard 

Chairman 

The Honorable Bill Nelson 

Ranking Minority Member 

Subcommittee on Strategic Forces 

Committee on Armed Services 

United States Senate 

Subject: Defense Acquisitions: Risks Posed by DOD’s New Space Systems 

Acquisition Policy

On November 18, 2003, we testified before the Subcommittee on the Department of 

Defense’s (DOD) new acquisition policy for space systems. The new acquisition 

policy, issued in October 2003, sets the stage for decision making for DOD’s 

investment in space systems, which currently stands at more than $18 billion annually 

and is expected to grow considerably over the next decade. You requested that we 

provide additional comments on several issues relating to the new policy and other 

space acquisition issues. Your specific questions and our answers are discussed 

below. 

1. The Air Force maintains that its Defense Space Acquisition Board (DSAB) process 

allows earlier identification of problems and senior level attention, which will 

improve management and lower risk. Does GAO have any concerns with the DSAB 

process? 

Our concern is not with earlier identification of problems or the added senior level 

attention the new process calls for, but with earlier investment decisions, which are 

also called for. Under the new process, the DSAB may approve product development 

to begin before DOD knows whether technologies can work as intended. As a result, 

it will make major investment commitments without really knowing what resources 

will be required to deliver promised capability. Moreover, the policy encourages 

development of leading-edge technology within product development, that is, at the 

same time the program manager is designing the system and undertaking other 

product development activities. DOD believes this approach will allow space 

systems to better incorporate leading-edge technologies. But as our work has 
Page 2 GAO-04-379R DOD Space Acquisition Policy 

repeatedly shown, such concurrency within space and other weapon system 

programs increases the risk that significant problems will be discovered as the 

system is integrated and built, when it is more costly and time-consuming to fix them. 

Moreover, as we testified, the knowledge-building approach for space stands in sharp 

contrast to that followed by successful programs and the approach recommended by 

DOD’s revised acquisition policy for weapon systems. Successful programs will not 

commit to undertaking product development unless they have high confidence that 

they have achieved a match between what the customer wants and what the program 

can deliver. Technologies that are not mature continue to be developed in an 

environment that is focused solely on technology development. This system puts 

programs in a better position to succeed because they can focus on design, system 

integration, and manufacturing. Further, our work has shown that taking an 

evolutionary approach to improving capability increases the likelihood of delivering 

that capability to the war fighter sooner than the revolutionary approach the Air 

Force continues to support in the new space policy. 

2. Does GAO believe that the process put into place in the new space 

acquisition policy by which cost estimates are derived will provide better 

cost estimates? 

No. Although some process changes will be made, the underlying causes of 

underestimating costs remain. 

DOD is adopting new methodologies and tools to enhance cost estimates, and it is 

enlisting assistance from DOD’s Cost Analysis Improvement Group (CAIG) to 

conduct independent cost estimates using cost estimating teams drawn from a broad 

spectrum of the cost-estimating community. Moreover, programs are now required to 

resolve differences between their cost estimates and estimates produced by the 

independent teams. In the past, cost-estimating groups have developed estimates 

that were different, leaving decision makers to select one estimate or combine a few. 

However, under the new space acquisition policy, cost estimates do not have to be 

based on the knowledge that technologies can work as intended. History has shown 

that cost estimates not based on such knowledge are significantly understated. 

Moreover, incentives that work against providing good estimates have not changed. 

Unlike the commercial world where the focus is on delivering a product to market, 

DOD’s system focuses on competing for resources from oversubscribed budgets. In 

the competition for funding, managers are encouraged to launch product 

developments before technologies are mature. Because funding is competitive and 

DOD’s forecasts of costs, schedules, and performance are largely based on immature 

technologies and other unknowns, estimates tend to be squeezed into insufficient 

profiles of available funding. In fact, pressures to underestimate costs may increase 

over the next decade as DOD plans to undertake a number of new, challenging space 

programs—which are expected to require an additional $4 billion in the next 4 years 

alone. Costs beyond that period are as yet unknown but are likely to be considerably 

higher. 
Page 3 GAO-04-379R DOD Space Acquisition Policy 

3. What is GAO's view on the Air Force policy related to full funding? 

DOD’s acquisition policy for other weapon systems requires a commitment for full 

funding at milestone B—the start of product development and the point at which 

DOD should have knowledge that technologies can work as intended. However, the 

new space acquisition policy does not require DOD to commit to fully fund a space 

program either when this knowledge has been obtained or at any point in the 

development process. Hence, there is no guarantee that the resources needed to 

meet requirements on any individual program will be there when needed—

particularly as DOD moves forward with its new programs. 

This represents another important departure from the development approach 

followed by successful programs. Our prior work1

 has found that if a product’s 

business case measures up, that is a company is assured that there is a market or 

need for the product and that it has the right knowledge in hand to develop the 

product with firm cost and schedule estimates, the company then commits to the 

entire development of the product, including the financial investment. In other 

words, corporate resources are made available to the development team so that 

product success is not compromised. As noted earlier, because DOD begins too 

many programs, its resources are always oversubscribed. By requiring program 

managers to continually justify funding, DOD runs a risk of foreclosing the ability for 

sound planning and execution. 

4. The Young Panel was not convinced of the merits of competition in some 

circumstances, particularly when the incumbent has performed well and 

"owns" the expertise and the government would incur significant cost in 

choosing another contractor for follow-on systems. Does GAO have a 

view on the merits or demerits of competition in space programs? 

Competition can provide natural incentives for an organization to be more efficient 

and more innovative. These incentives work in DOD’s favor. However, it is also 

important to recognize that competition can take various forms. For example, DOD 

can increase competition by using shadow contractors, pursuing alternative sensor 

designs, and breaking acquisitions into smaller blocks. DOD can also optimize its 

investment in weapon systems by competing air, land, sea, and space-based 

capabilities. By pursuing these various options, DOD would have greater assurance 

that it is obtaining the best value when it must select a prime contractor for follow-on 

systems. 

5. How effective can competition be when we have so few major contractors 

capable of executing large and complex space programs? 

While there are only a few contractors currently capable of implementing large and 

complex space programs, there are many more capable of building specific satellite 

components and technologies. Thus, by increasing competition at the mission 

payload or sensor level and breaking acquisitions into smaller pieces, DOD can 

 

1

 U.S. General Accounting Office, Best Practices: A More Constructive Test Approach Is Key to Better 

Weapon System Outcomes, GAO/NSIAD-00-199 (Washington, D.C., July 31, 2000).
Page 4 GAO-04-379R DOD Space Acquisition Policy 

expand the universe of contractors competing for work. Over the long run, this could 

enable more contractors to build the expertise and knowledge needed to manage 

large space programs. It would also require DOD to have significant insight into the 

lower tiers of the industry.

6. Is there a path to making competition a useful element in healthy 

programs? 

Managing the industrial base is one of the most critical determinants of acquisition 

success. According to DOD studies, this not only means injecting competition early 

on to ensure that the highest performing and most cost-effective technologies and 

designs are being pursued, but adequately defining work; establishing shorter, more 

manageable contract periods; and providing the right incentives for contractors. 

Following an evolutionary development path would better enable programs to take 

these kinds of actions. It would also foster a healthier industrial base because it 

would get programs into production sooner. Also important is ensuring that 

programs have the right capability to evaluate contractor proposals and to manage 

the contracts once they are in place. As DOD’s studies of space programs show, the 

government will invariably encounter problems when too much responsibility is 

handed over to contractors and too little oversight is provided. 

We have also found that the path to healthier programs is characterized by having an 

open systems design. Such a design is characterized by (1) well defined, widely used, 

preferably nonproprietary interfaces and protocols between systems, subsystems, 

and components and (2) an explicit provision for system expansion or upgrade 

through incorporation of additional higher performance subsystems and components 

with minimal negative impact on the existing system. Open systems design allows 

competing developers to offer additional features and capabilities. With this 

approach, the government might be able to minimize dependence on a specific 

contractor. Also, upgrades can be added without replacing the entire system. Costs 

across the board--development, production, operations, and support--can thereby be 

reduced. 

7. Does GAO believe that space programs will be less schedule driven under 

the new acquisition policy? 

No. In the past, DOD has taken a schedule-driven versus a knowledge-driven 

approach to the acquisition process for space and other weapons systems with the 

justification that capabilities were urgently needed. In other words, commitments 

were made to achieving certain capabilities without knowing whether technologies 

being pursued could really work as intended. As a result, time and costs estimates 

were consistently exceeded, and steps essential to containing costs, maximizing 

competition among contractors, and testing technologies were shortchanged. 

Perversely, programs actually took longer when rushed at the start. Moreover, DOD 

often lacked assurance that it was even pursuing the best technical solution because 

alternatives were not analyzed or they were eliminated in order to meet schedule 

pressures. When technology did not perform as planned, assigning additional 

resources in terms of time and money became the primary option for solving 

problems, since customer expectations about the products’ performance already 

became hardened. 
Page 5 GAO-04-379R DOD Space Acquisition Policy 

The new space acquisition process does not change this approach or the incentives 

that drive it. Rather, it encourages programs to enter into product development 

without knowledge that technologies can work as intended. Moreover, for new 

programs like the Transformational Satellite (TSAT) and Space Based Radar (SBR), 

DOD is still setting initial satellite launch dates before this knowledge has been 

obtained. By contrast, DOD’s acquisition policy for non-space systems establishes 

mature technologies—that is, technologies demonstrated in a relevant environment—

as critical before entering product development. By encouraging programs to do so, 

the policy for non-space systems puts programs in a better position to deliver 

capability to the war fighter in a timely fashion and within funding estimates because 

program managers can focus on the design, system integration, and manufacturing 

tasks needed to produce a product. 

8. The requirements for the Space-Based Infrared System High system 

(SBIRS High) still continue to change. In GAO’s report, you highlight 

several examples, including batteries and solar cell panels. From the 

report, I gather that GAO finds that the Air Force’s efforts to limit 

requirements changes to only those that are "urgent and compelling" are 

better, but that they are not successfully eliminating the growth of 

requirements. You mention at least $203 million in new requirements. Is 

this a correct interpretation of the new "urgent and compelling 

approach?" 

Prior to the restructuring, the SBIRS High program office exerted no control over 

requirements changes, leaving many decisions on requirements to its contractors or 

within lower management levels of the program office. As part of the SBIRS High 

program restructuring, the Air Force established an advisory program management 

board to oversee requirements changes. The board’s role is to ensure that new 

requirements are urgent and compelling, that they reflect an appropriate use of funds, 

and that decisions about requirements are more transparent. Air Force leadership, 

not the SBIRS High program office, made the decision that the new requirements 

were urgent and compelling enough to address. 

We believe that establishing the board is a positive step and should help manage 

requirements changes more effectively. Nevertheless, the board will still be 

challenged to ensure some discipline in requirements setting, since there is a diverse 

group of Air Force and other DOD users that have an interest in SBIRS High and 

there are increasing demands for surveillance capabilities. Currently, there are 

several proposed requirements changes on the table that could have a significant 

impact on the program. 

9. The GAO report also indicates that software development problems 

continue to be a problem. This problem is not limited to SBIRS High, 

however. What recommendations can you make to address this 

continuing problem? 

Problems with software development in DOD weapons systems are well known. For 

example, the Defense Science Board reviewed selected DOD software intensive 
Page 6 GAO-04-379R DOD Space Acquisition Policy 

systems and found that programs lacked well thought-out, disciplined program 

management and/or software development processes. The programs lacked 

meaningful cost, schedule, and requirements baselines, making it difficult to track 

progress against them. These findings are echoed by the work of DOD’s Tri-Service 

initiative. Because weapon systems are becoming increasing dependent on 

software, lax management and oversight over software development can be 

detrimental to a program, as it was for SBIRS High. 

There are steps we have identified in an ongoing review for the Senate Committee on 

Armed Services that DOD could take to address this problem. Chief among them is 

to require programs to apply best practices for software development and acquisition, 

many of which have been identified by the Software Engineering Institute at Carnegie 

Mellon University and packaged into continuous improvement models and guidance. 

In adopting these models, organizations would take a more disciplined and rigorous 

approach toward managing or overseeing software development. At the same time, 

organizations need to provide the right environment to reduce software development 

risk. This means establishing an environment comprised of an evolutionary software 

development approach that relies on well-understood, manageable requirements and 

a desire to continuously improve development processes. It also means adopting and 

using a host of metrics to track cost and scheduling deviations; requirements changes 

and their impact on software development efforts; testing efforts; as well as efforts to 

detect and fix defects. Also important is to integrate these practices into existing 

acquisition policies and improvement plans as well as to enforce the use of these 

practices within individual programs. 

10. The GAO conclusion is that SBIRS High is still a program in trouble. To 

remedy this problem, GAO recommends that the Secretary reconvene the 

independent review team, or a similar body, to provide an assessment of 

the restructured program and concrete guidance for addressing the 

program's underlying problems. To play devil's advocate for a moment, 

how will another review of this program improve its chances of technical, 

budget, and schedule success?

The fundamental problem with the SBIRS High program has been the failure to 

develop key knowledge at critical junctures early in the development of the system, 

that is, before major investments were made. The program is now paying the price 

for this lack of knowledge development. Although the restructuring of the program 

in 2002 improved management and oversight capabilities, it did not go far enough in 

addressing the underlying problems with system design, integration, and software 

development. Another independent and in-depth technical review of the program is 

important to ensure that these problems are more clearly understood and that there 

are no other hidden problems lurking. At the same time, such a review will keep 

attention focused and heighten oversight of the program. Moreover, until it becomes 

standard to make knowledge-based decisions on DOD programs, ad hoc reviews such 

as the one we call for may be the only way to bring transparency to the decision 

making process. 
Page 7 GAO-04-379R DOD Space Acquisition Policy 

11. SBIRS High is clearly a highly visible troubled program. How 

representative is it of space programs in general? Is it unique or are the 

problems identified present throughout the space acquisition effort? 

We recently reported2

 that the majority of satellite programs over the past couple 

decades, like SBIRS High, cost more than expected and took longer to develop than 

planned. SBIRS High is one of the few weapon systems programs to exceed the 25 

percent cost threshold established in 10 U.S.C. 2433, but the problems affecting other 

programs have been equally dramatic. For example, cost estimates for the Advanced 

Extremely High Frequency (AEHF) communications satellite program grew by $1.2 

billion from 1999 through 2001, while the program experienced a 2-year delay in the 

launch of the first satellite. And while DOD has spent several billion dollars over the 

past 2 decades to develop low-orbiting satellites that can track ballistic missiles 

throughout their flight, it has not launched a single satellite to perform this capability. 

A key underlying problem with many programs has been the desire to achieve 

revolutionary advancements in capability instead of evolutionary advancements. 

Such an approach meant that requirements exceeded resources (time, money, and 

technology) at the time of product development, setting the stage for costly and timeconsuming rework later in the program. More specifically, in reviewing our past 

reports, we found that: (1) requirements for what the satellite needed to do and how 

well it must perform were not adequately defined at the beginning of a program or 

were changed significantly once the program had already begun; (2) investment 

practices were weak, e.g., cost estimates were optimistic or potentially more costeffective approaches were not examined; (3) acquisition strategies were poorly 

executed, e.g., competition was reduced for the sake of schedule or DOD did not 

adequately oversee contractors; and (4) technologies were not mature enough to be 

included in product development. All of these problems affected SBIRS High and 

AEHF. One or more affected the Space Tracking and Surveillance System (STSS) and 

the predecessor SBIRS programs as well as Milstar, the Global Positioning System 

(GPS), and the National Polar-orbiting Operational Environmental Satellite System 

(NPOESS). 

Because DOD took a schedule-driven approach instead of a knowledge-driven 

approach to the acquisition process, activities essential to containing costs, 

maximizing competition among contractors, and testing technologies were 

compressed or not done. Like SBIRS High, many programs also encountered 

problems in setting requirements due to the diverse array of organizations with 

competing interests involved in overall satellite development—from the individual 

military services, to testing organizations, contractors, civilian agencies, and in some 

cases international partners. Requirements setting for SBIRS High was particularly 

problematic because the government put too much responsibility on its contractors 

to balance these competing interests—a problem recognized in DOD’s own study of 

SBIRS High and other studies of space acquisition problems. 

 

2

 U.S. General Accounting Office, Military Space Operations: Common Problems and Their Effects 

on Satellite and Related Acquisitions, GAO-03-825R (Washington, D.C.: June 2, 2003). 
Page 8 GAO-04-379R DOD Space Acquisition Policy 

In our view, new programs like the Transformational Satellite (TSAT) will likewise be 

unable to make a match between needs and resources at the onset of product 

development because DOD’s new space acquisition policy encourages product 

development to begin without knowing that technologies can work as intended to 

meet capability needs. 

-- -- -- -- -- 

In preparing answers to your questions, we relied on our prior work on DOD’s new 

space acquisition policy,3

 best practices in weapon system acquisitions, and our 

reviews of specific space acquisitions. Because we relied on previously issued work, 

we did not obtain comments from DOD on a draft of this letter. We conducted our 

work from December 2003 through January 2004 in accordance with generally 

accepted government auditing standards. 

We are sending copies of this letter to the Secretaries of Defense and the Air Force 

and interested congressional committees. We will also make copies available to 

others upon request. In addition, the report will be available at no charge on the GAO 

Web site at http://www.gao.gov. 

If you or your staff have any questions concerning this report, please contact either 

me or Katherine Schinasi at (202) 512-4841. Key contributors to this letter were 

Cristina Chaplain, Sigrid McGinty, Art Gallegos, Maricela Cherveny, John Oppenheim, 

and Mike Hazard. 

Robert E. Levin 

Director, Acquisition and Sourcing Management 

(120318) 

 

3

U.S. General Accounting Office, Defense Acquisitions: Improvements Needed in Space Systems 

Acquisition Management Policy, GAO-03-1073 (Washington, D.C.: September 15, 2003) and Defense 

Acquisitions: Improvements Needed in Space Systems Acquisition Policy to Optimize Growing 

Investment in Space, GAO-04-253T (Washington, D.C.: November 18, 2003). 
This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
GAO’s Mission The General Accounting Office, the audit, evaluation and investigative arm of 

Congress, exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability of the

federal government for the American people. GAO examines the use of public 

funds; evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good 

government is reflected in its core values of accountability, integrity, and 

reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older 

products. The Web site features a search engine to help you locate documents

using key words and phrases. You can print these documents in their entirety, 

including charts and other graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web 

site daily. The list contains links to the full-text document files. To have GAO email this list to you every afternoon, go to www.gao.gov and select “Subscribe to 

e-mail alerts” under the “Order GAO Products” heading. 

The first copy of each printed report is free. Additional copies are $2 each. A 

check or money order should be made out to the Superintendent of Documents. 

GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to

a single address are discounted 25 percent. Orders should be sent to: 

U.S. General Accounting Office 

441 G Street NW, Room LM

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800 

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

Obtaining Copies of 

GAO Reports and

Testimony 

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Public Affairs