Defense Acquisitions: Improvements Needed in Space Systems Acquisition Policy to Optimize Growing Investment in Space
Published: 2003-11-18T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense is spending nearly $18 billion annually to develop, acquire, and operate satellites and other spacerelated systems. The majority of satellite programs that GAO has reviewed over the past 2 decades experienced problems that increased costs, delayed schedules, and increased performance risk. In some cases, capabilities have not been delivered to the warfighter after decades of development. DOD has recently implemented a new acquisition policy, which sets the stage for decision making on individual space programs. GAO was asked to testify on its assessment of the new policy.
section_title: What GAO Found
paragraphs: Similar to all weapon system programs, we have found that the problems being experienced on space programs are largely rooted in a failure to match the customer's needs with the developer's resources--technical knowledge, timing, and funding--when starting product development. In other words, commitments were made to satellite launch dates, cost estimates, and delivering certain capabilities without knowing whether technologies being pursued could really work as intended. Time and costs were consistently underestimated. DOD has recognized this problem and recently revised its acquisition policy for non-space systems to ensure that requirements can be matched to resources at the time a product development starts. The space community, however, in its newly issued policy for space systems, has taken another approach. As currently written, and from our discussions with DOD officials about how it will be implemented, the policy will not result in the most important decision, to separate technology development from product development to ensure that a match is made between needs and resources. Instead, it allows major investment commitments to be made with unknowns about technology readiness, requirements, and funding. By not changing its current practice, DOD will likely perpetuate problems within individual programs that require more time and money to address than anticipated. More important, over the long run, the extra investment required to address these problems will likely prevent DOD from pursuing more advanced capabilities and from making effective tradeoff decisions between space and other weapon system programs.

section_title: Letter
paragraphs: I am pleased to be here today to discuss the Department of Defense’s  (DOD) new space acquisition policy. This policy will be critical as DOD  strives to optimize its investment in space—which currently stands at  more than $18 billion annually, and is expected to grow considerably over  the next decade. DOD’s space acquisitions have experienced problems  over the past several decades that have driven up costs by hundreds of  millions, even billions of dollars, stretched schedules by years, and  increased performance risks. In some cases, capabilities have not been  delivered to the war fighter after decades of development.
Similar to all weapon system programs, we have found that the problems  being experienced on space programs are largely rooted in a failure to  match the customer’s needs with the developer’s resources—technical  knowledge, timing, and funding—when starting product development.  While DOD’s new policy for space acquisitions may help to illuminate gaps  between needs and resources, it will not help DOD to close this gap. More  specifically, the policy allows programs to continue to develop  technologies after starting product development, which not only means  that costs and schedule will be more difficult to estimate, but that there  will be more risk that DOD will encounter technical problems that could  disrupt design and production and require more time and money to  address than anticipated. More important, over the long run, the extra  investment required to address these problems may likely prevent DOD  from pursuing more advanced technologies and from making effective  tradeoff decisions between space and other weapon system programs.
By contrast, DOD is taking steps to better position its other acquisition  programs for success. Its revised acquisition policy for non-space systems  separates technology development and product development.
subsections: section_title: The Importance of DOD’s Space Systems is Growing
paragraphs: DOD’s current space network is comprised of constellations of satellites,  ground-based systems, and associated terminals and receivers. Among  other things, these assets are used to perform intelligence, surveillance,  and reconnaissance functions; perform missile warning; provide  communication services to DOD and other government users; provide  weather and environmental data; and provide positioning and precise  timing data to U.S. forces as well as national security, civil, and  commercial users.
All of these systems are playing an increasingly important role in military  operations. According to DOD officials, for example, in Operation Iraqi  Freedom, approximately 70 percent of weapons were precision-guided,  most of those using Global Positioning System (GPS) capabilities. Weather  satellites enabled war fighters to not only prepare for, but also take  advantage of blinding sandstorms. Communication and intelligence  satellites were also heavily used to plan and carry out attacks and to  assess post-strike damage. Some of DOD’s satellite systems—such as  GPS—have also grown into international use for civil and military  applications and commercial and personal uses. Moreover, the demand for  space-based capabilities is outpacing DOD’s current capacity. For  example, even though DOD has augmented its own satellite  communications capacity with commercial satellites, in each major  conflict of this past decade, senior military commanders reported  shortfalls in capacity, particularly for rapid transmission of large data files,  such as those created by imagery sensors.
DOD is looking to space to play an even more pivotal role in future  military operations. As such, it is developing several families of new,  expensive, and technically challenging satellites, which are expected to  require dramatically increased investments over the next decade. For  example, DOD is building new satellites that will use laser optics to  transport information over long distances in much larger quantities than  radio waves. The system, known as the Transformational Satellite, or  TSAT, is to be the cornerstone of DOD’s future communications  architecture. Many space, air, land, and sea-based systems will depend on  TSAT to receive and transmit large amounts of data to each other as DOD  moves toward a more “network centric” war-fighting approach. DOD is  also building a new space-based radar (SBR) system, which is to employ  synthetic aperture radar and other advanced technologies to enable DOD  to have 24-hour coverage over a large portion of the Earth on a continuous  basis and allow military forces a “deep-look” into denied areas of interest,  on a non-intrusive basis without risk to personnel or resources. SBR itself  is expected to generate large amounts of imagery data, and it will rely on  TSAT to deliver this data to war fighters.
As figure 1 shows, the costs of these and other new efforts will increase  DOD’s annual space investment significantly. For example, based on the  2003 President’s budget, acquisition costs for new satellite programs and  launch services in the next 4 years are expected to grow by 115 percent— from $3.5 billion to about $7.5 billion. Costs beyond that period are as yet  unknown. While DOD’s budget documents show a decrease in 2009 for  these systems to $6.4 billion—they do not include procurement costs for  some of the largest programs, including TSAT, GPS III, SBR, Space  Tracking and Surveillance System (STSS), and Space-Based Surveillance  System (SBSS), which DOD will begin fielding beginning 2011. Nor do  these numbers reflect the totality of DOD’s investment in space. For  example, ground stations and user equipment all require significant  investment and that investment will likely increase as the new programs  mature.
Table 1 identifies specific programs factored into our analysis of upcoming  investments. It also shows that DOD will be fielding many of the new  programs within just a few years of each other.
section_title: Grounding Decisions in Knowledge is Vital for DOD’s Space Investment
paragraphs: For the past 6 years, we have been examining ways DOD can get better  outcomes from its investment in weapon systems, drawing on lessons  learned from the best, mostly commercial, product development efforts.  Our work has shown that leading commercial firms expect that their  managers will deliver high quality products on time and within budgets.  Doing otherwise could result in losing a customer in the short term and  losing the company in the long term. Thus, these firms have adopted  practices that put their individual programs in a good position to succeed  in meeting these expectations on individual products. Collectively, these  practices ensure that a high level of knowledge exists about critical facets  of the product at key junctures and is used to make decisions to deliver  capability as promised. We have assessed DOD’s space acquisition policy  as well as its revised acquisition policy for other weapon systems against  these practices.
Our reviews have shown that there are three critical junctures at which  firms must have knowledge to make large investment decisions. First,  before a product development is started, a match must be made between  the customers’ needs and the available resources—technical and  engineering knowledge, time, and funding. Second, a product’s design  must demonstrate its ability to meet performance requirements and be  stable about midway through development. Third, the developer must  show that the product can be manufactured within cost, schedule, and  quality targets and is demonstrated to be reliable before production  begins. If the knowledge attained at each juncture does not confirm the  business case on which the acquisition was originally justified, the  program does not go forward. These precepts hold for technically  complex, high volume programs as well as low volume programs such as  satellites.
In applying the knowledge-based approach, the most-leveraged investment  point is the first: matching the customer’s needs with the developer’s  resources. The timing of this match sets the stage for the eventual  outcome—desirable or problematic. The match is ultimately achieved in  every development program, but in successful development programs, it  occurs before product development begins. When the needs and resources  match is not made before product development, realistic cost and  schedule projections become extremely difficult to make. Moreover,  technical problems can disrupt design and production efforts. Thus,  leading firms make an important distinction between technology  development and product development. Technologies that are not ready  continue to be developed in the technology base—they are not included in  a product development.
With technologically achievable requirements and commitment of  sufficient resources to complete the development, programs are better  able to deliver products at cost and on schedule. When knowledge lags,  risks are introduced into the acquisition process that can result in cost  overruns, schedule delays, and inconsistent product performance. As we  recently testified, such problems, in turn, can reduce the buying power of  the defense dollar, delay capabilities for the war fighter, and force  unplanned—and possibly unnecessary—trade-offs in desired acquisition  quantities and an adverse ripple effort among other weapon programs or  defense needs. Moreover, as DOD moves more toward a system-of- systems approach—where systems are being designed to be highly  interdependent and interoperable—it is exceedingly important that each  individual program stay on track.
subsections: section_title: Decisions on Space Programs Have Not Been Sufficiently Grounded in Knowledge
paragraphs: Our past work has shown that space programs have not typically achieved  a match between needs and resources before starting product  development. Instead, product development was often started based on a  rigid set of requirements and a hope that technology would develop on a  schedule. At times, even more requirements were added after the program  began. When technology did not perform as planned, adding resources in  terms of time and money became the primary option for solving problems,  since customer expectations about the products’ performance already  became hardened.
For example, after starting its Advanced Extremely High Frequency  (AEHF) communications satellite program, DOD substantially and  frequently changed requirements. In addition, after the launch failure of  one of DOD’s legacy communications satellites, DOD decided to  accelerate its plans to build AEHF satellites. The contractors proposed,  and DOD accepted, a high risk schedule that turned out to be overly  optimistic and highly compressed, leaving little room for error and  depending on a precise chain of events taking place at certain times.  Moreover, at the time DOD decided to accelerate the program, it did not  have funding needed to support the activities and manpower needed to  design and build the satellites quicker. The effects of DOD’s inability to  match needs to resources were significant. Total program cost estimates  produced by the Air Force reflected an increase from $4.4 billion in  January 1999 to $5.6 billion in June 2001—a difference of 26 percent.  Although considered necessary, many changes to requirements were  substantial, leading to cost increases of hundreds of millions of dollars  because they required major design modifications. Also, schedule delays  occurred when some events did not occur on time, and additional delays  occurred when the program faced funding gaps. Scheduling delays  eventually culminated into a 2-year delay in the launch of the first satellite.  We also reported that there were still technical and production risks that  need to be overcome in the AEHF program, such as a less-than-mature  satellite antenna system and complications associated with the production  of the system’s information security system.
Another example can be found with DOD’s Space-Based Infrared System  (SBIRS)–High program, which is focused on building high-orbiting  satellites that can detect ballistic missile launches. Over time, costs have  more than doubled for this program. Originally, total development costs  for SBIRS-High were estimated at $1.8 billion. In the fall of 2001, DOD  identified potential cost growth of $2 billion or more, triggering a  mandatory review and recertification under 10 U.S.C. section 2433.  Currently, the Air Force estimates research and development costs for  SBIRS-High to be $4.4 billion. We reported that when DOD’s SBIRS-High  satellite program began in 1994, none of its critical technologies were  mature. Moreover, according to a DOD-chartered independent review  team, the complexity, schedule, and resources needed to develop SBIRS- High, in hindsight, were misunderstood when the program began. This led  to an immature understanding of how requirements translated into  detailed engineering solutions. We recently reported to this subcommittee  that while the SBIRS restructuring implemented a number of needed  management changes, the program continues to experience problems and  risks related to changing requirements, design instability, and software  development concerns. We concluded that if the Air Force continues to  add new requirements and program content while prolonging efforts to  resolve requirements that cannot be met, the program will remain at risk  of not achieving, within schedule, its intended purposes—to provide an  early warning and tracking system superior to that of its current ballistic  missile detection system.
DOD has also initiated several programs and spent several billion dollars  over the past 2 decades to develop low-orbiting satellites that can track  ballistic missiles throughout their flight. However, it has not launched a  single satellite to perform this capability. We have reported that a primary  problem affecting these particular programs was that DOD and the Air  Force did not relax rigid requirements to more closely match technical  capabilities that were achievable. Program baselines were based on  artificial time and/or money constraints. Over time, it became apparent  that the lack of knowledge of program challenges had led to overly  optimistic schedules and budgets that were funded at less than what was  needed. Attempts to stay on schedule by approving critical milestones  without meeting program criteria resulted in higher costs and more slips in  technology development efforts. For example, our 1997 and 2001 reviews  of DOD’s $1.7 billion SBIRS-Low program (which was originally a part of  the SBIRS-High program) showed that the program would enter into the  product development phase with critical technologies that were immature  and with optimistic deployment schedules. Some of these technologies  were so critical that SBIRS-Low would not be able to perform its mission if  they were not available when needed. DOD eventually restructured the  SBIRS-Low program because of the cost and scheduling problems, and it  put the equipment it had partially built into storage. In view of the  program’s mismatch between expectations and what it could achieve, the  Congress directed DOD to restructure the program (now under the  responsibility of the Missile Defense Agency) as a research and  development effort.
section_title: New Space Policy Allows Programs to Go Forward with Key Unknowns
paragraphs: DOD’s new space acquisition policy may help increase insight into gaps  between needs and resources, but it does not require programs to close  this gap before starting product development. In other words, the new  policy does not alter DOD’s practice of committing major investments  before knowing what resources will be required to deliver promised  capability.
There are tools being adopted under the new policy that can enable DOD  to better predict risks and estimate costs. Similar tools are also being  adopted by other weapon system programs. For example:    DOD is requiring that all space programs conduct technology maturity  assessments before key oversight decisions to assess the maturity level  of technology.
DOD is requiring space programs to more rigorously assess  alternatives, consider how their systems will operate in the context of  larger families of systems, and think through operational, technical,  and system requirements before programs are started.
The new policy seeks to improve the accuracy of cost estimates by  establishing an independent cost estimating process in partnership  with DOD’s Cost Analysis Improvement Group (CAIG) and by adopting  methodologies and tools used by the National Reconnaissance Office.  To ensure timely cost analyses, the CAIG will augment its own staff  with cost estimating personnel drawn from across the entire national  security space cost estimating community.
Moreover, to facilitate faster decision-making on programs, the policy also  calls for independent program assessments to be performed on space  programs nearing key decision points. The teams performing these  assessments are to be drawn from experts who are not directly affiliated  with the program, and they are to spend about 8 weeks studying the  program, particularly the acquisition strategy, contracting information,  cost analyses, system engineering, and requirements. After this study, the  team is to conclude its work with recommendations to the Under  Secretary of the Air Force, as DOD’s milestone decision authority for all  DOD major defense acquisition programs for space, on whether or not to  allow the program to proceed, typically using the traditional “red,”  “yellow”, and “green” assessment colors to indicate whether the program  has satisfied key criteria in areas such as requirements setting, cost  estimates, and risk reduction.
The benefits that can be derived from tools called for by the space  acquisition policy, however, will be limited since the policy allows  programs to continue to develop technologies while they are designing the  system and undertaking other product development activities. As  illustrated below, this is a very different and important departure from  DOD’s acquisition policy for other weapon systems.
As we reported last week, the revised acquisition policy for non-space  systems establishes mature technologies—that is, technologies  demonstrated in a relevant environment—as critical before entering  product development. By encouraging programs to do so, the policy puts  programs in a better position to deliver capability to the war fighter in a  timely fashion and within funding estimates because program managers  can focus on the design, system integration, and manufacturing tasks  needed to produce a product. By contrast, the space acquisition policy  increases the risk that significant problems will be discovered late in  development because programs are expected to go into development with  many unknowns about technology. In fact, DOD officials stated that  technologies may well enter product development at a stage where basic  components have only been tested in a laboratory, or an even lower level  of maturity. This means that programs will still be grappling with the  shapes and sizes of individual components while they are also trying to  design the overall system and conduct other program activities. In  essence, DOD will be concurrently building knowledge about technology  and design—an approach with a problematic history that results in a cycle  of changes, defects, and delays. Further, the consequences of problems  experienced during development will be much greater for space programs  since, under the new space acquisition policy, critical design review  occurs at the same time as the commitment to build and deliver the first  product to a customer. It is thus possible that the design review will signify  a greater commitment on a satellite program at the same time less  knowledge will be available to make that commitment.
An upcoming decision by DOD on the new TSAT program represents the  potential risks posed by the new space acquisition policy. The $12 billion  program is scheduled to start product development in December 2003,  meaning that the Air Force will formally commit to this investment and, as  required by law, set goals on cost, schedule and performance. However,  at present, TSAT’s critical technologies are underdeveloped, leaving the  Air Force without the knowledge needed to build an effective business  case for going forward with this massive investment. In fact, most of the  technologies for TSAT are at a stage where most of the work performed so  far has been based on analytical studies and a few laboratory tests or, at  best, some key components have been wired and integrated and have been  demonstrated to work together in a laboratory environment. The program  does not know yet whether TSAT’s key technologies can effectively work,  let alone work together in the harsh space environment for which they are  intended. Yet the space acquisition policy allows the Air Force to move the  program forward and to set cost, schedule, and performance goals in the  face of these unknowns. Moreover, the Air Force has scaled back its AEHF  program, whose technologies are more mature, to help pay for TSAT’s  development. Making tradeoff decisions between alternative investments  is difficult at best. Yet doing so without a solid knowledge basis only  compounds the risk of failures. Our work on program after program has  demonstrated that DOD’s optimism has rarely been justified.
section_title: Changes Needed to Optimize DOD’s Investment in Space
paragraphs: The growing importance of space systems to military and civil operations  requires DOD to achieve timely delivery of high quality capability. New  space systems not only need to support important missions such as missile  defense and reconnaissance, they need to help DOD move toward a more  “network centric” warfighting approach. At the same time, given its desire  to transform how military operations are conducted, DOD must find ways  to optimize its overall investment on weapon systems since the  transformation will require DOD to develop new cutting edge systems  while concurrently maintaining and operating legacy systems—a costly  proposition. Recognizing the need to optimize its investment, DOD has  expressed a desire to move toward an “effects-based” investment  approach, where decisions to acquire new systems are made based on  needs and joint interests versus annual budgets and parochial interests.
Changing the new space acquisition policy to clearly separate technology  development from product development is an essential first step toward  optimizing DOD’s space investment and assuring more timely delivery of  capability since it enables a program to align customer expectations with  resources, and therefore minimize problems that could hurt a program in  its design and production phase. Thus, we recommended that DOD make  this change in our recent report on the new space acquisition policy.  DOD did not agree with our recommendation because it believed that it  needs to keep up with the fast-paced development of advanced  technologies for space systems, and that its policy provides the best  avenue for doing so. In fact, it is DOD’s long-standing and continuous  inability to bring the benefits of technology to the war fighter in a timely  manner that underlies our concerns about the policy for space  acquisitions. In our reviews of numerous DOD programs, including many  satellite developments, it has been clear that committing to major  investments in design, engineering, and manufacturing capacity without  knowing a technology is mature and what resources are needed to ensure  that the technology can be incorporated into a weapon system has  consistently resulted in more money, time, and talent spent than either  was promised, planned for, or necessary. The impact of such high risk  decisions has also had a damaging effect on military capability as other  programs are taxed to meet unplanned cost increases and product units  are often cut because unit costs increase and funds run out. Moreover, as  it moves toward a more interdependent environment, DOD can simply no  longer afford to misestimate the cost and time to field capabilities—such  as TSAT—since they are needed to support other applications.
Further, policy changes are just a first step toward optimizing DOD’s  investment in space and other weapon systems. There are also some  changes that need to be made at a corporate level to foster a knowledge- based acquisition approach. As we have reported in the past, DOD needs  to remove incentives that drive premature product development decisions.  This means embracing a willingness to invest in technology development  outside a program as well as alleviating pressures to get new acquisition  programs approved and funded on the basis of requirements that must  beat out all other alternatives. Other changes—some of which have been  recognized by recent DOD studies on space acquisitions—include:    Keeping key people in place long enough so that they can affect  decisions and be held accountable. Part of the solution would be to  shorten product development times.
Providing program offices with the capability needed to craft  acquisition approaches that implement policy and to effectively  oversee the execution of programs by contractors.
Realigning responsibilities and funding between science and  technology organizations and acquisition organizations to enable the  separation of technology development from product development.
Bringing discipline to the requirements-setting process by demanding a  match between requirements and resources.
Designing and implementing test programs that deliver knowledge  when needed, including reliability testing early in design.
Lastly, DOD leadership can use this knowledge-based approach to  effectively rebalance its investment portfolio. For programs whose original  justification was based on assumptions of cost, schedule and performance  that have not been realized, having a consistent set of standards allows  DOD and the Congress to reevaluate alternatives and make investment  decisions across programs that increase the likelihood that the war fighter  will have the best possible mix of capabilities in a timely fashion.
In conclusion, using an approach for managing weapon system  investments based on knowledge instead of promises can help DOD fully  leverage the value of its investment dollars. At a time when the nation is  facing a large and growing fiscal gap, DOD’s $150 billion annual  investment in the acquisition of new weapons is the single largest area of  discretionary spending. While there are differing views on what weapons  DOD should or should not invest in and how much should be invested,  there cannot be any disagreement that within this fiscal environment, once  a consensus has been reached on the level of investment and the specific  weapons to be acquired, we should get those weapons for what was  estimated in the budget. While DOD’s revised acquisition policy for non- space systems puts DOD on a better footing toward this end, DOD’s  acquisition policy for space systems does not because it allows programs  to proceed into product development before knowing what their true costs  will be. Therefore, we continue to recommend that DOD modify its policy  to separate technology development from product development so that  needs can be matched with available technology, time, and money at the  start of a new development program.
Mr. Chairman and Members of the Subcommittee, this concludes my  statement. I would be happy to respond to any questions that you or other  members of the Subcommittee may have.
section_title: Scope and Methodology
paragraphs: In preparing for this testimony, we relied on previously issued GAO  reports on DOD’s space acquisition policy, common problems affecting  space acquisitions, SBIRS-High and other individual programs, as well as  our reports on best practices for weapon systems development. We also  analyzed DOD’s Future Years Defense Program to assess investment  trends. In addition, we reviewed DOD reports on satellite acquisition  problems. We conducted our review between October 29 and November  14, 2003 in accordance with generally accepted government auditing  standards.
For future information, please contact Katherine Schinasi or Bob Levin at  (202) 512-4841 or by email at schinasik@gao.gov or levinr@gao.gov  Individuals making key contributions to this testimony include Cristina  Chaplain, Jean Harker, and Art Gallegos.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.