Defense Acquisitions: Improvements Needed in Space Systems Acquisition Policy to Optimize Growing Investment in Space
Released: 2003-11-18T00:00:00

Letter

The Importance of DOD’s Space Systems is Growing

DOD’s current space network is comprised of constellations of satellites, ground-based systems, and associated terminals and receivers. Among other things, these assets are used to perform intelligence, surveillance, and reconnaissance functions; perform missile warning; provide communication services to DOD and other government users; provide weather and environmental data; and provide positioning and precise timing data to U.S. forces as well as national security, civil, and commercial users.

All of these systems are playing an increasingly important role in military operations. According to DOD officials, for example, in Operation Iraqi Freedom, approximately 70 percent of weapons were precision-guided, most of those using Global Positioning System (GPS) capabilities. Weather satellites enabled war fighters to not only prepare for, but also take advantage of blinding sandstorms. Communication and intelligence satellites were also heavily used to plan and carry out attacks and to assess post-strike damage. Some of DOD’s satellite systems—such as GPS—have also grown into international use for civil and military applications and commercial and personal uses. Moreover, the demand for space-based capabilities is outpacing DOD’s current capacity. For example, even though DOD has augmented its own satellite communications capacity with commercial satellites, in each major conflict of this past decade, senior military commanders reported shortfalls in capacity, particularly for rapid transmission of large data files, such as those created by imagery sensors.

DOD is looking to space to play an even more pivotal role in future military operations. As such, it is developing several families of new, expensive, and technically challenging satellites, which are expected to require dramatically increased investments over the next decade. For example, DOD is building new satellites that will use laser optics to transport information over long distances in much larger quantities than radio waves. The system, known as the Transformational Satellite, or TSAT, is to be the cornerstone of DOD’s future communications architecture. Many space, air, land, and sea-based systems will depend on TSAT to receive and transmit large amounts of data to each other as DOD moves toward a more “network centric” war-fighting approach. DOD is also building a new space-based radar (SBR) system, which is to employ synthetic aperture radar and other advanced technologies to enable DOD to have 24-hour coverage over a large portion of the Earth on a continuous basis and allow military forces a “deep-look” into denied areas of interest, on a non-intrusive basis without risk to personnel or resources. SBR itself is expected to generate large amounts of imagery data, and it will rely on TSAT to deliver this data to war fighters.

As figure 1 shows, the costs of these and other new efforts will increase DOD’s annual space investment significantly. For example, based on the 2003 President’s budget, acquisition costs for new satellite programs and launch services in the next 4 years are expected to grow by 115 percent— from $3.5 billion to about $7.5 billion. Costs beyond that period are as yet unknown. While DOD’s budget documents show a decrease in 2009 for these systems to $6.4 billion—they do not include procurement costs for some of the largest programs, including TSAT, GPS III, SBR, Space Tracking and Surveillance System (STSS), and Space-Based Surveillance System (SBSS), which DOD will begin fielding beginning 2011. Nor do these numbers reflect the totality of DOD’s investment in space. For example, ground stations and user equipment all require significant investment and that investment will likely increase as the new programs mature.

Table 1 identifies specific programs factored into our analysis of upcoming investments. It also shows that DOD will be fielding many of the new programs within just a few years of each other.

Grounding Decisions in Knowledge is Vital for DOD’s Space Investment

For the past 6 years, we have been examining ways DOD can get better outcomes from its investment in weapon systems, drawing on lessons learned from the best, mostly commercial, product development efforts. Our work has shown that leading commercial firms expect that their managers will deliver high quality products on time and within budgets. Doing otherwise could result in losing a customer in the short term and losing the company in the long term. Thus, these firms have adopted practices that put their individual programs in a good position to succeed in meeting these expectations on individual products. Collectively, these practices ensure that a high level of knowledge exists about critical facets of the product at key junctures and is used to make decisions to deliver capability as promised. We have assessed DOD’s space acquisition policy as well as its revised acquisition policy for other weapon systems against these practices.

Our reviews have shown that there are three critical junctures at which firms must have knowledge to make large investment decisions. First, before a product development is started, a match must be made between the customers’ needs and the available resources—technical and engineering knowledge, time, and funding. Second, a product’s design must demonstrate its ability to meet performance requirements and be stable about midway through development. Third, the developer must show that the product can be manufactured within cost, schedule, and quality targets and is demonstrated to be reliable before production begins. If the knowledge attained at each juncture does not confirm the business case on which the acquisition was originally justified, the program does not go forward. These precepts hold for technically complex, high volume programs as well as low volume programs such as satellites.

In applying the knowledge-based approach, the most-leveraged investment point is the first: matching the customer’s needs with the developer’s resources. The timing of this match sets the stage for the eventual outcome—desirable or problematic. The match is ultimately achieved in every development program, but in successful development programs, it occurs before product development begins. When the needs and resources match is not made before product development, realistic cost and schedule projections become extremely difficult to make. Moreover, technical problems can disrupt design and production efforts. Thus, leading firms make an important distinction between technology development and product development. Technologies that are not ready continue to be developed in the technology base—they are not included in a product development.

With technologically achievable requirements and commitment of sufficient resources to complete the development, programs are better able to deliver products at cost and on schedule. When knowledge lags, risks are introduced into the acquisition process that can result in cost overruns, schedule delays, and inconsistent product performance. As we recently testified, such problems, in turn, can reduce the buying power of the defense dollar, delay capabilities for the war fighter, and force unplanned—and possibly unnecessary—trade-offs in desired acquisition quantities and an adverse ripple effort among other weapon programs or defense needs. Moreover, as DOD moves more toward a system-of- systems approach—where systems are being designed to be highly interdependent and interoperable—it is exceedingly important that each individual program stay on track.

Decisions on Space Programs Have Not Been Sufficiently Grounded in Knowledge

Our past work has shown that space programs have not typically achieved a match between needs and resources before starting product development. Instead, product development was often started based on a rigid set of requirements and a hope that technology would develop on a schedule. At times, even more requirements were added after the program began. When technology did not perform as planned, adding resources in terms of time and money became the primary option for solving problems, since customer expectations about the products’ performance already became hardened.

For example, after starting its Advanced Extremely High Frequency (AEHF) communications satellite program, DOD substantially and frequently changed requirements. In addition, after the launch failure of one of DOD’s legacy communications satellites, DOD decided to accelerate its plans to build AEHF satellites. The contractors proposed, and DOD accepted, a high risk schedule that turned out to be overly optimistic and highly compressed, leaving little room for error and depending on a precise chain of events taking place at certain times. Moreover, at the time DOD decided to accelerate the program, it did not have funding needed to support the activities and manpower needed to design and build the satellites quicker. The effects of DOD’s inability to match needs to resources were significant. Total program cost estimates produced by the Air Force reflected an increase from $4.4 billion in January 1999 to $5.6 billion in June 2001—a difference of 26 percent. Although considered necessary, many changes to requirements were substantial, leading to cost increases of hundreds of millions of dollars because they required major design modifications. Also, schedule delays occurred when some events did not occur on time, and additional delays occurred when the program faced funding gaps. Scheduling delays eventually culminated into a 2-year delay in the launch of the first satellite. We also reported that there were still technical and production risks that need to be overcome in the AEHF program, such as a less-than-mature satellite antenna system and complications associated with the production of the system’s information security system.

Another example can be found with DOD’s Space-Based Infrared System (SBIRS)–High program, which is focused on building high-orbiting satellites that can detect ballistic missile launches. Over time, costs have more than doubled for this program. Originally, total development costs for SBIRS-High were estimated at $1.8 billion. In the fall of 2001, DOD identified potential cost growth of $2 billion or more, triggering a mandatory review and recertification under 10 U.S.C. section 2433. Currently, the Air Force estimates research and development costs for SBIRS-High to be $4.4 billion. We reported that when DOD’s SBIRS-High satellite program began in 1994, none of its critical technologies were mature. Moreover, according to a DOD-chartered independent review team, the complexity, schedule, and resources needed to develop SBIRS- High, in hindsight, were misunderstood when the program began. This led to an immature understanding of how requirements translated into detailed engineering solutions. We recently reported to this subcommittee that while the SBIRS restructuring implemented a number of needed management changes, the program continues to experience problems and risks related to changing requirements, design instability, and software development concerns. We concluded that if the Air Force continues to add new requirements and program content while prolonging efforts to resolve requirements that cannot be met, the program will remain at risk of not achieving, within schedule, its intended purposes—to provide an early warning and tracking system superior to that of its current ballistic missile detection system.

DOD has also initiated several programs and spent several billion dollars over the past 2 decades to develop low-orbiting satellites that can track ballistic missiles throughout their flight. However, it has not launched a single satellite to perform this capability. We have reported that a primary problem affecting these particular programs was that DOD and the Air Force did not relax rigid requirements to more closely match technical capabilities that were achievable. Program baselines were based on artificial time and/or money constraints. Over time, it became apparent that the lack of knowledge of program challenges had led to overly optimistic schedules and budgets that were funded at less than what was needed. Attempts to stay on schedule by approving critical milestones without meeting program criteria resulted in higher costs and more slips in technology development efforts. For example, our 1997 and 2001 reviews of DOD’s $1.7 billion SBIRS-Low program (which was originally a part of the SBIRS-High program) showed that the program would enter into the product development phase with critical technologies that were immature and with optimistic deployment schedules. Some of these technologies were so critical that SBIRS-Low would not be able to perform its mission if they were not available when needed. DOD eventually restructured the SBIRS-Low program because of the cost and scheduling problems, and it put the equipment it had partially built into storage. In view of the program’s mismatch between expectations and what it could achieve, the Congress directed DOD to restructure the program (now under the responsibility of the Missile Defense Agency) as a research and development effort.

New Space Policy Allows Programs to Go Forward with Key Unknowns

DOD’s new space acquisition policy may help increase insight into gaps between needs and resources, but it does not require programs to close this gap before starting product development. In other words, the new policy does not alter DOD’s practice of committing major investments before knowing what resources will be required to deliver promised capability.

There are tools being adopted under the new policy that can enable DOD to better predict risks and estimate costs. Similar tools are also being adopted by other weapon system programs. For example: DOD is requiring that all space programs conduct technology maturity assessments before key oversight decisions to assess the maturity level of technology.

DOD is requiring space programs to more rigorously assess alternatives, consider how their systems will operate in the context of larger families of systems, and think through operational, technical, and system requirements before programs are started.

The new policy seeks to improve the accuracy of cost estimates by establishing an independent cost estimating process in partnership with DOD’s Cost Analysis Improvement Group (CAIG) and by adopting methodologies and tools used by the National Reconnaissance Office. To ensure timely cost analyses, the CAIG will augment its own staff with cost estimating personnel drawn from across the entire national security space cost estimating community.

Moreover, to facilitate faster decision-making on programs, the policy also calls for independent program assessments to be performed on space programs nearing key decision points. The teams performing these assessments are to be drawn from experts who are not directly affiliated with the program, and they are to spend about 8 weeks studying the program, particularly the acquisition strategy, contracting information, cost analyses, system engineering, and requirements. After this study, the team is to conclude its work with recommendations to the Under Secretary of the Air Force, as DOD’s milestone decision authority for all DOD major defense acquisition programs for space, on whether or not to allow the program to proceed, typically using the traditional “red,” “yellow”, and “green” assessment colors to indicate whether the program has satisfied key criteria in areas such as requirements setting, cost estimates, and risk reduction.

The benefits that can be derived from tools called for by the space acquisition policy, however, will be limited since the policy allows programs to continue to develop technologies while they are designing the system and undertaking other product development activities. As illustrated below, this is a very different and important departure from DOD’s acquisition policy for other weapon systems.

As we reported last week, the revised acquisition policy for non-space systems establishes mature technologies—that is, technologies demonstrated in a relevant environment—as critical before entering product development. By encouraging programs to do so, the policy puts programs in a better position to deliver capability to the war fighter in a timely fashion and within funding estimates because program managers can focus on the design, system integration, and manufacturing tasks needed to produce a product. By contrast, the space acquisition policy increases the risk that significant problems will be discovered late in development because programs are expected to go into development with many unknowns about technology. In fact, DOD officials stated that technologies may well enter product development at a stage where basic components have only been tested in a laboratory, or an even lower level of maturity. This means that programs will still be grappling with the shapes and sizes of individual components while they are also trying to design the overall system and conduct other program activities. In essence, DOD will be concurrently building knowledge about technology and design—an approach with a problematic history that results in a cycle of changes, defects, and delays. Further, the consequences of problems experienced during development will be much greater for space programs since, under the new space acquisition policy, critical design review occurs at the same time as the commitment to build and deliver the first product to a customer. It is thus possible that the design review will signify a greater commitment on a satellite program at the same time less knowledge will be available to make that commitment.

An upcoming decision by DOD on the new TSAT program represents the potential risks posed by the new space acquisition policy. The $12 billion program is scheduled to start product development in December 2003, meaning that the Air Force will formally commit to this investment and, as required by law, set goals on cost, schedule and performance. However, at present, TSAT’s critical technologies are underdeveloped, leaving the Air Force without the knowledge needed to build an effective business case for going forward with this massive investment. In fact, most of the technologies for TSAT are at a stage where most of the work performed so far has been based on analytical studies and a few laboratory tests or, at best, some key components have been wired and integrated and have been demonstrated to work together in a laboratory environment. The program does not know yet whether TSAT’s key technologies can effectively work, let alone work together in the harsh space environment for which they are intended. Yet the space acquisition policy allows the Air Force to move the program forward and to set cost, schedule, and performance goals in the face of these unknowns. Moreover, the Air Force has scaled back its AEHF program, whose technologies are more mature, to help pay for TSAT’s development. Making tradeoff decisions between alternative investments is difficult at best. Yet doing so without a solid knowledge basis only compounds the risk of failures. Our work on program after program has demonstrated that DOD’s optimism has rarely been justified.

Changes Needed to Optimize DOD’s Investment in Space

The growing importance of space systems to military and civil operations requires DOD to achieve timely delivery of high quality capability. New space systems not only need to support important missions such as missile defense and reconnaissance, they need to help DOD move toward a more “network centric” warfighting approach. At the same time, given its desire to transform how military operations are conducted, DOD must find ways to optimize its overall investment on weapon systems since the transformation will require DOD to develop new cutting edge systems while concurrently maintaining and operating legacy systems—a costly proposition. Recognizing the need to optimize its investment, DOD has expressed a desire to move toward an “effects-based” investment approach, where decisions to acquire new systems are made based on needs and joint interests versus annual budgets and parochial interests.

Changing the new space acquisition policy to clearly separate technology development from product development is an essential first step toward optimizing DOD’s space investment and assuring more timely delivery of capability since it enables a program to align customer expectations with resources, and therefore minimize problems that could hurt a program in its design and production phase. Thus, we recommended that DOD make this change in our recent report on the new space acquisition policy. DOD did not agree with our recommendation because it believed that it needs to keep up with the fast-paced development of advanced technologies for space systems, and that its policy provides the best avenue for doing so. In fact, it is DOD’s long-standing and continuous inability to bring the benefits of technology to the war fighter in a timely manner that underlies our concerns about the policy for space acquisitions. In our reviews of numerous DOD programs, including many satellite developments, it has been clear that committing to major investments in design, engineering, and manufacturing capacity without knowing a technology is mature and what resources are needed to ensure that the technology can be incorporated into a weapon system has consistently resulted in more money, time, and talent spent than either was promised, planned for, or necessary. The impact of such high risk decisions has also had a damaging effect on military capability as other programs are taxed to meet unplanned cost increases and product units are often cut because unit costs increase and funds run out. Moreover, as it moves toward a more interdependent environment, DOD can simply no longer afford to misestimate the cost and time to field capabilities—such as TSAT—since they are needed to support other applications.

Further, policy changes are just a first step toward optimizing DOD’s investment in space and other weapon systems. There are also some changes that need to be made at a corporate level to foster a knowledge- based acquisition approach. As we have reported in the past, DOD needs to remove incentives that drive premature product development decisions. This means embracing a willingness to invest in technology development outside a program as well as alleviating pressures to get new acquisition programs approved and funded on the basis of requirements that must beat out all other alternatives. Other changes—some of which have been recognized by recent DOD studies on space acquisitions—include: Keeping key people in place long enough so that they can affect decisions and be held accountable. Part of the solution would be to shorten product development times.

Providing program offices with the capability needed to craft acquisition approaches that implement policy and to effectively oversee the execution of programs by contractors.

Realigning responsibilities and funding between science and technology organizations and acquisition organizations to enable the separation of technology development from product development.

Bringing discipline to the requirements-setting process by demanding a match between requirements and resources.

Designing and implementing test programs that deliver knowledge when needed, including reliability testing early in design.

Lastly, DOD leadership can use this knowledge-based approach to effectively rebalance its investment portfolio. For programs whose original justification was based on assumptions of cost, schedule and performance that have not been realized, having a consistent set of standards allows DOD and the Congress to reevaluate alternatives and make investment decisions across programs that increase the likelihood that the war fighter will have the best possible mix of capabilities in a timely fashion.

In conclusion, using an approach for managing weapon system investments based on knowledge instead of promises can help DOD fully leverage the value of its investment dollars. At a time when the nation is facing a large and growing fiscal gap, DOD’s $150 billion annual investment in the acquisition of new weapons is the single largest area of discretionary spending. While there are differing views on what weapons DOD should or should not invest in and how much should be invested, there cannot be any disagreement that within this fiscal environment, once a consensus has been reached on the level of investment and the specific weapons to be acquired, we should get those weapons for what was estimated in the budget. While DOD’s revised acquisition policy for non- space systems puts DOD on a better footing toward this end, DOD’s acquisition policy for space systems does not because it allows programs to proceed into product development before knowing what their true costs will be. Therefore, we continue to recommend that DOD modify its policy to separate technology development from product development so that needs can be matched with available technology, time, and money at the start of a new development program.

Mr. Chairman and Members of the Subcommittee, this concludes my statement. I would be happy to respond to any questions that you or other members of the Subcommittee may have.

Scope and Methodology

In preparing for this testimony, we relied on previously issued GAO reports on DOD’s space acquisition policy, common problems affecting space acquisitions, SBIRS-High and other individual programs, as well as our reports on best practices for weapon systems development. We also analyzed DOD’s Future Years Defense Program to assess investment trends. In addition, we reviewed DOD reports on satellite acquisition problems. We conducted our review between October 29 and November 14, 2003 in accordance with generally accepted government auditing standards.

For future information, please contact Katherine Schinasi or Bob Levin at (202) 512-4841 or by email at schinasik@gao.gov or levinr@gao.gov Individuals making key contributions to this testimony include Cristina Chaplain, Jean Harker, and Art Gallegos.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.