Space Acquisitions: Actions Needed to Expand and Sustain Use of Best Practices
Released: 2007-04-19T00:00:00

Letter

Space Acquisitions Continue to Face Cost and Schedule Increases

The majority of major acquisition programs in DOD’s space portfolio have experienced problems during the past two decades that have driven up cost and schedules and increased technical risks. Several programs have been restructured by DOD in the face of delays and cost growth. At times, cost growth has come close to or exceeded 100 percent, causing DOD to nearly double its investment in the face of technical and other problems without realizing a better return on investment. Along with the increases, many programs are experiencing significant schedule delays—as much as 6 years—postponing delivery of promised capabilities to the warfighter. Outcomes have been so disappointing in some cases that DOD has had to go back to the drawing board to consider new ways to achieve the same, or less, capability. Some programs have been able to work through the bulk of technical problems they were facing and are on track to meet revised targets, albeit at higher costs and with delayed deliveries. Others, however, continue to face setbacks.

The following chart compares original cost estimates and current cost estimates for the broader portfolio of major space acquisitions for fiscal years 2007 through 2012. The wider the gap between original and current estimates, the fewer dollars DOD has available to invest in new programs.

The next two figures reflect differences in unit costs and total costs for satellites from the time the programs officially began to their most recent cost estimate. As the second figure notes, in several cases, DOD has had to cut back on quantity and capability in the face of escalating costs. For example, two satellites and four instruments were deleted from National Polar-orbiting Operational Environmental Satellite System (NPOESS) and four sensors are expected to have fewer capabilities. This will reduce some planned capabilities for NPOESS as well as planned coverage. Likewise, the Space Based Infrared System (SBIRS) High missile detection program deferred capabilities, such as mobile data processors for the Air Force and the Army and a fully compliant backup mission control facility, and it pushed off a decision to procure the third and fourth satellites, which will not meet SBIRS High requirements for coverage. Despite such measures, unit costs for both programs are still considerably higher than originally promised. In addition to SBIRS High and NPOESS, the programs featured in the figures include the Advanced Extremely High Frequency (AEHF) satellites, the Wideband Global SATCOM (WGS) and the Mobile User Objective System (MUOS), which are all communications satellites, and the Global Positioning System (GPS) II.

The next chart highlights the additional estimated months needed to complete programs. These additional months represent time not anticipated at the programs’ start dates. Generally, the further schedules slip, the more DOD is at risk of not sustaining current capabilities. For this reason, DOD began an alternative infrared system effort, known as the Alternative Infrared Satellite System (AIRSS), to run in parallel with the SBIRS High program.

Some programs, such as AEHF and WGS, have worked through the bulk of technical and other problems that were causing large schedule increases and cost delays. For example, the AEHF program, which has been in the final stages of development for almost 3 years, resolved issues related to its cryptographic equipment and is on track to meet a revised date for first launch. The WGS program completed rework on improperly installed fasteners, and contractors have redesigned computers to rectify data transmission errors. The program expects a first launch in June 2007. As noted in our figures, the MUOS program, which began more recently than AEHF and WGS, is generally meeting its targets, though it has yet to enter into the more difficult stages of satellite production, integration and test.

By contrast, the SBIRS High program still faces considerable risks. Recent GAO work for this subcommittee, for example, shows that the program is diverging from cost and schedule targets just months after rebaselining due to problems related to assembly, integration, and testing and that the contractor’s estimates for addressing these issues are overly optimistic. Defense Contract Management Agency reports also show that software development efforts are behind schedule—by as much as 32 percent. In addition, the contractor has already spent about 28 percent, or $66 million, of its management reserve from April 2006 to November 2006. This reserve is designed to last until 2012, but at the current rate, is likely to be depleted by May 2008. If this trend continues, $500 million in additional reserve will be needed. As noted earlier, DOD initiated an alternative effort—AIRSS—to ensure it would have continued capabilities. However, we have questions as to whether AIRSS is being pursued as a “plan B” program, as originally envisioned. Rather than seek to maintain continuity of operations, the program is focused on advancing capabilities because program managers believe there are no viable alternatives. We also found that there was disagreement among DOD stakeholders as to whether there were alternatives or not, and there was concern that the AIRSS schedule may be too compressed. Our analysis also found that there was a high degree of concurrency in the program’s schedule, which may be limiting DOD’s ability to gain knowledge from planned demonstrations and increased the potential for costly rework further in the program.

The GPS Block IIF program is also at a high risk of cost increases and schedule delays. Since our last annual assessment of the GPS Block IIF program, the program has revised its acquisition program baseline to account for cost increases and schedule delays, and requested an additional $151 million to cover these costs. The number of IIF satellites to be procured was reduced from 19 to 12. Further, the launch date of the first IIF satellite continues to slip. The original baseline showed an initial launch availability date of December 2006, but DOD’s current baseline shows July 2009—a slip of about 2.5 years. The program also learned that the contractor’s earned value management reporting system was not accurately reporting cost and schedule performance data. A DOD report also recently found that development of user equipment has not been synchronized with the development of satellites and control system, increasing the risk of substantial delays in realistic operational testing and fielding of capabilities. GPS is taking measures to address these problems. For example, this year, it did not award its contractor $21.4 million in award fees. In December 2005, GAO recommended that DOD improve its use of award fees for all weapon system contracts by specifically tying them to acquisition outcomes. A review of a sample of programs, including SBIRS High, found that this was generally not done.

The NPOESS program is also still at risk of more cost increases and schedule delays. In June 2006, DOD certified the NPOESS program to Congress, and with agreement from its program partners, DOD restructured the program. Now the NPOESS program acquisition costs are estimated to be about $11.5 billion—an increase of about $3 billion over the prior cost estimate. Before the contract was awarded, in 2002, the life cycle cost for the program was estimated to be $6.5 billion over the 24-year period from the inception of the program in 1995 through 2018. The delivery of the first two NPOESS satellites has been delayed by roughly 4 and 5 years, and as noted earlier, the number of satellites to be produced has been reduced from six to four. In addition, the number of instruments was reduced from 13 (10 sensors and 3 subsystems) to 9 instruments (7 sensors and 2 subsystems), and 4 of the remaining sensors will have fewer capabilities. The NPOESS program will incorporate any number of the deleted instruments if additional funding is provided from outside the NPOESS program. The program restructure will result in reduced satellite data collection coverage, requiring dependence on a European weather satellite for coverage during midmorning hours. Although the program has reduced the number of satellites it will produce, the cost per satellite is more than 150 percent above the original approved program baseline. The NPOESS program is now updating the cost, schedule, performance baselines and acquisition strategy, and coordinating the changes with the three agencies. The program expects these documents to be approved later this year. While work is continuing on key sensors, the program still faces potential problems in their development.

The Space Based Space Surveillance System (SBSS) system—not featured on the charts above because it is not yet a formal acquisition program— is also encountering problems. The SBSS system is to replace an aging sensor on an orbiting research and development satellite and improve the timeliness of data on objects in geosynchronous orbit. As currently planned, the initial block will consist of a single satellite and associated command, control, communications, and computer equipment. Subsequent SBSS efforts will focus on building a larger constellation of satellites to provide worldwide space surveillance of smaller objects in shorter timelines. In late 2005, an independent review team found that the program’s baseline was not executable; that the assembly, integration, and test plan was risky; and that the requirements were overstated. The SBSS program was restructured in early 2006 due to cost growth and schedule delays. The restructuring increased funding and schedule margin; streamlined the assembly, integration, and test plan; and relaxed requirements. The launch of the initial satellite was delayed to April 2009—a delay of about 18 months. Cost growth due to the restructure is about $130 million over initial estimates.

Last, additional cost increases are expected for the Evolved Expendable Launch Vehicle (EELV) program, but for reasons that are different than the ones being experienced on the satellite programs. In recent years, program cost has risen 79 percent, with a cost per unit increase of 135 percent. A chief reason for cost increases is a decline in the commercial launch market upon which the program’s business case was based. Cost increases are also a result of additional program scope, including mission assurance, assured access to space, and earned value management systems reporting. In addition, satellite vehicle weight growth and satellite launch delays have contributed to cost increases. In December 2006, Boeing and Lockheed Martin initiated a joint venture (United Launch Alliance, or ULA) that will combine the production, engineering, test, and launch operations associated with U.S. government launches of Boeing Delta and Lockheed Martin Atlas rockets. Though the EELV program office expects long-term savings to be achieved through this arrangement, the cost per launch under a recently negotiated Buy III acquisition strategy will be higher than under Buy I. According to the Air Force, this is because the contractors will incur additional costs to allow the government to perform the necessary oversight not required under Buy I. (Exact estimates of the new cost per launch have not been finalized by the program office yet.) Under the new strategy, EELV will be transitioning from a fixed-price arrangement, where launches were essentially procured as a service, to a combination of a cost-plus and fixed-price contracts. The arrangement will allow the government to exercise more oversight and to incentivize contractors through the use of award fees. But to realize these benefits, the Air Force will need to ensure it has resources (skills, expertise, and tools) to begin accumulating and analyzing detailed cost, schedule, performance, design, and technical data. In addition, it will be important to assess progress in achieving longer-term savings envisioned under ULA as well as to ensure that the combined assets of the contractors are adequately protected.

Underlying Reasons for Cost and Schedule Growth

Our past work has identified a number of causes behind the cost growth and related problems, but several consistently stand out. First, on a broad scale, DOD starts more weapon programs than it can afford, creating a competition for funding that encourages low cost estimating, optimistic scheduling, overpromising, suppressing of bad news, and, for space programs, forsaking the opportunity to identify and assess potentially better alternatives. Programs focus on advocacy at the expense of realism and sound management. Invariably, with too many programs in its portfolio, DOD is forced to continually shift funds to and from programs— particularly as programs experience problems that require additional time and money to address. Such shifts, in turn, have had costly, reverberating effects.

Figure 5 illustrates the negative cycle of incentives that come when programs compete for funding. Table 1 highlights specific areas where we found the original cost estimates of programs to be optimistic in their assumptions.

Second, as we have previously testified and reported, DOD has tended to start its programs too early, that is before it has the assurance that the capabilities it is pursuing can be achieved within available resources and time constraints. This tendency is caused largely by the funding process, since acquisition programs attract more dollars than efforts concentrating solely on proving technologies. Nevertheless, when DOD chooses to extend technology invention into acquisition, programs experience technical problems that require large amounts of time and money to fix. Moreover, when the approach is followed, cost estimators are not well positioned to develop accurate cost estimates because there are too many unknowns. Put more simply, there is no way to estimate how long it would take to design, develop, and build a satellite system when critical technologies planned for that system are still in relatively early stages of discovery and invention.

GAO, Space Acquisitions: DOD Needs to Take More Action to Address Unrealistic Initial Cost Estimates of Space Systems, GAO-07-96, (Washington, D.C.: Nov. 17, 2006). the full capability. Increasingly, DOD has preferred to make fewer but heavier, larger, and more complex satellites that perform a multitude of missions rather than larger constellations of smaller, less complex satellites that gradually increase in sophistication. This has stretched technology challenges beyond current capabilities in some cases and vastly increased the complexities related to software—a problem that affected SBIRS High and AEHF, for example.

In addition, several of the space programs discussed above began in the late 1990s, when DOD structured contracts in a way that reduced government oversight and shifted key decision-making responsibility onto contractors. This approach—known as Total System Performance Responsibility, or TSPR—was intended to facilitate acquisition reform and enable DOD to streamline a cumbersome acquisition process and leverage innovation and management expertise from the private sector. However, DOD later found that this approach magnified problems related to requirements creep and poor contractor performance. In addition, under TSPR, the government decided not to obtain certain cost data, a decision that resulted in the government having even less oversight of the programs and limited information from which to manage the programs. Further, the reduction in government oversight and involvement led to major reductions in various government capabilities, including cost-estimating and systems-engineering staff. The loss of cost-estimating and systems- engineering staff in turn led to a lack of technical data needed to develop sound cost estimates.

DOD Is Implementing Best Practices on Two New Efforts

Over the past decade, GAO has examined successful organizations in the commercial sector to identify best practices that can be applied to space and weapon system acquisitions. This work has identified a number of practices, which we have recommended that DOD adopt. Generally, we have recommended that DOD separate technology discovery from acquisition, follow an incremental path toward meeting user needs, match resources and requirements at program start, and use quantifiable data and demonstrable knowledge to make decisions to move to next phases. DOD is making efforts to instill these practices on two programs reviewed this year: the Transformational Satellite Communications System and the Space Radar program. Specifically: Successful organizations we have studied ensure that technologies are mature, that is, proven to work as intended before program start. Both TSAT and Space Radar are attempting to do this. According to their plans, critical technologies should reach at least a Technology Readiness Level (TRL) 6 by program start, meaning the technologies have been tested in a relevant environment. This stands in sharp contrast to previous programs, which have started with immature technologies, such as SBIRS and NPOESS, and it reflects the implementation of a “back to basics” policy advocated this past year by the Under Secretary of the Air Force. If these programs adhere to the TRL 6 criteria, they will greatly reduce the risk of encountering costly technical delays, though not completely. There are still significant inherent risks associated with integrating critical technologies and with developing the software needed to realize the benefits of the technologies. Moreover, the best practice programs we have studied strive for a TRL 7, where the technology has been tested in an operational environment, that is, space.

Successful organizations defer more ambitious technology efforts to corporate research departments (equivalent to science and technology (S&T) organizations in DOD) until they are ready to be added to future increments. Both programs have deferred more ambitious technology development efforts to the science and technology environment. TSAT, for example, deferred the wide-field of view multi-access laser communication technology, and is contributing about $16.7 million for “off-line” maturation of this technology that could be inserted into future increments. It has laid out incremental advances in other capabilities over two increments. Space Radar has deferred lithium-ion batteries, more efficient solar cells, and onboard processing for its first increment, and like TSAT, is contributing toward their development by S&T organizations. At this time, Space Radar has not defined details of an increment beyond the first one.

Successful organizations extensively research and define requirements before program start to ensure that they are achievable, given available resources, and that they do not define requirements after starting programs. Both programs have also employed systems engineers to help determine achievability of requirements. The TSAT program has reached agreement on requirements with its users— primarily in terms of what will be included in the first several blocks of the program and what will not be included. The Space Radar program has instituted several processes designed to achieve consensus on requirements across a range of diverse users. It still needs to formalize agreements related to these processes and also identify key performance parameters. This is important because Space Radar is to be shared by the military and intelligence communities—each with different specific needs for the system and very specific roles and responsibilities with regard to the data being produced by Space Radar and its users. It has been reported recently that conflicts in roles and responsibilities have arisen on dissemination of data being produced by a small tactical satellite (TacSat 2) recently launched by DOD for use by military commanders.

It remains to be seen whether TSAT and Space Radar will take additional steps that successful organizations take to position programs for success. For example: The organizations we have studied do not go ahead with program start milestone decisions if there are still gaps between requirements and resources. TSAT and Space Radar have indicated that they intend to do the same, but there are external pressures on both programs to provide needed capabilities.

The organizations we have studied hold program managers accountable for their estimates and require program managers to stay with a project to its end. We have made recommendations to DOD to instill similar practices departmentwide, but these have yet to be implemented. Further, there are still incentives in place to keep program managers’ tenures relatively short. Promotions, for example, often depend on having varied management experience rather than sustained responsibility for one program.

The organizations we studied have developed common templates and tools to support data gathering and analysis and maintain databases of historical costs, schedule, quality, test, and performance data. Cost estimates themselves are continually monitored and regularly updated through a series of gates or milestone decisions that demand programs assess readiness and remaining risk within key sectors of the program as well as overall cost and schedules. We saw indications that TSAT and Space Radar were using more robust tools to analyze risks, costs, and schedule than programs have done in the past. However, it remains to be seen how these practices will be reflected in official cost estimates. In the past, we have found space program estimates were simply too optimistic and that independent estimates produced by DOD’s Cost Analysis Improvement Group were not being used. DOD agreed with our findings and asserted it was taking actions to address them.

Actions Needed to Sustain Commitment to Improvements

The Air Force’s continued efforts to instill best practices on Space Radar and TSAT are good first steps toward addressing acquisition problems in the space portfolio. They represent significant shifts in thinking about how space systems should be developed as well as commitment from senior leadership. But sustaining these reforms will not be easy. The programs are not immune to funding pressures that have encouraged too much optimism. They are also being undertaken as DOD is addressing shortfalls in critical technical, business, and program management skills. Further, processes and policies key to sustainment and broader use of best practices have not been changed to further reflect the kinds of changes taking place on Space Radar and TSAT.

First, new programs still must compete for limited funding. As DOD seeks to fund Space Radar and TSAT, it will be (1) undertaking other new, costly efforts, including GPS III, SBSS, and AIRSS; (2) addressing cost overruns associated with programs like SBIRS High and GPS; and (3) facing increased pressures to increase investments in assets designed to protect space systems. In total, these efforts will increase DOD’s investment for all major space acquisitions from $6.31 billion to $9.22 billion, or about 46 percent over the next 3 years. More may be needed if technical, software, and other problems on current programs worsen. At the same time, investment needs for other weapon systems are also on the rise, while long-term budget forecasts indicate that considerably fewer dollars will be available for discretionary spending in coming years rather than more.

In prior reports, we have stated that as long as too many programs compete for too few dollars in DOD, programs will be incentivized to produce optimistic estimates and suppress bad news. They will view success as securing the next installment of funds versus delivering capability within cost and schedule goals. We have recommended that DOD guide its decisions to start space and other weapons acquisition programs with an overall investment strategy that would identify priorities for funding so that space systems that are expected to play a critical role in transformation, such as Space Radar and TSAT, could be prioritized along with other legacy and transformational systems.

Let me take a moment to illustrate why an investment strategy is critical. We have reported in the past that DOD and the Air Force have waited too long to establish priorities or make trade-off decisions. We have also reported that frequent funding shifts have hurt programs that were performing well or further damaged troubled programs. We have also reported cases where DOD and the Air Force have walked away from opportunities to save costs in lot buys or leverage knowledge already gained in legacy programs in favor of starting new programs that promise much more advanced capability but have little knowledge to back up that promise. Today, DOD is on track to cut short the AEHF program in order to pursue TSAT. It has stated it may also do the same for SBIRS to pursue AIRSS. In both cases, DOD would be forgoing savings that it had already negotiated for lot buys and in effect, paying significantly more for nonrecurring engineering. While these decisions have the potential to enable DOD to obtain advanced capability sooner (provided best practices are followed on the new programs), they should have been made much earlier and more strategically in order to stem investment losses.

DOD’s own reports recognize that investment planning needs to be instilled in weapon acquisitions. A February 2007 report, in response to a requirement in the John Warner National Defense Authorization Act for Fiscal Year 2007, outlines steps that DOD is taking to better prioritize and fund programs. The initiatives include (1) establishing a new concept decision review to provide decision makers with an early opportunity to evaluate trade-offs among alternative approaches to meeting a capability need, (2) testing portfolio management approaches in selected capability areas to facilitate more strategic choices about how to allocate resources across programs, and (3) capital budgeting as a potential means to stabilize program funding. While these developments are promising, we recently reported that such initiatives do not fundamentally change DOD’s existing service-centric framework for making weapon system investment decisions. Moreover, it will take some time to determine their success in enabling more effective funding prioritization.

Second, space programs are facing capacity shortfalls. These include shortages of staff with science and engineering backgrounds as well as staff with program management and cost estimating experience. During our review this year, the TSAT program cited shortages of space acquisition personnel as a key challenge that increases risk for the program. Due to broader Air Force cuts in workforce, the program did not expect to be able to fill technical positions needed to accompany plans to ramp up spending. During our review of DOD’s space cost estimating function, Air Force space cost estimating organizations and program offices said that they believed their cost-estimating resources were inadequate to do a good job of accurately predicting costs. Because of the decline in in-house cost-estimating resources, space program offices and Air Force cost-estimating organizations are now more dependent on support contractors. At 11 of 13 program offices we informally surveyed, contractors accounted for 64 percent of cost-estimating personnel. This reliance raised questions from the cost-estimating community about whether numbers and qualifications of government personnel were sufficient to provide oversight of and insight into contractor cost estimates. In addition to technical and cost estimating skills, DOD and GAO studies have also pointed to capacity shortfalls in program management. According to DOD’s Young Panel report, government capabilities to lead and manage the space acquisition process have seriously eroded, in part because of actions taken in the acquisition reform environment of the 1990s. During our 2005 review of program management, we surveyed DOD’s major weapon system program managers and interviewed program executive officers, who similarly pointed to critical skill shortages in program management, systems engineering, and software development. The Air Force and DOD recognize these shortfalls and are taking actions to address them, but these will take time to implement. It is important that in the interim, the Air Force identify and take steps to grow or retain skill sets that should be organic, such as highly specialized knowledge of certain military space technologies. During both our cost estimating and space system reviews, program officials noted that it can take several years for new technical staff to build knowledge and skills unique to military space.

Our past work has also pointed to capacity shortfalls that go beyond workforce. For example, in 2006, we reported that cost estimation data and databases are incomplete, insufficient, and outdated. And in our testimony last year, we pointed to limited opportunities and funding for space technologies, and the lack of low-cost launch vehicles. It is our understanding that the Air Force and DOD are working to address all of these shortfalls. Budget plans show, for example, an increase of nearly $11 million in funding for the space test program beginning in 2009— about 23 percent.

Last, policies that surround space acquisition need to be further revised to ensure best practices are instilled and sustained. For example, DOD’s space acquisition policy does not require that programs such as TSAT and Space Radar achieve a TRL 6 or higher for key technologies before being formally started (KDP B). Instead, it is suggested that TRL 6 be achieved at preliminary decision review (KDP C) or soon after. Given that there are many pressures and incentives that are driving space and other weapon programs to begin too early and to drive for dramatic rather than incremental leaps in capability, DOD needs acquisition policies that ensure programs have the knowledge they need to make investment decisions and that DOD and Congress have a more accurate picture of how long and how much it will take to get the capability that is being promised. In addition, although the policy requires that independent cost estimates be prepared by bodies outside the acquisition chain of command, it does not require that they be relied upon to develop program budgets. Officials within the space cost estimating community also believed that the policy was unclear in defining roles and responsibilities for cost estimators. We continue to recommend changes be made to the policy—not only to further ingrain the shift in thinking about how space systems should be developed, but to ensure that the changes current leaders are trying to make can be sustained beyond their tenure.

In closing, we support efforts to instill best practices on programs like Space Radar and TSAT. They are critical to enabling DOD to break the cycle of space acquisition problems by matching resources to requirements before program start. We encourage DOD to build on this momentum by extending a best practice approach to its entire space portfolio. For newer efforts, such as AIRSS, this means reexamining requirements and alternative means of satisfying those requirements and clarifying the true purpose of the program. For current programs, such as SBIRS, this means continuing to track risks and dedicating resources necessary to mitigate those risks, leveraging management tools such as earned value management analyses, and finding ways to incentivize contractors to perform well. For the broader portfolio, this means ensuring programs have all the right resources to enable success. These include adequate levels of funding accompanied by short- and long-term investment plans, adequate skills and capabilities, as well as data, policy, and processes, accountability and leadership support.

Appendix I: Scope and Methodology

In preparing for this testimony, we relied on previously issued GAO reports on assessments of individual space programs, common problems affecting space system acquisitions, and DOD’s space acquisition policy. We also relied on our best practices studies, which have examined pressures and incentives affecting space system acquisition programs, the optimal levels of knowledge needed to successfully execute programs, and complementary management practices and processes that have helped commercial and DOD programs to reduce costs and cycle time. In addition, we analyzed DOD’s Selected Acquisition Reports to assess cost increases and investment trends. We conducted our review between March 19 and April 13, 2007 in accordance with generally accepted government auditing standards.

Appendix II: Contacts and Acknowledgments

For in information, please contact Cristina Chaplain at 202-512-4841 or chaplainc@gao.gov. Individuals making contributions to this testimony include, Art Gallegos, Jeff Barron, Tony Beckham, Noah Bleicher, Greg Campbell, Maricela Cherveny, Claire Cyrnak, Jean Harker, and Rich Horiuchi.