DOD Space Systems: Additional Knowledge Would Better Support Decisions about Disaggregating Large Satellites
Released: 2014-10-30T00:00:00

Letter

Background

DOD is considering alternative approaches—including disaggregation—to providing space-based capabilities to address challenges related to high- cost satellite programs and increasing threats to its space system capabilities. The size, capability, and complexity of DOD’s satellites have tended to increase over the years, making them more costly to build. For example, Milstar, the protected communications satellite that preceded AEHF, weighed about 10,000 pounds, whereas the AEHF satellite weighs about 13,600 pounds and provides ten times the communications capacity. Additionally, SBIRS satellites are more capable than the Defense Support Program satellites they are to replace, offering greater sensor sensitivity and supporting a broader set of missions. According to DOD officials, it has made economic sense to maximize satellite capabilities as much as launch vehicle constraints would allow to fully utilize the capacity of the selected launch vehicle—which can cost well over $100 million for the vehicles DOD primarily uses. Additionally, certain requirements were consolidated to more efficiently provide capabilities, that is, fewer but more capable satellites could execute more missions than in the past.

This approach of building large, multifunctional satellites also increased acquisition risks since the design and manufacturing of satellites became more complex. Total acquisition costs for the AEHF satellite constellation, for example, have more than doubled, from $6.7 billion at program inception in 2001 to $14.5 billion currently. This was partially due to the acquisition of an additional satellite, but also due to technical challenges and associated schedule delays of about 6 years for the launch of the first satellite. Our reports over the years have found that the significant cost and schedule growth for AEHF, SBIRS, and the environmental satellites DOD was acquiring in a similar fashion were partly due to design and manufacturing complexity. We also reported that DOD’s tendency to build large, monolithic satellites limited its ability to provide capabilities sooner and contributed to higher costs through developing extensive new designs, custom-made spacecraft, and payloads to meet the needs of multiple users. At the same time, other issues drove up costs for these programs including immature technologies, requirements instability, concurrent development and production, poor management and oversight, and reduced testing. We have reported that DOD has taken actions to address many of these issues, and in recent years, DOD has largely overcome cost and schedule problems for the programs currently in production, and additional satellites of the same design are now being launched.

Reducing acquisition cost and risk are not the only reasons DOD is considering new approaches. According to Air Force Space Command, U.S. space systems face intentional and unintentional threats, which have increased rapidly over the past 20 years. These include radio frequency interference (including jamming), laser dazzling and blinding, kinetic intercept vehicles, and ground system attacks. Additionally, the hazards of the already-harsh space environment (e.g., extreme temperature fluctuations and radiation) have increased, including numbers of active and inactive satellites, spent rocket bodies, and other fragments and debris. In response, recent governmentwide and DOD strategic and policy guidance have stressed the need for U.S. space systems to be survivable or resilient against such threats.

According to the Air Force, to maintain protected satellite communications, SBIRS, and space based environmental monitoring capabilities, new satellites and/or payloads would be needed beginning in 2024, 2025, and 2015, respectively. impacts of decisions could reach far beyond the space segment, affecting ground control systems, satellite user equipment, and communications networks.

While this report is focused on the concept of disaggregation, it is important to note that DOD officials we spoke with stressed that disaggregation is only one of a number of approaches DOD is considering for its future space system designs. For example, in addition to disaggregation, DOD is considering the possibility of making satellites more maneuverable and building in defense capabilities to protect themselves as a means to increase survivability. Another approach under consideration is to evolve, or enhance, the capabilities of the large multifunctional satellites DOD is currently building. According to these officials, the primary considerations for studying these approaches and making decisions on which to use relate to finding the right balance between:

Affordability—the determination that the life cycle cost of a program is in line with long-range plans;

Resilience—the ability of a system to support the functions necessary for mission success in spite of hostile action or adverse conditions;and

Capability—the ability to execute specified actions.

To build consensus, and to conduct a more rigorous analysis of options, DOD is currently conducting studies that will consider future system architectures. Among these are Analyses of Alternatives (AOAs) that compare the effectiveness, suitability, risks, and estimated costs of alternatives for future protected satellite communications, SBIRS, and space-based environmental monitoring.design and components to pursue is critical because most of a program’s total life-cycle cost is influenced by decisions made before it is approved to start development.

Disaggregation Offers an Array of Benefits and Limitations, but Significant Barriers to Implementation Exist

It is not yet known whether and to what extent disaggregation, on a broad scale, can enable DOD to reduce acquisition costs and increase the resilience of its satellite systems. DOD has had limited experience with disaggregating satellites and is still in the relatively early stages of quantifying the costs and benefits of disaggregating its satellites. The potential benefits of disaggregation, such as reduced costs associated with shorter development cycles and simpler designs, have been discussed in various Air Force publications. But less has been said about potential limitations, such as changes that would need to be made to interconnect systems—such as ground stations, user equipment, and communications networks—and the investment those changes would require. Moreover, many potential benefits can also be accompanied by drawbacks. For instance, with capabilities distributed across multiple platforms, rather than centralized onto just a few satellites, it may be more difficult for an adversary to target all assets to attack full system capabilities. However, with increased numbers of satellites, the space environment becomes more congested, increasing risk of radio frequency interference and the potential for on-orbit contact with and damage from debris or other assets. Given the breadth of impacts disaggregation can have, it may not be possible to have complete knowledge of its effects. But experts agree that decisionmaking would benefit from assessments that look well beyond a single satellite program.

Further, even if DOD concludes disaggregation would have net positive effects, there are significant longstanding barriers to implementation. For instance, our work continues to find that there are time gaps—sometimes years—in aligning the delivery of satellites with associated user equipment and ground systems, which means that satellites may be in orbit for a long time with limited use. Without resolving the leadership, management, and oversight issues that have led to these delays, time gaps may well be exacerbated by disaggregation. Furthermore, disaggregation may require a more complex infrastructure and additional investments to support the satellites.

Disaggregation Offers a Range of Potential Benefits and Limitations

DOD, civilian agencies, industry officials, and researchers have identified a broad range of potential benefits and limitations associated with disaggregating military space systems that span numerous aspects of space systems acquisitions and operations. Views on the potential benefits and limitations are widely distributed among the various experts, even within DOD. Because these potential effects are dependent on various disaggregation approaches and have not yet been validated in practice for large, complex space systems, some points may appear to contradict one another. Further, some potential effects may not be unique to disaggregation, but may also apply to other changes from the current approach. Tables 1 through 5 describe the potential benefits and limitations in terms of affordability and life cycle costs, resilience, capability, the acquisition process, and the industrial base identified by experts. The contents of the tables are not exhaustive, but each of the potential benefits and limitations included were identified by more than one expert as important considerations for deciding whether to pursue a disaggregated approach.

Affordability and Life Cycle Costs

Affordability—the feasibility of producing and supporting a program throughout its life cycle and within budget—is one of the key drivers behind considering disaggregation as an option, according to experts. Disaggregation may offer a more affordable approach to providing capabilities because of the potential for reduced per-satellite costs over the long term. Acquiring smaller, less complex satellites may require less time and effort to develop and produce, for example. This may be in part due to improved requirements discipline, as more frequent production rates may allow program managers to delay new requirements to the next production cycle instead of incorporating them into ongoing timelines mid- stream. However, a larger number of small satellites may be needed to provide the same level of capability overall, and the transition from existing system designs to disaggregation could increase costs in the near term. Additionally, given the current budget environment facing DOD, the transition to a new approach may be cost prohibitive, as resources may not be available to maintain and operate a legacy system and develop and field a new system at the same time. Table 1 describes how affordability could be impacted by disaggregation.

Another main driver experts cited for examining disaggregation is the potential for increased resilience—the ability of a system to support the functions necessary for mission success in spite of hostile action or adverse conditions, according to one DOD definition—of space systems. Distributing capabilities across more satellites will increase the number and diversity of potential targets for an adversary and may make it more difficult for an adversary to decide which assets to attack, though some argue an adversary may be more likely to attack disaggregated satellites because they may be considered lower risk in terms of escalating hostilities. Potential effects of disaggregation in terms of resilience are listed in table 2.

Experts also cite capability—the ability to execute specified actions—as a key driver for considering disaggregation. Some experts assert disaggregation will offer enhanced space-based capability, largely due to increased opportunities to insert new technologies, while others argue that disaggregation may reduce capability overall by losing the combined capacity achieved by operating multiple payloads on a single satellite. Table 3 details the range of potential benefits and limitations of disaggregation with respect to capability.

The acquisition process—the way DOD buys, develops, and produces defense systems to provide capabilities to the warfighter—is another area that may be impacted by disaggregation. According to some experts, disaggregation may allow DOD to implement new business practices, though others are concerned about the ability of the existing process to manage smaller, more frequent acquisitions. Table 4 expands on the potential benefits and limitations of disaggregation related to DOD’s acquisition process.

Experts identified various potential effects on the industrial base—the capacity to produce and maintain U.S. military services and equipment— due to disaggregation. Some stated disaggregation may provide greater stability for the industrial base, while others said it may be more disruptive, especially for existing suppliers. Details on potential benefits and limitations for the industrial base due to disaggregation are provided in table 5.

DOD Faces Significant Longstanding Barriers to Implementation

Benefits and limitations aside, there are significant, persistent barriers to implementing a disaggregation approach. Most of these relate to leadership shortcomings over systems and assets that need to work together to deliver space-based capabilities, such as satellite control and data processing facilities, user terminals, and satellite launch vehicles and facilities. Figure 2 depicts these interdependent segments. For each of these segments, DOD is already facing considerable management and oversight challenges that disaggregation could well exacerbate.

Disaggregation could require DOD to make significant cultural and process changes in how it acquires space systems—for instance by relying on new contractors, relinquishing control to providers who host government payloads on commercial satellites, using different contracting methods, and executing smaller but more numerous and faster-paced acquisition programs. It will likely require DOD to be more flexible and agile when it comes to satellite acquisitions, especially with regard to coordinating satellite delivery with interdependent systems, such as user equipment. Yet, we have reported that DOD’s culture has generally been resistant to changes in space acquisition approaches and that fragmented responsibilities have made it very difficult to coordinate and deliver interdependent systems. At present, although some improvements in leadership have been made, there is still no single person or organization that is held accountable for balancing needs against wants, ensuring coordination among the many organizations involved with space systems acquisitions, and ensuring that resources are directed where they are most needed. These challenges pose barriers for even the best-run efforts and they likely will be exacerbated and pose significant implementation barriers if DOD decides to disaggregate its space systems.

Gaps in the Delivery of Satellites, Ground Systems, and User Terminals

One of the principal, persistent outgrowths tied to fragmented leadership is that development of satellites often outpaces that of ground systems and user terminals (such as those on airplanes, ground vehicles, and ships), leading to underutilized on-orbit satellites and delays in getting new capabilities to end users. In some cases, gaps in delivery can add up to years, meaning that a satellite is launched but not effectively used for years until ground systems become available. One reason DOD has been unable to align the delivery of space system segments is because budgeting authority for the segments is spread across the military services and there is no single authority to ensure programs are funded in a manner that would align their deliveries. There are other reasons that have contributed to gaps in the delivery of space system segments, including poor acquisition management (requirements instability, underestimation of technical complexity, and poor contractor oversight), program funding instability, and changing information security requirements. These challenges could intensify with the potentially larger numbers and novel configurations of satellites, payloads, and other components of a disaggregated approach.

Stove-Piped Satellite Control Networks

Over the past decade, DOD has increasingly deployed standalone satellite control operations networks, which are designed to operate a single satellite system, as opposed to shared systems that can operate multiple kinds of satellites. Dedicated networks can offer many benefits to programs, including possible lower risks and customization for a particular program’s needs. However, we have reported they can also be more costly and have led to a fragmented, and potentially duplicative, approach which requires more infrastructure and personnel than shared operations. Moreover, DOD has not embraced practices and technologies that could reduce satellite control costs and increase efficiency. more satellites developed by a potentially broader base of suppliers, DOD risks increasing its reliance on standalone satellite networks, especially since it has lacked a long-term plan for satellite control operations.

Satellite Launch Costs

GAO, Satellite Control: Long-Term Planning and Adoption of Commercial Practices Could Improve DOD’s Operations, GAO-13-315 (Washington, D.C.: April 18, 2013). competition is focused on the larger launch vehicles DOD primarily uses for its satellites.

DOD Has Begun to Examine Disaggregation for Its Space Systems but Has Yet to Verify Effects

DOD has initiated and completed studies and demonstrations, including AOAs that examine disaggregated concepts for certain systems. While AOAs and other studies can provide initial insights, such as rough order of magnitude costs of selected disaggregated scenarios, they are not intended to comprehensively assess the effects of disaggregation. Moreover, the lack of common measures for key factors such as resilience may limit the effectiveness of these assessments. Technology demonstrations are providing an additional avenue for gaining knowledge about disaggregation, but demonstrations to date have been limited, concentrating more on technical feasibility and less on operational feasibility.

DOD is Beginning to Develop Knowledge on Disaggregation but Lacks Key Assessment Measures

DOD has multiple ongoing and completed efforts to examine aspects of disaggregation (see table 6). These include AOAs—which are designed to identify a range of solutions that could provide needed capability and support a decision on the most cost effective solution—as well as other studies and demonstrations that assess the potential for applying disaggregation concepts. We have reported on the important role of AOAs in establishing a sound basis for initiating programs, including providing insight into the technical feasibility and costs of alternatives.AOAs carry significant weight in the decisionmaking process, in part because they involve participation and oversight by all military services. Many of the other studies and demonstration projects related to disaggregation are Air Force efforts with a focused application to Air Force missions or programs.

It is too early to determine the extent to which the AOAs will assess disaggregation for AEHF, SBIRS, and WSF because the AOAs are ongoing. While DOD plans to assess a disaggregated scenario in each of the AOAs for the mission areas of these programs, it is not yet clear how thorough the assessments will be in terms of the potential benefits and limitations experts identified. However, the intent of the AOAs is not to examine the merits of disaggregation on its own, but rather as one of the many options that may or may not provide solutions. The additional studies beyond the AOAs have been useful in providing results to inform the ongoing AOAs, officials told us, though some have been regarded as inconclusive because they were not conducted with sufficient analytical rigor or did not consider the capabilities, risks, and trades in a holistic manner. For example, according to the Office of the Secretary of Defense’s Office of Cost Assessment and Program Evaluation, one study of satellite communication architectures contained insufficient data to support the conclusion that one architectural approach was more resilient than others. In addition, the study’s cost estimates did not consider important factors, such as ground control and user terminal costs, in calculating the implications of changing architectures. Other studies on specific applications and system components of disaggregation may provide useful information in those narrow areas, though they are not set up to cover a broader range of potential impacts of disaggregation.

Further, DOD lacks common measures for resilience that can be used consistently in AOAs and other analysis. DOD leaders have emphasized resilience as a priority when considering future systems. For example, officials told us DOD is conducting an internal review on various aspects of space resilience in the strategic portfolio, which includes disaggregation as one potential method for affecting resilience in the space segment. Officials acknowledge the need for measures and DOD is taking steps to develop them. For example, in February 2014, DOD hosted a forum convening DOD and industry experts to discuss resilience and work toward developing an agreed-upon taxonomy. The Space and Missile Systems Center, under Air Force Space Command, is also working to develop a standard assessment methodology to quantify the resilience of proposed enhancements to certain systems, including overhead persistent infrared (OPIR) and satellite communications systems. These efforts may not be completed in time to apply common measures across alternatives in the AOAs in our review; instead, resilience will be assessed on a relative basis by comparing alternatives and determining which is more or less resilient than another, Air Force officials told us. Without a consistent method to measure resilience that can be used across DOD, though, it may be difficult to determine how much resilience is needed and whether an alternative meets that need.

Demonstration Projects Are Helping to Build Knowledge but Have Not Concentrated on Operational Aspects

Demonstration projects related to disaggregation have provided technological insight and valuable lessons that DOD may use in future projects. For example, the CHIRP program demonstrated how new infrared sensor technology can be applied in OPIR missions and whether the capability of commercial spacecraft technologies could support mission needs. For future hosted payloads, DOD learned that it is better to finish payload upgrades before contracting for launch and integration services to prevent schedule delays and take advantage of cost savings. DOD also learned that conducting the necessary up-front systems engineering to better configure the payload for the host vehicle could help ensure the payload’s performance without harming the host. The Defense Advanced Research Projects Agency’s (DARPA) System F6 project also provided a lesson learned with regard to the utility of efforts not in direct support of a mission. DARPA officials said the lack of a driving mission behind System F6—which was initiated with technology development as a goal rather than support of a specific mission—limited its ability to directly inform operational systems. The project did provide useful outcomes that could be applied to future disaggregated architectures, though, such as the development of data management protocols for managing multiple satellites.

While technology demonstrations have been playing an important role in assessing the technical feasibility of disaggregation, they have not focused on operational feasibility, that is, how to introduce a disaggregated approach into a program of record and make it work from a holistic perspective. This would include assessing, for example, how a potential technical or design change would impact contracting how it would impact interconnected systems such as ground terminals and control stations and the extent to which changes would be needed to these systems; how data from disaggregated satellites would be integrated and how information assurance would be maintained with hosting operational payloads on commercial or foreign satellites.

In addition to studies and technology demonstrations, officials we interviewed emphasized the importance of testing the operational viability of disaggregation—to provide real data to assess the effectiveness of the concept—and gaining practical experience by disaggregating a mission system. Along these lines, we have previously recommended that DOD follow an incremental path toward introducing significant advancements or changes into weapon programs and ensure critical technologies are proven to work. Though disaggregation may not be introducing revolutionary technology changes, it may be introducing design, architectural, and operational changes that could have far-reaching effects.

Conclusions

Disaggregation of satellites may offer a viable option for addressing affordability and resilience challenges that DOD is facing. But it is not a simple solution. The changes to satellite designs that are being contemplated could have far-reaching effects on requirements, supporting infrastructure, management and oversight of acquisitions, industry, and other areas. They could also require upfront investments—which, under DOD’s current fiscal constraints, may not be feasible—and more leadership attention. DOD does not yet have the knowledge it needs to make changes on a wide scale nor has it addressed underlying challenges to space acquisition that could be exacerbated by disaggregation. Without a determined and disciplined effort to develop information about the full range of disaggregation issues—including operations—decisions on future space capabilities could be under- informed and opportunities missed. For example, a premature decision to pursue a large-scale disaggregated design could lead to a poor investment. Alternatively, aspiring programs could overlook the benefits of disaggregation in favor of status-quo approaches.

DOD is taking steps to gain knowledge, particularly in terms of technical feasibility, costs, and technical alternatives. The ongoing studies, including the AOAs for protected satellite communications, SBIRS, and space-based environmental monitoring, if comprehensively conducted, should help to determine some of the impacts of changing DOD’s current space system designs and find the right balance among system characteristics of affordability, capability, and resilience. However, until resilience measures are developed and agreed upon, resilience assessments—whether as part of the AOA studies or not—will be of limited value. Further, even when these studies are completed, there may still be significant uncertainties about how disaggregation could work and there will still be significant risks in implementation if key challenges, such as aligning satellite and ground systems, are not addressed. Because most of DOD’s knowledge about disaggregation resides in paper studies and a limited number of demonstration efforts, continued or expanded demonstration efforts—including those to provide operational capabilities or otherwise aimed at quantifying the benefits, limitations, and feasibility of space system disaggregation—conducted before wide-scale change is implemented could go a long way towards gaining empirical data to help DOD develop quantifiable estimates and verify the wide range of potential benefits and limitations disaggregation may yield.

Recommendations for Executive Action

Before making decisions on whether to disaggregate DOD’s protected satellite communications, SBIRS, or environmental monitoring satellite systems, we recommend that the Secretary of Defense direct the Under Secretary of Defense for Acquisition, Technology and Logistics to take the following three actions: 1. Comprehensively examine—either through the AOA studies or through other assessments—the full range of disaggregation issues, including those that go beyond the satellite systems themselves; 2. Develop common measures for resilience; and 3. Expand demonstration efforts to examine the operational feasibility of disaggregation by empirically quantifying its benefits and limitations as well as addressing longstanding barriers that could hinder its implementation.

Agency Comments and Our Evaluation

We provided a draft of this report to DOD for comment. In its written comments, DOD concurred with our first two recommendations and partially concurred with our third recommendation. The comments are reprinted in appendix II. DOD also provided technical comments which we incorporated as appropriate.

In concurring with our first recommendation, DOD agreed that the extent to which disaggregation could help reduce costs and increase the resilience of satellites is not yet known, and AOAs and other forms of assessments should comprehensively examine the range of disaggregation issues. DOD reiterated that alternatives for future satellite constellations must balance required capability, affordability, and resilience. Additionally, DOD stated that it continues to examine how different architectural alternatives affect its acquisition process, its relationship with industry partners, and the space industrial base as a whole.

In concurring with our second recommendation, DOD agreed that developing a common definition and measures for resilience is important. During the course of our review, DOD officials we interviewed stated that a consensus on a definition for resilience had not yet been achieved. However, in commenting on this recommendation, DOD stated that its Defense Space Council has approved a definition of resilience, contained in its October 2012 Space Policy. In response, we revised our recommendation to focus on the need for common measures for resilience. DOD stated it is beginning to implement resilience measures into space systems requirements and acquisition processes, and that it agrees the development of metrics to measure space system resilience is needed.

In partially concurring with our third recommendation, DOD agreed that disaggregation requires continued analysis as one method of providing increased space system resilience and emphasized that disaggregation is just one approach. DOD stated the challenges of a disaggregated ground system are well-known and demonstrated in existing architectures. However, our work found that multiple stakeholders—including DOD officials—are concerned about the unknown impacts of disaggregation on ground systems, such as whether existing systems could support it. Further assessment of the potential effects on the ground system— including how interconnected systems could be impacted and whether changes would be needed—will be necessary to ensure DOD has a complete understanding of the challenges it may face. DOD also stated it is well-versed in managing constellations with many satellites and networks with diverse users. However, disaggregation, depending on how it is carried out, could represent a significant departure from DOD’s current space system architectures, operational experience, as well as acquisition strategy and oversight. Consequently, we continue to maintain that gaining experience on the operational feasibility of disaggregation through expanded demonstrations could provide DOD with valuable knowledge for deciding a way forward for its space system acquisitions.

We are sending copies of this report to the appropriate congressional committees and the Secretary of Defense. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The Senate Armed Services Committee, in its report, 113-44, accompanying S. 1197, a bill for the National Defense Authorization Act for Fiscal Year 2014, mandated that we assess the potential benefits and drawbacks of disaggregating key military space systems—to include the use of hosted payloads—with a focus on whether disaggregation offers decreased acquisition and life cycle costs and increased survivability compared to more traditional approaches to acquiring capabilities.1, The report mandated that we examine disaggregation for three military space systems in particular: Advanced Extremely High Frequency (AEHF), Space Based Infrared System (SBIRS), and Weather Satellite Follow-on (WSF). This report (1) describes the potential benefits and limitations of disaggregating military space systems, and (2) assesses the extent to which DOD is ready to make informed decisions regarding disaggregating AEHF, SBIRS, and WSF.

S. Rep. No. 113-44, at 165 (2013). In its report the Senate Armed Services Committee mandated that we report on the results of our engagement by March 31, 2014. As agreed with Committee staff, we satisfied this reporting requirement by including preliminary results of this engagement in our written statement for a March 12, 2014 hearing on military space issues before the Subcommittee on Strategic Forces, Senate Committee on Armed Services (see GAO, Space Acquisitions: Acquisition Management Continues to Improve but Challenges Persist for Current and Future Programs, GAO-14-362T (Washington, D.C., Mar. 12, 2014)), to be followed by this report.

Information Officer; Joint Chiefs of Staff; U.S. Strategic Command; military service headquarters, including the Offices of Executive Agent for Space Staff, the Assistant Secretary of the Air Force (Acquisitions- Directorate of Space Programs), and the Office of the Chief of Naval Operations; Air Force Space Command; applicable program offices at the Air Force Space and Missile Systems Center; Naval Research Laboratory; the Defense Advanced Research Projects Agency; the Air Force Scientific Advisory Board; and the National Geospatial-Intelligence Agency. Outside of DOD, we met with government officials from the Department of Commerce, including the National Oceanic and Atmospheric Administration, and the National Aeronautics and Space Administration. We also interviewed representatives from contractors representing work in a range of mission areas (e.g., satellite communications, weather monitoring, etc.), product and service types (e.g., building satellites, integrating systems, etc.), and level of involvement in military space systems within the scope of our review. In addition, we spoke with third-party organizations such as non-profit research and industry organizations. Based on these document reviews and discussions, we compiled a list of potential benefits and limitations that experts determined to be important factors for consideration regarding disaggregation and subsequently verified the list with those experts. Each of the potential benefits and limitations identified in our report are supported by statements from at least two experts, generally representing multiple types of organizations (i.e., DOD, civilian agency, industry, and third-party). Further, we reviewed prior GAO work to determine if longstanding space system acquisition challenges pose potential barriers to implementing a disaggregated approach.

To assess the extent to which DOD is ready to make informed decisions regarding disaggregating AEHF, SBIRS, and WSF, we reviewed prior GAO work on DOD space systems acquisitions and identified criteria for early acquisition planning and development efforts to minimize cost and schedule growth. We interviewed officials from the DOD offices responsible for conducting and reviewing the ongoing AOAs in the scope of our review, as well as contributing offices, and reviewed available guidance for conducting each of the AOAs. We also reviewed available documentation about and interviewed the relevant DOD officials involved in previous or ongoing studies and demonstrations related to disaggregation. We assessed the completed and ongoing efforts—to include the AOAs, other studies, and demonstrations—against the criteria from prior GAO work and the potential benefits and limitations identified by experts to determine the extent to which these efforts provide a comprehensive assessment of disaggregation and prepare DOD to make an informed decision.

We conducted this performance audit from November 2013 to October 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, key contributors to this report were Art Gallegos, Assistant Director; Jenny Chanley; Brenna Guarneros; Kristine Hassinger; Rich Horiuchi; Carol Petersen; Jeff Sanders; Roxanna Sun; Bob Swierczek; Jay Tallon; and Oziel Trevino.