Space Research: Content and Coordination of Space Science and Technology Strategy Need to Be More Robust
Released: 2011-07-19T00:00:00

Letter

Background

In general, the S&T community includes government research laboratories and testing facilities as well as contractors and academic institutions that support these facilities. This community conducts research and development to support military or intelligence applications, such as space or weapon systems. While intelligence community funding levels for S&T are classified, DOD uses Research, Development, Test and Evaluation (RDT&E) funds for S&T work, including space S&T work, some of which is classified.

Space S&T efforts are undertaken by many government organizations. While DOD and the intelligence community comprise the vast majority of organizations involved in space S&T, several civilian government organizations are also involved, including the National Aeronautics and Space Administration (NASA), the National Oceanic and Atmospheric Administration (NOAA), and the Department of Energy (DOE). NASA’s space S&T efforts are significant and drive advances in space science, technology, and exploration and often can involve technology transfers to DOD and other agencies. NOAA, which provides space weather information to other government organizations and the public, conducts research in the development of new satellite sensors, creates new applications for using satellite data, and develops innovative approaches for handling increased data rates as well as increases in computing power and data storage. DOE develops sensors that collect space weather data, and its laboratories often collaborate on space S&T efforts that are sponsored by the Defense Advanced Research Projects Agency, the Air Force Research Laboratory, or the Naval Research Laboratories. Strategy developers told us that NASA and NOAA were not consulted in the development of the strategy because there was no statutory requirement to do so. According to strategy developers, DOE was involved in the development of the strategy even though their involvement was not required.

DOD RDT&E investment is separated into seven discrete investment categories known as budget activities. Within the DOD S&T community, the first three categories—which represent basic research, applied research, and advanced technology development activities, and are collectively known as S&T activities—use RDT&E funds. Figure 1 describes the three categories of DOD S&T investment.

Congress required DOD to develop and implement a space S&T strategy in 2004. DOD was not required to collaborate with DNI in developing the 2004 strategy. DOD’s 2004 strategy described six specific investment areas: assured access to space; responsive space capability; assured space operations; spacecraft technology; information superiority; and the S&T workforce. The most significant space S&T goals were identified within each area, with the exception of S&T workforce. The strategy stated it would be necessary to successfully develop and demonstrate the requisite technologies in a relevant environment within the short-term, defined to be within the next 5 years, and the long-term, defined to be in the year 2020 or beyond. The 2004 strategy also discussed implementation mechanisms for the strategy, the importance of transitioning new technology to fielded capability, and an approach to assess progress toward achieving the goals. Though there are many diverse organizations carrying out S&T efforts related to space and a considerable amount being invested, DOD did not update its space S&T strategy between 2004 and 2011.

Our review of DOD’s 2004 space S&T strategy found that it provided a foundation for coordination among space S&T efforts but lacked detail in key areas needed to achieve the strategy’s goals. We found that DOD had taken an initial positive step in optimizing investments in space S&T projects by establishing short- and long-term goals. However, we also identified significant challenges or barriers for DOD in implementing the strategy such as inadequate funding visibility, decreased testing resources, workforce deficiencies, and long-standing incentives that encourage technology development to take place within acquisition programs rather than the S&T community. We recommended that the strategy contain stronger linkages to DOD’s requirements-setting process, identify additional measures for assessing progress in achieving strategic goals, address barriers to achievement, and include all efforts related to space S&T. In addition, we recommended establishing protocols and mechanisms for enhancing coordination and knowledge sharing among the DOD S&T community, acquisition programs involved in space, and DOD intelligence agencies. DOD agreed with our recommendations.

Space S&T Strategy Addresses Most Statutory Requirements, but Lacks Robust Detail

Our comparison of the strategy against the statutory reporting requirements found that the strategy addresses eight statutory requirements and DOD has plans to address two other statutory requirements. For two of the requirements involving strategy implementation, DOD has asserted that its normal budgetary process will suffice. While the requirements were met, additional information that could enable DOD to successfully implement the strategy was not included. See table 1 for our assessment of the strategy.

The contents of the strategy address four primary areas: (1) short- and long-term goals; (2) goal implementation; (3) goal assessment; and (4) the transition of space S&T technology to space acquisition programs. The strategy outlines a total of 54 space S&T goals—30 short-term and 24 long-term, and describes existing, routine DOD processes such as program planning and budgeting procedures to help implement the strategy. It also mentions that periodic structured and informal programmatic and technical reviews are used to assess progress toward achievement of DOD goals and objectives and describes how there is no distinct process for transitioning space S&T products toward ultimate application in acquisition programs, while also describing several examples considered “success stories” in the transition of space S&T. While DOD addresses, or plans to address, the statutory requirements, in some instances, the inclusion of more detailed information could have allowed an opportunity for more successful implementation of the strategy. Specifically, in relation to the strategy’s goals, it does not establish a newly developed implementation plan for the achievement of the goals. Instead, the strategy describes a plan for implementation where DOD components essentially implement the strategy as a routine element of their existing program planning and budgeting procedures while employing processes that are specifically tailored to each component’s mission function. Assistant Secretary of Defense, Research and Engineering (ASD (R&E)) officials, who served as leads in developing the strategy (strategy developers) also cited a program called Reliance 21 as helping to manage and implement its entire S&T portfolio, including space S&T. While creation of a newly developed implementation plan was not a specific statutory requirement, it potentially could have provided a more delineated, exacting process for successfully achieving the strategy’s goals.

Also, DOD strategy developers told us that, as part of implementing the entire strategy, they did not specifically direct DOD components and research laboratories to meet the statutory requirements to (1) identify their research projects in support of the strategy that contribute directly and uniquely to the development of space technology, or (2) inform top DOD officials of their planned budget and planned schedule for executing those projects. However, officials explained that research components and research laboratories would do these activities as part of the normal DOD budgetary process. DOD strategy developers also told us that their entire S&T portfolio, including space S&T, is managed using the existing Reliance 21 process, including implementation planning, execution, coordination, and review. They further explained that the Reliance 21 S&T Strategic Overview process fosters awareness and joint planning among senior S&T leadership and enhances coordination among DOD’s S&T investment managers. In addition, the ASD(R&E), in conjunction with the S&T Executive Committee, conducts an annual review of all DOD S&T investments, and Space-related Advanced Technology Demonstrations are reviewed yearly at the Air Force Applied Technology Council held as part of the Air Force Space Command Space S&T Council. Strategy developers further added that cooperation, collaboration, and partnerships among the S&T organizations are also achieved through a variety of mechanisms such as technology forums, workshops, conferences, project and program reviews, international agreements, partnerships, and on-site liaisons.

Another example where key details were not present in the strategy is the establishment of 54 total goals without any prioritization. Though prioritization is not a statutory requirement, given the breadth and scope of space S&T development activities and issues we have identified in the past with respect to a lack of coordination and prioritization, it is important, and indeed a best practice, that goals be prioritized. We have also identified several, additional strategic planning best practices in the next section of this report that were not followed in the development of the strategy. If the strategy had more closely followed these best practices, it potentially could have allowed for the development of a more robust and useful space S&T strategy that addresses some of the major challenges in space S&T.

Fundamental Challenges Facing the Space S&T Community and Best Practices Not Addressed

While the content of the space S&T strategy meets statutory requirements, it does not address fundamental challenges facing the space S&T community. These challenges include human capital shortages, growing fiscal pressures, and the difficulty in transitioning space S&T to acquisition programs. While adopting best practices in strategic planning for future versions of the strategy is not required under the current statutory requirements, we identified some best practice elements that could be used to develop well-structured goals in any strategic plan.

Recent DOD-sponsored and congressional studies have addressed concerns over the DOD S&T laboratory personnel and the loss of talent in the space workforce. In addition, our prior reports have identified challenges facing the space S&T community. We have also consistently reported on today’s challenging budgetary environment and that many of DOD’s problems with poor cost and schedule outcomes on acquisition programs can be attributed to deficiencies (1) in strategic planning for critical technologies such as S&T investments, and (2) in processes for technology development and transition to acquisition programs. In fact, in S&T areas across DOD, we have found challenges in transitioning new technology from the laboratory to acquisition programs. The new space S&T strategy acknowledges there is no single process for transitioning space S&T products, and that transition is tailored to the nature of the technology being developed and the ultimate application. Strategy developers told us they recognize that having many different processes for technology transition to acquisition programs is a fact of life and they do not endorse a one-size-fits-all approach. However, as we have previously reported, DOD does not use a process with criteria that would allow lab and program managers to know when a technology is ready to transition.

In our assessment, we identified some strategic planning best practice elements that, while not required for the strategy, should be part of well- structured goals in strategic plans. Most of these strategic planning best practices are contained in the Government Performance and Results Act of 1993, designed to provide a basis for the establishment of government strategic planning and performance management, as well as the Office of Management and Budget (OMB) guidance designed to provide specific information to government agencies on the preparation and submission of strategic plans. We have also discussed DOD strategic planning best practices in our prior reports. If incorporated, these best practice elements could improve the usefulness of future strategy versions and position DOD to better address fundamental challenges in space S&T. In the absence of these more detailed best practice elements, the usefulness of the strategy for decision making may be limited. Adopting these best practice elements in future versions of the strategy is not required under the current statutory requirements, but we believe that incorporating them will ultimately improve the foundation the strategy provides for space S&T.

Identify Required Human Capital: Both the U.S. government and industry face substantial shortages of scientists and engineers and difficulty in recruiting new personnel because the space industry is one of many sectors competing for the limited number of these professionals. A recent U.S. House of Representatives study concluded that the space workforce is facing significant loss of talent and expertise and the challenge exists to smoothly transition to a new space workforce. Also, a recent study done for DOD on S&T observed that while the DOD S&T laboratory infrastructure was once world leading, the flow of research science, technology development, and engineering expertise is not as robust as it once was and, for the most part, it has declined to the point where most DOD S&T people are project managers who monitor research being done by others outside of DOD. While the discussion of human capital needs is considered a strategic planning best practice, the strategy does not include a discussion of the human capital currently engaged in space S&T or the human capital required to achieve the strategy’s goals.

Identify Required Funding: Our prior work has shown that, in general, a lack of investment information can adversely affect the ability to avoid unnecessary duplication, control costs, ensure basic accountability, anticipate future costs, and measure performance. Funding required to implement and achieve the goals was not compiled or included in the strategy. Strategy developers told us that since they were not required to report funding information, they did not collect funding information from the various DOD research laboratories and components involved in space S&T and they did not impose funding constraints in developing the strategy. In addition, DNI officials told us there was no attempt to incorporate information on space S&T funding amounts associated with the intelligence community in the strategy. Furthermore, strategy developers told us that achieving top-level visibility for DOD space S&T funding is not a simple task and would require substantial effort. However, when asked about the biggest challenge to achieving the goals, strategy developers told us it was maintaining consistent funding in a funding- constrained environment. In addition to funding information, neither an analysis of past trends nor future funding needs were included in the strategy, both of which could have established a recommended level of consistent funding. These efforts could potentially help mitigate risk to space S&T against the backdrop of growing national government fiscal imbalance and budget deficits that are straining all federal agencies’ resources.  Prioritize Initiatives: The federal government faces real fiscal limitations and will have to make difficult choices about upcoming priorities, but the strategy does not go beyond what was required and prioritize the goals in a more definitive way than classifying them as either short- or long-term. We have reported that prioritizing initiatives enables evaluation in terms of overall importance to the portfolio and can help decision makers when allocating resources. Strategy developers told us criteria were not developed to either accept or reject goals for inclusion in the strategy or to prioritize a goal as either short- or long-term. Instead, they told us that meetings were convened to discuss and review draft goals and that the categorization of goals as either short- or long-term was based upon the professional judgment of the meeting participants with respect to the current state of technology and potential to achieve capabilities within nominal time frames. The strategy currently lists 30 short-term and 24 long-term goals—22 more goals than were identified in the 2004 strategy—and gives no indication which ones are more important than the others. Strategy developers also added that more specific time frames or goal prioritization were not included in the strategy because that would require the assignment of resource (i.e., funding) commitments, which was not required. Since strategy goals were not prioritized, it will be difficult to determine which space S&T goals are the most important to the space S&T community if trade-off decisions are necessary.  Establish Ways to Measure Progress: Performance measures can be used to assess the value of projects relative to goals, demonstrate results and provide useful information for decision makers. The strategy did not develop new metrics or performance measures that could be used to assess whether the strategy’s goals are being achieved. Rather, the strategy indicates that various periodic structured, and informal, programmatic and technical reviews are used to evaluate the effectiveness and quality of space S&T investments and assess progress toward achievement of objectives and goals. It is unclear how these reviews can help assess the larger strategic goals developed in the strategy that are meant to apply to DOD and the intelligence community. It is also unclear how the Reliance 21 program helps to specifically assess the progress of, and provide linkage to, the goals established in the strategy. Further, Reliance 21 reviews do not involve an examination of space S&T within the intelligence community. Without established ways to measure progress toward strategy goals, it will be more difficult to measure the progress and achievement of space S&T goals and implement corrective actions if needed.  Establish Process for Revising Goals in the Future: Criteria to establish and revise goals could help to improve performance and results. The strategy, however, does not articulate the process to be used to revise the goals in the future for the next version of the strategy. Strategy developers told us there were no criteria established to initially help formulate goals and that the general consensus of the officials involved determined which goals were included in the strategy. While this same process could be used to make revisions to the goals, the absence of criteria will likely make future revisions more difficult to track and understand, and certainly more ad hoc.

Coordination Efforts in Strategy Development Were Limited and Implementation Efforts Are too Early to Assess

Although most organizations involved with the space S&T strategy participated significantly in developing its short- and long-term goals, participation among some organizations in developing other aspects of the strategy was more limited. DOD and DNI officials told GAO that their interpretation of the 2009 statute directing development of the strategy was that it did not require that the intelligence community be involved to the full extent in some aspects of the strategy. Moreover, although their involvement was not required by the statute, other agencies with investments in space S&T, such as NASA and NOAA, were not involved in the strategy’s development. Together, the intelligence community and these other agencies conduct a significant amount of space S&T development. By limiting their involvement, DOD may have missed an opportunity to leverage these activities and optimize its own S&T spending. Also, since the strategy has only recently been issued, it is too early to evaluate the effectiveness of the coordination mechanisms planned to implement the strategy.

Strategy Development Coordination within DOD Was Limited

According to an ASD (R&E) official, his office served as the lead in development of the strategy and explained that the methodology began with identifying DOD, DNI, and other space S&T stakeholders. As required, strategy developers consulted with the directors of DOD research laboratories and other DOD research components, as well as the heads of other DOD organizations that have interests in space S&T. Officials from most of the DOD laboratories and components said that they had participated in establishing the strategy goals. Seven of eight organizations we interviewed reported they were tasked to compile and submit their goals related to space S&T. Goals included in the final strategy were based on consensus agreement. Strategy developers organized the strategy goals under the space functional areas used in the National Security Space Plan. Goals were categorized as either short- or long-term, based on the professional judgment of the strategy developers with respect to the current state of technology and the potential to achieve the desired capabilities within nominal time frames.

While it is clear that DOD research laboratories and components were consulted, and most assisted in the development of the strategy, it is also clear from our discussions with these organizations that their involvement was typically limited to contributing to the establishment of short- and long-term space S&T goals. Although workgroups were established to develop each of the strategy’s sections, responses varied concerning the level of involvement. Some organizations’ participation in the development of the strategy’s sections that discuss the implementation, assessment, and transition of space S&T was limited. Some research laboratories and components reported playing larger roles in the workgroups than others.

DOD strategy developers told us that, in developing the strategy, they did not direct DOD research laboratories or components to identify and provide a compilation of their space S&T projects being worked on or planned. On the basis of the statute, they were not specifically required to do so. They also were not required to, nor did they ask these research laboratories and components to, provide planned budget information associated with their space S&T projects. Officials explained that they did not believe this was necessary or would provide value to the development of the strategy and said they have processes in place, such as the Reliance 21 program, to help facilitate the coordination of space S&T projects within DOD. Reliance 21, however, is a DOD program and does not involve reviews of space S&T projects within the intelligence community.

Strategy Development Coordination between DOD and the Intelligence Community Was Limited

DNI, while not required to in the development of the strategy, did not provide a compilation of space S&T projects, nor the associated planned budget information for assessment as the strategy was being developed. While the statutory provision for the strategy was amended in 2009 to require DNI to jointly develop the strategy with DOD, other parts of the statute do not specifically call for DNI’s involvement. DOD and DNI officials told us they interpreted this to mean the intelligence community was not required to be involved to the full extent in some aspects of the strategy. Specifically, DNI officials told us that they interpret the statute to apply to the establishment of goals for DOD in the strategy, and that there is no requirement that these goals take into consideration the goals of the intelligence community. According to DNI officials, only where the DOD goals coincide with the already established goals of the intelligence community do they plan to work jointly toward goal accomplishment. Further, DNI officials told us that while the strategy addresses plans for implementing the goals established, these plans only apply to DOD and the strategy does not include implementation plans for the intelligence community. While coordination between DOD and DNI was limited in development of the strategy, strategy developers told us that apart from the development, they do coordinate regularly with each other on space S&T projects and also use on-site liaison personnel at the Air Force Research Laboratory and the National Reconnaissance Office. Strategy developers also provided some limited information on meetings and councils in which agencies within the two organizations participate.

Although DOD did coordinate with DNI and the intelligence community, we believe greater coordination among the stakeholders would allow for the formation of a single strategic plan to guide this important area. In the past, we have raised concerns about DOD strategic planning and the degree to which DOD and DNI collaborate on space strategic planning. In 2008, we reported to Congress that we were concerned there was no overarching strategic guidance in place to link the defense and intelligence communities’ future space programs, plans, and new space concepts. The National Reconnaissance Office (NRO), which served as the lead agent for DNI and the primary participant from the intelligence community in the development of the strategy, typically develops sophisticated space capabilities and is critically important to space intelligence. While the amount of money NRO spends on space S&T is classified, it is the premier space reconnaissance organization in the world and has established a priority to improve this area of investment.

Further, strategy developers stated that they did not coordinate with NASA or NOAA in developing the strategy because the statute did not direct that these agencies be involved. NASA and NOAA are both involved in significant space S&T efforts with NASA’s fiscal year 2012 budget request including over $1 billion for space research and technology. NASA has a strategic plan of its own with a primary mission to drive advances in space science, technology, and exploration and can be involved in technology transfers to DOD and other agencies. While DOD and DNI did not coordinate with NASA in the development of the strategy, NASA and DOD periodically coordinate on S&T projects as part of the National Science and Technology Council. NASA’s procedural requirements also recommend they search research and technology literature prior to investing in new research areas to minimize duplication of effort and look for opportunities to augment research and technology efforts from other agencies. NOAA conducts research in the development of new satellite sensors, new applications of satellite data, new approaches for handling increased data rates, as well as increased computing power and data storage.

Conclusions

If done well, strategic planning provides the foundation for the most important things organizations do each day and fosters informed communication between organizations and their stakeholders. Strategic planning provides decision makers with a framework to guide program efforts and the means to determine if these efforts are achieving the desired results. While the strategy was a first step, it unfortunately was not a rigorous, comprehensive strategic plan. Instead, it embraces the status quo without laying out a path for assuring effective and efficient progress. The space S&T strategy could have gone beyond statutory requirements and provided the basis for a rigorous, comprehensive space S&T program, consistent with economic trends and budgetary constraints to ensure the United States continues to possess the advantages that space provides DOD and the intelligence community. Improving coordination and incorporating changes in future versions would help ensure the strategy addresses space S&T challenges, and help supports agency investments. Addressing these kinds of factors would enable DOD and the intelligence community to have a more effective strategy to guide this critical area of investment.

Recommendations for Executive Action

To optimize government investment in space S&T and address key challenges, we recommend that the Secretary of Defense (who would direct the Assistant Secretary of Defense, Research and Engineering and the DOD Executive Agent for Space) and the Director of National Intelligence make the following three improvements to enhance the next version of the space S&T strategy:  Develop a specific implementation plan that provides a detailed process for achieving the strategy’s goals. Include information on required human capital; required funding; prioritization; ways to measure progress against the goals; and process(es) for revising goals to address the challenges in space S&T.  Enhance coordination between the DOD space S&T community, the intelligence space S&T community, and NASA and NOAA in the development of the strategy so that the space S&T area can be examined strategically.

Agency Comments

In written comments on a draft of this report, DOD concurred with all three of our recommendations to enhance the next version of the space S&T strategy. DNI did not offer any comments on the draft report provided for their review.

DOD's written comments are reprinted in appendix II.

We are sending copies of this report to the Secretary of Defense, the Assistant Secretary of Defense, Research and Engineering, the DOD Executive Agent for Space, the Director of National Intelligence, and the Director of the Office of Management and Budget. The report also is available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Staff members making key contributions to this report are listed in appendix III.

Appendix I: Scope and Methodology

To determine the extent the 2011 space science and technology (S&T) strategy addressed statutory requirements, we compared the strategy contents to the congressional requirements in 10 U.S.C. Section 2272 and asked the Department of Defense (DOD) and Director of National Intelligence (DNI) officials for additional details when necessary. We also met with DOD and DNI officials to discuss and collect information on the methods, approaches, and analyses used to support the content of the strategy.

We also identified existing challenges in space S&T documented in prior government and GAO reports and compared these challenges to the areas covered by the strategy’s contents. Further, we identified best practices in strategic planning and compared the strategy’s contents to these best practices to determine if there were ways to improve it. We obtained the strategic planning best practices from the Government Performance and Results Act of 1993, Office of Management and Budget Circular No. A-11, Part 6, Preparation and Submission of Strategic Plans, Annual Performance Plans, and Annual Program Performance Reports, July 2010, and prior GAO reports.

To determine the extent of coordination efforts used to develop and implement the strategy, we discussed the coordination methods used in the strategy’s development with officials from the Office of Assistant Secretary of Defense, Research and Engineering (formerly the Director, Defense Research and Engineering); Office of the Director of National Intelligence (including National Reconnaissance Office officials); Defense Advanced Research Projects Agency; Missile Defense Agency; Army Materiel Command; Office of the Assistant Secretary of the Army for Acquisition, Logistics, and Technology; Army Space and Missile and Defense Command; Naval Research Laboratory; Office of Naval Research; Office of the Assistant Secretary of the Air Force for Acquisition; Air Force Research Laboratory; and the Department of Energy. Since the strategy has only recently been issued, and given the time constraints of our review, it was too early to assess the mechanisms and processes outlined in the strategy for its implementation.

We also reviewed other relevant high-level space strategic plans including the National Security Strategy, the National Security Space Strategy, the Quadrennial Defense Review, the National Space Policy, the Defense Science and Technology Strategy, the DOD Research and Engineering Strategic Plan, and the DOD Space Science and Technology Strategy (2004).

We conducted this performance audit from April 2011 to July 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Art Gallegos, Assistant Director; Tim Persons; Marie Ahearn; Don Springman; LeAnna Parkey; and Laura Greifner made key contributions to this report.