Technology Development: New DOD Space Science and Technology Strategy Provides Basis for Optimizing Investments, but Future Versions Need to Be More Robust
Published: 2005-01-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense (DOD) is depending heavily on new space-based technologies to support and transform future military operations. Yet there are concerns that efforts to develop technologies for space systems are not tied to strategic goals for space and are not well planned or coordinated. In the National Defense Authorization Act for 2004, the Congress required DOD to develop a space science and technology (S&T) strategy that sets out goals and a process for achieving those goals. The Congress also required GAO to assess this strategy as well as the required coordination process.
section_title: What GAO Found
paragraphs: DOD's new strategy for space S&T met four of the nine requirements set out by the Congress and plans are in place to meet the remaining requirements. These included requirements for setting short- and long-term goals and a process for achieving those goals as well as requirements that focused on ensuring the strategy was developed with laboratories, research components, and other organizations involved in space S&T and ensuring the strategy would be reviewed by appropriate entities and revised periodically. In addition to meeting these requirements, GAO found that development of the strategy itself helped spur collaboration within the DOD space S&T community since it required diverse organizations to come together, share knowledge, and establish agreement on basic goals. Since the strategy has only recently been issued, it is too early to assess whether the direction and processes outlined in the strategy will be effective in supporting and guiding future space S&T efforts. Moreover, DOD officials are still working out the details of some implementation mechanisms. However, in order to better position DOD for successful implementation, GAO believes that the plan should contain stronger linkages to DOD's requirements setting process, identify additional measures for assessing progress in achieving strategic goals, and explicitly cover all efforts related to space S&T. Moreover, there are formidable barriers that stand in the way of optimizing DOD's investment in space S&T. DOD does not have complete visibility over all spending related to space S&T, including spending occurring within some S&T organizations and acquisition programs. Without a means to see where funding is being targeted, DOD may not be able to assure all spending on technology development is focused on achieving its goals. The S&T community itself may not have resources critical to achieving DOD's goals. In recent years, funding and opportunities for testing for the space S&T community have decreased. And, concerns have grown about the adequacy of the space S&T workforce. DOD acquisition programs continue to undertake technology development that should be occurring within an S&T environment, which is more forgiving and less costly than a delivery-oriented acquisition program environment. Until this is done, cost increases resulting from technology problems within acquisitions may keep resources away from the S&T community. By using the strategy as a tool for assessing and addressing these challenges, DOD can better position itself for achieving its goals and also strengthen the S&T base supporting space.

section_title: Letter
paragraphs: The Department of Defense (DOD) is looking to its space systems to play  an increasingly pivotal role in future military operations. As such, it is  developing several families of new, expensive, and technically challenging  space systems, eventually including constellations of satellites that will  employ laser optics to transport information over long distances in much  larger quantities than radio waves; a new generation of global positioning  technology; and advanced infrared sensors, radar sensors, and  environmental monitoring sensors. At the same time, DOD is seeking to  improve technologies and materials that are critical to enhancing satellite  performance, such as propulsion systems, cooling systems, onboard and  ground processing systems, and materials used to protect technologies  and spacecraft in the harsh space environment.
A broad array of entities is charged with responsibility for developing the  science and technology (S&T) supporting space systems, including  research laboratories and test facilities belonging to the military  departments and DOD as well as industry and academic organizations that  perform research and development for these organizations or for specific  DOD space acquisition programs. From fiscal years 2004 through 2009,  DOD plans to spend about $3.8 billion on S&T efforts exclusive to space  applications within its military laboratories and contracts through its  laboratories. Considerably more money will be spent on projects that have  space and terrestrial applications (for example, propulsion technologies  and advanced materials) as well as on technology development that occur  within acquisition programs. In addition, outside agencies, such as the  National Aeronautics and Space Administration (NASA), also invest in  S&T efforts that can support DOD space systems and may benefit from  S&T efforts being carried out by DOD.
Though there are many diverse organizations carrying out S&T efforts  related to space and a considerable amount being invested, DOD has not  had an overarching strategy for space S&T in recent years that sets goals  for these efforts, helps direct how investments should be spent, and tracks  the overall progress of DOD’s investment in space S&T. Moreover, there  have been concerns that the level of collaboration and coordination  among all DOD S&T organizations involved in space has not been  adequate, leading to redundant or unnecessary investments in some areas  or even too little investment in areas where it is critical for the United  States to maintain a lead over other nations. There has also been concern  that technologies have difficulty transitioning from the laboratories to  DOD’s acquisition programs. In addition, our previous reports have shown  that weapon system acquisition programs have taken on technology  development that should occur in an S&T environment. In doing so,  acquisition programs have not been able to align customer expectations  with resources, and therefore minimize problems that could hurt the  program in its design and production phases. In fact, many of the space  programs we have reviewed over the past several decades have incurred  unanticipated cost and schedule increases because they began without  knowing whether technologies could work as intended and invariably  found themselves addressing technical problems in a more costly  environment.
The National Defense Authorization Act for Fiscal Year 2004 (the act)  required DOD’s Executive Agent for Space and its Director of Defense  Research and Engineering (DDR&E) to develop and implement a space  S&T strategy. The act required us to review and assess the S&T strategy  and the effectiveness of the coordination process among DOD S&T  elements and to report our findings by September 1, 2004. As discussed  with committee staff, our objectives were to (1) assess whether the  strategy meets the act’s requirements, (2) identify additional criteria above  and beyond the act that could enhance the usefulness of the strategy, and  (3) identify barriers that may hamper DOD’s ability to successfully  enhance S&T efforts for space. We provided a briefing on our review to  your committees on our findings on September 1, 2004. This report details  our findings.
In conducting our work, we reviewed DOD and military department policy  documents on S&T activities, as well as pertinent S&T reports and related  material, to determine DOD’s progress in achieving program mission  objectives. We assessed the DOD space S&T strategy for compliance with  the 2004 Defense Authorization Act. We developed additional criteria with  which to assess the space S&T strategy and identified barriers that may  influence DOD’s ability to successfully implement S&T efforts for space. In  doing so, we consulted with subject matter experts and reviewed our  previous best practice reports. We reviewed documents from and  conducted interviews with officials in DOD, the Army, the Navy, the Air  Force, the Defense Advanced Research Projects Agency (DARPA), the  Missile Defense Agency (MDA), NASA, and military department research  laboratories. We also analyzed the fiscal year 2004 virtual Major Force  Program for Space and unclassified DOD budget documents to identify the  amount of space research, development, test and evaluation (RDT&E)  funding for fiscal years 2003 to 2009 and confirmed with DOD officials  responsible for maintaining this information that our analysis was correct.  Our review was conducted from November 2003 to November 2004 in  accordance with generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: Generally, DOD’s S&T community (which includes DOD laboratories and  testing facilities as well as contractors and academic institutions that  support these facilities) conducts research and develops technologies to  support military applications, such as satellites or weapon systems. Like  the acquisition community in DOD, the S&T community uses RDT&E  funds, but the S&T community’s work precedes the acquisition cycle.  Weapon system program managers, who receive most of DOD’s RDT&E  budget, apply generic technologies to specific systems. Figure 1 highlights  activities the S&T community is involved in along with the RDT&E budget  categories, or “activities,” which are used to fund these efforts. More  details on both are provided in appendixes I and II.
The S&T community carries out its work within the first three categories  of research and development listed above. DOD has specified that the  work within the fourth category—testing and evaluation of prototypes of  systems or subsystems in a high fidelity or realistic environment—involves  efforts before an acquisition program starts product development.  However, according to DOD officials, it is assumed that either the S&T  community or an acquisition program may carry out this work, and  traditionally, weapon system acquisition programs have taken on  technology development within this stage. After this point, any additional  development is to be completed as part of a formal acquisition or product  development phase under the authority of the weapon system manager  and apart from the S&T community.
The DOD DDR&E is responsible for the overall direction, quality, and  content of the agency’s S&T efforts. Each of the military departments— Army, Air Force, and Navy—has its own S&T programs, as do DOD  organizations such as DARPA, Defense Threat Reduction Agency, MDA,  and the National Reconnaissance Office (NRO). The DOD Executive Agent  for Space—who is also the space milestone decision authority for all space  major defense acquisition programs, the Under Secretary of the Air Force,  and the Director of the NRO—also influences S&T efforts for space since  he decides whether significant investments in space systems are to move  forward in the development process.
There are mechanisms within the space community and DOD designed to  ensure S&T efforts are coordinated and are focused on achieving broader  goals and that redundancy is minimized. Within the space community, a  forum called the Space Technology Alliance was established in 1997 to  coordinate the development of space technologies with an eye toward  achieving the greatest return on investment. Its membership includes the  Air Force, the Army, the Navy, MDA, DARPA, and NRO. At the DOD-wide  level, there is a Defense Science and Technology Strategy, which lays out  goals for DOD-wide S&T efforts based on goals set by higher-level  documents, such as the Quadrennial Defense Review. This strategy is  used, in turn, to develop a DOD-wide basic research plan, which reflects  DOD’s objectives and planned investments for basic research conducted  by universities, industry, and laboratories and a DOD-wide technology  area plan, which does the same for applied research and advanced  technology development. There is also a Joint Warfighting S&T Plan,  which ties S&T projects to priority future joint warfighting capabilities  identified by higher-level documents. These overall plans, in turn, are used  by DOD laboratories to direct investments in S&T. They are also used by  the Office of the Secretary of Defense to provide guidance to the military  departments and the defense agencies as they develop and vet their  proposed budgets. In addition, DOD puts together teams of outside  experts in 12 technology areas to assess whether particular investments  across DOD’s S&T community are redundant or unnecessary. These are  known as Technology Area Reviews and Assessments. The teams make  recommendations to a board comprised of senior DOD S&T officials and  chaired by the DDR&E for action to terminate, adjust, and/or enhance  investments to better align the S&T program to comply with the planning  document guidance. The DDR&E, which reports to the Under Secretary of  Defense (Acquisition, Technology and Logistics), has oversight of the  RDT&E budget activities used to research and develop new technologies,  specifically, RDT&E budget activities 1 (basic research), 2 (applied  research), and 3 (advanced technology development). Recently, the  DDR&E was given oversight of RDT&E budget activity 4 (advanced  component development and prototypes) in an effort to ensure this  development had sufficient oversight from the S&T community.
section_title: DOD’s Space S&T Strategy Addresses the Act’s Requirements
paragraphs: The act required DOD to develop a strategy for its space S&T efforts that  identified short- and long-term space S&T goals; a process for achieving  the goals, including an implementation plan; and a process for assessing  progress made toward achieving the goals. The act also required DOD to  coordinate its strategy development efforts. The strategy, yet to be  delivered to the Congress at the time of our review, met four of nine  requirements, and plans are in place to meet the remaining five. We found  that the strategy provides a foundation for enhancing coordination among  space S&T efforts since it does specify overall goals and that it establishes  several mechanisms to help senior leaders gauge whether investments are  focusing on those goals. However, since the strategy has only recently  been issued, it is too early to assess whether the direction and processes  outlined in the strategy will be effective in supporting and guiding future  space S&T efforts.
The strategy identified goals for space S&T along six main areas—assured  access to space, responsive space capability, assured space operations,  spacecraft technology, information superiority, and S&T workforce.  Except for the goal of enhancing the workforce, the strategy laid out short- term goals (within 5 years) and long-term goals (in the year 2020 or  beyond). Under spacecraft technology, for example, the strategy identified  a short-term goal of on-orbit assessment of satellite servicing and repair  and long-term goals of on-orbit assembly, deployment, repair, and  upgrades. Under assured space operations, the strategy identified a short-  term goal of detecting, identifying, and characterizing natural and man- made objects in space and a long-term goal of complete space situational  awareness. According to S&T community officials we spoke with, the  mere identification of goals should be useful in helping DOD laboratories  and other S&T facilities to direct their investment as this type of guidance  had not been provided for space previously.
The strategy also establishes several mechanisms for implementation.  Primarily, it calls for semiannual space S&T summit meetings to  coordinate user expectations, highlight technologies, provide guidance,  and establish priorities. DDR&E officials, agency S&T executives as well  as Service Program Executive Officers for Space who will ultimately  transition new capabilities, and major command leadership will attend  these meetings. The strategy also implements an Industry Independent  Research and Development coordination conference, where industry and  government officials can come together to collaborate in their S&T  planning activities. Details on both of these mechanisms are still being  worked out, according to the developers of the strategy.
The strategy also identifies some tools and measures that will be used to  track progress in meeting goals. These tools and measures include  “technology roadmaps,” which identify timelines, milestones, and  transition dates for specific projects as well as interdependencies with  other projects and “technology readiness level” (TRL), an analytical tool  that assesses the maturity level of technology. Our prior work has found  TRLs to be a valuable decision-making tool since it can presage the likely  consequences of incorporating a technology at a given level of maturity  into a product development.  Appendix III details criteria for each TRL.
In addition, DOD has plans in place to ensure that the strategy is reviewed  and revised, as necessary, annually and that it be made publicly available  for review by congressional defense committees. Other DOD S&T entities  will be provided the strategy to support the planning, programming, and  budgeting processes. DOD also plans to include the strategy as an annex  to the National Security Space Plan, even though the plan is thought to be  a lower-level tactical document and not a strategic document.
The developers of the strategy worked with a wide range of organizations  in establishing goals, measures, and implementation plans. These include  military department laboratories, DARPA, intelligence agencies, MDA, the  Air Force Space Command, NASA, the Space and Missile Systems Center,  the U.S. Strategic Command, the National Security Space Office, and  others. Officials within the space community we spoke with commented  that it has historically been difficult to gain agreement from these  organizations. Even though they all have ties to space, these organizations  have different views as to what overall goals the space community should  strive for and how they should be achieved. According to officials within  the space community we spoke with, just getting these organizations to  work together and to gain agreement was a significant benefit to the  community at large since it helped foster more collaborative working  relationships and greater knowledge sharing.
section_title: Additional Criteria Are Not Included in the Act That May Enhance the Strategy
paragraphs: In addition to the requirements specified by the act, we found that  optimizing space S&T efforts also depends on whether (1) the strategy is  clearly linked to other strategies and plans; (2) all DOD space S&T efforts  are covered by the strategy; and (3) the strategy identifies metrics beyond  TRLs that focus on success. Linkage to other strategies and plans is  important to providing clear guidance to S&T laboratories and other  organizations making investments since there are a number of DOD-wide  “strategies” for S&T as well as a number of space-related higher level  strategic plans as well as tactical plans relating to S&T. Coverage of all  space S&T efforts is important since S&T is carried out not only by DOD  laboratories but also by large acquisition programs and other agencies that  have a large stake or investment in space S&T. For example, NRO  develops new satellites for the intelligence community and could  potentially leverage its S&T efforts with DOD’s. Lastly, having additional  measures beyond TRLs is important to gauging the success of the  implementation of the strategy as well as the relevancy and feasibility of  specific progress toward achieving DOD’s overall goals for space. We  found that the strategy clearly identified linkages to some, but not all, key  plans and strategies, and it did not provide coverage over all S&T efforts  or establish additional measures.
subsections: section_title: Links to Other Strategies and Plans
paragraphs: The space S&T strategy identifies links to higher-level documents, such as  the National Security Space Strategy, which sets overall strategic goals for  DOD space and identifies capabilities to be pursued, and the Defense S&T  Strategy, which provides overall goals for DOD S&T based on higher-level  strategic documents. The strategy also references lower-level plans  including the National Security Space Plan discussed earlier and DOD- wide S&T plans, such as the Basic Research Plan, the Defense Technology  Area Plan, and the Joint Warfighting S&T Plan. However, the strategy did  not provide links to other documents and assessments that impact the  space S&T community. For example, it is unclear how the document will  link to DOD’s Space Technology Guide, which describes the current state  of space and space-related technology activities underway, including key  enabling technologies, that is, those that “must be done right” since they  play a pivotal role in making revolutionary advancements in space  applications. The guide is being revised and could serve as a useful  implementation tool for the new space S&T strategy. It is also unclear how  the strategy links to architectures in areas such as responsive space  operations, protection for space mission assurance, and integrated  intelligence, surveillance and reconnaissance being developed by the  National Security Space Office. These architectures are to define the  future desired state for DOD’s space assets. It is important that DOD  reflect these other documents in the new space S&T strategy so that the  space community clearly understands where the strategy fits in relation to  other plans and guides and can ensure decision making is consistent.  Moreover, by establishing closer links with the Space Technology Guide  and architectures under development, DOD may have more avenues to  implement its short- and long-term goals.
In addition, the Joint Chiefs of Staff did not participate in the development  of the strategy, including offices responsible for DOD’s new Joint  Capabilities Integration and Development System (JCIDS). JCIDS is  replacing DOD’s requirements generation process for major acquisitions in  an effort to shift the focus to a more capabilities-based approach for  determining joint warfighting needs rather than a threat-based approach  focused on individual systems and platforms. Under JCIDS, boards  comprised of high-level DOD civilians and military officials are to identify  future capabilities needed around key functional concepts and areas, such  as command and control, force application, and battlespace awareness,  and to make trade-offs among air, space, land, and sea platforms in doing  so. Although the JCIDS officials were not required to participate in  developing the strategy, it is important that they do so in the future since  their work could have a significant impact on the direction of investments  for space S&T projects.
section_title: Coverage of All S&T Efforts
paragraphs: The space S&T strategy does not explicitly address technology  development efforts within DOD acquisition programs. According to DOD  officials, space acquisition programs are typically using RDT&E funds  from budget activity 4 to mature technology and build the first two  satellites. Our analysis showed that space acquisition programs plan to  spend as much as $16 billion from fiscal years 2004 through 2009 on  budget activity 4. Our annual assessments of space systems have shown  that the portion of the $16 billion that is to be spent on maturing  technology (which we could not readily separate from the portion spent  building the first two satellites) is often being used to carry out activities  that should be carried out in an S&T environment. For example, the  Transformational Satellite program, which is focused on building  advanced communication satellites, entered system development in early  2004 with only one of seven critical technologies matured to a point of  being tested in a relevant environment. Most of the technologies were at a  TRL 3, meaning analytical studies and some laboratory tests had been  conducted, but components had not yet been demonstrated to work  together. If DOD does not explicitly include acquisition programs in the  space S&T strategy, it will not be able to ensure the S&T community has  oversight over a considerable amount of ongoing technology development.
We were not provided access to NRO to discuss how it collaborated with  the DDR&E and the Executive Agent for Space in developing the space  S&T strategy and how they intended to work with the DDR&E and the  Executive Agent for Space in implementing the strategy. However, DOD  officials stated that NRO had participated in the development of the  strategy and would participate in all S&T coordination activities identified  by the space S&T strategy. Moreover, according to DOD officials, NRO and  other intelligence agencies already participate in some DOD space S&T  coordination and review efforts, such as the Space Technology Alliance. In  addition, the DDR&E and the DOD Executive Agent for Space are  continuing to work on increasing coordination between DOD and the  intelligence community. DOD officials also noted that the current  Executive Agent for Space also serves as the Director of NRO, which has  helped to increase coordination between the intelligence community and  DOD. While these efforts may be helping to increase coordination between  DOD and the intelligence S&T communities, it is still important to  specifically include the DOD intelligence agencies in the strategy itself and  to identify protocols that can help foster greater knowledge sharing  between both communities.
section_title: Success Measures
paragraphs: While the strategy identifies TRLs as a measure for tracking progress, it  does not prescribe metrics that focus on the value of S&T projects relative  to specific goals or knowledge being gained from projects. Such metrics  would help provide a foundation for assessing progress in achieving  strategic goals. Strategy developers stated that technology development  organizations are better suited to develop and use their own specific  metrics to measure success because different technologies may require  different types of metrics. The developers stated that by design, the  strategy sets the direction but leaves it up to the laboratories and other  S&T entities to establish their own metrics. However, they acknowledged  that some of the organizations they worked with did not have adequate  metrics. It is important that DOD attempt to identify and use metrics that  help assess progress, since these will enable DOD to evaluate investments  against its short- and long-term goals and make informed investment  decisions.
section_title: Barriers May Hamper Strategy Implementation
paragraphs: Though the new space S&T strategy takes important first steps toward  optimizing investments, there are significant barriers that will make it  difficult to make advancements in the way S&T efforts are planned,  managed, and transitioned into acquisition programs. Some barriers relate  specifically to the space community—principally, incomplete RDT&E  funding visibility, inadequate testing resources, and workforce  deficiencies. These can potentially be addressed through further study,  resource shifts, increased management attention, and/or changes to how  funding is captured. Other barriers are more systemic and require more  difficult management and cultural changes to be made throughout DOD.  Nevertheless, until barriers are largely removed, the impact of a new  strategy for space S&T may be limited. The developers of the strategy  agreed that the barriers we identified were important and needed to be  addressed through efforts beyond the development of the strategy.
subsections: section_title: Visibility over S&T Spending on Space Is Incomplete
paragraphs: The current budget process does not readily capture all RDT&E funding  for space S&T efforts. In 2001, DOD established a “virtual” Major Force  Program for space to increase the visibility of resources allocated for  space activities. This is a programming mechanism that aggregates most  space-unique funding by military department and function. However, the  mechanism does not align funding with RDT&E budget activities, making  it more difficult for DOD to assess the balance of funding among basic  research, applied research, and advanced technology development. In  working with DOD officials to categorize the virtual Major Force Program  by RDT&E budget activity, we identified about $3.8 billion from fiscal  years 2004 through 2009 for budget activities 2 (applied research) and 3  (advanced technology development). However, funding for budget activity  1 (basic research) cannot be specifically associated to either space or  terrestrial platforms, and therefore does not appear in the virtual Major  Force Program, which is focused on space-unique funding. Funding in  RDT&E budget activities 2 and 3 that is not space unique is also not  captured. In addition, some DOD agencies develop space assets but have  primary missions that are not associated with space and are therefore, not  included in the virtual Major Force Program. For example, MDA’s space  efforts are not included in the virtual Major Force Program for space even  though MDA is developing a new generation of missile tracking satellite  systems using advanced infrared sensors. MDA plans to spend about $4.12  billion on this system from fiscal years 2004-2009, and a considerable  portion of this funding is expected to be used to mature technologies for  future satellites. Moreover, DARPA reports its space funding by project so  space S&T efforts cannot be readily identified without additional  knowledge of whether these projects are space related. Currently, DARPA  has funded about $200 million annually on projects that are space unique  and considerably more on projects that have both space and terrestrial  applications. Until the virtual Major Force Program or some other tool can  capture and categorize the total amount of RDT&E dollars supporting  space-unique S&T projects at a minimum, DOD will be limited in guiding  and directing all space investments.
section_title: Testing Resources Declining
paragraphs: Testing resources for space technologies are on the decline. In particular,  funding for testing has decreased, costs to launch experiments have  increased, and opportunities have been reduced with the loss of the space  shuttle, which had been partially used for DOD-related technology  experiments. DOD’s Space Test Program, which is designed to help the  S&T community find opportunities to test in space relatively cost- effectively, was funded at $62.3 million in fiscal year 1990 but only   $38.6 million in fiscal year 2004 (see fig. 2). And because the cost to launch  experiments has increased, the program has only been able to launch an  average of seven experiments annually in the past 4 years (see fig. 3).  According to Space Test Program officials, demand for testing has not  diminished. S&T officials cited dwindling testing resources as a barrier to  their efforts. While the strategy states that appropriate resources need to  be allocated for on-orbit testing, it does not address how this can or will be  done.
section_title: Space S&T Workforce Facing Shortages
paragraphs: The workforce needed to carry out S&T for space is facing shortages. DOD  officials cited staff shortages with science and engineering backgrounds  and had more concerns about the future since their workforces were  reaching retirement age. These concerns were echoed by DOD and  industry studies. A 2002 study on the space research and development  industrial base conducted by Booz Allen Hamilton, for example, found that  over half of the current space R&D workforce is over 45 years old and that  departure of key talent could be especially worrisome in 10 years, as  scientists and engineers now in the 45- to 49-year-old group begin to retire  from the workforce and are replaced by a smaller pool of less experienced  personnel. In its report, the Space Commission noted that both industry  and the U.S. government face substantial shortages of scientists and  engineers and that recruitment of new personnel is difficult since the  space industry is one of many sectors competing for the limited number of  trained scientists and engineers. Booz Allen noted that areas in which  either recruitment efforts are difficult or a critical mass is lacking include  systems engineering and software engineering. The 2004 National Defense  Authorization Act directed the Secretary of Defense to promote the  development of space personnel career fields within each of the military  departments. However, we recently reported that the military services  vary in the extent to which they have identified and implemented  initiatives to develop and manage their space cadres. Moreover, the space  S&T strategy itself merely lays out goals for workforce without identifying  actions or resources needed to achieve those goals.
In recognizing that more needs to be done to develop, attract, and retain  staff with critical skills, the Defense Authorization Act for Fiscal Year 2005  Conference Report directed DOD to develop detailed implementation  plans for enhancing the space cadre and to study the ability of academia,  industry, and government to educate and train a community of space  professionals and to address the definition and development of key  competencies and skill levels in the areas of systems engineering, program  management, financial management, operations, and tactics. We believe  that S&T skill areas should also be included in the strategy given the  importance of advancing space technologies and potential future  workforce shortages.
section_title: Investment Strategy Needed to Support S&T Planning
paragraphs: DOD does not yet have a departmentwide investment strategy that could  provide a good foundation for space S&T planning. While desired  capabilities are regularly identified by military commanders and are vetted  through strategic reviews, such as the Quadrennial Defense Review, DOD  has limited ability to make trades among space, air, land, and sea  platforms in deciding how best to meet those capabilities, document those  decisions, and follow through on those decisions. For example, DOD  would like to achieve persistent surveillance to enhance military  operations. But it has not been decided how much of the earth needs to be  covered and the extent to which air-based assets, such as unmanned  reconnaissance aircraft, can achieve this capability versus space-based  assets, such as the planned space-based radar system. If DOD conducted  thorough and independent analyses of alternatives weighing the pros and  cons of using different combinations of both assets and made trade-off  decisions that could be enforced across the military services, the S&T  community could have a better basis for deciding how much S&T dollars  should go toward space-based radar technologies versus technologies  supporting air platforms.
The need for an investment strategy DOD-wide or for particular functional  areas has been cited in a variety of recent studies, including a 1999  Defense Science Board study on tactical battlefield communications and a  2004 study by the Center for Strategic and International Studies. The  recently established JCIDS process is designed to identify future  capabilities by functional areas and to make trades between space and  other platforms. However, it is unknown as to how this work will translate  into an investment strategy that could be used to enhance S&T planning.  And it is unknown how effectively decisions made through JCIDS will be  enforced. DOD has also made changes to its Planning, Programming,  Budgeting, and Execution process to provide higher-level guidance to the  budgeting process. However, it is also unclear as to how effectively these  changes will be implemented over time and whether they can serve as a  foundation for directing science and technology investments.
section_title: Funding Process Encourages Technology Development to Occur within Acquisition Programs
paragraphs: We have previously reported that an S&T environment is more forgiving  and less costly than a delivery-oriented acquisition program environment.  Events such as test “failures,” new discoveries, and time spent in attaining  knowledge are considered normal in this environment, while they are seen  as a negative event in an acquisition program. Moreover, separating  technology development and product development enables organizations  to align customer expectations with resources, and therefore minimize  problems that could hurt a program in its design and production phases.  Budget realities within DOD, however, make it more advantageous to fund  technology development in an acquisition program. Historically, S&T  organizations receive about 20 percent of DOD’s research and  development budget, while weapon system programs receive about 80  percent. The money going toward S&T is spread over several thousand  projects, while the money going toward weapons systems is spread out  over considerably fewer projects. This “distribution of wealth” makes it  easier to finance technology development within an acquisition program.  In addition, even though more money is distributed to weapon systems,  there is still considerable competition for funding. Such competition  makes it advantageous for programs to include in their design immature  technologies that offer significant performance gains. Within the space  community, there is also a perception that the length of time it takes to  develop space systems (which have only “one shot” at incorporating  technologies) demands that DOD push for continual advancement of  technologies, even after starting an acquisition program.
The impact of acquisition programs taking on technology development  that should be done in an S&T environment is considerable. Our work over  the past several decades has shown that this practice invariably leads to  unanticipated cost and schedule increases for space and other weapon  system programs since technical problems occurring within acquisition  require more time and money to fix. For some large programs for space,  cost increases have amounted to billions of dollars and delayed schedules  by years. Aside from removing technology development from a more  protective environment and from S&T oversight processes, problematic  acquisitions may also rob the S&T community and other acquisition  programs of investment dollars.
Some actions have been taken recently to address this dilemma. In  particular, DOD issued a revised directive in November 2003 expanding  the DDR&E’s oversight authority to include efforts to develop advanced  components and prototypes—RDT&E budget activity 4. According to  DDR&E officials, this authority was intended to keep technology  development out of the acquisition programs and within the S&T  community, but it will take at least 2 years to determine its success. In  addition, DOD’s revised acquisition policy for weapon systems encourages  programs not to commit to undertaking product development until  technologies are matured, that is, at a minimum tested in a relevant  environment (TRL 6) and preferably in an operational environment (TRL  7). However, in October 2003, DOD also issued a separate acquisition  policy for space, which allows technology development to continue into  product development up until a decision is made to build the first product.  At the time of our review, DOD was revising the space acquisition policy  and reexamining how long technology development should continue  within an acquisition program.
section_title: Conclusions
paragraphs: DOD has taken an initial positive step in optimizing investments in space  S&T projects by establishing short- and long-term goals, which can be  used to direct spending by S&T organizations, and by establishing a forum  by which senior leaders can assess whether spending is going in the right  direction. However, there will be significant challenges ahead for DOD in  implementing the strategy. Namely, DOD must maintain momentum  toward greater collaboration, which began under this effort. This will not  be an easy task, given the varied and competing interests of organizations  with a stake in DOD’s space S&T investment and the fact that the strategy  does not explicitly cover organizations that fall outside the realm of  traditional DOD S&T oversight. Moreover, there are formidable barriers  that stand in the way of achieving and measuring progress, including  inadequate funding visibility, decreased testing resources, workforce  deficiencies, and long-standing incentives that encourage technology  development to take place within acquisition programs rather than the  S&T community. By using the strategy as a tool for assessing and  addressing these challenges, DOD can better position itself for achieving  its goals and also strengthen the S&T base supporting space.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Defense direct the (1) Executive  Agent for Space and (2) the Under Secretary of Defense (Acquisition,  Technology and Logistics) (to whom the DDR&E reports) to make the  following improvements to space S&T strategic planning.
Establish protocols and mechanisms for enhancing coordination and  knowledge sharing between the DOD S&T community, acquisition  programs involved in space, and DOD intelligence agencies.
Ensure that the space S&T strategy fully reflects warfighter needs by  establishing links between space S&T strategic planning and DOD’s  new JCIDS. In addition, establish links to architectural development  processes to assure that S&T projects align with future technology  requirements identified in space-related architectures.
Continue to ensure that DOD has the right tools for measuring progress  in achieving its goals for space by identifying metrics that could be  used for assessing the value of S&T projects relative to strategic goals  and knowledge being gained relative to goals.
Develop plans for addressing barriers to achieving strategic goals for  S&T, including deficiencies in RDT&E funding visibility, testing  resources, and workforce. A first step would be to include skills critical  to S&T in the workforce study identified in the Fiscal Year 2005  Defense Authorization Act Conference Report.
section_title: Agency Comments
paragraphs: In commenting on a draft of this report, DOD concurred with our   recommendations and identified actions being taken to address them. (See  app. V for DOD’s comments.)
We are sending copies of this report to the Secretaries of Defense and the  Air Force and interested congressional committees. We will also make  copies available to others upon request. In addition, the report will be  available at no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (937) 258-7915. Key contributors to this report were Cristina  Chaplain, Maricela Cherveny, Jean Harker, and Rich Horiuchi.
section_title: Appendix I: Budget Activity Descriptions
paragraphs: Basic research is systematic study directed toward greater  knowledge or understanding of the fundamental aspects of  phenomena and of observable facts without specific applications  towards processes or products in mind. It includes all scientific  study and experimentation directed towards increasing  fundamental knowledge and understanding in those fields of the  physical, engineering, environmental, and life sciences related to  long-term national security needs. It is farsighted high-payoff  research that provides the basis for technological progress.
Applied research is systematic study to understand the means to  meet a recognized and specific need. It is a systematic expansion  and application of knowledge to develop useful materials, devices,  and systems or methods. Applied research may translate promising  basic research into solutions for broadly defined military needs,  short of system development. Applied research precedes system- specific technology investigations or development.
Advanced technology development includes development of  subsystems and components and efforts to integrate them into  system prototypes for field experiments and/or tests in a simulated  environment. The results of this type of effort are proof of  technological feasibility and assessment of subsystem and  component operability and producibility rather than the  development of hardware for service use. Projects in this category  have a direct relevance to identified military needs. Program  elements in this category involve pre-acquisition efforts, such as  system concept demonstration, joint and service-specific  experiments, or technology demonstrations, and generally have  technology readiness levels (TRLs) of 4, 5, or 6. Projects in this  category do not necessarily lead to subsequent development or  procurement phases, but should have the goal of moving out of  space science and technology (S&T) and into the acquisition  process within the future years defense program.
Advanced component development and prototypes consists of  efforts necessary to evaluate integrated technologies or prototype  systems in a high fidelity and realistic operating environment.  These activities include system-specific efforts that help expedite  technology transition from the laboratory to operational use.  Emphasis is on proving component and subsystem maturity prior to  integration in major and complex systems and may involve risk  reduction initiatives. Advanced component development and  prototypes efforts are to occur before an acquisition program starts  product development.
System development and demonstration consists of newly initiated  acquisition programs and includes engineering and manufacturing  development tasks aimed at meeting validated requirements prior  to full-rate production. Characteristics of this activity involve mature  system development, integration, and demonstration to support a  production decision.
RDT&E management support includes efforts to sustain and/or  modernize the installations or operations required for general  RDT&E. Such efforts may relate to test ranges, military  construction, maintenance support of laboratories, operation and  maintenance of test aircraft and ships, and studies and analyses in  support of the RDT&E program.
Operational system development includes development efforts to  upgrade systems that have been fielded or have received approval  for full-rate production and anticipate production funding in the  current or subsequent fiscal year.
section_title: Appendix II: Funding on Technology Development within Science and Technology and Acquisition Communities
paragraphs: Appendix II: Funding on Technology  Development within Science and Technology  and Acquisition Communities Funding going toward a variety of projects and sources.
section_title: Appendix III: Technology Readiness Levels and Their Definitions
subsections: section_title: 1. Basic principles observed and reported
paragraphs: Lowest level of technology readiness. Scientific research begins to be  translated into applied research and development. Examples might  include paper studies of a technology’s basic properties.
Invention begins. Once basic principles are observed, practical  applications can be invented.
Active research and development is initiated. This includes analytical  studies and laboratory studies to physically validate analytical  predictions of separate elements of the technology. Examples include  components that are not yet integrated or representative.
Basic technological components are integrated to establish that the  pieces will work together. This is relatively “low fidelity” compared to the  eventual system. Examples include integration of “ad hoc” hardware in a  laboratory.
Fidelity of breadboard technology increases significantly. The basic  technological components are integrated with reasonably realistic  supporting elements so that the technology can be tested in a simulated  environment. Examples include “high fidelity” laboratory integration of  components.
Representative model or prototype system, which is well beyond the  breadboard tested for TRL 5, is tested in a relevant environment.  Represents a major step up in a technology’s demonstrated readiness.  Examples include testing a prototype in a high fidelity laboratory  environment or in simulated operational environment.
Prototype near or at planned operational system. Represents a major  step up from TRL 6, requiring the demonstration of an actual system  prototype in an operational environment, such as in an aircraft, vehicle,  or space.
Technology has been proven to work in its final form and under  expected conditions. In almost all cases, this TRL represents the end of  true system development. Examples include developmental test and  evaluation of the system in its intended weapon system to determine if it  meets design specifications.
Actual application of the technology in its final form and under mission  conditions, such as those encountered in operational test and  evaluation. Examples include using the system under operational  mission conditions.
section_title: Appendix IV: Organizations That Participated in Developing the Space Science and Technology Strategy
section_title: Appendix V: Comments from the Department of Defense