Small Business Innovation Research: DOD's Program Has Developed Some Technologies that Support Military Users, but Lacks Comprehensive Data on Transition Outcomes
Published: 2014-07-23T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: DOD relies on its research and development community to identify, pursue, and develop new technologies that improve and enhance military operations and ensure technological superiority over adversaries. The SBIR program is a key mechanism for DOD to use small businesses to meet its research and development needs; stimulate technological innovation; foster and encourage participation by minority and disadvantaged persons in technological innovation; and increase private sector commercialization of innovations derived from federal research and development funding. DOD is the largest SBIR participant in the federal government, with over $1 billion spent annually on the program.
This testimony is based primarily on a report GAO issued in December 2013 and addresses: (1) practices the military departments use to facilitate the transition of SBIR technologies, (2) the extent to which these technologies are successfully transitioning to military users, such as weapon system programs or warfighters in the field, and (3) DOD's efforts to meet fiscal year 2012 NDAA transition reporting requirements. This statement draws from the 2013 report and other work GAO has conducted on technology transition activities in DOD's science and technology programs.
section_title: What GAO Found
paragraphs: Transitioning technologies from defense research and technology development programs, such as through the Small Business Innovation Research (SBIR) program, to military users has been a long-standing challenge for the Department of Defense (DOD). Over the past decade, Congress and DOD have taken several steps to address transition challenges in DOD's SBIR program. For example, the military departments can offer additional SBIR funding to certain awardees to supplement or extend technology development projects in order to move them closer to transition. Additionally, each of the military departments has a network of transition facilitators who work directly with small businesses, military research laboratories, and the acquisition community to foster transition opportunities. Further, in fiscal year 2012, Congress provided federal agencies the opportunity to use more of SBIR funding (up to 3 percent) for program administrative purposes, including activities that facilitate transition. However, at times, promising technologies are not taken advantage of because their potential has not been adequately demonstrated, they do not meet military requirements, or users are unable to fund the final stages of development and testing.
GAO found that DOD's SBIR program has developed some technologies that successfully transitioned into acquisition programs or fielded systems, but the extent of transition is unknown because comprehensive and reliable transition data are not collected. The military departments collect information on selected transition “success stories” on a somewhat ad hoc basis from SBIR program officials, acquisition program officials, prime contractors, or directly from small businesses. In addition to these less formal transition tracking efforts, the military departments use, to varying degrees, two data systems—Company Commercialization Reports and the Federal Procurement Data System-Next Generation—to identify transition results program-wide. While these systems provide high-level commercialization information that the departments use to track progress in achieving overall program goals, the systems have significant gaps in coverage and data reliability concerns that limit their transition tracking capabilities. In addition, the systems are not designed to capture detailed information on acquisition programs, fielded systems, or on projects that did not transition.
The National Defense Authorization Act (NDAA) for fiscal year 2012 directed DOD to begin reporting the number and percentage of SBIR projects that transition into acquisition programs or to fielded systems, among other things. DOD acknowledged that it may need to modify its existing data systems or develop new tools to compile more complete and accurate technology transition data. At the end of 2013, DOD was still assessing how to comply with the new transition reporting requirements, and had not established a specific plan, as GAO had recommended, for how and when it would be able to meet the requirements. In a recent update, DOD officials confirmed that alternatives are still being evaluated and no plan for improving the tracking and reporting of technology transition has been completed. Without better information on technology transition outcomes, questions will remain as to whether the DOD SBIR program is providing the right technologies at the right time to users, using effective approaches to select, develop, and transition technologies, and providing tangible benefits.

section_title: Letter
paragraphs: I am pleased to be here today to discuss the Department of Defense’s  (DOD) Small Business Innovation Research (SBIR) program and its  efforts to transition technologies to military users, such as weapon system  acquisition programs and the warfighters in the field. DOD relies on its  research and development community—government research  laboratories, test facilities, industry, and academia—to identify, pursue,  and develop new technologies that improve and enhance military  operations and ensure technological superiority over adversaries. The  SBIR program is a key mechanism for the department to (1) use small  businesses to meet its research and development needs; (2) stimulate  technological innovation; (3) foster and encourage participation by  minority and disadvantaged persons in technological innovation; and (4)  increase private sector commercialization of innovations derived from  federal research and development funding. DOD is the largest SBIR  participant in the federal government, with over $1 billion spent annually  on SBIR contract awards, which are implemented across 13 military  departments and defense agency components.
The Small Business Administration (SBA) is responsible for establishing  the broad policy and guidelines under which individual agencies operate  SBIR programs. Within DOD, the Office of Small Business Programs  (OSBP) oversees SBIR program activities, develops policy, and manages  program reporting. This office generally relies on the components, such  as the departments of the Army, Air Force, and Navy, to oversee and  execute their own SBIR program activities. Each component has flexibility  to tailor its SBIR program to meet its needs, including determining what  type of research to pursue, which projects to fund, and how to monitor  ongoing projects. The SBIR program is structured into three phases,  which are described in table 1.
In the context of the SBIR program, commercialization is defined broadly  to include the process of developing, producing, and delivering products,  processes, technologies, or services for sale to, or use by, the federal  government or commercial markets. For DOD, a primary goal of  commercialization is the transition of SBIR-developed technologies to  weapon system programs or directly to warfighters in the field.
In December 2013, we reported on the practices the military departments  use to facilitate the transition of technologies developed through the SBIR  program, the extent to which technologies are successfully transitioning to  military users, and DOD’s efforts to meet transition reporting requirements  established by the National Defense Authorization Act (NDAA) for fiscal   My testimony today focuses primarily on the findings of our  year 2012.2013 report, but also draws from other work we have conducted on  technology transition activities in DOD’s science and technology  enterprise. More detail on our scope and methodology is included in  these issued products. The work on which this testimony is based was  conducted in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Several Mechanisms Have Been Established Over Time to Help Facilitate SBIR Technology Transition
paragraphs: Transitioning technologies from defense research organizations and  technology development programs, such as the SBIR program, to military  users has been a long-standing challenge for DOD. To address  technology transition challenges in DOD’s SBIR program, Congress and  DOD have established several program provisions and mechanisms over  the past decade, including the following:
Congress directed the establishment of the Commercialization Pilot  Program, subsequently renamed the Commercialization Readiness  Program, in fiscal year 2006 to accelerate the transition of SBIR- funded technologies to Phase III, especially those that lead to  acquisition programs and high priority military requirements, such as  fielded systems. As part of the program, Congress authorized the  military departments to use up to 1 percent of SBIR funding to  administer the program. This funding is used to provide assistance to  SBIR awardees, including efforts to enhance networking and build  relationships among small businesses, prime contractors, and DOD  science and technology and acquisition communities.
Each military department’s SBIR program also has a network of  transition facilitators who manage the Commercialization Readiness  Program and other SBIR activities that support technology transition.  The facilitators are located at military laboratories, acquisition centers,  and program executive offices and work directly with government  stakeholders to help ensure projects are responsive to warfighter  needs. Although the roles and responsibilities vary somewhat across  the military departments, in general, transition facilitators foster ties  among small businesses, military research laboratories, and the  acquisition community in support of transition opportunities, and  monitor project progress, including outcomes. Further, the military  departments conduct conferences and workshops to provide  opportunities for SBIR companies to interact with users and showcase  their projects. In the fiscal year 2012 reauthorization of the SBIR  program, Congress authorized a pilot effort to allow the DOD SBIR  program the opportunity to use more of their SBIR funds (up to 3  percent) for program administration, technical assistance, and  commercialization and outreach activities.
In addition, DOD SBIR components may offer special awards that  supplement or extend Phase II projects. The military departments  have developed mechanisms to do this, which provide awardees  additional SBIR funding to move Phase II projects closer to transition.  In some cases, the military departments require formal technology  transition agreements or matching funding from intended military  users as a condition to receiving the additional Phase II funding.  Technology transition agreements, which Air Force and Navy officials  reported using, help manage project expectations and formalize  stakeholder commitments by outlining cost, schedule, and  performance expectations for transition to occur. Matching funds from  intended users, which are required by the Navy for some projects, can  help create greater buy-in for transition because the intended users  have a monetary stake in the project.
In fiscal year 2011, Congress required the Secretary of Defense to  establish a program to accelerate the fielding of technologies  developed pursuant to Phase II of the SBIR program, technologies  developed by defense laboratories, and other innovative  technologies.established the Rapid Innovation Program, which received about $430  million in fiscal year 2011 and $175-$225 million annually in  subsequent years. According to DOD officials, the majority of projects  awarded in the first few years of the program have been to SBIR  Phase II awardees. We are currently conducting a review of the  program and its transition outcomes for the Senate Armed Services  Committee.
As a result of this new requirement, the Secretary  Sometimes technologies are not ready to transition when needed  because they may still be too risky or costly to adopt. Further, at other  times, promising technologies are not taken advantage of because their  potential has not been adequately demonstrated or recognized, they do  not meet military user requirements, or users are unable to fund the final  stages of development and testing. As we have reported in the past on  DOD science and technology programs, factors that facilitate successful  transition outcomes include selecting the right projects—those that  address military needs, have realistic cost and schedule expectations,  and have technologies that can be matured or demonstrated—and  ensuring early and sustained commitments from intended users and other  key stakeholders throughout projects.
section_title: DOD’s SBIR Program Has Developed Some Technologies that Support Military Users, but Lacks Comprehensive Data on Transition Outcomes
paragraphs: In our 2013 review, we found that the military department SBIR programs  have identified some technologies that successfully transitioned into  acquisition programs or fielded systems over the past several years, but  the extent of transition is unknown because comprehensive and reliable  transition data are not collected. The military departments collect  information on selected transition “success stories” on a somewhat ad  hoc basis from SBIR program officials, acquisition program officials, prime  contractors, or directly from small businesses. These success stories  cover a broad range of technologies and products. One SBIR transition  example from the Air Force was an antenna that transitioned to an  unmanned air system that was undergoing operational evaluations in  Afghanistan to demonstrate identification and detection capabilities for  improvised explosive devices. Another transition success reported by the  Army was a sensor system for identifying structural fatigue or damage on  Black Hawk helicopters that was undergoing further testing. In addition to  less formal transition tracking efforts, the military departments use, to  varying degrees, two data systems—Company Commercialization  Reports (CCR) and the Federal Procurement Data System-Next  Generation (FPDS-NG)—to identify transition results for their programs.  While these systems provide high-level commercialization information  that the departments use to track progress in achieving overall program  goals, the systems have significant gaps in coverage and data reliability  concerns that limit their transition tracking capabilities. In addition, the  systems are not designed to capture detailed information on acquisition  programs, fielded systems, or on projects that did not transition. Table 2  more fully describes these transition data sources and their limitations.
section_title: DOD is in the Early Stages of Developing a Plan to Improve Technology Transition Reporting
paragraphs: New reporting requirements in the NDAA for fiscal year 2012 directed  DOD to begin reporting new SBIR-related transition information to SBA,  which is to be included in SBA’s annual report to designated  congressional committees. This requirement includes reporting on the  number and percentage of Phase II SBIR projects that transitioned into  acquisition programs or to fielded systems, effectiveness of incentives  provided to DOD program managers and prime contractors, and  additional information specific to the transition of projects funded through  the Commercialization Readiness Program. This type of information is not  currently captured by the existing data sources described in table 2.
At the time of our 2013 review, DOD was still assessing how to comply  with the new transition reporting requirements directed by Congress, and  no specific plan that included a time line for meeting the requirements had  been established. DOD acknowledged that it may need to modify its  existing data systems or develop new tools to compile more complete and  accurate technology transition data, but cited several challenges to  obtaining better data. One challenge we found in 2013 was that the  military departments and components define technology transition  differently—with definitions ranging from any commercialization dollars  applied to a project, to only when a technology is actually incorporated  into a weapon system or in direct use by the warfighter—and no  consensus had been reached on a standard definition to use. We  recommended that DOD establish a common definition of technology  transition for all SBIR projects as a key step to support annual reporting  requirements. Additionally, according to DOD SBIR officials, tracking  transition outcomes can be challenging because of the sometimes  lengthy period that can occur between SBIR project completion and  transition to a DOD user. In some cases, the time lag can be several  years and make it difficult to track projects and obscure a project’s SBIR  linkages. Time lags can occur because of delays in funding availability,  additional development or testing needs before transition, or schedule  delays encountered by intended users. Officials also stated that limited  resources for administrative activities constrain the SBIR program’s ability  to effectively follow up on the transition outcomes for completed projects.  Conversely, military users, such as weapon system acquisition programs,  often do not dedicate resources to monitor or track SBIR projects and  their likelihood to transition.
In 2013, DOD officials indicated that addressing the transition reporting  requirements would be a long-term effort because of data collection  challenges such as those identified in table 2. As a first step, DOD  initiated an assessment last year of different options for enhancing  transition data, which included examining whether CCR or other existing  DOD data sources could be modified to improve reporting, such as  Selected Acquisition Reports—annually required for major defense  acquisition programs. In addition, opportunities to build more SBIR  awareness directly into acquisition program activities were being  considered, such as including SBIR-specific provisions in acquisition  strategy documents or formal program reviews. Despite these activities,  DOD had not established a plan for how and when it would be able to  meet the reporting requirements and begin to provide the technology  transition information expected by Congress. As such, we recommended  in 2013 that DOD develop a plan to meet new technology transition  reporting requirements that will improve the completeness, quality, and  reliability of SBIR transition data, and report this plan to Congress,  including specific steps for improving the technology transition data. In a  recent update to our 2013 work, DOD officials confirmed that alternatives  are still being evaluated and no plan for how and when it would improve  the tracking and reporting of technology transition has been completed.
While we recognize there are challenges to improving transition data, we  continue to believe it is important for DOD to develop and implement a  plan for obtaining more comprehensible and reliable measures of  transition. Without better information on technology transition, questions  will remain as to whether the DOD SBIR program is providing the right  technologies at the right time to users, using effective approaches to  select, develop, and transition technologies, and providing tangible  benefits. As we have reported in the past through other work on DOD  science and technology activities, tracking technology transitions and the  impact of those transitions, such as cost savings or deployment of the  technology in a product, provides key feedback that can inform the  management of programs. In particular, we found that leading commercial  companies tracked technology transition not only to enable them to  measure success, but also to assess their processes and determine what  changes are necessary to improve transition rates. In addition, we found  a few examples of unique efforts within DOD where transition outcomes  were being effectively monitored and documented. The Navy, for  instance, uses a Transition Review Board to assess whether projects in  its Future Naval Capabilities technology development program are being  utilized in systems that support Navy warfighters. For example, the  Navy determined that of the 155 technology products the Future Naval  Capabilities program delivered to acquisition programs between fiscal  years 2006-2011, 21 percent were subsequently deployed to fleet forces,  35 percent were still with the acquisition programs, and 44 percent failed  to deploy. For projects that did not successfully deploy, the board  assessed whether there were other benefits achieved, such as whether  technologies were leveraged for follow-on science and technology work.  The board also identifies obstacles to transition, such as loss of interest  by the user or inadequacy of funding. These findings are then used to  inform the Navy’s annual review process and inform future science and  technology investment decisions.
Overcoming the challenges to obtaining better technology transition  information may ultimately require closer collaboration between the DOD  SBIR and acquisition communities. While incremental improvements may  be possible by modifying the existing CCR and FPDS-NG data systems  and increasing SBIR program managers’ capacity to track projects,  greater insights into transition outcomes and the benefits the technologies  provide to military users may not be achieved without additional  information obtained from users, such as acquisition program managers.  In an environment of declining budgets, it is important that information on  technology transition outcomes for SBIR projects be improved for DOD to  identify the extent to which the program is supporting military users and  determine whether existing monitoring and transition efforts are working  effectively. We recognize that the goal is not to transition all technologies  funded through SBIR, because not all technologies will be demonstrated  successfully. Nonetheless, it is important to ensure that the right  technologies are transitioning and to not allow these technologies to fail  for the wrong reasons.
Chairman Graves, Ranking Member Velázquez, and Members of the  Committee, this completes my prepared statement. I would be pleased to  respond to any questions that you may have at this time.