Small Business Innovation Research: DOD's Program Has Developed Some Technologies that Support Military Users, but Lacks Comprehensive Data on Transition Outcomes
Released: 2014-07-23T00:00:00

Letter

Several Mechanisms Have Been Established Over Time to Help Facilitate SBIR Technology Transition

Transitioning technologies from defense research organizations and technology development programs, such as the SBIR program, to military users has been a long-standing challenge for DOD. To address technology transition challenges in DOD’s SBIR program, Congress and DOD have established several program provisions and mechanisms over the past decade, including the following:

Congress directed the establishment of the Commercialization Pilot Program, subsequently renamed the Commercialization Readiness Program, in fiscal year 2006 to accelerate the transition of SBIR- funded technologies to Phase III, especially those that lead to acquisition programs and high priority military requirements, such as fielded systems. As part of the program, Congress authorized the military departments to use up to 1 percent of SBIR funding to administer the program. This funding is used to provide assistance to SBIR awardees, including efforts to enhance networking and build relationships among small businesses, prime contractors, and DOD science and technology and acquisition communities.

Each military department’s SBIR program also has a network of transition facilitators who manage the Commercialization Readiness Program and other SBIR activities that support technology transition. The facilitators are located at military laboratories, acquisition centers, and program executive offices and work directly with government stakeholders to help ensure projects are responsive to warfighter needs. Although the roles and responsibilities vary somewhat across the military departments, in general, transition facilitators foster ties among small businesses, military research laboratories, and the acquisition community in support of transition opportunities, and monitor project progress, including outcomes. Further, the military departments conduct conferences and workshops to provide opportunities for SBIR companies to interact with users and showcase their projects. In the fiscal year 2012 reauthorization of the SBIR program, Congress authorized a pilot effort to allow the DOD SBIR program the opportunity to use more of their SBIR funds (up to 3 percent) for program administration, technical assistance, and commercialization and outreach activities.

In addition, DOD SBIR components may offer special awards that supplement or extend Phase II projects. The military departments have developed mechanisms to do this, which provide awardees additional SBIR funding to move Phase II projects closer to transition. In some cases, the military departments require formal technology transition agreements or matching funding from intended military users as a condition to receiving the additional Phase II funding. Technology transition agreements, which Air Force and Navy officials reported using, help manage project expectations and formalize stakeholder commitments by outlining cost, schedule, and performance expectations for transition to occur. Matching funds from intended users, which are required by the Navy for some projects, can help create greater buy-in for transition because the intended users have a monetary stake in the project.

In fiscal year 2011, Congress required the Secretary of Defense to establish a program to accelerate the fielding of technologies developed pursuant to Phase II of the SBIR program, technologies developed by defense laboratories, and other innovative technologies.established the Rapid Innovation Program, which received about $430 million in fiscal year 2011 and $175-$225 million annually in subsequent years. According to DOD officials, the majority of projects awarded in the first few years of the program have been to SBIR Phase II awardees. We are currently conducting a review of the program and its transition outcomes for the Senate Armed Services Committee.

As a result of this new requirement, the Secretary Sometimes technologies are not ready to transition when needed because they may still be too risky or costly to adopt. Further, at other times, promising technologies are not taken advantage of because their potential has not been adequately demonstrated or recognized, they do not meet military user requirements, or users are unable to fund the final stages of development and testing. As we have reported in the past on DOD science and technology programs, factors that facilitate successful transition outcomes include selecting the right projects—those that address military needs, have realistic cost and schedule expectations, and have technologies that can be matured or demonstrated—and ensuring early and sustained commitments from intended users and other key stakeholders throughout projects.

DOD’s SBIR Program Has Developed Some Technologies that Support Military Users, but Lacks Comprehensive Data on Transition Outcomes

In our 2013 review, we found that the military department SBIR programs have identified some technologies that successfully transitioned into acquisition programs or fielded systems over the past several years, but the extent of transition is unknown because comprehensive and reliable transition data are not collected. The military departments collect information on selected transition “success stories” on a somewhat ad hoc basis from SBIR program officials, acquisition program officials, prime contractors, or directly from small businesses. These success stories cover a broad range of technologies and products. One SBIR transition example from the Air Force was an antenna that transitioned to an unmanned air system that was undergoing operational evaluations in Afghanistan to demonstrate identification and detection capabilities for improvised explosive devices. Another transition success reported by the Army was a sensor system for identifying structural fatigue or damage on Black Hawk helicopters that was undergoing further testing. In addition to less formal transition tracking efforts, the military departments use, to varying degrees, two data systems—Company Commercialization Reports (CCR) and the Federal Procurement Data System-Next Generation (FPDS-NG)—to identify transition results for their programs. While these systems provide high-level commercialization information that the departments use to track progress in achieving overall program goals, the systems have significant gaps in coverage and data reliability concerns that limit their transition tracking capabilities. In addition, the systems are not designed to capture detailed information on acquisition programs, fielded systems, or on projects that did not transition. Table 2 more fully describes these transition data sources and their limitations.

DOD is in the Early Stages of Developing a Plan to Improve Technology Transition Reporting

New reporting requirements in the NDAA for fiscal year 2012 directed DOD to begin reporting new SBIR-related transition information to SBA, which is to be included in SBA’s annual report to designated congressional committees. This requirement includes reporting on the number and percentage of Phase II SBIR projects that transitioned into acquisition programs or to fielded systems, effectiveness of incentives provided to DOD program managers and prime contractors, and additional information specific to the transition of projects funded through the Commercialization Readiness Program. This type of information is not currently captured by the existing data sources described in table 2.

At the time of our 2013 review, DOD was still assessing how to comply with the new transition reporting requirements directed by Congress, and no specific plan that included a time line for meeting the requirements had been established. DOD acknowledged that it may need to modify its existing data systems or develop new tools to compile more complete and accurate technology transition data, but cited several challenges to obtaining better data. One challenge we found in 2013 was that the military departments and components define technology transition differently—with definitions ranging from any commercialization dollars applied to a project, to only when a technology is actually incorporated into a weapon system or in direct use by the warfighter—and no consensus had been reached on a standard definition to use. We recommended that DOD establish a common definition of technology transition for all SBIR projects as a key step to support annual reporting requirements. Additionally, according to DOD SBIR officials, tracking transition outcomes can be challenging because of the sometimes lengthy period that can occur between SBIR project completion and transition to a DOD user. In some cases, the time lag can be several years and make it difficult to track projects and obscure a project’s SBIR linkages. Time lags can occur because of delays in funding availability, additional development or testing needs before transition, or schedule delays encountered by intended users. Officials also stated that limited resources for administrative activities constrain the SBIR program’s ability to effectively follow up on the transition outcomes for completed projects. Conversely, military users, such as weapon system acquisition programs, often do not dedicate resources to monitor or track SBIR projects and their likelihood to transition.

In 2013, DOD officials indicated that addressing the transition reporting requirements would be a long-term effort because of data collection challenges such as those identified in table 2. As a first step, DOD initiated an assessment last year of different options for enhancing transition data, which included examining whether CCR or other existing DOD data sources could be modified to improve reporting, such as Selected Acquisition Reports—annually required for major defense acquisition programs. In addition, opportunities to build more SBIR awareness directly into acquisition program activities were being considered, such as including SBIR-specific provisions in acquisition strategy documents or formal program reviews. Despite these activities, DOD had not established a plan for how and when it would be able to meet the reporting requirements and begin to provide the technology transition information expected by Congress. As such, we recommended in 2013 that DOD develop a plan to meet new technology transition reporting requirements that will improve the completeness, quality, and reliability of SBIR transition data, and report this plan to Congress, including specific steps for improving the technology transition data. In a recent update to our 2013 work, DOD officials confirmed that alternatives are still being evaluated and no plan for how and when it would improve the tracking and reporting of technology transition has been completed.

While we recognize there are challenges to improving transition data, we continue to believe it is important for DOD to develop and implement a plan for obtaining more comprehensible and reliable measures of transition. Without better information on technology transition, questions will remain as to whether the DOD SBIR program is providing the right technologies at the right time to users, using effective approaches to select, develop, and transition technologies, and providing tangible benefits. As we have reported in the past through other work on DOD science and technology activities, tracking technology transitions and the impact of those transitions, such as cost savings or deployment of the technology in a product, provides key feedback that can inform the management of programs. In particular, we found that leading commercial companies tracked technology transition not only to enable them to measure success, but also to assess their processes and determine what changes are necessary to improve transition rates. In addition, we found a few examples of unique efforts within DOD where transition outcomes were being effectively monitored and documented. The Navy, for instance, uses a Transition Review Board to assess whether projects in its Future Naval Capabilities technology development program are being utilized in systems that support Navy warfighters. For example, the Navy determined that of the 155 technology products the Future Naval Capabilities program delivered to acquisition programs between fiscal years 2006-2011, 21 percent were subsequently deployed to fleet forces, 35 percent were still with the acquisition programs, and 44 percent failed to deploy. For projects that did not successfully deploy, the board assessed whether there were other benefits achieved, such as whether technologies were leveraged for follow-on science and technology work. The board also identifies obstacles to transition, such as loss of interest by the user or inadequacy of funding. These findings are then used to inform the Navy’s annual review process and inform future science and technology investment decisions.

Overcoming the challenges to obtaining better technology transition information may ultimately require closer collaboration between the DOD SBIR and acquisition communities. While incremental improvements may be possible by modifying the existing CCR and FPDS-NG data systems and increasing SBIR program managers’ capacity to track projects, greater insights into transition outcomes and the benefits the technologies provide to military users may not be achieved without additional information obtained from users, such as acquisition program managers. In an environment of declining budgets, it is important that information on technology transition outcomes for SBIR projects be improved for DOD to identify the extent to which the program is supporting military users and determine whether existing monitoring and transition efforts are working effectively. We recognize that the goal is not to transition all technologies funded through SBIR, because not all technologies will be demonstrated successfully. Nonetheless, it is important to ensure that the right technologies are transitioning and to not allow these technologies to fail for the wrong reasons.

Chairman Graves, Ranking Member Velázquez, and Members of the Committee, this completes my prepared statement. I would be pleased to respond to any questions that you may have at this time.