Small Business Innovation Research: DOD's Program Supports Weapon Systems, but Lacks Comprehensive Data on Technology Transition Outcomes
Published: 2013-12-20T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: To compete in the global economy, the United States relies heavily on innovation through research and development. The Small Business Innovation Development Act of 1982 initiated SBIR programs across federal agencies in an effort to stimulate innovation through small businesses. DOD spends over $1 billion annually to support SBIR awards. The Conference Report accompanying the National Defense Authorization Act for Fiscal Year 2013 mandated that GAO assess the transition of technologies developed through the DOD SBIR program. This report examines (1) practices the military department SBIR programs use to facilitate the transition of SBIR technologies, and (2) the extent to which SBIR technologies are transitioning to DOD users, including major weapon system acquisition programs. GAO reviewed SBIR program documentation and data. GAO also interviewed officials from DOD's Office of Small Business Programs and the military departments to determine the practices used to facilitate technology transition and assess SBIR transition outcome data.
section_title: What GAO Found
paragraphs: The Small Business Innovation Research (SBIR) programs within the military departments use a variety of practices and tools to facilitate technology transition--the act of passing technologies developed in the science and technology environment on to users such as weapon system acquisition programs or warfighters in the field. GAO identified some common transition practices and tools across SBIR programs. For example, specific initiatives, such as the Commercialization Readiness Program, are used by each SBIR program and focus resources on enhancing technology transition opportunities. Transition facilitators are also used by each program to provide a network of personnel who manage SBIR activities that support technology transition. GAO also found some different practices and tools used to support technology transition efforts, such as the Navy Transition Assistance Program, which provides consulting services and helps showcase SBIR projects in an effort to improve small businesses' abilities to transition their projects. Transition facilitation efforts are supported by administrative funds provided through each program's SBIR budget and from other funds received from their respective military department. A recent increase in the amount of administrative funding that can come from SBIR budgets is expected to help the programs enhance their transition facilitation efforts.
GAO was unable to assess the extent of technology transition associated with the military department SBIR programs because comprehensive and reliable technology transition data for SBIR projects are not collected. Transition data systems used by DOD provide some transition information but have significant gaps in coverage and data reliability concerns. The military departments have additional measures through which they have identified a number of successful technology transitions, but these efforts capture a limited amount of transition results. SBIR transition reporting requirements recently established by Congress have led DOD to evaluate its options for providing transition data. GAO identified several challenges to attaining complete and accurate technology transition data. For instance, the lack of a common definition for technology transition across SBIR programs could cause reporting inconsistencies. Additionally, tracking transition can be challenging because of the sometimes lengthy period between SBIR project completion and transition to a DOD user. DOD initially plans to use transition data from Company Commercialization Reports--viewed by DOD as the best available source--to meet the new transition reporting requirements. However, SBIR officials indicated that addressing transition reporting requirements is a longer-term effort, and there is no specific plan including a time line for when DOD will be able to support those requirements. Without a plan that establishes a time line, it is unclear how and when DOD will begin to provide the technology transition information expected by Congress. Although Congress did not specify when reporting was to begin, it expects DOD to report new transition-related information to the Administrator of the Small Business Administration to meet the new reporting requirements. However, unless DOD communicates its plan and accompanying time line, the congressional committees to whom the Small Business Administration reports may be unaware of the data quality issues with the transition-related information DOD plans to use to support reporting in the near term.
section_title: What GAO Recommends
paragraphs: GAO recommends that DOD establish a common definition of technology transition for SBIR projects, develop a plan to track transition that will improve the completeness, quality, and reliability of transition data, and report to Congress its plan for meeting new SBIR technology transition reporting requirements. DOD partially concurred with these recommendations, but cited challenges to improving transition data. GAO believes options are available to address the challenges.

section_title: Letter
paragraphs: To compete in the global economy, the United States relies heavily on  innovation through research and development. Recognizing the potential  for small businesses to be a source of significant innovation, Congress  enacted the Small Business Innovation Development Act of 1982. The  act established the Small Business Innovation Research (SBIR) Program  for use by defense and civilian agencies to stimulate technological  innovation and utilize small businesses to meet federal technology  needs. The Department of Defense (DOD) has 13 SBIR program  components (programs) and spends over $1 billion annually on SBIR  contract awards. These programs provide DOD with opportunities to  acquire technological innovations that can improve military capabilities  and ensure superiority over adversaries. The conference report  accompanying the National Defense Authorization Act for Fiscal Year  2013 mandated that GAO assess the transition of technologies developed  through the DOD SBIR program. This report examines (1) practices the  military department SBIR programs use to facilitate the transition of  technologies, and (2) the extent to which SBIR technologies are  transitioning to DOD users, including major weapon system acquisition  programs.
In conducting this review, we focused on the SBIR programs within the  three military departments—Air Force, Army, and Navy—which in recent  years have comprised about 75 percent of the total DOD SBIR program  budget. We evaluated documentation and interviewed officials from  DOD’s Office of Small Business Programs and the three military  departments on processes and practices used to facilitate technology  transition—the act of passing technologies developed in the science and  technology environment on to users—and track transition outcomes. We  reviewed information on the SBIR program structure and processes for  selecting and managing technology projects, and the practices used to  support the transition of SBIR technologies to DOD acquisition programs  and other military users. In addition, we reviewed practices and data  systems that DOD uses to track SBIR projects and their outcomes. We  also reviewed available documentation on these practices and systems,  and interviewed DOD officials to assess whether they provide accurate,  reliable, and comprehensive data on the transition of technologies to  military users. Any limitations that were identified for these data collection  practices and data systems are discussed later in this report.
We conducted this performance audit from April 2013 to December 2013  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. Please see Appendix I for  further details on the scope and methodology used for this work.
subsections: section_title: Background
paragraphs: DOD relies on its science and technology community—DOD research  laboratories, test facilities, industry, and academia—to identify, pursue,  and develop new technologies that address military needs. The DOD  SBIR program is one mechanism through which DOD attempts to  accomplish its science and technology goals and develop technologies  that contribute to weapon systems or transition directly to warfighters for  use in the field. Within DOD, the Office of Small Business Programs  oversees the department’s SBIR program activities, develops policy, and  manages program reporting. This office generally relies on the agencies,  such as the Army, Air Force, and Navy, to oversee and execute their own  SBIR program activities. Each agency has flexibility to tailor its SBIR  program to meet its needs, including determining what type of research to  pursue, which projects to fund, and how to monitor ongoing projects.
To initiate the project award process, SBIR programs work with the  science and technology and acquisition communities to generate and  prioritize research and development topics. These topics describe  technical areas of interest and capability needs, which the programs use  in their solicitations for proposals from small businesses. DOD conducts  three solicitations each year where small businesses compete for Phase I  contract awards that are expected to respond to the needs identified in  each topic. Once awarded, SBIR projects are managed through a three- phase program structure, which is outlined in table 1.
The number of Phase I and Phase II projects varies from year to year  based on technology needs and funding availability. Table 2 shows the  budgets and project awards reported for the military department SBIR  programs in fiscal year 2012.
We and others have previously found that DOD and its technology  development programs, such as the SBIR program, have encountered  challenges to transitioning their technologies to acquisition programs or  directly to the warfighter for use in the field. For instance, in our past  work we found several reasons why technologies may not transition,  including insufficient maturity, inadequate demonstration or recognition by  users of a technology’s potential, and unwillingness or inability of  acquisition programs to fund final stages of development.
To address SBIR technology transition challenges, DOD, the Small  Business Administration, and Congress have established additional  program provisions, incentives, and reporting requirements. For example,  the Commercialization Readiness Program was initiated to accelerate the  transition of SBIR funded technologies to Phase III, especially those that  lead to acquisition programs and high priority military requirements, such  as fielded systems. military departments to use up to 1 percent of SBIR funding for  administrative activities that facilitate transition. This funding is used to  support program staff and contractors who provide assistance to SBIR  awardees, including efforts to enhance networking and build relationships  among small businesses, prime contractors, and DOD science and  technology and acquisition communities.
The National Defense Authorization Act for Fiscal Year 2006 authorized the Commercialization Pilot  Program under the Secretary of Defense and the Secretary of each military department. Pub. L. No.  109-163, § 252. The National Defense Authorization Act for Fiscal Year 2012 continued the program  and renamed it the Commercialization Readiness Program. Although the program may support any  Phase III awards, such as technology transition to commercial products, DOD is required to provide  goals to increase the number of Phase II SBIR contracts that lead to technology transition into  programs of record or fielded systems and to use incentives to meet those goals. Pub. L. No. 112-81,  § 5122(a).
Requires DOD to set a goal to increase the number of Phase II  contracts awarded that lead to technology transition into acquisition  programs or fielded systems, and use incentives to encourage  program managers and prime contractors to meet the goal.
Requires that DOD report specific transition-related information to the  Administrator of the Small Business Administration for inclusion in an  annual report to designated congressional committees. This includes  reporting the number and percentage of Phase II contracts that led to  technology transition into acquisition programs or fielded systems,  information on the status of each project that received funding through  the Commercialization Readiness Program and efforts to transition  those projects, and a description of each incentive used to meet the  department’s transition goal.
Authorizes DOD to establish goals for the transition of Phase III  technologies in subcontracting plans for contracts of $100 million or  more, and to require prime contractors on such contracts to report the  number and dollar amount of contracts entered into by prime  contractors for Phase III projects.
Sets the ceiling for discretionary technical assistance that can be  provided annually for all Phase I and Phase II projects at $5,000 per  project. Programs can use this funding to assist awardees in making  technical decisions on projects, solving technical problems,  minimizing technical risks, and commercializing projects.
Establishes a pilot effort to allow DOD SBIR programs to use not  more than 3 percent of their SBIR budgets for, among other things,  program administration, technical assistance, and the implementation  of commercialization and outreach initiatives.
section_title: Military Department SBIR Programs Use a Variety of Practices, Tools, and Administrative Funds to Support Technology Transition Efforts
paragraphs: The military department SBIR programs use several management  practices and tools to support technology transition efforts. We identified  some common transition elements across the programs, but also found  some differences in how each program approaches its technology  transition efforts. The programs’ technology transition efforts are  supported through use of administrative funds coming from their SBIR  budgets and other funds provided by their respective military department.  The transition facilitation practices, tools, and funds used to promote the  transition of SBIR technologies include the following:  Early focus on transition through topic generation and project  selection: Technology transition efforts begin with topic generation  and project selection processes that emphasize the pursuit of projects  for which there is a demonstrated military need and potential transition  opportunities. To do this, the military department programs formally  engage stakeholders from the science and technology and acquisition  communities in generating and endorsing topics for SBIR solicitations.  In proposing topics and selecting projects, programs have to balance  their desire for technological innovation with meeting pressing  warfighter needs. SBIR officials stated that projects that pursue  incremental improvements generally are more likely to deliver the  technical capability expected for technology transition to occur. In  contrast, they noted that projects that focus more on “leap-forward”  technology innovations that can support future warfighting needs tend  to require more long-term development and have greater technical  and transition risks. DOD policy requires that at least 50 percent of  military department topics are endorsed by the acquisition community,  such as program executive offices. This helps ensure that the  acquisition community is engaged with the SBIR programs and that a  significant portion of projects are dedicated to addressing specific  needs identified by military users.
Phase II transition initiatives: Transition-focused activities increase  as Phase II projects progress, commensurate with an increasing  technology maturity and understanding of a project’s potential  opportunities for use. In particular, the military department SBIR  programs target transition opportunities through their  Commercialization Readiness Programs and other initiatives that  provide additional support to select Phase II projects. In some cases,  the SBIR programs require formal technology transition agreements  or matching funding as a condition to receiving additional Phase II  funding. Technology transition agreements, which Air Force and Navy  officials reported using, help manage project expectations and  formalize stakeholder commitments by outlining cost, schedule, and  performance expectations for transition to occur. Matching funds from  intended users, which are required by the Navy for some projects, can  help create greater buy-in for transition because the intended users  have a monetary stake in the project.
Transition facilitators: Each military department SBIR program has  a network of transition facilitators who manage the Commercialization  Readiness Program and other enhancement efforts, as well as  broader SBIR activities that support technology transition. The  facilitators are located at military labs, acquisition centers, and  program executive offices to work directly with government  stakeholders and help ensure projects are responsive to warfighter  needs. They also help small businesses identify and position  themselves for opportunities to transition their SBIR technologies.  Although the roles and responsibilities vary somewhat across the  programs, in general, transition facilitators assist with topic generation  and prioritization; foster communication among small businesses,  research laboratories, and the acquisition community in support of  transition opportunities; and monitor project progress, including  outcomes.
Navy Transition Assistance Program: The Navy established an  additional program over a decade ago to prepare its SBIR participants  for technology transition opportunities. The Transition Assistance  Program is a voluntary 11-month program with, on average, about  two-thirds of Phase II recipients participating each year. It provides  consulting services focused on improving the small businesses’  abilities to transition their SBIR products, including assistance in  transition planning and developing marketing tools. Under the  program, profiles are used to describe the expected capability, level of  technology maturity, and potential technology transition opportunities  for each project. These profiles are available in electronic form  through a web-based portal called the Navy Virtual Acquisition  Showcase, and support the annual Navy Opportunity Forum  conference. The conference provides Transition Assistance Program  participants with direct exposure and one-on-one opportunities to  interact with prospective transition partners in the government and  industry.
Other transition facilitation tools: SBIR programs also use  technology roadmaps and formal relationship-building activities, such  as conferences and workshops, to support transition efforts.  Technology roadmaps are schedule-based planning documents used  to identify opportunities for SBIR technology insertion into acquisition  programs or direct use by the warfighter. Conferences and  workshops, such as the annual Beyond Phase II conference hosted  by the Office of Small Business Programs, are used by the programs  to provide opportunities for SBIR Phase II companies to interact  directly with prospective government and industry users and  showcase their projects.
Administrative funds: The technology transition practices and tools  used by the programs are supported by administrative funds provided  through their SBIR budgets as well as non-SBIR sources from their  respective agencies. The Commercialization Readiness Program and  discretionary technical assistance provisions enable programs to use  portions of their SBIR budgets to fund administrative activities,  including transition support. For fiscal year 2012, this funding totaled  about $12 million across the three military departments. The fiscal  year 2012 SBIR reauthorization included a new provision—which  DOD officials advocated—that allows the programs to use up to 3  percent of their funds to support administrative activities, which is in  addition to funds available through the Commercialization Readiness  Program and discretionary technical assistance. SBIR officials stated  that although this additional funding allowance is in the initial stages of  being used, they believe these funds will help enhance transition  facilitation measures for their programs going forward. Additional  agency funding outside of the SBIR budget is also used to manage  programs and support transition activities, but the amount of such  funding is not readily identifiable because the military departments do  not all require that the amount of funding used to support associated  administrative efforts be documented.
section_title: DOD Lacks Comprehensive Data on SBIR Technology Transition Outcomes, but Some Transition Successes Have Been Identified
paragraphs: We were unable to assess the extent of technology transition associated  with the military department SBIR programs because comprehensive and  reliable technology transition data are not collected. Tracking  mechanisms used by DOD—Company Commercialization Reports (CCR)  and the Federal Procurement Data System-Next Generation (FPDS- NG)—provide some information on SBIR Phase III activities, but these  mechanisms have significant gaps in coverage and data reliability  concerns that limit their transition tracking capabilities. The military  departments have additional measures through which they have identified  a number of successful SBIR transitions to DOD acquisition programs  and directly to fielded systems, but these efforts capture a limited amount  of transition information. DOD is assessing how to comply with the new  transition reporting requirements directed by Congress, but has yet to  develop a plan that will support identification and annual reporting of the  extent to which SBIR technologies transition to DOD acquisition programs  or to fielded systems.
subsections: section_title: SBIR Programs Use Three Resources to Identify Transition Results, but Each Has Inherent Data Limitations
paragraphs: The military department SBIR programs rely, to varying degrees, on two  data systems—CCR and FPDS-NG—as well as their own agency-specific  data collection activities to identify transition results. Table 3 more fully  describes the data sources used and their limitations.
Although the CCR and FPDS-NG data systems do not capture complete  data on the transition of SBIR technologies, they do provide high-level  commercialization information that the SBIR programs use to track  progress in achieving program goals. Because the data help support  program management efforts, the Office of Small Business Programs and  the military departments, to varying degrees, take steps to verify the  quality of CCR and FPDS-NG data. For example, the Army assesses and  validates CCR data for its projects on an ongoing basis. This process  involves comparing recent updates to the database with FPDS-NG  contract data and internal Army tracking data to confirm the accuracy of  commercialization funding reported by the small businesses. The Navy  SBIR program uses FPDS-NG as its primary source of commercialization  data and employs similar validation techniques to improve the accuracy of  commercialization data tracked through this system. By comparing  contracts in FPDS-NG flagged as SBIR-related to DOD contract  management systems, the Navy is able to verify the accuracy of Phase III  awards data tied to government contracts. Both the Army and Navy  officials acknowledged, however, that even with their data validation  efforts, problems persist because of the limitations of the Company  Commercialization Reports and FPDS-NG.
The military department programs have developed some internal  capabilities to track certain projects and provide insight into the types of  capabilities enabled by them. Like Company Commercialization Reports  and FPDS-NG, these capabilities do not provide comprehensive transition  information, but may help the departments to gain more insight into  transition outcomes for some technologies developed within SBIR  programs and to respond to DOD and congressional inquiries about  program results. In particular, the programs identify transition success  stories for a limited number of projects, ranging from Phase III awards for  additional research and development to transition to major acquisition  programs or fielded systems. Information on these success stories can  come from SBIR program officials, acquisition program officials, prime  contractors, or directly from the small businesses. The Air Force’s  database of identified transition successes includes 95 transition stories  dating back to 2004. The Army’s program produces an annual report  describing transition outcomes for 20-30 successful projects. The Navy’s  program maintains a searchable database of SBIR projects that includes  profiles on select transitioned projects as well. Table 4 provides examples  of transition outcomes for projects identified through our review of these  reporting mechanisms.
SBIR program officials within the military departments emphasized that, in  addition to their broader program efforts to identify transition outcomes,  some acquisition organizations have implemented their own practices to  track transition. For example, the Navy Program Executive Office for  Submarines tracks the transition of SBIR technologies to its acquisition  programs by managing a list of companies, the value of contract awards,  the specific program office associated with each contract award, and the  SBIR technology associated with the award. The office indicated that 20  active Phase III awards associated with its acquisition program efforts are  being tracked.
section_title: DOD Is in the Early Stages of Developing a Plan to Improve Technology Transition Tracking and Reporting
paragraphs: The National Defense Authorization Act for Fiscal Year 2012 mandated  that DOD report new transition-related information to the Administrator of  the Small Business Administration who will report this information  annually to designated congressional committees. This reporting will  include information on the number and percentage of Phase II projects  that transition into acquisition programs or to fielded systems, the efficacy  of steps taken by DOD to increase the number of transitioned projects,  and additional information specific to the transition of projects funded  through Commercialization Readiness Programs. In order to provide  more complete and accurate transition data to support the new reporting  requirements, DOD recognizes it may need to modify its existing data  systems or develop new tools to better capture the transition results for  SBIR projects.
According to the Office of Small Business Programs, DOD’s response to  the new reporting requirements is still being evaluated, in part because  there are several challenges to compiling complete and accurate  technology transition data. One such challenge we found was variation  across the military departments in their definitions of technology  transition. Specifically, transition definitions ranged from any  commercialization dollars applied to a project, to only when a technology  is actually incorporated into a weapon system or in direct use by the  warfighter. The Office of Small Business Programs acknowledged that a  standard DOD SBIR definition of technology transition must be ensured  before the congressionally-required reporting begins. Standards for  internal control state that management should establish procedures to  ensure that it is able to achieve its objectives, such as being able to  compile and report consistent, complete, and accurate data.according to SBIR officials, tracking transition outcomes can be   Additionally,  challenging because the sometimes lengthy period between SBIR project  completion and transition to a DOD user can obscure a project’s SBIR  linkages. Time lags can occur because of delays in transition funding  availability, additional development or testing needs before transition, or  schedule delays encountered by intended users. During the time between  project completion and transition, personnel associated with projects may  change and technologies may evolve. This increases the likelihood that  transitions associated with SBIR technologies go unacknowledged. SBIR  officials within the military departments also stated that limited resources  for administrative activities constrain their ability to effectively follow up on  the transition outcomes for completed projects.
Although the Office of Small Business Programs acknowledges the  limitations of CCR data, the initial plan is to use this data source—viewed  by DOD as the best available—as the primary means for beginning to  address the new transition reporting requirements. Additionally, in an  effort to improve DOD’s future technology transition reporting and its  understanding of transition results in general, the Office of Small  Business Programs has initiated an assessment of different options for  enhancing transition data. For example, as part of this assessment, DOD  is examining whether CCR could be modified to improve reporting.  Additionally, existing DOD reporting mechanisms, such as Selected  Acquisition Reports—annually required for major defense acquisition  programs—are being considered as potential vehicles for supporting  SBIR technology transition reporting. Opportunities to build more SBIR  awareness directly into acquisition activities are being considered as well,  such as including provisions in acquisition strategy documents or formal  program reviews. According to the Office of Small Business Programs,  DOD intends to issue a policy directive in fiscal year 2014 that will provide  guidance for implementing overall SBIR program requirements. However,  SBIR officials indicated that addressing technology transition reporting  requirements is viewed as a longer-term effort because of the challenges  we have discussed, and no specific plan including a time line has been  established for when DOD will be able to support those requirements.  Without a plan that establishes a time line, it is unclear how and when  DOD will begin to provide the technology transition information expected  by Congress. Although Congress did not specify when reporting was to  begin, it expects DOD to report new transition-related information to the  Administrator of the Small Business Administration to meet the National  Defense Authorization Act for Fiscal Year 2012 requirement. However, as  stated above, DOD expects this to be a longer-term effort and designated  congressional committees may not be aware of when DOD will likely have  developed the capability to provide comprehensive and accurate data.
Further, unless DOD communicates its plan and accompanying time line,  these committees may be unaware that the transition-related information  DOD plans to provide in the near-term to address the National Defense  Authorization Act for Fiscal Year 2012 requirements has data quality  issues. Standards for internal control emphasize the need for federal  agencies to establish plans to help ensure goals and objectives can be  met, including compliance with applicable laws and regulations. Further,  communicating internal control efforts on a timely basis to external  stakeholders, such as congressional committees, helps ensure that  effective oversight can take place.
section_title: Conclusions
paragraphs: The SBIR program efforts within DOD provide opportunities for small  businesses to develop new technologies that may improve current U.S.  military capabilities and provide innovative solutions to address future  needs of the warfighter. However, information on technology transition  outcomes for SBIR projects is limited. Consequently, DOD cannot identify  the extent to which the program is supporting military users. The Office of  Small Business Programs is taking steps to respond to new technology  transition reporting requirements, but has not yet determined how and  when it will more completely and reliably track and report on the extent of  transition for SBIR technologies. While initial reporting efforts are  expected to use existing data systems, such as CCR, DOD will need to  overcome the inherent limitations of data collected through those systems  if it expects to provide a comprehensive picture of transition outcomes.
section_title: Recommendations for Executive Action
paragraphs: To improve tracking and reporting of technology transition outcomes for  SBIR projects, we recommend that the Secretary of Defense direct the  Office of Small Business Programs to take the following three actions:  1.  Establish a common definition of technology transition for all SBIR  projects to support annual reporting requirements;  2.  Develop a plan to meet new technology transition reporting  requirements that will improve the completeness, quality, and  reliability of SBIR transition data; and  3.  Report to Congress on the department’s plan for meeting the new  SBIR reporting requirements set forth in the program’s fiscal year  2012 reauthorization, including the specific steps for improving the  technology transition data.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a copy of a draft of this report to DOD for review and  comment. Written comments from the department are included in  appendix II of this report.
DOD partially concurred with our recommendations. In its response, DOD  stated that it has established a working group that is currently working  with all stakeholders to develop a common definition of technology  transition for all SBIR projects. DOD also agreed that it is important to  improve the completeness, quality, and reliability of SBIR transition data,  but noted that it has significant concerns related to the difficulty in actually  capturing the data. The department indicated that the full scope of data  collection challenges and associated resource needs is unknown at this  time.
While we recognize there are challenges to improving transition data, we  believe there are avenues already available that DOD could pursue to  improve transition data that may not require extensive resource  commitments. For example, DOD’s SBIR program could work more  closely with its acquisition community to track transition outcomes. As  outlined in this report, some acquisition organizations have developed  their own practices to track transition outcomes, which the program may  be able to leverage for use on a broader scale. In addition, DOD could  consider greater use of contracting provisions to require contractors to  report on SBIR project activities, or use existing program reporting  mechanisms, such as Selected Acquisition Reports, to capture additional  transition information.
We believe that collection of better data is not only needed to support the  congressional reporting requirements, but also to help DOD assess the  efficacy of existing transition efforts and the benefits the program yields  for the warfighter. DOD stated it will continue with initiatives that seek to  improve the collection of SBIR technology transition data. However, it did  not specify if or when it intends to develop a plan for meeting the  transition reporting requirements. We continue to believe a plan that  includes a time line for when DOD will begin to support reporting  requirements should be provided to the designated congressional  committees in the near term to make clear the limitations of reported  transition data and the department’s approach to improving the data over  time.
We are sending copies of this report to appropriate congressional  committees and the Secretary of Defense. In addition, this report will be  available at no charge on the GAO website at http://www.gao.gov. If you  or your staff have any questions concerning this report, please contact  me at (202) 512-4841 or by email at sullivanm@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be  found on the last page of this report. Key contributors to this report are  listed in appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: To identify what processes are used by the Department of Defense  (DOD) to facilitate transition for Small Business Innovation Research  (SBIR) technologies, we reviewed prior reports by GAO, DOD and other  organizations, such as the National Research Council and the Rand  Corporation, as well as reviewed DOD policies, procedures, and funding  information. Using this information, we scoped our work to focus on the  SBIR activities conducted by the Air Force, Army, and Navy. These three  organizations typically receive about three-fourths of the annual SBIR  funding that supports the 13 participating DOD organizations. With the  military department SBIR programs as our focus, we interviewed DOD  officials from the Office of Small Business Programs and the SBIR  program offices at the Air Force, Army, and Navy on practices and tools  used to facilitate technology transition. In addition, we interviewed and  collected documentation from DOD officials within the acquisition  community concerning their use of and interactions with the SBIR  program. Specifically, we interacted with officials at the Air Force Life  Cycle Management Center and Air Force Research Laboratory; the Army  Aviation and Missile, Research, Development, and Engineering Center;  the Naval Sea Systems Command; and the F-35 Joint Program Office.  This included interviewing SBIR program management and transition  facilitation personnel at each location, as applicable.
Similarly, to assess the extent to which SBIR technologies are  transitioning to DOD users, we met with officials in the Office of Small  Business Programs, military department SBIR program offices, and the  aforementioned military acquisition organizations to discuss what data are  available to measure transition of SBIR technologies to acquisition  programs, or directly to warfighters in the field. We determined that DOD  uses two primary data systems—Company Commercialization Reports  and the Federal Procurement Data System-Next Generation. We  discussed with DOD officials what data are collected by these systems,  how the data are validated and used, and whether there are limitations to  the data collected. We also reviewed available documentation on the  systems. In assessing data limitations, we discussed with SBIR officials  whether the systems provide accurate, reliable, and comprehensive data  on SBIR projects that transition to military users. In addition, we  interviewed military department officials about other data collection  practices they may have implemented to track SBIR projects and results.  Any limitations that were identified for the data collection practices and  data systems used to identify technology transition outcomes for SBIR  projects are discussed in this report.
We conducted this performance audit from April 2013 to December 2013  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, John Oppenheim, Assistant  Director; Danielle Greene; Victoria Klepacz; Sean Merrill; Scott Purdy;  and Sylvia Schatz also made key contributions to the report.