Federal Research: Observations on the Small Business Innovation Research Program
Published: 2005-06-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Since it was established in 1982, GAO has consistently reported on the success of the Small Business Innovation Research (SBIR) program in benefiting small, innovative companies, strengthening their role in federal research and development (R&D), and helping federal agencies achieve their R&D goals. However, through these reviews GAO has also identified areas where action by participating agencies or the Congress could build on the program's successes and improve its operations. This statement for the record summarizes the program's successes and improvements over time, as well as the continuing challenge of assessing the long term results of the program.
section_title: What GAO Found
paragraphs: Between July 1985 and June 1999, GAO reviewed, reported, and testified on the SBIR program many times at the request of the Congress. While GAO's work focused on many different aspects of the program, it generally found that SBIR is achieving its goals to enhance the role of small businesses in federal R&D, stimulate commercialization of research results, and support the participation of small businesses owned by women and/or disadvantaged persons. Participating agencies and companies that GAO surveyed during the course of its reviews generally rated the program highly. GAO also identified areas of weaknesses and made recommendations that, if addressed, could strengthen the program further. Some of these concerns related to (1) duplicate funding for similar, or even identical, research projects by more than one agency, (2) inconsistent interpretations of extramural research budgets by participating agencies, (3) geographical concentration of awards in a small number of states, and (4) lack of clarification on the emphasis that agencies should give to a company's commercialization record when assessing its proposals. Most of GAO's recommendations for program improvement have been either fully or partially addressed by the Congress in various reauthorizations of the program or by the agencies themselves. One issue that continues to remain somewhat unresolved after almost two decades of program implementation is how to assess the performance of the SBIR program. As the program has matured, the Congress has emphasized the potential for commercialization as an important criterion in awarding funds and the commercialization of a product as a measure of success for the program. However, in 1999, GAO reported that the program's other goals also remain important to the agencies. By itself, according to some program managers, limited commercialization may not signal "failure" because a company may have achieved other goals, such as innovation or responsiveness to an agency's research needs. GAO identified a variety of reasons why assessing the performance of the SBIR program has remained a challenge. First, because the authorizing legislation and the Small Business Administration's (SBA) policy directives do not define the role of the company's commercialization record in determining commercial potential and the relative importance of the program's goals, different approaches have emerged in agencies' evaluations of proposals. Second, GAO found that it has been difficult to find practical ways to define and measure the SBIR program's goals in order to evaluate proposals. For example, the authorizing legislation lacks a clear definition of "commercialization," and agencies sometimes differed on its meaning. Finally, GAO reported that as the emphasis on commercialization had grown, so had concerns that noncommercial successes may not be adequately recognized. For example, program managers identified various projects that met special military or medical equipment needs but that had limited sales potential.

section_title: Letter
paragraphs: We are pleased to have the opportunity to comment on the Small Business  Innovation Research (SBIR) program. Since the program’s inception, we  have consistently reported on its success in benefiting small, innovative  companies, strengthening their role in federal research and development  (R&D), and helping federal agencies achieve their R&D goals. However,  through these reviews we have also identified areas where action by  participating agencies or the Congress could build on the program’s  successes and improve its operations. Over the life of the program these  recommendations have largely been implemented. This statement will  discuss the program’s successes as well as the continuing challenge of  assessing the long term results of the SBIR program.
As a competitor in the global economy, the United States relies heavily on  innovation through research and development. The potential of small  businesses to be sources of significant innovation led the Congress to  increase government funding for R&D projects with commercial potential  that are conducted by small high-technology companies. In this context,  the Small Business Innovation Development Act of 1982 established the  SBIR program to stimulate mission-related technological innovation, use  small businesses to meet federal R&D needs, foster participation by  minority and disadvantaged persons in technological innovation, and  increase private sector commercialization of innovations derived from  federal R&D. The act provided for a three-phased program: phase I to  determine the feasibility and scientific and technical merit of a proposed  research idea; phase II to further develop the idea, taking into account its  commercial potential; and phase III to commercialize the resulting product  or process with no further SBIR funding.
The original program was reauthorized in 1986, extending the program’s  expiration date from 1988 to 1993. In 1992, it was reauthorized by the  Small Business Research and Development Enhancement Act to expand  and improve the program, to emphasize its goal of increasing private  sector commercialization, to increase participation by small businesses,  and to improve the government’s dissemination of program-related  information. In addition, the act increased funding for phase-I and phase- II proposals to $100,000 and $750,000, respectively, with adjustments once  every 5 years for inflation and changes in the program. The program was  again reauthorized in 2000 by the Consolidated Appropriations Act of  2001, which directed the Small Business Administration (SBA) and  participating agencies to, among other things, expand the scope of  publicly available information on specific grants and to annually report on  their SBIR programs. In addition, the act requires award recipients to  provide information to help SBA evaluate the program. The SBIR program  is currently scheduled to expire on September 30, 2008.
Current law requires every federal department with an R&D budget of  $100 million or more to establish and operate a SBIR program funded by a  set percentage of that agency’s extramural R&D budget—originally 1.25  percent and now 2.5 percent. In addition, agencies with R&D spending  above $20 million are directed to establish goals for financing small  business R&D projects at levels higher than the previous year. As of fiscal  year 2004, 12 federal agencies participated in the SBIR program, including  the departments of Agriculture, Commerce, Defense (DOD), Education,  Energy, Health and Human Services (HHS), Homeland Security, Housing  and Urban Development, and Transportation; the Environmental  Protection Agency; the National Aeronautics and Space Administration  (NASA); and the National Science Foundation (NSF). Each agency  manages its own program, while SBA plays a central administrative role,  such as issuing policy directives and annual reports for the program.  Awards from three agencies—DOD, National Institutes of Health, and  NASA—account for the majority of SBIR funds. From its inception in  fiscal year 1983 through fiscal year 2003, federal agencies have awarded  over $15 billion for more than 76,000 projects.
subsections: section_title: SBIR Program Has Generally Met Its Goals
paragraphs: Between July 1985 and June 1999, we reviewed, reported, and testified on  the SBIR program many times at the request of the Congress. While our  work focused on many different aspects of the program, we generally  found that SBIR is achieving its goals to enhance the role of small  businesses in federal R&D, stimulate commercialization of research  results, and support the participation of small businesses owned by  women and/or disadvantaged persons. Participating agencies and  companies that we surveyed during the course of our reviews generally  rated the program highly. Specific examples of program success that we  identified include the following:    High-quality research. Throughout the life of the program, awards have  been based on technical merit and are generally of good quality. For  example, in 1989 we reported that according to agency officials, more than  three-quarters of the research conducted with SBIR funding was as good  as or better than other agency-funded research. Agency officials also rated  the research as more likely than other research they oversaw to result in  the invention and commercialization of new products. When we again  looked at the quality of research proposals in 1995, we found that while it  was too early to make a conclusive judgment about the long-term quality  of the research, the quality of proposals remained good, according to  agency officials.
Widespread competition. The SBIR program successfully attracts many  qualified companies, has had a high level of competition, and consistently  has had a high number of first-time participants. Specifically, we reported  that the number of proposals that agencies received each year had been  increasing. In addition, as we reported in 1998, agencies rarely received  only a single proposal in response to a solicitation, indicating a sustained  level of competition for the awards. We also found that the agencies  deemed many more proposals worthy of awards than they were able to  fund. For example, the Air Force deemed 1,174 proposals worthy of  awards in fiscal year 1993 but funded only 470. Moreover, from fiscal years  1993 through 1997, one third of the companies that received awards were  first-time participants. This suggests that the program attracts hundreds of  new companies annually.
Effective outreach. SBIR agencies consistently reach out to foster  participation by women-owned or socially and economically  disadvantaged small businesses. For example, we found that DOD’s SBIR  managers participated in a number of regional small business conferences  and workshops that are specifically designed to foster increased  participation by women-owned and socially and economically  disadvantaged small businesses.
Successful commercialization. SBIR successfully fosters  commercialization of research results. At various points in the life of the  program we have reported that SBIR has been successful in increasing  private sector commercialization of innovations. For example, past GAO  and DOD surveys of companies that received SBIR Phase II funding have  determined that approximately 35 percent of the projects resulted in the  sales of products or services, and approximately 45 percent of the projects  received additional developmental funding. We have also reported that  agencies were using various techniques to foster commercialization. For  example, in an attempt to get those companies with the greatest potential  for commercial success to the marketplace sooner, DOD instituted a Fast  Track Program, whereby companies that are able to attract outside  commitments/capital for their research during phase I are given higher  priority in receiving a phase II award.
Helping to serve mission needs. SBIR has helped serve agencies’ missions  and R&D needs. Agencies differ in the emphasis they place on funding  research to support their mission and to support more generalized  research. Specifically, we found that DOD links its projects more closely  to its mission. In comparison, other agencies emphasize research that will  be commercialized by the private sector. Many of the projects DOD funded  have specialized military applications while NIH projects have access to  the biomedical market in the private sector. Moreover, we found that SBIR  promotes research on the critical technologies identified in lists developed  by DOD and/or the National Critical Technologies Panel. Generally  agencies reviewed these listings of critical technologies to develop  research topics or conducted research that fell within one of the two lists.
section_title: Improvements Made to the SBIR Program Over Time
paragraphs: We have also identified areas of weaknesses and made recommendations  that, if addressed, could strengthen the program further. Many of our  recommendations for program improvement have been either fully or  partially addressed by the Congress in various reauthorizations of the  program or by the agencies themselves. For example,    Duplicate funding. In 1995, we identified duplicate funding for similar, or  even identical, research projects by more than one agency. A few  companies received funding for the same proposals two, three, and even  five times before agencies became aware of the duplication. Contributing  factors included the fraudulent evasion of disclosure by companies  applying for awards, the lack of a consistent definition for key terms such  as “similar research,” and the lack of interagency sharing of data on  awards. In response to our recommendations, SBA strengthened the  language agencies use in their application packages to clearly warn  applicants about the illegality of entering into multiple agreements for  essentially the same effort and developed Internet capabilities to access  SBIR data for all of the agencies. In SBA’s view, the stronger language  regarding the illegality of seeking funding for similar or identical projects  addresses the need to develop consistent definitions to help agencies  determine when projects are “similar.”
Inconsistent interpretations of extramural research budgets. In 1998, we  found that while agency officials adhered to SBIR’s program and statutory  funding requirements, they used differing interpretations of how to  calculate their “extramural research budgets.” As a result some agencies  were inappropriately including or excluding some types of expenses. To  address our recommendation that SBA provide additional guidance on  how participating agencies were to calculate their extramural research  budgets, the Congress in 2000 required that the agencies report annually to  SBA on the methods used to calculate their extramural research budgets.
Geographical concentration of awards. In 1999, in response to  congressional concerns about the geographical concentration of SBIR  awards, we reported that companies in a small number of states,  especially California and Massachusetts, have submitted the most  proposals and won the majority of awards. The distribution of awards  generally followed the pattern of distribution of non-SBIR expenditures for  R&D, venture capital investments, and academic research funds. We  reported that some agencies had undertaken efforts to broaden the  geographic distribution of awards and that the program implemented by  the National Science Foundation had been particularly effective. Although  we did not make any recommendations on how to improve the program’s  outreach to states receiving fewer awards, in the 2000 reauthorization of  the program, Congress established the Federal and State Technology  Partnership Program to help strengthen the technological competitiveness  of small businesses, especially in those states that receive fewer SBIR  grants.
Clarification on commercialization and other SBIR goals. Finally, in  response to our continuing concern that clarification was needed on the  relative emphasis that agencies should give to a company’s  commercialization record and SBIR’s other goals when evaluating  proposals, in 2000 the Congress required companies applying for a second  phase award to include a commercialization plan with their SBIR  proposals. This requirement partially addressed our concern. Moreover, in  the spring of 2001, SBA initiated efforts to respond to our recommendation  to develop standard criteria for measuring commercial and other  outcomes of the SBIR program, such as uniform measures of sales and  developmental funding, and incorporate these criteria into its Tech-Net  database. Specifically, SBA began implementing a reporting system to  measure the program’s commercialization success. In fiscal year 2002,  SBA further enhanced the reporting system to include commercialization  results that would help establish an initial baseline rate of  commercialization. In addition, small business firms participating in the SBIR program are required to provide information annually on sales and  investments associated with their SBIR projects.
section_title: Assessing the Performance of the SBIR Program Remains a Challenge
paragraphs: One issue that continues to remain somewhat unresolved after almost two  decades of program implementation is how to assess the performance of  the SBIR program. As the program has matured, the Congress has  emphasized the potential for commercialization as an important criterion  in awarding funds and the commercialization of a product as a measure of  success for the program. However, in 1999, we reported that the program’s  other goals also remain important to the agencies. By itself, according to  some program managers, limited commercialization may not signal  “failure” because a company may have achieved other goals, such as  innovation or responsiveness to an agency’s research needs. We identified  a variety of reasons why assessing the performance of the SBIR program  has remained a challenge.
First, because the authorizing legislation and SBA’s policy directives do  not define the role of the company’s commercialization record in  determining commercial potential and the relative importance of the  program’s goals, different approaches have emerged in agencies’  evaluations of proposals. As a result, the relative weight that should be  given to the program’s goals when evaluating proposals remains unclear.  Innovation and responsiveness to an agency’s needs, for example, may  compete with the achievement of commercialization. In the view of many  program managers, innovation involves a willingness to undertake R&D  with a higher element of risk and a greater chance that it may not lead to a  commercial product; responsiveness to an agency’s needs involves R&D  that may be aimed at special niches with limited commercial potential.  Striking the right balance between achieving commercial sales and  encouraging new, unproven technologies is, according to the program  managers, one of the key ingredients in the program’s overall success.
Second, we found that it has been difficult to find practical ways to define  and measure the SBIR program’s goals in order to evaluate proposals. For  example, the authorizing legislation lacks a clear definition of  “commercialization,” and agencies sometimes differed on its meaning. This  absence of a definition makes it more difficult to determine when a  frequent winner is “failing” to achieve a sufficient level of  commercialization and how to include this information in an agency’s  review of the company’s proposal. Similarly, efforts to define and measure  technological innovation, which was one of the program’s original goals,  have posed a challenge. Although definitions vary, there is widespread  agreement that technological innovation is a complex process, particularly  in the development of sophisticated modern technologies.
Finally, we reported that as the emphasis on commercialization had  grown, so had concerns that noncommercial successes may not be  adequately recognized. For example, program managers identified various  projects that met special military or medical equipment needs but that had  limited sales potential. These projects would be helpful in reducing the  agency’s expenditures and meeting the mission of the agency but may not  be appropriately captured in typical measurements of commercialization.  In general, we found that program managers valued both noncommercial  and commercial successes and feared that the former might be ignored in  emphasizing the latter.
To help evaluate the performance of the program, in the 2000  reauthorization of SBIR, Congress required SBA to develop a database that  would help the agency collect and maintain in common format necessary  program output and outcome information. The database is to include the  following information on all phase II awards: (1) revenue from the sale of  new products or services resulting from the SBIR funded research, (2)  additional investment from any non-SBIR source for further research and  development, and (3) any other description of outputs and outcomes of  the awards. In addition, the database is to include general information for  all applicants not receiving an award including an abstract of the project.
In conclusion, Mr. Chairman, our work has shown that, overall, the SBIR  program has been successful in meeting its goals and that the Congress  and the agencies have implemented actions to strengthen the program  over time. However, an assessment of the program’s results remains a  challenge because of the lack of clarity on how much emphasis the  program should place on commercialization versus other goals.
For further information, please contact Anu Mittal at (202) 512-3841 or  mittala@gao.gov.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.