Advanced Technology Program: Inherent Factors in Selection Process Are Likely to Limit Identification of Similar Research
Published: 2005-05-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Advanced Technology Program (ATP) supports research that accelerates the development of high-risk technologies with the potential for broad-based economic benefits for the nation. Under the program, administrators at the National Institute of Standards and Technology are to ensure that they do not fund research that would be conducted in the same period without ATP funding. Between 1990 and September 2004, ATP funded 768 projects at a cost of about $2.3 billion. There is a continuing debate over whether the private sector has sufficient incentives to undertake research on high-risk, high-payoff emerging technologies without government support, such as ATP. This testimony discusses the results of GAO's April 2000 report, Advanced Technology Program: Inherent Factors in the Selection Process Could Limit Identification of Similar Research (GAO/RCED-00-114) and provides updated information. GAO determined (1) whether ATP had funded projects with research goals that were similar to projects funded by the private sector and (2) if ATP did, whether its award selection process ensures that such research would not be funded in the future.
section_title: What GAO Found
paragraphs: The three completed ATP-funded projects GAO reviewed, which were approved for funding in 1990 and 1992, addressed research goals that were similar to those already funded by the private sector. GAO chose these 3 projects from among the first 38 completed projects, each representing a different technology sector: computers, electronics, and biotechnology. These three technology sectors represent 26 of the 38 completed ATP projects, or 68 percent. The projects included an on-line handwriting recognition system, a system to increase the capacity of existing fiber optic cables for the telecommunications industry, and a process for turning collagen into fibers for human prostheses use. In the case of the handwriting recognition project, ATP provided $1.2 million to develop a system to recognize cursive handwriting for pen-based (i.e., without a keyboard) computer input. GAO identified several private firms that were conducting similar research on handwriting recognition at approximately the same time the ATP project was funded. In fact, this line of research began in the late 1950s. In addition, GAO identified multiple patents, as early as 5 years prior to the start of the ATP project, in the field of handwriting recognition. GAO found similar results in the other two projects. Two inherent factors in ATP's award selection process--the need to guard against conflicts of interest and the need to protect proprietary information--make it unlikely that ATP can avoid funding research already being pursued by the private sector in the same time period. These factors, which have not changed since 1990, make it difficult for ATP project reviewers to identify similar efforts in the private sector. For example, to guard against conflicts of interest, the program uses technical experts who are not directly involved with the proposed research. Their acquaintance with ongoing research is further limited by the private sector's practice of not disclosing its research efforts or results so as to guard proprietary information. As a result, it may be impossible for the program to ensure that it is consistently not funding existing or planned research that would be conducted in the same time period in the absence of ATP financial assistance. GAO made no recommendations in its April 2000 report.

section_title: Letter
paragraphs: We are pleased to be here today to discuss our past work, as well as to  provide some updated information, on the funding that the Advanced  Technology Program (ATP) provides for private research. As you know,  ATP was established in 1988 to support research that accelerates the  development of high-risk technologies with the potential for broad-based  economic benefits for the nation. Under the provisions establishing ATP,  program administrators at the National Institute of Standards and  Technology (NIST) are to ensure that they are not funding existing or  planned research that would be conducted in the same time period in the  absence of ATP financial assistance. Between 1990 and September 2004,  ATP funded 768 projects at a cost of about $2.3 billion in federal matching  funds.
Research can provide both private benefits, which accrue to the owners of  the research results, and societal benefits, which accrue to society as a  whole. In some instances, the private sector does not fund research that  would be beneficial to society because doing so might not provide an  adequate return on a firm’s investment. To address this situation, the  federal government, through tax credits or direct public funding, supports  research that has very broad societal benefits, such as basic research and  research focused on developing technologies in areas such as public  health and nutrition, energy conservation, and environmental protection.  However, there is a continuing debate over whether the private sector has  sufficient incentives to undertake research on high-risk, high-payoff  emerging and enabling technologies without government support, such as  ATP.
In this context, in our prior work, we determined (1) whether, in the past,  ATP had funded projects with research goals that were similar to projects  funded by the private sector and (2) if we identified such cases, whether  ATP’s award selection process ensures that such research would not be  funded in the future. To determine whether ATP has funded projects  similar to private sector projects, we chose 3 of the first 38 completed  projects, each representing a different technology sector: biotechnology;  electronics; and information, computers, and communications. These  three technology sectors represent 26 of the 38, or 68 percent, of the ATP  projects completed by 1999. We reviewed the ATP project files and held  discussions with industry and academic experts, technical reviewers, and  award recipients to assist in our examination of these projects. We also  conducted patent searches on the technical areas associated with each of  the three projects. Our objective was not to provide an evaluation of the  quality of the research funded by ATP or the private sector, nor the impact  these projects may or may not have had on their respective industries. To  address the second objective, we reviewed ATP’s award selection process.  We did not review the overall management of the program. We performed  our initial work from October 1999 through April 2000, and developed  updated information in May 2005, in accordance with generally accepted  government auditing standards.
subsections: section_title: Background
paragraphs: ATP, which began funding projects in fiscal year 1990, was intended to  fund high-risk research and development (R&D) projects with broad  commercial and societal benefits that would not be undertaken by a single  company or group of companies, either because the risk was too high or  because the economic benefits of success would not accrue to the  investors. ATP is viewed as a mechanism for fostering investment in areas  in which societal returns would exceed private returns. ATP has addressed  other opportunities to achieve broader societal goals, such as small  business participation, as well as the establishment of joint ventures for  high-risk technologies that would be difficult for any one company to  justify because, for example, the benefits spread across the industry as a  whole. Thus, ATP is seen by some as a means of addressing market failure  in research areas that would otherwise not be funded, thereby facilitating  the economic growth that comes from the commercialization and use of  new technologies in the private sector. Advocates of the program believe  that the government should serve as a catalyst for companies to cooperate  and undertake important new work that would not have been possible in  the same time period without federal participation. Critics of the program  view ATP as industrial policy, or the means by which government rather  than the marketplace picks winners and losers.
ATP provides funding through cooperative agreements—a type of  financial assistance in which the federal government is substantially  involved in project management. ATP offers these agreements through  announced annual competitions. It provides multiyear funding to single  companies and to industry-led joint ventures. The proposal review and  selection process is a multistep process based on NIST regulations. In  general, these steps include a preliminary screening, technical and  business reviews, semifinalist identification, oral reviews, ranking, and  final selection. At the beginning of each round of ATP competitions, NIST  establishes Source Evaluation Boards (SEBs) to ensure that all proposals  receive careful consideration. Each SEB is comprised of NIST technical  experts as well as outside specialists with backgrounds in business and  economics. ATP supplements the SEBs with outside technical reviewers,  generally federal government experts in the specific industry of the  proposal. Independent business experts are also hired on a consulting  basis, including high-tech venture capitalists, people who teach strategic  business planning, retired corporate executives from large and small high- tech businesses, as well as economists and business development  specialists. All SEB members and outside reviewers must sign  nondisclosure statements, agree to protect proprietary information, and  certify that they have no conflicts of interest.
As part of the proposal evaluation process, ATP uses the external  reviewers to assess the technical and business merit of the proposed  research. Each proposal is sponsored by both technical and business SEB  members, whose roles include identifying reviewers, summarizing  evaluative comments, and making recommendations to the SEB. The SEB  evaluates the proposals, selects the semifinalists, conducts oral interviews  with semifinalists, and ranks the semifinalists. A source selecting official  makes the final award decisions based on the ranked list of proposals  from the SEB.
The three projects that we reviewed received funding through the ATP  competitions announced in 1990 and 1992. In those years, the selection  criteria included scientific and technical merit, potential broad-based  benefits, technology transfer benefits, the proposing organization’s  commitment level and organizational structure, and the qualifications and  experience of the proposing organization’s staff. Each of the five selection  criteria was weighted at 20 percent. Today, these same selection criteria  are used but are grouped into two categories, each weighted at 50 percent.  The “Scientific and Technical Merit” category addresses a variety of issues  related to the technical plan and the relevant experience of the proposing  organization. The second category, “Potential for Broad-Based Economic  Benefits,” addresses the means to achieving an economic benefit and  commercialization plans, as well as issues related to the proposer’s level of  commitment, organizational structure, and management plan. Technical  and business reviewers complete documentation, referred to as technical  and business evaluation worksheets, that address various aspects of these  criteria.
section_title: Three ATP Projects Addressed Similar Research Goals to Projects in the Private Sector
paragraphs: The three completed projects that we reviewed addressed research goals  that were similar to goals the private sector was addressing at about the  same time. Each of the three projects was from a different sector of  technology—computers, electronics, and biotechnology. The projects  include (1) an on-line handwriting recognition system for computer input,  (2) a system to increase the capacity of existing fiber optic cables for the  telecommunications industry, and (3) a process for turning collagen into  fibers for human prostheses use.
subsections: section_title: ATP Project on Handwriting Recognition
paragraphs: Both the ATP project and several private sector projects had a similar  research goal of developing an on-line system to recognize natural or  cursive handwritten data without the use of a keyboard. This technology  would make computers more useful where keyboard use is limited by  physical problems or in situations where using a keyboard is not practical.  On-line handwriting recognition means that the system recognizes  handwritten data while the user writes. The primary technical problem in  handwriting recognition is that writing styles vary greatly from person to  person, depending upon whether the user is in a hurry, fatigued, or subject  to a variety of other factors. While the technology for obtaining  recognition of constrained careful writing or block print writing was  commercially available, systems for cursive writing recognition were not  commercially available because of the greater handwriting variability that  was encountered.
The ATP project we reviewed sought to develop an on-line natural  handwriting recognition system that was user-independent and able to  translate natural or cursive handwriting. Communication Intelligence  Corporation (CIC) was the award recipient. CIC used its ATP funding of  $1.2 million from 1991 to 1993 to build its own algorithms and models for  developing its handwriting recognition system. During the project, CIC  created a database that includes thousands of cursive handwriting samples  and developed new recognition algorithms. Some of this technology has  been incorporated into a registered software product that has the ability to  recognize cursive writing in limited circumstances.
According to the experts we interviewed, as well as literature and patent  searches, several companies were attempting to achieve a similar goal of  handwriting recognition through their research around the same time that  the ATP project received funding. Some of the key players in the private  sector conducting research on cursive handwriting recognition included  Paragraph International (in collaboration with Apple Computer) and  Lexicus (which later became a division of Motorola). For example, Apple  licensed a cursive handwriting recognition system from a Soviet company,  Paragraph International, according to articles published in computer  magazines in October 1991. According to these sources, this technology  provided Apple with a foundation for recognizing printed, cursive, or  block handwritten text.
Another indication of research with a similar goal appeared in the October  1990 edition of PC Week, which reported that “handwriting recognition is  an emerging technology that promises increased productivity both for  current microcomputer owners and for a new breed of users armed with  hand-held ‘pen-based’ computers.” Similarly a technical journal article  indicated that there was renewed interest in the 1980s in this field of on- line handwriting recognition, from its advent in the 1960s, because of more  accurate electronic tablets, more compact and powerful computers, and  better recognition algorithms.
Moreover, according to the U.S. Patent and Trademark Office’s (PTO)  database, over 450 patents were issued on handwriting recognition  software, concepts, and related products from 1985 through 1999,  indicating that research of a similar goal was being conducted around the  time of the ATP project. Given the fact that it can take many years  between the time a research project takes place and the time that an  outcome is realized, this time period for a patent search allowed us to  determine whether there was research ongoing during the time of the ATP  project. The dates of the patents actually occurred sometime after the  research was conducted. And, as we reported in a prior report, the time  between the point when a patent application is filed until the date when a  patent is issued, or the application is abandoned, ranged from 19.8 months  to 21 months, adding additional time to when the research was done.
section_title: ATP Project on Capacity Expansion of Fiber Optic Cables
paragraphs: Another ATP project we reviewed, which proposed to develop a system to  increase the capacity of existing fiber optic cables for the  telecommunications industry, also had a similar goal to that of research in  the private sector. At the same time, firms in the private sector were  attempting to increase the number of light signals that can be transmitted  through a single strand of fiber optic cable using a technology called  wavelength division multiplexing (WDM). In the 1980s, telephone  companies laid fiber optic cables across the United States and other  countries to create an information system that could carry significantly  more data than the copper wires they replaced. Tremendous increases in  cable traffic, primarily from the Internet, have crowded these cables. WDM  technology was aimed at providing a cost-effective alternative to the  expensive option of installing additional fiber optic cables.
Accuwave Corporation (Accuwave) was the ATP award recipient.  Accuwave used its ATP funding of approximately $2 million from March  1993 through March 1995 to develop a wavelength division multiplexing  system that would substantially increase the number of signals that could  be transmitted through a single optical fiber strand, using the concept of  volume holography. Volume holography uses holograms to direct multiple  light signals simultaneously through a single fiber strand. Accuwave was  able to make improvements on these issues but not enough to fully  develop and market a successful WDM system for the telecommunications  market. In 1996, a competitor beat Accuwave to the market. After the  completion of the ATP project, Accuwave filed for bankruptcy protection  due to its inability to successfully commercialize a wavelength division  multiplexing system.
Other private firms were involved in research with a similar goal of  increasing the capacity of fiber optic cable at about the same time as  Accuwave was conducting its research. Conceptual research on such  systems dates back to the early 1980s, but development and  commercialization did not flourish until the mid- to late-1990s. Bell Labs  (now Lucent Technologies), Nortel Networks, and Ciena Corporation,  among others, were considered some of the major competitors in the  industry. In the early 1990s, these firms were attempting to develop WDM  technology using different methods and materials. For example, Ciena  Corporation developed a system that incorporated fiber-Bragg gratings,  which are filters embedded directly onto fiber optic cable that help to  separate multiple light signals through a single fiber strand.
We also found an indication of WDM-related research through a review of  issued patents. According to PTO’s database, over 2,000 patents were  issued related to wavelength division multiplexing components, systems,  and concepts from 1985 through 1999. The patents issued ranged from 10  patents in 1985 to 493 in 1999.
section_title: ATP Project on Regenerating Tissues and Organs
paragraphs: Both the ATP project and private sector projects we identified in the tissue  engineering field had similar broad research goals of developing biological  equivalents for defective tissues and organs utilizing diverse technical  approaches. ATP’s project proposed procedures for extracting, storing,  spinning, and weaving collagen (the main constituent of connective tissue  and bones) into fibers suitable for human prostheses that could induce the  body’s cells to regenerate lost tissue. Tissue Engineering, Inc., received  ATP’s award of about $2 million for use over the years 1993 through 1996.  The company’s long-term and yet unrealized goal is to transplant these  prostheses into humans, after which the collagen framework, or scaffold,  would induce the growth and function of normal body cells within it,  eventually remodeling lost human tissue and replacing the scaffold.
Within the very innovative field of tissue engineering, however, many  competitors were attempting to achieve similar broad research goals.  Organogenesis, the Collagen Corporation, Integra LifeSciences, Advanced  Tissue Sciences, Genzyme Tissue, Osiris Therapeutics, Matrix  Pharmaceuticals, and ReGen Biologics are key players in the market to  develop structures that could replace or regenerate cells, tissues, and  organs such as skin, teeth, orthopedic structures, cartilage, and valves. A  number of these companies have subsequently received ATP awards. In  addition, universities and medical schools have researchers investigating  the many possibilities to engineer human tissues, and eventually complex  organs, such as the liver, pancreas, and heart. According to one expert,  there is a great deal of competition within the field of tissue engineering.
Although the Tissue Engineering, Inc. research focused on the use of  collagen as the basis for these structures, other companies were pursuing  a variety of technical approaches for addressing the goal of developing  biological equivalents for defective tissues and organs. In addition to  research in collagen, other companies and researchers have also been  attempting to create human tissues and organs from other biological  materials, synthetics, and hybrid products, which are both biologic and  synthetic. For example, researchers from the Massachusetts Institute of  Technology (MIT) developed an artificial skin product using collagen and  a natural polymer. Several companies have since developed comparable  products. In 1986, researchers from MIT and a hospital in Massachusetts  began inserting cells into scaffolds created of biodegradable polymer. As  the cells multiply, tissues form. The magazine BusinessWeek reported this  concept as “an elegantly simple concept that underlies most engineered  tissue.” Two competitors, Integra LifeSciences and Organogenesis,  reported that they were also doing work on the use of collagen in various  applications. Although their technical approaches were different than the  ATP project, the broad research goals were similar.
In addition to our discussions with experts and literature searches, patent  research shows that there was activity related to the field of tissue  engineering prior to and during the ATP project. According to a search  done on the PTO website, at least 370 patents were issued related to cell  culturing, scaffolding or matrix development, and tissue engineering from  1985 through 1999. Experts have also indicated that there are several  patents related to the field, with a considerable amount of overlap in the  technologies described in those patents.
section_title: ATP’s Award Selection Process Is Unlikely to Avoid Funding Similar Research
paragraphs: Two factors in ATP’s award selection process could result in ATP’s  funding research similar to research that the private sector would fund in  the same time period. These two factors are inherent in the review process  and limit the information the reviewers have on similar private sector  research efforts. Due to conflict-of-interest concerns, technical reviewers  are precluded from being directly involved with the proposed research,  making them less likely to know about all the research in an area. Also, the  information available about private sector research is limited because of  the private sector practice of not disclosing research results. Until a patent  is issued, a private sector firm generally publishes very few details about  the research to protect proprietary information. Therefore, it is difficult for  the reviewers to identify other cutting-edge research.
subsections: section_title: ATP’s Conflict-of-Interest Provision Limits Its Ability to Identify Similar Research
paragraphs: ATP selection officials rely on outside technical reviewers to evaluate a  proposal’s scientific and technical merit. All reviewers must certify that  they have no conflicts of interest. To minimize possible conflicts of  interest, the technical reviewers are generally federal government  employees who are experts in the specific technology of the research  proposal but are not directly involved with the proposed research area.  Although this approach helps to guard against conflict of interest, it has  inherent limitations on the program’s ability to identify similar research  efforts. The technical reviewers rely on their own knowledge of research  underway in the private sector. One of the technical reviewers we  interviewed said that he did not personally know of other companies that  were doing similar work. However, he believed that it was unlikely that  there were not dozens of others working on the same issue.
Proprietary Information Limits ATP’s Ability to Identify Similar Research  ATP reviewers are significantly limited in their ability to identify similar  research efforts by an inherent lack of information on private sector  research. Although ATP officials use several sources, such as colleagues,  conferences and symposia, and current technical literature, to try to  identify research efforts conducted by the private sector and the federal  government, this information is often proprietary. Most of the private  sector and university experts we consulted agreed that it can be very  difficult to identify the specific research that private sector firms are  conducting, especially considering the competitive nature of most  industries. The early release of information on a company’s research could  be costly to the firm. If a competing firm could determine the nature and  progress of another company’s research, it could help the competitor to  develop and commercialize an identical or higher-quality product before  the other firm. At the very least, the early release of research information  by a firm can give competitors an idea as to the focus of the firm’s  strategic plan. Thus, many firms are very careful about releasing detailed  information related to research and development activities they are  conducting.
In conclusion, Mr. Chairman, the process ATP follows to select projects  for funding is limited in its ability to identify similar research efforts in the  private sector. Our retrospective look at the three ATP research projects  showed that their goals were similar to research goals already being  funded by the private sector. Examining the process that ATP uses to  select projects, we found two inherent factors—the need to guard against  conflicts of interest and the need to protect proprietary information—that  limit ATP’s ability to identify similar research efforts in the private sector.  These two factors have not changed since the beginning of the program.  We recognize the valid need to guard against conflicts of interest and to  protect proprietary information; thus, we did not recommend any changes  to the award selection process. However, we believe that it may be  impossible for the program to ensure that it is consistently not funding  existing or planned research that would be conducted in the same time  period in the absence of ATP financial assistance.
Mr. Chairman, this concludes my prepared statement. I would be happy to  respond to any questions that you or Members of the Subcommittee may  have.
section_title: Contacts and Acknowledgements
paragraphs: For further information about this testimony, please contact Robin M.  Nazzaro at 202-512-6246. Diane Raynes, Carol Herrnstadt Shulman, and  Jessica Evans made key contributions to this statement.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.