Advanced Technology Program: Inherent Factors in Selection Process Are Likely to Limit Identification of Similar Research
Released: 2005-05-26T00:00:00

Letter

Background

ATP, which began funding projects in fiscal year 1990, was intended to fund high-risk research and development (R&D) projects with broad commercial and societal benefits that would not be undertaken by a single company or group of companies, either because the risk was too high or because the economic benefits of success would not accrue to the investors. ATP is viewed as a mechanism for fostering investment in areas in which societal returns would exceed private returns. ATP has addressed other opportunities to achieve broader societal goals, such as small business participation, as well as the establishment of joint ventures for high-risk technologies that would be difficult for any one company to justify because, for example, the benefits spread across the industry as a whole. Thus, ATP is seen by some as a means of addressing market failure in research areas that would otherwise not be funded, thereby facilitating the economic growth that comes from the commercialization and use of new technologies in the private sector. Advocates of the program believe that the government should serve as a catalyst for companies to cooperate and undertake important new work that would not have been possible in the same time period without federal participation. Critics of the program view ATP as industrial policy, or the means by which government rather than the marketplace picks winners and losers.

ATP provides funding through cooperative agreements—a type of financial assistance in which the federal government is substantially involved in project management. ATP offers these agreements through announced annual competitions. It provides multiyear funding to single companies and to industry-led joint ventures. The proposal review and selection process is a multistep process based on NIST regulations. In general, these steps include a preliminary screening, technical and business reviews, semifinalist identification, oral reviews, ranking, and final selection. At the beginning of each round of ATP competitions, NIST establishes Source Evaluation Boards (SEBs) to ensure that all proposals receive careful consideration. Each SEB is comprised of NIST technical experts as well as outside specialists with backgrounds in business and economics. ATP supplements the SEBs with outside technical reviewers, generally federal government experts in the specific industry of the proposal. Independent business experts are also hired on a consulting basis, including high-tech venture capitalists, people who teach strategic business planning, retired corporate executives from large and small high- tech businesses, as well as economists and business development specialists. All SEB members and outside reviewers must sign nondisclosure statements, agree to protect proprietary information, and certify that they have no conflicts of interest.

As part of the proposal evaluation process, ATP uses the external reviewers to assess the technical and business merit of the proposed research. Each proposal is sponsored by both technical and business SEB members, whose roles include identifying reviewers, summarizing evaluative comments, and making recommendations to the SEB. The SEB evaluates the proposals, selects the semifinalists, conducts oral interviews with semifinalists, and ranks the semifinalists. A source selecting official makes the final award decisions based on the ranked list of proposals from the SEB.

The three projects that we reviewed received funding through the ATP competitions announced in 1990 and 1992. In those years, the selection criteria included scientific and technical merit, potential broad-based benefits, technology transfer benefits, the proposing organization’s commitment level and organizational structure, and the qualifications and experience of the proposing organization’s staff. Each of the five selection criteria was weighted at 20 percent. Today, these same selection criteria are used but are grouped into two categories, each weighted at 50 percent. The “Scientific and Technical Merit” category addresses a variety of issues related to the technical plan and the relevant experience of the proposing organization. The second category, “Potential for Broad-Based Economic Benefits,” addresses the means to achieving an economic benefit and commercialization plans, as well as issues related to the proposer’s level of commitment, organizational structure, and management plan. Technical and business reviewers complete documentation, referred to as technical and business evaluation worksheets, that address various aspects of these criteria.

Three ATP Projects Addressed Similar Research Goals to Projects in the Private Sector

The three completed projects that we reviewed addressed research goals that were similar to goals the private sector was addressing at about the same time. Each of the three projects was from a different sector of technology—computers, electronics, and biotechnology. The projects include (1) an on-line handwriting recognition system for computer input, (2) a system to increase the capacity of existing fiber optic cables for the telecommunications industry, and (3) a process for turning collagen into fibers for human prostheses use.

ATP Project on Handwriting Recognition

Both the ATP project and several private sector projects had a similar research goal of developing an on-line system to recognize natural or cursive handwritten data without the use of a keyboard. This technology would make computers more useful where keyboard use is limited by physical problems or in situations where using a keyboard is not practical. On-line handwriting recognition means that the system recognizes handwritten data while the user writes. The primary technical problem in handwriting recognition is that writing styles vary greatly from person to person, depending upon whether the user is in a hurry, fatigued, or subject to a variety of other factors. While the technology for obtaining recognition of constrained careful writing or block print writing was commercially available, systems for cursive writing recognition were not commercially available because of the greater handwriting variability that was encountered.

The ATP project we reviewed sought to develop an on-line natural handwriting recognition system that was user-independent and able to translate natural or cursive handwriting. Communication Intelligence Corporation (CIC) was the award recipient. CIC used its ATP funding of $1.2 million from 1991 to 1993 to build its own algorithms and models for developing its handwriting recognition system. During the project, CIC created a database that includes thousands of cursive handwriting samples and developed new recognition algorithms. Some of this technology has been incorporated into a registered software product that has the ability to recognize cursive writing in limited circumstances.

According to the experts we interviewed, as well as literature and patent searches, several companies were attempting to achieve a similar goal of handwriting recognition through their research around the same time that the ATP project received funding. Some of the key players in the private sector conducting research on cursive handwriting recognition included Paragraph International (in collaboration with Apple Computer) and Lexicus (which later became a division of Motorola). For example, Apple licensed a cursive handwriting recognition system from a Soviet company, Paragraph International, according to articles published in computer magazines in October 1991. According to these sources, this technology provided Apple with a foundation for recognizing printed, cursive, or block handwritten text.

Another indication of research with a similar goal appeared in the October 1990 edition of PC Week, which reported that “handwriting recognition is an emerging technology that promises increased productivity both for current microcomputer owners and for a new breed of users armed with hand-held ‘pen-based’ computers.” Similarly a technical journal article indicated that there was renewed interest in the 1980s in this field of on- line handwriting recognition, from its advent in the 1960s, because of more accurate electronic tablets, more compact and powerful computers, and better recognition algorithms.

Moreover, according to the U.S. Patent and Trademark Office’s (PTO) database, over 450 patents were issued on handwriting recognition software, concepts, and related products from 1985 through 1999, indicating that research of a similar goal was being conducted around the time of the ATP project. Given the fact that it can take many years between the time a research project takes place and the time that an outcome is realized, this time period for a patent search allowed us to determine whether there was research ongoing during the time of the ATP project. The dates of the patents actually occurred sometime after the research was conducted. And, as we reported in a prior report, the time between the point when a patent application is filed until the date when a patent is issued, or the application is abandoned, ranged from 19.8 months to 21 months, adding additional time to when the research was done.

ATP Project on Capacity Expansion of Fiber Optic Cables

Another ATP project we reviewed, which proposed to develop a system to increase the capacity of existing fiber optic cables for the telecommunications industry, also had a similar goal to that of research in the private sector. At the same time, firms in the private sector were attempting to increase the number of light signals that can be transmitted through a single strand of fiber optic cable using a technology called wavelength division multiplexing (WDM). In the 1980s, telephone companies laid fiber optic cables across the United States and other countries to create an information system that could carry significantly more data than the copper wires they replaced. Tremendous increases in cable traffic, primarily from the Internet, have crowded these cables. WDM technology was aimed at providing a cost-effective alternative to the expensive option of installing additional fiber optic cables.

Accuwave Corporation (Accuwave) was the ATP award recipient. Accuwave used its ATP funding of approximately $2 million from March 1993 through March 1995 to develop a wavelength division multiplexing system that would substantially increase the number of signals that could be transmitted through a single optical fiber strand, using the concept of volume holography. Volume holography uses holograms to direct multiple light signals simultaneously through a single fiber strand. Accuwave was able to make improvements on these issues but not enough to fully develop and market a successful WDM system for the telecommunications market. In 1996, a competitor beat Accuwave to the market. After the completion of the ATP project, Accuwave filed for bankruptcy protection due to its inability to successfully commercialize a wavelength division multiplexing system.

Other private firms were involved in research with a similar goal of increasing the capacity of fiber optic cable at about the same time as Accuwave was conducting its research. Conceptual research on such systems dates back to the early 1980s, but development and commercialization did not flourish until the mid- to late-1990s. Bell Labs (now Lucent Technologies), Nortel Networks, and Ciena Corporation, among others, were considered some of the major competitors in the industry. In the early 1990s, these firms were attempting to develop WDM technology using different methods and materials. For example, Ciena Corporation developed a system that incorporated fiber-Bragg gratings, which are filters embedded directly onto fiber optic cable that help to separate multiple light signals through a single fiber strand.

We also found an indication of WDM-related research through a review of issued patents. According to PTO’s database, over 2,000 patents were issued related to wavelength division multiplexing components, systems, and concepts from 1985 through 1999. The patents issued ranged from 10 patents in 1985 to 493 in 1999.

ATP Project on Regenerating Tissues and Organs

Both the ATP project and private sector projects we identified in the tissue engineering field had similar broad research goals of developing biological equivalents for defective tissues and organs utilizing diverse technical approaches. ATP’s project proposed procedures for extracting, storing, spinning, and weaving collagen (the main constituent of connective tissue and bones) into fibers suitable for human prostheses that could induce the body’s cells to regenerate lost tissue. Tissue Engineering, Inc., received ATP’s award of about $2 million for use over the years 1993 through 1996. The company’s long-term and yet unrealized goal is to transplant these prostheses into humans, after which the collagen framework, or scaffold, would induce the growth and function of normal body cells within it, eventually remodeling lost human tissue and replacing the scaffold.

Within the very innovative field of tissue engineering, however, many competitors were attempting to achieve similar broad research goals. Organogenesis, the Collagen Corporation, Integra LifeSciences, Advanced Tissue Sciences, Genzyme Tissue, Osiris Therapeutics, Matrix Pharmaceuticals, and ReGen Biologics are key players in the market to develop structures that could replace or regenerate cells, tissues, and organs such as skin, teeth, orthopedic structures, cartilage, and valves. A number of these companies have subsequently received ATP awards. In addition, universities and medical schools have researchers investigating the many possibilities to engineer human tissues, and eventually complex organs, such as the liver, pancreas, and heart. According to one expert, there is a great deal of competition within the field of tissue engineering.

Although the Tissue Engineering, Inc. research focused on the use of collagen as the basis for these structures, other companies were pursuing a variety of technical approaches for addressing the goal of developing biological equivalents for defective tissues and organs. In addition to research in collagen, other companies and researchers have also been attempting to create human tissues and organs from other biological materials, synthetics, and hybrid products, which are both biologic and synthetic. For example, researchers from the Massachusetts Institute of Technology (MIT) developed an artificial skin product using collagen and a natural polymer. Several companies have since developed comparable products. In 1986, researchers from MIT and a hospital in Massachusetts began inserting cells into scaffolds created of biodegradable polymer. As the cells multiply, tissues form. The magazine BusinessWeek reported this concept as “an elegantly simple concept that underlies most engineered tissue.” Two competitors, Integra LifeSciences and Organogenesis, reported that they were also doing work on the use of collagen in various applications. Although their technical approaches were different than the ATP project, the broad research goals were similar.

In addition to our discussions with experts and literature searches, patent research shows that there was activity related to the field of tissue engineering prior to and during the ATP project. According to a search done on the PTO website, at least 370 patents were issued related to cell culturing, scaffolding or matrix development, and tissue engineering from 1985 through 1999. Experts have also indicated that there are several patents related to the field, with a considerable amount of overlap in the technologies described in those patents.

ATP’s Award Selection Process Is Unlikely to Avoid Funding Similar Research

Two factors in ATP’s award selection process could result in ATP’s funding research similar to research that the private sector would fund in the same time period. These two factors are inherent in the review process and limit the information the reviewers have on similar private sector research efforts. Due to conflict-of-interest concerns, technical reviewers are precluded from being directly involved with the proposed research, making them less likely to know about all the research in an area. Also, the information available about private sector research is limited because of the private sector practice of not disclosing research results. Until a patent is issued, a private sector firm generally publishes very few details about the research to protect proprietary information. Therefore, it is difficult for the reviewers to identify other cutting-edge research.

ATP’s Conflict-of-Interest Provision Limits Its Ability to Identify Similar Research

ATP selection officials rely on outside technical reviewers to evaluate a proposal’s scientific and technical merit. All reviewers must certify that they have no conflicts of interest. To minimize possible conflicts of interest, the technical reviewers are generally federal government employees who are experts in the specific technology of the research proposal but are not directly involved with the proposed research area. Although this approach helps to guard against conflict of interest, it has inherent limitations on the program’s ability to identify similar research efforts. The technical reviewers rely on their own knowledge of research underway in the private sector. One of the technical reviewers we interviewed said that he did not personally know of other companies that were doing similar work. However, he believed that it was unlikely that there were not dozens of others working on the same issue.

Proprietary Information Limits ATP’s Ability to Identify Similar Research ATP reviewers are significantly limited in their ability to identify similar research efforts by an inherent lack of information on private sector research. Although ATP officials use several sources, such as colleagues, conferences and symposia, and current technical literature, to try to identify research efforts conducted by the private sector and the federal government, this information is often proprietary. Most of the private sector and university experts we consulted agreed that it can be very difficult to identify the specific research that private sector firms are conducting, especially considering the competitive nature of most industries. The early release of information on a company’s research could be costly to the firm. If a competing firm could determine the nature and progress of another company’s research, it could help the competitor to develop and commercialize an identical or higher-quality product before the other firm. At the very least, the early release of research information by a firm can give competitors an idea as to the focus of the firm’s strategic plan. Thus, many firms are very careful about releasing detailed information related to research and development activities they are conducting.

In conclusion, Mr. Chairman, the process ATP follows to select projects for funding is limited in its ability to identify similar research efforts in the private sector. Our retrospective look at the three ATP research projects showed that their goals were similar to research goals already being funded by the private sector. Examining the process that ATP uses to select projects, we found two inherent factors—the need to guard against conflicts of interest and the need to protect proprietary information—that limit ATP’s ability to identify similar research efforts in the private sector. These two factors have not changed since the beginning of the program. We recognize the valid need to guard against conflicts of interest and to protect proprietary information; thus, we did not recommend any changes to the award selection process. However, we believe that it may be impossible for the program to ensure that it is consistently not funding existing or planned research that would be conducted in the same time period in the absence of ATP financial assistance.

Mr. Chairman, this concludes my prepared statement. I would be happy to respond to any questions that you or Members of the Subcommittee may have.

Contacts and Acknowledgements

For further information about this testimony, please contact Robin M. Nazzaro at 202-512-6246. Diane Raynes, Carol Herrnstadt Shulman, and Jessica Evans made key contributions to this statement.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.