October 1999

National Institute of Standards and Technology Technology Administration U.S. Department of Commerce

Business Planning and Progress of 

Small Firms Engaged in

Technology Development Through

the Advanced Technology Program

ADVANCED TECHNOLOGY PROGRAM

0

50

100

150

200

250

300

350

Millions of Dollars

35%

25%

20%

5%

15%

NISTIR 6375
UNITED STATES OF AMERICA

DEPARTMENT OF COMMERCE

U.S. DEPARTMENT OF COMMERCE Economic Assessment Office

Technology Administration Advanced Technology Program

National Institute of Standards and Technology Gaithersburg, Maryland 20899

NISTIR 6375

Business Planning and Progress of 

Small Firms Engaged in Technology

Development Through the Advanced

Technology Program

Jeanne W. Powell October 1999

U.S. DEPARTMENT OF COMMERCE

William M. Daley, Secretary

TECHNOLOGY ADMINISTRATION

Gary R. Bachula, Acting Under Secretary

of Technology

NATIONAL INSTITUTE OF STANDARDS

AND TECHNOLOGY

Raymond G. Kammer, Director
ii Abstract

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

ABSTRACT

This report assesses participation, strategies, and progress of firms engaged in technology development

through partnerships with the Advanced Technology Program (ATP). The focus is on the performance

of small firms (those with fewer than 500 employees) compared with medium-to-large firms (grouped

together and referred to as “larger” firms in this report, and defined as firms with 500 or more

employees). Using 1997 data collected through the ATP’s Business Reporting System (BRS) from

firms receiving ATP awards in 1993 through 1996, the report identifies factors contributing to the

success of small firms in ATP competitions; and it addresses whether small firms are adopting effective

“strategies for success,” using ATP funding to accelerate R&D cycles and attract increased private

funding, and overcoming at least the initial hurdles to commercial success and broad diffusion of their

technologies. For purposes of comparison, parallel analyses are presented for larger firms funded by

the ATP during the same period. Whether or not the firm is a participant in a joint-venture project is

examined as a source of variation in the results for small firms as compared with larger firms. 

The study finds that small companies have been successful in winning ATP awards in competition with

larger firms. Their success likely relates to their ambitious plans for numerous applications, including

many “new-to-the-world” and high-performance products. Small firms also appear to be following

strategies for success cited in the literature. While collaboration activity is widespread across both

small firms and larger firms, small firms are placing more emphasis on strategic alliances with

customers and other supply chain linkages than larger firms and more frequently report significant

effects of collaboration on identifying customer needs, and on raising capital. Most small firms as well

as larger firms have experienced acceleration of their R&D in the critical race to market as a result of

the increased levels of investment and of collaborative activity with other organizations. On the other

hand, larger firms more frequently report that they require ATP funding to undertake their projects at

all. At similar stages of their projects, a larger percentage of small firms have set up pilot production,

begun production, and/or have earned revenues from their ATP-funded technologies. Small firms in

single-company projects are taking advantage of early opportunities for revenues from spin-off

activities to support cash flow. Small firms in joint ventures appear to be making particularly strong

progress towards actual production of products incorporating ATP-funded technology. Larger firms

are more likely than small firms to be pursuing new or improved process technologies. Evidence to

date, at a time when most of the firms are still in the R&D phase, suggests that small firms funded by

the ATP are progressing at a pace at least equivalent to larger firms and engaging in collaborative and

other commercialization activities needed for business success and economic impact. Whether small

firms will ultimately be as successful as larger firms in gaining adoption of their technologies and

achieving broad economic impact will likely not be known until many years after their period of ATP

funding ends.

KEYWORDS: acceleration; Advanced Technology Program; collaboration; commercialization;

program evaluation; small firms; technology development.
Acknowledgments iii

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

ACKNOWLEDGMENTS

This report reflects the contributions of the hundreds of ATP-funded organizations that have provided

reports of business planning and progress related to their ATP projects through the ATP’s Business

Reporting System (BRS). Statistical analysis of data collected across the portfolio of ATP-funded

organizations has been invaluable to ATP program evaluation efforts such as this one and to meeting

ATP’s responsibilities under the Government Performance and Results Act. At the project level, data

reported by the individual project participants are also useful internally, on a confidential basis, for

project management. 

Many individuals across the ATP and NIST deserve recognition for their roles in the BRS data

collection effort and in the work performed for this report. Stephen F. Weber, Building and Fire

Research Laboratory, designed the original BRS platform and developed the original BRS software and

its subsequent revisions. Karen Lellock conducts the day-to-day data collection activities related to the

BRS, including follow-up on late respondents and error detection. Karen extracted the data sets used

in the analyses included in this report from the broader BRS database and compiled the majority of the

summary statistics. Holly Jackson compiled the patent information and performed the Z statistical tests

of the mean where quantitative data was available. Jo Ann Brooks, graphic artist, prepared the

graphical illustrations. Maryellen Kelley and Robert Sienkiewicz provided comments and suggestions

concerning the earlier conference paper from which this report was derived. Many of their suggestions

were incorporated in the extended analyses presented in this report. Rosalie Ruegg provided general

guidance along with insightful and detailed reviews. 

Thanks are also due to the other reviewers within the ATP and NIST for their many helpful comments:

Cita Furlani, Acting Deputy Director of the ATP, and Stephen F. Weber, Economist, Building and Fire

Research Laboratory. 
iv Preface

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

PREFACE

The Advanced Technology Program (ATP), administered by the U.S. Department of Commerce’s

National Institute of Standards and Technology, partners with U.S. industry to fund the development

of high risk and enabling technologies, with significant potential for stimulating U.S. economic growth.

Through a highly competitive peer review process, projects with strong technical and

business/economic merit are selected for funding. The ATP shares with industry part of the research

costs; the companies are responsible for the rest, as well as for all business planning, product

development, and other future commercialization costs. 

From its inception in 1990 through 1998, the ATP has held 39 competitions and announced 431 awards,

a funding commitment of nearly $1.4B. The awards involve about 1,000 participating organizations,

including for-profit companies of all sizes, universities, state and federal laboratories, and other nonprofit organizations. Although the ATP has no “set-aside” budget designated for small businesses,

small businesses have been highly successful in ATP’s selection process. Of the total 431awards to

date, 190 awards were to small companies (companies with fewer than 500 employees) proposing

single-company projects, and 48 awards were to joint venture/consortia led by small businesses. Taking

into account other small-firm members of joint ventures, a total of over 400 small businesses participate

directly in ATP projects. In addition, hundreds of small businesses are subcontractors to the “directly

participating” companies. 

This report expands upon a research paper entitled “Small-Firm Experience in the Advanced

Technology Program” presented at the High-Technology Small Firms Conference, University of

Twente, the Netherlands, June 4-5, 1998. The earlier paper compared experiences of ATP-funded small

firms with the full portfolio of organizations funded in the same time period. The current report directly

contrasts small firms (those with fewer than 500 employees) with medium-to-large firms (grouped

together and referred to as “larger” firms in this report, and defined here as firms with 500 or more

employees). It uses test statistics to illustrate the strength of differences between small firms and larger

firms. It further expands the analysis to identify differences between firms receiving single company

awards and those receiving joint venture awards. 

This report is offered as part of ATP’s on-going evaluation program. To provide feedback to ATP

management and to meet the many outside requests for program performance information, the ATP’s

Economic Assessment Office (EAO) conducts a multi-component evaluation program. Other studies

by EAO staff and outside contractors include detailed case studies of ATP-funded projects, statistical

and econometric studies, status reports on the accomplishments of completed projects, methodological

studies, and other topics of interest to the evaluation community. For copies of EAO reports, see

“publications” at the EAO Web site (http://www.eao.atp.nist.gov), or contact the EAO directly at (301)

975-4332.
Executive Summary v

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

EXECUTIVE SUMMARY

This report provides a statistical overview of participation, strategies, and progress of small firms (with

fewer than 500 employees) compared with medium-to-large firms (grouped together and referred to as

“larger” firms in this report, and defined here as firms with 500 or more employees) engaged in

technology development. The report is based on data collected in 1997 from 401 firms in 210 projects

funded by the Advanced Technology Program (ATP) during the period 1993 through 1996. The

emphasis in this report is on the firm rather than the project, particularly as concerns joint ventures. 

The report represents a snapshot comparison of small firms with larger firms at a point in time, but at

various stages of their technology development projects. Some of the ATP-funded projects in the data

set have just begun, while others have recently been completed. Business plans and progress evolve

continually over the course of the research projects, as well as subsequently. The report focuses on the

unique challenges faced by small firms engaged in technology development and commercialization, and

documents the experiences of small firms that have received partial government funding for technology

development projects. 

Progress reports from individual firms and aggregate summary statistics provide some answers to the

following questions: a) Have small firms been able to compete successfully against larger firms for ATP

awards? Why? b) Are small-firm participants in ATP adopting effective commercialization strategies

for achieving the business and economic goals of their projects? c) What effect does ATP funding have

on the R&D being performed and on the participating firms? and d) Even though most projects are still

in the R&D phase, are there signs of commercial success and economic impact from small-firm

participation in the ATP?

Have small firms been able to compete successfully against larger firms for ATP awards? Why?

Small firms have been very successful in winning ATP awards. Across the portfolio of 431 ATP

awards to date, 190 have gone to small firms in single-company projects, and 48 have gone to joint

ventures led by small firms, adding up to more than half the awards. Forty percent of all participants

in ATP awards to date (excluding subcontractors) have been small businesses. Taking into account

subcontractors would tip the majority of participants to small businesses. 

Included in the analyses presented in this report are 167 small firms and 234 larger firms.

-Many ATP-funded small firms are very small. Twenty percent of the small firms covered in

this report, including a number of start-up companies, have fewer than 10 employees; 40% have

fewer than 20 employees; only 24% have as many as 100 employees. The smallest firms are

more likely to be in single-company projects than in joint-venture projects.

-The majority of ATP-funded “larger firms” (with 500 or more employees) are quite large.

Sixty-one percent of those covered in this report are Fortune 500 firms or equivalent in size to

the Fortune 500 firms. The larger firms, whether medium-size or Fortune 500 firms, are more

likely to be in formal ATP joint ventures than in single-company projects.
vi Executive Summary

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

An analysis of the goals of small firms for products and processes anticipated to result from their ATPfunded technologies, compared with those of larger firms, helps explain the success of small firms in

ATP competitions. The challenging nature of the goals set by small firms is a major factor in their

success in terms of meeting both technical and economic project selection criteria. 

-Small firms as a group envision more applications (uses) of their ATP-funded technology and

are engaged in planning for a larger number of applications than are larger firms. The number

of potential applications of a technology is significant because it supports the idea that the

technology is “enabling” (or “generic”) in nature. The ATP seeks to fund enabling (or generic)

technologies. Furthermore, small-firm applications are more likely to entail either “new-to-theworld” solutions where there is essentially no product or service based on the same technology

currently in the market, or highly ambitious performance improvement goals or cost

improvement goals. “New-to-the-world” products and major performance or cost

improvements imply cutting edge technologies rather than incremental changes in an existing

technology. 

-“New-to-the-world” solutions are more commonly reported by small firms or larger firms in

single-company projects than by firms of either size in joint ventures, and small firms in singlecompany projects appear to have particularly ambitious product performance goals. 

-Larger firms in single-company projects report comparable percentages of applications for

“new-to-the-world” solutions as smaller firms, but for other applications the magnitude of their

performance and cost goals appears somewhat less ambitious than for small firms. This may

be attributable to a greater emphasis by larger firms on advances needed to enable production

in high-volume and at lower costs of existing products. Larger firms in joint ventures may be

expected on average to have less ambitious goals for their own participation in the ATP project

because they sometimes play a support role, for example, testing and customer validation. In

such cases, other firms in the joint ventures are performing the more technically challenging

R&D tasks and building the enabling product components and complex integrated systems. For

the projects taken as a whole, the goals of the joint-venture projects may well be as or more

ambitious than those of the single-company projects.

Perhaps the ambitious project goals small firms tend to set for themselves are to some extent reflective

of “youthful eagerness” and inexperience rather than realistic expectations. Answers to subsequent

research questions addressed in the report cast some light on this issue, but the ultimate answer likely

lies in future observations of actual outcomes.

Are small-firm participants in ATP projects adopting effective commercialization strategies for

achieving the business and economic goals of their projects?

ATP-funded small firms appear to exhibit characteristics of successful small firms described in the

business literature. For example, their emphasis on “new-to-the-world” solutions and aggressive

performance goals, and their pursuit of large numbers of applications, indicate that they are risk takers

and that they are pursuing quality and innovativeness, and emphasizing organizational flexibility, rather
Executive Summary vii

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

than improvements in cost, price, and economies of scale. 

Many “strategies for success” cited in the literature appear to have been adopted by ATP-funded small

firms. The frequent mix of plans to produce a product in-house and also to license their technology to

others suggests that small firms are pursuing their competitive advantage in selecting the specific

application(s) for which they will seek to develop their own manufacturing capability, and that they tend

towards realism about the breadth of markets into which they can supply their own products given their

limited resources. Many appear to realize the need for supply-chain linkages with other firms for at

least some applications and are already benefitting from such collaborations.

-Licensing of the ATP-funded technology to others is significantly more common a

commercialization strategy for small firms than for larger firms, and is reported as a primary

strategy for 38% of small-firm applications, compared with only 17% of larger-firm

applications. It is a particularly important strategy for small firms in single-company projects,

53% of whose applications will entail licensing as a primary commercialization strategy.

Licensing allows small firms to produce a product in-house while also providing its customers

with the security of a second source. This may be particularly important for firms establishing

their very first manufacturing capability and entering markets where their customers anticipate

significant and rapid scale-up in volume. Licensing their technology also allows firms to diffuse

their technology to a broader spectrum of markets than they could supply competitively with

their own manufactured products, while increasing their income stream. 

-Production of a product in-house is the most common primary strategy for both small firms and

larger firms, with 62% of small-firm applications and 66% of larger-firm applications involving

plans for production of a product in-house. Larger firms in single-company projects plan to

produce a product or service in-house for 73% of applications, and plan to license their

technology to other firms for only 26% of applications. Larger firms have less need than small

firms to transfer their technology to a second source supplier in their own markets, but like

small firms, many larger firms appear to see opportunities to pursue broader markets through

licensing. 

-Collaborative R&D activity has been vigorous for firms of all sizes. About two-thirds of small

firms and of larger firms in single-company projects report collaborative activity. Many small

firms, in particular, reported benefits from collaboration in forming relationships needed for

business success, such as identification of customer needs and attraction of capital investment.

Both small firms and larger firms commonly reported that their collaborations were effective

in stimulating creative thinking, obtaining R&D expertise, and accelerating market entry. On

the negative side, small firms in joint ventures appear more sensitive to increased management

and coordination costs that have resulted from their collaborations than larger firms. Thirteen

percent of small firms in joint ventures reported a significant increase in management and

coordination costs as a result of their joint-venture collaboration. By contrast, only seven

percent of larger firms in joint ventures, seven percent of small firms in single-company

projects, and three percent of larger firms in single-company projects reported negative effects

of collaboration.
viii Executive Summary

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

What effect does ATP funding have on R&D being performed and on the participating firms?

Small technology-based firms face a trade-off between the time required to undertake technically

challenging R&D aimed at highly innovative products and the race to market new products. Many are

start-ups, with no existing customer base. They seek ways to overcome possible barriers to customer

acceptance of innovative “new-to-the-world” solutions that require a change in existing ways of doing

things. ATP funding helps firms accelerate their R&D efforts, increasing the chances that resulting new

products will reach the market within the critical window of opportunity. ATP funding appears to

improve the credibility of small firms with potential investors and new customers. At the same time,

ATP funding appears to strengthen the interest in and commitment of firms of all sizes to undertaking

more challenging, broader-scope R&D efforts than they would undertake on their own.

-A large percentage of firms of all sizes reported that ATP funding had accelerated their R&D.

Firms in single-company projects indicated they were “Ahead” more frequently than those in

joint ventures. (Ninety-seven percent of small firms in single-company projects and 95% of

larger firms in single-company projects indicated they were “Ahead”, compared with 81% of

both small and larger firms in joint ventures.) Among those reporting they were “Ahead,” larger

firms reported more frequently than small firms that they would not have pursued the project

at all without ATP funding, with larger firms in single-company projects reporting this most

often.

-ATP funding has enabled firms to undertake R&D projects that are more challenging and

ambitious in scope than they could undertake on their own, rather than “more of what they

would have done anyway.” ATP funding appears to have been particularly effective in

encouraging larger firms in single-company projects to engage in more challenging R&D than

they would normally undertake, with 89% indicating the level of technical challenge they would

bear had increased as a result of ATP funding. 

-Both small firms and larger firms appear to have increased the amounts of their own investment

from internal/industry funds in R&D related to the ATP-funded technology area (above and

beyond ATP funding and what they would have invested without ATP), with small firms

reporting a greater increase on average than larger ones.

-The ATP award appears to have had an important “halo” effect for small firms in establishing

credibility with investors, customers, and suppliers. Increased credibility with customers is

most common for small firms in joint ventures. Increased credibility with investors is most

common for small firms in single-company projects. Increased credibility with management

(e.g., credibility of the technology being developed, and/or credibility of the principal

investigator responsible for the ATP proposal, with management) is most common for larger

firms, although nearly as large a percentage of small firms reported it, perhaps indicating

increased credibility of the technology with the company owner and/or with investors.
Executive Summary ix

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Are there signs of commercial success and economic impact from small-firm participation in the ATP?

Assessed after comparable periods of ATP funding, while still in the R&D phase and not yet having

fully achieved the technical goals of their projects, small firms appeared to be keeping pace with larger

firms or were slightly ahead in early stages of commercialization. 

-After comparable periods of ATP funding, small firms had made about the same or a greater

level of progress as larger firms in producing production prototypes, setting up pilot production,

beginning production, and actually earning revenues. Twenty-six percent of small firms

reported actual revenues earned, compared with 11% of larger firms.

-Small firms in joint ventures had progressed further than those in single-company projects and

further than larger firms in either type of project against a number of commercialization

milestones. For example, more small firms in joint ventures had developed production

prototypes for at least one application, had adopted process improvements, had set up pilot

production or a commercial demonstration, and had started production for at least one

application. 

-Small firms in single-company projects were ahead of those in joint ventures in terms of

actually reporting revenues, even though these early revenues more often derived from the sale

of prototypes or early spin-off products, than from actual production in a major application. The

data suggest that small firms (many of which are very small start-ups) respond to the need to

generate cash flow at the earliest possible time by seeking opportunities to gain name

recognition and by building customer relationships at relatively early stages of technology

development. They then build on these relationships as their technologies develop further and

achieve their full potential.

-Small firms are taking the necessary steps to protect intellectual property and establish the basis

for licensing their technology despite the high cost of filing for patents. Measured after two and

after three years of ATP funding, a larger percentage of small firms than larger firms had filed

for at least one patent on their ATP-funded technology. Across all firms included in the study,

39% of small firms and 31% of larger firms had filed for at least one patent.

The rapid progress towards commercialization of small firms in joint ventures found in this study merits

closer scrutiny, and needs to be observed over time. It may be that this group of small firms, on

average, has the advantage of a somewhat larger size and greater experience than the small firms in

single-company projects. In addition, the collaborative efficiencies specific to formal joint ventures

may be providing a significant advantage to the small companies in those joint ventures. Further

research is needed to probe this hypothesis and also to examine the varied roles and progress of larger

companies in joint ventures more closely. 

Taken as a whole, the results in this report indicate that to date small firms have been successful in

winning ATP awards to develop technologies underlying a large number of “new-to-the-world” or high

performance products. These companies are using a mix of in-house production and licensing
x Executive Summary

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

strategies, have taken advantage of collaborations to achieve these goals, and are vigorously engaged

in plans and activities required for successful commercialization. Although only a few of the projects

have been completed, some of the innovative new technologies are already under commercialization.

Most require substantial additional investment after ATP funding ends to achieve their potential. With

additional time, ATP can determine whether its small firms, in single-company projects or in formal

ATP joint ventures, will fully achieve their ambitious R&D and commercial goals and achieve broad

impact through their myriad of planned applications.
Table of Contents xi

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

TABLE OF CONTENTS

ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii

PREFACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

EXECUTIVE SUMMARY ....................................................... v

TABLE OF CONTENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

1. INTRODUCTION ............................................................ 1

2. LITERATURE REVIEW OF ISSUES IMPORTANT TO ATP-FUNDED SMALL FIRMS

............................................................................... 3

2.1 Advantages of Small Firms ............................................... 3

2.2 Strategies for Success of Small Firms ...................................... 3

2.3 The Trade-off Between Time-to-Innovate and Time-to-Market .................. 4

2.4 Financing R&D and the Challenge of Raising Capital .......................... 4

2.5 The Role of Collaboration and Strategic Alliances ............................ 5

3. THE DATA AND METHODOLOGY ........................................... 6

3.1 The Role of Small Firms in the ATP’s Award Portfolio ........................ 6

3.2 The Distribution of Firms Included in Study – by Firm Size and Project Type ....... 6

3.3 The Stage of Project Completion of Firms Included in This Study – by Firm Size and

Project Type ........................................................ 8

3.4 Supplementary Analyses ................................................. 9

4. PROJECT GOALS AND EXPECTED COMMERCIAL ADVANTAGE . . . . . . . . . . . . . 11

4.1 Number of Commercial Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.2 Project Goals and Expected Commercial Advantage . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.2.1 “New-to-the-world” Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.2.2 Performance and Cost Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4.2.3 Process Versus Product Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

5. STRATEGIES FOR COMMERCIALIZATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5.1 Technology Licensing and/or In-house Production . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5.2 Adoption of New Manufacturing Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

5.3 Strategic Alliances Across the Supply Chain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
xii

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

6. COLLABORATIVE R&D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6.1 Collaboration Activity in ATP Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6.2 ATP Responsibility for Collaboration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

6.3 Effects of Collaboration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

6.4 Anecdotal Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

6.4.1 Small Firms in Single-Company Projects: . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

6.4.2 Small Firms in Joint-Venture Projects: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

6.4.3 Larger Firms in Single-Company Projects . . . . . . . . . . . . . . . . . . . . . . . . . . 29

6.4.4 Larger Firms in Joint-Venture Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

7. EFFECTS OF ATP FUNDING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

7.1 Acceleration of R&D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

7.2 Change in Scope, Level of Risk, or Ability To Take on Long-Term Research . . . . . . 34

7.3 Change in Level of Industry Investment from Internal Funds . . . . . . . . . . . . . . . . . . . 35

7.3.1 Direction of Effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

7.3.2 Estimated Magnitude of Effect on Industry Investment . . . . . . . . . . . . . . . . 36

7.3.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

7.4 Increased Credibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

7.5 Anecdotal Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

7.5.1 Small Firms in Single-Company Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

7.5.2 Small Firms in Joint-Venture Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

7.5.3 Larger Firms in Single-Company Projects . . . . . . . . . . . . . . . . . . . . . . . . . . 42

7.5.4 Larger Firms in Joint-Venture Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

8. PROGRESS TOWARDS COMMERCIALIZATION OF ATP-FUNDED

TECHNOLOGIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

8.1 Progress of Firms Against Some Early Commercialization Milestones . . . . . . . . . . . . 44

8.1.1 Progress of Small Firms Compared with Larger Firms . . . . . . . . . . . . . . . . 44

8.1.2 Progress of Small Firms and Larger Firms in Single-Company Projects

Compared with Those in Joint Ventures . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

8.2 Progress in Obtaining Patents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

9. SUMMARY OF FINDINGS AND FUTURE WORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

9.1 Summary of Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

9.2 Implications for Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

APPENDIX A. Z Test Statistics for Differences in Population Proportions . . . . . . . . . . . . . . 55

APPENDIX B. Z Test Statistics for Differences in Population Means . . . . . . . . . . . . . . . . . . 57
Table of Contents xiii

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

LIST OF FIGURES

Figure 1. SIZES OF FIRMS AND TYPES OF PROJECTS

Small firms (<500 employees) and larger firms (> 500 employees) included in the study

........................................................................ 7

Figure 2. PROJECT GOALS AND EXPECTED COMMERCIAL ADVANTAGE

Comparison of small firms (<500 employees) with larger firms (> 500 employees) with

commercialization plans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Figure 3. STRATEGIES FOR COMMERCIALIZATION

Comparison of small firms (<500 employees) with larger firms (> 500 employees) with

commercialization plans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Figure 4. COLLABORATION EXPERIENCE IN ATP PROJECTS

Comparison of small firms (<500 employees) with larger firms (> 500 employees) . . . . . 23

Figure 5. HAS COLLABORATION BEEN EFFECTIVE?

Comparison of small firms (< 500 employees) with larger firms (> 500 employees) reporting

collaboration experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Figure 6. EFFECTS OF ATP FUNDING

Comparison of small firms (<500 employees) with larger firms (> 500 employees) . . . . . 32

Figure 7. INCREASED CREDIBILITY

Comparison of small firms (<500 employees) with larger firms (> 500 employees) . . . . . 39

Figure 8. PROGRESS TOWARDS COMMERCIALIZATION

Comparison of small firms (<500 employees) with larger firms (> 500 employees) after two

years of ATP funding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

LIST OF TABLES

Table 1. Stage of Project Completion

For Small Firms (<500 employees) and Larger Firms (> 500 employees) Included in the

Study ................................................................... 9

Table 2. Planned Commercial Applications of the ATP-Funded Technologies–by Type of

Project 

Small Firms (<500 employees) and Larger Firms (> 500 employees) . . . . . . . . . . . . . . . . 11

Table 3. Project Goals and Expected Commercial Advantage–by Type of Project

Small firms (<500 employees) and larger firms (> 500 employees) . . . . . . . . . . . . . . . . . 14

Table 4. Strategies for Commercialization–by Type of Project

Small firms (<500 employees) and larger firms (> 500 employees) . . . . . . . . . . . . . . . . . 19

Table 5. Collaboration Experience–by Project Type

Percentage of firms for small firms (<500 employees) and larger firms (> 500 employees)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Table 6. Effects of ATP Funding–by Type of Project

Percentage of firms for small firms (<500 employees) and larger firms (>500 employees)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
xiv

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Table 7. Increased Credibility–by Project Type 

Percentage of firms for small firms (< 500 employees) and larger firms (> 500 employees)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Table 8. Progress Towards Commercialization–by Project Type

Small firms (< 500 employees) and larger firms (> 500 employees) . . . . . . . . . . . . . . . . . 47

Table 9. Patents Filed 

Small firms (<500 employees) and larger firms (> 500 employees) . . . . . . . . . . . . . . . . . . 48
1. Introduction 1

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

1. INTRODUCTION

This report assesses the participation, strategies, and progress of small firms compared with larger firms

engaged in technology development through partnerships with the Advanced Technology Program

(ATP). The ATP partners with U.S. industry to fund the development of high risk and enabling

technologies, with significant potential for stimulating U.S. economic growth. The ATP is administered

by the National Institute of Standards and Technology (NIST), within the U.S. Department of

Commerce. 

The ATP seeks to accelerate the development and commercialization of new, and highly innovative

technologies with strong promise for expanding U.S. economic growth. Selection criteria include not

only high technical merit but also the potential for broad economic benefits to the nation, evidence of

the need for federal funding, and a credible plan for achieving the national economic benefits described.

Members of small firms, typically with limited resources and business experience, often ask whether

they can be successful in competition with larger firms against the ATP’s broad selection criteria and

demanding selection process. Others have questioned whether small firms which have produced awardwinning proposals will be able to execute relatively large-scale technology development projects on a

timely basis and ultimately commercialize the technologies and generate broad economic benefits. 

After ten years of experience and a substantial history of awards to a mix of small firms and larger

firms, it seems an appropriate time for the ATP to address whether small firms have been successful

in winning ATP awards, pursuing their R&D plans, developing effective business strategies, and

overcoming at least the initial hurdles to commercial success and broad diffusion of their technologies.

Using data collected in 1997 through the ATP’s Business Reporting System (BRS), the study seeks

answers to the following research questions: 

1) Have small firms been able to compete successfully against larger firms in ATP

competitions? Why?

2) Are small-firm participants in ATP projects adopting effective commercialization strategies

for achieving the business and economic goals of their projects?

3) What effect does ATP funding have on R&D being performed and on the participating firms?

4) Even though most projects are still in the R&D phase, are there signs of commercial success

and economic impact from small-firm participation in the ATP?

For purposes of comparison, parallel analyses are presented for larger firms funded by the ATP during

the same period. Whether or not the firm is a participant in a joint-venture project is examined as a

source of variation in the results for small firms as compared with larger firms.

Consistent with the Small Business Administration’s classification of small technology-based firms,

“small firms” are defined as for-profit firms with fewer than 500 employees. For purposes of this study,
2 1. Introduction

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

“larger firms” are defined as all other for-profit firms; i.e., those with 500 or more employees. The

larger-firm group thus includes a) firms that ATP designates as large firms, defined as Fortune 500

firms or their equivalent in size, and b) medium-sized firms, including all other firms with 500 or more

employees. 

The report first reviews the current literature on small technology-based firms to understand both the

unique advantages and special challenges of small firms engaged in technology development and to

identify small-firm “strategies for success.” Then, using data collected through ATP’s Business

Reporting System, for firms funded in the period 1993-1996, the study addresses the research questions

against the context of small-firm characteristics and success strategies cited in the literature. 

This study provides the ATP with a better understanding of the unique roles, risks, and opportunities

for its small-firm participants and an assessment of small-firm plans and progress to date. Those

involved in international science and technology policy or evaluation may also benefit from learning

about the experience of small firms funded by the ATP. Both of these communities may further benefit

from learning more about the ATP’s experience in tracking early business progress and economic

impact of government cost-shared R&D through the technology development stage into early phases

of commercialization, and in developing a comprehensive database as an evaluation tool. 
2. Literature Review of Issues Important to ATP-Funded Small Firms 3

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

2. LITERATURE REVIEW OF ISSUES IMPORTANT TO ATP-FUNDED

SMALL FIRMS

A number of issues of relevance to the ATP and related government S&T programs emerge from the

literature as it relates to identifying special advantages and challenges of small technology firms in the

marketplace. Although there is no single small-company strategy that is recommended by the many

studies, there are a set of “strategies for success” that are suggested by the literature.

2.1 Advantages of Small Firms

In the process of downsizing, breaking up, and shedding many of their operations, larger companies are

creating new opportunities for small businesses. Larger firms are outsourcing basic tasks, including

R&D previously performed in-house. Changes in the international marketplace include the possible

demise of large-scale production as it was practiced in the past, an increase in uncertainty and market

fragmentation, an explosion in products and services, an emphasis on customized products, superior

quality, and fast response time, and a de-emphasis on cost, price, and economies of scale [Lefebvre et

al., 1990; Phillips, 1991; Winger, 1994; Lawler, 1997]. Like many others, Winger [1994] observes that

“entrepreneurs are risk takers” and that smaller, less hierarchical firms have the organizational

flexibility and freedom to take more risks than larger ones on an aggressive time scale. 

2.2 Strategies for Success of Small Firms

Both Yap and Souder [1994] and Litvak [1992] identify a broad set of factors critical to a small firm’s

success. In their study of 12 entrepreneurial high-tech electronics firms, Yap and Souder found that 1)

products requiring a high degree of modification of customer behavior were negatively correlated with

success; 2) speed-to-market should not be achieved at the expense of innovativeness; 3) under

conditions of high market uncertainty, highlighting superior performance and uniqueness was

counterproductive, while highlighting high product compatibility fostered success; and 4) under

conditions of low market uncertainty and low technical uncertainty, comparison of the new product with

an existing one was a determinant of success provided the customer was not required to form a new

frame of reference. (In this study, “customers” were industrial purchasers of electronic components for

use in industrial or institutional applications.) Yap and Souder [1994] postulated that small firms were

not viewed by customers as influential enough (and market resources were not adequate) to create a new

class of product; therefore, their message was not deemed credible. The resulting prescription was rather

conservative: “Stay close to core technologies; seek niche opportunities, and don’t overextend.” Litvak

[1992] considered a broader set of issues. Among the “strategies for success” that seemed paramount,

he cited: 1) A global marketing niche strategy; 2) concentration on products for which competitive

advantage can be sustained; 3) integration of R&D and marketing strategies, and successful transition

from a research orientation to a product development and manufacturing orientation; 4) use of

government R&D grants and other government business for cash flow in tight periods and to enhance

credibility of the technology-based firm to potential investors; 5) patents for the purposes of

establishing a market niche and enhancing firm credibility in the eyes of potential investors; and 6) use

of horizontal or vertical strategic alliances to promote corporate growth.
4 2. Literature Review of Issues Important to ATP-Funded Small Firms

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

These studies raise a number of questions to be considered in this paper: Are ATP-funded small firms,

which as a group are willing to take technical risks required for innovation, also able to meet the

challenges on the business side? Do they exhibit the management and growth characteristics needed to

succeed in the international marketplace? For example, are they emphasizing superior quality, rapid

response, and multiple-product strategies rather than cost reduction and economies of scale? Are they

taking steps to maximize their commercial advantage and minimize the challenges to the successful

introduction of highly innovative products into the marketplace? Are they making effective use of

collaboration and strategic alliances? 

Cycle-time reduction, the trade-off between time-to-innovate and time-to-market, financing challenges,

and multi-firm collaboration are also recurring themes in the literature on factors in small business

success. These are areas where programs like the ATP may be able to contribute to the success of small

firms. 

2.3 The Trade-off Between Time-to-Innovate and Time-to-Market

The increasing emphasis on acceleration of R&D has been cited in numerous studies, including those

mentioned above. The research by Ali et al. [1995] explored the relationship among technical content,

innovation, cycle time, and breakeven time. These authors found that greater technical content and

innovation lengthen cycle time. Where innovative product advantage produced some incompatibility

with customers’ existing way of doing things, it seemed to increase breakeven time for the innovator.

The Ali et al. research poses an interesting question for the ATP and similar programs: By accelerating

the R&D process and catalyzing collaborative linkages of small firms with customers, do programs like

the ATP provide a mechanism for offsetting the negative effect on cycle time of developing innovative

technology and the negative effects on both cycle time and customer acceptance of developing highly

innovative, “new-to-the-world” products? 

 

2.4 Financing R&D and the Challenge of Raising Capital

Financing concerns limit the level of R&D and commercialization strategies of small firms.

Himmelberg and Peterson [1994] demonstrated that “because of capital market imperfections, the flow

of internal finance is the principal determinant of the rate at which small high-tech firms acquire

technology through R&D.” Gompers and Lerner [1999] investigated the availability of support for

entrepreneurial technology firms. They reported that although venture capitalists have mechanisms for

dealing with information problems in debt and equity markets, their funding is uneven, typically

concentrating on a few narrow technologies and a few geographic locations. They further noted that

individual investments by the angel community are typically very small compared with venture capital.

Piper and Lund [1996], researchers in the U.K., offer some insights for the U.S., as well, particularly

for angel investments. The small scale of the seed-capital financial sector and the reluctance to invest

in early-stage technology firms may be due to 1) perceived risks, 2) failures to understand the

technology, and 3) perceived low average rates of return. 

Winger [1994] noted that the U.S. Department of Defense’s procurement of high-tech products in the

1980’s served to reduce the risks and remove market uncertainties associated with new technology.
2. Literature Review of Issues Important to ATP-Funded Small Firms 5

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Unlike most U.S. DoD and other federal procurement programs, the ATP relies on industry for partial

funding of R&D (including full product development costs) and on the commercial marketplace for

customers. Although it takes a different approach than the defense and other mission-based programs,

the ATP’s documented, rigorous award process and cost sharing of technology development can

similarly serve to reduce technical risk. Its requirement for up-front integration across technical goals

and long-run goals for commercial applications also likely reduces market risk by bringing the technical

goals more in line with ultimate commercial requirements.

2.5 The Role of Collaboration and Strategic Alliances

Jonash [1996] expects 80% of technology-related growth in companies to derive from external

relationships of various types. Maynard [1996] states that “strategic alliances are quickly becoming a

necessity for firms across the spectrum of industries and for a wide variety of purposes.” On the other

hand, collaborations create substantial management complexity. The 10-year study performed by

Slowinski [1996] of 50 strategic alliances (21 in biotechnology, 21 in electronics, and eight in machine

vision) showed that only about 25% of alliances were considered successful by both partners; “50

percent didn’t meet stated objectives;” and the remainder were “disasters.” Nevertheless, collaborations

offer opportunities for tackling larger and more complex problems and realizing cost efficiencies

relative to single-firm solutions; and they may further encourage product innovation, bring financial

stability, expand product portfolios, forge new supplier relationships, and provide for risk sharing.

Small firms are finding that a strategic partner is often required for credibility with investors [Maynard,

1996]. Jonash [1996] reported that though “the jury is still out on the effectiveness, value, and risk of

increasing technology collaborations, there is clearly no retreat from the general trend.”

The ATP actively encourages collaborative relationships, with emphasis on appropriate small-firm

participation. Is there evidence that the ATP’s effort to catalyze collaborations is working for small

firms? Are ATP collaborations overcoming the typical difficulties and exhibiting some of the positive

characteristics cited above; for example, encouraging product innovation, bringing financial stability,

forging new supplier relationships, and encouraging risk sharing? Do collaborations appear to be

contributing to the success of ATP-funded small-firms?

These and other questions raised in this chapter are examined in subsequent chapters. Chapter 4 of this

report examines the project goals and the nature of the commercial advantage sought by small firms in

the ATP compared with larger firms, against the backdrop of advantages of small firms reported by

Winger [1997] and others. Chapter 5 examines the commercialization strategies of ATP-funded small

firms compared with larger firms against a number of the “strategies for success” cited by Yap and

Souder [1994], Litvak [1992], and others. Chapter 6 focuses on R&D collaborations and some of their

effects. Chapter 7 examines the effects of ATP funding in helping small firms, as well as larger firms,

address the issue raised by Ali et al. [1995] concerning the trade-off between R&D cycle time and

development of highly innovative new products. Chapter 7 also considers the role of ATP funding in

stimulating increased financing of high-risk R&D from industry sources. Finally, earlier chapters

having explored whether conditions for success identified in the literature appear to have been met,

Chapter 8 looks at early signs of commercial success and economic impact from small-firm

participation in ATP projects.
6 3. The Data and Methodology

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

3. THE DATA AND METHODOLOGY

3.1 The Role of Small Firms in the ATP’s Award Portfolio

Since receiving its first congressional appropriation of just $10 million in 1990, the Advanced

Technology Program has made 431 awards, amounting to $1.4 billion, with industry committing an

additional $1.4 billion in cost share. Among this portfolio, 190 small firms received awards for singlecompany projects, 48 small firms are the “lead” participant in joint venture awards. A total of 409 small

firms, representing 40% of all participants and 46% of for-profit company participants in the awards,

have either received an award directly as a single-company proposer or have been a participant in an

ATP joint venture. Many other small firms are involved in ATP projects through subcontractor

relationships with the awardees. Small firms quite clearly have been highly successful in winning ATP

awards despite concerns expressed by some small firms that they might not be able to compete for

funding against long-established, larger firms in the ATP’s rigorous proposal evaluation process. 

3.2 The Distribution of Firms Included in Study – by Firm Size and Project Type

This report is based on data collected through ATP’s Business Reporting System (BRS) from firms

receiving ATP awards in competitions conducted in 1993 through 1996. The ATP’s Business

Reporting System (BRS) is a comprehensive data collection tool for tracking, on a regular basis,

progress of projects in pursuing business plans and achieving anticipated economic benefits outlined

in the project proposals. Electronically administered, the survey system has been implemented for

projects funded since 1993, from their inception. (Powell [1997] provides a more detailed picture of

the structure and coverage of the BRS.) This report draws upon the BRS reports filed through

September 1997 from 401 firms in 210 ATP-funded projects initiated during the period 1993 through

1996. These reporting firms represent 99% of the projects funded during the period except those

canceled or terminated prematurely, 99% of single company awardees, and approximately 90% of forprofit participants in joint venture projects that are subject to the business reporting requirement. (Some

firms in very large joint ventures but with very limited roles are not required to report. Universities and

other not-for-profit research organizations are not included in the study because most do not plan to

commercialize the results of their ATP-funded research.) 

Figure 1 shows the distribution of firms covered in the study by size of firm and type of project. Of the

401 firms included, 167 were small firms participating in 124 projects as either a single-company award

recipient or participant in a joint-venture award; 234 were larger firms participating in 127 projects as

either a single-company award recipient or joint-venture participant. (Some projects involved both

small firms and larger firms.) Forty-two percent of the firms for which data was available were small

firms; i.e., the percentage of small firms in the BRS for which data was available for this study is very

similar to the percentage of small firms among all participants in ATP awards to date.
3. The Data and Methodology 7

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

(37)

(105)

(21)

(71)

Mediumsize

firms

39%

(92)

Fortune 500

or

equivalent

61%

(142)

234 LARGER FIRMS

In Single-Company Projects

25%

In Joint Ventures

75%

167 SMALL FIRMS

In Single-Company Projects

45%

In Joint Ventures

55%

(13)

(11)

(20)

(17)

(17)

(16)

(12)

(22)

(8)

(20)

(10)

50-99

employees

14%

(24)

100-249

employees

17%

(28)

1-9 employees

20%

(34)

20-49

employees

22%

(37)

10-19

employees

20%

(33)

(1)

250-499

employees

7%

(11)

Figure 1. SIZES OF FIRMS AND TYPES OF PROJECTS

Small firms (<500 employees) and larger firms (> 500 employees) included in the study
8 3. The Data and Methodology

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

For consistency with the Small Business Administration’s classification of small technology firms, ATP

designates firms with fewer than 500 employees as “small.” ATP-funded small firms typically are very

small. Twenty percent of those in the study had fewer than 10 employees, a number of which. A

number of these were start-up companies. Forty percent had fewer than 20 employees, and 62% had

fewer than 50 employees at the time of the ATP award. Only 24% had as many as 100 employees.

Small firms receiving single-company awards are typically somewhat smaller than small firms in joint

ventures. Fifty-one percent of small-firms receiving single-company awards had fewer than 20

employees, but only 31% of small firms in joint ventures had fewer than 20 employees.

On the other hand, many ATP-funded firms that fall outside the “small-firm” category are quite large.

As shown in Figure 1, 61% of the 234 “larger” firms included in the study are either members of the

Fortune 500 or equivalent in size to the Fortune 500. (Subsidiaries are classified according to the size

of the parent firm.) For purposes of this report, all the non-small companies are grouped together and

called “larger.” This group is contrasted with the “small-firm” group.

Small firms more frequently receive single-company awards than larger firms; larger firms are primarily

in joint ventures. Separate analyses were performed for participants in single-company projects and

joint-venture projects among the small-firm and larger-firm groups to separate some effects of project

type from effects of firm size. (See Section 3.4.) 

The technology areas addressed by small firms compared with larger firms are somewhat different, with

small firms relatively more often involved in biotechnology and electronics projects and larger firms

more engaged in materials development, manufacturing, and chemicals/chemical processing. (Smallfirm involvement in information and computing technologies was similar to that of larger firms.) Thus

differences in technology area may account for some of the differences found between small firms and

larger firms. The effects of these differences were not explored in this study although such work is

planned for the near future. 

3.3 The Stage of Project Completion of Firms Included in This Study – by Firm Size and

Project Type

Table 1 illustrates the stages of project completion of the firms covered by the study at the time the data

was analyzed. In the lower table, one sees that small firms were at a slightly later stage on average,

having filed somewhat higher percentages of Close-out and Third Anniversary reports and a smaller

percent of First Anniversary Reports. We also see that the largest percentages of reports available were

from small firms and larger firms in joint-venture projects filing their First Anniversary Report (i.e.,

their report after one year of ATP funding). ATP funded many large joint ventures in FY 1995, with

funding and R&D actually starting in 1996; thus, a large number of project participants had completed

just one full year of funding when data for the study was collected in 1997. FY 1997 and later projects

had not yet started. 

In FY 1993, the first year of awards included in the BRS, only one competition was held, and many

fewer awards made than in subsequent years, including only three joint ventures. This accounts for the

small number of Third Anniversary and Close-out reports and for the fact that substantially more firms
3. The Data and Methodology 9

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

 Most

Recent

Report

Baseline

(Beginning of

Project)

First

Anniversary

(After 1 year)

Second

Anniversary

(After 2 years)

Third 

Anniversary

(After 3 years)

Close-out

(End-ofproject)

Total

Number of Firms Reporting by Award Year

Small

Firms

Larger

Firms

Small

Firms

Larger

Firms

Small

Firms

Larger

Firms

Small

Firms

Larger

Firms

Small

Firms

Larger

Firms

Small

Firms

Larger

Firms

FY 1993 1 1 2 8 9 11 10 21 21

FY 1994 2 2 6 10 46 60 8 12 62 84

FY 1995 7 16 56 98 13 11 1 2 77 127

FY 1996 7 2 7 2

Total 16 20 63 108 60 73 8 9 20 24 167 234

Percentage of Firms Reporting by Project Type

SingleCompany

Projects

8% 29% 35% 40% 41% 23% 24% 100

%

100%

JointVenture

Projects

11% 11% 45% 50% 32% 28% 9% 5% 3% 6% 100

%

100%

All Projects 10% 9% 38% 46% 36% 31% 5% 4% 11% 10% 100

%

100%

Table 1. Stage of Project Completion

For Small Firms (<500 employees) and Larger Firms (> 500 employees) Included in the Study

in single-company projects than in joint-venture projects had completed their projects (23% of small

firms in single-company projects and 24% of larger firms in single-company projects compared with

3% of small firms in joint ventures and 6% of larger firms in joint ventures).

3.4 Supplementary Analyses

The small-firm and larger-firm groups were each subdivided into firms in single-company projects and

firms in joint-venture projects for further analysis. Separate analyses of the subgroups help to

differentiate effects of firm size from effects of project type. The separate analyses of the singlecompany and joint-venture subgroups were particularly important for this study because the reporting

unit for many of the analyses of commercialization plans and progress is the individual application,

rather than the company or the project. Given that small firms and larger firms in single-company

projects reported a larger number of applications, on average, than those in joint ventures and given that

larger firms are more frequently found in joint-venture projects than in single-company projects, the

results of analyses across applications where small firms are compared with larger firms partially reflect

the effects of the two types of projects rather than just the size of firm. The supplementary analyses of

small firms and larger firms by type of project help to remove the “weighting” effects of differences
10 3. The Data and Methodology

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

in the numbers of applications reported on by firms in single-company projects and joint ventures. To

further adjust for the effect of detailed reporting across all individual applications rather than across

firms, separate assessments of progress at the firm and project levels were performed where deemed

most likely to affect the outcome of the analysis. 

Nevertheless, some of these interlocking differences between small firms and larger firms and singlecompany and joint-venture projects are better captured in the statistical analyses performed for this

study than others. Single-company projects and joint-venture projects have somewhat different

characteristics in that joint-venture projects are often longer, larger-scope, and more likely to take on

infrastructural and process technology development. As a result, the joint ventures, on average, may

offer fewer early product opportunities. Focusing the analyses on the application or the company rather

than the project has a further diluting effect in results for joint ventures in that companies in joint

ventures play a variety of roles, ranging from highly challenging R&D to customer testing and

validation. The different characteristics of single-company projects and joint-venture projects, and the

different roles among participants of joint ventures, likely account for some of the differences in results

for small firms compared with larger firms. Further analysis a the project level, including use of case

studies, is needed to probe differences identified in this study further.

The analysis by firm size may be masking other effects as well. For example, the results for small

firms, particularly those in single-company projects, may reflect the large percentage of very small

firms, including start-up companies, in the group. The results may also reflect technological differences

of small firms compared with larger firms given the relatively larger concentrations of biotechnology

and electronics R&D projects among small firms, compared with heavier concentrations of R&D in

materials and manufacturing for larger firms. 

 

Tests of statistical significance were performed to provide supplementary indicators of the relative

strength of differences between small firms and larger firms. These tests entailed computation of Z test

statistics of differences in population proportions for small firms and larger firms for responses

involving ordinal-level data (for example, qualitative responses concerning the degree of progress).

Where ratio-level data were available (for example, patent counts), Z test statistics of differences in

population means for small firms and larger firms were computed. All results of tests for statistical

significance reported are based on two-tail tests. There was little theoretical basis for assuming a priori

that small firms would exhibit higher or lower population proportions than larger firms or higher or

lower population means for many, if any, of the characteristics examined. The Z test results are

presented in detail in Appendices A and B.

It was not necessary to perform these tests of significance to isolate effects of random sampling error,

because the analyses encompassed nearly the entire population of ATP firms funded during the period

of 1993-1996. Nevertheless, these tests (and additional Chi Square tests performed but not reported)

assisted in identifying which sets of results involved greater variation between small and larger firms

and hence in selecting which to illustrate in the report. These tests also provide a basis for projecting

the results of this study to all projects funded since the inception of the ATP in 1990. 
4. Project Goals and Expected Commercial Advantage 11

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Small Firms Larger Firms

Single

Company

Projects

Joint

Venture

Projects

All Small

Firms

Single

Company

Projects

Joint

Venture

Projects

All Larger

Firms

Number of Applications 191 168 359 125 275 400

Average Number of

Applications Per Firm 2.62* 2.10 2.35*** 2.15* 1.83 1.92***

Notes ( See Appendix B): * Z test statistic for difference in population means is significant at .10 level.

***Z test statistic for difference in population means is significant at .01 level.

Source: Business Progress Reports from 361 firms with commercialization plans in 208 projects funded 1993-1996.

Table 2. Planned Commercial Applications of the ATP-Funded Technologies–by Type of Project

Small Firms (<500 employees) and Larger Firms (> 500 employees)

4. PROJECT GOALS AND EXPECTED COMMERCIAL ADVANTAGE 

Chapter 3 provided a statistical snapshot of the success of small firms in winning ATP awards and

showed how small firms are distributed among the total group of firms and types of projects analyzed

in this study. This chapter, focusing on the project goals and expected commercial advantage of small

firms compared with larger firms, helps explain why small firms have been so successful in winning

ATP awards. The analyses encompass nearly 800 commercial applications for ATP-funded

technologies being planned by a total of 361 organizations in 208 projects, including 359 applications

being developed by 153 small firms in 118 projects.

The number and diversity of commercial applications expected to result from a technology development

project reflect its enabling character and suggest further opportunities for spillover benefits. The pursuit

of “new-to-the-world” products and/or leaps in expected values of performance attributes of the

anticipated future products usually implies substantial technical challenge or innovation, and risk

taking. Large performance, cost, and other advantages of proposed technology development projects,

of course, contribute to the innovator’s commercial advantage. Of great importance to the ATP is that

they also create opportunities for spillover benefits to downstream customers and users. (See [Jaffe,

1996] for a more complete discussion of the framework for analyzing spillovers used by the ATP.) 

4.1 Number of Commercial Applications

Small firms report more applications (uses) of their ATP-funded technology and are engaged in

planning for a larger number of applications than are larger firms. As noted above, this is important

to the ATP because it suggests the technology being developed is “enabling,” i.e., has potential to

benefit a wide spectrum of customers or users in other firms and even other industries. 

Table 2 summarizes the numbers of planned commercial applications planned by small firms and larger

firms. The Z test statistic for the difference in means of 2.35 applications per firm for small firms

compared with 1.92 applications per firm for larger firms was significant at the .01 level. Separate

analyses conducted for single-company and joint-venture participant subgroups showed that small firms
12 4. Project Goals and Expected Commercial Advantage

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

in single-company projects reported plans for a larger number of applications, on average, than larger

firms in single-company projects, and small firms in joint ventures reported plans for a larger number

of applications, on average, than larger firms in joint ventures. The Z test statistic for the difference in

means for small firms in single-company projects compared with larger firms in single-company

projects was significant at the .10 level. In many cases, ATP’s small firms are start-ups or have no

products currently under commercialization. They appear to be planning a number of different

commercialization options, some with short-term potential for raising cash and others longer-term,

requiring accomplishment of more ambitious project goals. 

4.2 Project Goals and Expected Commercial Advantage

Figure 2 describes the ATP project goals for small firms compared with larger firms, which, if achieved,

are expected to result in commercial advantage in their specific markets. The types of goals and

resulting commercial advantage expected by small firms were somewhat different from those expected

by larger firms. 

4.2.1 “New-to-the-world” Solutions

Small-firm applications more frequently entail either “new-to-the-world” solutions, where there are

essentially no competing products or services currently, or ambitious performance improvement goals

or cost improvement goals. These aggressive product performance and cost goals thus support the

ATP’s desire to fund technically challenging “breakthrough” technologies with potential to spawn entire

new product lines and even new industries. 

For small firms, 37% of applications were considered “new-to-the-world” solutions, i.e., innovative

products aimed at entirely new markets, compared with 31% for larger firms. The Z test statistic for

the difference in population proportions of small firms reporting a “new-to-the-world solution” was

significant at the .10 level. 

Table 3 breaks the application results down by type of project. Across both small firms and larger

firms, participants in single-company projects planned more “new-to-the-world” products among their

applications than joint-venture participants did (41% of applications of small firms and 38% of

applications of larger firms in single-company projects compared with 32% of applications of small

firms and 28% of applications of larger firms in joint ventures). Apparently firms pursuing “new-tothe-world” products, for example, those at relatively early stages of the innovation process, are less

likely to be involved in ATP joint ventures than those seeking performance and/or cost improvements,

although one or more firms in any given joint venture might pursue “new-to-the-world” products. 
4. Project Goals and Expected Commercial Advantage 13

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Notes (see Appendix A): * Z test statistic for difference between two population proportions is significant at .10 level.

 *** Z test statistic for difference between two population proportions is significant at .01 level.

Source: Business Progress Reports from 361 firms with commercialization plans in 208 ATP projects funded 1993-1996.

For those expecting performance and/or cost advantages:

0

20

40

60

80

100

22

34

69

58

9 8

Manuf.

Process

***

Product

***

Service

Type of Commercialization

Expected

Percent of Applications

Small

Firms

(359 Applications)

Larger

Firms

(400 Applications)

100

Performance Improvement Goals

20

40

60

80

0

0%

19

34

9

23

12 13

17

7

43

23

1 - <25% 25 - <50% 50 - <99% 100-999+%

***

Percent of Applications

 207 Applications - Small Firms

259 Applications - Larger Firms

17

38

50 - 99%

***

20

40

60

80

100

0

Cost Reduction Goals

38

42

15

28

9

13

0% 1 - <25% 25 - <50%

Percent of Applications

207 Applications - Small Firms

259 Applications - Larger Firms 

20

40

60

80

100

Performance Cost Performance

& Cost

New

Solution

*

Other

20 18

9

33

37 38

31

0

4 4 6

Commercial Advantage

Expected

Percent of Applications

Percent Cost Reduction

Percent Improvement

Figure 2. PROJECT GOALS AND EXPECTED COMMERCIAL ADVANTAGE

Comparison of small firms (<500 employees) with larger firms (> 500 employees) with commercialization

plans
14 4. Project Goals and Expected Commercial Advantage

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Percentage of Applications

Small Firms Larger Firms

Single

Company

Projects

Joint

Venture

Projects

All Small

Firms

Single

Company

Projects

Joint

Venture

Projects

All Larger

Firms

Type of Commercialization

Expected

 Process

 Product

12%

79%

34%

59%

22%

69%

31%

63%

35%

55%

34%

58%

Commercial Advantage

 New Solution 41% 32% 37% 38% 28% 31%

Performance Improvement

Goals

 100-999+% 49% 38% 43% 30% 20% 23%

Cost reduction goals

 50-99% 46% 31% 38% 17% 16% 17%

Source: Business Progress Reports from 361 firms with commercialization plans for 759 applications in 208 projects

funded 1993-1996.

Table 3. Project Goals and Expected Commercial Advantage–by Type of Project

Small firms (<500 employees) and larger firms (> 500 employees)

4.2.2 Performance and Cost Improvements

For most of the remaining applications, performance advantages over existing products were more

important than cost reductions for both small firms and larger firms. This finding is likely related to

the greater emphasis on product applications compared with process applications for both groups. 

Small firms had aggressive performance and cost goals for more applications than larger firms.

Performance improvements of 100-999+% were expected for 43% of small-firm applications, but only

for 23% of relevant applications of larger firms. Cost reductions of 50-99% were expected for 38% of

small-firm applications, but only for 17% of relevant applications of larger firms. The Z test statistic

of differences in population proportions for small firms compared with larger firms was significant at

the .01 level for both of these performance and cost goal ranges.

These differences held whether small firms were in single-company projects or participants in joint

ventures. As shown in Table 3, small firms reported performance goals of 100-999+% improvement

for a higher percentage of applications than larger firms in either type of project (49% of applications

for small firms in single-company projects and 38% of applications for small firms in joint ventures

compared with 30% of applications for larger firms in single-company projects and 20% of applications

for larger firms in joint ventures).
4. Project Goals and Expected Commercial Advantage 15

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

4.2.3 Process Versus Product Applications

Larger-firm applications more frequently emphasized process improvements and related cost-reduction

goals than did small-firm applications. Thirty-four percent of larger-firm applications involved

manufacturing process applications rather than products or services, compared with 22% of small-firm

applications. The Z test statistic of the difference in population proportions was significant at the .01

level for the percentage of small-firm applications involving products compared with the percentage

of larger-firm applications involving products, and similarly, but in the reverse direction, for process

applications.

This finding is consistent with the work of Utterback [1994] and others who have noted that advances

during stages of innovation in a new product area are dominated by achievement of functionality for

products with new capabilities, while later generations of product advances are more oriented to

increased process efficiencies. New and improved processes can be innovative and enabling in that they

may actually expand the capabilities of the product group to enable a broader pool of applications where

cost and specific performance features are more critical than they were for the initial products

introduced. For example, new processes may have significant effects on product size, ruggedness, or

reliability, along with increasing manufacturing and integration efficiencies. In addition to expanding

the opportunities for firms to build new markets, new processes may provide economies of scale needed

to compete in existing international markets and more mature product areas. 

Small firms with limited, or no, existing product lines, such as is the case for the start-ups, and with no

large-scale process operations are more likely to be addressing the earlier stages of innovation (i.e.,

those involving new products, or products with new capabilities) rather than process efficiencies. As

shown in Table 3, only 12% of applications of small firms in single-company projects involved

processes. Small firms in joint ventures are more likely to be pursuing process applications than small

firms in single-company projects. Percentages of applications involving processes were nearly the same

for small firms in joint ventures as for larger firms in joint ventures (34% compared with 35%), and

both of these percentages were greater than for larger firms in single-company projects, which reported

31% process applications. 

Small firms in joint ventures may be drawn into process work through their collaborative R&D with

larger firms. A contributing factor may be that the ATP-funded small firms in joint ventures are slightly

larger, and presumably somewhat more mature and more likely to have at least some manufacturing

capability, than those in single-company projects. 

4.3 Conclusions

The data seem to capture the different “personalities” of small firms and larger firms in the ATP

portfolio. The enthusiasm and energy of the small firms for taking on ambitious problems are visible

and appealing and indicative of why small firms have been successful against the ATP selection criteria.

Whether the aggressive goals of the small companies are achievable, or even appropriately realistic, is

less clear at this stage but subsequent chapters will shed some light on the progress they are making.
16 4. Project Goals and Expected Commercial Advantage

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

On the surface, small firms appear to be attempting more ambitious project goals than larger firms,

particularly in joint ventures. However, this effect may actually not reflect differences in the inherent

difficulty of the projects being undertaken. The following factors appear to contribute to the statistical

difference: 1) Small firms tend to be at a different stage of the innovation process than larger ones. This

is consistent with the substantial number of very small, start-up companies. 2) Larger firms are more

likely to be pursuing new manufacturing processes where the magnitude of the innovation may not be

captured as clearly in the survey questions concerning project goals as for applications where the firms

are engaged in technology development at an earlier stage of the innovation cycle and are focused on

products rather than processes. 3) Smaller firms tend to have different roles in joint-venture projects

than larger firms. 4) Participants in joint ventures play a variety of types of project roles, ranging from

performing the most innovative R&D to serving as test bed demonstration sites to serving as suppliers;

therefore, the level of difficulty of the tasks and related goals of various members of joint ventures

must be expected to be more variable than for firms in single-company projects. To the extent that,

within joint ventures, larger firms are more likely to perform the test bed/demonstration role and small

firms are more likely to be engaged in earlier-stage R&D, one would expect that these small firms on

average would have opportunities for more applications and would exhibit more aggressive product

performance and cost goals than the larger firms in the joint ventures. Thus, even though the

contrasting differences in goals by size of company are provocative, the findings are not conclusive.

 Additional analysis is required at the project level to probe these issues.
5. Strategies for Commercialization 17

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

5. STRATEGIES FOR COMMERCIALIZATION

The ambitious project goals of ATP-funded small firms, emphasizing “new-to-the-world” solutions and

aggressive product performance, suggest that these firms have characteristics of successful small firms

described in the business literature. The pursuit of large numbers of applications and of innovative

product content, rather than cost, price, and economies of scale, reflect multiple-product strategies and

suggest that the firms as a group have the organizational flexibility needed to take on the related risks.

But are small firms employing effective commercialization “strategies for success” for achieving their

goals for high performance products and numerous applications? This chapter looks at the

commercialization strategies being employed by ATP-funded small-firms compared with larger firms

to see whether those of small firms seem to be geared to handle the special challenges facing small

firms.

Figure 3 illustrates some of the similarities and differences in strategies for commercialization planned

by small firms compared with larger firms. The most significant differences are that: a) small firms

planned to license their ATP-funded technology for more of their applications than larger firms did

(whether through patent license or a software license); b) small firms planned to form alliances for

commercialization of more applications than larger firms, most frequently with customers; and c) larger

firms were more likely to be planning to use their technology in-house for process applications than

small firms. (The Z test statistics for the differences in population proportions of small-firm

applications and larger-firm applications for which these were primary strategies were significant at the

.01 level.) In-house production of a product or service was the most common strategy for both small

firms and larger firms. Larger firms planned this strategy for 66% of applications and small firms for

62% of applications. 

5.1 Technology Licensing and/or In-house Production

Small firms appear to be pursuing a mix of in-house production and licensing strategies, even for some

of the same applications. Patent licensing allows small firms to produce a product in-house while also

providing its customers with the security of a second source. This may be particularly important for

firms establishing their very first manufacturing capability and entering markets where their customers

downstream anticipate significant and rapid scale-up in volume. 

Licensing of the ATP-funded technology also allows firms to diffuse their technology to a broader

spectrum of markets than they could attempt to supply competitively. For at least some applications,

licensing represents revenue opportunities for firms without the need to make major expenditures to

develop specialized manufacturing facilities and distribution channels. Software firms, because the

costs of product development and production are often much lower than for manufactured products,

are sometimes able to produce a number of products for substantially different applications and markets

despite their limited resources and staff. They generally use a licensing mechanism to authorize use

of their software products.
18 5. Strategies for Commercialization

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

100

0

20

40

60

80

38

17 19

15

32

40

11

28

Percent of Applications

Licensing to Others

Primary

***

Secondary Possible N/A

0

20

40

60

80

100

62

66

18

7 10 7 10

20

Percent of Applications

Producing Product/Service

In-house

Primary Secondary Possible N/A

100

0

20

40

60

80

22

34

12 14 13

19

53

33

Percent of Applications

Adopting Process for

In-house Use

Primary

***

Secondary Possible N/A

0

20

40

60

80

100

Percent of Applications

21

10

17

9

30

23

32

58

Forming Alliances With

Distributors

Primary

***

Secondary Possible N/A

20

40

60

80

100

0

Percent of Applications

24

20

9 9

29

41

27

41

Forming Joint Production

Alliances

Primary

***

Secondary Possible N/A

Note (see Appendix A): *** Z test statistic for difference between two population proportions is significant at .01 level.

Source: Business Progress Reports from 361 firms with commercialization plans in 208 projects funded 1993-1996.

Small Firms (361 Applications) Larger Firms (397 Applications)

0

20

40

60

80

100

Percent of Applications

29

18 19

11

30 32

22

39

Forming Alliances With

Customers

Primary

***

Secondary Possible N/A

Figure 3. STRATEGIES FOR COMMERCIALIZATION

Comparison of small firms (<500 employees) with larger firms (> 500 employees) with commercialization

plans
5. Strategies for Commercialization 19

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Percentage of Applications

Small Firms Larger Firms

Single

Company

Projects

Joint

Venture

Projects

All Small

Firms

Single

Company

Projects

Joint

Venture

Projects

All Larger

Firms

Licensing to others

 Primary

 Secondary

53%

22%

22%

15%

38%

19%

26%

22%

13%

12%

17%

15%

Producing product/service

In-house

 Primary

 Secondary

60%

17%

65%

18%

62%

18%

73%

4%

63%

9%

66%

7%

Adopting process for inhouse use

 Primary

 Secondary

16%

10%

28%

15%

22%

12%

35%

13%

33%

14%

34%

14%

Forming alliances with

customers

 Primary

 Secondary

30%

21%

28%

15%

29%

19%

24%

8%

16%

12%

18%

11%

Forming joint production

alliances

 Primary

 Secondary

27%

22%

12%

27%

20%

24%

8%

10%

10%

10%

9%

9%

Forming alliances with

distributors

 Primary

 Secondary

27%

19%

14%

14%

21%

17%

21%

7%

4%

10%

10%

9%

Source: Business Progress Reports from 361 firms with commercialization plans for 758 applications in 208 projects

funded 1993-1996.

Table 4. Strategies for Commercialization–by Type of Project

Small firms (<500 employees) and larger firms (> 500 employees)

Licensing was a more common strategy for applications being pursued by firms in single-company

projects–both small firms and larger firms–than by those in joint ventures; and percentages were highest

for small firms in single-company projects. As shown in Table 4, “Licensing to others” was a primary

strategy for 38% of all small-firm applications, compared with 17% of applications planned by all larger

firms. Small firms with single-company awards cited it as the primary strategy for 53% of applications,

compared with 26% of applications for larger firms with single-company awards. 

Larger firms in joint ventures have less need than small firms to transfer their technology to a second

source supplier in their own markets, but like small firms, many larger firms appear to see opportunities

to pursue broader markets through licensing than they can supply competitively with their own

products.
20 5. Strategies for Commercialization

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Licensing would appear to be an appropriate strategy for some, if not all, applications for many very

small firms without joint venture partners as potential customers for their technology. Licensing is of

special interest to the ATP because it increases the chances of spillover benefits through additional

technology development and diffusion into new markets. Nevertheless, patent licenses are not easy to

negotiate because, in essence, they transfer risks for further development of the technology, for

manufacturing, and for marketing and sales to the licensee.

Larger firms in single-company projects reported the largest percentage of applications involving inhouse production as the primary strategy. Consistent with their greater emphasis on a licensing

strategy, small firms in single-company projects reported the lowest percentage of applications

involving in-house production as the primary strategy. 

5.2 Adoption of New Manufacturing Processes

Larger firms more frequently than small firms plan to adopt new/improved processes for in-house use

as their primary strategy, as one would expect from findings about project goals of larger firms in

Chapter 4. Larger firms reported that they planned to “adopt a process for in-house use” for 34% of

applications, compared with 22% of applications for small firms. As shown in Table 4, percentages

of applications involving in-house processes were higher for larger firms in both single-company

projects and joint ventures than for small firms in either type of project, although small firms in joint

ventures more frequently planned in-house process applications than small firms in single-company

projects. Process applications were about equally common for larger firms in single-company projects

and joint ventures (for 35% of applications and 33% of applications respectively); however, only 16%

of small firms in single-company projects planned process applications.

5.3 Strategic Alliances Across the Supply Chain

Small firms appear to be using approaches in addition to licensing to bridge the gap to the marketplace.

Figure 3 summarizes their plans for strategic alliances for commercialization along with those of larger

firms. Such alliances were consistently a primary or secondary strategy for a higher percentage of

small-firm applications than larger-firm applications. As one might expect, “forming alliances with

customers” was the most common primary or secondary alliance strategy for small-firm applications.

This strategy was cited as primary or secondary for 48% of small-firm applications but only 29% of

larger-firm applications. “Forming joint production alliances” was a primary or secondary strategy for

44% of small-firm applications, but only 18% of larger-firm applications. “Forming alliances with

distributors” was a primary or secondary strategy for 38% of small-firm applications but only 19% of

larger-firm applications. The Z test statistics for the difference in population proportions were

significant at the .01 level for each of these alliance strategies as a primary/primary or secondary

strategy. Overall, small firms seem to recognize the need for supply chain linkages for at least some

applications. 

As shown in Table 4, strategic alliances with customers were cited as a primary or secondary strategy

most frequently by small firms in single-company projects, and next most frequently by small firms in
5. Strategies for Commercialization 21

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

joint ventures, and least frequently by larger firms in joint ventures. (It was a primary or secondary

strategy for 51% of applications of small firms in single-company projects, compared with 43% of

applications for small firms in joint ventures, 32% of applications for larger firms in single-company

projects, and 28% of larger firms in joint ventures.) 

The findings that small firms in single-company projects are seeking alliances with customers for more

applications than small firms in joint ventures, and that larger firms in single-company projects

likewise plan alliances with distributors and customers as a primary strategy for a higher percentage of

applications than do larger firms in joint ventures, are somewhat surprising given that ATP joint

ventures often “build in” such supply chain linkages. In these “supply-chain” joint ventures, firms have

some degree of access to the technology of potential suppliers and/or customers while it is being

developed and have the opportunity to strengthen firm-to-firm relationships over the course of the

R&D. In reporting their plans for such alliances, many firms participating in joint ventures likely see

the joint venture as a partial substitute for other types of commercialization alliances. Nevertheless,

results obtained in this study show that many firms are planning alliances for commercialization outside

the bounds of ATP joint ventures, and small firms, particularly those without joint-venture support, are

more likely than larger firms to be seeking to build customer or supplier alliances. 
22 6. Collaborative R&D

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

6. COLLABORATIVE R&D

The ATP encourages small firms to come together with larger firms, universities, and other research

organizations in collaborative R&D relationships as part of its legislative mandate. The availability of

partial funding through the ATP provides incentives for industry partners and others with

complementary technical and market strengths to share risks and tackle difficult R&D problems of

common interest. 

Some ATP-funded joint ventures consist of the legislated minimum of just two for-profit companies;

others include twenty or more companies of different sizes, and universities or other research

organizations as well. These R&D collaborations extend beyond the bounds of “official” ATP joint

ventures. Firms in both single-company projects and joint ventures engage in subcontractor

arrangements and other informal collaborations.

Collaborative R&D appears to represent an important component of the “strategies for success” of ATPfunded small firms. This chapter explores the participation of small firms in ATP-funded R&D

partnerships and the role of the ATP in stimulating these collaborative relationships. It then provides

evidence showing that many of these collaborations are achieving R&D efficiencies, most are

overcoming the typical management difficulties, and many are assisting small firms in building

customer relationships that will be needed to commercialize their innovative technologies successfully.

6.1 Collaboration Activity in ATP Projects

Figure 4 illustrates the amount of collaboration activity engaged in by firms having participated in the

ATP for at least one year and the degree to which the ATP was considered responsible for the

collaboration. Collaborative activity has been quite vigorous. Eighty-two percent of larger firms and

76% of small firms cite collaboration experience on their ATP project. The higher percentage for larger

firms is likely due to the higher percentage of larger firms in joint ventures. The Z test statistic for the

difference in population proportions of small firms and larger firms reporting collaboration experience

was not significant. (See Appendix A.) 

Subcontractor and other alliance activity engaged in by firms in single-company projects, which

primarily are small firms, does not fully compensate for the greater joint-venture activity among the

larger firms. Nevertheless, 66% of small firms in single-company projects and 67% of larger firms in

single-company projects reported collaboration experience, indicative of considerable collaboration

outside the bounds of the ATP research joint ventures among both small firms and larger firms. (The

breakdown of collaboration activity by type of project is included in Table 5.)

Separate data from project proposals and BRS reports show that small-firm subcontractors are

commonly used by both small firms and larger firms. Many small firms serve as a subcontractor to

another firm in a single-company project or a joint venture rather than seek formal joint-venture

participant status because they have difficulty meeting the joint-venture cost share. (Joint-venture

participants must cover at least 50% of total project costs; the cost share need not be provided equally

by all the partners. Subcontractors have no cost-share requirement.)
6. Collaborative R&D 23

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Source: Business Progress Reports from 360 firms in 202 ATP projects funded 1993-1996 after at least one year of ATP funding.

Larger

Firms

(211)

Small

Firms

(149)

0

20

40

60

80

100 Collaboration Experience?

Percent of Firms

YES NO

76

82

24

18

Of those reporting

collaboration experience:

20

40

60

80

100

0

Was ATP Responsible?

Percent of Firms

No/Not at All/

Not Sure

Moderate Slightly

Extent

Great

Extent

66 67

26

18

4

9

4 6

Larger

Firms

(173)

Small

Firms

(112)

Figure 4. COLLABORATION EXPERIENCE IN ATP PROJECTS

Comparison of small firms (<500 employees) with larger firms (> 500 employees)
24 6. Collaborative R&D

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

6.2 ATP Responsibility for Collaboration

According to most firms, the ATP was responsible for their collaborative activities, with 66% of small

firms and 67% of larger firms reporting that ATP was responsible “to a great extent”; i.e., ATP was

thought to be at least a catalyst in the formation of many of the collaborative relationships. As shown

in Table 5, a higher percentage of joint-venture participants than participants in single-company projects

attributed their collaborative activities to the ATP. Thus the ATP may be more critical to the formation

of formal joint ventures than to use of subcontracts for R&D services or to less formal partnerships with

suppliers or potential customers. 

6.3 Effects of Collaboration

Large percentages of firms of all sizes reported significant positive effects from collaboration. Figure

5 illustrates the results for the subset of effects where differences in the numbers of small firms

compared with larger firms experiencing the effect were substantial or the effect was particularly

widespread. A higher percentage of small firms than larger firms reported a “significant” effect from

collaboration in “identifying customer needs” (55% of small firms compared with 38% of larger firms

reported a “significant” effect), “stimulating creative thinking” (78% of small firms compared with 66%

of larger firms reported a “significant” effect), and “attracting investment capital” (20% of small firms

compared with 12% of larger firms reported a “significant” effect). Among the other effects illustrated,

“accelerating market entry” was reported to be a “significant” effect by 56% of small firms compared

with 54% of larger firms, and “obtaining R&D expertise” was reported to be a “significant” effect by

56% of small firms compared with 50% of larger firms. 

The Z test statistics for the difference in the population proportions of small firms and larger firms that

considered the effect “significant” were significant at the .01 level for “identify customer needs,” at the

.05 level for “stimulating creative thinking,” and at the .10 level for “attracting investment capital.” 

Percentages of small firms and larger firms reporting an effect on “obtaining R&D expertise” and

“accelerating market entry” were very similar. Combining the “significant” and “moderate” responses

for “accelerating market entry,” the difference in percentages for small firms and larger firms increased,

but the Z test statistic for the difference in the population proportions was not significant. (See

Appendix A.) 

Differences between the percentages of small firms and larger firms that reported a “significant” or

“significant or moderate” effect on “increasing project management and coordination costs” were

somewhat greater, with small firms more frequently reporting the effect; however, the Z test statistics

for the differences in population proportions were not significant. (See Appendix A.)
6. Collaborative R&D 25

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Larger

Firms

(173)

Small

Firms

(112)

Notes (see Appendix A): * Z test statistic for difference between two population proportions is significant at .10 level.

 ** Z test statistic for difference between two population proportions is significant at .05 level.

 *** Z test statistic for difference between two population proportions is significant at .01 level.

Source: Business Progress Reports from 285 firms in 159 ATP projects funded 1993-1996 after at least one year of ATP funding.

100

60

80 In attracting capital investment?

73

52

100

0

20

40

60

80

In stimulating creative thinking?

Percent of Firms

Significantly

**

Moderately Little/None/

Not Sure

2

5

32

17

66

78

0

20

40

60

80

100

In identifying customer needs?

Percent of Firms

Significantly

***

Moderately Little/None/

Not Sure

32

21

30

24

38

55

0

20

40

60

80

100

56

50

31 34

13 16

Little/None/

Not Sure

Significantly Moderately

In obtaining R&D expertise?

0

20

40

60

80

100

56 54

30

25 21

14

Little/None/

Not Sure

Significantly Moderately

In accelerating market entry?

0

20

40

60

80

100

11

6

47

43

51

42

Little/None/

Not Sure

Significantly Moderately

To what extent did it increase

management/coordination costs?

Percent of Firms

Percent of Firms

Percent of Firms

0

40

Significantly

*

Moderately Little/None/

Not Sure

20

Percent of Firms

15

28

12

20

Figure 5. HAS COLLABORATION BEEN EFFECTIVE?

Comparison of small firms (< 500 employees) with larger firms (> 500 employees) reporting collaboration

experience
26 6. Collaborative R&D

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Small Firms Larger Firms

Single

Company

Projects

Joint

Venture

Projects

All Small

Firms

Single

Company

Projects

Joint

Venture

Projects

All Larger

Firms

Collaboration experience? 

 Yes

66% 83% 76% 67% 88% 82%

Was ATP responsible?

 Great extent 

 Moderate extent

60%

31%

70%

22%

66%

26%

56%

10%

69%

21%

67%

18%

Specific effects:

 Identify customer needs?

 Significantly 

 Moderately 

42%

31%

63%

19%

55%

24%

31%

26%

40%

32%

38%

30%

Stimulate creative thinking?

 Significantly

 Moderately

78%

16%

78%

18%

78%

17%

62%

39%

67%

31%

66%

32%

Attract capital investment?

 Significantly

 Moderately

29%

27%

13%

28%

20%

28%

13%

8%

12%

16%

12%

15%

Obtain R&D expertise?

 Significantly

 Moderately

53%

33%

58%

30%

56%

31%

51%

28%

50%

35%

50%

34%

Accelerate market entry?

 Significantly

 Moderately

53%

33%

58%

28%

56%

30%

49%

21%

55%

25%

54%

25%

Increase management/

coordination costs?

 Significantly

 Moderately

7%

47%

13%

48%

11%

47%

3%

41%

7%

43%

6%

43%

Source: Business Progress Reports from 360 firms in 202 projects funded 1993-1996–after at least one year of funding.

Table 5. Collaboration Experience–by Project Type

Percentage of firms for small firms (<500 employees) and larger firms (> 500 employees)

As shown in Table 5, responses from small and larger firms varied depending on whether they were in

single-company projects or joint-venture projects. As one would expect given the supply-chain linkages

involved in many ATP-funded joint ventures, firms in joint ventures experienced a significant effect

on “identifying customer needs” more frequently than those in single-company projects within the same

size group (63% of small firms and 40% of larger firms in joint ventures compared with 42% of small

firms and 31% of larger firms in single-company projects). Firms in joint-venture projects more

frequently reported a significant acceleration effect than those in single-company projects of either size

class, with larger firms in joint ventures reporting this effect nearly as frequently as small firms (58%

for small firms and 55% for larger firms in joint ventures compared with 53% of small firms in singlecompany projects and 49% of larger firms in single-company projects). Also, firms in joint-venture
6. Collaborative R&D 27

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

projects more frequently reported that project coordination and management costs had increased

significantly as a result of collaboration, with more small firms reporting this effect than larger firms

for each size class (13% for small firms and 7% for larger firms in joint ventures compared with 7%

of small firms and 3% of larger firms in single-company projects). Only a small percentage of all firms

reported problems from collaboration. 

Small firms in joint ventures more frequently reported a significant effect on obtaining R&D expertise

than small firms in single-company projects (58% compared with 53%); however, the difference in the

frequency of this effect for larger firms in joint-venture projects and single-company projects was less

than for small firms (50% compared with 51%).

Firms in single-company projects reported experiencing a “significant” effect of collaboration on

“attraction of capital” more frequently than firms in joint ventures. More small firms in single-company

projects reported a “significant” effect for “attracting investment capital” than small firms in joint

ventures (29% of small firms in single-company projects compared with 13% of small firms in joint

ventures); likewise, more larger firms in single-company projects reported this effect than larger firms

in joint venture projects, but the percentages of larger firms reporting a “significant” effect were much

smaller and there was little variation in the results for the two types of projects (13% of larger firms in

single-company projects reported a “significant” effect compared with 12% of larger firms in joint

venture projects). 

For the remaining effects, the frequency of the effect appeared more closely related to firm size than

to the type of project. Small firms in single-company projects and small firms in joint ventures more

frequently reported a “significant” effect of collaboration on “stimulating creative thinking” than larger

firms in either single-company projects or joint ventures (78% for small firms in both single-company

projects and joint ventures compared with 67% for larger firms in joint ventures and 62% for larger

firms in single-company projects). 

Together, these results appear to show that both small firms and larger firms are engaging actively in

R&D collaborations and/or other alliances that have potential to contribute to the success of their ATPfunded technology development projects and to ultimate commercial success. Small firms more

frequently judged the effects of their collaborations “significant” than larger firms. Small firms thus

appeared to be somewhat more “sensitive” to specific benefits and costs of collaborations than larger

firms; i.e., they seemed more likely to judge an effect as “significant,” perhaps because small firms were

more likely to judge that these effects had a significant impact on their overall chances of success. For

effects where supply-chain linkages were particularly important, for example, “identifying customer

needs,” and “accelerating market entry,” firms in formal ATP joint ventures experienced a “significant”

effect from collaboration more frequently than those engaged in subcontractor or other relationships

outside of a formal ATP joint venture. On the other hand, companies with subcontractor or other

collaborative relationships outside of joint ventures appeared to experience effects on “stimulating

creative thinking” and “obtaining R&D expertise” comparable to the effects obtained through formal

ATP joint ventures. Finally, joint-venture collaboration did not appear to provide a frequent,

“significant” assistance in helping small firms (or larger firms) attract capital investment. Small firms

may find other relationships more effective for this purpose.
28 6. Collaborative R&D

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

6.4 Anecdotal Comments

Anecdotal comments from individual firms help to illustrate further the statistical findings and provide

some additional insights. In their anecdotal comments, small firms tend to emphasize the following

benefits of collaboration: a) the increased breadth of research, b) the increased number and expanded

scope of applications, c) acceleration in time to market as a result of combining complementary R&D

resources from multiple firms, and d) improved access to customers for testing and for “advice”

concerning product needs. A number of small firms expressed some concern about project delays due

to conflicts with partner companies. The following are a few of the many positive and negative

comments received from companies.

6.4.1 Small Firms in Single-Company Projects:

“Collaboration with research groups which has resulted from our ATP involvement has

significantly expanded the possible applications for the ... technology and has accelerated the

development of products for these applications.”

“Collaborations have allowed [us] to generate additional intellectual property and have guided

us toward thinking about new product ideas.”

“The manpower-multiplying and synergistic creative effects of using subcontractors has

accelerated the R&D process to make it possible to plan and begin to initiate business alliances

for the technology that capitalize on a fast-approaching market opportunity.”

“We have received expert advice in our commercialization efforts. We have been able to

ascertain general characteristics of the market for our product.”

“Several contractors ... were extremely helpful in facilitating our progress. Although they were

technically subcontractors, they acted as if, and we felt as if, they were true collaborators with

us.”

6.4.2 Small Firms in Joint-Venture Projects:

“In general, the collaboration has allowed us to contact new potential collaborators and

markets. Some of these markets are for new equipment using our technology in ways we had

not considered.”

“Intercorporate collaboration has fostered business relationships which are likely to continue

beyond the ATP into product development and commercialization. This should accelerate the

application of the ATP-developed technology.”

“The biggest positive impact is from the collective view of marketplace implementation and

areas of value for the customer.”

“Collaboration has helped us gain access to application testing from potential customers.
6. Collaborative R&D 29

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

While this has leveraged our resources and reduced development costs, it also has resulted in

significant delays in the development program due to schedule and priority conflicts at partner

companies.”

“Through our collaboration with ... we now have access to potential customers (the colleagues

of our collaborators) who would have been less receptive had we approached them on our

own.” 

“Working with specialized skills of member firms in collaboration is a positive impact. Dealing

with differing member firm commercialization objectives can be a negative impact.”

“On the positive side, there has been significant information transferred back and forth, in both

technical and marketing areas. On the negative side, collaboration has introduced its own set

of problems communicating, coordinating, and establishing administrative functions.”

“Positive - added prestige of program due to recognition associated with a “name” partner.

Negative - coordination difficulty and need to agree on program resource allocation and

direction.”

6.4.3 Larger Firms in Single-Company Projects

“The collaboration would not have occurred without ATP support. Also, it has generally raised

awareness of the benefits in leveraging [our] research dollars to do high risk technology

development projects. It has helped facilitate a growing relationship with ... in other areas of

[our] business and we have an excellent relationship with Professor .... The collaboration with

Professor ... has also provided us with early access to several outstanding doctoral candidates

and post-doctoral fellows for potential recruiting purposes.”

“Securing a contract with the ... to install a pilot demonstration ... facilitated a number of

discussions which has identified the real customer needs and therefore accurate functional and

performance requirements of the products. [Our] collaboration with ... University (as a

subcontractor) has led to the acquisition of some fundamental optimization technology.”

“There are two types of impacts: intellectually through our subcontract with ... and practically

through our subcontracts with ... and others by getting access to expertise and equipment which

we could not easily duplicate due to resource constraints. Without such collaborations the pace

of the project would be significantly slower and considerably costlier.”

“Our subcontractor, ... University, has done an outstanding job. With ATP approval we have

increased their statement of work by $24,000 enabling them to continue to reduce program risk

during our last project year. We were also pleased to host a graduate student from the

university this summer to work on the project. This provides the student with industrial

experience.”
30 6. Collaborative R&D

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

6.4.4 Larger Firms in Joint-Venture Projects

“The ATP award and its broad publication has brought many potential collaborators to our

doors. Our notoriety has also attracted more competitors to the field. We have used the ATP

model to set up a non-ATP collaboration with ... University.”

“[Our ATP project helped us establish] more disciplined approach to setting hard measurable

targets since common reviews are held with combined company managements.”

“[The ATP project] facilitated communications about potential business relationships outside

of the ATP program. [The] project has raised some concerns regarding confidentiality and

intellectual property.”

“Positive: [The ATP project led to] reduction of ‘not invented here syndrome’; reduction of

capital cost on equipment available from JV partner; positive factor in negotiating other

contract research with outside company that involves complementary technologies of both JV

partners.

Negative: [The ATP project led to] difficulties experienced in initial collaboration with JV

partner revolving around issue of trust (viewing the other company as a potential competitor).”

“Not all of the business models of the participating companies were compatible which resulted

in some serious business negotiation problems relative to intellectual property.”
7. Effects of ATP Funding 31

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

7. EFFECTS OF ATP FUNDING

The trade-off between R&D cycle time and the time required to develop innovative new products with

a high level of technical content poses a challenge to all firms. The challenge is compounded for small

firms trying to raise capital for high-risk technology development projects in private financial markets

and seeking to build their first customer relationships. These challenges help to create a role for the

ATP: Under its legislated mandate, the ATP explicitly aims to make a difference over what industry

and private financial markets can and would do on their own in funding technically risky, but broadly

enabling R&D projects. 

The results presented in this chapter provide evidence that the ATP funding is making a difference –

for firms of all sizes – over what industry would do on its own. In some cases, ATP funding enables

projects to be undertaken that likely would not happen otherwise. Small firms and larger firms receive

direct financial assistance for development of risky – and disruptive – new technologies leading to

innovative “new-to-the-world” products that the companies could not pursue on their own. More

frequently, the differences are in terms of the time required to do the R&D; the nature, scale, and scope

of work that would be performed; and the level of industry’s own financial investment in the R&D.

In addition to providing taxpayer support for industry R&D that the private sector is unable to fund on

its own and in addition to stimulating companies to increase their own level of funding to these projects,

the ATP award appears to assist firms in raising funds in the private capital markets. In particular, small

firms have received a benefit from the announcement of an ATP award in the form of a “halo effect”

with investors and others in their supply chains. Figure 6 illustrates effects of ATP funding on

acceleration of R&D, on the scope and nature of R&D conducted, and on the amount of industry funds

invested in the ATP-funded technology areas. 

7.1 Acceleration of R&D

Acceleration of R&D has been cited in the literature as an increasingly important “strategy for success”

where the flexibility and aggressiveness of small firms would appear to provide a distinct advantage.

ATP funding helps firms accelerate their R&D projects and thus increases the chances that technical

challenges can be overcome and resulting new products will reach the market within the critical window

of opportunity. Acceleration is part of ATP’s legislated mandate and has been frequently cited in past

studies as an important effect of ATP funding [Laidlaw, 1997; Martin et al., 1998; Powell, 1997;

Silber, 1996]. 

Most firms, regardless of size, report some acceleration in their R&D. In analyses shown in Figure 6,

88% of small firms and 85% of larger firms reported they were already ahead in the R&D cycle as a

result of ATP funding. 

The acceleration effect appears more common for firms in single-company projects than for those in

joint ventures. As shown in Table 6, under “Position in R&D Cycle,” 97% of small firms in singlecompany projects and 95% of larger firms in single-company projects reported they were “ahead” in

the R&D cycle compared with 81% of small firms and of larger firms in joint ventures.
32 7. Effects of ATP Funding

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

0

20

40

60

80

100

Change in Level of Industry

Investment from Internal Funds

25

37

57

32

15

28

3 3

No Project

w/o ATP

**

Increased

*** NoEffect

Decreased/

Not Sure

Percent of Firms

147 Small Firms

207 Larger Firms

0

20

40

60

80

100 Position in R&D Cycle

88 85

8 8

4 7

Percent of Firms

Ahead Same Place Behind or

Not Sure

146 Small Firms

 208 Larger Firms

Notes (see Appendix A): * Z test statistic for difference between two population proportions is significant at .10 level.

 ** Z test statistic for difference between two population proportions is significant at .05 level.

 *** Z test statistic for difference between two population proportions is significant at .01 level.

Source: Business Progress Reports from 331-354 firms in 196-200 ATP projects funded 1993-1996 after at least one year of funding.

Small

Firms

Larger

Firms

0

20

40

60

80

100

R&D Scope

11

18

80

70

8 10

1 2

No Project

w/o ATP

*

Increased

**

No

Effect

Decreased/

Not Sure

Percent of Firms

137 Small Firms

194 Larger Firms

0

20

40

60

80

100 Of those "Ahead" in R&D Cycle

By how much?

19 19

45

39

6 4

28

35

2 3

Percent of Firms

<1 yr 1 - <3 yrs 3+ yrs No

Project

w/o ATP

Not Sure

129 Small Firms

176 Larger Firms

20

40

60

80

100

Ability to Take on Technical Challenges

6 9

79 74

14 14

0 1 3

No Project

w/o ATP

Increased No

Effect

Decreased/

Not Sure

Percent of Firms

137 Small Firms

194 Larger Firms

Percent of Firms

40

60

80

100

Pursuit of

Long-Term Research

68

63

0

20

7 7

22

3 4

No Project

w/o ATP

Increased No

Effect

Decreased/

Not Sure

137 Small Firms

194 Larger Firms

26

Figure 6. EFFECTS OF ATP FUNDING

Comparison of small firms (<500 employees) with larger firms (> 500 employees)

 
7. Effects of ATP Funding 33

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Small Firms Larger Firms

Single

Company

Projects

Joint

Venture

Projects

All Small

Firms

Single

Company

Projects

Joint

Venture

Projects

All Larger

Firms

 Position in R&D Cycle

 Ahead

 –No project w/o ATP

 --Ahead by 1-<3 years

 --Ahead by >3 years 

97%

34%

36%

8%

81%

22%

54%

5%

88%

28%

45%

6%

95%

40%

33%

9%

81%

33%

41%

2%

85%

35%

39%

4%

R&D Scope

 No project w/o ATP

 Increased 

 6%

88%

15%

74%

11%

80%

 7%

87%

22%

63%

18%

70%

Technical Challenge

 No project w/o ATP

 Increased 

 3%

82%

 8%

76%

 6%

79%

 2%

89%

12%

68%

9%

74%

Long-Term Research

 No project w/o ATP

 Increased 

 3%

72%

10%

64%

 7%

68%

 2%

69%

9%

60%

7%

63%

Change in industry

investment

 No project w/o ATP

 Increased 

28%

60%

23%

54%

25%

57%

41%

34%

35%

31%

37%

32%

Average change in industry

investment Per Firm $613K $780K $704K $499K $296K $354K

Source: Business Progress Reports from 331-354 firms in 196-200 ATP projects funded 1993-1996 after at least one

year of funding.

Table 6. Effects of ATP Funding–by Type of Project

Percentage of firms for small firms (<500 employees) and larger firms (>500 employees)

The consistency in results for larger firms and small firms in single-company projects and again for

single-company projects and joint ventures, suggests that firms in single-company projects have

experienced a greater effect from ATP funding on acceleration of their R&D than those in joint-venture

projects. On the other hand, fewer of the joint-venture participants had completed their projects at the

time the data was collected. It is possible that joint-venture participants will experience additional

acceleration as their projects progress and joint venture start-up difficulties are overcome. 

Of those experiencing acceleration, 79% of small firms and 78% of larger firms felt they were either

ahead by at least one year or would not have pursued the project without ATP. Some minimum level

of acceleration effect is likely a direct reflection of the increased amount of R&D that can be performed

each year if ATP and industry sources of funding are combined.

Among those indicating they were “ahead,” larger firms stated more frequently that they would not have

pursued the project at all without ATP than small firms (35% of larger firms compared with 28% of

small firms). For firms of each size, percentages were higher for participants in single-company
34 7. Effects of ATP Funding

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

projects than for joint-venture participants. Percentages for larger firms in single-company projects

were the highest and nearly twice as high as the percentages for small firms in joint ventures. Larger

firms appear to be more likely than small firms to follow clear-cut decision criteria as to which R&D

projects they will undertake. They generally have more choices and are less dependent on any one

R&D path. In essence, ATP funding can serve to reduce the risk-adjusted hurdle rate on a multi-million

dollar R&D investment. As a result, ATP funding may cause a risky project may move from “out-ofbounds” to “within bounds.” 

Larger amounts of acceleration, i.e., the firm was ahead by at least three years or would not have

pursued the project at all without ATP funding, were more frequently reported by firms in singlecompany projects than in joint ventures. Percentages were again highest for larger firms in singlecompany projects. Forty-nine percent of larger firms in single-company projects reported they were

either ahead by at least three years or would not have pursued the project without ATP funding,

compared with 42% of small firms in single-company projects, 35% of larger firms in joint ventures,

and 27% of small firms in joint ventures. 

Lesser amounts of acceleration; i.e., the firm was ahead by at least one but less than three years, were

more frequently reported by firms in joint ventures than in single-company projects. (More detailed

examination of the data, not illustrated, shows further that among those reporting at least one but less

than three years acceleration, firms in joint ventures reported acceleration of less than two years more

frequently than firms in single-company projects.) Again differences may relate to the fact that singlecompany projects on average have a somewhat shorter period of ATP funding than joint-venture

projects and were slightly further along, on average, at the time of data collection. Over 30% of firms

in single-company projects had completed their projects at the time of data collection (35% of larger

firms in single-company projects and 32% of small firms in single-company projects), while just over

10% of firms in joint ventures had completed their projects (13% of larger firms and 12% of small

firms). Differences may also relate to start-up delays experienced in joint ventures which may be

overcome as the projects progress. 

7.2 Change in Scope, Level of Risk, or Ability To Take on Long-Term Research

This section examines whether the availability of federal funds through the ATP affected the

willingness of small firms and larger firms to undertake more ambitious projects than they would have

without ATP funding; i.e., to engage in projects with a broader scope, and/or projects entailing higher

technical risk or a longer R&D cycle. Parallels are drawn with the analyses of project goals reported

in Chapter 4.

In general, large percentages of firms of all sizes reported that ATP had caused an increase in “R&D

Scope,” “Technical Risk,” and/or “Long-Term Research.” Though similar percentages of small and

larger firms reported an “Increase,” as shown in Figure 6, the percentage was slightly higher for small

firms, particularly for scope and risk. The Z test statistic for differences in population proportions of

small firms and larger firms reporting an increase was significant only for “R&D Scope.” When the

responses “Increased” were combined with responses “No project w/o ATP,” Z test statistics for

differences in proportions of small firms and larger firms reporting either effect were not significant

although percentages remained somewhat higher for small firms than for larger firms, with 91% of
7. Effects of ATP Funding 35

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

small firms reporting either an increased R&D scope or “No project w/o ATP” compared with 88% of

larger firms, and with 85% of small firms reporting either increased Technical Challenge or “No project

w/o ATP” compared with 83% of larger firms.

More detailed analyses illustrated in Table 6 suggest that the responses of different firms may vary more

with the type of project they are engaged in–single-company or joint venture–than the size of the firm.

One observes that, for each firm size, the percentages of combined responses “Increased” and “No

project w/o ATP” were consistently higher for firms in single-company projects than for firms in joint

ventures – for all three characteristics, “R&D Scope,” “Technical Challenge,” and “Long-Term

Research.” 

These results seem to parallel and reinforce those reported in Chapter 4. As reported in the earlier

chapter, firms in single-company projects are more frequently pursuing “new-to-the-world” product

applications and have larger numbers of applications, on average, per firm than those in joint ventures.

Firms in single-company projects – whether small firms or larger firms – also seem more frequently

to be undertaking longer-term projects and/or projects entailing greater risk or breadth with ATP

funding than they would have undertaken without ATP funding. This suggests that the goals of these

firms for applications of their technologies are therefore more ambitious than they would have been

without the ATP awards. For joint venture projects, the ATP’s role as catalyst for collaboration may

be the more important effect. 

The ATP’s effect on project risk, and presumably then on the “ambitiousness” of new products, as well,

appears to be greatest for larger firms in single-company projects. Many small firms are pursuing

innovative technologies, the development and commercialization of which represents the core and

central purpose of the firm. They often would pursue these technologies at some level with or without

the ATP as long as the business survived. This would appear consistent with the findings above for

acceleration of R&D and with Smith et al. [1991] who suggest that larger firms face a greater variety

of R&D options than small firms for obtaining access to technologies. Larger firms may assess the

benefits and risks of each choice more carefully than small firms in making their R&D decisions.

7.3 Change in Level of Industry Investment from Internal Funds

7.3.1 Direction of Effect

Differences in the interest and willingness of small firms to finance and to apply scarce resources to

high risk and enabling research compared with larger firms could signal differences in the motivations

of the various segments of the R&D community in proposing to the ATP. As shown in Figure 6, 37%

of larger firms indicated they would not have made any investment in the project without ATP funds,

compared with 25% of small firms. (The Z test statistic for the difference in population proportions for

“Change in level of Industry Investment from Internal Funds – No project w/o ATP” was significant

at the .05 level.) These results parallel those for “Ahead – R&D Cycle” (above), but the differences

between small firms and larger firms are even greater here.

Of those firms which would have pursued the project without ATP funding, a larger percentage of small

firms than larger firms reported that they had increased their level of industry investment in the project
36 7. Effects of ATP Funding

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

(54% of small firms compared with 32% of larger firms). The Z test statistic for the difference in

population proportions for “Change in level of Industry Investment from Internal Funds – Increased”

was significant at the .01 level.) 

When firms reporting “No project w/o ATP” are combined with those reporting an “Increased” level

of industry investment, the percentage of small firms reporting one of these effects is greater than the

percentage for larger firms, despite the offsetting effects for the two sets of responses. The Z statistic

for the difference in population proportions for small and larger firms reporting “No project w/o ATP”

or “Increased” for “Change in level of Industry Investment” (82% of small firms compared with 69%

of larger firms) continues to be significant at the .01 level. (See Appendix A.) 

Looking at the results in Table 6 for the joint-venture and single-company projects separately,

differences due to firm size appear greater than differences due to the type of project. Small firms more

frequently reported either “No project w/o ATP” or “Increased” than larger firms, with the highest

percentages being reported by small firms in single-company projects (88% of small firms in singlecompany projects and 77% of small firms in joint ventures provided one of these responses, compared

with 75% of larger firms in single-company projects and 66% of larger firms in joint ventures). 

Given their financial constraints, their typical dependency on development of a single technology (or

narrow set of technologies), and their need to pursue development of the technology whether or not they

received ATP funding, one would expect small firms, on average, to invest about the same amount of

industry funds whether or not they received any ATP funding. ATP funding seems to have had a

leveraging effect in increasing private sector funding of R&D (above and beyond the ATP’s

contribution) for a substantial percentage of small firms. Small firms may have been able to attract

more private funding than they had expected, perhaps as a result of the ATP award (see Section 7.4

below), or they may have incurred more indirect costs than anticipated in scaling up the firm to

undertake the enlarged ATP project. 

The fact that small firms in joint ventures reported either “No project w/o ATP” or “Increased” more

frequently than larger firms in joint ventures suggests the joint-venture opportunity may be more critical

to small firms than larger firms in their decision to undertake an investment at all or at the level required

for the ATP joint-venture project. 

7.3.2 Estimated Magnitude of Effect on Industry Investment

The issue of the effects of ATP funding on the amount of investment industry is willing to undertake

from its own internal resources was explored further by examining estimates provided by companies

of the amount of increased investment from industry (non-ATP) sources. Small firms provided

estimates indicating that ATP funding has served to catalyze an additional $704K per firm, on average,

of industry spending in the ATP-funded areas. In comparison, larger firms reported estimates

amounting, on average, to $354K per firm. Separating estimates provided by small firms and larger

firms into those from single-company project participants and joint-venture participants, we see that

small firms in joint ventures have increased their spending from industry sources by the largest

amounts, on average, of any subgroup, followed by the small firms in single-company projects ($780K

for small firms in joint ventures compared with $613K for small firms in single-company projects and
7. Effects of ATP Funding 37

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

$499K for larger firms in single-company projects and $296K for larger firms in joint ventures). Larger

firms in both single-company projects and joint ventures increased their investments by lesser amounts

than small firms in either type of project. However, larger firms in single-company projects reported

larger increased levels of investment (above and beyond that provided by the ATP) on average than

larger firms in joint ventures. 

The finding that a larger percentage small firms than larger firms have increased their level of

investment from industry funds as a result of ATP funding is reinforced by the finding that the average

increase, per firm, is higher for small firms than larger firms. The greater effect for small firms is

somewhat unexpected given a) the tighter financial constraints on small firms, b) the lower overhead

rates for small firms, and c) the larger percentage of small firms with single-company projects than joint

ventures. (Small-firm recipients of single-company ATP awards must cover their indirect costs as a

minimum cost share, while larger firms equivalent to the Fortune 500 must cover 60% of total project

costs. Joint venture members must cover more than 50% of project costs although larger firms may

subsidize small firms as long as industry covers more than 50% of total project costs.) The results

obtained from these analyses suggest that some small firms are covering more than their minimum

requirement, and furthermore that those in joint ventures, on average, are not being heavily subsidized

by larger firms. 

These findings seem to indicate that small firms in both single-company projects and joint ventures are

more committed to and have a greater stake in their ATP projects than larger firms in either singlecompany projects or in joint ventures. Furthermore, this effect may be stronger for small firms in joint

ventures. For one thing, the ATP-funded technology area is more likely to be the strategic focus of the

small firm receiving ATP funding than for a larger firm. The 50+% minimum cost share requirement

for joint venture participants may further tend to increase the average level of investment for joint

venture participants more than for participants in single-company projects. 

The smaller levels of increased industry investment (above and beyond ATP funding) on average for

larger firms may reflect more varied degrees of commitment of larger firms to their ATP projects,

especially joint-venture projects. Some ATP-funded joint ventures have a substantial number of

partners, with varying levels of financial commitment and R&D participation among partners.

Individually, larger firms may be making relatively small financial contributions if they are primarily

performing testing and validation services or sharing the R&D costs of universities with a large number

of other firms. 

It is also possible that larger firms found it more difficult to estimate the amount of change in their

investment in the project than small firms. For example, a somewhat higher percentage of larger firms

than small firms had indicated that their level of investment had increased or would not have occurred

at all without ATP funding but then failed to provide an estimate of the change in the amount of their

investment, thus causing a downward bias in the calculated average increased investment per firm for

larger firms. On the other hand, a few small firms in joint ventures provided very large estimates. (In

two cases at least, these estimates were based on new capital investment attributable to the ATP award

that is being devoted to developing the ATP-funded technology.) If the largest two values reported by

small firms in joint ventures are removed and the mean recalculated, the average increase in investment

for this group drops significantly – to approximately $428K per firm. 
38 7. Effects of ATP Funding

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

7.3.3 Conclusion

Overall, these findings on the effects of ATP funding suggest that 1) the ATP has (as intended) a

leveraging effect both in encouraging industry to pursue more ambitious technology development

projects, aimed at more distant, future products, and in stimulating industry to increase its own internal

funding of such projects, 2) larger firms more frequently require ATP funding to make any investment

of their own in the ATP-funded project area, and 3) the increased amount of industry investment is

higher on average for small firms than for larger firms and is the highest for small firms in joint

ventures. Note, however, that this type of quantitative, counterfactual information (quantitative

estimates of what would have occurred in the absence of ATP funding) is difficult to obtain. The larger

amounts of increased investment reported by small firms likely reflect new private investment capital

raised by small firms subsequent to their ATP awards, and may be due, at least partially, to the “halo

effect” of the ATP award described below. 

7.4 Increased Credibility

An ATP award is the culmination of a rigorous peer review process involving scientists knowledgeable

in the field and experienced business specialists. The announcement of an ATP award may help to

validate the technology in the eyes of those in a position to benefit from it. For example, the award

announcement may be a positive signal as to the technology’s value to those in a position to provide

investment capital or ultimately to use the technology in manufacturing and commerce. Attraction of

investment capital or customer interest serves to improve the viability of the firm and may be

particularly helpful to young, technology-based firms.

Figure 7 illustrates the results of an analysis of the ATP “halo effect” on small firms compared with

larger firms. Small firms more frequently reported enhanced credibility with investors, with customers,

and with suppliers than larger firms. The Z test statistic for the difference in population proportions of

small firms and larger firms which reported increased credibility was significant for all three

relationships at the .01 level. The most commonly reported effect for both small firms and larger firms

was increased credibility with customers. This effect was reported by 84% of small firms compared

with 70% of larger firms. On the other hand, increased credibility with investors was frequent for small

firms, but not very frequent for larger firms, most of whom likely relied on internally-generated

company funds for financing. (Seventy-five percent of small firms reported increased credibility with

investors compared with only 24% of larger firms.) 

A high percentage of both small firms and larger firms reported increased credibility with management,

with larger firms reporting this effect slightly more frequently than small firms (85% of larger firms

reported this effect compared with 81% of small firms). In small firms, the individuals responsible for

the ATP award often “were management,” and the company president often helped manage the R&D.

In larger firms, “management” was likely to be a distinctly different group of individuals without a

direct R&D role but with control over which R&D projects the company pursued and over which R&D

jobs were maintained as R&D priorities shifted. A few small and larger firms reported that the ATP

award had actually decreased their credibility with management. In these cases, perhaps management

was not aware of the ATP project prior to the award and not committed to it; or perhaps subsequent

difficulties, for example, problems in joint venture formation, raised concerns with management.
7. Effects of ATP Funding 39

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Note (see Appendix A): *** Z test statistic for difference between two population proportions is significant at .01 level.

Source: Business Progress Reports from 353 firms in 201 ATP projects funded 1993-1996 after at least one year of funding.

0

20

40

60

80

100

75

24 25

76

Percent of Firms

With Investors

Increased

***

No Change

or Don't Know

0

20

40

60

80

100

Percent of Firms

Increased

***

No Change

or Don't Know

84

70

16

30

With Customers

0

20

40

60

80

100

69

52

31

48

With Suppliers

Increased

***

No Change

or Don't Know

0

20

40

60

80

100

81

85

3 2

16 13

With Management

Increased Decreased No Change

or Don't Know

146 Small

Firms

207 Larger

Firms

Figure 7. INCREASED CREDIBILITY

Comparison of small firms (<500 employees) with larger firms (> 500 employees)
40 7. Effects of ATP Funding

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Small Firms Larger Firms

Single

Company

Projects

Joint

Venture

Projects

All Small

Firms

Single

Company

Projects

Joint

Venture

Projects

All Larger

Firms

With Investors?

 Increased 87% 66% 75% 33% 20% 24%

With Customers?

 Increased 79% 89% 84% 74% 68% 70%

With Suppliers?

 Increased 72% 67% 69% 59% 49% 52%

With Management:

 Increased

 Decreased

85%

0%

77%

6%

81%

3%

93%

2%

81%

2%

85%

2%

Source: Business Progress Reports from 353 firms in 201 ATP projects funded 1993-1996 after at least one year of

funding.

Table 7. Increased Credibility–by Project Type 

Percentage of firms for small firms (< 500 employees) and larger firms (> 500 employees)

Table 7 presents small-firm and larger-firm results for single-company projects and joint-venture

projects separately. Increased credibility with investors was more frequent among participants in

single-company projects than joint ventures. (Among small firms, 87% of single-company project

participants reported this effect compared with 66% of joint venture participants; among larger firms,

33% of single-company project participants reported this effect compared with 20% of joint venture

participants.) Increased credibility with customers and increased credibility with suppliers were

reported more frequently by small firms in joint ventures than by small firms in single-company

projects. Small firms in joint ventures likely experienced enhanced credibility with customers and

suppliers from both their joint ventures, many of which include potential customers or improved access

to customers through various participants, and from the effect of the ATP award announcement on

outside parties. Small firms in single-company projects, many of whom are focused on short-term cash

needs for survival, were most likely to find the award announcement helpful in increasing the interest

and confidence of private investors.

Larger firms with single-company projects cited a greater “halo” effect than larger firms in joint

ventures, and the effect was most frequent with customers and with management. Larger firms in

single-company projects were perhaps pursuing R&D in areas apart from their main lines of business

and needed to attract customer interest to maintain management support.

Overall, small firms appear to be more sensitive than larger firms to the effects of the ATP award on

increasing their credibility with external business partners, perhaps because the ATP award is more

important to small firms than larger firms in enabling new relationships with external parties.

Increased credibility with management was more common for single-company project participants than

joint venture participants, and most common for larger firms in single-company projects, although

nearly as large a percentage of small firms reported it. (Among larger firms, 93% of single-company
7. Effects of ATP Funding 41

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

 project participants reported this effect compared with 81% of joint venture participants; among small

firms, 85% of single-company project participants reported this effect compared with 77% of joint

venture participants.) This finding for large firms appears complementary to that above showing that

larger percentages of larger firms in single-company projects than in joint ventures experienced

increased credibility with customers. The finding for small firms may indicate increased credibility

with owners who were not directly involved in the proposal or with investors.

The small number of larger-firm respondents who reported decreased credibility with management were

about equally divided between single-company and joint-venture participants, but the few small firms

reporting decreased credibility with management were all joint venture participants. This finding seems

to confirm that a few incidences of decreased credibility with management may occur in cases where

management is not closely involved in the proposal process and/or joint venture complications have

occurred.

7.5 Anecdotal Comments

Anecdotal comments from individual firms help provide a clearer picture of the effect of ATP funding

summarized in the statistics. Both small and larger firms describe the broader scope, riskier, and

longer-term nature of the R&D resulting from their ATP awards. Small firms seem to emphasize the

scope, breadth, and longer-term nature of the research enabled by ATP funding. Larger firms seem to

emphasize the increased amount of R&D and the riskier projects enabled by ATP funding than they

would not have explored otherwise, along with a new interest in collaboration.

7.5.1 Small Firms in Single-Company Projects

“[The ATP award] accelerated the R&D process by teaching core (research) group and

product development groups to work more closely. The firm became more R&D based during

this period, in part due to credibility of the NIST project.”

“[The ATP award] permitted the project to be brought to the level [needed] to understand the

technology’s best potential and solve many restraining technology issues.”

“This company and project would not have been funded without the ATP. ATP support allowed

us to take a very risky technology out of a university laboratory, develop it through the

prototype stage, and enlist substantial investments from outside sources to fund

commercialization.”

“The project is reaching the stage of development [needed] to facilitate the attraction of outside

investment. Having the means to begin the research and development in earnest gave [us] a

critical time advantage to adapt to changing opportunities for this technology. Finally, the

purpose of this technically high risk venture is to create a tool that will change for the better

how R&D in the life sciences is done. With funding, that goal is being achieved.”

“ATP funding has enabled [us] to advance beyond where we would have been without ATP

funding, allowing us to expend effort on new areas.”
42 7. Effects of ATP Funding

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

“Approximately half of our R&D effort over the past year has been due to the ATP. Thus in

relative terms, the effects have been enormous, very much enabling.”

“[ATP funding] has dramatically expanded both the breadth and the subsequent depth of our

R&D efforts.”

“We were a good match for the ATP program because we had a high-risk technology targeted

at a unrealized, but high potential market. We could not attract private investment because of

this risk and early market status. Now, as the market is beginning to appear and attract many

new entries we are positioned with a low-cost, innovative technology solely as a result of the

ATP.”

7.5.2 Small Firms in Joint-Venture Projects

“Without the ATP funding, we would have pursued only the short-term (quick-to-market)

solution, which would not have fully addressed the market requirements.”

“[ATP funding] allowed us to do research on alternative uses of our systems that we would not

have done without the funding. This will hopefully open up new markets.”

“ATP funding has allowed us to proceed faster with projects that are high risk and high

return.” 

“ATP funding has greatly increased our research efforts. Without ATP we would have been

forced to continue to market mature products and would have been unlikely to launch new

products in a timely enough manner to impact the marketplace.”

“It is our primary project, especially in regards to our core technology, so it gets a lot of

attention from all within our organization, from the Board of Directors to the machine

operators. It is forming the basis for all future ... R&D. It is extremely critical.”

“ATP funding has been the primary driver for our R&D efforts since the start of the project.”

“[This funding] has allowed development and insight into the real world market that could not

have been obtained otherwise.”

7.5.3 Larger Firms in Single-Company Projects

“ATP funding provided the leverage needed to justify this broad research project.”

“The ATP funding has enabled [us] to pursue very high risk research on ... long before it would

normally have been pursued.”

“ATP funding has both broadened the scope of our R&D efforts and focused our attention on

maintaining and strengthening our core competencies. We have looked increasingly at longer-
7. Effects of ATP Funding 43

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

range projects to create sustained growth in the business and looked for creative ways to

manage the risk associated with these efforts. Additionally, the ATP award has generated more

interest in multiparty projects to bring ourselves closer to the markets we serve.”

“These funds have allowed the company to explore some very risky technology that would not

have been explored otherwise. A development team has been assembled solely for this project,

thereby shortening the time of R&D.”

“ATP funding allows us to focus on the high risk, high payoff aspects of the research. Without

this funding we would probably only be able to work on short-term solutions for subsystems.

ATP funds allow us to address the entire system.”

7.5.4 Larger Firms in Joint-Venture Projects

“[ATP funding] has provided our R&D efforts with a broader scope that can quickly be

transferred to other sectors of our business.”

“[ATP funding] has provided the catalyst to focus resources in R&D.” 

“[ATP funding] has increased focus and effort, more collaboration with partners, and [speed]

to market.”

“Management is beginning to value pure R&D more than in the past. In this age, advanced

research is the only way to gain or maintain a competitive edge over competition.”

“We are increasingly becoming more interested in long-term work on higher risk R&D. We

also are concentrating on collaboration networks to share the cost associated with this type of

effort.”

“[ATP funding] has increased our overall commitment to the project.”

“[ATP funding] allowed us to develop a system that would reach a much broader market than

that originally thought possible. The risk of the ... R&D simply could not be justified by

investors outside of that cost sharing approach used through ATP.”
44 8. Progress Towards Commercialization of ATP-Funded Technologies

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

8. PROGRESS TOWARDS COMMERCIALIZATION OF ATP-FUNDED

TECHNOLOGIES

Early chapters reported that small firms are pursuing ambitious research goals aimed at “new-to-theworld” or high-performance products and that they often plan a number of different products for

different niche markets. It was further noted that larger firms are more likely than small firms to be

developing new material or process technology that can be applied to high-volume products. This

chapter explores whether small firms are overcoming the many technical, financial, and marketplace

challenges in their paths and making comparable progress towards commercialization to that being

made by larger, more established firms. (Note that the ATP can fund only R&D costs. Project

participants must bear the full costs associated with product development and other commercialization

activities. The BRS tracks early commercialization activities undertaken at the companies’ own

expense, including early, spin-off product revenues.) 

8.1 Progress of Firms Against Some Early Commercialization Milestones

Assessed after two years of ATP funding, while still in the R&D phase and not yet having achieved all

the technical goals of their projects, small firms appear to be keeping pace with larger firms or are

slightly ahead in their early stages of commercialization. Findings for a selected group of early

commercialization activities are illustrated in Figure 8. Separate analyses were performed for 1) the

percentage of all applications being pursued for which the specific activity had occurred; and 2) the

percentage of firms having completed the activity for at least one application. 

8.1.1 Progress of Small Firms Compared with Larger Firms

In producing production prototypes, setting up pilot production, beginning production, and actually

earning revenues, small firms seem to have made about the same or more progress as larger firms.

Analyses of the percentage of firms having completed the commercialization activity for at least one

application showed greater differences between small firms and larger firms than the analyses

performed across all applications. Significantly more small firms had set up pilot production or a

commercial demonstration for at least one application (54% of small firms compared with 31% of larger

firms), had begun production for at least one application (20% of small firms compared with 5% of

larger firms), and more small firms had earned product revenues (26% of small firms compared with

11% of larger firms). The Z test statistics for the differences in the population proportions of small firms

and larger firms reporting these signs of progress were significant at the .05 level for “setup pilot

production/commercial demo” and “begun production” and at the .10 level for “product revenues

earned.” 

Small firms reported a mix of “early” and “later” applications or spin-off activities stemming from

different phases of their technology development projects. Many small firms appeared to be in the

process of commercializing one or more “early” applications, sometimes through selling production

prototypes. Larger firms appear to pursue few if any “early” applications, and thus most larger firms

show less progress towards commercialization.
8. Progress Towards Commercialization of ATP-Funded Technologies 45

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

0

20

40

60

80

100

0

20

40

60

80

100

Percent of Applications

Percent of Firms

49 50

62

58

YES YES

Made a Production Prototype?

Notes (see Appendix A): * Z test statistic for difference between two population proportions is significant at .10 level.

 ** Z test statistic for difference between two population proportions is significant at .05 level.

Source: Business Progress Reports from firms in 83 projects with commercialization plans funded in 1993-1995--after two years of funding.

Setup Pilot Production/Commercial Demo

YES**

20

40

60

80

100

0

Percent of Applications

YES**

34

21

0

20

40

60

80

100

54

31

Percent of Firms

YES** YES**

Begun Production

Percent of Applications

0

20

40

60

80

100

11

3

20

0

20

40

60

80

100

5

Percent of Firms

YES

Product Revenues Earned

0

20

40

60

80

100

15

10

Percent of Applications

YES*

11

26

0

20

40

60

80

100

Percent of Firms

Small Firms (50-51)

111/122 Applications

Larger Firms (62/65)

110/112 Applications

YES YES

Percent of Applications

0

20

40

60

80

100

37 32

0

20

40

60

80

100

38

45

Percent of Firms

Adopted Process

Improvements

Figure 8. PROGRESS TOWARDS COMMERCIALIZATION

Comparison of small firms (<500 employees) with larger firms (> 500 employees) after two years of ATP

funding
46 8. Progress Towards Commercialization of ATP-Funded Technologies

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Larger firms had adopted process improvements for a higher percentage of applications than small firms

(37% of applications for larger firms compared with 32% of applications for small firms). However,

the difference in the percentage of applications for which small firms reported adopting process

improvements compared with larger firms (i.e., 5%) was not as large as might be expected given that

larger firms planned to “adopt process improvements” as a primary or secondary strategy for a far

greater percentage of applications than small firms (48% of applications for larger firms compared with

34% of applications of small firms). Furthermore, a larger percentage of small firms than larger firms

reported having adopted a process improvement for at least one application. Many small firms appear

to be implementing some degree of process improvements over the course of their projects, while

considering their application to be primarily a “product” rather than a “process.” 

The progress of small firms towards actual production and revenues confirms that many small firms are

seeking early commercial opportunities for product spin-offs. Many ATP-funded small firms recognize

the need to move aggressively to begin revenue generation and thereby improve cash flow – and also

to establish a market niche – at the earliest possible time in order to improve their financial viability.

This appears to be occurring once the technology has economic value but before the technology goals

are fully achieved or the development process completed. 

8.1.2 Progress of Small Firms and Larger Firms in Single-Company Projects Compared with Those in

Joint Ventures

Differences in progress for small firms and larger firms also reflect the different distribution of types

of projects among small firms and larger firms. As noted previously, larger firms more frequently are

joint-venture participants, and their projects tend to be longer, broader in scope, and more

infrastructural. On average, the joint-venture projects likely provide less opportunity for spin-off

products after only two years of project funding. Table 8 reports the separate analyses of small firms

and larger firms by type of project.

The results for the small companies in joint ventures are particularly interesting. Small firms in joint

ventures reported greater progress after two years of ATP funding than small firms in single-company

projects for a number of key milestones towards commercial production: having developed a

production prototype for at least one application, having set up pilot production or a commercial

demonstration for at least one application, having begun production for at least one application, and

having adopted process improvements. A higher percentage of small firms in joint ventures had

achieved these milestones than in single-company projects for all activities examined except “product

revenues earned.” Percentages were low in general for “begun production,” and zero for larger firms

in joint ventures. In all cases, percentages for small firms in joint ventures were also greater than for

larger firms in joint ventures or for larger firms in single-company projects. Small firms in joint

ventures appear to have made the greatest and/or most rapid progress for all activities considered except

for “product revenues earned.” 
8. Progress Towards Commercialization of ATP-Funded Technologies 47

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Small Firms Larger Firms

Single

Company

Projects

Joint

Venture

Projects

All Small

Firms

Single

Company

Projects

Joint

Venture

Projects

All Larger

Firms

Percentage of Applications for which participants report this activity

Made a production prototype 38% 62% 49% 48% 52% 50%

Set up Pilot Production/

Commercial Demo 23% 48% 34% 15% 25% 21%

Begun Production 7% 16% 11% 9% 0% 3%

Adopted Process

Improvements 32% 33% 32% 28% 41% 37%

Product Revenues Earned 15% 16% 15% 4% 13% 10%

Percentage of firms for which participants report this activity for at least one application

Made a production prototype 50% 75% 62% 43% 67% 58%

Set up Pilot Production/

Commercial Demo 42% 67% 54% 22% 36% 31%

Begun Production 12% 29% 20% 13% 0% 5%

Adopted Process

Improvements 40% 48% 45% 30% 43% 38%

Product Revenues Earned 29% 22% 26% 7% 13% 11%

Source: Business Progress Reports from 113 firms with commercialization plans funded 1993-1995–After two years of

funding.

Table 8. Progress Towards Commercialization–by Project Type

Small firms (< 500 employees) and larger firms (> 500 employees)

Percentages of small firms reporting they had earned product revenues were higher for firms in singlecompany projects than for joint-venture participants. This confirms that small firms in single-company

projects, which on average are younger and smaller than those in joint venture projects (see Chapter 3),

have been particularly energetic in obtaining revenues for at least some early applications or spin-off

products. This finding appears consistent with the greater urgency for these firms to generate cash flow

and some degree of name recognition.

It is possible that at least some of the differences between larger firms and small firms, particularly for

“begun production” and “product revenues earned,” relate to differences in attitudes regarding obtaining

early production and sales revenue and also regarding reporting on it to the ATP. The very small

percentages of larger firms reporting either “production begun” or “revenues earned” may indicate a

somewhat greater “disconnect” in larger firms between the individuals chiefly responsible for

performing and reporting on the ATP project and those in the manufacturing and sales divisions. Not

only may larger firms be less interested than small firms in pursuing sales of prototypes (“worth to
48 8. Progress Towards Commercialization of ATP-Funded Technologies

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Number of patents Patents per firm Percentage of firms with

at least 1 patent

Small Firms 91 .60 25%

 After one year 13 .21 11%

 After two years 47 .78 32%

 After three years/

 project end

31 1.11 39%

Larger Firms 142 .67 19%

 After one year 47 .43 15%

 After two years 63 .86 19%

 After three years/

 project end

32 1.0 31%

Source: Business Progress Reports and project close-out records.

Table 9. Patents Filed 

Small firms (<500 employees) and larger firms (> 500 employees)

bother” ratio for prototype sales may not be very high for larger firms), but also larger firms may

distinguish more clearly between pilot production and commercial production in that they do not

consider production to begin until the technology is transferred to the business units and some degree

of scale up is underway.

Overall, based on data reported after two years of ATP funding, small firms in joint ventures appear to

be making particularly rapid progress against all measures examined in Figure 8 and Table 8. A

number of factors may help account for this. The sheer ambitiousness and aggressiveness of small

firms as indicated by their goals for “new-to-the-world” and high-performance products (see Chapter

4), combined with collaboration efficiencies specific to joint-venture participation, may be having a

beneficial effect. Also, the characteristics of small firms in joint ventures may differ somewhat from

those in single-company projects. For example, small firms in joint ventures are somewhat larger, on

average, than those in single company projects (many of which are start-ups – see Chapter 3), which

suggests that they may be stronger financially and have more business experience. Perhaps jointventure partners (small or larger firms) are more willing to team with small firms which have some

level of established business credibility and recognized technological success. Further research is

needed to probe these hypotheses. 

8.2 Progress in Obtaining Patents

The final measure of commercialization progress examined was the number of patents filed. Patents

will be needed for firms to license their ATP-funded technologies and, in many cases, to provide

adequate protection of intellectual property. As noted in the literature review, patents may also be

needed for small firms to establish credibility and start building a market niche. Table 9 provides a

summary of patents filed, including the averages per firm, for small firms and for larger firms. To
8. Progress Towards Commercialization of ATP-Funded Technologies 49

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

account for the different stages of R&D across projects, separate averages were computed for firms at

different levels of project completion. The “after one year” and “after two years” subgroups of larger

firms reported more patents filed per firm, on average, than these subgroups of small firms. On the

other hand, the “after three years/project completed” subgroup of small firms reported more patents

filed per firm, on average, than the comparable subgroup of larger firms. The Z test statistics for the

differences in the population means for small firms and larger firms were not significant either for the

small and larger-firm groups as a whole or for the subgroups of firms reporting after one, two, or three

years/project end. (See Appendix B.) In most cases, the patents will not actually be granted until after

the ATP project ends. 

In the analyses just described, the intense patent activity of just a few larger firms affected the outcome

for larger firms. To help compensate, separate analyses were performed for firms in each subgroup that

had filed at least one patent. These analyses showed that a higher percentage of small firms than larger

firms had filed at least one patent – after two years, after three years/project completed, and across all

time periods. Overall, small firms appear to be taking the necessary steps to pursue their intellectual

property and licensing strategies, and small firms appear to be engaging in a comparable amount of

patent activity to larger firms despite the high cost of patenting.
50 9. Summary of Findings and Future Work

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

9. SUMMARY OF FINDINGS AND FUTURE WORK

9.1 Summary of Findings 

This study describes the participation, strategies, and progress of small firms receiving ATP awards in

the period 1993 through 1996, compared with larger firms receiving awards in that period. The study

shows that small firms have indeed been successful in winning ATP awards and suggests that the

aggressive technology development goals of small firms, often aimed at “new-to-the-world” or high

performance products for a large number of applications, is a likely reason for their success against ATP

selection criteria. 

Most ATP-funded small firms are pursuing strategies needed for ultimate commercial success and

technology diffusion; for example, they are entering into R&D collaborations with larger firms,

universities, and other research organizations, and forming strategic alliances with potential customers.

In obtaining ATP funding, they are taking advantage of opportunities for federal funding geared

specifically to ambitious technology development goals. Small firms as well as larger firms seem to

be using ATP awards to leverage internal company funding to expand the level, scope, and challenge

of their R&D efforts and simultaneously to shorten the R&D cycle in the race to bring technically

innovative products to market ahead of the competition. Small firms are progressing against major

production milestones towards commercialization at a pace at least equivalent to larger firms. Together,

these characteristics and signs of progress appear to indicate that ATP-funded small firms are pursuing

necessary “strategies for success” and are making considerable progress in early stages of

commercialization. 

Among small firms, those in single-company projects more frequently exhibit characteristics of

potentially successful firms or have achieved more progress in some areas than those in joint-venture

projects. For example, small firms in single-company projects report plans for more applications of

their ATP-funded technologies (many through patent licensing), and anticipate “new-to-the-world”

products or very high performance gains for more applications than those in joint ventures; they are

more likely to have experienced increased credibility with investors and suppliers and to have earned

product revenues for at least one application. Furthermore, the percentages of small firms in singlecompany projects reporting these activities or effects are larger than the percentages of larger firms in

either type of project reporting these activities or effects. 

On the other hand, small firms in joint-venture projects are more likely to report significant benefits

from collaboration than small firms in single-company projects (whose collaboration involved

subcontractors or informal strategic alliances rather than formal ATP joint-venture relationships), and

also appear to have made particularly rapid progress towards commercialization. Small firms in jointventure projects report “significant” beneficial effects of collaboration more frequently than those in

single-company projects in a) identifying customer needs, b) obtaining R&D expertise, and c)

accelerating market entry. (They also are more likely to report increased management and coordination

costs.) Small firms in joint-venture projects also report more frequently than those in single-company

projects that they have experienced greater credibility with customers and that they have achieved major

milestones towards commercialization, for example, completing a production prototype, setting up pilot

production or a commercial demonstration, actually beginning production, and adopting process
9. Summary of Findings and Future Work 51

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

improvements for at least one application. The percentages of small firms in joint ventures reporting

these effects or activities were larger than the percentages of larger firms in either type of project

reporting these effects or achievements. 

Larger firms tend to exhibit different characteristics than small firms. Larger firms in single-company

projects, in particular, report more frequently than small firms either that they would not have

performed the project at all without ATP or that the ATP award had enabled them to expand their R&D

scope or to undertake tasks with greater technical challenge. Similarly, they report either that their

R&D has been accelerated by at least three years or they would not have undertaken the project at all

without ATP, and that the project or research area has experienced increased credibility with

management as a result of the ATP award. 

Larger firms in joint ventures appear to play a variety of roles that are difficult to capture in summary

statistics alone. Information from other sources shows that some lead challenging and complex R&D

efforts involving a large number of small firms and larger firms, universities, and other research

organizations; others perform testing and validation services and provide requirements for the

technology as potential customers, leaving other members of their joint ventures to carry out the more

challenging research tasks. 

Evidence presented suggests that small firms are “keeping up” and engaging in collaborative and other

commercialization activities needed both for firm success and national impact. Time and more in-depth

analyses will be needed to establish whether small firms are able to achieve large economic impact in

the long run through the myriad of applications they plan for their ATP-funded technologies.

9.2 Implications for Future Work

Ex post evaluation of the long-run economic impact of ATP-funded technologies developed by small

firms, as well as by larger firms, lies principally in the future. Most of the projects ATP has funded are

still at a relatively early stage although a growing number are now completed [Long, 1999]. About 15

percent of the ATP-funded projects covered in the BRS database had been completed at the time the

data was analyzed. Many of the technologies will require considerable additional development, and

some will need to undergo FDA or other approval before commercialization in applications requiring

the level of technical capabilities envisioned in project proposals. The full economic impacts across

diverse applications and industries will typically unfold over many years after the initial ATP-funded

research. 

Thus, this study provides only the beginning of the story. To determine how successful small firms will

be compared with larger, more established firms in bringing their technologies into the marketplace,

and in achieving adoption within their industries, diffusion to other industries, and significant economic

impact, ATP will need to track its awardee companies for many years after the period of ATP funding.

Additional research also is needed to examine some of the underlying factors that contribute to the

differences between small firms and larger firms identified in this study, for example, to study

differences in the length of the R&D cycle or stage in the innovation cycle typical of ATP projects in

different technology areas and to study whether the characteristics of small firms identified in this study

reflect the large percentage of very small companies, many of which are start-ups. Additional research
52 9. Summary of Findings and Future Work

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

is needed to examine the relationships among firms of different sizes and among other types of

participating organizations across joint ventures. Nicholas Vonortas has made a start in this direction

with his research concerning the use of research joint ventures to achieve business diversification

[1999]. More time must pass before the effects of joint-venture participation on the longer-range

commercial progress and economic impact of small firms and larger firms can be adequately assessed.

This report represents part of ATP’s effort to use BRS data to investigate a subset of ATP participants;

i.e., small firms. As the BRS database grows, new types of analyses become feasible. Future work will

1) expand data collection concerning the nature and magnitude of the roles of different project

participants, particularly in joint ventures, 2) extend the BRS to post-ATP project data collection, 3)

analyze other ATP subgroups, such as different technology areas, and 4) use the BRS in conjunction

with other data sources to support a variety of evaluation studies, including case studies and

econometric analyses. A number of ATP-funded studies are underway which explore some of these

issues and techniques [Ruegg, 1998].
References 53

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

REFERENCES

Ali, Abdul; Krapfel, Robert Jr; and LaBahn, Douglas (1995). Product Innovativeness and Entry

Strategy: Impact on Cycle Time and Break-even Time, Journal of Product Innovation

Management, 12, pp. 54-69.

Gompers, Paul and Lerner, Josh (1999). Money Chasing Deals?: The Impact of Fund Inflows on

private Equity Valuations. Cambridge, MA: Harvard University and National Bureau of

Economic Research. Journal publication forthcoming.

Himmelberg, Charles P. and Petersen, Bruce C. (1994). R&D and Internal Finance: A Panel Study of

Small Firms in High-Tech Industries, Review of Economics and Statistics, 76, pp. 38-51.

Jaffe, A.B. (1996). Economic Analysis of Research Spillovers: Implications for the Advanced

Technology Program. Gaithersburg, MD: National Institute of Standards and Technology.

Jonash, Ronald S. (1996). Strategic Technology Leveraging: Making Outsourcing Work for You,

Research-Technology Management, March April 1996, pp. 19-25.

Laidlaw, Frances J. (1997). Acceleration of Technology Development by the Advanced Technology

Program: The Experience of 28 Projects Funded in 1991. Gaithersburg, MD: National

Institute of Standards and Technology, NISTIR 6047.

Lawler, Edward E. III (1997). Rethinking Organization Size, Organizational Dynamics, Autumn 1997,

pp. 24-35.

Lefebvre, Louis; Lefebvre, Elizabeth; and Poupart, Robert (1990). The Shape of the New Winner:

Innovativeness and the Strategic Edge in Small Firms, National Productivity Review, 9, pp.

313-320.

Lerner, J. (1996). The Government as Venture Capitalist: The Long-Run Impact of the SBIR Program,

NBER Working Paper 5753.

Litvak, Isaiah A. (1992). Winning Strategies for Small Technology-Based Companies, Ivey Business

Quarterly, 57, pp. 47-51.

Long, William F., Performance of Completed Projects: Status Report Number 1. Gaithersburg, MD:

National Institute of Standards and Technology, NIST Special Publication 950-1.

Maynard, Roberta (1996). Striking the Right Match, Nation’s Business, 84, pp. 18-28.

Martin, Sheila A.; Winfield, Daniel L.; Kenyon, Anne E.; Farris, John R.; Bala, Mohan V.; and

Bingham, Tayler H. A (1998). Framework for Estimating the National Economic Benefits of

ATP Funding of Medical Technologies: Preliminary Applications to Tissue Engineering

Projects Funded from 1990 to 1996. Research Triangle Park: Research Triangle Institute.

Phillips, Bruce D. (1991). The Increasing Role of Small Firms in the High-Technology Sector:

Evidence from the 1980s. Business Economics, 26, pp. 40-46.

Piper, Adrian and Lund, Melanie (1997). The Financing of Technology-based Small Firms: An

Update, Bank of England Quarterly Bulletin, May 1997, pp. 210-213.

Powell, Jeanne W. (1997). Advanced Technology Program--Development, Commercialization, and

Diffusion of Enabling Technologies: Progress Report for Projects Funded 1993-1995.

Gaithersburg, MD: National Institute of Standards and Technology, NISTIR 6098.

Ruegg, Rosalie T. (1988). The Advanced Technology Program, Its Evaluation Plan, and Progress in

Implementation, Journal of Technology Transfer, 23, Summer 1998, pp. 5-9. For information

on the World Wide Web concerning ATP’s evaluation program and studies completed and

underway, go to http://www.atp.nist.gov and click on the listing for the Economic Assessment

Office, and from there go to publications.
54 References

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Silber & Associates (1996). Survey of Advanced Technology Program 1990-1992 Awardees:

Company Opinion About the ATP and Its Early Effects. Gaithersburg, MD: National Institute

of Standards and Technology.

Slowinski, Gene; Seelig, Gerard; and Hull, Frank (1996). Managing Technology-Based Strategic

Alliances Between Large and Small Firms, S.A.M. Advanced Management Journal, 61, pp. 42-

47.

Smith, Helen Lawton; Dickson, Keith; and Smith, Stephen Lloyd (1991). There Are Two Sides to

Every Story: Innovation and Collaboration Within Networks of Large and Small Firms.

Research Policy, October 1991.

Utterback, J. (1994). Mastering the Dynamics of Innovation, Harvard Business School.

Vonortas, N. (1999). Business Diversification Through Research Joint Ventures: Advanced

Technology Program. Gaithersburg, MD: National Institute of Standards and Technology.

Winger, Alan R. (1994). Is Big Really Bad? Business Economics, 29, pp. 38-42.

Yap, Chee Meng and Souder, William E. (1994). Factors Influencing New Product Success and Failure

in Small Entrepreneurial High-Technology Electronics Firms, Journal of Product Innovation

Management, 11, pp. 418-432

NIST publications listed above are available on the Web at http://www.atp.nist.gov/atp/pubs.htm.
Appendix A. Z Test Statistics for Differences in Population Proportions 55

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Weighted Z Statistic

Variable Small Firms Larger Firms Population for Population

(Population) (Percent) (Population) (Percent) Proportion Proportion Significance

Project Goals and Expected

 Commercial Advantage

 Type of Commercialization

 Process 359 0.22 400 0.34 0.283 -3.663 ***

 Product 359 0.69 400 0.58 0.632 3.137 ***

 Commercialization Advantage Expected

 New Solution 359 0.37 399 0.31 0.338 1.743 *

 Cost, Cost & Perf., or Perf. 359 0.57 399 0.65 0.612 -2.257 **

 Performance Improvement Goals

 100-999+% 207 0.43 259 0.23 0.319 4.603 ***

 Cost reduction goals

 50-99% 207 0.38 259 0.17 0.263 5.114 ***

Strategies for Commercialization

 Licensing to Others

 Primary 361 0.38 397 0.17 0.270 6.504 ***

 Primary or Seconday 361 0.57 397 0.32 0.439 6.927 ***

 Producing Product/Service in-house

 Primary 361 0.62 397 0.66 0.641 -1.147

 Primary or Seconday 361 0.80 397 0.73 0.763 2.265 **

 Adopting Process for In-house Use

 Primary 361 0.22 397 0.34 0.283 -3.664 ***

 Primary or Seconday 361 0.34 397 0.48 0.413 -3.909 ***

 Forming Alliances with Customers

 Primary 361 0.29 397 0.18 0.232 3.581 ***

 Primary or Seconday 361 0.48 397 0.29 0.380 5.381 ***

 Forming Joint Production Alliances

 Primary 361 0.20 397 0.09 0.142 4.328 ***

 Primary or Seconday 361 0.44 397 0.18 0.304 7.773 ***

 Forming Alliances with Distributors

 Primary 361 0.21 397 0.10 0.152 4.209 ***

 Primary or Seconday 361 0.38 397 0.19 0.280 5.816 ***

Collaboration Experience

 Collaboration Experience?

 Yes 149 0.76 211 0.82 0.795 -1.389

 Was ATP Responsible?

 Great extent 112 0.66 173 0.67 0.666 -0.175

 Great or moderate extent 112 0.92 173 0.85 0.878 1.760 *

 Specific effects:

 Identify Customer Needs?

 Significant 112 0.55 173 0.38 0.447 2.819 ***

 Significant or Moderate 112 0.79 173 0.68 0.723 2.027 **

 Stimulate Creative Thinking?

 Significant 112 0.78 173 0.66 0.707 2.174 **

 Significant of Moderate 112 0.95 173 0.98 0.968 -1.410

 Attract Capital Investment?

 Significant 112 0.20 173 0.12 0.151 1.840 *

 Significant or Moderate 112 0.48 173 0.27 0.353 3.624 ***

 Obtain R&D Expertise?

 Significant 112 0.56 173 0.50 0.524 0.991

 Significant or Moderate 112 0.87 173 0.84 0.852 0.696

 Accelerate Market Entry?

 Significant 112 0.56 173 0.54 0.548 0.331

 Significant or Moderate 112 0.86 173 0.79 0.818 1.494

 Increase Management/Coordination Costs?

 Significant 112 0.11 173 0.06 0.080 1.523

 Significant or Moderate 112 0.58 173 0.49 0.525 1.486

APPENDIX A. Z Test Statistics for Differences in Population Proportions
56 Appendix A. Z Test Statistics for Differences in Population Proportions

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

Weighted Z Statistic

Variable Small Firms Larger Firms Population for Population

(Population) (Percent) (Population) (Percent) Proportion Proportion Significance

Effects of ATP Funding

 R&D Scope

 Increased 137 0.80 194 0.70 0.741 2.046 **

 No Project w/o ATP 137 0.11 194 0.18 0.151 -1.752 *

 Increased OR No project w/o ATP 137 0.91 194 0.88 0.892 0.868

 Take on Technical Challenges

 Increased 137 0.79 194 0.74 0.761 1.050

 No project w/o ATP 137 0.06 194 0.09 0.078 -1.005

 Increased OR No project w/o ATP 137 0.85 194 0.83 0.838 0.487

 Long-Term Research

 Increased 137 0.68 194 0.63 0.651 0.940

 No project w/o ATP 137 0.07 194 0.07 0.070 0.000

 Increased OR No project w/o ATP 137 0.75 194 0.70 0.721 0.999

 Position in R&D Cycle

 Ahead 146 0.88 208 0.85 0.862 0.807

 Ahead by how much?

 At least one year 129 0.51 176 0.43 0.464 1.384

 No project w/o ATP 129 0.28 176 0.35 0.320 -1.294

 At least one year OR No project w/o AT 129 0.79 176 0.78 0.784 0.210

 Change in Industry Investment

 Increased 147 0.57 207 0.32 0.424 4.690 ***

 No project w/o ATP 147 0.25 207 0.37 0.320 -2.385 **

 Increased OR No project w/o ATP 147 0.82 207 0.69 0.744 2.762 ***

Increased Credibility

 Credibility with Investors

 Increased 146 0.75 207 0.24 0.451 9.484 ***

 Credibility with Customers

 Increased 146 0.84 206 0.70 0.758 3.022 ***

 Credibility with Suppliers

 Increased 146 0.69 207 0.52 0.590 3.199 ***

 Credibility with Management

 Increased 146 0.81 207 0.85 0.833 -0.993

 Decreased 146 0.03 207 0.02 0.024 0.603

Progress Towards Commercialization

 Production Prototype

 Across all applications 111 0.49 110 0.50 0.495 -0.149

 For at least one application 50 0.62 62 0.58 0.598 0.429

 Set Up Pilot Production

 Across all applications 111 0.34 110 0.21 0.275 2.163 **

 For at least one application 50 0.54 62 0.31 0.413 2.458 **

 Began Production

 Across all applications 111 0.11 110 0.03 0.070 2.328 **

 For at least one application 50 0.20 62 0.05 0.117 2.456 **

 Adopted Process Improvements

 Across all applications 122 0.32 112 0.37 0.344 -0.804

 For at least one application 51 0.45 65 0.38 0.411 0.761

 Earned Product Revenues

 Across all applications 110 0.15 73 0.10 0.130 0.985

 For at least one application 46 0.26 46 0.11 0.185 1.853 *

Level of Significance Critical Value for Z test statistic--two-tail test

* = .10 1.645

** = .05 1.960

***= .01 2.576
Appendix 57

Business Planning and Progress of Small Firms Engaged in Technology Development Through the Advanced Technology Program

APPENDIX B. Z Test Statistics for Differences in Population Means

Variable Mean

Small

Firms

n

Small

Firms

s2

Small

Firms

Mean

Larger

Firms

n

Larger

Firms

s2

Larger

Firms

Z

Applications

Average Number of 

Applications Per Firm--

 All Projects

2.346 153 2.583 1.923 208 1.521 2.719

***

Average Number of

Applications Per Firm--

 Single-Company Projects

2.616 73 3.073 2.155 58 1.783 1.708

*

Average Number of

Applications Per Firm--

 Joint Ventures

2.1 80 2.041 1.833 150 1.402 1.430

Patents

Average Number of

Patents Per Firm

0.603 151 1.948 0.666 213 10.10 -0.257

Average Number of 

Patents Per Firm--

 After One Year

0.206 63 0.586 0.435 108 2.36 -1.297

Average Number of 

Patents Per Firm--

 After Two Years

0.783 60 2.444 0.863 73 23.17 -0.134

Average Number of 

Patents Per Firm--

 After Three Years or

 Close-out

1.107 28 3.433 1 32 6.710 0.186

Level of Significance Critical Value of Z Test Statistic–two-tail test

* = 0.10 1.645

** = 0.05 1.960

***= 0.01 2.576