The Role of Demonstrations in Federal

R&D Policy

July 1978

NTIS order #PB-284387
Library of Congress Catalog Card Number 78-600070

For sale by the Superintendent of Documents, U.S. Government Printing Office

Washington, D.C. 20402 Stock No. 052-003-00557-3
FOREWORD

The Federal Government now spends about $28 billion per year on research and

development (R&D) activities and facilities in the United States. With another $20 billion

per year from the private sector, the total national investment in R&D approaches $50

billion annually.

Large though this total is, it portrays only a small portion of the overall impact of

R&D on the economy and the quality of life in our society. Research and development

is the engine that drives the currents of change in our civilization. From R&D stem the inventions, techniques, and processes that propel innovations through our economic and

social system. Moreover, it has been estimated that, on the average, each person engaged in R&D eventually generates 6 to 10 other jobs throughout the economy. As a

consequence, the $48 billion annual national investment in R&D has a massive

multiplier effect on our entire socioeconomic system.

Therefore, it behooves Congress to consider this investment carefully and pay close

attention to the ways in which it is allocated and used, as well as to the framework of

laws, regulations, incentives, and constraints whereby the fruits of scientific research and

development are converted into operational results.

Furthermore, R&D and the process of innovation help to determine the options and

establish many of the parameters whereby specific technologies can be assessed for their

potential impacts on society. In assessing a particular technology, the Office of

Technology Assessment (OTA) compares its advantages and disadvantages with those

of alternative technologies and assesses its impact on economic, social, environmental,

and political factors within a perspective of probable future human needs, capabilities,

and values.

To carry out its assessments effectively, OTA needs a thorough understanding of

the Nation’s R&D effort and of the process whereby R&D results are converted into

useful innovations. While helping to strengthen and integrate OTA’S overall assessment

activities, such understanding also enables OTA to assist Congress in better shaping the

national investment in R&D by developing more soundly based R&D policies and

priorities. Thus through such understanding, OTA can more effectively fulfill its mandate

to give Congress early indication of the impacts of technological change.

In response to these needs and the urging of a number of congressional committees

and individual Members, the OTA Board authorized a Program of R&D Policies and

Priorities, which became operational in May 1976.

Recognizing that such an assessment cannot be carried out effectively through a

single, comprehensive project which attempts to address all facets of the problem, the

Program was designed to proceed through a series of manageable, interrelated studies

which will help to build an understanding of how to maximize the beneficial impacts of

our total R&D enterprise.

The Program has operated with the guidance of three interrelated Advisory Panels

made up of distinguished leaders of science, technology, industry, labor, the professions, and the consumer, environmental, and public interest movements.

The Panel on the Health of the Scientific and Technical Enterprise, chaired by Dr.

Harvey Brooks, Benjamin Peirce Professor of Technology and Public Policy at Harvard

University, has been concerned with ways we can maintain and enhance the health and

vitality of the entire scientific and technical enterprise.

. . .

111
demonstrations which have

RUSSELL W. PETERSON

Director

Office of Technology Assessment

iv
OTA R&D Policies and

Priorities Program Staff

Eltis Mottur, Assistant Director, OTA and Program Manager

John H. Young, Project Director

Judith Angerman Dorothy S. Poole

Contractors

The body of this report was prepared by:

Thomas K. Glennan, Jr.

William F. Hederman

Leland L. Johnson

Richard A. Rettig

of the RAND Corporation

OTA Publishing Staff

John C. Holmes, Publishing Officer

Kathie S. Boss Joanne Heming
Program on R&D Policies and

Priorities Steering Committee

Russell W. Peterson

Director

Office of Technology Assessment

Jerome B. Wiesner

Chairman

Technology Assessment Advisory Council

Lewis M. Branscomb

Chairman

Panel on the Applications of Science and

Technology

Application

Technology

Preston T. Bankston

Deputy Director, Office of Science and Technology

Mississippi Fuel and Energy Management Commission

Harvey Brooks

Chairman

Panel on Health Of the Scientific and

Technical Enterprise

Gilbert F. White

Chairman

Panel on Decision making on R&D Policies and

Priorities

Ellis Mottur

Ex Officio

of Science and

Advisory Panel

Lewis M. Branscomb, Chairman

Vice President and Chief Scientist, IBM Corporation

Alice Tepper Marlin

Executive Director

Council on Economic Priorities

Barry R. Bloom

President, Central Research

Pfizer, Inc.

Irving Bluestone

Vice President, Director

United Auto Workers

General Motors Division

Edward E. David, Jr.

President

EXXON Research and Engineering Company

Charles J. Hitch

President

Resources for the Future, Inc.

C. Lester Hogan

President and Chief Executive Officer

Fairchild Camera and Instrument Corporation

Claire Nader

Independent Consultant

Arthur S. Obermayer

President

MOLECULON Research

Robert M. Solow

Institute Professor

Corporation

Massachusetts Institute of Technology

Philip H. Trezise

Senior Fellow

The Brookings Institute

Herbert F. York

Professor of Physics

University of California at San Diego

vii
SUMMARY

Demonstration projects have become increasingly popular as innovative responses to a broad

spectrum of national problems. Federal expenditures for demonstration projects, including social

program demonstrations, have grown to over $1

billion annually, and further growth appears

likely. Yet their effectiveness has been limited.

Demonstrations are frequently used in areas,

such as energy, where there is controversy concerning what constitutes desirable and timely innovations. To effectively evaluate proposed and

ongoing demonstrations in such areas, Congress

must be able to develop a set of realistic expectations for a demonstration’s outcome. In order to

provide this capability, the present report

develops:

● A perspective that permits analysis of

demonstrations in many policy contexts;

and

● Guidelines to aid the evaluation of individual proposals for demonstration projects.

The perspective and guidelines are derived

from analyzing the lessons learned in both social

and hardware demonstrations.

Perspective

The perspective developed here encompasses

both the different purposes of demonstration

projects and the principal factors that determine

success.

What is a Demonstration?

The term “demonstration” is ambiguous. It

can mean to test an innovation for the purpose of

formulating national policy. Or, it can mean to

show others the relative advantage of an innovation for the purpose of persuading them to use it.

We define a demonstration here as:

A project in which an innovation is operated

at or near full scale in a realistic environment in

order to (1) formulate national policy, and/or (2)

promote the use of an innovation.

The term “innovation” may refer to a new program, product, or process.

30-78- 2

Policy-Formulating Demonstrations

We refer to the first type of demonstration

project as a policy-formulating demonstration. Federal decisionmakers are its principal audience. The income maintenance experiments

that examine the administrative feasibility, costs,

and impacts of a variety of income transfer programs provide an example of this type of demonstration. Demonstrations that provide the necessary technical or economic information for setting

regulations and standards are also a type of

policy-formulating demonstration. The Refan

program to reduce noise from commercial jet aircraft and inform regulatory decisions of the

Federal Aviation Administration is an example.

Typically, policy-formulating demonstrations are

intended to provide information to Federal decisionmakers about:

● Technological and administrative feasibility

of instituting a policy or adopting an innovation.

● Expected economic, environmental, and

social impacts of the policy or innovation.

Policy-Implementing Demonstrations

We refer to demonstrations to promote the use

of an innovation as policy-implementing

demonstrations. Solar heating demonstrations

are an example. The criterion of success for this

type of demonstration is diffusion of the innovation from the demonstration site. Thus, those

non-Federal decisionmakers who control the rate

of diffusion of an innovation are the principal audience for these demonstrations. In addition to

the factors mentioned above for policy-formulating demonstrations, policy-implementing demonstrations are typically intended to provide information on:

●

●

●

●

Costs of adopting and using the innovation.

Reliability of that innovation in use.

Demand for the innovation.

Feasibility of implementing the innovation

at the adopter’s site.

The common denominator of both types of

demonstrations is the generation of information

for decisionmaking, and a single project may inix
corporate elements of both types of demonstrations.

Who Expects What From a Demonstration?

Demonstrations often serve important political

as well as information functions, such as providing:

● A compromise between those who prefer a

large-scale operating program and those

who prefer nothing.

● A means of expressing concern for a national problem.

● A response by executive agencies to

pressures to show the usefulness of their

R&D program.

Furthermore, because demonstration projects

are an instrument of transition from R&D to use,

an innovation may be simultaneously moving:

FROM

Small scale in the laboratory

Control by R&D personnel .

Technical criteria of success.

Federal management . . . . .

TO

. . Full scale in the field;

. . Control by operating

personnel;

. Institutional criteria of

success;

. . Private sector management

or State and local management.

The range of different but limited perspectives

involved makes a transition such as this difficult

to bring about and complicates the evaluation of

proposed demonstrations.

All of these factors imply that various interested parties often have different objectives

and expectations from a demonstration project.

Some may view it primarily as a test of an innovation; others may view it as a promotion of

an innovation; still others may view it primarily as

a means of expressing concern for a national

problem.

These different objectives and expectations

make the evaluation of a demonstration difficult

and necessarily judgmental. Nonetheless, such

judgments can be informed by realistic expectations of a demonstration’s outcome.

Institutional Environment, Technology,

and Success

There are two essential requirements for a successful demonstration project: (1) clearly specifyX

ing the relevant information to be generated, and

(2) generating reliable information once specified. The specification of the relevant information

depends largely on the institutional environment; reliability of the information generated

depends largely on the technology (pp. 22-27).

An institutional environment is characterized

by the users of an innovation, its suppliers,

markets, and regulators of those markets. A welldeveloped institutional environment has two

characteristics—a tradition of using the results of

R&D, and an accepted Federal role. In such an

environment there is generally consensus among

the various participants as to what constitutes a

desirable innovation and when a demonstration

is appropriate. An example of such a welldeveloped institutional environment is the

agricultural sector with its system of experiment

stations, land-grant colleges, and extension

agents. Conversely, the components of the institutional environment in the elementary and

secondary education system are unevenly

developed, poorly linked, and frequently in

disagreement over the process of change in the

schools. In the public sector, there is often a lack

of consensus—and even fundamental value and

goal conflicts—regarding the desirability of innovations. In the private sector, the discipline of

the marketplace tends to force a greater degree

of consensus as to what constitutes a desirable innovation.

The term “technology” refers here to

knowledge for the production and delivery of

goods and services. Some technologies, such as

central-station electric power generation, are well

developed, with well-characterized inputs and

outputs and a good understanding of their relationship; other technologies are poorly developed, as in education and law enforcement.

Demonstrations employing poorly developed

technologies are less likely to be perceived as

generating reproducible results than those

employing well-developed technologies.

The Federal role in a given policy sector determines the scope of the Federal R&D effort in that

sector, and consequently, the role of demonstrations (pp. 9-15). In many areas of domestic

policy a major Federal role is relatively new and

often controversial. In areas where the Federal

role is still controversial, and it is difficult either to

reach consensus as to relevant outcomes or to
generate reliable information, the effectiveness

and role of demonstrations remain to be clearly

determined. The experience in agriculture suggests that an extended period of time is required

for an effective and accepted Federal role to be

established.

Guidelines for Evaluating Proposed

Demonstration Projects

The following questions focus attention on key

factors that influence success in demonstration

projects (pp. 31-41). The brief discussions attempt to provide the necessary perspective by

which one can evaluate critically the answers to

these questions.

1. Are the goals for a demonstration project clearly articulated and agreed

upon?

A divergence of goals and expectations among

funders, performers, and potential audiences

may often be inevitable. Nonetheless, to avoid

an inadvertent lack of clarity in goals, an effort

should be made to ensure that program managers and performers of demonstrations share a

common understanding of the purposes of

demonstrations. Specifying how the results of a

demonstration are to be evaluated can be an effective device for clarifying goals.

2. Given the purposes of a demonstration project, ‘have the information

needs of the demonstration’s audience been adequately considered?

A demonstration may be intended to inform a

congressional debate on a welfare system. Or, a

demonstration may be intended to promote the

commercialization of a new energy technology.

In either case, the demonstration should be

designed to address the critical issues upon which

the decisions will turn, as perceived by the relevant decisionmakers themselves.

This assumes, of course, that the necessary

decisions can be effectively informed by a

demonstration. For example, policy-implementing demonstrations by themselves have been

weak means for bringing about institutional

change. In such cases, other measures, where

available, should be considered as alternatives or

complements to demonstration projects (pp.

48-51) .

3. If the demonstration is intended to

promote the diffusion of an innovation, have key actors in the institutional environment been involved?

It is not only important that an institutional environment be sufficiently well developed to generate a consensus on the criteria for successful innovations. Diffusion of an innovation is also

enhanced by the actual involvement of an institutional environment’s key actors in the planning of

a policy-implementing demonstration.

An illustrative example is provided by a

demonstration of mechanized refuse collection

aimed at reducing labor costs through smaller

collection crews. The mechanism was designed

by city personnel, but no garbage truck outfitting

firm was involved in the demonstration. Despite

striking success at the demonstration site, no

commercial firm could be found to market the innovation, and there has consequently been little

diffusion.

4. Is the technology incorporated in a

demonstration sufficiently reproducible to be credible?

Projects incorporating poorly developed technologies seldom lead to similar operations in

other locations. For example in education,

significant innovation appears to occur at a site

only when there are major adaptations in the innovation to meet local needs. The apparent requirement for a unique implementation of an innovation at each potential adoption site tends to

preclude its replication elsewhere.

Support for local problem-solving efforts

where diffusion is not a relevant success criterion

might be better termed “subsidized local development” than a demonstration project. Although

few such efforts have survived withdrawal of the

Federal subsidy, such support might still be useful

where a genuine commitment to address real

problems can be distinguished from mere opportunism in response to available funds.

Extensive adaptation of an innovation to meet

specific local needs may also occur where

technologies of substantial reproducibility are

used in a larger “system .“ Examples include mass

xi
transit, law enforcement hardware, and mining.

For technologies of low to intermediate reproducibility, successful replication at multiple sites

may be necessary to stimulate diffusion.

5. Is the technology being demonstrated

well in hand?

Even though a technology may be potentially

reproducible with great reliability, its development may not have proceeded to the point

where it is well in hand. There is persuasive

evidence that attempts to demonstrate a

technology that is not well in hand adversely affect diffusion.

For example, extensive development work

was required during the demonstration of a

desalination process in Freeport, Tex. Although

this work led to important improvements in performance, it also caused interruptions in plant

operations. Potential adopters of desalination

plants mistakenly perceived these interruptions

as an indication that the desalination process was

unreliable, and there has been no diffusion.

This example illustrates the importance of considering the alternative of a full-scale test at a testbed facility when a technology is not yet well in

hand. Such testing would avoid both conveying

the adverse impression of unreliability to potential adopters and facilitate engineering solutions

to technical problems by removing real-world

operating constraints.

6. Is there sufficient time and operational flexibility for the demonstration

to meet its objectives?

Strict time constraints generated by policy or

political needs have seriously impaired demonstrations from achieving diffusion success.

, operational flexibility is essential for

coping with unanticipated difficulties that fre-

** appear in such projects.

7. Is there sufficient evidence of commitment to the innovation by the performer?

Cost sharing and initiative by non-Federal participants in a policy-implementing demonstration

provide an important test of user need. It maybe

difficult to distinguish opportunism as a response

to available funds from a commitment to address

real problems without a significant measure of

cost sharing.

Failure of the private sector to assume a

substantial portion of the costs and risks is itself

informative, and may indicate the technological,

market, institutional, or environmental uncertainties are too high. A demonstration would then be

premature without prior steps to reduce such

uncertainty.

Alternatively, the innovation may simply be

uneconomic. Failure to commercialize would

then have nothing to do with the lack of

knowledge that could be produced by a

demonstration project. In such a case, policies

that change the incentive structure facing the

potential innovator should be considered as an

alternative to a demonstration project, or the

commercialization effort should be delayed until

further R&D or changed economic conditions

make it more attractive.

8. Does the design of the demonstration

project reflect the experiences of past

demonstrate ions?

Any well-conceived and well-executed demonstration project strives to reduce uncertainty in

the various dimensions of a problem. However,

the failure of a demonstration may highlight

dimensions of a problem whose significance was

not adequately appreciated. The insights gained

can be used to reformulate national policy or be

incorporated into the design of future demonstrations.

For example, at the time of the Atomic Energy

Commission’s first power reactor demonstrations, electric utilities could not obtain liability insurance against nuclear accidents. The recognition of this barrier to the diffusion of nuclear

power led to passage of the Price-Anderson Act

which imposed a legal limit on damages that

xii
could be claimed after a nuclear accident. By Conclusion

creating a more favorable institutional environment, further demonstrations were able to suc- Despite their obvious potential, demonstraceed in promoting the diffusion of nuclear power. tions can be easily misused. The perspective and

Further demonstrations by themselves, however, guidelines developed here are to help Congress

would have been unable to overcome this barrier more effectively utilize this important policy inwhich had nothing to do with the lack of informa- strument for pursuing national goals.

tion that could be produced by a demonstration

project.

acid
CONTENTS

Chapter Page

I.

II.

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Purposes of This Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ........4

Conceptual Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ........5

Review of the Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .....5

Guide lines for Congress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .....5

Format of theStudy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ........6

The Historical Contex tof Demonstrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

The R&D Tradition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ........9

Agriculture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ........9

National Defense. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......10

Basic and Applied Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

Domestic Public Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......11

Regulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......12

Energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......13

TheSocialActionTradition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..14

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......15

III. A Conceptual Frame work for the Consideration of DernonStration

Projects in Federal RaDPolicy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

The PurposesofDemonstrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......19

Demonstrations intheR&D Tradition . . . . . . . . . . . . . . . . . . . . . . . . . . . .......19

Demonstrations aslnstruments inthe political process . . . . . . . . . . . . . . . . . . . . ..21

General FactorsAffectingthe SuccessofDemonstrations . . . . . . . . . . . . . . . . . . . . 22

TheTechnology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......22

The institutional Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24

IV. Factors Affectingthe Success ofDemonstrations . . . . . . . . . . . . . . . . . . . . . 31

Factors Affecting the Success of Policy-Implementing Demonstrations . . . . . . . . . , .31

Technologies With Low Reproducibility. . . . . . . . . . . . . . . . . . . . . . . . . . .......31

Technology ThatIsWellinHand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......33

Demonstrationsin Well-Developed Institutional Environments. . . . . . . . . . . . . . . .34

Requirementsfor Cooperative Efforts . . . . . . . . . . . . . . . . . . . . . . . . . . . .......34

User Need. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......35

Time and Operational Flexibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......36

Factors Affecting the Success of Policy-Formulating Demonstrations . . . . . . . . . . ...37

Perceived Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......38

Perceived Relevance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. ....38

Continuing CloseTiesWith Policymakers . . . . . . . . . . . . . . . . . . . . . . . . .......40

Including PointsofConflict in Demonstration Design . . . . . . . . . . . . . . . . . . .....40

V. ImplicationsforCongress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Reasons for Use ofDemonstration Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...45

xv
CONTENTS—continued

Chapter Page

The Demonstration Project as a Political Tool. . . . . . . . . . . . . . . . . . . . . . .......45

The Absence of Alternatives to Demonstration Projects . . . . . . . . . . . . . . . . . . . . .45

The Demonstration as an Instrument of Transition: Inherent Difficulties . . .......46

Incentives for Government Agencies to Use Demonstrations . . . . . . . . . . .......46

Implications for Congress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. ..47

Alternative Strategies to Promote the Use of R&D . . . . . . . . . . . . . . . . . . . . .. ...,48

Change Market Incentives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . , ..49

Modify the Institutional Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......49

Subsidize Local Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Utilize Existing Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. ...50

Suggested Questions To Be Examined in Congressional Review of

Demonstration Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .51

The Future Useof Demonstration Projects . . . . . . . . . . . . . . . . . . . . . . . . .......53

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

xvi
Chapter I

INTRODUCTION
Chapter I

INTRODUCTION

Demonstration projects have become increasingly important in Federal Government research

and development (R&D) programs. They are

used in such diverse areas as energy, transportation, environmental quality, health, water resources, aeronautics, education, and income

maintenance. The resources allocated to demonstrations are substantial. One study estimated

that approximately $625 million of the Federal

civilian R&D budget was allocated to demonstrations in FY 1974; 1 an update of that estimate

placed the funds for demonstrations at roughly

$860 million for civilian R&D in FY 1977.2

Another study estimated funds of $400 million

for social program demonstrations in FY 1976. 3

Reliable data for prior years are not available; it is

clear, however, that Federal support for demonstrations has increased at a rapid rate and that

continued increases in funding are likely.

It might be reasonable to assume from their

apparent popularity that demonstrations are a

well understood and highly effective instrument

of Federal policy. But this is not the case.

Demonstrations are poorly understood and their

effectiveness is open to question.

One reason for our limited understanding of

demonstrations is the lack of an agreed upon

‘Walter S. Baer, Leland L. Johnson, and Edward W.

Merrow, Analysis of Federally Funded Demonstration Projects: Final Report, The RAND Corporation, R- 1926 -DOC,

April 1976, p. 2. Hereafter, this study will be cited as

Federal Demonstrations. A supplementary volume of case

studies is Waker Baer, C. Johnston Conover, Cheryl Cook,

Patricia Fleischauer, Bruce Goeller, William Hederman,

Leland Johnson, Edward Merrow, Richard Rettig, and John

Wirt, Analysis of Federally Funded Demonstration Projects:

Supporting Case Studies, The RAND Corporation, R-1927-

DOC, April 1976. This volume will be cited as Federal

Demonstrations: Case Studies.

‘See Walter S. Baer, Leland L. Johnson, and Edward W.

Merrow, “Government-Sponsored Demonstrations of New

Technologies.” Science, Vol. 196, May 27, 1977, p. 951.

3

See the Report of the Study Committee on Social R&D,

The Federal investment in Knowledge of Social Problems,

The National Academy of Science, Washington, D. C.,

forthcoming, 1977.

definition. An analyst, several years ago, referred

to “the ‘demonstration-research’ project as the

major instrument for social planning in American

communities today. ”4 Another analyst of social

programs distinguished among experimental, developmental, and demonstration projects, defining the latter as aimed at “showing administrative

and/or political feasibility. ”s A forthcoming study

of social research and development by the National Academy of Sciences (NAS) defines a

demonstration as “a small-scale program undertaken in an operational setting for a finite period

of time to test the desirability of a proposed

course of action .“s A recent study of energy

policy defined the purpose of energy technology

demonstrations as “providing hardware and

nonhardware information with sufficient reliability and credibility to inform commercial utilization

decisions.’” Last, the most extensive empirical

study to date of demonstrations referred to “activities undertaken at sufficient scale so that

results can be easily translated into regular commercial operations” and distinguished this from

pilot plant and field test activities that “involve

operation on a smaller scale to determine technical feasibility, to identify major problem areas,

and to provide early estimates of costs. ”8

The various usages of demonstrations reflect

semantic ambiguity about the term. There are,

however, two principal meanings that underlie

most discussions. The first is that demonstrations

are intended to prove: to test, validate, and

prove the innovation under consideration. The

second is that demonstrations are intended to

show others the relative advantages of an in-

‘Martin Rein, Social Policy: Issues oj Choice and

Change, Random House, New York, 1970.

‘Walter Williams, Social Policy Research and Analysis:

The Experience in the Federal SoCml Agenc~es, American

Elsevier Publishing Company, New York, 1971, pp. 53-54.

‘The Federal Investment in Knowledge O) Socia) Problems, op. cit.

‘Don E. Kash et al , Our Energy Future, The University

of Oklahoma Press, Norman, Okla., 1976, p. 25

‘Federal Demonstrations, p. 19.

3
novation for the purpose of persuading them to

use it. Incorporating both these meanings into

our analysis, we have adopted the following

definition:

A demonstration is a project, involving an innovation and operated at or near full scale in a

realistic environlnent, for the purpose of (1) formulating national policy or (2) promoting the use

of the innovation.

The term “innovation, ” as used in this study,

may refer to a new program, product, or process.

The “use of an innovation” in our definition encompasses the stages of adoption, implementation, and incorporation.

The ways that Congress has provided statutory authorization for demonstrations indicate the

range of their intended uses. Frequently, a broad

authorization for R&D activity, without specific

reference to demonstrations, has provided the

basis for agency demonstration projects. The

Maritime Administration, for example, has conducted its ship development and construction

demonstration program under authority to conduct “research and development activities. ” A

second pattern has been congressional authorization of demonstrations for which the objectives

have been broadly defined. The Bureau for

Education of the Handicapped, for instance, has

broad authority to support “demonstrations

relating to education of handicapped children”

and “demonstrations relating to physical education or recreation for handicapped children. ”

Congress has also authorized demonstrations

and specified their objectives. The solid waste

demonstration authority of the Environmental

Protection Agency is for “the development and

application of new and improved methods of collecting and disposing of solid waste” as well as

“processing and recovering materials and energy

from solid waste. ” Congress has on occasion provided specific authority for particular demonstration projects. In 1956, for instance, Congress

authorized the “construction, outfitting, and

preparation for operation . . . of a nuclearpowered ship, ” which became the IV. S. Sauannah. Nevertheless, the two primary meanings of

proving and of showing are clear, even from

these various statutory formulations.9

————.——.

Vhis material is drawn from an analysis by Christopher J.

Conover of The RAND Corporation, “Federal Demonstration Projects: Statutory Language to Fund Demonstration

Projects, ” prepared for the Federal Demonstrations study.

4

More important, perhaps, than the absence of

clarity about the meaning of demonstrations is

that the results of their use have been discouraging. Some of the main conclusions of one recent

study, for example, include the following:

“demonstration projects have a narrow scope for

effective use; “ “demonstration projects appear to

be weak tools for tackling institutional and

organizational barriers to diffusion ;“ “large demonstration projects with heavy Federal funding

are particularly prone to difficulty. ”lo From other

sources, come additional criticisms. 1

1 Demonstrations, it is argued, provide little generalized information because they are often characterized

by poor research designs. They seldom are

replicated beyond their initial sites, nor do they

often lead to commercialization. Many conclude

that demonstrations provide little additional information that could not be obtained more inexpensively by other means.

Thus, Congress has good reason to be concerned with demonstrations. First, because Congress appropriates funds for demonstrations, the

substantial amount of resources invested in them

is in itself reason for congressional attention.

What is the Federal Government receiving for its

money? What might be better alternative uses of

these funds? Second, because Congress provides

authorization for demonstrations, an understanding of their appropriate uses and limits is needed

for informed decisionmaking on new initiatives.

Should authorization be broad or specific, providing substantial or limited administrative discretion? Should demonstrations be authorized by

themselves or in relation to other policy instruments and tools? Third, because statutory

authority is reviewed regularly and performance

of programs assessed periodically through the

reauthorization and oversight processes, criteria

for the review and evaluation of demonstrations

can be helpful. When should demonstrations be

employed? How should they be managed? What

are the most likely predictors of success?

Purposes of This Study

● To develop a conceptual framework for the

analysis of demonstration projects. ———

‘“Federal Demonstrations, pp. v, vi.

“See, for example, Alice M. Rivlin and P. Michael Timpane (eds.), Planned Variation in Education: Should We

Giue Up Or Try Harder?, The Brookings Institution,

Washington, D. C., 1975.
● To review and synthesize the literature on

demonstrations in relation to this conceptual framework.

● To draw out implications of the analysis for

congressional action.

Each of these purposes deserves further comment.

Conceptual Framework

Demonstrations, as is indicated above, have

been used both with physical technologies and

social programs. Yet no study has attempted to

analyze hardware and social demonstrations

together. One objective of the conceptual framework developed below is to permit the analysis of

demonstrations in many policy contexts, including both physical and social technologies. A

related objective is to determine whether there

are lessons that can be learned from the use of

hardware demonstrations that have utility for

social demonstrations, and uice uersa. Last, an

objective of this study has been to understand the

contribution of demonstrations to the different

stages of the policy process by distinguishing between those that generate information useful for

formulating national policy and those undertaken

for the purpose of promoting the utilization of a

technology.

Review of the Literature

The literature that constitutes the basis for this

study is indicated in the bibliography. It falls into

four principal categories. First, there are two

studies that focus directly upon demonstrations

as policy instruments, both done by the RAND

Corporation. One of these is the Federal Demonstrations study, cited above, performed for the

Department of Commerce. The other is the

“change agent” study of educational demonstrations conducted by Berman and McLaughlin for

the Office of Education. 12 The second category of

literature consists of two retrospective analyses of

specific social experiments, both sponsored by

“See Paul Berman and Milbrey Wallin McLaughlin et al.,

Federal Programs Supporting Education Change, Vols. l-V,

The RAND Corporation, R-1589 l-HEW, September 1974

and R 1589 2-5-HEW, April 1975,

The Brookings Institution. ’3 One, by Pechman

and Timpane, reviews the New Jersey negative

income tax experiment. 14 The other, by Rivlin

and Timpane, reviews the experience with

planned variation in education. 15 Third, there is a

general literature on R&D utilization and commercialization that is relevant to demonstrations

but in an indirect way. The forthcoming NAS

report on social R&D is a case of this type. 16

Finally, there are prescriptive analyses of R&D

issues. The analysis by Kash, et al., of U.S.

energy options illustrates this type of analysis. 1’

In this report, we draw upon the literature to support or challenge propositions that emerge from

our conceptual framework. Thus, this study is

not a conventional literature review but an

analytical interpretation and extension of the

literature.

Guidelines for Congress

Previous studies of demonstrations have

directed their policy recommendations primarily

to the Federal agencies sponsoring demonstrations. Congress, however, has responsibilities for

the authorization of demonstrations, appropriation of funds to support them, and review and

evaluation of program performance. An important purpose of this study, then, is to analyze the

literature and experience of demonstrations i n

the context of congressional concerns.

‘

3 For the purposes of this study, we do not distinguish

between social experiments and social demonstrations. The

major difference between these two types of projects lies in

the nature of the evaluation by which the effects are determined rather than in the fundamental purposes they serve.

See Henry W. Riecken and Robert F. Borouch (eds.),

Social Experimentation: A Method for Planning and

Evaluating Social lnteruention, Academic Press, New York,

1974, chapter 1.

‘*Joseph A. Pechman and P. Michael Timpane (eds. ),

Work incentives and Income Guarantees: The New Jersey

Negatiue Income Tax Experiment, The Brookings Institution, Washington, D. C., 1975.

“Alice M. Rivlin and P. Michael Timpane (eds. ), Planned

Variation in Education: Should We Give Up Or Try

Harder?, The Brookings Institution, Washington. D. C.,

1975,

“See also Arthur D. Little, Inc., Federal Funding of

Civilian Research and Development, Volume 1: Summary,

Washington, D. C., February 1976.

“Don E. Kash et al., Our Energy Future, University of

Oklahoma Press, Norman, Okla., 1976.

5
Format of the Study

Chapter 11 includes a brief overview of the

historical context from which demonstrations

have evolved. Chapter 111 develops a conceptual

framework for analyzing policies for usin g

demonstrations. This framework is used in chapter IV to develop several propositions concerning

factors likely to affect the success of demonstrations. In the concluding chapter, chapter V, we

trace the implications of the analysis for Congress.

As this report will make clear, experience and

the research literature provide no infallible guides

to good policy concerning demonstrations. Problems that arise from using demonstrations are the

result of the institutional complexity of the public

and private sectors and the workings of a democratic government. We hope that a treatment of

this complexity will be helpful to Congress and

others for future action.

6
THE

Chapter II

HISTORICAL CONTEXT

OF DEMONSTRATIONS
T H E I I

Demonstrations as an explicit policy instrument have emerged from two distinct traditions.

They have evolved, first, within the context of

the Federal Government’s support of R&D and,

second, in the context of the Federal social action

programs of the 1960’s. It is both appropriate

and useful, therefore, to sketch the origins of

demonstrations in these two traditions so that we

can better understand their contemporary manifestations. And because the R&D tradition is

older, broader, deeper, and the source of much

of our thinking about demonstrations. it will

receive somewhat greater emphasis than the

social action tradition.

The R&D Tradition

The Federal Government currently supports

R&D for a number of purposes at an annual level

of $23 billion. Historically, six major patterns of

support can be identified, each of which is

discussed briefly below. These patterns are

agriculture, national security, basic and applied

research, domestic public policy, regulation, and

energy. (Though agriculture and energy are

logically subordinate to domestic policy, they are

historically distinct and of sufficient importance to

warrant separate consideration. ) An analysis of

these six patterns can reveal important similarities

and differences in the purposes for which the

Federal Government has supported R&D.

Because demonstrations have been justified

mainly within the context of Federal R&D programs, then, an understanding of these patterns

can contribute to an understanding of

demonstrations.

Agriculture

Agriculture is often regarded as possessing a

comprehensive system for generating practical

research results and transmitting them into application. This “system” includes a highly differentiated research establishment, supporting activities from basic research through development

Chapter II

HISTORICAL CONTEXT

OF DEMONSTRATIONS

State, regional, and National laboratories. It

also includes educational programs, located

mainly at State land-grant colleges, which provide training for the next generation of farmers

and agricultural scientists and continuing education for agricultural producers, processors, and

even consumers. In the State experiment stations, the system has the capacity to test promising new research results on the appropriate

scale as well as exemplify the utility of validated

results. Through the extension service and a

variety of other communication channels, the

agricultural system has the means for transmitting

valuable new information from the research

establishment to agricultural producers, processors, and consumers.

Several factors should be kept in mind

concerning the agricultural system. 1 First, as with

many of the other policy areas we will discuss,

the development of Federal R&D policy was accompanied by fundamental political conflict.

There was no formally sanctioned Federal role in

agriculture until the 1860’s. The Southern States,

guided by the doctrine of States rights, successfully opposed creation of a Federal Department of Agriculture and a federally supported

system of agricultural schools prior to the Civil

War. This constitutional controversy was resolved only when the South seceded from the

Union. The massive realignment of political

power and reorientation of political philosophy

that then occurred in Congress resulted in significant new legislation in 1862: a Department of

Agriculture was established, the Merrill Act

authorized land-grant colleges of “agriculture and

mechanical arts, ” and the Homestead Act

opened the lands of the West.2

Within a sanctioned Federal role in agriculture,

‘The primary source for this section is A. Hunter J1.pee,

Science in the Federal Government, Harvard University

Press, Cambridge, Mass., 1957, PP. 109-114. 149-183,

348-350 et passim.

‘Ibid, pp. 149-151.

9
a second consideration is that the system of

research, education, experimentation, and extension developed slowly. The Federal Department itself went through a slow process of development which rejected a scientific disciplineoriented pattern of research organization in favor

of an agricultural problem focus.3 There was a

concurrent, but also lengthy, development of

State experiment stations in conjunction with the

land-grant colleges, but Federal funding and

Federal efforts at coordination took 25 years to

develop. The extension service, moreover, was

not authorized by Federal statute until 1914.

Regional laboratories, and the national Beltsville

laboratory, came even later. That a system that

evolved over at least three generations has not

been readily duplicated by others should not be

surprising.

Several other points are noteworthy for this

discussion. The institutional framework that

developed in agriculture was much more the

product of politicians and practical men than of

scientists.’ The framework itself, however, did

not dictate results and the “validation” of the

system came only as scientific results in entomology, animal diseases, and botany began to

show practical results of the 1880’s and 1890’s.

In this context, the experiment station provided

the logical site for the “demonstrations” of the

day: laboratory results that appeared promising

could be “proved” in the agricultural setting, and

validated results could be demonstrated to skeptical farmers.

National Defense

The major share of Federal R&D in the postWorld War 11 period has been for national security purposes. Most of the national security R&D

has been funded by the Department of Defense

(DOD), though the Atomic Energy Commission

(AEC) and its successor agencies have provided

funds for R&D on nuclear warhead development. Furthermore, the creation of the National

Aeronautics and Space Administration (NASA)

in 1958 and the rationale for the Apollo-manned

lunar exploration mission in the 1960’s can be at3

This development alone took several decades (ibid., pp.

157-169)

‘The deep involvement of the States through the landgrant colleges and the experiment stations practically insured attention to the system by political officials (see ibid.,

PP. 169-176 et passim).

10

tributed to the international threat from the space

program of the Union of Soviet Socialist

Republics.

The role of the Federal Government in national security has never been an issue, inasmuch as “the common defense” is a central part

of the Constitution. Defense-related R&D, therefore, has evolved as an integral component of

DOD and the uniformed services. It has been

supported for the purposes of providing and

maintaining a technologically advanced military

capability, generating technical options for the

future, and hedging against technological surprise by international adversaries.

Demonstrations per se do not constitute part

of the vocabulary of defense R&D. On the other

hand, certain R&D activities can be identified in

the defense context as quite similar to demonstrations as defined in this report. These include

testing a complex weapons system (usually at the

prototype stage) at the level of systems integration of a number of established component technologies in a simulation of a real-world operational environment; and conducting competition

among prototype weapons systems for the purpose of choosing the preferred system. These activities have their analogues in the policyimplementing and policy-formulating demonstrations discussed below.

The distinguishing characteristic of defense

R&D is that a strong relationship exists between

the R&D investment and the incorporation of the

results of that investment in advanced weapons

systems and component technologies. In institutional terms, a single, integrated organizational

system is both responsible for generating new

military technology and for purchasing suc h

technology. It should be emphasized, though,

that the search for the appropriate organizational

arrangements to relate R&D, weapons systems

procurement, and force posture needs has been

a long and complicated one in the three decades

since World War 11.5 As complex as this institutional system is, it remains simple in the fundamental relationships that govern organizations

5 A useful overview of the institutional evolution of

defense R&D is presented in Herbert F. York and G. Allen

Greb, “Military Research and Development: A Postwar

History,” Bulletin of the Atomic Scientists, Vol. 33, No. 1,

January 1977, pp. 13-26. York was the first Director of

Defense Research and Engineering.
within it. Defense R&D and the equivalent of

demonstrations within the R&D context, therefore, are characterized by strong relationships

between R&D inputs and defense products and

by a fundamentally integrated, and relatively

simple, institutional system.

Basic and Applied Research

World War 11 had a powerful “demonstration

effect” on conceptions of the Federal Government’s role in R&D. b Basic research became

recognized as a critical contributing factor to major technological advance. The connection was

most obvious between nuclear physics and

nuclear fission, but was also dramatically illustrated in the case of penicillin. Consequently, in

the postwar era, several Federal agencies developed with the support of basic research as a

primary mission.

The policy rationale for this basic research was

largely utilitarian (science for its applications) with

a strong aesthetic component (science for its own

sake). The rationale that emerged as welfare

economists turned their attention to the area was

that basic research generated “external benefits”

that could not be fully captured by the sponsor. 7

This led to an assumption that systematic underinvestment in basic research by the private sector

would occur relative to the socially optimal level

of support and then to an a priori justification for

Government support of basic research. This

argument was extended to include applied

research whose results were not easily appropriable by private firms for commercial application. The theoretical limit on the Government’s

R&D activity was thus defined by appropriability

of results: it was inappropriate for the Federal

‘See, for example, J. Merton England, “Dr. Bush Writes

a Report: ‘Science—The Endless Frontier, ’ “ Science. Vol.

191, Jan. 9, 1976, pp. 41-47, and Daniel J. Kevles, “The

National Science Foundation and the Debate over Postwar

Research Policy, 1942-1945: A Political Interpretation of

‘Science—The Endless Frontier,’ “ 1S1S, Vol. 68, No. 241,

1977, pp. 4-26.

‘See Richard R. Nelson, “The Economics of Invention: A

Survey of the Literature, ” Journal of Business Vol. XXXII,

No. 2, April 1959, pp. 101-127, and Kennetb J. Arrow,

“Economic Welfare and the Allocation of Resources for Invention, “ in Richard R, Nelson (ed, ), The Rate and Direction of Inuentiue Activity: Economic and Social Factors, a

Report of the National Bureau of Economic Research,

Princeton University Press, Princeton, N. J., 1962, pp

609-626,

Government to fund work appropriable by the

private sector, since firms could be expected to

finance such R&D themselves. The operational

limit on Government’s R&D activity was essentially set by drawing a boundary between applied

research and development. a In this context, the

question of demonstrations did not arise because

application of R&D results was left to the private

sector.

Domestic Public Policy

The Federal Government supports R&D in a

number of areas of domestic policy. Many of

these R&D efforts include demonstrations.

The area of health and medicine, however,

differs from many other policy areas in notable

ways. First, the rationale for Federal Government

support of medical research, largely through the

National Institutes of Health (NIH), is a product

of several factors—the successful applications of

medical research results in World War 11 (e. g.,

penicillin, antimalarial drugs, treatment of burn

and trauma),9 the limited resources of private

philanthropy in sustaining an expanding medical

research enterprise after the war, 10 the lack of in

terest by the pharmaceutical industry in supporting fundamental research, and the lack of

opposition by organized medicine. 11 Second,

because the main element of NIH strategy over

the years has been to invest proportionately more

in fundamental research than in applied or

clinical research, interest in demonstrations has

been relatively weak.12 Third, in response to

‘See Richard R. Nelson, Merton J. Peck, and Edward D.

Kalachek,Technology, Economic Growth, and Public

Policy, The Brookings Institution, Washington. D. C.. 1967,

pp. 171-211 et passim; George Eads and Richard R. Nelson,

“Governmental Support of Advanced Civilian Technology—power Reactors and the Supersonic Transport, ” public

Policy, Vol. 19, 1971, pp. 405-427. and George Eads,

“U.S. Support for Civilian Technology Economic Theory

and Political Practice,” Research Policy, Vol. 3, 1974, pp.

2-16.

9See E.C. Andrus et al. (eds. ), Aduances in Military

Medicine, Vols. 1 and II, Little, Brown and Company,

Boston, Mass., 1948.

‘“See England op. cit., p. 44.

*’See American Medical Association, Report oj the Commission on Medical Research, Chicago, 111., 1967.

IZA very useful historical overview is presented in

Stephen P Strickland, Pol/tics, Science, and Dread

Disease: A Short History of United States Nledlcal Research

Policy, Harvard University Press, Cambridge, Mass., 1972.

11
more recent pressures to concern itself with the

application of medical research results, NIH has

supported an increasing number of controlled

clinical trials. These trials emphasize the validation of research results in the clinical setting

rather than promotion of widespread use of such

results, and are thus analogous to demonstrations intended to prove a technology. There are

cancer and heart-disease control programs, on

the other hand, that are more analogous to exemplification demonstrations. These control programs, however, coexist rather uneasily with the

more rigorous research-oriented activities of

NIH.

In the mid-1960’s, Federal Government R&D

resources were increasingly allocated to a

number of domestic policy areas—urban mass

transit, postal services, housing and urban affairs,

education, manpower, and law enforcement and

criminal justice. The R&D efforts in these areas

took place as two large secular shifts, sometimes

overlapping, occurred. One was that the importance of the public sector increased relative to

the private sector as, for example, in urban mass

transit. The other was that the importance of the

Federal Government increased relative to State

and local government. In some policy areas, an

expanded Federal role paved the way for greater

Federal R&D, e.g., education; in others R&D

constituted the “entering wedge” for an

expanded Federal role, e.g., criminal justice.

These domestic policy areas have two characteristics of consequence for demonstrations. On

the one hand, the scientific understanding of central phenomena and the technical base undergirding operational activity are not well

developed, thus the contribution of R&D is often

problematic. At the same time, the institutional

environment in these areas is highly complex,

consisting of public and private organizations

functioning at all levels of government—Federal,

State and local. ” The authority of Federal agencies is frequently limited, while responsibility is

distributed in a fragmented way throughout the

interorganizational network that constitutes the

institutional environment. Thus, the Federal

R&D agency seldom, if ever, has anything but a

weak relationship with the agencies actually

“For a fascinating illustration of this point, see Donald

Schon, “The Blindness System, ” The Public fnterest, No.

18, Winter 1970, pp. 25-38.

responsible for policy and program implementation. Although demonstrations have become an

important part of R&D activity in these policy

areas, their utility has been limited by both

technical and institutional factors, as discussed at

greater length in the next chapter.

Regulation

The scope of Federal Government regulatory

activity has expanded markedly in recent years

from its more traditional concerns with “markets,

rates, and the obligation to serve” to include the

conditions under which goods and services are

produced and the characteristics of those

goods. This expansion has blurred, if not

erased, the prior distinction between regulated

and unregulated industries. In addition, the

scope of Federal regulation has also come to include activity by lower levels of government, as in

the case of water pollution control standards applied to municipal sewage disposal. This expansion of regulatory activity has drawn the attention

of an increasing number of commentators; 15 it is

not our purpose to review this discussion here.

There is one aspect of the newer forms of

regulation, however, that does bear strongly

upon the use of demonstrations as Federal policy

instruments. Regulations and standards are increasingly based upon technical, as distinct from

economic, criteria. This is true, for instance, in

the case of S02 stack-gas scrubbers, fabric flammability standards, and aircraft engine noise

standards. l6 The precise values of these standards are often a direct function of the technological state of the art; statutes sometimes require

the “best available technology” as an additional

criterion. An increasing portion of Federal R&D

funds is directed to generating the technical information required for these new standardsetting activities. In this context, demonstrations

“See William Lilley, 111, and James C. Miller, III, “The

New ‘Social’ Regulation, ” The Public Interest, No. 47, Spring 1977, p. 53.

“See, for example, Charles L. Schultze, “The Public Use

of the Private Interest,” Harper’s, Vol. 254, May 1977, pp.

43-62, and Regulation, a new journal initiated by the

American Enterprise Institute in summer 1977.

l6 See the following cases in Federal Demonstrations:

Cale Studies–’’Refuse Firing Demonstration (Solid-Wasteto-Fuel-Conversion Plant), ” “Resource Recovery from

Refuse, ” “Poultry Waste Processing, ” and the “Refan Jet

Engine Program .“

12
have begun to take on an important role in proving or verifying given technologies. This represents the use of demonstrations in a policyformulating manner.

A concomitant of basing regulatory standards

on technical criteria is that the regulations provide

a powerful force for diffusing technology. Indeed, the coercive quality of regulations probably

provides a more powerful incentive to adopt new

technologies than do the usual market forces.

While this naturally provides an important opportunity for the Government to promote technological change, it simultaneously creates a

responsibility to ensure that the change is, in fact,

socially desirable. Demonstrations seem likely to

be an important aid in making judgments concerning the worth of proposed standards.

Energy

Federal Government R&D investment in

energy was, through the 1960’s, primarily directed to the development of nuclear power

through the programs of the AEC. A small

amount of coal research was supported by the

Department of the Interior, but little R&D was invested in other energy sources or energy conservation.

The AEC initiated the Power Reactor Demonstration Program in the mid-1950’s, and this program was instrumental in the adoption of nuclear

power by the private utilities. 17 The initial objective of this program was to generate R&D information and to involve commercial firms and

utilities in the construction of nuclear powerplants. Two successive rounds emphasized construction of small power reactors for use in rural

areas and foreign export and the development

and exemplification of larger reactors.

The Federal Government’s role in energy was

markedly altered in 1973 by the action of the

Organization of Petroleum Exporting Countries

(OPEC) in raising the price of OPEC-produced

oil. The AEC gave way to the Energy Research

and Development Administration and the latter,

in turn, to the new Department of Energy. R&D

activity was increased across all energy technologies and, in the most significant policy departure, “commercialization” of new technologies

“See Wendy Allen, Nuclear Reactors for Generating

Electricity: U.S. Development from 1946 to 1963, The

RAND Corporation, R-21 16-NSF, June 1977.

became an important Federal goal.18

It is this altered Federal role with responsibility

for “commercialization” that makes the energy

area distinct from other domestic policy areas.

The Federal Government is now engaged in the

complex task of relating public investments in the

development of new energy technologies to

strategies for seeing those technologies used for

energy production and conservation in t h e

private sector. ’9 The matching of public and

private investment decisions, choice processes,

and decision criteria is being worked out in a very

complicated manner. Demonstrations, as it turns

out, happen to be one of the policy instruments,

however imperfect, being used in the search for

an improved match. 20

Conclusions

Several conclusions emerge from the above

discussion. First, the Federal role in a given

policy area establishes the scope of the Federal

R&D effort in that area. In national security, the

Federal Government is primary and its role is

comprehensive in scope. In basic and applied

research, however, the Federal role is limited to

that research generating external benefits, and

duplication of private sector development work is

not warranted. A second conclusion is that the

several processes of generating scientific knowledge, incorporating that knowledge into useful

technology, and building the associated institutional and professional capital nearly always require a long period of time. This is clearly the

case in agriculture and national security, two of

the most highly developed policy areas from an

R&D perspective. There is no reason to suppose

it less true in areas where scientific understanding

and technological capability are less developed.

Furthermore, if the Federal role authorizes or

mandates that the R&D effort deal with the

“See MIT Energy Laboratory Policy Study Group,

Government Support for the Commercialization of New

Energy ‘Technologies: An Analysis and Exploration of the

Issues, Cambridge, Mass., November 1976.

“See Leland L. Johnson, Edward W. Merrow, Walter S.

Baer and Arthur J. Alexander, Alternative institutional Arrangements for Developing and Commercializing Breeder

Reactor Technology, The RAND Corporation, R-2069 -

NSF, November 1976.

20 See Don E. Kash et al., Our Energy Future, University

of Oklahoma Press, Norman, Okla., 1976, pp. 25-26 et

passim.

13
utilization of R&D results, the use of demonstrations will be a derivative response. In agriculture

and defense, for instance, the comprehensive

Federal role led long ago to the equivalent of integrating demonstrations into the institutions and

procedures of these sectors. In basic and applied

research, on the other hand, because the Federal

role is limited, the question of demonstrations

has never been raised. In domestic policy, regulation, and energy, where R&D is performed to

generate innovation, demonstrations are prevalent. Even so, it is not surprising that demonstrations have yet to be firmly established in domestic

policy, regulation, and energy. These areas are

relatively new, deal continuously with distribution

of costs and benefits of policies among social

groups, have complex institutional relationships

with the private sector and with other levels of

Government, and often manifest lack of consensus about the appropriate Federal role. Finally,

the effectiveness of demonstrations will be determined by the contribution of the R&D effort to

the technical problems of the operational world

and by the institutional environment in which that

relationship is set. This point is elaborated analy -

tically in chapter III.

The Social Action Tradition

Demonstrations as a policy instrument have

their roots in a second tradition, one of social action, z

’ This tradition has as its concern those individuals, families, and groups deemed by society to have too few of the necessities and

amenities that constitute an acceptable physical,

economic, and social standard of living. Specifically, this tradition has championed the cause of

the poor, the disadvantaged, and those who encounter discrimination and is manifest in policies

like income maintenance, education of disadvantaged children, provision of adequate housing and health care, job training programs, and

the like. Demonstrations here frequently have

been organized and directed by individuals with

little scientific or technical training—social

workers, city and regional planners, teachers and

school administrators, and public administrators.

“See Martin Rein, Social Policy: Issues of Choice and

Change, Random House, New York, 1970, pp. 138-152 et

passim.

14

The mid-1960’s witnessed a significant and

rapid expansion of Federal Government activities

directed to the alleviation of a number of social

ills. In the early days of the War on Poverty, for

instance, “new programs to help the poor tumbled out of the White House and Congress in

rapid succession, and idealistic Government officials worked frantically to get them started and

confidently looked forward to quick and visible

results. “22 A wide range of far-reaching social

legislation was enacted in a similar manner during this era.

Demonstration projects were a prominent

feature of these social action programs. Often it

was the case that demonstrations funded by nonFederal sources preceded the emergence of

Federal programs; they constituted “prototypes”

in the sense of indicating in particular instances

the main lines that a larger invention might

follow. ” Demonstrations also were initially proposed as limited efforts to learn “what works, ”

only to be expanded to a broader program by

Congress, as was the case for Model Cities. A

third instance was the use of demonstrations to

initiate a service provision program as in the community mental health centers funded by the National Institute of Mental Health.

Although the term “demonstration” suggested

an effort to validate social intervention strategies,

little about these demonstrations was systematic.

They were described as “random innovation” by

Rivlin. 24 (Try enough innovation approaches to a

problem and some are bound to workout; or so it

was thought. ) Seldom were these demonstrations subject to good program evaluation, so little

knowledge emerged of how effective programs

could be used to make other projects more successful.

Even so, social action demonstrations commended themselves to policy makers for impor22Alice M. Rivlin and P. Michael Timpane, (eds. ), Planned Variation in Education: Should We Giue Up Or Try

Harder?, The Brookings Institution, Washington, D. C.,

1975, p. 3..

“See, for instance, Terrance Keenan, “The Health

Record of Private Foundations, ” Journal of Health Politics,

Po/icy and Law, Vol. 2, No. 1, Spring 1977, pp. 11-19, for

discussion of the efforts of private philanthropic foundations

to “craft and test the prototypes upon which government

programs are modeled. ”

“Alice M. Rivlin, “Systematic Thinking Jor Social Action,

The Brookings Institution, Washington, D. C., 1971, pp.

87-90.
tant reasons. They were a readily available

means for making a prompt Federal response to

a pressing social need, thus satisfying the “do

something” injunction to policy makers. Beyond

this, their execution occurred at local government levels, thus indicating action “where it

counts. ” In addition, they implied a strategy of

rational social change—first demonstrations,

then full-scale programs, thus carrying promise

for Iongrun achievement. Furthermore, they

were an accommodation to the problem of scarce

resources, requiring substantially fewer resources

than full-scale programs of intervention .25 This

accommodation had a dual appeal: liberals could

approve this as a strategy of “getting a foot in the

door, ” while conservatives could regard it as a

second-best solution where the preferred course

was no Federal program. Finally, demonstrations

quite often were important in providing symbolic

recognition by the Federal Government of claims

for attention or resources by the important constituency group. These political uses of demonstrations, more prominent in the social action

tradition than in the R&D tradition, retain their

attractiveness in many situations today.

Gradually, for a combination of reasons,

demonstrations in the social policy area began to

be drawn into a more formal R&D tradition. This

development was supported by research personnel in the agencies and by the Office of Management and Budget. A body of doctrine was developed about good and bad practices in the

management of demonstrations. Project goals

became more specific. The time duration of projects was more clearly indicated. Expectations

“See Rein, op cIt , pp 139-140.

were reduced. In some cases, demonstrations

began to be viewed as a stage in program R&D

development not unlike that characteristic of the

program in the R&D tradition.

Conclusions

Demonstrations in the social policy area today

constitute one important means in the search by

policy makers for ways to learn about “what

works. ” These policy makers have been taught by

the experience of the 1960’s and 1970’s that

“random innovation” leads to few generalizations

that are widely applicable. They are generally

convinced of the need for systematic accumulation of knowledge as a basis for the establishment

of sound social policy. But they are also increasingly aware of the limits of R&D programs in

general, and of demonstrations in particular. to

generate that knowledge. The rhetoric surrounding demonstrations in the social policy area is

similar, if not identical, to that in areas involving

hardware demonstrations. But the confidence

that demonstrations are the logical next step to

move research results into operational uses is not

as strong. Thus, we find demonstrations have

developed from two quite different traditions, but

that social demonstrations are increasingly being

guided by an R&D perspective. The fact that the

language and aspirations of social and hardware

demonstrations are increasingly similar may

mean that social policy and action are steadily being undergirded by a systematically developed

knowledge base. Alternatively, it may mean that

the language of the R&D tradition is only serving

to mask temporarily the limited capacity of R&D

to generate an adequate knowledge base for

social policy.

15
—.

Chapter Ill

A CONCEPTUAL FRAMEWORK

FOR THE CONSIDERATION OF

DEMONSTRATION PROJECTS

IN FEDERAL R&D POLICY
Chapter Ill

A CONCEPTUAL FRAMEWORK

FOR THE CONSIDERATION OF

DEMONSTRATION PROJECTS

IN FEDERAL R&D POLICY

Our approach to demonstrations is from an

R&D perspective. It is not a narrowly technical

perspective, but one that explicitly recognizes the

way that political and technical factors interact in

demonstration projects. In this way, we believe

that a conceptual framework can help in understanding the uses and limits of demonstrations as

a policy instrument.

This chapter sets forth a conceptual framework

within which to examine demonstration projects.

We examine the purposes of demonstration projects in meeting R&D objectives and other policy

goals and review the general factors that affect

the probability that demonstration projects will

achieve these purposes. Throughout we draw

upon the literature to provide illustrations and

support for the analysis.

The Purposes of Demonstrations

The purposes of demonstrations can be considered from two quite distinct perspectives: purposes related to the development and application

of knowledge through an R&D process, and their

uses within a political process.

Demonstrations in the R&D Tradition

Within the R&D tradition there are two fundamental purposes for a demonstration project:

● To aid Federal policy makers in deciding

whether or not to adopt a policy.

● To promote the use of an innovation.

Demonstration projects in which the first pur-

‘We define innovation as a program, product. or production process in the particular existing context: innovation is

not necessarily new in the sense of “first instance. ”

pose predominates we refer to as policy-formulating demonstrations; projects in which the

second purpose predominates we call policyimplementing Ciemonstrat#ons.

A good example of policy-formulating demonstrations is the income maintenance experiments

conducted by the Office of Economic Opportunity (OEO) and the Department of Health, Education, and Welfare. These experiments were begun in the late 1960’s and continue at the present

time. They have sought to provide information to

policy makers concerning the effects of income

maintenance payments upon the work incentives

of the participants in the programs, the administrative feasibility of implementing such a program, and the difficulties of integrating income

maintenance programs with other service programs such as manpower training. The emphasis

upon the work incentives reflected the beliefs of

the demonstration’s sponsors concerning the

likely focus of a policy debate over welfare

reform. The sponsors in OEO felt that a major

congressional concern would be whether individuals who receive some form of guaranteed income would withdraw from the labor force and

become permanent members of the welfare

population.2

An example of a policy-implementing demonstration is those conducted by the Shipbuilding

Research, Development, and Demonstration

program; through this program, the Maritime

Administration is attempting to stimulate innovation in the shipbuilding industry by organizing an

interindustry consortium of shipbuilding firms to

propose and assist in managing demonstration

projects. The projects with this program invoke

‘Joseph A. Pechman and P. Michael Timpane (eds.),

Work incentives and income Guarantees: The New Jersey

Negatiue Income Tax Experiment, The Brookings Institution, Washington, D. C., 1975, pp. 15-24.

19
the development and testing of a new technological device or production method, which is followed by a formal demonstration in a shipyard to

which all industry engineers are invited. Most of

the projects are small and concern the pragmatic

“nuts and bolts” problems of shipbuilding, not

large-scale demonstrations of significantly new

technologies. The projects have included new

techniques of welding, surface coating, materials

handling, and automation.3

In all cases in this paper, our perspective is that

of a Federal policy maker. This caveat is important. It is quite possible that the firm or institution

carrying out a project the Federal Government

views as a policy-implementing demonstration

conceives the project as a means of formulating

its own “policies. ” But because we are examining

demonstrations sponsored by the Federal Government, our analysis is from the perspective of

the Federal policy maker.

Moreover, individual demonstration projects

can possess elements of both policy-formulating

and policy-implementing purposes. A project

that begins as a test of the worth of an innovation

may ultimately serve as an exemplary demonstration for other sites that are funded under a

new Federal program created as a result of the

test. Alternatively, a demonstration project may

be initiated to promote what is considered to be a

proven concept but, as a result of adverse experiences in the early stages of the project, it becomes a means of testing the worth of the concept. A project may also possess different purposes, as perceived by the various actors concerned with its execution. The funder of a project

may view the project as a test of a concept; the

manager, as a means of promoting the use of an

innovation whose worth has already been

proved.

Although a demonstration may possess

elements of both purposes, the distinction is important. If policy makers do not specify the purposes of a demonstration, agencies conducting

the demonstration are free to view its purposes

from the perspective that best suits their individual needs. This may result in the promotion

of the use of innovations that Congress or

executive policy makers do not feel are proved:

demonstrations that these policy makers view as

policy-formulating are viewed by the agencies as

3

Federal Demonstrations: Case Studies, pp. E-1 - E-6.

20

policy-implementing. Alternatively, agencies

may treat programs as experimental that Congress sees as a means of implementing desirable

policies.

For either policy formulation or policy implementation, the value of demonstrations should

be based upon their ability to provide more credible information to either policy makers or potential users of innovations than would be available

from other R&D methods. A project’s scale and

the realistic environment within which it is tried

are the major contributors to this credibility.

Policy-formulating demonstrations should

provide information to Federal policy makers

about:

●

●

●

●

Technological and administrative feasibility

of instituting the policy or adopting the innovation on a wider scale.

Expected economic, environmental, and

social impactsof the policy or innovation.

Relative merits of alternative policies or innovations intended to meet the same

needs.

Unanticipated consequences of adopting a

particular policy or innovation.

In seeking to provide this information, demonstrations may help to clarify the nature of the

policy problem itself. For example, the New

Jersey income maintenance experiment highlighted the sensitivity of program costs to the frequency and manner of assessment of a family’s

income, an issue that was given little attention in

early legislative planning. The experiment also

provided detailed information on the variability of

the incomes of the working poor.4

Negative information, information concerning

the shortcomings of an innovation, may also be

an important output of policy-formulating demonstrations. Because these demonstrations take

place before a commitment to a course of policy,

negative results can be used to support a decision

to abandon consideration of the policy. ’

Policy-implementing demonstrations

should provide many of the same types of in4

Pechman and Timpane, op. cit, pp. 207-214.

5

0E0’s experiment with Educational Performance Contracting is a good example. See Edward M. Gramlich and

Patricia P. Koshel, Educational Performance Contracting,

An Evaluation oj an Experiment, The Brookings Institution,

Washington, D. C., 1975.
formation to potential users of information. However, the emphasis is typically placed upon

● Costs of adopting and using the innovation.

● Reliability of that innovation in use.

● Demand for the innovation.

 Feasibility of implementing the innovation

at the adopter’s site.

Inherent in the definition of purposes of

demonstrations is a specification of audiences.

For policy-formulating demonstrations, the audience is the Federal Government and the constituencies that seek to influence Federal policy; for

policy-implementing demonstrations, potential

users of the innovation. Two classes of users can

be identified: “final” users—private consumers,

local governmental units, or individual firms; and

“intermediary” users—suppliers of equipment,

for example. Planning for a demonstration

should involve clear identification of the audience

and a determination of their information needs.

These two types of demonstrations should be

distinguished from two other types of activities

that share some characteristics of demonstration

projects. We do not consider large-scale engineering tests of hardware components to be

demonstrations. Such projects, which typically

test scaled-up components of an innovation

process or product, neither test a full “system”

nor take place in a realistic operating environment. For example, aircraft engines may be run

in large test cells or even mounted on test aircraft

before being integrated into prototype systems.

Such activities are potentially quite important to

the development of new products but should be

considered part of development rather than an

instrument for promoting the transition from

development to use.

The second type of activity that we exclude is

the projects that are created as a result of

Government subsidies for local development. In

a number of social programs, for example, funds

are provided to local political jurisdictions to

develop innovative activities. The Law Enforcement Assistance Administration (LEAA) has provided such funds to police departments; Title IVC

of the Elementary and Secondary Education Act

supports innovative projects that relate to local or

State school needs. Projects such as these are

generally not considered as means of promoting

the use of a particular innovation. Rather, they

place the responsibility for innovating on the local

jurisdiction itself.

Demonstrations as Instruments

i n the Political Process

The purposes of demonstrations can also be

viewed in terms of their usefulness within our

political system. Demonstration projects are a

means of showing political initiative at relatively

low budgetary cost. They are an attractive compromise between the advocates of sweeping

policy or program change and the supporters of

the status quo. They can help promoters of a

policy to “get a foot in the door” to build a

constituency. By demonstrating a concern for an

issue or a particular group, demonstrations can

become significant symbolic acts. They can assist

Federal R&D managers to gain a political constituency. Demonstrations permit an R&D program

manager to maintain some momentum in his

program if the political support for expansion

does not otherwise exist. Finally, demonstration

projects can serve the needs of traditional pork

barrel politics.

These are important functions within our

governmental system because there are many

issues on which compromises are needed. In the

absence of such compromises, we would often

reach an impass between advocates and opponents of a policy. A demonstration project

may permit concrete experience with a policy to

be obtained to settle the debate. It may provide a

vehicle through which a particular interest group

can establish a dialogue with elements of the

Government. A demonstration can provide an

important governmental gesture of concern for

individuals facing difficult problems when the

Government does not know how to solve those

problems.

The fact that a demonstration is a compromise

poses substantial problems for a project with

R&D objectives. The parties to the compromise

have a variety of objectives and frequently the

compromise itself will not specify which of these

objectives should predominate. The F O1 1O W

Through Program illustrates this problem well.

Follow Through was proposed as a program to

provide continuing compensatory educational

services to schools with disadvantaged children

coming from Headstart preschool programs. Advocates of the program argued that such continued services would prevent the students from

losing the gains they had achieved in their

21
Headstart programs. Opponents argued that

there were insufficient funds to support such a

program and in any case there was no evidence

that it would be effective. The compromise was a

“planned variations” program to test the effectiveness of different program designs. b

The advocates of an operating program

sought quick implementation and, as a consequence, most of the programs were incompletely

designed. In addition, the experimental design

necessary to detect differences among programs

was deemed inappropriate by the operating officials. The absence of clearly defined goals led

the evaluators to collect data having only

marginal usefulness. While there have been

some useful outcomes, it seems unlikely that

they are commensurate with the costs that were

incurred. An important cause of this low yield

was the conflicting goals of the parties to the

compromise and the failure to resolve these conflicts in the design of the program.

The political uses of demonstrations will lead

to projects with multiple and often conflicting objectives. Those groups that are involved will frequently possess “hidden agendas” that may interfere with the R&D objectives of the project.

The absence of a clear consensus about goals

may weaken project management and lead to inconclusive evaluations. A major task of both the

executive branch and Congress should be to

recognize both the political and R&D functions of

demonstration projects and to design projects in

a manner that minimizes the adverse interactions

of the goals.

In the next section we address the factors that

affect the success of demonstration projects as

part of the R&D tradition rather than as instruments in the political process. Our concluding

section will consider how the political and R&D

uses of demonstrations can usefully be melded

together.

General Factors Affecting the

Success of Demonstrations

Many factors affect the success of a particular

demonstration project: the quality of the project

‘Alice M. Rivlin and P. Michael Timpane (eds. ), Planned

Variation in Education: Should We Give Up Or Try

Harder?, The Brookings Institution, Washington, D. C.,

1975, chapter 1.

22

staff, the appropriateness of its goals, the adequacy of the funding, and the nature of the political and physical environment. Moreover, these

factors will affect the outcome of demonstration

projects in every policy area. Our interest,

however, is not to investigate how an individual

project can be most successfully executed once a

decision has been made to initiate it. Rather, we

seek to understand the types of situations in

which well-executed demonstrations are most

likely to be successful in reaching their objectives.

Our review of the literature leads us to suggest

that two factors are likely to be of prime importance in determining the success of a demonstration project: first, the nature of the technology

that is being demonstrated; second, the nature of

the institutional environment within which a

demonstration must be carried out.

The Technology

As we have noted, we use the term

“technology” to denote the inputs, outputs, and

theory relating them that are associated with the

innovation under consideration, 7 The technology

of the automobile is thus the inputs of personal

time and invested capital, the output of transportation services, and the theories that relate the inputs to outputs that are embodied in the automobile itself. Two points deserve note. First, the

definition of a technology is a matter of convenience. We can discuss the technology of an automobile or we can enlarge the concept to discuss

the technology of an urban transportation

system. The choice of the scope of the technology depends upon the purposes for which the

concept is used. In this report, the scope is associated with the innovation whose demonstration

is being contemplated. If the innovation is an

electric automobile, the technology would be the

automobile itself. If the innovation were a van

carpool system, however, the technology would

extend beyond the vehicle to encompass the system of roads, the manner in which carpools are

formed, the location of work and living sites as inputs, and transportation services as outputs. The

theories that relate these inputs are no longer

solely embodied in physical entities but extend to

institutions and behavioral responses.

Second, the theories and inputs associated

‘Our use of the term “technology” is similar to that of

economists who are concerned with production functions.
with differing technologies vary substantially in

quality. These variations are important to understanding the impact that the quality of a technology has on the usefulness of demonstration projects as policy tools. Technologies differ in the

degree to which their inputs involve important

uncontrollable variability. Thus, the inputs of an

education technology such as a reading program

include teachers and students whose capabilities

vary substantially. The technology associated

with a large coal powerplant involves inputs of

coal and water whose characteristics are predictable and have little variance for a single plant.

Technologies also differ in the degree to which

the inputs interact and change in the course of

the transportation to outputs. In education, the

qualities of teachers can be modified as they work

to change the skills and other attributes of the

students. For example, teachers can become

more or less directive in their instruction or

change their allocations of time among students.

In the case of the powerplant, the coal does not

change its attributes in unpredictable ways as it

enters the process of producing power. Finally,

technologies differ in the nature of their

associated theories. For a technology such as

power production, the theories that underlie the

transformation of the inputs to outputs and residuals are quite deterministic and provide high confidence that certain inputs will lead to predictable

outputs. In the case of education, the theories

that relate inputs to outputs either do not exist or

can predict outputs only

ranges.

For some technologies,

proved through research,

testing. While we may not

within very broad

theory can be imdevelopment, and

be sure that a new

design for an internal combustion engine will

have high fuel efficiency and useful power output, we are confident that the developing and

testing will allow us to perfect the technology’s

theory in the sense that we have used the term.

In other cases, we are far less confident that we

can produce such theory through R&D processes. For example, incentive systems for encouraging the insulation of houses can be designed and tested but will probably never reach a

point where we may confidently predict that a

particular individual, faced with the incentives,

will make a particular decision. Even without the

capability to make such exact predictions, however, decisionmakers can reduce the uncertainty

concerning the behaviors of groups of individuals. Thus, while the theory may not be as deterministic as that associated with internal combustion engines, it is useful to a policy maker and it

can be improved through R&D. In other areas,

however, theory may be at such a primitive stage

that it cannot be readily perfected through R&D.

This may well be the case with police patrol procedures, where the relationship between the frequency of patrols, the type of patrol, and the

type of site on the one hand and the occurrence

of crimes on the other seems to be beyond our

present capacities to discover.8

These qualities of a technology can be conveniently characterized in essentially two ways. The

first of these is the degree to which a technology

is reproducible from site to site. With what confidence can the policy maker or the industrial firm

say that the use of the innovation will result in the

transformation of inputs into desired outputs?

This will depend upon the variability of the inputs, their tendency to interact or modify themselves in the course of the transformation, and

the quality of the theory that exists. In general.

technologies that are substantially embodied in

hardware would be quite reproducible, while

those that involve processes of human interaction

would be relatively nonreproducible. Some

technologies occupy a middle ground. For example, transportation systems have some reproducibility but will vary substantially by site.

The second quality of a technology is the

degree to which research, development, and

testing can reduce uncertainty about the likely

outputs to be associated with a particular set of

inputs. Designing and testing prototype models

of automobile engines can yield high-confidence

estimates of the performance of the engine. Tests

of curriculum materials, on the other hand, yield

only low-confidence estimates concerning the

performance of those materials because of the

weak theory upon which they are based and the

variability of the inputs to the education process.

If the uncertainties surrounding the relationship

between the inputs and outputs of a technology

have been substantially reduced prior to its incorporation as a demonstration, we will refer to

the technology as being well in hand.

“Jan Chaiken,

Police Activities,

1977.

What is Known About Deterrent Effects of

The RAND Corporation, P-5735-1, July

23
The Institutional Environment

A demonstration will result in a number of outcomes. Some of these will simply be what we

have referred to as outputs of the technology

associated with an innovation; others will be

associated with the impacts that the use of the innovation has upon the environment, the behaviors of groups of citizens, or the employment in a

community. A major reason for conducting a

demonstration project is to improve knowledge

concerning the effect of using the innovation on

these outcomes.

The context in which the demonstration is carried out determines the scope of relevant outcomes. The needs and interests of the target audiences determine the specific outcomes measured. If, for example, the demonstration is intended to influence local police departments to

adopt and use a new piece of equipment, the

outcomes examined are those that are important

to officials in these departments. The choice of a

site or sites should include operational environments that are similar to the sites of most

potential adopters. If the demonstration is

intended to aid in the congressional debate concerning a new welfare system, it should be

designed to address questions that Congress and

its constituents consider important.

Thus, the relevant scope of outcomes for a

demonstration is determined by the institutional

environment in which it will be implemented.

This environment encompasses the collection of

organizations and the linkages among those

organizations that are involved in selecting and

implementing the innovation being demonstrated. The developers of the innovation,

ultimate users, regulatory bodies, markets, and

the Federal agencies that fund the demonstration

are components of the institutional environment.

The concept of a “technology delivery system”

suggested by Wenk and that of a “selection environment” used by Nelson and Winter are

roughly equivalent concepts.9

‘See Edward Wenk, Jr., “The Social Management of

Technology,” in John E. Mock (cd.), Science for Society,

proceedings of the National Science Conference held at

Atlanta, Ga., Oct. 12-14, 1970; The Committee on Public

Engineering Policy, National Academy of Engineering,

Priorities for Research Applicable to National Needs, report

to the National Science Foundation, Washington, D. C.,

1973; Arthur A. Ezra, “Technology Utilization: Incentives

and Solar Energy, ” Science, Vol. 187, Feb. 28, 1975, pp.

707-713; Richard T. Nelson and Sidney G. Winter, “In

Search of Useful Theory of Innovation, ” Research Policy,

Vol. 6, 1977, pp. 36-76.

24

The institutional environment has importance

to Federal officials for two reasons. If the Federal

Government were contemplating using a demonstration to help determine the worth of a policy or

the desirability of promoting the use of a particular innovation, the relevant scope of outcomes as set by the institutional environment

would determine the inputs and outputs that

must be measured to make credible inputs to the

policy process. If, on the other hand, the Government were considering a demonstration to

promote the use of an invention, the institutional

environment would be the medium in which the

use would take place; the important actors and

communications links in this environment would

be the critical determinants of the way in which

the innovation is adopted and used.

The basic proposition about institutional environment that emerges from the literature is that

innovation is strongly affected by the nature and

the workings of the institutions in a policy sector.

There are some familiar examples that help to

clarify this point. In the area of national security,

it is widely believed that the development, procurement, and development of new weapons

systems is facilitated because the Federal Government is the sponsor of the relevant R&D, the

buyer of the R&D product (usually a weapons

system), and the user of that product. Even

though the institutional environment in the national security area is complex, the dominant position of the Federal Government simplifies the

process of innovation relative to other areas involving intergovernmental or public/private relationships. 10

A second example is that of the Bell System.

Within the framework of the American Telephone and Telegraph Company are the Bell

Laboratories, the R&D organization; Western

Electric, the production organization; and the

Long Lines Division and the operating telephone

companies, the organizations providing service.

Thus, institutions within the AT&T organization

are specialized in their functions, yet they are

highly integrated concerning the manner in

which each plans its operations and the ways in

which each transfers the information or products

to the others. It is argued by the Bell System, and

believed by many, that this complex group of in-

‘“Herbert F. York and G. Allen Greb, “Military Research

and Development: A Postwar History, ” Bulletin Of the

Atomic Scientists, Vol. 33, No. 1, January 1977, pp.

13-26.
stitutions, tied together as they are, facilitates the

generation and diffusion of new communications

technologies. 11

A third example is found in the agricultural

sector. The U.S. Department of Agriculture

(USDA) has developed an institutional system

over an extended period of time that is thought to

be quite effective in ensuring the flow of innovation in agriculture. The USDA directly supports a

national laboratory, four regional laboratories,

and satellite research laboratories through the

Agriculture Research Service. Through the Cooperative State Research Service, it finances

research jointly with the States in colleges of

agriculture and State experiment stations across

the country. The Agricultural Extension Service,

funded jointly by Federal, State, and county

funds, supports the agricultural extension agents

who are responsible for the dissemination of

R&D results to prospective users and the needs

of users to the research system. Thus, a strong

system need not be centralized. 12

These three examples illustrate relatively welldeveloped institutional environments. In contrast, consider the elementary and secondary education system. The National Institute of Education, the National Science Foundation, and the

Office of Education provide funds at the national

level. Fifty State departments of education

assume a variety of stances toward promoting

the use of new innovations in local school

districts. Most education costs are for personnel.

Training professionals in the school system is

mainly the province of a large number of teacher

preparation colleges that are not widely known

for their interest in or conduct of R&D activities.

Decisions concerning the adoption of new curriculum materials vary widely among States and localities. A beleaguered and embryonic network

of R&D centers established by the Office of Education over a decade ago has failed to achieve

notable successes. A persistent hostility exists

between the practitioner community and the research community, which makes it difficult for

communications to occur between them. The

textbook manufacturers place little reliance on

“See H.W. Bode, Synergy: Technical Integration and

Technological Innovation in the Bell System, Bell

Laboratories, Murray Hill, N. J., 1971.

‘2A. Hunter Dupree, Science in the Federal Government,

Harvard University Press, Cambridge, Mass., 1957.

systematic materials development and testing.

The components of the institutional environment

are unevenly developed, poorly linked, and frequently in disagreement over the process of

change in the schools. 13

An important contributor to the nature of the

institutional environment is the degree of consensus that exists concerning the appropriate Federal

role in a particular policy area. In chapter II, we

briefly sketched the evolution of the Federal role

in a number of policy areas. In areas such as national security or agriculture, a strong Federal

role is viewed as legitimate and is generally accepted and understood by the elements of the institutional environment. Similarly, there is fairly

general agreement concerning the limits of the

Federal role in promoting the use of basic research. However, in many areas of domestic policy and in energy policy there is considerable

debate over the appropriate Federal role.

The strength of the rationale for Federal involvement in these “mixed responsibility” sectors

varies substantially; more importantly, it is subject to a great deal of controversy. The involvement of the Federal Government in these sectors

has come about because important political constituencies have felt that the performance of

these sectors was inadequate for the needs of the

country. For example, the Federal Government

entered the health care financing area because

the health needs of the elderly and the poor were

not viewed as being adequately addressed by the

existing Federal policy in these areas. State and

local officials, whose operations are being curtailed by Federal activities, see Federal officials as

interfering in their legitimate policy responsibilities. When demonstrations become a tool to further controversial Federal policies, their legitimacy is questioned.

This problem is exacerbated by increasing

pressures from Congress and the public to take

results of publicly supported research “off the

shelf” and put them into practice. While we may

question whether sufficient R&D exists “on the

shelf” to justify this concern, the pressure has led

Federal R&D agencies to promote the commer13 For a description of the education R&D system and the

institutional environment which surrounds it, see The Status

of Education Research and Development in the United

States: 1976 Databook, The National Institute of Education,

Washington, DC., 1976.

25
cialization and use of R&D activities and to provide demonstration projects with a promient role

in this effort. The Federal Government must

choose specific technologies for demonstration.

These demonstrations will be chosen to advance

Federal policy goals, ghals that maybe in conflict

with those of State and local authorities or the

private sector. In the situation where such conflict

exists, the probability of success in promoting the

use of a new innovation or technology will be

low.

Another general factor that shapes the institutional environment is the extent to which an R&D

tradition is a part of the operations of a policy sector. For example, in the military sector, in civilian

technology, in agriculture, or in parts of the

health sector, there is a long tradition of using

R&D results to improve performance. In contrast, R&D in support of the improvement of the

criminal justice or educational systems is relatively recent. In the absence of a tradition of using

R&D and of training emphasizing the utility of

R&D to important actors within a sector, it is

unlikely that R&D and demonstration will play a

major role in the near future.

Judging the Development of Institutional Environments. -The examples and discussion suggest criteria by which to judge the degree of development of these institutional environments. A well-developed environment might

have the following attributes:

A set of institutions necessary for the entire

process of research, development, commercialization, and application.

Established, agreed-upon roles and responsibilities of these institutions (including the

Federal role).

Communication paths among these institutions along which information critical to the

R&D planning, utilization, commercialization, and innovation process can flow.

Communications among institutions that

are sufficiently frequent and strong to facilitate the movement of an innovation into

utilization.

Well-developed criteria at each stage of the

innovation process for evaluating the desirability of the movement from one stage to

the next.

These criteria for a developed institutional environment emphasize qualities that are likely to

26

lead to routine and continuing innovative activities. Equally important, sectors that have the

degree of development implied by these criteria

are likely to have developed a consensus about

the relevant scope of outcomes by which a prospective innovation should be evaluated; conversely, in less developed institutional environments, there is likely to be considerable disagreement over the outcomes of greatest importance.

The Interaction Between Technology and

the Institutional Environment. - An appreciation of these factors and the manner in which

they affect the success of demonstrations can be

gained by an examination of several specific

examples.

Central power generation can be contrasted

with solar heating and cooling, a decentralized

form of “power” generation. In both cases, the

technologies seem reproducible. We know a

great deal about their performance and are confident that we can learn more. Some aspects of

these technologies are currently well in hand

while others require further development. However, these two technologies are used in two

quite different institutional environments.

Central power station technologies are developed and used in an environment where the suppliers are known; the regulatory apparatus is in

place; and the market is orderly. ” In contrast,

solar heating and cooling is being developed and

used in a less developed environment; its application is controlled by hundreds of building

codes. The industry that supplies the technology

is new and unstructured. Financial and tax policies for installations are not yet routinized.

Regular sources of maintenance are not yet applicable. Suitable criteria for making decisions

about whether or not to invest in the technology

are not yet widely agreed upon. Thus, while we

are confident that we know or can know a great

deal about the performance of the technology,

the institutional environment for facilitating the

“This institutional environment is less well developed

now than several years ago because of the advent of both

energy and environmental concerns. These concerns have

resulted in new and as yet incompletely specified roles for

the Federal Government. The regulatory structure governing powerplants has been elaborated and new groups have

joined in powerplant design decisions Still. according to the

criteria presented in this section, central power generation

occurs in a developed institutional environment.
application of technology is not well developed. 15

Day care in industrial and community settings

provides an example of how institutional environments of differing levels of development can

shape the relevant scope of outcomes for demonstration projects. There is little agreement

among child care specialists concerning the levels

of inputs. competencies of care givers, and types

of activities that will produce various outcomes in

children in day care settings. The standards that

should govern day care financed by the Federal

Government have been

debate for a number of

considerable disagreement

the appropriate measures

care. 

subject to vigorous

years. There is also

over what constitutes

of outcome for day

In the public sector, day care is typically provided by a variety of institutions for the benefit of

the children, as well as the parents. Accreditation

for day care centers varies substantially among

jurisdictions. The standards for training of care

givers are highly variable. Nationally, the level of

subsidization for day care and the restrictions

placed upon the operation of programs that

receive subsidies are still matters of debate. The

institutional environment is poorly developed.

The relevant scope of outcomes is broad. The

technology has low reproducibility.

————.—

“For an interesting discussion of institutional environments in general and that surrounding solar heating in

particular, see Arthur A. Ezra, “Technology Utilization: incentives and Solar Energy, ” science, VOI. 187. Feb. 28?

1975, pp. 707-713.

‘

6A brief history of the controversy surrounding Federal

support for day care is contained in Gilbert Y. Steiner, The

Children Cause, The Brookings Institution, Washington,

D.C. , 1976, pp. 14-35.

Several years ago, as a result of the general

concern over extending day care to more parents

and the pressures from feminist and other

groups, a number of industrial firms experimented with day care. These firms provided day

care to small groups of their own employees on a

trial basis .1’ The firms quickly decided that the

relevant outcome of the day care they supported

would be reduced absenteeism and lower turnover among their personnel. These day care

centers met existing local standards for good day

care but did not consider provision of education

to children as a relevant goal. The demonstrations suggested that relatively few parents found

day care provided at the work site as desirable or

no improvement in turnover or absenteeism. The

idea seems to be largely abandoned. 18

In this case, within an industrial setting there

was agreement over the outcomes that were

important. Responsibility for the demonstration

was clearly assigned. And there was general

agreement over the meaning of the outcomes of

the demonstration among the potential providers

of industrial day care. The institutional environment was well developed. Moreover, the

technology associated with these outcomes was

reproducible. While there may be no reproducible means of providing day care that maximizes

cognitive growth in children, there is no problem

in creating day care centers that meet specified

(input) standards.

“Using our terminology, they conducted policyformulating demonstrations.

‘*’’Minding the Kid, Frustrating for Companies,” New

York Times, Sept. 11, 1977, p.F9.

27
Chapter IV

FACTORS AFFECTING THE

SUCCESS OF DEMONSTRATIONS

.
Chapter IV

FACTORS AFFECTING THE

SUCCESS OF DEMONSTRATIONS

The previous section developed a conceptual

framework with which to discuss factors that affect the success of both policy-formulating and

policy-implementing demonstrations. In this

chapter we advance several propositions concerning the determinants of success of demonstrations and assess the evidence contained in the

literature concerning these propositions. As in

the last chapter, our emphasis is on the use of

demonstrations as a logical step in an R&D process. Because the experience with policy-implementing demonstrations is rich, we begin our

discussion there.

Factors Affecting the Success of

Policy-implementing Demonstrations

Given the manner in which we have defined

policy-implementing demonstrations, the criteria

by which to judge their success is straightforward:

did the demonstration result in the diffusion and

use of the demonstrated innovation in other

sites? An innovation may be an industrial process, a new product, or a program design. In the

course of the innovation’s diffusion to other sites,

it may be expected to change somewhat in response to the needs and interests of that site. It is

a matter of judgment as to when the innovation

has sufficiently changed that it should no longer

be viewed as the same innovation.

The analysis above together with the literature

that we have reviewed suggest that a number of

propositions concerning the use of demonstrations that should guide policy makers at the

Federal level:

● Implementation demonstrations having

technologies with low reproducibility are

unlikely to lead to successful diffusion.

● If a technology is well in hand, a policyimplementing demonstration has a higher

●

●

●

●

probability of success than if the technology

is not well in hand.

Implementation demonstrations that are

conducted in a well-developed institutional

environment where there is a clearly accepted Federal role are more likely t o

achieve diffusion success.

An innovation requiring cooperative action

among elements of an institutional environment will be less likely to diffuse than one

with an application within the scope of a

single institution.

User need, as exemplified by non-Federal

initiative and cost/risk sharing, is an important factor in the success of policyimplementing demonstrations.

Lack of time constraints and operational

flexibility are important to the success of

policy-implementing demonstrations.

Each of these propositions is discussed in detail

in the following,

Technologies With Low Reproducibility

If the technology is perceived as being

nonreproducible, a demonstration incorporating

that technology will not be likely to lead to successful widespread diffusion of similarly performing processes or projects because potential users

will not be able to reproduce the demonstrated

process. Projects incorporating nonreproducible

technologies may provide ideas to other potential

users; they may suggest pitfalls to avoid in 1

plementing a project; they may even provit

equipment and materials that can be used b

others. But they seem unlikely to lead to similar

operations in other locations.

Available evidence is consistent with this proposition. Education, for example, possesses technology with low reproducibility. In their study of

Federal programs promoting educational

change, Berman and McLaughlin found no significant diffusion of the innovative projects sup31
ported under the programs. Their study showed

that (1) local staff preferred to develop their own

material, even when prepared material was available; (2) successful projects that differed substantially from usual practice were viewed as nonreplicable by staff from other locations; (3) rather

than search for alternatives, local staff preferred

to solve problems by using information already

known to local personnel; and (4) local personnel

tended to view their own site as unique. As a

consequence of these characteristics, the Change

Agent study hypothesized that significant innovation could be implemented only through a process of “mutual adaptation. ” It reported:

The initial design of an innovative project

must be adapted to the particular organizational

setting of the school, classroom, or other institutional hosts, and, at the same time, the organization and its members must adapt to the demands

of the project. Many educational innovations

may fail to have desirable effects because the

project is not adapted to the institutional setting

or vice versa during the implementation stage.2

This model of change requires unique interaction of the innovation with staff at each potential

adoption site. Given such a requirement, it

seems unlikely that exemplary projects at

selected demonstration sites will be replicated

elsewhere.

If diffusion is no longer a relevant success

criterion for such demonstrations because a

unique implementation of a technology with low

reproducibility is required at each site, Federal

policy makers should consider redefining the

goals of a program supporting such projects.

Perhaps these programs should consciously seek

site-specific development for use mainly at the

project site. In such cases, decisions to initiate a

project should be evaluated on the basis of the

costs and benefits associated with an individual

adoption. The use of the term “demonstration

project” would appear misleading for this type of

project. ’ We have referred to this type of pro-

‘Paul Berman and Milbrey Wallin McLaughlin, Federal

Programs Supporting Educational Change, The Findings in

Reuiew: Vol. IV, The RAND Corporation, R-1589/4-

HEW, April 1975, p. 1, hereafter referred to as the Change

Agent study.

‘Ibid, p. 6.

‘It should be noted, however, that experiences with this

type of program are not encouraging. The Change Agent

study found that only 5 percent of a sample of such local

development projects resulted in permanent incorporation

of the project.

32

gram as subsidized local development.

In a number of areas, innovative technologies

of substantial reproducibility are used in a larger

“system, ” where adaptation of the innovation

may occur. Examples include mass transit, law

enforcement hardware, and mining. Success of

demonstrations where such adaptation can take

place depends in part upon the perceived

similarity of the environment in which the demonstration is conducted to that of the potential

adapter.

Two examples from mass transit illustrate this

point. The Shirley Highway Express Bus Lanes

in the Washington, D .C., area used conventional

buses and highway and street lanes exclusively

designated for buses. The project measured actual ridership and thus avoided the theoretical

debates in the mass transportation community

about the relative importance to potential commuters of trip time, waiting time, transfers, or

comfort. The success of the system in attracting

commuters and in showing that the demonstration results are relevant to other metropolitan

areas with radial commuting corridors has led to

diffusion of the concept to other cities. ”

In contrast, the Personal Rapid Transit (PRT)

demonstration which, as we shall note, experienced many technical difficulties, was demonstrated in such a specialized situation that it

seems unlikely that it would have been persuasive even in the absence of technical problems. This system, involving unattended small

vehicles operating on a fixed track, was used to

connect parts of The University of West Virginia

in Morgantown. It seems unlikely that the conditions prevailing at this university campus would

be sufficiently similar to other contexts for this innovation to be persuasively demonstrated to city

planners. The demonstration did not provide

credible predictions of the relevant outcomes.

The existence of replicated projects may also

be important to the diffusion of innovations using

technologies with low reproducibility. Yin

discusses this issue with respect to innovation in

local law enforcement. In considering the Kansas

City Preventive Patrol Experiment as an exam-

‘Cheryl D. Hayes, “Toward a Conceptualization of the

Functions of Demonstrations, ” in Thomas K. Glennan, Jr.,

(cd.), Studies in the Managment of Social Research and

Development, The National Academy of Sciences,

Washington, D. C., forthcoming.
pie, he cautions that “findings . . . are n o t

facts.’” Unless successful results are experienced

at multiple sites, they may not be credible to

potential adopters. Diffusion of intermediate

strength technology innovations in the absence of

such replications is not likely.

Technology That Is Well in Hand

The degree to which a technology is in hand

will affect the success of policy-implementing

demonstrations. In projects using technologies

that are not well in hand, managers must concentrate on developing and improving the technology. The rhythm of the project will reflect the

technical needs of development activities. On the

other hand, the concerns of the adopters will frequently focus on questions such as the reliability

of the process, the administrative feasibility of implementing the process with the manpowe r

available at local sites, and the operating costs of

processes under routine conditions. If the technology is not well in hand, evidence concerning

these latter issues will be more speculative and

less clear. The demonstration may well be less

than effective in persuading adopters to make a

decision to implement a new innovation.

The Federal Demonstrations study provides

persuasive evidence that the lack of technology

well in hand adversely affects diffusion. This

study found little or no diffusion in any of the

demonstrations for which there was high technical uncertainty. Uncertainty was considered high

“when the technology has not previously been

prototype or field tested; or when basic problems with the technology are known to exist, and

techniques for dealing with them are not clear at

the time of the demonstrations. 6 With one exception, all of the examined demonstrations of

technologies with medium or low uncertainty

achieved at least some diffusion success unless

there were institutional problems. 7 A possible ex5Robert K. Yin, R&D Utilization by Local Services: Problems and Proposals ~or Further Research, The RAND Corporation, R-2020 -DOJ, December 1976, p. vii.

‘Federal Demonstrations. p. 31.

‘Medium technological uncertainty was considered to exist when technology had been tested at a lower scale but

uncertainties remained about performance at nearcommercial size or where uncertainties remained concerning performance of a new configuration of components.

Low technological uncertainty characterized projects using

existing components in configurations similar to previous

use. See Federal Demonstrations, p. 31.

planation for this finding is that the lack of uncertainty enables a realistic appraisal of the prospects for successful implementation. As a consequence, technologies that do not possess some

advantage over the status quo are not demonstrated.

The exception to this pattern—a demonstration of a seawater desalination process in

Freeport, Tex.—is instructive. The demonstration involved testing a large-scale version of a

desalination process previously tested only in a

pilot plant. The project was classified as having

medium preproject technological uncertainty.

The process required significant development

work during the course of the demonstration,

which resulted in important improvements in performance but also in interruptions in plant operations. At the end of the demonstration, the

technology could be considered well in hand.

However, potential adopters of desalination

plants mistakenly perceived these instructions at

the Freeport plant as an indication that the

desalination process was unreliable. These potential adopters failed to understand that the interruptions were necessary for technological

development work to occur.8

The PRT demonstration illustrates how the implementation of an undeveloped technology affects the operational results and the innovation’s

diffusion. The major leap in technology incorporated in the project led to rapid escalations in cost,

ultimately to a cost of over $60 million. To keep

costs down, the system was modified and curtailed, and the resulting system provided little

useful data on the possible spacing of vehicles,

the relative attractiveness of PRT to private

transportation, or other important measures.9

The Freeport desalination and the PRT cases

illustrate the importance of considering an alternative to a demonstration when a technology

contains some elements of uncertainty. When

high technological uncertainty exists, a full-scale

test at a test-bed facility should be considered.

This test of the technology at the developmentoriented facility would ease the development of

engineering solutions to technical problems by

removing real-world operating constraints.

“William F. Hederman, “Saline Water Conversion Plant,”

in Federal Demonstrations: Case Studies, pp. G 1-G46.

‘Federal Demonstrations, pp. 137-138.

33
Demonstrations in Well-Developed

Institutional Environments

Areas with well-developed institutional environments pose relatively minor problems for

Federal policies governing the use of demonstrations. Their very development means that the innovation system is working well and probably on

a continuing basis. Mutual agreement exists concerning when demonstrations are appropriate.

Sponsors and performers of demonstrations will

know which actors to involve and those actors

will understand the needs and patterns of success

within the policy area.

Poorly developed institutional environments

seem more likely to give rise to the type of goal

and value conflicts among the actors that have

been mentioned frequently in earlier sections of

this report. The absence of close ties among the

actors, lack of important institutional entities, or

the lack of consensus over the appropriate means

of bringing about and diffusing innovations all

lead to the possibility of such disagreements over

goals and values.

The empirical literature tends to support this

proposition. The Federal Demonstrations study

found that all demonstrations that examined

what had taken place in well-developed institutional environments had achieved at least some

diffusion success.

Moreover, the evidence indicates that the participation of the components of the institutional

environment is also important to diffusion success. The Federal Demonstration study found

that only one demonstration project that

excluded some active components of the institutional environment was diffused.

The exception to the observed pattern illuminates this proposition. A demonstration of mechanized refuse collection aimed at reducing labor

costs through smaller collection crews was a striking application success. The service, provision of

a mechanism to improve truck refuse collection,

was cost-effective and received strong public approval (exceeding 90 percent). However, the

mechanism was designed by city personnel, and

no garbage-truck outfitting firm could be found to

‘“Federal Demonstrations, pp. 51-53. The one exception

to this finding appears to have been examined too soon after

the demonstration for any meaningful assessment of diffusion to have been made.

market the new truck. Consequently, the diffusion observed was piecemeal and not selfsustaining, a disappointing result for an innovation based upon a reproducible technology that

was successful in its initial application.11

The innovative behavior of local governments

suggests the existence of goals other than those

that would normally be emphasized in the planning and execution of a demonstration project.

For example, in an investigation of technological

innovation in State and local services, Yin and

his colleagues found that evidence on the trial

and adoption or rejection of new technologies

supported two differing interpretations of how

local agencies operate, One model, termed a

problem-solving model, involves local identification of problems and a search for means to solve

the problems. The second emphasizes conditions

of bureaucratic self-interest that govern the innovations to be incorporated. Yin et al., in a

study of innovation, suggest that both models of

innovative behavior may operate at the State and

local level.

The important point to note in these findings is

that demonstrating a new technology is not an effective way to influence local problem solving

and the bureaucratic process. The character of

local innovation seems to spring from the characteristics of each locality and the needs that they

perceive, rather than from the opportunities presented by a demonstrattion project. This discrepancy may be part of the reason that Yin’s analysis

found no statistically significant relationship between a variety of Federal policies and successful

incorporation in local innovation. *3

Requirements for Cooperative Efforts

An innovation requiring cooperative action

among elements of an institutional environment

will be less likely to diffuse than one where its application is within the scope of a single institution.

Innovations can be significantly mismatched to

the institutional environment to which they are

intended to apply. In such instance, even if the

environment is developed and the technology is

free of uncertainty, diffusion may not take place.

“Federal Demonstrations, p. 96.

‘

zRobert K. Yin, Karen A. Heald and Mary E. Vogel,

Tinkering with the System, Chapter 5, Lexington Books,

Lexington, Mass., 1977.

‘

3

1 bid., Chapter 6.

34
The regional emergency medical service is a

case in point. The capability of specialized systems of emergency medical care to save lives has

been demonstrated. Necessary equipment,

personnel, and organizational procedures are

known and local governments and communities

seem to appreciate the need for such services.

Nonetheless, the development of these systems

has been sporadic.

A major reason for this sporadic development

is that the appropriate scale for such operations

exceeds the scale of most local political jurisdictions and hence requires cooperation across jurisdictional boundaries. Funds must be raised from

a variety of jurisdictions, central control and

management must be agreed upon, and a variety

of hospitals and other medical providers must cooperate. Because this type of joint effort is difficult to bring about, the diffusion of the innovation is slow and uncertain even though there has

been Federal funding to support it.

The use of cable telecommunications to provide social services seems likely to face similar

problems. Effective use of this medium will require cooperation among a number of social

service and regulatory agencies as well as several

levels of government. These agencies have disparate and conflicting goals and frequently resist

efforts of other agencies to enter their areas of

specialization. Diffusion of cable technology for

these uses will not occur easily.

User Need

A serious commitment to the innovation by the

performer is important to the successful operation of a demonstration. Both the Change Agent

study and the Federal Demonstrations study

found non-Federal initiative to be important to

demonstration success. The Change Agent

study’s description of the two types of project initiation encountered—opportunism and problem

solving—illustrates the problem well:

“The Comptroller General of the United States, Progress, but Problems, in Developing Emergency Medical Service Systems, Washington, D. C., July 13, 1976.

“For an interesting discussion of this point see William A.

Lucas, “Social Service Applications of Electronic Abundance, ” in Forrest Chisman and Glen Robinson (eds. ),

Communications for Tomorrow: Policy Perspectives for the

Future (tentative title), Aspen Institute, Washington, DC.,

forthcoming.

Projects generated essentially by opportunism

seemed to be a response to available funds and

were characterized by a lack of interest and commitment on the part of local participants—from

district administrations to classroom teachers. As

a result, participants were often indifferent to

project activities and outcomes, and little in the

way of serious change was ever attempted—or

occurred.

The problem-solving motive for projects

emerged primarily in response to locally identified needs and was associated with a strong

commitment to address these needs. Federal

funds were viewed as a way to support the local

solution—one which often broke new ground in

local educational practice. 16

Cost sharing by non-Federal participants is an

indication of interest. The Federal Demonstrations study found that demonstrations with large

shares of Federal funding (more than 90 percent)

had a poor chance of diffusion success.

The form of the cost sharing is also important.

For example, the share of costs for projects at the

State and local levels is frequently small and in

the form of contributed space and services. In

such instances, the type of opportunism that the

Change Agent study notes can easily occur. It

seems likely that larger contributions, involving

staff and financial resources, will assure that at

least some consideration is given to whether the

demonstration is consistent with the interests of

the local jurisdiction. For example, the mechanized refuse collection demonstration that was

cited earlier involved contributions of nearly 40

percent of the cost by the city of Scottsdale,

Ariz .

When the intent of the demonstration is to

promote the commercialization of a technology

by the private sector, a variety of forms of cost

sharing are possible. A close-ended contribution

can be made as was the case with the Atomic

Energy Commision’s Power Demonstration Reactor Program. 18 Loan guarantees can be provided as was contemplated in the proposed synthetic fuels legislation. 19 The demonstrating firm

“Change Agent study, p. 9

“Federal Demonstrations, p. 96.

18 Federal Demonstrations. Case Studies, pp I-55 and

1-56.

“U.S. Congress, House. 94th Cong., 2d Sess., Committee on Banking, Currency and Housing, Loan Guarantees

jor Demonstration of New Energy Technologies, report to

accompany H.R. 12112, Washington, D. C,, June 1976.

35
(or consortium of firms) can make a fixed contribution and the Government can support the remainder, as was proposed in the Clinch River

Breeder Reactor Demonstration .’” Tax subsidies

may be provided for specific types of investments. The Government may guarantee the

price of the output of a particular plant. Detailed

examination of these various methods of finance

is beyond the scope of this report, and in any

case must be made on a case-by-case basis.

Nonetheless, the literature provides a few guiding

principles.

The MIT Energy Laboratory Policy Group, in

examining policies for commercialization of

energy technologies, suggests three guidelines:

1. Subsidies for demonstration projects should

simulate the workings of the normal market

and should be as small as possible.

2. The mechanism for providing the subsidy

should ideally provide managers with a circumstance much the same as that in the unsubsidized case. In particular, it should be

neutral with respect to the choice of inputs

such as capital, labor, transportation, and

maintenance.

3. The financing mechanism should reveal the

full costs of the program and the detailed

cost performance of the individual

technologies. Loan guarantees are cited as

poor means of achieving this objective

because the degree of subsidy is not obvious. 21

These guidelines suggest simple, obvious cost

arrangements designed to elicit a commitment to

the effort by the private-sector firm.

Federal Demonstrations suggests that at the

beginning of planning for a demonstration project, a survey of possible participants should be

made to determine their willingness to participate

in the demonstration. Lack of willingness to

make substantial contributions to the cost may

well indicate that technological uncertainty is too

20 Leland L. Johnson, Edward W. Merrow, Walter S.

Baer, and Arthur J. Alexander, Alternative Institutional Arrangements for Developing and Commercializing Breeder

Reactor Technology, The RAND Corporation, R-2069 -

NSF, November 1976, pp. 117-120.

“Policy Study Group, Energy Laboratory, Government

Support for the Commercialization of New Energ y

Technologies, An Analysis and Exploration of the Issues,

MIT, Cambridge, Mass., November 1976, p. 11.

high to proceed with a demonstration, that

market demand is weak, that costs are high, or

that institutional factors or side effects are likely to

inhibit the use of the innovation .22 Thus, the

Federal Demonstrations study suggests that the

willingness to share costs may be an appropriate

“market test” for the demonstration of a new

technology.

This willingness cannot always be used as an

indicator of commitment. If the problem to which

the demonstration is addressed arises because of

State or local inattention or indifference, a strong

cost-sharing requirement may merely provide a

convenient excuse for that Government to avoid

response. There is no easy solution to this

dilemma.

Time and Operational Flexibility

In the Federal Demonstrations study, not one

demonstration that proceeded under strict time

constraints achieved any diffusion success. It is

also worth noting that none of these demonstrations produced the information necessary to

make an adoption decision, and only one was

successfully adopted at the demonstration site.

These time constraints had little to do with interest in diffusing the innovations. They were

primarily generated by other policy and political

needs .23

Operational flexibility allows the project performers to react to unanticipated events—either

to avoid or recover from negative events or to

take advantage of positive ones. Every demonstration encounters some unexpected situations

of varying importance. One reading project examined in the Change Agent study encountered

a freeze on the use of outside contractors who

were to print a locally developed test, the printing

of the wrong version of the test by student

printers, and a citywide teachers’ strike .24 In

another case, an attempt to extend a classroom

organization project to a junior high failed. By

completely redesigning the program consistent

with the elementary project but using different

techniques, a “significant degree of success” was

achieved .25

“Federal Demonstrations, p. 71.

231bid, pp. 53-54.

“Peter Greenwood et al., Federal Programs Supporting

Educational Change: The Process of Change, Vol. III, The

RAND Corporation, R-1589/3-HEW, April 1975, p. 43.

“Ibid, p. 44.

36
The Change Agent study found that attempts

to prescribe management techniques were “usually counterproductive, leading to nonimplementation or cooptation. ”26 The Federal Demonstrations study does not dwell on detailed management procedures for the hardware demonstrations except to state that “onsite management

was generally effective, ” meaning that project

management was not a major source of trouble in

the cases studied .27

Factors Affecting the Success of

Policy-Formulating Demonstrations

In comparison with policy-implementing

demonstrations, there has been relatively little

examination of the success or failure of policyformulating demonstrations. In part, this is

because there have not been many such demonstrations. More importantly, however, the complexity of the policymaking process makes it difficult to trace the use of information produced by

such demonstrations.

The making of policy seldom conforms to the

models of rational decisionmaking that are presented in the literature. At the Federal level,

policymaking extends over a long period of time

and often includes loosely coupled actions at

several levels of Government. For example,

debate over the Nation’s welfare system has been

going on for decades, with additions from time to

time of new information or new calls for national

concern. Perceptions concerning the fiscal health

of cities have shaped the nature of the program

solutions that have been advanced. The almost

inadvertent expansion of programs such as food

stamps changes the need for welfare reform.

Determining the role that a particular piece of information plays in shaping a particular policy is

virtually impossible,

Policymaking consists of identifying problems,

searching for possible courses of action, articulating and evaluating these courses of action,

and choosing one or a combination of the options. These are not necessarily sequential

events, however. The definition of the problem

may take place as a committee of Congress or a

legislative drafting group articulates and evalu2’Change Agent study, p. 26,

“Federal Demonstrations, p. 58,

ates proposed programs. A possible solution may

be shaped by policy actions in quite different

problem areas. The identification and articulation

of policy alternatives may occur in adversary proceedings or as the result of patient analytical staff

work.

Demonstration projects may make contributions to virtually all steps of the policymaking

process. The most obvious contribution is the articulation of the consequences of adopting a particular policy. Estimates of the costs, performance, and unanticipated effects of a policy or program may be obtained. However, the process of

designing the project may help to clarify the

nature of the problem and therefore to define the

objectives of a potential policy. Conducting a

demonstration may be a means of advocating the

consideration of a policy alternative by groups or

programs that have the interest and resources to

do so. It is even possible to imagine that an array

of projects which embody the various policy or

program alternatives could provide decisive information for choosing among policies.

Of course, many factors affect the outcome of

a policy debate besides the predicted performance of the policy itself. Distribution of benefits and costs among groups, regions, or industries may be important. The particular interests

and values of policy makers in important positions

will shape both the alternatives considered and

the choices made. Crises or events that require

immediate action may make quick decisions

imperative. Consequently, it will be very difficult

to provide sharp assessments of the success or

failure of demonstrations that seek to contribute

to the policymaking process.

The distinction between policy-formulating

and policy-implementing demonstrations was

originally made to categorize the purposes of

social demonstrations. 28 Social experiments such

as those dealing with housing allowances, income maintenance, health insurance, and utility

rate structures fit the description well. Other

social service and education projects can be identified that are tested in realistic environments to

determine if they should be put into effect in

large-scale programs.

“Cheryl D. Hayes, “Toward a Conceptualization of the

Function of Demonstrations, “ in Thomas K. Glennan, Jr.,

(cd.), Studies in the Management OJ Social Research and

Development, the National Academy of Sciences,

Washington, D. C., forthcoming.

37
While the concept of a policy-formulating

demonstration has not been used with more

technical hardware projects, it is clearly applicable. Within the military systems, tests of new

aircraft or weapons systems frequently provide a

basis for decisions as to whether or not to procure

the system and begin large production runs. A

number of demonstrations have also been carried

out as an aid to formulating regulatory policy. For

example, the aircraft industry and the airlines

under National Aeronautics and Space Adminillustration (NASA) sponsorship and with the participation of the Federal Aviation Administration

(FAA) designed and tested quieter engines to

reduce the noise at airports (Project Refan). Data

on costs and performance were obtained that

have helped to shape FAA policy in this area. ”

The detailed observation of Dunes airport landings and takeoffs of the Concorde supersonic aircraft is a demonstration to help establish appropriate noise level standards for such aircraft.

The complex nature of most policy decisions

precludes the sharp designation of criteria for

success or failure of a demonstration. Instead, we

suggest two attributes of a demonstration that

would seem to be necessary for success:

● Perceived quality of the information provides,

● Perceived relevance of that information to

the policymaking process.

We emphasize the term “perceived” because it is

difficult to specify intrinsic qualities of demonstrations that can be associated with all successful

demonstrations. “Success” in the creation of new

policy is likely to reflect the political resolution of

the value conflicts in question, and thus can be

only partially related to the outcome of a

demonstration project.

Perceived Quality

Policy makers in varied positions or at different

times will prefer different forms of evidence. For

example, during the late 1960’s and early

1970’s, many Federal executives pushed for improved experimental designs for demonstrations.

It was hoped that the use of such designs would

make the results of the demonstrations more

“Federal Demonstrations, p. 161: Peter W. House and

David W. Jones, Getting It Off the Shelj: A Methodology for

Implementing Federal Research, West View Press, Boulder,

Colo., 1977, pp. 232-234.

reliable and generalizable. Frequently, these

policy makers also sought independent evaluations because of their fear that evaluative information provided by the performers of the

demonstrations themselves would be subject to

self-serving bias. The observations of program

operators, State and local policy makers, or industry executives were viewed as unreliable and

insufficient bases upon which to make policy.

Other executives and many Members of Congress expressed quite different preferences. They

preferred the input of trusted colleagues or individuals who had provided useful and reliable

information in the past. They sought information

on a wider range of outcomes than was typically

provided by scientific evaluation designs. They

valued the ability to observe the actual demonstration, to talk with its managers and with clients

of the project.

The preferences for different types of information will depend in part upon the background factors mentioned above. In areas where there is a

tradition of R&D and where there is a belief in the

strength of the technological base, higher quality

scientific information will be important. The quality of the experimental design and the independence of the evaluator will be significant factors in

the policy makers’ judgment concerning worth of

the information. In those areas where there is no

such tradition or where the state of the technology is a matter of debate, more qualitative and

impressionistic information may be preferred.

Thus, these preferences are situation-specific.

Perceived Relevance

The relevance of the information depends

upon three factors: (1) the centrality to the policy

debate of the information produced by the demonstration, (2) the degree to which parties to the

policy debate jointly perceive the evidence as

valid, and (3) the timeliness of the demonstration

relative to the policy debate.

The central issues of a policy debate may be

difficult to predict. For example, the income

maintenance experiments provide information

on the labor force behavior of recipients of income maintenance payments. At the time these

experiments were initiated, this information was

expected to be a major issue in a debate on the

desirability of establishing a widespread income

maintenance system; indeed, in the early

38
debates this issue was important. However, the

debates have now continued for some 8 years

and other concerns have become more important. Today concerns are expressed about the

distribution of benefits among different classes of

recipients and different areas of the country, the

manner in which States will participate in the administration of the program, the degree to which

benefit levels will vary by area of the country, and

the effect the program will have on employment

programs. In part, this shift can be attributed to

Congress’s recognition that political issues surrounding welfare reform relate to the distribution

of costs and benefits. In part, however, the

reason for the receding relevance of the information provided by the income maintenance experiments may be that their findings were such as

to reduce the concern on the part of the lawmakers and the public that there would be

massive withdrawals from the labor force.

The success of a demonstration in aiding a

policy debate should depend upon the joint acceptance of the validity of the information by participants in that debate. If, for example, one goal

of a demonstration is to determine the environmental effects of a new process for the production of synthetic fuel and if both the promoters of the innovation and the interested environmental groups do not agree upon the

pollutants to be measured and the means of

measuring them, the evidence produced by the

demonstration and the policy debate is likely to

be less useful than would otherwise be the case.

In another context, one problem—although not

necessarily the most significant one—in determining the efficacy of Laetrile in the treatment of

cancer patients is that agreement among proponents and opponents as to what constitutes a

fair test is difficult to conceive.

The timing of demonstrations also poses

significant problems. Many policy decisions are

made as a result of forces quite independent of

the development and evolution of a technology.

When a crisis arises, such as the OPEC oil embargo or the urban riots, policies will be formulated and implemented whether or not there

is information from demonstrations or other

classes of R&D that would affect those policies.

Politicians in both executive and legislative

branches have short time horizons. They stand

for reelection at regular intervals or possess

career progression patterns that place them in

particular policy positions for short periods of

time. Policies are often made in response to

pressure from various interest groups. Because of

the fluidity of the situation and because of length

of time usually involved in the conduct of demonstrations, matching the results of demonstration activities to the policymaking process is

troublesome.

The difficulty in matching the timing of demonstrations to decisonmaking varies among

policies. Where the question is one of adopting

and promoting a major technological device such

as a new weapons system or a specific energyrelated process such as the breeder reactor, the

sensitivity of the policy to evidence concerning

the performance of the device may be so widely

appreciated that the policymaking will be paced

by the evolution of the technological work. A

policy-formulation demonstration in this case

would set the timing for policymaking.

In contrast, in cases where the technology is

clearly not perceived as central to the decisonmaking process, other imperatives that govern

the policymaking process will take over. Consider again the income maintenance experiment:

the underlying technology, the transfer of funds

to individuals contingent upon their other sources

of income, was perceived as being feasible from

the beginning. The demonstrations focused on

the work incentives, unanticipated outcomes,

and administrative processes. These issues were

not suffiently central to the policy debates that

the executive branch or Congress felt the debate

should await the completion of the demonstration.

This discussion leads us to advance two tentative propositions concerning factors affecting

the success of policy-implementing demonstrations:

●

●

The perceived relevance and usefulness of

policy-formulating demonstrations will be

enhanced when they are initiated and performed by agencies with close continuing

ties with policy makers.

Policy-formulating projects that give conscious attention to expected points of conflict in the policy debate and address these

conflicts in their design will be more successful than those that are developed along

the interests of just one party.
Continuing Close Ties With Policymakers

One theme pervades our discussion of factors

affecting the success of policy-formulating demonstrations. The success of such demonstrations

depends upon the styles and interests of relevant

policy makers. Demonstrations initiated and

managed by organizations that are closely linked

to the policy maker are likely to reflect those interests.

All the cited examples of policy-formulating

demonstrations have been initiated and managed

by organizations that bore close relationships to

executive branch policy makers. The social experiments have been sponsored and monitored by

Assistant Secretary level offices of policy planning and evaluation. The regulatory demonstrations were carried out by the cognizant regulatory

agency. We have reviewed no examples of policy-formulating demonstrations that have been

supported by relatively independent research

agencies such as the National Science Foundation or the National Institutes of Health. Thus, we

cannot provide evidence to test this proposition.

Although a major portion of the policymaking

process concerning the issues examined in these

experiments will be associated with the development of legislation, Congress has generally

played a minor role in the design of the experiments. Sponsors of demonstrations frequently do not relate closely to Congress. The housing

experiments, although mandated by law, were

implemented with little input from legislators.

The income maintenance experiments were designed on the basis of executive branch perceptions of what the policy issues would be. The

same was true of the health insurance experiment.

We have found two instances in which Congress mandated demonstrations to understand

the effects of a proposed policy change. The

1972 amendments to the Social Security Act

authorized a group of experiments and

demonstrations: 30

.

native methods for classifying providers, for

establishing prospective rates of payment, and

for implementing on a gradual, selective, or

other basis the establishment of a prospective

payment system.

Under this authorization, a number of demonstrations have been mounted; in addition, several

existing State efforts to limit the rate of increase

of hospital costs have been evaluated, which

essentially made these efforts policy-formulation

demonstrations .3*

In 1974, Congress authorized up to 20 experimental programs as a part of the Study of the

Effectiveness of Compensatory Education Programs. The primary purpose of these programs

was to determine the effects of changing the

bases upon which compensatory education funds

were allocated within a local education agency.

Under this authorization, 13 districts are e x -

perimenting with policies such as using academic

achievement rather than poverty criteria to

allocate compensatory education funds. 32

Including Points of Conflict in

Demonstration Design

The making of policy frequently involves

melding together a variety of conflicting views.

This is particularly true for the policies made by

Congress. As a consequence, we might hypothesize: policy-formulating demonstrations that

give conscious attention to the expected points of

conflict in the policy debate and address these

conflicts in their design will be more successful

than those that are developed along the interests

of just one party.

Explicit and participatory processes for seeking

the conflicting views were not apparent in the

planning of the demonstrations we examined.

However, in some cases, particularly those associated with the regulatory demonstrations, many

of the interested parties directly participated in

the conduct of the demonstration. In the case of

the Refan engine, manufacturers, the regulatory

to determine the relative advantages and

disadvantages of various alternative methods of

making payment on a prospective basis to

hospitals, skilled nursing facilities, and other providers of services for care and services provided

by them under Title XVII . . . including alter-

‘“Sec. 22 of Social Security Amendments of 1972.

31U. S. Department of Health, Education, and Welfare,

Social Security Administration, Office of Research and

Statistics, Research in Health Care Reimbursement,

Spring/Summer 1976, Publication No. (SSA) 77-11901,

Washington, D. C., 1976.

32National Institute of Education, Evaluating Compensatory Education, an interim Report on the NIE Compensatory Education Study, Washington, D. C., Dec.30, 1976,

Appendixes A and B.

40
agency, and the airlines all participated, and the

evidence from the demonstration does not seem

to have been a matter of dispute.

Kash et al., in a study of needed energy R&D,

proposed that many of the demonstrations of

energy supply technologies should be viewed as

means of obtaining evidence to resolve disputes

among concerned groups such as the environmentalists and industry .33 It certainly seems

desirable to assure that the concerns of various

parties in a dispute over the desirability of a

course of action should be reflected in the design

of the demonstration but it must be remembered

that some part—perhaps the major part—of the

dispute is over values possessed by various parties in a decision, and value conflict will not be

resolved by a demonstration project.

Even in cases where there are important

disputes over potentially knowable facts such as

the level of pollution or costs associated with a

particular production process, groups that expect

ultimately to oppose the adoption of an innova3

’Don E, Kash et al., Our Energy Future, University of

Oklahoma Press, Norman, Okla.. 1976, pp. 25-26.

tion (whatever its outcome) may oppose the conduct of a policy-formulating demonstration. Such

demonstrations are frequently seen as a means

by which a Government agency or industrial corporation that has already made its decision initiates action. These groups tend to doubt the

Government’s assurance that a demonstration is

focused primarily on obtaining information about

the desirability of pursuing a course of action.

This problem is heightened by the nature of

Government-industry relationships that seem

likely to lead to successful commercialization

efforts—i.e., close and sympathetic ties that promote a good exchange of reliable information.

If the two propositions advanced here are correct, Congress has important contributions to

make to the conduct of many policy-formulating

demonstrations. Where legislative action is expected to follow a demonstration, Congress

needs to assure that the design of the demonstration is relevant to its deliberations. In particular, it

must seek to ascertain that issues in which important political constituencies are likely to disagree

are treated in a manner that will help to resolve

the debate.

41
Chapter V

IMPLICATIONS FOR CONGRESS

.
Chapter V

IMPLICATIONS FOR CONGRESS

In this chapter we review the reasons that the

Federal Government may choose to use demonstration projects to promote the use of innovations. These reasons, together with the analysis

presented in the previous sections, suggest

several implications for Congress. Because demonstrations are not a uniformly attractive policy

instrument, we briefly review a number of alternative policies for promoting the use of innovations. We conclude with some questions to be examined in a congressional review of demonstration projects.

Reasons for Use of

Demonstrations Projects

The most important implication of the previous chapters is that demonstrations, as a component of the R&D process, have a relatively

narrow scope of usefulness. For policy-implementing demonstrations, the analysis and evidence suggest that desirable features for success

are a reproducible technology and one that is

“well in hand, ” together with a well-developed

institutional environment whose members are involved with the demonstration. Technologies

with low reproducibility or fragmented and undeveloped institutional environments of the types

associated with many domestic programs seem

unpromising areas for the use of demonstrations.

The situation is less clear for policy -

formulating demonstrations because the criteria

for success of such demonstrations are less welldefined and the experience, less extensive. They

typically are expensive activities relative to the

R&D that precedes them. On the other hand,

they may be inexpensive relative to the costs

(and risks) of a full-scale initiative.

The Demonstration Project as

a Political Tool

In contrast to their limited usefulness in the

R&D framework, demonstrations are considered

by many to be politically attractive. Demonstrations permit modestly priced responses to emerging political problems; they are, in a sense, a

means of symbolic action. Demonstration projects can show constituents that Washington is doing something for them. Demonstrations may be

a means of delaying policy decisions while additional information—both technical and political—is accumulated. Demonstrations are a convenient point of compromise between those who

would do much and those who would do little.

The Absence of Alternatives to

Demonstration Projects

Not only are demonstrations politically attractive but in many instances there appear to be few

feasible policy alternatives. If Congress presses

R&D funding agencies to promote the application of the results of their efforts, and the agencies are restricted to project grants (rather than to

changes in tax codes or widespread subsidy programs), demonstration projects, along with information dissemination systems, are about the

only tools available. If decisions concerning

regulatory policies—say defining an acceptable

level of effluents—will not be made in the absence of a specific case that forces such a determination, a demonstration project may be the

only way to force the decision. Because the traditional distribution of responsibilities between

Federal, State, and local government generally

precludes the direct intervention of the Federal

Goverment in local affairs, the demonstration

project may be the most attractive available tool

for trying to persuade the State or local communities of the importance of the national goal. Increases in regulated prices or changes in the tax

codes to make private-sector innovation more

profitable may be perceived as placing socially

unacceptable burdens on groups such as the

poor; demonstration projects are a possible

substitute. Thus, while the prospects for success with a demonstration project may not be

good, the demonstration may be the best

policy instrument available; and in some in45
stances, the importance of the problem may

dictate its use.

The Demonstration as an Instrument of

Transition: Inherent Difficulties

These qualities of demonstrations pose difficulties for Congress as it performs its legislative

functions. Moreover, the role of demonstration

projects as instruments of transition means that

Congress must proceed in the face of poor and

probably biased information. Ideas imbedded in

the project are moving from R&D to use;

technical criteria are being supplanted by institutional criteria of success. Projects are evolving

from the small scale typical of a laboratory to full

scale in the field. Control is shifting from R&D

personnel to operating personnel. In many instances, this transition is associated with a movement from public to private-sector sponsorship

and management. Alternatively, the innovation

may shift from Federal emphasis and sponsorship to State and local use. Not only is a transition such as this difficult to bring about, but it

gives rise to strong advocacy by individuals and

groups with a stake in the innovation.

R&D personnel who have developed a new

technology often wish to proceed to the next

logical step, a demonstration in the field. Industry, particularly that part associated with

earlier development work, may seek a subsidy

for continuation into the demonstration phase.

Final users such as State and local officials may

well see the innovation as a means for enlarged

political power or career advancement. Because

of the transitional nature of the demonstration,

however, most actors have only limited perspectives concerning the worth of the innovation.

The developer may not be aware of all the institutional impediments to the application of the

innovation. The final users may have insufficient

understanding of the nature of the technology

and may have a large personal stake in its application. The industrial firm may see the opportunity to obtain a subsidy for an effort that it

might otherwise have to fund itself. In any case,

Congress or the senior executive branch officials

will be faced with conflicting information.

Incentives for Government Agencies

to Use Demonstrations

When considering policies to promote the

commercialization of new technologies by the

46

private sector, Congress

note of the nature of the

the behavior of Federal

should take particular

incentives that govern

R&D agencies. These

agencies are under continual pressure from the

public and Congress to show that their programs

have resulted in technologies and knowledge that

are being used. One of the most straightforward

means of obtaining an example of use is to create

a demonstration project. A demonstration project may thus be viewed as a simple means of

placating critics rather than as a component of a

well-developed strategy to promote the use of

the results of an R&D program.

Even in public bureaucracies where there is a

thoughtful policy for promoting the use of technology, there are some undesirable incentives.

As we have noted, it is the reality of the R&D

process that there is both uncertainty concerning

the worth of an idea that is being developed and

a likely division of opinion about the most likely

outcome of its application. This should result in

dropping or sharply redirecting a substantial proportion of R&D efforts before they reach fruition.

In the private sector, decisions to continue or

cease development efforts are disciplined by the

realities of the market, which provides a measure

of outcome to which all members of the firm must

adhere. In general, the structure of a business

firm also makes it clear who will be responsible

for various levels of decision. If a member of the

firm does not agree with the decision that is

made, he has little recourse except to leave the

firm.

In the bureaucracy of the public sector, the incentives are different. Frequently, the bureaucracy will possess multiple goals. In energy, for example, program goals encompass both the development of a technological base and the commercialization of new technologies. More importantly, decisons concerning projects to be initiated or continued are subject to considerable

public scrutiny. Public officials do not have the

luxury of simply making a decision because they

have the responsibility and authority to make that

decision; they also must justify those decisions

before a considerable number of outsiders, many

of whom have quite different views concerning

the prospects for a technology, the goals of a program, or the appropriate policy tools to be used.

Employees of the agency or their friends outside

the agency can frequently appeal the decisions to

higher agency officials or to Congress itself. The
debate over the demonstration of the Clinch

River breeder reactor shows that even a Cabinet

Secretary and the President are unable to make

decisions that cannot be overturned.

The incentives of the Government program

manager may also be somewhat different from

those of his counterpart in the private sector. The

latter program manager will frequently report to

higher level corporate officers who have had considerable experience with the development of

new products and processes. This program manager can easily be rewarded for knowing when to

cut off a development as well as for promoting

promising ventures because the firm’s management appreciates the risks associated with new

product development. In contrast, the public official may frequently have to face the scrutiny of an

array of elected officials and citizens who do not

share his views of the uncertainty and who take

the cessation of a project as an indication of

failure on the part of the program manager. As a

consequence, he may push for a demonstration

project despite its low probability of success in

promoting the use of an innovation.

These qualities of a Government agency seem

likely to inhibit its effective participation in making

sound decisions concerning the commercialization of products and processes developed. It will

take such bureaucracies a long period of time to

make decisions because so many points of view

must be considered. Similarly, it will take time to

implement decisions when they are controversial

because of the several levels at which the decisions can be overturned. It may well be that the

necessary absence of profit-oriented goals within

public bureaucracies will increase the influence of

personal career-oriented incentives of Government officials in the decision process.

Implications for Congress

The narrow scope of usefulness of demonstrations in achieving R&D objectives, the political attractiveness of those demonstrations, the difficulty of achieving the variety of transitions that

they seek to make, and the nature of the incentives for Federal R&D agencies suggest four characteristics of programs containing demonstration

projects that Congress might encounter.

1. Congress should expect a low rate of

success with demonstrations as a

means of promoting the use of a

technology.

The transition of an innovation from development to use is difficult under any circumstances.

It is likely that Federal officials promote technologies having objectives that are not fully shared

by either the private sector or by State and local

authorities. Demonstrations may frequently not

be the policy instrument of first choice but rather

the only politically feasible instrument available.

Each of these circumstances makes success difficult.

2. Congress should expect that the information it receives concerning the

potential of a proposed demonstration

wiII probably be biased and imperfect.

Again, the goals of transition mean that there

are few, if any, experts that have appropriate experience or are in institutional settings that do not

bias their judgments. Moreover, the fact that

demonstrations are frequently the product of a

political decision process means that the parties

to that process will have different goals for the activity and hence different assessments of likely

success.

3. Congress should expect that there w

be frequent confusion over the goa

of a demonstration project.

The inherent semantic confusion over the ter

II

s

n

“demonstration” constitutes the first problem.

Demonstrations can be used to prove as well as

to display a concept. In addition, the perspectives

of the actors involved in the demonstration process may be quite different. Some may be certain

that the innovation has been proved and seek

only to promote its use; some may be dubious

and seek to discover its worth; still others may

see the demonstration as an easy way to dispose

of a political problem and thus will be primarily

concerned with satisfying important constituen -

cies.

4. Congress should expect that the evaluation of the success or failure of a

demonstration will be difficult and

judgmental.

As policies are formulated, a demonstration

can provide important but rarely decisive infor47
mation. Moreover, the distribution of policymaking through time and among different institutions and levels of Government makes tracing

the value of the outcome of demonstrations nearly impossible.

While at first glance, evaluation of policy-implementing demonstrations appears to be simple,

our analysis suggests that it is not. The possibility

of goal conflict between Federal and non-Federal

sectors, the strength of a technology, or the

quality of an institutional environment all affect

the outcome. A particular demonstration project

may not have spawned a large group of replications but may have illuminated the institutional

problems so that future efforts to promote the use

of new technologies will be improved. The project may have brought a policy problem to the attention of local government and resulted in continued attempts at that level to deal with the

problem. It may have sustained an R&D capability in one or more private-sector firms that will

lead to successful new innovations. It may have

forced the resoution of uncertainties over environmental or other regulations that will make

subsequent investment planning easier.

In light of these expectations, Congress should

consider whether other types of policy or program actions will better serve their ends or will

complement and enhance the possibilities of successful demonstration projects.

Alternative Strategies to

Promote the Use of R&D

Five suggested strategies for improving the

chances of successful demonstration projects are:

● Conduct engineering tests.

● Change market incentives.

● Modify the institutional environment.

● Subsidize local development.

● Utilize existing projects.

The Congress, and the Federal Government in

general, can benefit by considering these strategies as either alternatives or complements to

demonstration projects.

Conduct Engineering Tests

The evidence suggests that if there is a high

degree of uncertainty associated with technology, a policy-implementing demonstration will

have a high probability of failing to achieve its objectives. If this is thought to be the case, a largescale prototype test or test-bed experiment may

be needed. It may be less expensive, quicker,

and more decisive than a demonstration project

that has to deal with the problems of a real-world

setting. If a major barrier to commercial use of a

new technology is technological uncertainty, the

resolution of that uncertainty may be a sufficient

means of promoting the use of the technology.

In the development of large-scale technology

there frequently is pressure both to advance

technology and to demonstrate its usefulness in a

single project; this appproach can have unfortunate outcomes. Experience with the development of major weapons systems is instructive,

although the problems arising in those systems

are not fully analogous to the policy problem

discussed here. The military has often attempted

to compress development times of weapons

systems by initially purchasing a sufficient

number of completed weapons systems permitting realistic operational testing. In many cases,

the production of these test systems has begun

before all the technological uncertainties have

been resolved. The result has been increased

costs, lengthened time schedules, and eroded

levels of performance.

Despite this experience, pressures for early

test and demonstration continue. Enthusiastic

supporters of a weapons system do not want it to

be discarded and thus seek the maximum commitment to the system. The industrial producers

want to minimize the risk of cancellation. Most

importantly, developers and sponsors of new

technologies perpetually seem to underestimate

the amount of uncertainty that exists with a new

technology. Thus,

● The Congress should seek to ensure

that it does not add to the incentives

for premature demonstration of an

undeveloped technology by pressing

to turn engineering tests into fullscale demonstrations.

IR. L. Perry, Reforms in

Corporation, P-5482, July

System Acquisition, The RAND

1975.

48
Change Market Incentives

Barriers to increased market penetration of an

innovation frequently have little to do with technological, market, and environmental uncertainties. For example, consider the case of synthetic

fuel plants. While there are some technical uncertainties and important issues concerning water

rights and environmental damage, the major

constraints on commercialization are economic.

The price of alternative sources of fuels are simply too low and are expected to remain low

enough to prevent the private sector from investing in large synthetic fuel plants.2 In part, the

low price reflects a decision to maintain these fuel

prices below the long-term replacement costs

through regulation in the interests of other policy

objectives. Moreover, the market does not take

account ef the value of the increment of “national

security” associated with substituting domestically produced synthetic fuel for foreign oil.

Perhaps, too, the market does not adequately

reflect the interests of future generations who will

suffer the consequences of depleted natural

fuels.

If the Government determines that it is in the

national interest to promote the design and construction of synthetic fuel plants, it could change

(or eliminate) the regulations on petroleum prices

or it could subsidize the output of synthetic fuel

plants. Both actions would allow potential producers of synthetic fuels to respond to marketlike forces in ways that seem profitable to them. If

such changes are made and alternative sources

of fuels are still not developed, either the

technology is so uneconomical or the environmental and social impacts so unacceptable that

the private sector cannot be expected to develop

such plants. If it is politically infeasible to change

the regulation of prices or to provide a continued

subsidy to new sources of fuel, partial sharing of

the cost of an initial “demonstration” plant may

be deemed an appropriate form of subsidy. In a

sense, the use of a demonstration is a “secondbest” solution to commercialization. One should

note, however, that in this instance it is unlikely

that the demonstration will lead to diffusion of the

technology. The fundamental reason for the lack

of commercialization has little to do with the

absence of the type of knowledge produced by

2

Edward W. Merrow, Constraints on the Commercialization OJ Oil Shale, The RAND Corporation, forthcoming.

the demonstration. Moreover, if the subsidy were

high, that is, if the Government paid a substantial

and possibly open-ended share of the cost, many

of the valuable indicators of economic viability

would be lost. In sum,

● When a demonstration is proposed to

overcome Government-created market imperfections, serious consideration should be given to altering those

imperfections as an alternative to

large-scale involvement in demonstration projects.

Modify the Institutional Environment

The analysis in chapter IV suggested that demonstration programs were not likely to be successful in promoting the diffusion of technologies

when the institutional environment is poorly

developed. As either an alternative or complement to demonstration projects, the Government

may wish to alter the institutional environment.

Earlier we noted that the Department of Agriculture has sought to develop a strong institutional environment as a means of improving the

productivity of agriculture through R&D. It was

able to do this in part because of the overwhelming importance of agricultural production in the

last century. The resulting system has been the

product of more than a century of development,

has many critics as well as admirers, but has provided a framework within which strong

technologies could develop and diffuse.

It is difficult to imagine that the Nation would

want to create for other sectors a system as elaborate as that serving agriculture. Less extensive

changes can be made, however. For example,

the current efforts to promote nationwide performance standards for construction can be seen as

a means of simplifying and strengthening the

institutional environment. Funding for improved

planning, analysis, and research capabilities in

local governmental units may be an effective

means of improving their capabilities to assess

and utilize new technology. Standards developed

by the National Bureau of Standards may help

the market perform more satisfactorily. Assistance to new industries, either through tax incentives or through specialized technical assistance,

may strengthen the institutional environment. On

occasion, the Federal Government can promote

49
improvements by bringing elements of State governments together to seek common perceptions

of social problems and solutions. Federal authorities can also foster the development of new curriculum and support the training of personnel in

the skills necessary to improve the innovative

capabilities of the State and local government.

As with any policy, such attempts to improve

the institutional environment should be preceded

by efforts to determine the real nature of the

problem. For example, it has been argued that

local schools lack the capacity for useful problem

solving. Consequently, proposals have been

made for educational extension agents, teacher

centers, technical assistance agencies, and other

forms of assistance. It is quite possible, however,

that the real reasons for the lack of desired improvements stem from the incentive structures

that face teachers and administrators at the local

level, from the type of training that is provided in

teacher’s colleges, or from the pressures that

changing societal norms are imposing upon the

schools. Developing better problem-solving skills

may accomplish little. In the absence of good

problem diagnosis, attempts to improve the institutional environment may simply lead to additional expensive complexity in the environment.

To paraphrase,

● Congress should regard policies that

seek to develop institutional environments as complements to R&D

policies.

Subsidize Local Development

When a technology of low reproducibility or a

poorly developed institutional environment is

coupled with a demonstration, Congress may

want to subsidize local development efforts to

focus attention on a problem of national concern.

Such subsidies permit local units of government

to implement the forms of new technologies that

they want and to modify them in ways that seem

appropriate to their particular situations.

There is a growing literature on these types of

programs. ’ In general, it has been found that a

‘See, for example, Robert K. Yin, Karen A. Heald and

Mary E. Vogel, Tinkering with the System, Lexington

Books, Lexington, Mass., Chapter 5, 1977; and Paul Berman and Milbrey Wallin McLaughlin, Federal Programs

Supporting Education Change, Vols. I-IV, The RAND Corporation, R-1589 1-4-HEW, April 1975.

disappointingly small number of programs survive the withdrawal of the Federal subsidy. Par-t

of this low-survival rate reflects the opportunistic

behavior that characterizes some local government and educational institutions. Also, the

bureaucratic needs of local governments may not

be served by many of the developments that are

attempted. Innovation is never easy, There is a

wide range of plausible explanations for the low

rate of incorporation of results, and it would be

premature to suggest that the observed rate of incorporation is the best that can be obtained.

Thus,

● The Congress should seek the type of

experimentation and evaluation that

will cast light on the best means to

provide subsidies for local development and to incorporate the findings

in new or revised legislation.

Utilize Existing Projects

For some social policies, existing projects or

programs may provide information that can only

be marginally improved by introducing a new

policy-formulating demonstration. In the

congressionally mandated program examining

means to provide incentives for efficient delivery

of health care, such an approach was taken.

Health care reimbursement systems already

under development or in use by several States

were evaluated to provide information about the

potential effects of several different approaches

to the problem.

There are many advantages to using existing

projects. Since they already exist, information

can be more quickly obtained than would be the

case with a new project. Many of the costs of

starting a project will already have been incurred.

It is possible that the information on implementation problems will be more reliable than would be

the case with a federally funded demonstration.

On the other hand, the existing project may inadequately reflect important policy options; the

project managers may resist being evaluated by

Federal agencies; or the project may be structured in ways that make determination of outcomes difficult or impossible. The last case would

occur if several policies had been simultaneously

implemented in a way that prevented the separation of outcomes attributable to each policy. To

summarize,

50
● In contemplating possible policy-formulating demonstrations, the execu -

tiue branch and Congress should first

consider existing sources of insight

concerning the policy.

Suggested Questions To Be

Examined in Congressional Review

of Demonstration Projects

In cases where these suggested strategies are

either undesirable or infeasible or where demonstration projects seem to be the most attractive

means for achieving a policy goal, the analysis

presented in this report indicates a number of

questions that Congress should bear in mind as it

performs its legislative functions:

Are the goals for a demonstration project

clearly articulated and agreed upon?

Given the purposes of a demonstration

project, have the information needs of the

demonstration’s audience been adequately

considered?

If the demonstration is intended to promote

the diffusion of an innovation, have the

relevant parts of the institutional environment been involved?

IS the technology underlying the innovation

adequately developed?

Is there sufficient evidence of commitment

to the demonstration by the performer?

Does the design of the demonstration project reflect the experiences of past demonstrations?

Each of these questions is discussed in the following pages.

● Are the goals for a demonstration project clearly articulated and agreed

upon?

In some instances, demonstration projects fail

simply because of conflicting goals among

funders, performers, and potential audiences. As

noted, divergence in opinions concerning goals

for demonstrations are inevitable given the project’s policy role. Inadvertent lack of clarity in

goals should be avoided, however. When authorizing demonstration projects or programs,

Congress should clearly state its intent. When

overseeing the conduct of programs under these

authorizations, Congress should seek to ensure

that program managers and the performers of

demonstrations share an understanding of the

purposes of demonstrations.

● Given the purposes of a demonstration project, have the information

needs of the demonstration’s audience been adequately considered?

Demonstrations, whether conducted to inform

Federal policy or to promote the use of an innovation, should be focused on the information

needs of its particular audience. A careful

analysis of their needs should precede the initiation of a demonstration. It is important to decide

what information a demonstration must provide

and how it will be conveyed. It is equally important to decide what information a demonstration

cannot generate and make that clear to the intended audience. The Congress should seek to

ensure that this has been done.

● If the demonstration is intended to

promote the diffusion of an innovation, have the relevant parts of the institutional environment been involved?

Evidence and common sense suggest that the

institutions important to the success of the demonstration should be involved in its planning and

conduct. This includes, for example, those who

must change the existing regulations to permit

the use of an innovation, the industrial firms that

will supply the innovation, and the professional

groups that will sanction its use.

Involvement does not necessarily mean participation. It is clear it would be difficult and timeconsuming to arrange the participation of all relevant elements of an institutional environment in

decisions concerning the design of a demonstration project. Involvement may include, for example, consultation, specialized dissemination of

results, participation in evaluating projects, as

well as participation in the demonstration project

itself. Congress should ascertain that adequate

attention has been given to this involvement.

● Is the technology underlying the innovation adequately developed?

Evidence in the literature suggests that when

substantial technological uncertainty surrounds a
proposed innovation, the purposes of the demonstration can be compromised. Costs will

escalate. Potential users of the innovation may

perceive it as unreliable. Compromises in the

conduct of the demonstration will then have to be

made to accommodate resolution of the technical

problems. Congress should seek to ensure that

the technology associated with the demonstration is sufficiently well in hand to preclude

technological problems from dominating the outcome of the demonstrations.

● Is there sufficient evidence of commitment to the demonstration by the performer?

The difficulties and the complex array of incentives associated with any demonstration project make commitment on the part of the participants an important factor in its success. In the

private sector, the best indicator of such commitment is the willingness to assume a substantial

share of the costs of the demonstration. Experience also suggests that projects that are conceived by the private or State and local sectors

themselves are more likely to have this commitment than projects developed in response to

detailed solicitation by Federal agencies.

In a few cases, this type of indicator maybe inappropriate. A willingness to invest funds implies

a belief that the conduct of the demonstration is

clearly consistent with the goals of a firm or

organization. For some innovations, however,

there will be Federal interest in promoting goals

that are not currently consistent with those of

private firms or public agencies. For example, the

use of cable television to deliver social services to

the aged and the poor may require changes in

the behaviors of social service agencies and costs

to cable television franchises that are unlikely to

be recovered through fees. It may be impossible

to implement an innovation like this in the

absence of almost total support from the Federal

Government. Demonstrations of programs dealing with a new and difficult clientele face similar

problems. Much of the Federal involvement in

elementary and secondary education was

brought about by the failure of the Nation’s

school systems to deal adequately with the needs

of disadvantaged students. Given that these

schools did not consider this problem to be important, it is unlikely that they would have been

willing to make a substantial financial commit52

ment to demonstration projects

problems of the disadvantaged.

dealing with the

In general, however, Congress should abandon cost sharing as a measure of commitment

only if the goals of the demonstration are of

significant national importance, and if the potential gains from the demonstration outweigh the

higher probability of its failure when the performers are unwilling to make financial commitments.

● Does the design of the demonstration

project reflect the experiences of past

demonstrations?

The failure of a demonstration project to contribute to formulating a policy or promoting the

use of an innovation is not necessarily an indication of wasted public funds. As we have

repeatedly noted, the task that a demonstration

project seeks to perform is difficult. Apparent

project failures may well reflect problems with the

institutional environment, conflicting goals at differing levels of government, or other factors

discussed in this report rather than poor management or inadequate funding.

These failures are likely to contain important

lessons and may point the way to better future

policy and program design. For example, while

the ambitious demonstration of educational television has left comparatively little in the way of

significantly changed patterns of instruction, it

has heightened our knowledge concerning the

process for changing these instructional patterns.

Cable television demonstrations and experiments

have clarified the conflict between the incentives

driving the expansion of commercial markets for

cable systems and the desire to use such systems

to enhance the access to services by disadvantaged groups in society. Demonstrations of solar

heating and cooling have served to highlight aspects of the institutional environment to be

developed.

The lessons learned from these demonstrations can lead to policies to encourage the development of the institutional environment. They

can provide guidance for future technological

developments. Congress should promote efforts

to learn these lessons, and, where appropriate,

encourage new demonstrations that reflect them.
—

The Future Use of sponsors and performers. But demonstrations

Demonstration Projects are also easily misused. They can lead to waste,

frustration, and discouragement. We hope that

Demonstrations constitute fascinating policy the concepts and guidelines developed in this

tools. They provide opportunities to try innova- report will help Congress and others to improve

tions; they can be used to promote important the use of demonstrations in pursuing national

causes; they are exciting experiences, for both goals.

53
BIBLIOGRAPHY
—.

BIBLIOGRAPHY

Allen, Wendy, Nuclear Reactors for Generating

Electricity: U.S. Development from 1946 to

1963, The RAND Corporation, R-21 16-NSF,

June 1977.

American Medical Association, Report of the

Commission on Medical Research, Chicago,

111., 1967.

Andrus, E. C., et al. (eds. ), Advances in Military

Medicine, Vols. I and II, Little, Brown and

Company, Boston, Mass., 1948.

Arrow, Kenneth J., “Economic Welfare and the

Allocation of Resources for Invention, ” in

Richard R. Nelson (cd.), The Rate and Direction of Inuentiue Actiuity: Economic and

Social Factors, a Report of the National

Bureau of Economic Research, Princeton

University Press, Princeton, N. J., 1962.

Arthur D. Little, Inc., Federal Funding of Civilian

Research and Deuelopment, Volume 1: Summary, Washington, D. C., February 1976.

Baer, Walter, et al., Analysis of Federally

Funded Demonstration Projects: Supporting

Case Studies, The RAND Corporation, R1927 -DOC, April 1976.

Baer, Walter S., Leland L. Johnson, and Edward W. Merrow, Analysis of Federally

Funded Demonstration Projects: Final Report,

The RAND Corporation, R-1926 -DOC. April

1976.

Baer, Walter S., Leland L. Johnson, and Edward W. Merrow, “Government-Sponsored

Demonstrations of New Technologies, ”

Science, Vol. 196, May 27, 1977.

Berman, Paul, Milbrey Wallin McLaughlin, et al.,

Federal Programs Supporting Education

Change, Vols. I-V, The RAND Corporation,

R-1589/l-HEW, September 1974 and R1589/2-5-HEW, April 1975.

Bode, H. W., Synergy: Technical Integration and

Technological innovation in the Bell System,

Bell Laboratories, Murray Hill, N. J., 1971.

Chaiken, Jan, What is Known About Deterrent

Effects of Police Activities, The RAND Corporation, P-5735-1, July 1977.

Committee on Public Engineering Policy. National Academy of Engineering, Priorities for

Research Applicable to National Needs, report

to the National Science Foundation, National

Academy of Engineering, Washington, D. C.,

1973.

Comptroller General of the United States, Progress, but Problems, in Developing Emergency

Medical Service Systems, Washington, D. C.,

July 13, 1976.

Dupree, A. Hunter, Science in the Federal

Government, Harvard University Press, Cambridge, Mass., 1957.

Eads, George, “U.S. Support for Civilian Technology, Economic Theory, and Political Practice, ” Research Policy, Vol. 3, 1974.

Eads, George, and Richard R. Nelson, “Governmental Support of Advanced Civilian Technology —Power Reactors and the Supersonic

Transport, ” Public Policy, Vol. 19, 1971.

England, J. Merton, “Dr. Bush Writes a Report:

‘Science—the Endless Frontier, ’ “ Science,

Vol. 191, Jan. 9, 1976.

Ezra, Arthur A., “Technology Utilization: Incentives and Solar Energy, ” Science, Vol. 187,

Feb. 28, 1975.

Gramlich, Edward M., and Patricia P. Koshel,

Educational Performance Contracting, An

Evaluation of an Experiment, The Brookings

Institution, Washington, D. C., 1975.

Hayes, Cheryl D., “Toward a Conceptualization

of the Functions of Demonstrations, ” in

Thomas K. Glennan, Jr. (cd.), Studies in the

Management of Social Research and Development, The National Academy of Sciences,

Washington, D. C., forthcoming.

House, Peter W., and David W. Jones, Getting It

Off the Shelf: A Methodology for implementing Federal Research, West View press,

Boulder, Colo., 1977.

Johnson, Leland L., Edward W. Merrow, Walter

S. Baer, and Arthur J. Alexander, Alternative

institutional Arrangements for Developing and

Commercializing Breeder Reactor Technology, The RAND Corporation, R-2069-NSF,

November 1976.

Kash, Don E., et al., Our Energy Future, University of Oklahoma Press, Norman, Okla.,

1976.

Keenan, Terrance, “The Health Record of Private Foundations, ” Journal of Health Politics,

Policy and Law, “ Vol. 2, No. 1, Spring 1977.

57
Kevles, Daniel J., “The National Science Foundation and the Debate over Postwar Research

Policy, 1942-1945: A Political Interpretation

of ‘Science—The Endless Frontier, ’ “ /S1S,

Vol. 68, No. 241, 1977.

Lilley, William, III, and James C. Miller, 111, “The

New ‘Social’ Regulation, ” The Public Interest,

No. 47, Spring 1977.

Lucas, William A., “Social Service Applications

of Electronic Abundance, ” in Forrest Chisman

and Glen Robinson (eds. ), Communications

Jor Tomorrow: Policy Perspectives for the Future (tentative title), Aspen Institute, Washington, D. C., forthcoming.

Merrow, Edward W., Constraints on the Commercialization OJ Oil Shale, The RAND Corporation, forthcoming.

‘&Minding the Kid. Frustrating for Companies, ”

New York Times, Sept. 11, 1977, p. F9.

MIT Energy Laboratory Policy Study Group,

Government Support for the Commercialization of New Energy Technoogies: An Analysis

and Exploration of the Issues, Novembe r

1976.

National institute of Education, Evaluating Compensatory Education, an interim report on the

NIE Compensatory Education Study, Washington, D. C., Dec. 30, 1976, Appendixes A

and B.

Nelson, Richard R., “The Economics of Invention: A Survey of the Literature, ” Journal of

Business, Vol. XXXII, No. 2, April 1959.

Nelson, Richard R., Merton J. Peck, and Edward

D. Kalachek, Technology, Economic Growth,

and Public Policy, The Brookings Institution,

Washington, D. C., 1967.

Nelson, Richard R., and Sidney G. Winter, “In

Search of Useful Theory of Innovation, ”

Research Policy, Vol. 6, 1977.

Pechman, Joseph A., and P. Michael Timpane

(eds. ), Work incentives and Income

Guarantees: The New Jersey Negative income Tax Experiment, The Brookings Institution, Washington, D. C., 1975.

Perry, R. L., Reforms in System Acquisition, The

RAND Corporation, P-5482, July 1975.

Riecken, Henry W., and Robert F. Borouch

(eds. ), Social Experimentation: A Method for

Planning and Evaluating Social lntervention,

Academic Press, New York, 1974.

Rein, Martin, Social Policy: issues of Choice and

Change, Random House, New York, 1970.

Rivlin, Alice M., Systematic Thinking for Social

Action, The Brookings Institution, Washington, D. C., 1971.

58

Rivlin, Alice M., and P. Michael Timpane (eds.),

Planned Variation in Education: Should We

Give Up or Try l-larder?, The Brookings Institution, Washington, D. C., 1975.

Schon, Donald, “The Blindness System, ” The

Public Interest, No. 18, Winter 1970.

Schultze, Charles L., “The Public Use of the Private Interest, ” Harper’s, Vol. 254, May 1977.

Social Security Administration, Office of Research and Statistics, Research in Health Care

Reimbursement, Spring/Summer 1976, Publication No. (SSA)77-11901, Washington,

D.C. , 1976.

Steiner, Gilbert Y., The Children’s Cause, The

Brookings Institution, Washington, D. C.,

1976.

Stricland, Stephen P., Politics, Science, and

Dread Disease: A Short History of United

States Medical Research Policy, Harvard

University Press, Cambridge, Mass. 1972.

Study Committee on Social R&D, The Federal

Investment in Knowledge of Social Problems,

The National Academy of Sciences, Washington, D. C., forthcoming.

The Status of Education Research and Development in the United States: 1976 Databook,

National Institute of Education, Washington,

D. C., 1976.

U.S. Congress, House, 94th Congs., 2d Sess.,

Committee on Banking, Currency and Housing, Loan Guarantees for Demonstration of

New Energy Technologies, report to accompany H.R. 12112, Washington, D. C.,

June 1976.

Wenk, Edward, Jr., “The Social Management of

Technology,” in John E. Mock (cd.), Science

for Society, proceedings of the National

Science Conference held in Atlanta, Ga., Oct.

12-14, 1970.

Williams, Walter, Social Policy Research and

Analysis: The Experience in the Federal Social

Agencies, American Elsevier Publishing Company, New York, 1971.

Yin, Robert K., R&D Utilization by Local Seruices: Problems and Proposals for Further

Research, The RAND Corporation, R-2020 -

DOJ, December 1976.

Yin, Robert K., Karen A. Heald, and Mary E.

Vogel, Tinkering with the System, Lexington

Books, Lexington, Mass., 1977.

York, Herbert F., and G, Allen Greb, “Military

Research and Development: A Postwar History, ” Bulletin oj the Atomic Scientists, Vol.

33, No. 1, January 1977.
———.— . —.—— — — .-— — —

ACKNOWLEDGMENTS

During the course of this study we benefited from numerous conversations with colleagues at RAND. We are indebted to Alvin Harman, Milbrey McLaughlin, Dan Weiler, and

Al Williams for their helpful comments during the early stages of the study. Walter Baer, Paul

Berman, Edward Merrow, Barbara Williams, and Robert Yin both contributed to the conceptualization of the study and provided thoughtful critiques of drafts along the way.

We benefited greatly from two meetings with the Panel on Applications of Science and

Technology, Office of Technology Assessment. The wide range of perspectives represented

on the panel resulted in challenging and productive exchanges. Don E. Kash, Nathan

Caplan, and Richard R. Nelson provided helpful comments on an earlier draft. Finally, we express particular appreciation to our Program Officer, John Young, whose interest led us to

undertake this research and whose continued thoughtful questioning has contributed substan -

tially to the report.

I 1

) I 1, ‘ . I . 1 I 1’ 1~. 1 .,( 1 } 1[ i I - f - . I

59