Federally Funded Research: Decisions for

a Decade

May 1991

OTA-SET-490

NTIS order #PB91-198101
Recommended Citation:

U.S. Congress, Office of Technology Assessment, Federally Funded Research: Decisions for

a Decade, OTA-SET-490 (Washington, DC: U.S. Government Printing Office, May 1991).

For sale by the Superintendent of Documents

U.S. Government Printing Office, Washington, DC 20402-9325

(order form can be found in the back of this report)
Foreword

The Nation relies on federally funded research to address many national objectives. With

global competition, changing student demographics, rising demands for research funds, and

the prospect of constricted budgets, Congress and the executive branch must make difficult

choices in supporting U.S. science and engineering. The House Committee on Science, Space,

and Technology asked OTA to examine the Federal research system—a conglomeration of

many separate systems that sponsor, oversee, and perform research-and the challenges that

it will face in the 1990s.

Given the exceptional history, strength, and character of U.S. research, there will always

be more opportunities than can be funded, more deserving researchers competing than can be

sustained, and more institutions seeking to expand than the prime sponsor—the Federal

Government--can fund. The objective for government, then, is to ensure continued funding

for a full portfolio of first-rate research and a high-caliber research work force to assure

long-term scientific progress. This report analyzes what OTA identifies as four pressing

challenges for the research system i-n the 1990s: setting priorities in tiding, understanding

trends in research expenditures, preparing human resources for the future research work force,

and supplying appropriate data for ongoing research decisionmaking. Managing the Federal

research system requires more than funding; it means devising ways to retain the diversity and

creativity that have distinguished U.S. contributions to scientific knowledge.

The advisory panel, workshop participants, reviewers, and other contributors to this study

were instrumental in defining the key issues and providing a range of perspectives on them.

OTA thanks them for their commitment of energy and sense of purpose. Their participation

does not necessarily represent endorsement of the contents of this report, for which OTA bears

sole responsibility.

~zf #&# M---

JOHN H. GIBBONS

w Director

iii
Federally Funded Research: Decisions for a Decade

Advisory Panel

Bernadine Healy, Panel Chair

Chairman, Research Institute, Cleveland Clinic Foundation

William Carey Herman Postma

Consultant to the President Senior Vice President

Carnegie Corp. Martin Marietta Energy Systems

Purnell Choppin

President

Howard Hughes Medical Institute

Herbert Doan

Consultant

Midland, MI

Gertrude Elion

Scientist Emeritus

Burroughs Wellcome

Robert Fossum

Professor, Electrical Engineering

Southern Methodist University

S. Allen Heininger

Corporate Vice President

Monsanto Co.

Donald Holt

Director

Illinois Agricultural Experiment Station

Larry Smarr

Director

National Center/Supercomputer Applications

University of Illinois, Urbana

Amy Walton

Manager

Science Data Analysis and Computing Laboratory

Jet Propulsion Laboratory

Sheila Widnall

Abby Mauze Rockefeller Professor of

Aeronautics and Astronautics

Massachusetts Institute of Technology

George Wise

Specialist-Communications

General Electric

Ex-Officio Members

Ted G. Berlincourt

Director

Research and Laboratory Management

Todd LaPorte Office of the Secretary of Defense

Professor of Political Science

University of California, Berkeley James Hays

Acting Senior Science Advisor

Leon Lederman National Science Foundation

Professor of Physics

University of Chicago Ruth L. Kirschstein

Director

Cora Marrett National Institute of General Medical Sciences

Professor of Sociology National Institutes of Health

University of Wisconsin, Madison F. Karl Willenbrock

William Massy Assistant Director

Vice President for Finance Scientific, Technological and International Affairs

Stanford University National Science Foundation

Alan McGowan

President

Sister Agency Member

Scientists’ Institute for Public Information

Richard Rowberg

Director, Science Policy Research Division

Robert Myers Congressional Research Service

Assistant Professor of Agronomy Library of Congress

University of Missouri, Columbia

NOTE: OTA appreciates and is grateful for the valuable assistance and thoughtful critiques provided by the advisory panel members.

The panel does not, however, necessarily approve, disapprove, or endorse this report. OTA assumes full responsibility for the

report and the accuracy of its contents.

iv
Marsha Fenn

Office Administrator

Federally Funded Research: Decisions for a Decade

OTA Project Staff

John Andelin, Assistant Director, OTA

Science, Information, and Natural Resources Division

Nancy Carson, Program Manager

Science, Education, and Transportation

Daryl E. Chubin, Project Director

Elizabeth Robinson, Principal Analyst

James Netter, Research Assistant

Katherine Evans, Summer Intern

Administrative Staff

Harvey Averch

Florida International University

David S. Birdsell

City University of New York

Susan E. Cozzens

Rensselaer Polytechnic Institute

Patrick W. Hamlett

University of Missouri, Rolla

Kathi E. Hannal

Churchton, MD

Institute for Scientific Information, Inc.

Henry G. Small, principal investigator

Gay Jackson

Administrative Secretary

Contractors

Tamara Cymanski

Secretary

Political Economy Research Institute, Inc.

John W. Sommer, principal investigator

Mark A. Pollock

Temple University

James D. Savage

University of Virginia

Herbert W. Simons

Temple University

Techmonitor, Ltd.

Ron Johnston, principal investigator

IP~.~e ow Sefior analyst July-December 1990.
Contents

Page

Chapter 1. Summary and Issues for Congress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Chapter 2. The Value of Science and the Changing Research Economy . . . . . . . . . . . . . . . . . . . 49

Chapter 3. The Federal Research System: The Executive and Legislative Branches . . . . . . . . . 71

Chapter 4. The Federal Research System: The Research Agencies . . . . . . . . . . . . . . . . . . . . . . . . 97

Chapter 5. Priority Setting in Science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .137

Chapter 6. Understanding Research Expenditures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

Chapter 7. Human Resources for the Research Work Force . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

Chapter 8. Data on the Federal Research System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Appendix A. Major Legislation Enacted Since 1975 Affecting

U.S. Research and Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .261

AppendixB. The Top 1OO Institutions Ranked by Amount of Federal R&D

Funding Received: Fiscal Year 1989 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .263

Appendix C. Funding Allocation in Six Federal Research Agencies . . . . . . . . . . . . . . . . . . . . . . . 266

Appendix D. Academic and Basic Research Decisionmaking in Other Countries . . . . . . . . . . . 272

Appendix E. Top lOO U.S. Academic Institutions Ranked by Citation Impact 1981-88 . . . . . 283

Appendix F. Contractor Reports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

Appendix G. Workshop Participants and Reviewers and Contributors. . . . . . . . . . . . . . . . . . . . . 286

Appendix H. References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 305
CHAPTER 1

summary and Issues for congress
Contents

Page

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

This Report and Its Origins . . . . . . . . . . . . . . . . . . . . . . . ..,..+.. . . . . . . . . . . . . . . . . .,.,,.4. . . . . . 4

Trends in Federal Research Funding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Federally Funded Research in the 1990s . . . . . . . . . . . . . . . . . . . , +. . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Issues and Options for Congress . . . . . . . . . . . . . . . . . . . . . ...,..,. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

ISSUE l: Setting Priorities in the Support of Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Discussion . . . . . . . . . . . . . . . . . . . . . . . ...+..... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

ISSUE 2: Understanding Research Expenditures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ● . . . . . . . . . . 22

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

ISSUE 3: Adapting Education and Human Resources To Meet Changing Needs . . . . . . . . . . . . . 28

Discussion . . . . . . . . . . . . . . . ....**.. . . . . . . . . . . . . . . . . . . . . . . . . +....,.. ,.+...., ......,, + 29

ISSUE 4: Refining Data Collection and Analysis To Improve Research Decisionmaking .+... 36

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .....,.+. . . . . . . . . . . . . . . . . . . . . . 36

Toward Policy Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ,. 41

Boxes

Box Page

l-A. How Much Is Enough? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..,,.+.. . . . 12

1-B. Criteria for Research Decisionmaking in Agency Program s.. .,.+.... . . . . . . . . . . . . . . . . . . + 16

l-C. How Many “Scientists’’ Are There? . . . . . . . . . . ,,.+.... . . . . . . . . . . . . . . . . . . . . . . . . . . . +... 38

Figures

Figure Page

l-1. Federally Funded Research (Basic and Applied): Fiscal Years 1%0-90 .. .. +.. .,. ..+.,... 5

l-2. Doctoral Scientists and Engineers in Academic R&D: 1977-87 .+, .. .. ., ... .., .+.,.., . . . 5

l-3. Federally Funded Research in the Major Research Agencies Fiscal Years 1960-90 +...... 8

l-4. Federally Funded Research by Broad Field: Fiscal Years 1960-90 . . . . . . . . . . . . . . . . . . . . . . . 8

l-5, Federally Funded Research by Performer Fiscal Years 1969-90 . . . . . . . . . . . . . . . . . . . . . . . . 9

l-6. Federal R&D Obligations by State (1985) and at Universities and Colleges (1989) . . . . . . . . 9

l-7. Cost Scenarios for the Science Base and Select Megaprojects: Fiscal Years 1990-2005 ..., 19

l-8. Estimated Cost Components of U.S. Academic R&D Budgets: 1958-88.....+......,,.+. 24

l-9. Percentage Distribution of Doctorates by Science and Engineering Field: 1960-89 . . . . . . . . 30

l-10. Federal Support of Science and Engineering Graduate Students, 1969 and 1988........+. 30

l-11. Science and Engineering Degrees: 1966-88 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Tables

Table Page

l-1. Tensions in the Federal Research System......,,,.. . . . . . . . . . . . . . . . . . . . . . . . . . . ..+. . . . . . 7

l-2. Federally Funded Research in the 1980s and 1990s .. .. .. .. .. .. . .+ ... ......+. . . . . . . . . . . . 11

l-3. Summary of Issues and Possible Congressional Responses . . . . . . . . . . . . . . . . . . . . . . . . ...++. 13

l-4. Congressional Authorization Committees and Appropriations Subcommittees With

Significant Legislative Authority Over R&D . . . . . . . . . . . . . . . . .. .. .. .. .. .,, .+. . . . . . . . . . . . 15

l-5. Desired Data and Indicators on the Federal Research System ...+,... . . . . . . . . . . . . . . . . . . +. 42

l-6. Summary of Possible Congressional, Executive Branch, and Research Performer Actions . . . 43
CHAPTER 1

Summary and Issues for Congress

Introduction

Research provides extraordinary benefits to society through the creation of new knowledge and the

training of scientists and engineers. The research and

higher education system in the United States is the

envy of the world, and has a long history of

advancing the state of scientific knowledge. This is

known as “scientific progress”: “. . . not the mere

accumulation of data and information, but rather the

advancement of our codified understanding of the

natural universe and of human behavior, social and

individual." 1

These advances have addressed such

goals as enhancing the Nation’s public health,

military security, prestige, educational achievement,

work force, technological development, environmental quality, and economic competitiveness.

To say only that research contributes to national

goals, however, simplifies and understates a complex system. Research is no longer a remote,

scientist- or engineer-defined activity resulting in

new knowledge for society. Perhaps it never was.

“Deeply held political values of democratic accountability and public scrutiny have naturally and

inevitably impinged on science policy. Demands for

observable benefits from public investment in science increase. ’ Such demands have led to claims

that scientitic research has a signficant and direct

impact on the economy, and that an investment in

knowledge is a downpayment on the products and

processes that fuel U.S. economic growth and

productivity. 3 Economists admit, however, that the

difficulties in measuring the benefits of research

" . . . are hard to exaggerate. ’ The Nation now

expects that in addition to knowledge, science and

engineering will contribute to U.S. prestige and

competitiveness abroad, create new centers of research excellence on a broad geographic basis,

continue to provide unparalleled opportunities for

Photo credit: Research Triangle Institute

Scientists at the Research Triangle Institute, NC,

synthesize chemicals for cancer research. Scientific

research takes place in many settings in the United States.

education and training, and nurture a more diverse

research work force.

Thus, the Federal Government funds research to

achieve more than specific national goals. By doing

so, it invests in knowledge-and the people who

produce it—not only for its intrinsic worth (which

can be considerable), but also for the value knowledge acquires as it is applied.

Scientific research is typically split into two

categories, “basic” and “applied.” Basic research

pursues fundamental concepts and knowledge (theories, methods, and findings), while applied research

focuses on the problems in utilizing these concepts

and forms of knowledge. OTA does not generally

 Brooks, “Knowledge and Action: The Dilemma of Science Policy in the 70s,” vol. 102, spring 1973, p. 125. Unless otherwise

stated, ‘science’ in this report includes the social and behavioral sciences as well as the natural sciences and engineering. “Research” refers to a creative

activity ongoing in of these fields.

 “The Public and Science Policy,” Science, Technology, & Human Values, vol. 7, spring 1982, p. 13. 3

See Edwin Mansfield, “The Social Rate of Return From Academic Research,” Research Policy, forthcoming 1991; and James D. Adams,

“Fundamental Stocks of Knowledge and Productivity Journal of Political Economy, vol. 98, No. 4, 1990, pp. 673-702.

‘Quoted in Eugene Garfield, “Assessing the Benefits of Science in Terms of Dollars and Sense,”The Scientist, vol. 4, No. 22, Nov. 12,1990, p. 14.

The source is Nathan Rosenberg and David C. Mowery, Technology and the Pursuit of Economic Growth (New NY: Cambridge University Press,

1989).

- 3 -
4 • Federa//y Funded Research: Decisions for a Decade

distinguish between these categories in this report,

because policymakers, especially Congress, make

very few decisions in which the two are separate. In

particular, research agency program managers rarely

allocate monies on the basis of a project’s basic or

applied classification, and divisions of research

funding into these categories are often unreliable.5

This Report and Its Origins

In December 1989, the House Committee on

Science, Space, and Technology requested that OTA

assist it in understanding the state of the federally

funded research system—its goals, research choices,

policies, and outcomes—and the challenges that it

will face in the 1990s. By requesting a study of the

state of the Nation’s research system and of alternative approaches the Federal Government could take

in funding research, the Committee sought information on the nature and distribution of research

funding and decisionmaking. Direct congressional

involvement in research decisionmaking is growing,

and annual agency appropriations seem more closely

tied to specific goals-and tough choices among

them-than ever before.6 As one member put it:

. . . the payoffs for the Nation are so great that

increased investments in science and technology are

only prudent. However, even if we could double the

science budget tomorrow, we would not escape the

need to establish priorities. ...7

The Federal Government has sustained an illustrious history of support for research. Underlying this

relationship between government and the scientific

community was a social contract or ‘‘trusteeship,’

developed after the scientific breakthroughs spurred

by World War II, that delegated much judgment on

Federal research choices to scientific experts.8

Perhaps the epitome of the trusteeship was the

research grant, which created a new relationship

between the Federal Government and the research

performer, especially the principal investigator in

universities. 9 This social contract implied that in

return for the privilege of receiving Federal support,

the researcher was obligated to produce and share

knowledge freely to benefit—in mostly unspecified

the public good.10 and long-term ways—

Since the 1960s, Federal funding for research

(both basic and applied) has increased from roughly

$8 billion in 1960 (1990 dollars) to over $21 billion

in 1990 (see figure l-l). Funding increased quickly

in the early 1960s during the “golden years’ for

research, after the launch of the Sputnik satellite, the

escalation of the Cold War, and the Presidential

commitment to land men on the Moon. Once these

challenges had been met, research funding decreased

5Aq@er-wnqago it was noted ht.o “The precise partitioning of all basic research into components is, of course, largely arbitrary. Basic research

can be classified in terms of its motivation-as culture, as an adjunct to educatiou as a means to accomplish nonscientific goals of the society; of its

sources o~support—whether mission-oriented agency or science-oriented agency; of its pe~ormers-whether university, government laboratory, or

private industry; or of its ctuzracter—whether ‘little science’ or ‘big science. ’ Any one of these classitlcations, if applied consistently, cover all basic

science, but none is wholly satisfacto~. ., .“ See National Academy of Sciences, Committee on Science and Public Policy, BasicResearch andNational

GoaZs, A Report to the Committee on Science and Astromutics, U.S. House of Representatives (Washington DC: March 1965), p. 9, italics added. This

was independently confiied by extensive OTA interviews with research agency persomel, spring-s ummer 1990. Today, research is also sometimes

labeled ‘strategic, “ “targeted,’ or “precompetitive,” for example. For an update and discussion see Harvey AverclL ‘The Political Bconomy of R&D

Thxonomies,” Research Policy, forthcoming 1991.

6see ~atio~ ~de~y of Sciences, Federal Science ad Technology B&get priorities: New Perspectives and Procedures (wuhkgto@ DC:

National Academy Press, 1989); and U.S. Congress, House Committee on Science, Space, and Technology, Subcommittee on Science, Research, and

Technology, The Hearings inadequacy, Direction and Priontiesfor the Amen”can Science and Technology Effort, IOlst Cong., Feb. 28-Mar. 1, 1989

(Washington, DC: U.S. Government Printing OMce, 1989).

TDoug Wdgrem ~n of the House Subcommittee on Science, Research, and Technology, in House Committee on Science, Space, and

Technology, op. cit., footnote 6, pp. 1-2.

8Re5emhm apl-ed activi~ of ~eFeder~ Govement can~ traced to ~o 1~~kvol~es:V~evmBush’s 1945 ‘ ‘A Report to the President

on a Program for Postwar Scientific Research” (subsequently known as Science; The Endless Frontier), which instigated the creation of an agency-the

National Science Foundation-whose dual mission was the promotion of research and science education, and Science and PubZic Policy, or the 1947

Steelman Repo~ which championed a crosscutting policy role for managing federally funded research. For interpretations, see J. Merton England, A

Patronfor Pure Science: The National Science Foundation’ sFormative Years, 1945-57 (Washington DC: National Science Foundation, 1982); and

Deborah Shapley and Rustum Roy, Lost at the Frontier (Philadelphia, PA: 1S1 Press, 1985).

gsee U.S. Con=ess, House Committee on Science and Technology, Task Force on Science Policy, A History of Science policy in the United States,

1940-1985, 99th Cong., September 1986 (Washington DC: U.S. Government Printing Office, September 1986), pp. 19-20. Also see Rodney W. Nichols,

“Mission-Oriented R&D,” Science, vol. 172, Apr. 2, 1971, pp. 29-37.

I%or examinations, see Bruce L.R. Smith, American Science Policy Since World War ZZ (Washington DC: The Brookings Institution, 1990),

especially chs. 1 and 3; Gene M. Lyons, The Uneasy Partnership: Social Science and the Federal Government in the Twentieth Century (New York

NY: Russell Sage Foundation 1969); and U.S. Congress, OffIce of Technology Assessmen~ The ReguZatoryEnvironnwntfor Science, O’E4-TM-SET-34

(Washington, DC: U.S. Government Printing Office, February 1986), pp. 15-16.
Chapter l—Summary and Issues for Congress ● 5

Figure 1-1—Federally Funded Research (Basic and .

Applied): Fiscal Years 1960-90

(in billions of 1982 dollars)

20

15

10

5

0

J\.

/’”

7“

I I 1 I I 1 I I I I 1 1 I I 1 1 I I ( I 1 I I I I I I I L

20

15

10

5

0

1960 1965 1970 1975 1980 1985 1990

Figure 1-2—Doctoral Scientists and Engineers in

Academic R&D: 1977-87

In thousands

160

140

120

100

80

60

4 0

20

0 —

See note —

—

1977 1979 1981 I 198 3 1985 1987

— Basi c + Applied ~ Total research

NOTE: Figures were converted into constant 1982 dollars using the GNP

Implicit Price Deflator. For 1990 (current dollars), basic research =

$11.3 billion, applied research= $10.3 billion, and total research=

$21.7 billion. 1990 figures are estimates.

SOURCE: National Science Foundation, Federal Funds for Research and

Development, Detailed Historical Tables: Fiscal Years 1955-

7990 (Washington, DC: 1990), table A; and National Science

Foundation, Selected Data on Federal Funds for Research and

Development: Fiscal Years 1989, 1990 and 1991 (Washington,

DC: December 1990), table 1.

slightly and leveled off from the late 1960s until the

mid-1970s. From 1975 onward, however, Federal

research funding again increased, due in large part to

the expansion in health and life sciences research.ll

Along with this increase in research funding, the

number of academic researchers grew steadily,

NOTE: There was a change in the wording of the National Science

Foundation survey questionnaire of academic Ph.D.s in 1987:

respondents were asked to identify whether “research” was their

primary or secondary work activity. This change may have resulted

in an artificially large increase from 1985 to 1987 in “academic

researchers.” Prior to 1987, Ph.D.s in academia were only asked to

identify their primary work activity.

SOURCE: National Science Board, Science & Engineering lndicators7989, NSB 89-1 (Washington, DC: U.S. Government Printing

Office, 1989), appendix table 5-17 and p. 115.

perhaps by as much as 60 percent from 1977 to 1987

(see figure 1-2).12 More generally, from 1980 to

1988, scientists and engineers in the work force grew

by an average of 7.8 percent per year, four times the

annual rate for total employment. 13 Not surprisingly,

the competition for research funds among these

scientists and engineers also intensified. By the late

Ilsee Natiorlal Science Fo~&tioq Federal FU&S for Research and Development-Detailed Historical Tables: Fiscal years 1955-1990

(Washington, DC: 1990). For discussions, see William D. Carey, “R&D in the Federal Budget: 1976- 1990,” Science and Technology and the Changing

World Order, Colloquium Proceedings, Apr. 12-13, 1990, S.D. Sauer (cd.) (Washingto~ DC: American Association for the Advancement of Science,

1990), pp. 43-51; and Genevieve J. Knezo, “Defense Basic Research Priorities: Funding and Policy Issues,” CRS Report for Congress (Washington

DC: Oct. 24, 1990).

lzNote, however, that there was a change in the wording of the National Science Foundation Survey qUeStiO tie, which may have resulted in an

artificially large increase from 1985 to 1987 in those that ident@ “research” as their primary or secondary work activity. Prior to 1987, Ph.D.s in

academia were only asked to identify their primary work activity. This probably underestimated the number of academic Ph.D. researchers in the United

States. See National Science Board, Science & Engineering indicator.+-1989, NSB 89-1 (Washington+ DC: Congressional Research Service, 1989),

app. table 5-17.

131bid., p. 67.
6 ● Federally Funded Research: Decisions for a Decade

1980s, researchers supported by the Federal Government had become increasingly restive over funding.

Today, many say that their lives as researchers have

become more stressful and laden with the paperwork

of proposal applications and accountability for

awarded finds, inhibiting the creativity and joy of

the research process. 14 Theyy cite the declining

fraction of meritorious proposals that are funded,

new investigators lacking the support to set up

independent research groups, and the fear that U.S.

students will turn their careers away from academic

science and engineering.15

Today, because the scientific community has the

capability to undertake far more research than the

Federal Government supports, policymakers and

sponsors of research must continuously choose

between competing “goods.” (The tensions underlying these choices are summarized in table l-l.)

Controversies over the support of younger scientists

and established researchers, “have” and “havenot” institutions, and tradeoffs among fields are all

manifestations of the consequences of choices

perceived by various segments of the "scientific

community." 16 Scientific community, as used here,

refers to apolitical entity. Like other sectors, science

contributes to national goals and competes for

Federal resources. At a more practical level, the

scientific community invoked by Congress and the

Presidential Science Advisor refers to a heterogeneity of professional associations, lobby activities, and

actual research performers. (These disciplinary or

subject-specific divisions and interest groups more

accurately correspond to what OTA calls ‘‘research

communities.

Additional funding for science and engineering

research would certainly be a good investment of

Federal resources. There is much that could be done,

and many willing and able people and institutions to

do it. The focus of this report, however, is not on the

level of investment, but on the “Federal research

system.” As the sum of the research programs and

efforts that involve the support of the Federal

Government, the “system” is best characterized as

the conglomeration of many separate systems, each

with constituencies inside and outside of science.17

How these participants compete, cooperate, and

interact in processes of Federal decisionmaking

determines which research is funded by the agencies

and performed by scientists and engineers.

If large increases in the budget were to materialize, it would not necessarily relieve system stresses

for long. Additional research funding would certainly allow the pursuit of more scientific opportunities and yield fruitful gains, but it would also enlarge

the system and increase the number of deserving

competitors for Federal support. Thus, such stresses

must be addressed with other policies. In the short

term, the government faces a rising budget deficit.

Congress has set targets to reduce the deficit and

eventually to balance the budget.18 In this fiscal

climate, the research system may not be able to

maintain the growth in Federal funding of research

that it experienced in the 1980s. Regardless of

funding levels, however, issues of management,

tiding, and personnel remain.

Given the extraordinary strength of the U.S.

research system and the character of scientific

research, there will always be more opportunities

14Science: The End of the Fronfier7 A ~epfi from ~nM. ~derm~ president.Elect to &e Bo~d of Dir~tors of the American Association fOr dle

Advancement of Science (Washingto~ DC: American Association for the Advancement of Science, Jan. 31, 1991).

ls~ese w~e tie Profient issues, for example, at the National Academy of Sciences/Institute of Medicine, “Fo~ on Suppotig Biom~c~

Research: Near-Term Problems and Options for Actiou” Washington DC, June 27, 1990. Recent discussion has paradoxically focused on the broad

field of the life sciences where Federal funding increases have been most generous for the last 15 years. In its initial effort to document change and stress

in the Federal research system created by an abundance of research applications, OTAfound that an increasing proportion could not be funded by various

research agencies due to budget limitations, rather than to deficiencies of quality. U.S. Congress, Office of Technology Assessment “proposal Pressure

in the 1980s: An Indicator of Stress on the Federal Research System” staff paper of the Science, Educatio@ and TnmsportationP rogr~ April 1990.

lbsee~ti~te of Medicine, Fu~ingHeaZthSciencesResearch:A Strategy to RestoreBalance (Washington DC: National Academy Press, November

1990). For insight into the contentiousness that greeted the Institute of Medicine repofi see Peter G. Gosse~ “A Clash of Scientilc Titans: Key Groups

Battle Over Funds for Medical Projects,”The Washington Post, Dec. 18/25, 1990, Health sectio~ p. 6.

ITAs one politic~ scientist writes: “. . . because the Federal R&D system is comprised of so many independent actors, each of whom tend to view

science and engineering from a relatively narrow perspective, the Fedeml R&D system proceeds virtually without plarming and coordination. If it

moves , . . it does so . . . oozing slowly and incrementally in several directions at once, with constantly changing boundaries and shape. ’ Joseph G.

Morone, “Federal R&D Structure: The Need for Change,” The Bridge, vol. 19, fall 1989, p. 6.

18~e debt held by me F~er~ Govement rewnfly topped $3.1 ~fion, ~d payments on me &bt exceed~ $255 bmion in fiscal JWX 1990. ThtX(2

figures are expected to rise signitlca.ntly in 1991 and 1992, with the costs of the war in the Persian Gulf and the bailouts of the Nation’s financial system.

For an explanation of the Budget Enforcement Act of 1990, see Lawrence J. Haas, ‘‘New Rules of the Game,” National Journal, vol. 22, No. 46, Nov.

17, 1990, pp. 2793-2797.
Chapter 1—Summary and Issues for Congress ● 7

Table l-l—Tensions in the Federal Research System

Centralization of Federal research planning

Concentrated excellence

“Market” forces to determine the shape of

the system

Continuity in funding of senior investigators

Peer review-based allocation

Set-aside programs

Conservatism in funding allocation

Perception of a “total research budget”

Dollars for facilities or training

Large-scale, multiyear, capital-intensive,

high-cost, per-investigator initiatives

Training more researchers and creating

more competition for funds

Emulating mentors’ career paths

Relying on historic methods to build the

research work force

++

‘++

‘++

++

++

++

++

‘(-+

++

++

‘++

‘++

‘+-+

SOURCE: Office of Technology Assessment, 1991.

than can be funded, more researchers competing

than can be sustained, and more institutions

seeking to expand than the prime sponsor—the

Federal Government-can fund. The objective,

then, is to ensure that the best research continues

to be funded, that a full portfolio of research is

maintained, and that there is a sufficient research

work force of the highest caliber to do the job.

This report is designed to support Congress in

achieving these goals.

Trends in Federal Research Funding

The research system has shown itself to be

remarkably robust over at least the last 30 years, and

it has done well with the resources it has received. To

develop multiple perspectives on the system, Federal funding can be examined by agency, broad field,

and category of recipient.

Figure 1-3 displays Federal funding trends for the

six largest research agencies.l9 Since 1973, the

Department of Health and Human Services (HHS,

largely through the National Institutes of Health—

NIH) has supported more research than any other

Federal research agency. In fiscal year 1989, HHS

Pluralistic, decentralized agencies

Regional and institutional development (to

enlarge capacity)

Political intervention (targeted by goal,

agency, program, institution)

Provisions for young investigators

Other funding decision mechanisms (agency

manager discretion, congressional earmarking)

Mainstreaming criteria in addition to scientific

merit (e.g., race/ethnicity, gender, principal investigator age, geographic region)

Risk-taking

Reality of disaggregate funding decisions

Dollars for research projects

Individual investigator and small-team, 1-5

year projects

Training fewer researchers and easing competition for funds

Encouraging a diversity of career paths

Broadening the participation of traditionally

underrepresented groups

supplied nearly twice the research funds of the next

largest research agency, the Department of Defense

(DOD). HHS and DOD were followed by the

National Aeronautics and Space Administration

(NASA), the Department of Energy (DOE), the

National Science Foundation (NSF), and the Department of Agriculture (USDA) .20

Reflecting the division of research funds by

agency and broad field, a 20-year time series is

shown in figure 1-4. Life sciences continues its

steady growth relative to other broad fields. In fiscal

year 1990, life sciences dominated Federal funding

at $8.9 billion (in 1990 dollars). Engineering was

funded at slightly less than one-half the level of

support given to the life sciences ($4.4 billion), as

were the physical sciences (roughly $4 billion).

Environmental and mathematics/computer sciences

were funded at $2.1 and $0.7 billion respectively,

and the social sciences together gathered $0.6

billion.

Turning to research performance, universities and

colleges in the aggregate are the largest recipients of

federally funded research (basic and applied, see

lgcon=ess is most interest~ incompfig research expendities to other elementa of the Federal budget. Thus, a deflator that represents exPendi~es

on products and services that are often bought throughout the United States-a “constant dollar” in the most general sense--is often the most useful

for congressional policy analysis. Given the problems with research-specitlc deflators and the advantage of a general-GNP deflator to compare

expenditures across the economy, all constant dollar graphs and tables in this report were calculated with the GNP Implicit Price Deflator for 1982 dollars

(See ch. 2).

mote that the order of these agencies would be changed if research and development or basic research were used to rank them. The remaining

agencies, not included in the top six, together fund less than 5 percent of the research supported by the Federal Government.
8 ● Federally Funded Research: Decisions for a Decade

Figure 1-3—Federally Funded Research in the Major

Research Agencies: Fiscal Years 1960-90

(in billions of 1982 dollars)

6 I

I

5

4

3

2

1

o ~

1960 1965 1970 1975 1980 1985 1990

— H H S + DO D + NASA

+ DOE + NSF + USDA

KEY: HHS=U.S. Department of Health and Human Services; DOD=U.S.

Department of Defense; NASA= National Aeronautics and Space

Administration; DOE=U.S. Department of Energy; NSF= National

Science Foundation; USDA=U.S. Department of Agriculture.

NOTE: Research includes both basic and applied. Figures were converted

to constant 1982 dollars using the GNP Implicit Price Deflator. 1990

figures are estimates.

SOURCE: National Science Foundation, Federal Funds for Research and

Development, Detailed Historical Tables: Fiscal Years 1955-

7990 (Washington, DC: 1990), table A; and National Science

Foundation, Selected Data on Federal Funds for Research and

Development: Fiscal Years 1989, 7990 and 1991 (Washington,

DC: December 1990), tables 4 and 5.

figure 1-5). From 1969 to 1990, Federal funding for

research at universities and colleges grew from over

$4 billion to nearly $8 billion (in constant 1990

dollars). In 1990, performance of research by

industry (at over $3 billion) and the Federal laboratories (at over $6 billion) are funded at lower levels.

For basic research alone (not shown), universities

and colleges are even more clearly the dominant

research performer at over $5 billion when compared with Federal laboratories, the next largest

basic research performer, at slightly over $2 billion.

Figure 1-4-Federally Funded Research by Broad

Field: Fiscal Years 1960-90 (in billions of 1982 dollars)

7 ,

6

-

5

-

‘4 /

0

1 I 1 I I 1 1 I I I I I I 1 I I I I I I

1969

—

1974 1979

Life sciences

Environmental

sciences

Math/computer

1984 1989

Engineering

Social

sciences

Physical sciences

sciences -

NOTE: Research includes both basic and applied. Fields not included in

this figure collectively accounted for $l.1 billion (4.9 percent) of all

federally funded research in 1990. Figures were converted to

constant 1982 dollars using the GNP Implicit Price Deflator. 1990

figures are estimates.

SOURCE: National Science Foundation, Federal Funds for Research and

Development, Detailed Historical Tables: Fiscal Years 1955-

7990 (Washington, DC: 1990), table 25; and National Science

Foundation, Selected Data on Federal Funds for Research and

Development: Fiscal Years 1989, 1990 and 1991 (Washington,

DC: December 1990), table 1.

The distribution of Federal research and development (R&D) funds has long been a contentious

issue—both in Congress and in the scientific community. As shown in figure 1-6, if these funds are

aggregated by the State of the recipient institution or

laboratory, then five States received 53 percent of

the R&D funds in fiscal year 1990 (California,
Chapter l—Summary and Issues for Congress ● 9

Figure 1-5—Federally Funded Research by Performer:

Fiscal Years 1969-90 (in billions of 1982 dollars)

3’+4

I

4

3

2

1

A A A A A

o I Y + I

1969 1974 1979 1984 1989

— Federa l -t- Industry * Universities

Government and colleges

-9- Nonprofits * FFRDC s --+ Other

KEY: FFRDCs include all Federally Funded Research and Development

Centers that are not administered by the Federal Government. Other

includes Federal funds distributed to State and local governments

and foreign performers.

NOTE: Research includes both basic and applied. Figures were converted

to constant 1982 dollars using the GNP Implicit Price Deflator. 1990

figures are estimates.

SOURCE: National Science Foundation, Federal Funds for Research and

Development, Detailed Historical Tables: Fiscal Years 1955-

1990 (Washington, DC: 1990), table 17; and National Science

Foundation, Selected Data on Federal Funds for Research and

Development: Fiscal Years 1989, 1990 and 1991 (Washington,

DC: December 1990), table 1.

Maryland, Massachusetts, New York, and Virginia). 21 (Research institutions are also not randomly dispersed across America; rather, they are

concentrated on the two coasts and the upper

Figure 1-6—Federal R&D Obligations by State (1985)

and at Universities and Colleges (1989)

Cumulative distribution of Federal R&D

obligations by State: 1985

Billions of dollars

50r1I

I

40

30 -

20 -

10 -

0

94% 97% 98% 100%

! i I I I i I 1 I , , ) ) ) 1 I 1 I I 1 1 , , ,

0 5 10 15 20 25 30 35 40 45 50

Number of States

Cumulative distribution of Federal R&D

expenditures at universities and

colleges: 1989

Billions of dollars

10 [

8 . 97%

6

-

4

-

2 “

o J

o 20 40 60 80 100 120 140 160 180 200

Number of universities

SOURCE: National Science Foundation, Geographic Patterns: R&D in the

United States, Final Report, NSF 90-316 (Washington, DC:

1990), table B-5; and National Science Foundation, Se/acted

Data on Academic Sdenca/Engineering R&D Expenditures,

Fiscal Year 1989, NSF 90-321 (Washington, DC: October

1990), table B-35 and CASPAR database.

midwest.) At the other end of the distribution, 15

States together received less than 2 percent of the

funds. At the institutional level, 10 universities

receive 25 percent of the Federal research funding,

zl~ese fiWes me present~ for resewh and development because figures for research alone are not available. Based on 1984 daw tie Gene~

Accounting Office found various patterns of concentration among perforrners: res~chers in 10 States submitted over one-halfof the proposals to the

National Science Foundation and the National Institutes of Heal@ supplied almost 60 percent of the proposal reviewers, and won over 60 percent of

the awards. See U.S. General Accounting OffIce, University Funding: Patterns of Distribution of Federal Research Funds to Universities (Washingto~

DC: February 1987), p. 43. These figures, however, ignore other relevant factors in judging the “fair” distribution of Federal research funds, such as

the total population of a State and the number of scientists and engineers living in it. No matter how fair the competitive process, the outcomes may still

be seen as ‘‘unfair.” Also see William C. Boesman and Christine Matthews Rose, “Equity, Excellence, and the Distribution of Federal Research and

Development Funds, ” CRSReportfor Congress (Washingto~ DC: Congressional Research Semice, Apr. 25, 1989).
10 ● Federally Funded Research: Decisions for a Decade

and only 30 universities account for 50 percent.

Funding is concentrated in 100 research universities

in 38 States. This reflects their importance to the

Nation’s research enterprise.

These data on the distribution of resources bear a

critical message: research capabilities-institutions

and people—take time to grow. It is not simply a

matter of ‘‘they who have, get. ” The reputation,

talent, and infrastructure of research universities

attract researchers and graduate students.22 Some

universities become assets not only in the production

of fundamental knowledge, but also in bridging

science and technology to other goals such as State

and regional economic development.

Federally Funded Research in the 1990s

Snapshots of federally funded research, comparing fiscal years 1980 and 1991, are provided in table

1-2. Research is a small portion of the total Federal

budget. Although the distribution of research funds

by agency sponsor, category of performer, and

stratum of academic institution has hardly changed

during this period, the activity has never been in

greater demand.

However, questions such as “Does the Nation

need more science?’ and “How much research

should the Federal Government support?’ have no

ready answers. Measures of distress and conflicts

over resource allocation within the scientific community do not address whether the Nation needs

more science. Other problems in the Federal research system do not derive from, but are exacerbated by, such stress. They include sparse participation by women and ethnic minorities in science,

indications that other nations are better able to

capitalize on the results of U.S. research than

American industry, and management problems that

have plagued many Federal research agencies. Only

some of these problems can be addressed solely by

the Federal Government, and long-term solutions

may not be found in adjusting Federal funding

levels. Rather, they reflect problems in the organization and management of research and competing

values within the scientific community .23

“How much is enough” depends on the goals of

the research system (see box l-A). The system by

definition takes on new goals, each of which can be

evaluated. But in the aggregate how these goals are

assimilated-by add-on or substitution-is not easily predicted. The challenge is not to determine what

fraction of the Federal budget would constitute

appropriate funding for scientific research. Rather,

OTA finds that under almost any plausible

scenario for the level of research funding in the

1990s, there are issues of planning, management,

and progress toward national goals to address.

Because the reach of science is now great,

decisions about the funding of research are intertwined with many Federal activities. Congress and

the executive branch, which make these decisions in

our form of government, will continue to wrestle

with scientific and other national priorities, especially those that help prepare for tomorrow’s science—renewing human resources throughout the

educational pipeline and building regional and

institutional capacity. History cautions against the

expectation that the scientific community will set

priorities across fields and research areas. Congress

must instead weigh the arguments made within each

area against desired national outcomes.

In the 1990s, the Federal research system will face

many challenges. OTA has organized them here

under four interrelated issues: 1) setting priorities for

the support of research; 2) understanding research

expenditures; 3) adapting education and human

resources to meet the changing needs of the research

work force; and 4) refining data collection, analysis,

and interpretation to improve Federal decisionmaking. (For a summary of issues and possible congressional responses, see table 1-3.) To craft public

policies for guiding the system, each issue is

outlined in the following discussion.

~~ti~tiom,Me the faculty researchers employed by them, accumulate ‘‘tivmmge. ” Among the many factors that influence Federal research

funding, institutional reputation is part of a cycle of credibility that gives investigators an edge in competition for scarce resources-the very resources

that strengthen the institution as a productive research performer, which builds more credibility, and so on. See Robert K. MertorL “The Matthew Effect

in Science, II: Cumulative Advantage and the Symbolism of Intellectual Property,” Zsis, vol. 79, No. 299, 1988, pp. 606-623.

~See Joshw~derberg, ‘‘Does Scientillc Progress Come From Projects or People?’ Current Contents, vol. 29, Nov. 27,1989, pp. 4-12. In this report,

OTA concentrates on Federal, especially agency, perspectives on research. Performer (researcher and institutional) responses to changes in Federal

policies and programs were included to broaden understanding of the Federal role vis-a-vis acudem”c research, since universities are the primary site

for research performance and most &ta are collected on universities. However, national laboratories and industry play targeted roles and figure

prominently in research funding decisions.
Chapter 1—Summary and Issues for Congress ● 11

Table 1-2—Federally Funded Research in the 1980s and 1990s (in percent)

Fiscal year 1980 Fiscal year 1991 (est.)

R&D as percent of total Federal budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.0 4.7

Total research as percent of Federal R&D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38.9 36.3

Basic researches percent of Federal R&D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.7 19.1

Basic researches percent of total Federal budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.8 0.9

Agency Fiscal year 1980 Fiscal year 1991 (est.)

Percent of total (basic) research funds distributed, by agency HHS/NiH

DOD

NASA

DOE

NSF

USDA

Other

Performer

29/24(38/35)

20(12)

14(12)

11(11)

8(17)

6(6)

7(4)

Fiscal year 1980

34/29(40/37)

15(8)

16(15)

12(14)

9(15)

5(5)

lo(4)

Fiscal year 1991 (est.)

Percent of total (basic) research funds, by performer Universities 32 (50) 36 (47)

Federal 32 (25) 30 (23)

industry 18 (7) 15 (9)

Nonprofits 6 (6) 8 (9)

FFRDCsa

11 (11) 11 (12)

Ranking Fiscal year 1980 Fiscal year 1988

Percent distribution of Federal R&D funds at academic institutions Top 10 25 25

Top 20 40 39

Top 50 68 65

Top 100 84 85

KEY: DOD=U.S. Department of Defense; DOE=U.S. Department of Energy; FFRDC=Federally Funded Research and Development Center; USDA=U.S.

Department of Agriculture; NSF-National Science Foundation; HHS/NIH=U.S. Department of Health and Human Services/National Institutes of Health;

NASA= National Aeronautics and Space Administration

aThe Categoy of FFRDCSincludes all Federally Funded Research and Development Centers that are not administered by the Federal Government.

NOTE: R& Ddata are based on Federal obligations; calculations involving the total Federal budget are based on outlays. Columns may not sum to 100 percent

due to rounding.

SOURCES: Office of Technology Assessment, 1991, based on National Science Foundation data; U.S. General Accounting Office data; Economic Report

of the President (Washington, DC: U.S. Government Printing Office, 1991); and Budget of the Urrited States Government: Fisca/ Year 7992

(Washington, DC: U.S. Government Printing Office, 1991).

Issues and Options for Congress

ISSUE 1: Setting Priorities in the

Support of Research

Priorities are set throughout the Federal

Government at many levels. At the highest

level, research priorities are compared to

conscience and nonengineering needs. At the

next level, priorities are set across research

fields, such as biomedicine and mathematics.

Within fields, agency programs reflect research opportunities in subfields and relevance

to national needs. Finally, research projects are

compared, ranked, and awarded Federal funds.

Although priority setting occurs throughout

the Federal Government, it falls short in three

ways. First, criteria used in selecting various

areas of research and megaprojects are not

made explicit and vary widely from area to

area. This is particularly true, and particularly

a problem, at the highest levels of priority

setting, e.g., in the President’s budget and the

congressional decision process. Second, there

is currently no mechanism for evaluating the

total research portfolio of the Federal Government in terms of progress toward many national objectives, although recent efforts by the

Office of Science and Technology Policy have

lead to some cross-agency planning, budgeting, and evaluation. Third, the principal criteria

for selection, scientific merit and mission

relevance, are in practice coarse filters. Con-
12 ● Federally Funded Research: Decisions for a Decade

Box l-A—How Much is Enough?

‘‘How much is enough money for research?’ is a question that can only be asked if it is clear what scientific

and engineering research in the United States is attempting to accomplish: research for what?

1.

2.

3.

4.

Is the primary goal of the Federal research system to fund the projects of all deserving investigators of

natural and social phenomena?

If so, then there will always be a call for more money, because research opportunities will always outstrip

the capacity to pursue them.

Is it to educate the research work force, or the larger science and engineering work force, needed to supply

the U.S. economy with skilled labor?

If so, then support levels can be gauged by the need for more technically skilled workers. Preparing students

throughout the educational pipeline will assure an adequate supply and diversity of talent.

Is it to promote economic activity and build research capacity throughout the United States economy by

supplying new ideas for industry and other entrepreneurial interests?

If so, then the support should be targeted in line with our efforts to pursue applied research, development,

and technology transfer.

Is it all of the above and other goals besides?

If so, then some combination of these needs must be considered in allocating Federal support.

Indicators of stress and competition in the research system do not address the question of whether science needs

more funding to do more science. Rather, they speak to the organization and processes of science and to the

competitive foundation on which the system is built and that sustains its vigor.

Education, economic activity, and other national goals have long been confronted by Congress and the

executive branch, Although the relative importance of these needs varies over time with new developments and

crises, their absolute importance has not been set. Thus, allocating resources to these needs has always been a

tradeoff, within a limited budget, against other national goals and the programs that embody them.

Because of its intrinsic merit and importance to the Nation, research has consistently been awarded funding

increases. But these do not compare to what some claim would be an appropriate level of funding for research to

pursue a full agenda of opportunities. Deciding if the Nation is pursuing enough research opportunities or if the

Nation needs more science is thus a complicated question, which requires that other decisions about the nature of

the research system and its goals be settled first. Table 1A-1 reports the costs of some potential science initiatives

as estimated in the late 1980s.

Table 1A-1—Sample Requests From the Research Community for Increased Funding

Field or agency Report or intiative Additional funds requesteda

NSF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Initiative to double the NSF budget $2.1 billion

NASA space science . . . . . . . . . . . . . Towards a New Era in Space: Realigning U.S. Policies to New Over $1 billion

Realitiesb

Neuroscience . . . . . . . . . . . . . . . . . . . .1990s Decade of the Brain lnitiatlveC Over $1 billion

USDA research grants . . . . . . . . . . . . Investing in Research d $0.5 billion

Behavioral and social sciences . . . . The Behavioral and Social Sciences: Achievements and $0.26 billion

Opportunitiese

Mathematical sciences . . . . . . . . . . . Renewing U.S. Mathematicsf

$0.12 billion

All academic research . . . . . . . . . . . . Science: The End of the Frontier? g Over $10 billion

KEY: NSF-National Science Foundation; NASA-National Aeronautics and Space Administration; USDA-U.S. Department of Agriculture

aAdjustd to 199o ddlars using the 1982 GNP Implicit Prka lhflatOr.

bNattia[ -my&~~~atimal A-of ~ineering, Committee on Space Policy, “Towards a NW Era h SWCXI: Realigning U-S.

Policies to New Realitbs,” Space Pdky, vol. 5, August 1989, pp. 237-255.

c“ ~rzjjn ~~$t-kw~mtkfs COUrt SuppoR,” Tbe SAMtiSL vd. 4, No. 21, @t. n, 1 ~, P. 8.

dNat~al R~ar& COundI, hwasting in Rtmaar~ (Washington, DC: NatkmaltiChmY Pre$$, l~g).

eNat~al Rmmrc+I CkUncil, ~i? 8duw&8/ Sd *~~: Achievumenis and @portum’t/es (Washington, DC: National Academy Press,

f~~~~al Re~amh ~~1, Rem”w U.S. MafhwMtkg; A Pfan b fhe 7990s (Washington, w: Nati~al A-dew ~~t 1~).

9&ienca: ?“he Encfof the Frcmtk? a report from Leon M. Ledwman, Pr@dent-EJect to the Board of Directors of the Arm&an Assodationfor the

Advancement of science (Washington, DC: American Aasodatbn for the Advancement of Sdence, Jan. 31, 1991).

SOURCE: Offica of Technology Assessment, 1991.
Chapter 1—Summary and Issues for Congress ● 13

Table 1-3--Summary of Issues and Possible Congressional Responses

Issue Possible congressional responses

Setting priorities for research Hearings on crosscutting priorities and congressional designation of

a body of the Federal Government to evaluate priority setting.

Application of criteria to: a) promote education and human resources,

b) build regional and institutional capacity in merit-based research

decisionmaking, and c) balance Iittle science and megaproject

initiatives.

Oversight of agency research programs that focuses on strategies to

fulfill the above criteria, and on responses to priority setting.

Coping with changing Encouragement of greater rest-accountability by the research agenexpenditures for cies and research performers (especially for indirect costs,

research megaprojects, and other multiyear initiatives).

Allowance for the agencies to pursue direct cost containment

measures for specific items of research budgets and to evaluate

the effectiveness of each measure.

Adapting education and Programs that focus investment on the educational pipeline at the

human resources to meet K-1 2 and undergraduate levels.

future needs Attention to diversity in the human resource base for research,

especially to the contributions of underparticipating groups.

Incentives for adapting agency programs and proposal requirements

to a changing model of research (where teams are larger, more

specialized, and share research equipment and facilities).

Refining data collection and Funding to: a) augment within-agency data collection and analysis on

analysis to improve re- the Federal research system, and b) increase use of research

search decision making program evacuation at the research agencies.

Encouragement of data presentation and interpretation for use in

policymaking, e.g., employing indicators and other techniques

that measure outcomes and progress toward stated objectives.

SOURCE: Office of Technology Assessment, 1991.

cerns for developing human resources and addition, since megaproject costs affect the

building regional and institutional capacity ability of other disciplines to start new, large

must also be considered; these criteria projects, megaprojects are candidates for

strengthen future research capability. While crosscutting priority setting.

not every project or agency will factor these

criteria equally, the total Federal research Discussion

portfolio must address these concerns.

Priority setting can help to allocate Federal

Priority-setting mechanisms that cut across

research fields and agencies, and that make

selection criteria more transparent, must be

strengthened in both Congress and the executive branch. Congressional oversight must

evaluate the total Federal research portfolio

based on national objectives, research goals,

and agency missions. In the executive branch,

Congress should insist, at a minimum, on

iterative planning that results in: a) setting

priorities among research goals, and b) applying (after scientific merit and program relevance) other criteria to research decisionmaking that reflect planning for the future. In

resources both when they are plentiful, as they were

in the 1960s, and when they are scarce, as expected

through the early 1990s.24 Governance requires that

choices be made to increase the benefits and

decrease the risks to the Nation. Priority setting

occurs throughout the Federal Government at many

levels. At the highest level, research priorities are

compared to conscience and nonengineering needs.

At the next level, priorities are set across research

fields, such as biomedicine and mathematics. Within

fields, agency research programs reflect research

opportunities in subfields and relevance to national

needs. Finally, research projects are compared,

ranked, and awarded Federal funds.

ucon~e~~ ~Wo~zed tie ~potimof Ptiotiv se~g in the Natioti Science and Technology Policy, Organization and fiofities Act of 1976

(Public Law 94-282), May 11, 1976. For an elucidation of the dilemmas inherent in priority setting, especially comparisons between “social merit” and

“scientitlc merit, ’ see A.M. Weinberg, Rejections on Big Science (Cambridge, MA: MIT Press, 1966). Also see Stephen P. Strickland, Research and

the Health of Americans (Lexingto~ MA Lexington Books, 1978).
14 ● Federally Funded Research: Decisions for a Decade

Photo credit: Department of Energy

Underground nuclear test craters dot Yucca Flat at the Nevada Test Site (NTS). In addition to nuclear testing, researchers at NTS

explore other scientific phenomena such as geologic and seismic problems.

Toward More Explicit Priority Setting

There are three problems with priority setting as

it is currently practiced in the Federal Government.

First, criteria used in selecting various areas of

research and megaprojects are not made explicit, and

vary widely from area to area. This is particularly

true, and particularly a problem, at the highest levels

of priority setting+. g., in the President’s budget

and the congressional decision process. The best

developed priority-setting mechanisms are within

the research agencies and at the agency program

level.

Second, there is currently no mechanism for

evaluating the total research portfolio of the Federal

Government in terms of progress toward many

national objectives. Research priorities must be

considered across the Federal research system, and

in particular, across the Federal agencies. What the

Federal Government values more or less in research

can be inferred in part from the Federal budget, but

there is no “research budget.” Federal support is

distributed across many executive agencies and falls

under the jurisdiction of a number of congressional

committees and subcommittees (see table 1-4).

Therefore, once allocations have been made to

agencies (by the Office of Management and

Budget-OMB) or to appropriations subcommittees

(by full appropriations committees), decisions are

made independently within narrow components of

what is after-the-fact called the research budget.

This hampers the implementation of crosscutting

comparisons by Congress.

During the 1980s, OMB was a surrogate for a

crosscutting agent, with Congress adding its own

priorities through budget negotiations. 25 Recent

efforts by the Office of Science and Technology

Policy (OSTP) have lead to cross-agency planning,

budgeting, and evaluation in certain research and

education areas. President Bush has invested more

power in OSTP to participate with OMB in deliberations over research spending, especially in targeted

 an see Elizabeth Baldwin and Christopher T. Hill, ‘‘The Budget Process and Large-Scale Science Funding, ” Review, 

1988, 13-16.
Chapter l—Summary and Issues for Congress . 15

Table 1-4-Congressional Authorization Committees and Appropriations

Subcommittees With Significant Legislative Authority Over R&D

Jurisdictions of authorization committees: a Agency

House:

Agriculture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Armed Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Energy and Commerce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Interior and Insular Affairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Science, Space, and Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Public Works and Transportation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Merchant Marine and Fisheries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Veterans’ Affairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Foreign Affairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Senate:

Agriculture, Nutrition, and Forestry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Armed Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Commerce, Science, and Transportation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Energy and Natural Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Labor and Human Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Environment and Public Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Veterans’ Affairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Foreign Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

USDA

DOD, DOE

DOE, ADAMHA, NIH, CDC, DOT

DOI

NASA, NSF, DOE, EPA, NOAA, DOT NIST DOI

NOAA, DOT

USDA, NOAA, DOT

VA

A.I.D.

USDA

DOD, DOE

NSF, NASA, DOT NOAA, NIST

DOE, DOI

NIH, ADAMHA, CDC, NSF

EPA

VA

A.I.D.

Jurisdictions of appropriations committees: a Agency

Labor, Health and Human Services, Education

and Related Agencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

HUD and Independent Agencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Energy and Water Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Interior and Related Agencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Agriculture, Rural Development, and

Related Agenciesb . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Commerce, Justice, State, the Judiciary,

and Related Agencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Transportation and Related Agencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Foreign Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Defense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

NIH, ADAMHA, CDC

NASA, NSF, EPA, VA

DOE

DOE, USDA, DOI

USDA

NOAA, NIST

DOT

A.I.D.

DOD

KEY: ADAMHA-Alcohol, Drug Abuse, and Mental Health Administration; A.1.D.=Agency for International Development; CDC=Centers for Disease Control;

DOD=JJ.S. Department of Defense; DOEAJ.S. Department of Energy; DOI-U.S. Department of the Interior; DOT=J.J.S. Department of Transportation;

EPA=U.S. Environmental Protection Ageney; HUD-U.S. Department of Housing and Urban Development; NASA=National Aeronautics and Space

Administration; NIH-National Institutes of Health; NIST=National Institute of Standards and Technology; NOAA=National Oceanographic and

Atmospheric Administration; NSF= National Science Foundation; USDA=U.S. Department of Agriculture; VA=4J.S. Department of Veterans Affairs.

aThe jufi.sdictions of the authorizing ~mmittee5 are not exclusive. For this table, repeated authorization of a number of R&D-related Pm9rams was r~uired

to establish jurisdiction.

bhe corresponding subcommittees of the Senate and House Committees on Appropriations have the same name with one exception: the Senate

Subcommittee on Agriculture, Rural Development, and Related Agencies and the House Subcommittee on Rural Development, Agriculture, and Related

Ageneies.

SOURCES: Office of Technology Assessment, 1991; and Elizabeth Baldwin and Christopher T. Hill, “The Budget Process and Large-Scale Scienee Funding,”

CRS Review, February 1988, p. 15.

Presidential priority areas such as high-performance Third, although scientific merit and mission

computing, global environmental change, and math- relevance must always be the chief criteria used to

ematics and science education.26 Since the Adminis- judge a research area or agency program’s potential

tration is moving in the direction of more centralized worth, they cannot always be the sole criteria. In

and coordinated priority setting, it is all the more particular, the application of criteria that augment

important for Congress to consider priority-setting scientific merit-which represent today’s judgmechanisms as well. ments of quality-would help meet tomorrow’s

Zb’rhe cle~est public stat~ent of executive branch priorities is contained in ‘‘Enhancing Research nd EXpmding tie Human Frontier,” Budget of

the United States Government, Fiscal Year 1992 (WaShing-tOq DC: U.S. Government Printing OffIce, 1991), pp. 35-76. The ground rules for setting

crosscutting priorities through the OffIce of Science and Technology Policy, Federal Coordinating Council on Science, Engineering, and Technology

Committee mechanism are detailed in the Office of Management and Budget (OMB) “terms of reference” memoranda (provided to OIA project staff

during an interview with Robert E. Grady, Associate Director, Natural Resources, Energy, and Science, and other OMB staff, Feb. 7, 1991.
16 ● Federally Funded Research: Decisions for a Decade

Box 1-B-Criteria for Research Decisionmaking in Agency Programs

Within agency research programs, research proposals have traditionally been selected for support on the basis

of expert peer or program manager judgments of scientific merit and program relevance. Many Federal agencies

are now finding that the introduction of other explicit criteria is important for research decisionmaking.l

For example, the National Science Board (NSB) established the following criteria for the selection of research

projects by the National Science Foundation (NSF): 1) research performer competence, 2) intrinsic merit of the

research, and 3) utility or relevance of the research. In addition, NSB included 4) the”. . . effect of the research on

the infrastructure of science and engineering. This criterion relates to the potential of the proposed research to

contribute to better understanding or improvement of the quality, distribution, or effectiveness of the Nation’s

scientific and engineering research, education, and manpower base. ”2

Under this fourth criterion, NSF includes:

. . . questions relating to scientific, engineering, and education personnel, including participation of women,

minorities, and disabled individuals; the distribution of resources with respect to institutions and geographical area;

stimulation of high quality activities in important but underdeveloped fields; support of research initiation for

investigators without previous Federal research support as a Principal Investigator or Co-Principal Investigator and

interdisciplinary approaches to research or education in appropriate areas.3

In short, this criterion defines the bases for using other criteria in addition to scientific merit in mainstream

allocations of research funds, and within set-aside programs, Set-aside programs, at NSF and elsewhere, underscore

the continuing need for ‘‘sheltered competitions’ for researchers who do not fare well in mainstream disciplinary

programs.

4

As acknowledged by NSB, although scientific merit and program relevance must always be the primary criteria

used to judge a research program or project’s potential worth, they cannot always be the only criteria. For most of

today’s research programs, there are many more scientifically meritorious projects than can be funded. Proposal

1

0TA interviews, Spring -summer 1990.

2Quoted in National Science Foundation, Grant.sfor Research and Education in science .ndEngineering: An Application Gui&, NSF

977 (wUhiIl@Ol& ~: August 1990), pp. 8-9.

31bid., p. 9.

‘bTA finds thaq m some programs at the National Science Foundation (NSF), the fourth criterion is not strongly heeded relative to the

olher three criteria in the merit review process (OTA interviews, spring- summer, 1990). NSF fac~ the impossible task of being all things to all

people. The organic act entrusts it with the support of the Nation’s hasic research and science education. In the academic institutions that form

NSF’s core ciientele these activities are not pursued in the same way or with the same vigor. Every rrxearc h program at NSF now impacts on

human resources for science and e W@=-@ This should remain foremost in mind when weighing policies for research programs.

objectives of research investment. Broadly stated, research initiatives on their contribution to underthere are two such criteria: strengthening education

and human resources at all stages of study (e.g.,

increasing the diversity and versatility of participants); and building regional and institutional capacity (including economic development by matching

Federal research support with funds from State,

corporate, and nonprofit sources).

Education and human resources criteria would

weigh research initiatives on their ‘‘production’ of

new researchers or technically skilled students.

Contributions to human resources include increasing participation in the educational pipeline (through

degree completion), the research work force, and the

larger science and engineering work force. Regional

and institutional capacity criteria would weigh

participating regions and institutions. Regional and

institutional capacity are important concerns in all

Federal finding, and encouraging new institutional

participants and development of research centers

strengthens the future capacity and diversity of the

research system. Some agency programs already

incorporate these criteria in project selection (see

box l-B).

Can Congress look to the scientific community

for guidance on setting priorities? The short answer

is ‘‘no. ’ Congress wishes—perhaps now more than

ever—that the scientific community could offer

priorities at a macro level for Federal funding.

Science Advisor Bromley and former Science Advisor Press have stated criteria and categories of
Chapter l—Summary and Issues for Congress ● 17

review could thus bean iterative process. First, a pool of proposals could be identified based on scientific merit and

program relevance, and those with exceptional human resources and/or research infrastructure potential so

indicated. The program manager, with or without the advice of expert peers, can then pick a balanced subset from

the pool. Any of several subsets might be equally meritorious-+.his is where selection criteria and judgment enter

the process. The result is a program research portfolio that can be reshaped in succeeding years.

OTA suggests that two broad criteria could be applied to research project selection: strengthening education

and human resources, and building regional and institutional capacity. How might these two additional criteria be

rated in research proposals?

● Education and human resources criteria would weigh proposals on their future production of new

researchers or technically skilled students. Outcome measure s would relate to undergraduate education,

graduate “ training, and characteristics of new Ph.D. s--the number and quality of those entering graduate

study and the research work force, respectively.

Contributions to human resources include increasing participation in the educational pipeline (through

degree completion), the research work force, and the larger science and engineering work force. With the

changing character of the student population, tapping the diversity of traditionally underrepresented groups

in science and engineering (e.g., women and U.S. minorities) is vital for the long-term health of the research

work force.

● Regional and institutional capacity criteria would weigh proposals on their contribution to underparticipating regions and institutions. Outcome measures would include the enhanced research competitiveness

of funded institutions; State, local, and private participation in the support of the research infrastructure; and

an enlarged role in training and employment in targeted sectors, industries, and fields.

Regional and institutional capacity are important concerns in all Federal finding, reflecting the interests of

taxpayers. While the major research universities are exemplary in their production of research, untapped

resources could be developed in other types of educational institutions throughout the United States. 5

Funding research to achieve all of these objectives will remain a prerogative of Congress. But decisions that

add tomorrow criteria to May’s, especially in the review of project proposals at the research agencies, will expand

the capability of the Federal research system.

5

If the Federal Government wishes to augment the economic health of a particular region, supporting research m that area is one means

of achieving it “Spin-offs” fium mseamh centers hWC &tlditiOIIIdiy iIXlpfOVCd had CCmOlniCS by emwumg@ development of technical

industries and local research infrastructures. They also ofkm contribute to local educational efforts and dirwtly provide tedmkal jobs for

residents. See U.S. Congress, Office of Technology Assessment Higher Education for Science and Engim’ng, OTA-TM-SET-52

(wMhirlgtom DC: Us. Gov ernrncnt Printing OffkC, March 1989).

priority that they consider essential for science.27 addition, priority setting is often resisted by the

Each emphasizes the separation of large projects

requiring new infrastructure from ‘‘small science. ’

Press further distinguishes human resources from

national crises and extraordinary scientific breakthroughs, whereas Bromley places national needs

and international security concerns above all else. 28

While the Press and Bromley formulations appear

to provide frameworks for priority setting, they do

not address the problem that there are few mechanisms for, and no tradition of, ranking research

topics across fields and subfields of inquiry. In

recipients-of Federal funding because it orders the

importance of research investments, which means

that some programs do not get funded and some

groups within the scientific community complain of

lack of support. Consequently, Congress and the

executive branch have found that the scientific

community cannot make crosscutting priority decisions in science. In particular, the traditional mechanism of peer review is clearly not suited to making

judgments across scientific fields. Some research

communities do set priorities within specific research areas. However, the practice is not universal

27 See Frank Press, ‘‘The Dilemma of the Golden Age,” Congressional Record, May 26, 1988, pp. E1738-E1740; and D. Allan Bromley, “Keynote

Address,” in Sauer, op. cit., footnote 11, p. 11. (This was augmented by “U.S. Technology Policy, “ issued by the Executive Office of the President

Office of Science and Technology Policy, Sept. 26, 1990).

28One effect of these rank-orders is the seeming creation of separate accounts, i.e., that choices could be made within each category and then across

categories. Of course, such choices are being made by various participants in the research system simultaneously.
18 ● Federally Funded Research: Decisions for a Decade

or widespread.29 Therefore, while recognizing the

preferences of researchers, the Federal Government

must set priorities at two levels: among scientifically

meritorious research areas and megaprojects, and

among agency programs.

Megaprojects and the Science Base

Key to the consideration of allocating public

funds for science and engineering research is the

simultaneous support of little and big science. Little

science is the backbone of the scientific enterprise,

and a diversity of research programs abounds. Not

surprisingly, many investigators and their small

teams shudder at the thought of organizing Federal

science funding around a principle other than

scientific merit-an approach that, in fact, is advocated by no one. They fear that setting priorities

would change the criteria by which research funds

are awarded. In particular, they seem to hear calls for

priority setting as calls to direct all of research along

specified lines, not as a means to assure that balance

is achieved. For example, one goal would certainly

be the maintenance of funding for a diverse science

research base,30 while other goals would include

training for scientists and engineers, and supplying

state-of-the-art equipment.

The Federal” Government also seeks to achieve

goals at many levels. These goals are likely to differ

between programs that pursue specific objectives

and those that seek primarily to bolster the science

base. For instance, the allocation of additional

monies to NIH for AIDS (acquired immunodeficiency syndrome) research, beginning in the late

1980s and continuing today, has been a clear

designation of an objective as a priority research

area. In addition, to enhance the science base in

specific research areas, such as environmental science and high-temperature superconductivity, the

Bush Administration has increased funding in certain fields. These increases, however, seem to be

dwarfed by the cost of a very few, but visible,

megaprojects.

Megaprojects are large, “lumpy,” and uncertain

in outcomes and cost. Lumpy refers to the discrete

nature of a project. Unlike little science projects,

there can be almost no information yield from a

megaproject until some large-scale investment has

occurred. Presumably, a successful science megaproject provides knowledge that is important and

unattainable by any other means. Because of the

large expenditures and long timeframes, many

science megaprojects are supported by large political constituencies extending beyond the science

community .31 Future decisions may center on ranking science megaprojects, since not all of them may

be supportable without eroding funding of the

science base (see figure 1-7).

There are few rules for selecting and funding

science megaprojects; the process is largely ad hoc.

From a national perspective, megaprojects stand

alone in the Federal budget and cannot be subject to

priority setting within a single agency. Nor can

megaprojects be readily compared. For example, the

Superconducting Super Collider (SSC) and the

Human Genome Project (HGP) are not big science

in the same sense. One involves construction of one

large instrument, while the other is a collection of

smaller projects .32

An issue raised about some megaprojects is their

contribution to science. For instance, the Space

Station has little justification on scientific grounds,33

especially when compared with the SSC, the HGP,

or the Earth Observing System, which have explicit

scientific rationales. On purely scientific grounds,

the benefits that will derive from investing in one

project are often incommensurable with those that

would be derived from investing in some other.34

z~or ex~ples, Se the Natioti Resemh Council, Renewing U.S. Mathematics: A Plan for the 1990s (wildlingtob DC: Natimd Acdmy press,

1990); and the National Research Council, The Behavioral and Social Sciences: Achievements and Opportum”ties (Washington, DC: National Academy

Press, 1988).

-s priority has beenpreeminent since the Federal support of research began. See House Committee on Science and Technology, op. cit., footnote

9.

31PM Kun~, “pie in the Sky: Big Science Is Ready for Bhtoff,” Congressional Quarterly, Apr. 28, 1990, pp. 1254-1260.

32~e reswch Suppofied by tie HunGnome project-HGP-my ~ve some scien~lc benefits before tie p@~t is complete. mUS, HGP my

not be big science in the strict sense of the deftition outlined above. See Tom Shoop, “Biology’s Moon Shot, ” Government Executive, February 1991,

pp. 1011, 13, 16-17.

3sFor an e~ly s~tement of ~view, see U.S. Congms, Offim of Tec~olo~ ~sessmen~ Civi/ian space Stations ad the U.S. Future in Space,

OZ4-STI-241 (Sprirgfleld, VA: National Technical Information Service, November 1984).

34~s is elaborated ~ ~ey Av~ch “~wing tie Costs of Feder~ Re~~cb” OTA con~ctor repofi August 1990. Also 5= J.E. Sigel et d.,

“Allocating Resources Among AIDS Research Strategies,” Policy Sciences, vol. 23, No. 1, February 1990, pp. 1-23.
Chapter 1

Summary and Issues for Congress ● 19

Figure 1-7—Cost Scenarios for the Science Base and Select Megaprojects: Fiscal Years 1990-2005

Current cost estimates for megaprojects

3 percent growth for

science base

(megaproject funding added on)

Constant dollars

Doubled current cost

3 percent growth for

science base

(megaproject funding added on)

Constant dollars

I

1990 1995 2000

3 percent growth for total

research funding

(megaproject funding included)

Constant dollars

I

1990 1995

estimates for megaprojects

2000

3 percent growth for total

research funding

(megaproject funding included)

Constant dollars

2005

2005 1990 1995 2000 2005

 Science base Human genome SSC EOS Space station

KEY: Super Collider; Observing System.

NOTE: These figures are schematic representations of projected costs for science projects. the figures on the left, the base projected to grow

at an annual rate of 3 percent above inflation. In the figures on the right, research funding projected to grow 3 percent above inflation.

The cost estimates for the are based on data from “The Outlook in Congress for 7 Major Big Science Projects,” Chronicle of Higher

Education, Sept. 12, 1990, p. A28, and Genevieve J. Congressional Research Service, Science Policy Research Division,

 Status and Funding, February 1991,” unpublished document, Feb. 21, 1991. “Science

SOURCE: Office of Technology Assessment, 1991.
20 ● Federally Funded Research: Decisions for a Decade

Photo credit: National Aeronautics and Space Administration

The Hubble Space Telescope (HST) is shown, still in the grasp of Space Shuttle Discovery, with only one of two solar panels

extended. Earth is some 332 nautical-miles away. HST is an example of a large scientific

mission at NASA.

However, because the problem of selecting among

science megaprojects has most in common with the

selection of complex capital projects, timeliness

(why do it now rather than later?) and scientific and

social merit must all be considered, as well as

economic and labor benefits. At present, for example, the Space Station has considerable momentum

as an economic and social project.

Other measures to evaluate and, if necessary,

compare megaprojects include the number and

diversity of researchers that can be supported, the

scientific and technological value of information

likely to be derived (i.e., the impact of the megaproject on the research community), and the ultimate

utility of the new equipment and/or facility. For

instance, if one project will support only a few

researchers, while a second of similar cost and

scientific merit will support a larger number of

researchers, then perhaps the second should be

favored. One might also expect preference for

megaprojects that can be cost-shared internationally

over those that cannot be. (Issues of costs in

megaprojects are discussed below.)

Once the context for priority setting is examined,

choices take on another dimension. What do U.S.

society and the Federal Government expect for their

research investment? What does the scientific community promise to deliver? The answers differ

among participants and over time. As Robert White,

President of the National Academy of Engineering,

states: “It may be time that we think about whether

our concern for the support of the science and
Chapter l—Summary and Issues for Congress ● 21

technology enterprise has diverted us from attention

to how we can best serve national needs. “35

Congressional Priority Setting

Since progress begets more opportunities for

research than can be supported, setting research

priorities may be imperative for shaping a successful

Federal research portfolio in the 1990s.36 To improve priority setting at a macro level, Congress

should hold biennial hearings specifically on the

state of the research system, including cross-field

priorities in science and engineering, and the criteria

used for decisionmaking within the cognizant research agencies.

For “objective-oriented” science and engineering that may or may not cross agencies, such as

high-temperature superconductivity research, Congress should allocate resources based on plans to

attain specific goals. In programs that seek primarily

to fortify the science base, such as those sustained by

NSF, Congress could judge progress toward goals

that reflect the research capacity of the scientific

community. While objective-oriented programs will

contribute to these goals, the burden falls largely on

science base programs to meet the goal of maintaining the research community. Congressional oversight of the research agencies could include questions of how their total research activities and

specific programs, such as multiyear, capitalintensive megaprojects, contribute to expanding

education and human resources, as well as to

building regional and institutional capacity.

If Congress determines that more thorough and

informed priority setting is required, the executive

branch must disclose the criteria on which its

priorities were set. OSTP is a candidate for this task.

Building on the Federal Coordinating Council for

Science, Engineering and Technology (FCCSET)

mechanism, which presently considers only certain

cross-agency research topics, OSTP could also

initiate broader priority setting. In the executive

branch, Congress should insist, at a minimum, on

iterative planning that results in: a) making tradeoffs

among research goals; and b) applying (after scientific merit and program relevance) other criteria to

research decisionmaking that reflects planning for

the future. In addition, since megaproject costs affect

the ability of other disciplines to start new, large

projects, megaprojects are candidates for crosscutting priority setting.37

Structural improvements to current priority setting, especially those that facilitate the budget

process and research planning within and across the

agencies, would also make the tradeoffs more

explicit and less ad hoc, and the process more

transparent. At a minimum, agency crosscutting

budgetary analysis38 and a separate congressional

cycle of priority-setting hearings (e.g., biennially)

could reduce uncertainty and reveal the relationships

among new and continuing projects, the support of

new investigators by each agency, and the changing

cost and duration estimates that currently bedevil all

participants in the Federal research system.

Congress could also initiate specific changes in

the executive agencies that would increase their

ability to respond to changing priorities. They would

include measures that encourage: 1) flexibility, so

that programs can be more easily initiated, reoriented, or terminated; 2) risk-taking, so that a

balanced portfolio of mainstream and “long-shot”

research can be maintained; 3) strategic planning, so

that agency initiatives can be implemented as

long-term goals; 4) coordination, so that crosscutting priorities can be pursued simultaneously in

many agencies; and 5) experimentation with finding

allocation methods, so that new criteria can be

introduced into project selection and evaluated to

ascertain the value added to decisionmaking.

It is symbolic that across the Federal research

system, national policymakers, sponsors, and performers alike have acknowledged that the funding

process would benefit from careful consideration of

35Robert M. White, “Science, Engineering, and the Sorcerer’s Apprentice,” Presidential Address to the National Academy of Engineering,

Washington DC, Oct. 2, 1990, p. 12.

~Brooks fites, “To&y ~ny of tie s-e negative signals that existed in 1971 are again evident. Will science recover to experience anew era of

prosperity as it did beginning in the late seventies, or has the day of reckoning that so many predicted ftily arrived?’ Harvey Brooks, “Can Science

Survive in the Modern Age? A Revisit After Twenty Years,’ Nationul Forum, vol. 71, No. 4, fall 1990, p. 33.

qTFor ex~ple, see Mssa Rub@ “science Budget: Hill Must Make Hard Choices Among Big-Money Projects,” congressional Quarterly Wt?euy

Report, vol. 49, Feb. 9, 1991, p. 363.

sg~s was a pficip~~omen~tion propos~ in National Academy of Sciences, op. cit., footnote 6.
22 ● Federally Funded Research: Decisions for a Decade

research priorities, especially at the macro level.39

Whether their exhortations lead to clearer research

agendas (including the suspension or postponement

of some activities) remains to be seen, and whether

these investments are balanced, well-managed, and

yield the desired consequences is hard to judge in

real time. But surely the policy process is enriched

by drawing a map of the choices, the benefits, and

the costs to be incurred by the scientific community

and the Nation.

ISSUE 2: Understanding Research

Expenditures

Summary

Many in the scientific community claim that

the “costs of doing research” are rising

quickly, especially that the costs of equipment

and facilities outpace increases in Federal

research funding. The most reliable data are

available from research agencies, and can be

analyzed at two levels: 1) total Federal expenditures for research, and 2) individual components of research project budgets. OTA finds

that Federal expenditures for research have

risen faster than inflation, and more researchers are supported by the Federal Government

than ever before. Salaries and indirect costs

account for the largest and fastest growing

share of these expenditures. However, these

findings do not truly address the claims expressed above, because of the numerous and

sometimes inconsistent meanings of the costs

of doing research.

Most research activities become cheaper to

complete with time, as long as the scope of the

problem and the standards of measurement do

not change. However, advances in technology

and knowledge are ‘‘enabling’ they allow

deeper probing of more complex scientific

problems. Experiments are also carried out in

an environment driven by competition. While

competition is part of the dynamic of a healthy

research system, competition drives up demand for tiding, because success in the

research environment often correlates highly

with the financial resources of research groups.

Direct cost containment by the research

agencies may not be an appropriate Federal

role, although Congress might direct the agencies to pursue specific measures at their

discretion and to evaluate their effectiveness.

Instead, greater cost-accountability could be

encouraged by the executive branch and Congress. In particular, the Federal Government

should seek to eliminate the confusion around

allowable indirect costs, and develop better

estimates of future expenditures, especially for

megaprojects where costs often escalate rapidly.

Discussion

Many researchers state as an overriding problem

that the ‘‘costs of doing research” have risen much

faster than inflation in the Gross National Product

(GNP), and Federal expenditures for research have

not kept pace with these rising costs. Included in the

costs of research are salaries, benefits, equipment,

facilities, indirect costs, and other components of

research budgets. Equipment and facilities are typically named as most responsible for increased

costs .40

However, addressing these claims is difficult,

because it is hard to define what is meant by the costs

of doing research. Research activities become

cheaper to complete with time, as long as the scope

of the problem and the standards of measurement do

not change. But this is not the way progress is made.

Advances in technology and knowledge are “enabling’ they allow deeper probing of more complex

problems. This is an intrinsic challenge of research.

There is an extrinsic challenge as well. Experiments are carried out in an environment that is

driven by competition. Competition is part of the

dynamic of a healthy research system. One sign of a

3% addition t. those ~it~ ~reviou~ly, se Ro~fl M. Rosemeig, Resident, Association of Amefican Univemities, ‘‘Ad&ess to the President’s

Opening Sessiou The Gerontological Society of Amencq ” 43rd annual meeting, Boston, MA, Nov. 16, 1990; John H. Dutton and Lawson Crowe,

“Setting Priorities Among Scientific Initiatives, ” American Scientist, vol. 76, No. 6, November-December 1988, pp. 599-603; Albert H. Teich

‘‘Scientists and Public Officials Must Pursue Collaboration To Set Research Priorities,’ The Scientist, vol. 4, No. 3, Feb. 5, 1990, pp. 17; and Tina M.

Kaa.rsbergand Robert L. Park, “Scientists Must Face the Unpleasant Task of Setting Priorities, ”The Chronicle of Higher Education, vol. 37, No. 23,

Feb. 20, 1991, p. A52.

‘See Janice Long, “Bush’s Science Advisor Discusses Declining Value of R&D Dollars,” Chemical & Engineering News, vol. 68, No. 17, Apr. 23,

1990, pp. 16-17; Science: The End of the Frontier?, op. cit., footnote 14; and OTA interviews at the University of Michigan and Stanford University,

July-August 1990.
Chapter 1—Summary and Issues for Congress ● 23

healthy research system is that it can expand to

produce more research. “Needs” in the research

environment are thus open-ended.

Although competition exists in the research community, it does not necessarily drive down costs, as

would be expected in typical “markets.” In an

earlier era, the chief cost of research was the annual

salary of the principal investigator (PI). Today, the

PI is often the head of a team with many players and

access to the latest research technologies. In the face

of inherent uncertainty about the eventual outcomes

of research,41 sponsors must apply various criteria in

predicting the likelihood of eventual project success,

such as access to sophisticated equipment or the

availability of appropriately trained personnel.

These criteria are often associated with higher rather

than lower costs. Success, therefore, often comes to

those who spend the most (especially if research

teams are relatively evenly matched). In fact, competitive proposals are often the most expensive and

low bids can actually decrease a proposer’s chance

of winning a grant. Because additional personnel

and sophisticated equipment are seen by sponsors as

being instrumental in the conduct of research, costs

are ultimately limited by what sponsors are willing

to spend.

Products, or ‘ ‘outputs, ’ ‘ of scientific research

have also traditionally defied measurement.42 Consequently, the price of research measured in economic terms-the cost per-unit output-is extremely difficult to estimate. Analysis using crude

measures of scientific “productivity’ suggests that

the cost of producing a published paper or performing a given scientific measurement has decreased:

with less than double the investment per year since

1965, more than double the number of papers are

published today in academia, and more than double

the number of Ph.D. scientists are employed in the

academic sector. 43 By these measures, science h a s

grown more productive (and consequently the cost

per-unit output of research has decreased).44 However, there is no metric to compare a ‘‘unit’ of

today’s research with one in the past.

Thus, ‘Are the costs of research going up?’ is not

a useful question for policy purposes. Research

expenditures by the Federal Government are

awarded and accounted for on an annual basis. What

gets included in these expenditures can be modified

by adjusting the scale and pace of scientific research.

Especially for basic research, these factors are

variable, though the competition for personal and

institutional recognition pushes PIs toward larger

teams and more sophisticated instrumentation. In

mission-oriented science, the rate of research maybe

dictated by pressing concerns (e.g., curbing the

AIDS epidemic is desired as quickly as possible).

For policy purposes, research costs equal expenditures: if the Federal Government provides more

finds, ‘ ‘costs’ ‘ will go up accordingly. A more

useful policy question might be: ‘‘Is Federal spending on individual components of research project

budgets reasonable?’ The Federal Government will

tend to have a different point of view on this question

from the research performer. OTA has explored both

perspectives.

Incomplete and murky data on research expenditures complicate questions on the costs of research.

Analysis of Federal expenditures for the conduct of

research must factor what Federal agencies are

willing to spend for personnel, facilities, and instrumentation, while analysis of expenditures by research performers is confounded by the expenditure

accounting schemes that vary from research institu41c&e, for ~=ple, RickdNel~oq ‘{me A~ocation of R~~ch and Development ReSO~ceS: some Problems of Public Policy,” Economics Of

Research andDevelopment, Richard fibout (cd.) (Columbus, OH: Ohio State University Press, 1965), pp. 288-308. Nelson points out that”. . . research

and development has economic value because the information permits people to do things better, and sometimes to do things that they did not know

how to do before. . . but] there is no simple way to evaluate the benefits society can expect from the knowledge created by different kinds of R&D. . . .“

@p. 293-294). Also see Mansfield, op. cit., footnote 3.

az~blishedpapers ~dpatents have been used as proxies, but they cannot be standardized.s= Susan E. Cozzem, ‘ ‘Literature-Based Data in Research

Evaluation: A Mamger’s Guide to Bibliometrics,” final report to the National Science Foundation Sept. 18, 1989.

4 30n tie fomer, see H.D. ~te and K.W. McC@ “Bibliometics, “ Annual Review of Information Science and Technology, vol. 2A, 1989, pp.

119-186; and on the latter, National Science Board, op. cit., foomote 12, tables 5-17 and 5-30.

44However, even if one accwpts ~ese definitions of rese~ch OUtpU~ tie productivity of research relative tO otier economic activities mi@t still be

stagnant. Economist William Baumol explains that research due to the price of labor rather than increases in its productivity, has an “ . . . inherent

tendency to rise in cost and price, persistently and cumulatively, relative to the costs and prices of the economy’s other outputs. ” He warns that”. . . the

consequence may be an impediment to adequate fimding of R&D activity, that is, to a level of funding consistent with the requirements of economic

efficiency and the general economic welfare. ” See W.J. Baumol et al., Productivity and American Leadership: The Long View (Cambridge, MA: MIT

Press, 1989), ch. 6, quotes from pp. 116, 124.
24 ● Federally Funded Research: Decisions for a Decade

6

5

4

3

2

1-

>

t

0

4

Figure 1-8-Estimated Cost Components of U.S. Academic R&D Budgets: 1958-88 (In billions of 1988 dollars)

I

1958 1963 1968 1973 1978 1983 1988

+ Senio r + Graduate * Other direct

scientists students

+ Indirect + Equipment + Facilities

tion to research institution.45 In addition, much of

the current debate over rising expenditures takes

place within a context of agency budget constraints

and pressures felt by research performers.

The most reliable data on Federal research expenditures are available from research agencies, and can

be analyzed at two levels: 1) total Federal expenditures for research, and 2) individual components of

research project budgets. OTA finds that total

expenditures on individual components of grants

have risen over inflation, but not nearly at the rate for

total Federal expenditures for research (see figure

1-8). Instead, growth in the size of the research work

force supported by the Federal Government seems to

account for the largest increase in Federal research

expenditures. Also, the largest component increases

of research project budgets are for salaries and

indirect costs.

SOURCE: Government-University-Industry Research Roundtable, Science and Technology in the Academic Enterprise: Status,

Trends and/Issues, (Washington, DC: National Academy Press,

1989), figure 2-43.

NOTE: Constant dollars were calculated using the GNP Implicit Price

Deflator.

DEFINITION OF TERMS: Estimated personnel costs for senior scientists

and graduate students include salaries and fringe benefits, such as

insurance and retirement contributions. Other direct costs include

such budget items as materials and supplies, travel, subcontractors,

computer services, publications, consultants, and participant support

costs. Indirect costs include general administration, department

administration, building operation and maintenance, depredation and

use, sponsored-research projects administration, libraries, and student-services administration. Equipment costs include: 1) reported

expenditures of separately budgeted current funds for the purchase of

research equipment, and 2) estimated capital expenditures for fixed or

built-in research equipment. Facilities costs include estimated capital

expenditures for research facilities, including facilities constructed to

house scientific apparatus.

DATA: National science Foundation, Division of Policy Research and

Analysis. Database: CASPAR. Some of the data within this

database are estimates, incorporated where there are discontinuities within data series or gaps in data collection. Primary data

source: National science Foundation, Division of Science Resource

Studies, “Survey of Scientific and Engineering Expenditures at

Universities and Colleges”; National Institutes of Health; American

Association of University Professors; National Association of State

Universities and Land Grant Colleges.

Trends in Components of Total

Federal Research Expenditures

Analyzing Federal expenditures for specific line

items of research budgets reveals interesting trends

(again see figure 1-8). First, reimbursements for

indirect costs are the fastest growing portion of

Federal research expenditures. Indirect costs is a

term that stands for expenses that research institutions can claim from the Federal Government for

costs that cannot be directly attributed to a single

research project, i.e., they are distributed over many

investigators who share research infrastructure and

administrative support. Federal support for indirect

costs has increased since the 1960s, with the largest

increases in the late 1960s and the 1980s. In 1958,

indirect cost billings comprised 10 to 15 percent of

Federal academic R&D funding. By 1988, that share

had risen to roughly 25 percent.% In addition, some

agencies allow more than other agencies in indirect

costs. For example, in 1988, the indirect cost as a

percent of the total R&D expenditures allowed at

dsFor an afiempt to compwe expen~~es at tsvo public and two private universities associated with the performance Of National Science

Foundation-funded research see G.W. Baughma~ ‘Impact of Inflation on Research Expenditures of Selected Academic Disciplines 1%7-1983,” report

to the National Science Foundation and the Natioti Center for Educational Statistics, NSF/PLN 8017815, NOV. 8, 1985. Also see Daniel E. Koshland,

“TheUndersideof overhead,’ Science, vol. 249, May 11, 1990, p. 3; and ‘The OverheaclQuestio~’ letters in respometo Koshland’s editorial, Science,

vol. 249, Jtiy 6, 1990, pp. 10-13.

~Natio~ Science Foundation, The State of Acodew”c Science and Engineering (wtlShingtOnj w: 19W), P. 121.
Chapter l—Summary and Issues for Congress ● 25

Photo credit: Bob Kalmbach, University of Michigan

These scientists are in an ion beam laboratory at the

University of Michigan. Research often requires state-ofthe-art equipment.

NIH was 30 percent, whereas it was less than 24

percent for NSF (a proportion unchanged since the

mid- 1980s).47

Second, increasing numbers of investigators and

rising salaries (and the benefits that go with them)

have driven up the price of the personnel component

of direct costs. University personnel speak of the

increased competition for faculty with other sectors

of the economy, and note that faculty salaries have

been rising significantly over inflation during the

last decade. The average total compensation (salaries and benefits) for academic Ph.D,s in the natural

sciences and engineering increased from $59,000

(1988 dollars) in 1981 to more than $70,000 in 1988.

In the same period, the number of full-time equivalent scientists and engineers employed in academic

settings rose steadily from about 275,000 to almost

340,000.4

Third, Federal support for academic research

equipment alone increased from $0.5 billion in 1968

(1988 dollars) to $0.9 billion in 1988. Despite

pronounced increases and improvements in equipment stocks in the 1980s, 36 percent of department

heads still describe their equipment as inadequate (to

conduct state-of-the-art research). This is in part due

to the reduction in the obsolescence time of equipment and instrumentation use since the late 1970s.49

Finally, the Federal share of all capital expenditures for academic facilities (which include both

research and teaching facilities) has never topped

one-third. Now it is less than 10 percent.50 For

university research facilities alone, the Federal

Government provided an estimated 11 and 16

percent, respectively, of private and public university capital expenditures in 1988-89. The government also supports research facilities through depreciation, operation, and maintenance charges

accounted for in the indirect cost rate. In 1988, the

Federal Government supplied nearly $1 billion to

support university infrastructure. Almost 20 percent

was for facilities depreciation, while the rest was

recovered for operation and maintenance costs.51

Academic administrators claim that with growing

frequency, aging laboratories and classroom buildings falter and break down,52 and many claim that

facility reinvestment has not kept pace with growing

needs. However, the. picture is not clear. For

example, when asked by NSF, a majority of the

research administrators and deans at the top 50

 p. and Universities, With on 

 for Change DC: 1988).

 Science and Issues

Washington DC: National Academy Press, October 1989), pp. 2-34 and 2-47, based on National Science data.

 science/Engineering SRS 

Washington, DC: June 1988).

 50t060percent of the facilities funds the States, and issues. roughly

 comes from the Federal Government, while another one-third is from See Michael Davey, and Mortar: A Summa and

 of Proposals to Meet Research Facilities Needs on College Campuses DC: Congressional Research Service, 1987).

 of 

 “Enhancing Research and Expanding the Human Frontier,” op. cit., footnote 26, pp. 61-62. The document states that: academic

 must provide a certification that its research facilities adequate (to perform the research proposed) a condition of accepting research

 The . . . $12 billion of needed, but unfunded capital projects. . . .“ reported in the National Science Foundation surveys of universities . . has

 had an apparent effect on the ability of universities to accept Federal research funds. ’

 Money to Reduce Huge Maintenance Estimated to Exceed $70 New Federal Help 

 The Chronicle of Higher Education, vol. 37, No. 6, Oct. 10, pp. Al, A34.
26 ● Federally Funded Research: Decisions for a Decade

research universities replied that their facilities were

“good to excellent,” whereas a majority of the

research administrators and deans in the schools

below the top 50 estimated that their facilities were

‘‘fair to poor.’ ’53

The crux of the facilities problem is that research

and academic centers can always use new or

renovated buildings, but how much is enough? Even

though “need” may not be quantified in the

different sectors of the research enterprise, a demand

certainly exists. For example, when NSF solicited

proposals for a $20 million program in 1989 to

address facilities needs, it received over 400 proposals totaling $300 million in requests.54

Federal Policy Responses to Increased Demand

Many Federal agencies have experimented with

grant-reducing measures, such as the salary caps

required by Congress and temporarily imposed by

NSF and NIH, the ceilings on indirect costs currently

in place at USDA, the elimination of cost-blind

reviews of proposals in some research programs at

NIH, the limitation of funds supplied in new grants

to researchers with multiple Federal grants at the

National Institute of General Medical Sciences, and

the institution of freed-price grants in some NSF

programs.

55 Congress could pursue permanent

grant-reducing measures to slow or limit increases in

research expenditures on individual research grants.

However, it may not be an appropriate Federal role

to dictate specific allowable costs in research

projects. In general, allowing market forces to

determine costs has been a tradition in Federal

policy.

Instead, greater cost-accountability could be encouraged. One benefit of cost-accountability could

be incentives for performers to spend less than what

was targeted in project budgets, and greater flexibility in expenditures for performers (e.g., researchers

could be encouraged to use the money saved one

year in the next year, a so-called no-cost extension).

Within such cost-accountability measures, Congress

might also direct the agencies to experiment with

cost-containment schemes and to evaluate their

effectiveness.

Greater cost-acountability is especially important

in the calculation of indirect cost rates. At present,

the guidelines for calculating costs are detailed in

conjunction with OMB Circular A-21 and have been

in force since 1979. Every major research university

has an indirect rate established for the current fiscal

year for recovery of costs associated with sponsored

research. These rates have evolved over many years

as a result of direct interaction and negotiation with

the cognizant Federal agency. There is a wide range

of indirect costs rates among universities, with most

noticeable differences between public and private

institutions (rates tend to be higher at private

institutions). Rates vary because of: 1) significant

differences in facilities-related expenditures, 2) underrecovery by some universities, 3) imposition of

limits by some government agencies in the negotiation process, and 4) diversity in assigning component expenditures as direct or indirect.56

However, confusion around what is contained in

the indirect cost rate is getting worse, not better. This

reflects, in part, the difficulty of separating expenditures along lines of research, instruction, and other

functions.57 Recent investigations by the Office of

Naval Research and the House Committee on

Energy and Commerce have also uncovered significant variation in the accounting of indirect costs by

the cognizant Federal agencies and research universities.58 These differences should be sorted out, and

more explicit and understandable guidelines devised.

S3Natio~ Science FoMdation, Scientific and Engineering Research Facilities at Universities and Colleges: 1988, NSF 88-320 (wm~to~ DC:

September 1988), p. 26.

‘See Jeffrey Mervis, “Institutions Respond in Large Numbers to Tiny Facilities Program at NIH, NSF,” The Scientist, vol. 4, No. 8, Apr. 16,1990,

p. 2.

ssFor a dismssion of VMOUS optious, see Barbara J. wlito~ “NIH Readies Plan for Cost Containrnen\” ~cience, vol. 250, NOV. 30, 19$W Pp.

1198-1 199; and Colleen Cordes, “Universities Fear That U.S. Will Limit Payments for Overhead Costs Lncurred by Researchers,” The Chronicle of

Higher Education, vol. 37, No. 3, Nov. 21, 1990, pp. A19, A21.

subassociation of American Universities, op. Cit., fOOhlOte 47.

5TEl=nor C. ~o-~d ~o~d L. ~dema, Natio~ Scienm Fo~datio~ Dk@orate for Scientilc, Technological, ~d hltelllatiOMl Aff&,

“Indirect Costs of Federally Funded Academic Researc~” unpublished paper, Aug. 3, 1984, p. 1.

58 See Marcia Barinag% “Statiord Saik Into a Stow” Science, vol. 250, Dec. 21, 1990, p. 1651; “Government bquiry,” Stanford Observer,

November-December 1990, pp. 1, 13; Colleen Cordes, ‘‘Conceding ‘Shortcomings,’ Stanford To Forgo $500,000 in Overhead on U.S. Contracts,” The

Chronicle of Higher Education, Jan. 30, 1991, vol. 37, No. 20, pp. A19, A22; and Colleen Cordes, ‘‘Stanford U. Embroiled in Angry Controversy on

Overhead Charges, ’ The Chronica2 of Higher Education, Feb. 6, 1991, vol. 37, No. 21, pp. Al, A20A21.
Chapter 7—Summary and issues for Congress ● 27

It is also important to stress accuracy in developing estimates of costs for megaprojects. When the

Federal Government ‘ ‘buys’ ‘ a megaproject, the

initial investment seems to represent a point of no

return. Once the go, no-go decision has been made

at the national level, the commitment is expected to

be honored. However, criteria for consideration in

the funding of a science megaproject could conceivably include: startup and maintenance costs, cost of

unanticipated delay, cost of users’ experiments, and

likely changes in the overall cost of the project from

initial estimate to completion. Some estimates for

science megaprojects double before the construction

is even begun, and costs of operating a big science

facility once it is completed are sometimes not

considered. 59

Megaprojects will always be selected through a

political process because of their scale, lumpiness,

and incommensurability. Since their costs, especially in following years, affect other disciplines’

abilities to start new, large projects, megaprojects

could well be considered as candidates for crosscutting, priority-setting analysis before the practical

point of no return. As the National Academy of

Sciences’ report on budget priorities reminds: "

. . . it is necessary to specify the institutions,

individuals, and organizations that will be served;

[and] the costs . . . of the program.”60 The cost of

investment for the Federal Government is an important criterion to apply to all scientific research,

including megaprojects.

Performer Expectations

Not all problems in research costs can be addressed by the Federal Government. Many researchers point to higher expectations, which require more

spending, and competition in the university environment. In the academic environment, researchers are

asked today to publish more papers, shepherd more

graduate students, and bring in more Federal funding

than their predecessors.6l If they do not meet these

expectations, some report a sense of failure.62 This

is true even if they have succeeded, but not by as

much or as quickly as they had hoped.

To boost research productivity and to compete

with other research teams, faculty attempt to leverage their time with the help of postdoctoral fellows,

nontenure track researchers, and graduate students

who are paid lesser salaries. Due to the shortage of

faculty positions for the numbers of graduate students produced, young Ph.D.s have been willing to

take these positions in order to remain active

researchers. This availability of ‘‘cheap labor” is

seen by many senior researchers and their institutions as the only way they can make ends meet in

competing for grants.63 This is a trend toward an

‘‘industrial model, ’ where project teams are larger

and responsibilities are more distinct within the

group.

@ While the expenditures charged to an

individual grant may be less (since more grants may

be required to support the diverse work of the group),

the overall cost of supporting a PI and the larger

group are greater.65

Some experiments have been attempted on U.S.

campuses to temper the drive for more research

publications (as a measure of productivity). For

example, at Harvard Medical School, faculty are

allowed to list only five publications for consideration in tenure reviews, with similar numbers set for

5~or ~xmple, ~ee K*@ ~p. ~it, fw~ote 31; and David p. =toq“me SSC T~es on a Life of Its Ow” Science, VO1. 249, Aug. 17, 1990,

pp. 371-372.

~ational Academy of Sciences, op. cit., footnote 6, p. 11.

61~~is e~wci~ly~e fien~prene~~mwmch ~eas such ~ biot~~olo~. see&nryE&kowi@ “13n@epreneuri~ Scientists and Entrepreneurial

Universities in American Academic Science,” Minerva, vol. 21, summer-autumn 1983, pp. 198-233.

62Science: The End of the Frontier? Op. Cit., fOOtnOte 14.

63~bor ~onofi~t Ah Fech ter, Executive D~tor, ~lce of sci~~lc ~d Enginee@ personnel, Natioti Research COUJICil, WriteS:

,,

. . . personnel costs constitute roughly 45 percent of total costs and . . . this percentage has remained reasonably stable over time. Given that salaries

of faculty (i.e., principal investigators) have been rising during the 1980s, this suggests that the staffing pattern of research projects has been changing,

with the input of PIs decreasing relative to . . . other, less expensive resourees. There is some evidence to support this hypothesis in the report of GUIRR

IGovernment-University-Industry Research Roundtable] . . . [that] finds in academia an increasing ratio of nonfaculty to faculty,” personal

communicatio~ Nov. 15, 1990. See Government-University-Indushy Research Roundtable, op. cit., footnote 48.

~E1sewhem t.his IMS been Cwed the ‘‘industrialization’ of SCience, Or ‘‘~. . a new collectivized form in which characteristics of both the academic

and industrialized modes are intermingled. ’ See John ZirnaU An Introduction to Science Studies (Cambridge, England: Cambridge University Press,

1984), p. 132 (elaborated below).

GSNoted at OTA wor~hop on the Costs of Research and Federal DeciSiO~“ g, July 19, 1990.

292-863 0 - 91 - 2
28 ● Federally Funded Research: Decisions for a Decade

66 Thus, the quality and ‘imporother promotions.

tance of the candidate’s selected set of papers is

stressed, though measuring these characteristics

remains controversial.67 However, strong incentives

militate against reducing research volume. Most

overhead is brought into the university by a small

number of research professors. (At Stanford, 5

percent of the faculty bring in over one-half of the

indirect cost dollars.) Any measure that would

reduce grant awards and publications produced by

these investigators would deprive the university of

revenues. In fact, many universities in tight financial

straits try to maximize the level of research volume.68

The Federal Government must seek to understand

better the trends in expenditures in the research

environment-specially variations across institutional settings-and craft government policies to

allocate resources effectively. Reliable analyses of

research expenditures at all of the Federal agencies

are not available. Future studies of expenditures

should look not only at the economic forces that

increase (and decrease) research expenditures, but

also at the sociology of research organizations,

including the demography of research teams and

institutional policies for sponsored projects.69

Federal agencies clearly must understand increasing demands to fund research, as research universities and laboratories are an invaluable resource for

the United States. Devising mechanisms for coping

with research expenditures is one of the central

challenges to the Federal system for funding research in the 1990s.

ISSUE 3: Adapting Education and Human

Resources To Meet Changing

Needs

Summary

Three issues are central to education and

human resources for the research work force:

1. Recent projections of shortages of Ph.D.

researchers in the mid-1990s have spurred

urgent calls to augment Ph.D. production in the

United States. OTA believes that the likelihood of these projections being realized is

overstated, and that these projections alone are

poor grounds on which to base public policy.

For instance, they assume continued growth in

demand in both academic and industrial sectors, independent of the level of Federal

funding. In both this and previous OTA work,

however, OTA has indicated the value to the

Nation—regardless of employment opportunities in the research sector-of expanding the

number and diversity of students in the educational pipeline (IS-12 and undergraduate) for

science and engineering, preparing graduate

students for career paths in or outside of

research, and, if necessary, providing retraining grants for researchers to move more easily

between research fields.

2. Total participation in science and engineering can be increased if the opportunities

and motivation of presently underparticipating

groups (e.g., women, minorities, and researchers in some geographic locations) are addressed. Federal legislation has historically

played an important role in recruiting and

retaining these groups. Also, “set-aside” programs (which offer competitive research grants

to targeted groups) and mainstream disciplinary programs are tools that can enlarge,

sustain, and manage the diversity of people and

institutions in the research system.

3. Research in many fields of science and

engineering is moving toward a larger, more

‘‘industrial’ model, with specialized responsibilities and the sharing of infrastructure. In

response, the Federal Government may wish to

acknowledge changes in the composition of

research groups and to enhance the opportunities and rewards for postdoctorates, nontenure

track researchers, and others.

fi’rhe Nation~ Science Fo~&tion alSO now limits the number of publications it will consider, as evidence of an applicant’s track record, inreviewing

grant proposals. See David P. Hamilto~ “Publishing By—and For?-the Numbers,” Science, vol. 250, Dec. 7, 1990, pp. 1331-1332.

bTSeeN,L. Geller et d., ‘‘Lifetime Citation Rates to Compare Scientists’ Work’ Social Science Research, vol. 7, No. 4, 1978, pp. 345-365; and A.L.

Porter et al., “Citations and Scientific Progress: Comparing Bibliometric Measures With Scientist Judgments,” Scientometrics, vol. 13, 1988, pp.

103-124.

@OTA interviews at Stanford University, Aug. 2-3, 1990.

69see Susan E. Comem et ~. (eds.), The Research system in Transition,fioceedings of a NAT() Adv~~dStidy kti~te, ~ Ciocco, Italy, OCt. 1-13,

1989 (Dordrecht, Holland: Kluwer, 1990).
Chapter I—Summary and Issues for Congress ● 29

Discussion

The graduate science and engineering (s/e) education system in the United States, especially at the

doctoral level, is the envy of the world. Foreign

nationals continue to seek graduate degrees from

U.S. institutions at an ever-growing rate.70 From

1977 to 1988, the number of Ph.D.s awarded in s/e

by U.S. universities increased by nearly 50 percent71

(for a breakdown by field and decade, see figure

1-9). This exemplary production of Ph.D.s continues

a noble tradition abetted by Federal research and

education legislation.

With passage of the National Defense Education

Act of 1958 (Public Law 85-864) in the wake of the

Sputnik launch, the Federal Government became a

pivotal supporter of pre- and postdoctoral science,

engineering, and indeed, non-s/e students.72 Additional programs were soon established by NSF,

NASA, NIH, and other Federal agencies. This period

of growth in Federal programs offering fellowships

(portable grants awarded directly to students for

graduate study) and traineeships (grants awarded to

institutions to build training capacity) was followed

by decreases in the 1970s.73 In s/e, this decline was

offset by the rise in the number of research

assistantships (RAs) for students awarded on Federal research grants to their mentors.

During the 1980s, RAs became the principal

mechanism of graduate s/e student support, increasing at 5 percent per annum since 1980, except in

agricultural sciences where RAs have actually declined. (A comparison of the types of graduate

student Federal support, 1969 and 1988, is presented

in figure 1-10.)74 This trend is consistent with the

growing “research intensiveness” of the Nation’s

universities: more faculty report research as their

primary or secondary work activity, an estimated

total in 1987 of 155,000 in academic settings.75

Thus, the Federal Government has historically

played both a direct and indirect role in the

production and employment of s/e Ph.D.s. Both as

the primary supporter of graduate student stipends

and tuition, and as a patron, mainly through research

grants, the Federal Government has effectively

intervened in the doctorate labor market and helped

shape the research work force.

Supplying the Research Work Force

The U.S. graduate research and education system

trains new researchers and skilled personnel for all

sectors of the Nation’s work force (and arguably for

some countries abroad). Since 1980, NSF estimates

that the total s/e work force (all degrees) has grown

at 7.8 percent per year, which is four times the annual

rate of growth in total employment. Scientists and

engineers represented 2.4 percent of the U.S. work

force in 1976 and 4.1 percent in 1988.76

While new s/e Ph.D.s have traditionally been

prepared for faculty positions in academia-almost

80 percent were employed in this sector in 198777—

in broad fields such as engineering and disciplines

such as computer science the demand for technical

labor outside of academia is great. Other fields, like

chemistry, benefit from having a large set of

potential academic and industrial employment opportunities. This diversity makes any labor market

70SW Natio~ Scien= Board, op. cit., footnote 12, p. 55; and National Research Council, Foreign andForeign-Born Engineers in the United States

(Washington, DC: National Academy Press, 1988). Although OTA uses the shorthand “scientists and engineers,” it recognizes the range of fields

represented by the term. They are encompassed by the degree-granting categories in the National Science Foundation’s Science Resources Studies

reports: engineering, physical sciences, environmental sciences, mathematical sciences, computer/information sciences, life (biologi~agric~~ral)

sciences, psychology, and social sciences.

TINational Science Fo~dation, Science andl?ngineering Doctorates: 1960-89, NSF 90-320 (WrA@to~ ~: 1990), table 1.

TzFor de~s, s= U.S. ConDess, ()&lce of T~~olo~ Ass~smen~ De~grophiC Trends and the &ien@C and Engineering Work Force,

OTA-TM-SET-35 (Washington DC: U.S. Government Printing Oft3ce, December 1985), pp. 44+49.

TsAssociation of Amefica.n Universities, The Ph.D. Shortage: The Federal Role (WasMgtOu DC: Jan. 11, l~o), PP. 15-16.

74~ ~lott~ by gender, this fiwe would look q~te diffe~nr. TraditiO~Y, women have nor r~ived as many fe~owships and baine~hips as men

or foreign students on temporary visas, are more dependent on personal or family resources during graduate study, and suffer higher attrition before

completing the Ph.D. See U.S. Co~ss, OffIce of Technology Assessment, Educating Scientists and Engineers: Grade School to Grad School,

OIX-SET-377 (Washingto~ DC: U.S. Government Printing (X13ce, June 1988), pp. 79-80; and National Science Foundation, Women and Minorities

in Science and Engineenng, NSF 90-301 (Washingto~ DC: January 1990), pp. 23-24.

T5Natio~ Science Bored, op. ~it., fw~ote 12, ~p. 46,57. These 155,~ rq~sent~ 37 ~rc~r of the doctorate scientists and engineers emplOyd

in the United States in 1987.

Ts~id., p. 67. Among Ph.D.s, the ratio of employed scientists to en@eerS iS 5 to 1.

77~id., app. table 5-19.
30 ● Federally Funded Research: Decisions for a Decade

Figure 1-9—Percentage Distribution of Doctorates by

Science and Engineering Field: 1960-89 (by decade)

Life sciences

Physical sciences

Engineering

Social sciences

Psychology

Mathematical

sciences a

Environmental

sciences

Percent of all doctorates earned

O 2 4 6 8 10 12 14 16 18

I I I I I

a Degrees in computer science were not awarded until the 

before then, computer science with mathematical 

SOURCE: National Science Foundation, Doctorates: 1960-89, NSF 90-320 (Washington, DC: 1990), detailed

statistical tables, table 1.

fluid and its forecasting difficult, but the major

components can be analyzed.78

Based on changing demographics and historical

trends in baccalaureate degrees, some studies have

projected that the scientific community will face a

severe shortage in its Ph.D. research work force

during the 1990s.79 However, there are pitfalls in the

methodologies employed in these projections of

Ph.D. employment demand.80 Predicting the deFigure I-l O-Federal Support of Science and

Engineering Graduate Students, 1969 and 1988 — (by type of support)

In percent

100

80

60

40

20

0

Fellowship

Traineeships

— Research

assistantships

100

80

60

40

20

0

1969 1988

Total

students

supported: 51,620 54,852

NOTE: Fellowships and traineeships were not reported separately in 1969.

SOURCE: National Science Board, Science and Engineering /ndicators7989, NSB 89-1 (Washington, DC: U.S. Government Printing

Office, 1989), appendix table 2-18; and National Science

Foundation, Graduate Student Support and Manpower Resources in Graduate Science Education, Fall 1969, NSF 70-40

(Washington DC: 1970), table C-1 la.

mand for academic researchers must also account for

enrollment and immigration trends, anticipated career shifts and retirements, and the intentions of new

entrants, as well as shifting Federal priorities and

available research funding. All of these are subject

to change, and may vary by institution, field, and

region of the country .81 In addition, OTA questions

 Eileen L. ‘‘Meeting the Scientific and Technical Requirements of the AmericanEconomy,” Science 

Policy, vol. 15, No. 5, October 1988, pp. 335-342; and National Research Council, The on Quality in 

(Washington, DC: National Academy Press, 1988).

 C. for Scientists and Engineers: A National Crisis in the Making, ” 246, 

 425-432.

 be of may be problematic. For

a discussion, see Howard P. ‘‘Supply, Human Capital, and the Average Quality Level of the Science and Engineering Labor Force, ”

Economics of Education Review, vol. 7, No. 4, 1988, pp. 405-421.

 see Ted ‘‘Studies of Academic Markets and Careers: Review, ‘‘ Academic Labor Careers, 

 and Ted Youn PA: The Press, 1988), pp. 8-27.
Chapter l—Summary and Issues for Congress ● 31

the ability of statistical analyses to predict future

demand for s/e Ph.D.s, especially as responses to

market signals and other societal influences are

known to adjust both interest and opportunities.

Even without the prospect of a slackening economy

in the 1990s, such projections would be unreliable.

Given the track record of these forecasting tools,

they are poor grounds alone on which to base public

policy. 82

Noting the uncertainty of projections, OTA finds

that concentration on the preparedness of the pipeline to produce Ph.D.s (i.e., increasing the number of

undergraduates earning baccalaureates in s/e) by

introducing flexibility into the system is the most

robust policy. If shortages begin to occur in a

particular field, not only should graduate students be

encouraged to complete their degrees (i.e., reducing

attrition), but prepared undergraduates should be

induced, through various proven Federal support

mechanisms, to pursue a Ph.D.83 Those scientists

who would have otherwise left the field might stay

longer, those who had already left might return, and

graduate students in nearby fields could migrate to

the field experiencing a shortage. If shortages do not

materialize, then the Nation’s work force would be

enhanced by the availability of additional highly

skilled workers.

OTA believes there are initiatives that maintain

the readiness of the educational pipeline to respond

to changing demands for researchers and that

enhance the diversity of career opportunities—

sectors and roles—for graduates with s/e Ph.D.s.84

Congress could urge NSF and the other research

agencies to intensify their efforts to maintain a

robust educational pipeline for scientific researchers

(and to let the labor market adjust Ph.D. employment). Funding could be provided for undergraduate

recruitment and retention programs, for grants to

induce dedicated faculty to teach undergraduates,

and for the provision of faculty retraining grants.88

Expanding Diversity and Research Capacity

Trends in the award of s/e degrees attest to 20

years of steady growth in human resources (see

figure 1-11). These data are a sustained record of

scientific education at the Ph.D. level. However, the

benefits of this education do not accrue equally to all

groups, and therefore to the Nation. Women and

U.S. racial and ethnic minorities, despite gains in

Ph.D. awards through the 1970s and 1980s, lag the

participation of white males. Relative to their

numbers in both the general and the undergraduate

populations, women and minorities (and the physically disabled) are underparticipating in the research

work force.86 Meanwhile, foreign nationals on

temporary visas are a growing proportion of s/e

Ph.D. recipients (and about one-half are estimated to

remain in the United States) .87

g~TAreached this conclusion afterex arnining the performance of various models of academic and industrirdlabor markets. See O.tXceof Technology

Assessment, op. cit., footnote 72, especially chs. 3 and 4. Recent independent confirmation of this conclusion appears in Alan Fechter, “Engineering

Shortages and Shortfalls: Myths and Realities, ” The Bridge, fall 1990, vol. 20, pp. 16-20,

Bssee offIce of Technology Assessment, op. Cit., fOOtnOte 74.

~See RVO repo~: U.S. Congress, office of Technology Assessment Elementa~ and Seconda~ Educan”on for Science and Enginee@,

O’E4-TM-SET-41 (Washington, DC: U.S. Government Printing Oft3ce, December 1988); and Higher Education for Science and Engineering,

O’E4-TM-SET-52 (Washingto~ DC: U.S. Government Printing OffIce, March 1989).

85~e Nation~ Science Fomdation, as pres~~d in ~efi enab~g l@s~tioq is equ~ly ~sponsible for science ~ucation ~d the SUppC)rt Of the

Nation’s basic research. It has gradually expanded its programs, long focused on the graduate end of the pipeline, to address issues in undergraduate

and K-12 education. For example, see National Science Foundatio@ Research on Key Issues in Science and Engineen”ng Education: Targeted Program

Solicitation, NSF 90-149 (Washington, DC: 1990). Perhaps faculty retmining programs, both to highlight changes in educational strategies and

developments in research should be considered. Retraining has been acknowledged as important for maintaining the engineering work force, and

retraining grants have been provided in some programs within the Department of Defense and other agencies. Additional research refraining grants could

certainly be financed by the research agencies and perhaps administered through the Federal laboratories. Retraining for teaching would fall primarily

to universities that wish to improve the classroom (i.e., undergraduate) teaching of its faculty. See National Research Council, op. cit., footnote 78; and

Neal Lane, ‘‘Educational Challenges and Opportunities,’ Human Resources in Science and Technology: Improving U.S. Competitiveness, Proceedings

of a Policy Symposium for Governmen~ Acaden@ and Industry, Mar. 15-16, 1990, Washingto~ DC, Betty Vetter and Eleanor Babco (eds.)

(Washington, DC: Commission on Professional in Science and Technology, July 1990), pp. 92-99.

gsDegrees done tell an incomplete story of fiture supply of scientists and engineers. For example, college attendance rates of 18- to 21-yw-olds v~

by gender and race. Since 1972, 35 to 40 percent of whites of both sexes in the cohort have attended college with Black rates in the 25 to 30 percent

range. By 1988, female attendance exceeded that of males and was rising, whereas male attendance of both races peaked in 1986-87 and declined

thereafter. See National Science Board, op. cit., footnote 12, p. 50, figure 2-2.

gTFor an Ovewiew, see Commission on professio~s in Science and Technology, Measuring National Needs for Scientists to the Year 2000, Report

of a Workshop, Nov. 30-Dec. 1, 1988 (Washington DC: July 1989), pp. 20-24. For more on graduate engineering educatio~ see Elinor Barber et al.,

Choosing Futures: U.S. and Foreign Student Views of Graduate Engineerin gEducation (New York NY: Institute of International Educatio~ 1990).
32 ● Federally Funded Research: Decisions for a Decade

Photo credit: Bob Kalmbach, (Universityof Michigan

These students are in a laboratory at the University of

Michigan. Laboratory classes are a crucial part of

undergraduate education in the physical sciences.

Increasing the participation at all educational

levels in s/e by traditionally underrepresented

groups is a challenge to the human resources goals

of the Federal research system. Enhancing the

participation of targeted groups at the Ph.D. level

will be particularly difficult, as the research work

force adjusts to changing fiscal conditions and

funding of research. As OTA found in an earlier

study, “. . . equal opportunity for participation in

higher education and in research for all groups is a

long-term social goal that will be achieved only with

steady national commitment and investments. ’ ’88

Congress could amend the Higher Education Act

(reauthorization is scheduled for the 102d Congress)

and the Science and Engineering Equal Opportunities Act to add provisions that address diversity in

research and science education funding, and emphasize undergraduate teaching opportunities at certain

categories of institutions such as historically Black

colleges and universities (HBCUS).89 Programs

targeted to U.S. minorities, women, and the physically disabled could help to expand the pool of

potential scientists and engineers. It is clear that in

Figure 1-1 I--Science and Engineering Degrees:

1966-88 (by level)

Bachelors

Degrees earned (in thousands)

150

100 100

5 0 5 0

Masters

Degrees earned (in thousands) 70 70

4 0 4 0

3 0 - -

3 0

20 - -

2 0

10 - -

10

I 1 [ 1 t [ o 1 1 ! , 0

1966 1971 1976 1981 1986

Doctorates

Degrees earned (in thousands) 25 I

2 0 -

15 -

10 

1 1

0 1

1966 1971 1976 1981 1986

25

2 0

15

10

5

0

SOURCE: National Science Foundation, and Engineering Degrees: 7966-88, A Source Book, NSF 90-312 (Washington, DC:

1990), detailed statistical tables, table 1.

 of Technology Assessment, op. cit., footnote 74, p. 

 of current provisions, see Margot A. Congressional Research Service, “Higher Education: Reauthorization of the Higher

Educational” Issue Brief, May 15, and Public Law 96-516,94 Stat, 3010, Section II, Science and Engineering Equal Opportunities Part

B, as amended by Public Law 99-159, 1982.

.
Chapter I—Summary and Conclusions ● 33

this particular realm of human resources, market

forces alone will not increase the participation of

these groups. Policy intervention is required and

Congress is empowered to intervene.

The capacity of the research system could also be

augmented by encouraging ‘‘have-not” institutions

to concentrate excellence in select research programs (departments and centers) and build from

there. Attempting to enter the top ranks of federally

funded research-intensive universities through

across-the-board enhancement of all research programs may lead to each program being unable to

garner enough support to improve research capability. Various programs that address geographical

diversity, such as the NSF Experimental Program to

Stimulate Competitive Research (EPSCoR), or

greater consideration of geography in funding allocation within the portfolios of mainstream scientific

merit-based programs, could build research capacity

that benefits States and regions as well as the Nation

as a whole.

Research and Education in Flux

Calls for the reform of higher education in the 21st

century are now emanating from many presidents of

research universities.90 These calls center on improved undergraduate education and abetter balance

between research and teaching. Many see a need to

change the reward system of the university, since

asking universities to augment the teaching of

undergraduates may be misplaced if faculty continue

to view this as a drain on time that would be better

spent doing research.91

The tension between research and teaching is

perpetuated by the provision of funds meant to

improve both the institution’s research performance

and teaching capability. A common perception

during the 1960s was that Federal dollars that

supported research also benefited undergraduate

teaching because these top researchers would communicate their excitement about developments ‘‘at

the laboratory bench” to undergraduate and graduate students alike. In the 1980s, with the separation

between research and undergraduate education becoming more pronounced, the connection between

research progress and the cultivation of human

resources grew more tenuous.92 These calls for

increased undergraduate teaching by faculty seek to

alter an academic research and teaching model in the

United States that is already under strain.

The predominant mode of academic research in

the natural sciences and engineering begins with a

research group that includes a PI (most often a

faculty member), a number of graduate students, one

or several postdoctoral scientists, technicians, and

perhaps an additional nonfaculty Ph.D. researcher.

While this group may be working on a single

problem funded by one or two grants, subsets of the

group may work on different but related problems

funded simultaneously by multiple project grants.

(In the social sciences, the groups tend to be smaller,

often numbering only the faculty member and one to

two graduate students.)

In addition, the dominant model to launch a career

as a young scientist is movement from one research

university to another with an assistant professorship,

the attainment of a first Federal research grant, and

the re-creation of the mentor’s professional lifestyle

(e.g., independent laboratory, graduate students,

postdoctorates). For an institution to subscribe to

this model tends to shift much of the actual

responsibility for awarding tenure from the department faculty to the Federal Government. While

university officials say there is “. . . no fried time in

which researchers are expected to become selfsufficient through outside grants . . . researchers

who have failed to win such grants are less likely to

~~ominent ~ong them me the two irMtitutions that OTA studied as part of this assessment Stanford and Michigan. See Karen Grassmuck “Some

Research Universities Contemplate Sweeping Changes, Ranging From Management and Tenure to Teaching Methods,” The Chronicle of Higher

Education, vol. 37, No. 2, Sept. 12, 1990, pp. Al, A29-31.

slThis wo~d includeno~g less @ a r~efinition of faculty scholarship that includes teaching. See Ernest L. Boyer, Scholarship Reconsidered:

Priorities of the Professoriate (Princeto~ NJ: The Carnegie Foundation for the Advancement of Teaching, 1990). Also see Alliance for Undergraduate

Educatio~ The Freshman Year in Science and Engineering: Old Problems, New F’erspectivesfor Research Universities (University Park, PA: 1990).

~See~thony B. Maddox and Renee P. Smith-Maddox,‘‘Developing Graduate School Awareness for Engineering and Science: A Model,’ JournaZ

of Negro Education, vol. 59, No. 3, 1990, pp. 479-490. This connection was also highlighted when institutions of higher education receiving Federal

assistance were required to provide certain information on graduation rates, reported by program and field of study. See Public Law 101-542, Title

I—Student Right-To-Know, Stat. 2381-2384, Nov. 8, 1990, p. 104.
34 ● Federally Funded Research: Decisions for a Decade

Photo credit: Jay Mangum Photography

Seventh graders observe research in a “cleanroom.” Seeing science at work is important for all age groups.

earn tenure than their colleagues who have found or national laboratory stretch the resources

such support’ ’93 experience of both participating institutions.94

and

There is doubtless a role for universities to play in

the diversification of research careers of recent New Models: University and Federal

Ph.D.s. New Ph.D.s find it difficult to entertain

alternative opportunities if they have no experience

with them. Thus, programs that offer a summer in a

corporate laboratory or part of an academic year at

a 4-year liberal arts college can help advanced

graduate students visualize working in settings other

than the university. Arrangements that link an

HBCU or liberal arts college to a research university

Other models of education could be encouraged

that feature a greater sharing of resources (e.g.,

equipment and space) and people (e.g., doctoral

students, nonfaculty researchers, and technicians).

Models that stress research in units other than

academic departments, research in nonacademic

sectors, and nonresearch roles in academia could be

entertained. Some Federal research agencies already

 ‘‘Yowger Scientists Feel Big Pressure Battle “ The Chronicle vol. 37, No. 4, Sept. 26,1990,

p. A16. As one researcher puts it: “Leading universities should make their own decisions about who their faculty are going to be, and not leave it to

the study sections of ND-I. ” Quoted in David Wheeler, “Biomedical Researchers Seek New Sources of Aid for Young Scientists,” The Chronicle of

Higher Education, vol. 36, No. 42, July 5, 1990, p. A23.

 National

Science Foundation will establish a communications network for information faculty exchange, workshops, and outreach to

elementary, secondary, and community college students. The participating universities are City College of New Howard, Maryland, Massachusetts

Institute of Technology, Morgan State, Pennsylvania State, and Washington. See “NSF Multi-Million Dollar Grants to Form 

Education Coalitions,” News, Oct. 9, 1990.
Chapter l—Summary and Issues for Congress ● .35

recognize the development of this form of teamwork

in their finding programs and support of the research

infrastructure. For example, these models are institutionalized in the centers programs sponsored by

NSF. Centers, which support individual researchers

(as faculty and mentors) as well, may represent a

new way of doing business for NSF. Centers are also

featured at NIH intra- and extramurally; at the

laboratories affiliated with DOD, DOE, and NASA;

and at the agricultural experiments stations funded

through block grants by USDA.95

Research in general is becoming increasingly

interdisciplinary, i.e., it requires the meshing of

different specializations to advance a research

area.

96 Academic departments house specialists by

discipline whose research will be performed in

units-enters, institutes, programs-that cut across

the traditional departmental organization on campus. Such organized research units have a history on

U.S. university campuses, but not as dominant

structures.97 However, as outlined above, in many

fields there is movement toward an industrial model

of research, characterized by larger research teams

and a PI who spends more time gathering funds to

support junior researchers who in turn devote their

full time to research. For many of today’s research

activities, this model seems to enhance productivity

and allow more complex research problems to be

tackled, by specializing responsibilities within the

research team and sharing infrastructure.98

The expanding size and complexity of research

teams under the responsibility of entrepreneurial PIs

and “lab chiefs’ fosters financial and organizational strains. To help ease the strains caused by a

transition in some parts of the research community

to an industrial model, the Federal research agencies

could encourage alternative models of education-inresearch that feature a greater sharing of resources

and people. While it is not the role of the Federal

Government to dictate university research or education policies, it can provide the impetus for examining and experimenting with those policies through

grant support.

Mainstream agency programs have always

awarded research funds to advance the state of

knowledge in their programmatic areas mainly on

the core criterion of “scientific merit. ” Though

difficult to define precisely, this is generally taken as

a necessary condition for funding. Recognition that

discipline-based agency programs favor investigator

track record in proposal review, but that other factors

reflect important objectives of research funding, led

to the creation of set-aside programs. These programs, originating both in Congress and within

agencies, restrict the competition for scarce funds

according to some characteristic of the investigator

or the proposal. Set-aside programs thus evaluate

proposals first and foremost on scientific merit, but

redefine the playing field by reducing the number of

competitors. (Examples discussed in the full report

include NIH’s Minority Biomedical Research Support Program; NSF’s aforementioned EPSCoR,

Presidential Young Investigator, and Small Grants

for Experimental Research programs; and the Small

Business Innovation Research programs conducted

by various Federal agencies.)

Taken together, such programs address the competitive disadvantage faced by young, minority, or

small business research performers; by researchers

and institutions in certain regions of the Nation; and

by ideas deemed ‘high-risk’ by expert peers or that

do not fit with traditional disciplinary emphases.

The proliferation of such programs over the last 20

years has been a response to the desire to enlarge

9sIn 1990, the NatiO~ science Fowtition supported 19 Engineering Research Centers and 11 Science and Technology Rwmch Centers (STCS)

at $48 million and $27 millioq respectively. Thus, together they account for less than 10 percent of the National Science Foundation’s budge~ while

providing a long-term funding base (5 to 11 years) for interdisciplinary and high-risk projects oriented to the applied, development, and commercial-use

end of the research continuum. See Joseph Palca and Eliot Marshall, “Bloch Leaves NSF in Mainstmxuq” Science, vol. 249, Aug. 24, 1990, p. 850.

In the block-grant, multi-investigator approach embodied by STCS: ‘‘NSF has rolled the dice on an experiment in science, and it will take some time

to know whether it has come up with a winner.”See Joseph Palc& “NSF Centers Rise Above the Sto~” Science, vol. 251, Jan. 4, 1991, pp. 19-22,

quote from p. 22.

%Fore=ple, s= A*L. Porter and D.E. ~ub@‘‘~ ~diwtorof Cross-Discip- Re~c~’ sciento~m”c~, VO1. 8, 1985, pp. 161-176; and Don

E. Kash, “Crossing the Boundaries of Disciplines,” Engineering Education, vol. 78, No. 10, November 1988, pp. 93-98.

97D61. Phillips ~dB-p-s. Shen (eds.), Research in the Age @~& ~re@Y-s~afe unive~@ @O@&r, co: wes.tview press, 1982). ~ee mOdels Of

organized research units (which me common in industry and the Fedeml laboratories) have taken rootoncampu~agricultural experiment stations, water

resources research centers, and engineering research centers, See Robert S. Friedman and Renee C. Friedman, “Science American Style: Three Cases

in Academe,” Policy Studies Journal, vol. 17, fall 1988, pp. 43-61.

SWW Z- op. cit., footnote 64, pp. 132-139, III other words, the traditional academic model of facul~-mentor plus graduate s~dent is todaY

accompanied by production units that demand more teamwork and sharing-what has long been commo~ for example, in astronomy, fusion, and

high-energy physics research.
36 Federally Funded Research: Decisions for a Decade

both the participation in, and the capacity of, the

Federal research system. But because the annual

finding for each program remains modest (typically

in the $10 million range), program impact is limited.

Without set-asides, the Federal Government

would have little confidence that once scientific

merit has been demonstrated, other differentiating

criteria would be applied to the funding of researchers. However, to a research system already strapped

for resources, the finding of such “tangential”

concerns is seen by some as diverting precious

dollars away from the core need to advance knowledge. 99

Human resources are perhaps the most important

component of the research system. Through support

of scientists and engineers, graduate students, and

the educational pipeline, the Federal Government is

instrumental in the creation of a strong research

work force, which has been expanding under this

support since the 1950s. In the 1990s, however, the

research work force-in its myriad forms of organization and scale of effort-has reached such a size

that it feels strain under the Federal Government’s

present approach to supporting the conduct of

research. In addition, accommodating to an expanding research work force, and to the changing ethnic

and racial composition of students in the educational

pipeline for science and engineering, poses challenges to the Federal research system. Human

resources issues have implications not only for the

number of participants in the research work force,

but also for the character of the research that new

entrants automatically bring to the Nation’s research

enterprise.

ISSUE 4:

Data

Refining Data Collection and

Analysis To Improve Research

Decisionmaking

Summary

collected on the health of the Federal

research system--dollars spent for research,

enrollments, and academic degrees awarded in

specific fields, and outcome measures such as

publications and citations—are extensive. In

other areas, however, data are scarce. For

instance, almost no consistent information

exists on the size and composition of the

research work force (as opposed to the total

science and engineering work force), or what

proportion is supported by Federal funds

(across agencies).

Most research agencies, with the exception

of NSF and NIH, devote few resources to

internal data collection. Consequently, most

analyses must rely on NSF and NIH data and

indicators alone, potentially generalizing results and trends that might not apply to other

agencies. Furthermore, it is not clear how

agency data are used to inform research decisionmaking, as some challenge current policy

assumptions and others are reported at inappropriate levels of aggregation.

OTA suggests additional information that

could be collected for different levels of

decisionmaking, concentrating in areas of policy relevance for Congress and the executive

branch. However, better information may not

be cost-flee. The idea is not merely to add to

data collection and analysis, but to substitute

for current activities not used for internal

agency decisionmaking or external accountability. Refried inhouse and extramural data

collection, analysis, and interpretation would

be instructive for decisionmaking and managing research performance in the 1990s.

Discussion

Many organizations collect and analyze data on

the research system. First and foremost is NSF, with

its numerous surveys, reports, and electronic data

systems that are publicly available. Certainly the

most visible compendium of data on the research

system is the biennial report, Science & Engineering

Indicators (SEI), issued since 1973 by the National

Science Board, the governing body of NSF.l00 Other

sources include the other Federal research agencies;

the National Research Council; the Congressional

Research Service; professional societies, especially

the American Association for the Advancement of

~Change comes incremen~y and at the margins of the enterprise. But if one were constructing the SyStem frOm SCmtC& mtim- criteria to

reflect the multiple objectives of research funding would be a key element to consider.

l~see Susan E. Cozzens, “Science Indicators: Description or Prescription?” OTA contractor report, September 1990. Note that Science &

Engineering Indicators (SEI) was named Science Indicators until 1987. SEI builds on data collected, published, and issued in many other reports

by the Science Resources Studies Division of the National Science Foundation.
Chapter l—Summary and Issues for Congress ● 37

Science; and other public and special interest

groups.

l0l

Together these databases and analyses provide a

wealth of information: time series on the funding of

research and development (R&D); expenditures by

R&D performer (e.g., universities and colleges,

industry, Federal laboratories), by source of funding,

and by type (basic, applied, or development);

numbers of students who enroll in and graduate with

degrees in s/e; characteristics of precollege science

and mathematics programs and students in the

education pipeline; and size, sectors of employment,

and activities of the s/e (especially Ph. D.) work

force.102 Detailed analyses of the Federal budget by

research agency are available each year, and impacts

on specific disciplines and industries can often be

found.

These publications provide a basis for understanding the Federal research system. But even with

each of these organizations devoting significant

resources to the collection of information, better data

are needed to guide possible improvements in the

system. 103 With its establishment, NSF was legislatively authorized as the Federal agency data liaison

and monitor for science and technology.104 Data can

be used to monitor, evaluate, anticipate, and generally inform decisionmakers-both within agencies

and within Congress. Although many data are

already collected, they are rarely matched to policy

questions. Other (or more) data could improve

decisionmaking.

Information for Research Decisionmaking

OTA defines four categories of data that could be

useful in decisionmaking: 1) research monies—how

they are allocated and spent; 2) personnel— characteristics of the research work force; 3) the research

process-how researchers spend their time and their

needs (e.g., equipment and communication) for

research performance; and 4) outcomes—the results

of research. Besides the considerable gaps and

uncertainties in measures of these components, the

most detailed analyses are done almost exclusively

at NSF and NIH, and not at the other major research

agencies. l05 These analyses may not generalize

across the Federal research system. Comparable data

from all of the agencies would be very useful to gain

a more well-rounded view of federally supported

research.

Perhaps the most fundamental pieces of information on the research system are the size, composition, and distribution of the research work force, and

how much is federally funded. Varying definitions

pose problems for data collection and interpretation

(for an example, see box l-C). These data are

important to understand the health and capacity of

the research system and its Federal components. In

addition, there is evidence that research teams are

changing in size and composition. This trend is also

important to measure since it affects the form and

distribution of Federal funding.

Second, information is needed on expenditures

(e.g., salaries, equipment, and indirect costs) in

research budgets; for all research performersacademia, Federal laboratories, and industry; and by

subfield of science and engineering. Data on how

Federal agencies allocate monies within project

budgets could also be compiled, and would illuminate how funding decisions are made within the

research agency and would help to clarify funding

levels in specific categories of expenditures. Better

cost accounting and forecasting for megaprojects is

IOIFor example, see Natiod Research Council, Surveying the Nation’s Scientists and Engineers: A Data System for the 1990s (wasmton, ~:

National Academy Press, 1990). Under multiagency support, the National Research Council is well known for collecting, analyzing, and disseminating

information on Ph.D. recipients. For a statement of its crosscutting role, see National Academy of Sciences, The NationaZ Research Council: A Unique

Institution (TVaahingtoU DC: National Academy Press, 1990). For a summary of major databases on science and engineering (individuals and

institutions), see National Research Counci~ Engineering PersormeZData Neea!sfor the 1990s (Washingto~ DC: National Academy Press, 1988), app.

A-2.

lo~or example, the bverment-u~vemi~-~dw~ Research Roundtable of the National Academy of Sciences, with data compiled by the Natioti

Science Foundation’s Policy Research and Analysis Division, provided much useful analysis on the state of academic R&D and changes since the early

1960s. Government-Industry-University Research Roundtable, op. cit., footnote 48.

los~ese effo~ must alSO be seen in the context of the massive Federal data system. The components most relevant to research me the data seri~

compiled and reported by the Census Bureau, the Bureau of Economic Analysis, the Bureau of Labor Statistics, and the National Center for Education

Statistics.

l~For the SCOP of th~e data co~ection and analysis responsibilities, see England, op. cit., footnote 8, app. 1.

IOsFOr example, the Natio@ ~titutes of Health sets aside 1 percent of its research budget for research evaluation and inte~ a~ysis of the

investigators and programs it supports. The Department of Energy, the National AerOMUtiCS and Space Administration the OfYIce of Naval Research

and the National Science Foundation have all conducted ad hoc inhouse evaluations of the research they support and the efficiency of the operations

needed to select and manage various research portfolios.
38 ● Federally Funded Research: Decisions for a Decade

Box l-C—How Many “Scientists” Are There?

How one defines a “scientist,” “engineer,” researcher,” or “postdoctorate” is in the eye of the beholder.

Depending on what data collection method is used, counting“ scientists and engineers (WE’s) can result in radically

different estimates. 1

Definition Number

S/E’s in the U.S. work force (defined by job held).. . . . . . . . . . . . 5,300,000

S/E’s in academia (defined by responses to surveys in academic institutions) . . . . . . . . . . . . . . . . . . . . 712,000

S/E Ph.D.s in basic research--all sectors (defined by responses to surveys of Ph. D.s).. . . . . . . . . . . . 187,000

S/E Ph.D.s in academia, where research is either their primaryor secondary work activity

(defined by responses to surveys of Ph.D.s) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155,000

S/E Ph.D.s in basic research in academia (defined by reponses to surveys of Ph.D.s) . . . . . . . . . . . . . 66,000

Full-time equivalent S/E investigators in Ph.D. institutlons (e.g., two resear chers who each spend

half-the on research would be counted as one full-time equivalent S/E investigator) . . . . . . . . . . . . 63,000

None of these definitions is the ‘right’ one. Rather, the appropriate definition depends on the purpose for

which the number is to be used Throughout this report, OTA refers to scientists and engineers in many ways:

e.g., by participation in the U.S. work force, by sector of employment, by work activity, by field, by highest

degree earned. The reader should keep in mind that the numbers can change by tenfold or more depending on

who is counted as a scientist or engineer.

1Most of the following numbers are taken from the National Science Board, kience & EngiIWWW“ in&atOrS4 989, NSB 89-1

w~ DC: 1989), amt are 1987 or 1988 estbwes by th NationaI !%mce ~’ S sckx% RcmaRxa Stlxfics DMsioa. l’llc

number of full-time cquivakm investigators is based on analysis by the Natkmd Sciuxc FoumWun ‘s R?*~ and halyais

Divisim as reported in Govcrnmm t-University-Industry R~ h RoundtablG Science and Tdnology in theAcaobticEruer@.w: tkztus,

Trenu3, and Issues (WaahingtOU DC: National Academy has, October 1989), p. 2-51.

surely needed. Continuous upward revisions of cost process would yield a firmer foundation on which to

estimates for megaprojects disrupt decisions about

their future funding priority.

Third, data on the research process could be

improved in amount and kind. One trend (mentioned

above) that OTA has noted, mostly with anecdotal

evidence and inferences from analyses of expenditures, is the increasing size of research groups, both

within the university structure and through Federal

support of centers. This trend has policy implications for the cost of research, its interdisciplinary

capabilities, the changing demographics of the work

force, and the aspirations of young researchers. It

also reflects how researchers may spend their time.

More data on ‘ ‘production units” in research, and

their dependence on Federal finding relative to other

sources, would augment enrollment, Ph.D. award,

and work activity data. Changes in the structure of

production units have also influenced the research

process and the volume---and perhaps the character----of outcomes.106 Information on the research

base funding allocation decisions, specifically: 1)

how researchers spend their time, 2) movement of

research teams toward a more industrial model in the

allocation of responsibilities, 3) changing equipment needs and communications technologies, and

4) requirements and average time to attain promotions in the scientific work force.

Evaluating Research Outcomes

Because of the fundamental and elusive nature of

research, measuring its outcomes-in knowledge

and education—is very difficult. The most elusive

outcome is cultural enrichment-the discovery and

growth of scientific knowledge. As OMB Director

Richard Darman has said (speaking of the proposed

Moon/Mars mission): “No one can put a price on

uplifting the Nation. Research has resulted in many

benefits and is funded precisely for this reason. This

kind of benefit is nearly impossible to measure.

However, there are some proxies.

106 The role of laboratory chief or team leader combines entrepreneurial andadmini“ strative/superviso~ tasks. Both are essential to the funding and

longevity of the productive research unit. On the emergence of the entrepreneurial role on campus, see Etzkowitz, op. cit., footnote 61.
Chapter 1—Summary and Issues for Congress ● 39

When looking at research as a contribution to

education, numbers of degrees can be tallied and

assertions made about skills added to the Nation’s

work force. When looking at research as creating

new knowledge, one tangible ‘‘output” is papers

published by scientific investigators to communicate new information to their scientific peers.

Communicating the results of scientific research to

colleagues through publication in the open literature

is considered to be an important feature of good

research practice. 107 perhaps the best approach is to

construct workable indicators and include a rigorous

treatment of their uncertainties.

One tool that has been vigorously developed for

measuring the outcomes of research is bibliometrics-the statistical analysis of scientific publications and their attributes.108 Intrinsic to scientific

publication is the referencing of earlier published

work on which the current work is presumably based

or has utilized in some way. References are a

common feature of the scientific literature, and by

counting how often publications are cited, bibliometrics can arrive at a weighted measure of publication impact—not only whether publications have

been produced, but also what impact those publications have had on the work of other scientists.l09

OTA has explored several examples of new data

sets that could be compiled using bibliometrics.l10

First, universities can be ranked according to an

output or citation measure-the citation rates for

papers authored by faculty and others associated

with each institution. 111 Instititutions can be ranked

by total number of cited papers, the total citations

received by all papers associated with each institution, and the ratio of number of citations to the

number of publications, namely, the average citations per cited paper. This appears to be a more

discerning measure than either publication or citation counts alone.

For example, a ranking of institutions by average

citation rates can be used in conjunction with the list

of top universities, in Federal R&D funding received, to link inputs with outputs. Together, these

measures illuminate differences in rank.112 Not only

can publishing entities be analyzed, but so can fields

of study. For instance, “hot fields, ’ in which the

rate of publication and citation increases quickly

over a short period of time, can be identified and

“related fields,‘‘ in which published papers often

cite each other, can be mapped.113 Because of

problems of interpretation in bibliometric analysis,

it should be seen as ‘‘value-added” to research

decisionmaking, not as stand-alone information.

Bibliometrics could be used to help monitor outcomes of research, e.g., publication output and other

information from the research system.114

Criteria that go beyond bibliometric data could be

specified for such evaluations. These criteria could

include the originality of research results, the

project’s efficiency and cost, impacts on education

and the research infrastructure, and overall scientific

merit. Such research project evaluation could be

loTFor example, see Leah A. Lievrouw, “Four Resarch Programs in Scientific Communication, ” Knowledge in Society, vol. 1, summer 1988, pp.

6-22; and David L. Hull, Science as a Process: An Evolutionary Account of the Social and Conceptual Development of Science (Chicago, IL: University

of Chicago Press, 1988).

108ReseUchers ~ western EWope ~ve ken p@m~ly active during the lggmo For example, see B.R. mand J. Irvtie, Research Foresight

(London, England: Pinter, 1989); and A.FJ. van Ram (cd.), Handbook of QWntitative Studies of Science and Technology (Amsterdam, Holland:

North-Holland, 1988).

l~nterpreting citation patterns remains a subject of contention. For caveats, see D.O. Edge, “Quantitative Measures of Communication in Science:

A critical Review,” History of Science, vol. 17, 1979, pp. 102-134. The deftitive overview is contained in Eugene Garfield, Citation lndem”ng: lts

Theoq and Application in Science, Technology and Humanities (New Yor& NY: John Wiley & Sons, 1979).

llOSee Hew S@ and David Pendlebw, ‘(F~er~ Support Of ~ad~g Edge Research: Report On a Method for Iden@@ hmovative Areas Of

Scien&lc Research and Their Extent of Federal Suppo~’ OTA contractor repo~ February 1989; and Henry Small, “Bibliometrics of Basic Researriu’

O’IA contractor repo~ September 1990.

11 l’rhe a~ysis IR1OW is basal on hqimte for ScienMlc Information databases and SUUI.1, op. cit., footnote 110.

1 lzAs part Of the agenda for future exploratio~ institutions rweivhg P rimarily directed funds or block grants (e.g., in agriculture) could be compared

with those that are investigator-initiated. This comparison would help to test the claim that targeted appropriations (e.g., earmarking) lead to the

production of inferior research. For discussio% see ch. 5 of the full OTA report.

llJFor example, see Angela MarteUO, “Governments Id in Funding 1989-90 ‘Hot Papers’ Research, “ The Scientist, vol. 4, No. 16, Aug. 20, 1990,

pp. 2@23.

114see us+ conpess, OffIce of ‘r’w.oloH Assessmm~ Research Funding as an lnvest~nf: can we Meu~re the Returns? Om-TM-SET-36

(Washington, DC: U.S. Government Printing OIXce, April 1986); and Ciba Foundation, The Evaluation of Scientific Research (New York NY: John

Wiley & Sons, 1989). For evidence on U.S. reseamh performance relative to seven other industrialized countries, see ‘No Slippage Yet Seen in Strength

of U.S. Science,” Science Watch, vol. 2, No. 1, January/February 1991, pp. 1-2,
40 ● Federally Funded Research: Decisions for a Decade

Photo credit: Jay Mangum Photography

A Research Triangle Park scientist accesses a computer

network. Computers can greatly enhance data collection

and presentation.

employed to augment agency decisions on funding

and administration of research programs. (Some

research agencies already utilize certain aspects of

research program evaluation.115)

Utilizing Data for Research Decisionmaking

In a policy context, information must be presented

to those who are in positions to effect change by

allocating or redirecting resources. In the diverse

structure of the Federal research system, research

decisions are made at many levels. For example, an

agency program manager requires data specific to

the purview of his/her program, while OMB and

OSTP must be aware of trends in science that span

broad fields, institutions, and agencies, as well as

those that apply only to specific fields, performers,

and research sponsors.

Drawing on NSF expertise as the possible coordinating “agency, information could be collected at

each agency on proposal submissions and awards,

research expenditures by line items in the budget,

and the size and distribution of the research work

force that is supported (including the funding that

this work force receives from other sources). Information must be available to decisionmakers for

evaluation as well as to illuminate significant trends.

Often data can be presented in the form of indicators, e.g., comparisons between variables, to suggest

patterns not otherwise discernible. NSF has pioneered and sustained the ‘creation of indicators for

science policy and has recently suggested monitoring several new indicators (e.g., indicators of proposal success rates, PI success rates, and continuity

Of NSF support).ll6

OTA agrees that new indicators could be very

useful, and also suggests elaborating them. These

could include measures of the active research

community (which would calibrate the number of

researchers actively engaged in research), and production units (which would track trends in the

composition of research teams by broad field and

subfield).

The combination of such indicators would give a

more precise estimate of the changing parameters of

the Federal research system.117 This information

would be invaluable to policymakers concerned

about the health of certain sectors of the system. To

produce such information, as part of ongoing agency

data collection and NSF responsibilities for collation

and presentation, extra resources would be needed

(at least in the near term). Over time, plans could be

developed to streamline NSF data and analysis

activities, such as a reduction in the number of

nonmandated reports issued annually, or expansion

of its inhouse and extramural ‘‘research on research. The idea is not merely to add to data

collection and analysis, but to substitute for current

activities that are not used for internal agency

 U.S. of Energy, of Program Analysis, of Energy of the Sciences

Program, DC: 1982). For a review of other evaluations, see National Academy of Sciences, Committee on Science,

Engineering, and Public Policy, The Quality of Research in Science DC: National Academy Press, 1982), app. C.

 Science “NSF Vital Signs: Trends in Research Support, Fiscal Years 198089, ” draft NOV. of 

indicators reported here were used for an National Science Foundation evaluation of ways to streamline the workload of program staff and the

external research community. See National Science of the MeritReview Task Force, NSF 90-113 (Washington DC: Aug. 23, 1990).

 that growth in is slowing or that the a research 

large or small relative to the resources supporting it? See “Policy Experts Ask a Heretical Question: Has Academic Science Grown

TOO Big?” The Chronicle of Higher Education, vol. 37, No. 3, Nov. 21, 1990, pp. Al, A22.
Chapter l—Summary and Issues for Congress ● 41

decisionmaking or external accountability. 1!8 If

there is a premium on timely information for

research decisionmaking, it must be declared (and

funded as) a Federal priority.

Congress could instruct every research agency to

develop a baseline of information, direct NSF to

expand its focus and coordinating function for data

collection and analysis, and direct OSTP (in conjunction with OMB) to devise a plan to increase the

reporting and use of agency data in the budget

process, especially crosscutting information in priority research areas. Using the FCCSET mechanism,

this has already been done for global change,

high-performance computing, and most recently,

science and mathematics education.

119 This mechanism seems to work and could be more widely

emulated.

In Summary, better data on the Federal research

system could be instrumental in the creation or

refinement of research policies for the 1990s. (For a

summary of data oriented to different users, see table

1-5.) The utility of data, of course, is judged by many

participants in the system: the needs of Congress are

usually agency- and budget-specific; 120 the agencies, in contrast, worry about the performance of

various programs and their constituent research

projects. While data collection by NSF and groups

outside the Federal Government has been instructive, it could be greatly enhanced. Much information

could be collected on the Federal research system

that maps trends, at different levels of aggregation

and units of analysis, for different users. However,

the existence of data does not ensure their utility.

The highest priority in data collection for research

policymaking in the 1990s is comparable data from

all of the agencies to help Congress maintain a

well-rounded view of federally supported research.

The second priority is data presented in forms that

are instructive at various levels of decisionmaking.

New data and indicators, grounded in the tradition of

the SEI volumes and extramural research on research, are needed to monitor changes in the Federal

research system.121 Finally, OTA finds that research

evaluation techniques, such as bibliometrics and

portfolio analysis, cannot replace judgments by

peers and decisionmakers, but can enrich them.

Ongoing project evaluation could keep agencies

alert to changes in research performance and augment program manager judgments about performers

and projects. In short, such evaluation could serve to

improve overall program effectiveness.

One of the functions of analysis is to raise

questions about the information that decisionmakers

have at their disposal, to assess its advantages and

disadvantages, and to define a richer menu of

options. 122 Improving the measurement process

could help to quantify existing opportunities and

problems, and pinpoint previously uncovered ones,

relevant to decisionmaking at all levels of the

Federal Government.

Toward Policy Implementation

Since the post-Sputnik era, both the U.S. capacity

to perform research and the demand for funds to

sustain scientific progress have grown. Federal

investments have fostered the research system,

managed through a pluralistic agency structure. This

structure has supported the largest and most producI lg~eNatio~ Science Fo~dation routinely conducts ‘uSer SurveyS. ’ If Science Resources Studies (SRS) knows fiomquestiormaim responses how

its various data reports are used--do they influence research or education policies? are they a source for administrators or facuhy-researchers? -then

NSF should have a sense of audience “consum ption’ and ‘ ‘utilization’ patterns. These would suggest which reports could be &opped, replac~ and

moditled. For an example of the SRS inventory of ‘‘intramural publications,’ see National Science Foundation Publications list: 1977-1987, NSF

87-312 (wdl@tO~ DC: Jdy 1987).

1190TA ~temiew with OffIce of Management and Budget staff, Feb. 7, 1991.

120& Sever~Natio~ Science Fomdation staff~ve indicat~ to o~ proj~t SW (perso~ communi~tions, october-~ember 1990), the fkieIICe

Advisor draws heavily on unpublished and newly published Science & Engineen”ng Indicators (SEX) data in preparing and presenting the

Administration’s policy proposals at congressional “posture hearings” early in the annual authorization process. Indeed, the production cycle of SEI

is geared to delivery of the volume as an input to this budget process.

lzlQ~titative ~~~not ~lce. ~omtion on tie contexts inw~chr=~chispfol-m~, and c~acteristicsof the performers individually ad

colhxtively, will provide clues to how t-he numbers can be interpreted and perhaps acted on. For example, see Daniel T. Layzell, “Most Research on

Higher Education Is Stale, Irrelevant, and of Little Use to Policymakers,’The Chronicle of Higher Education, vol. 37, No. 8, Oct. 24, 1990, pp. Bl,

B3.

1’221M5 le~s OTA to Su=e,q bit the resmch agencies, especially the National Science Foundation and its policy pmgHuM, remain in C1O* touch

with external analysts of the Federal research system. Keeping abreast of other new measurement techniques and findings related to people, funding,

andresearchactivities would be a modest but fruitful investment in extending inhouse capabilities and reftigknowledge of fedem.llyspomored research

performance.
42 ● Federally Funded Research: Decisions for a Decade

Table 1-5-Desired Data and Indicators on the Federal Research System

Primary users

Category Description Method Congress Agencies OMB OSTP

Agency funding

allocation method

Research expenditures

Research work force

Research process

Outcome measures

Indicators

Funding within and across fields and

agencies

Cross-agency information on proposal

submissions and awards, research

costs, and the size and distribution of

the research work force supported

Research expenditures in academia,

Federal and industrial laboratories,

centers, and university/industry collaborations

Agency allocations of costs within research project budgets, by field

Megaproject expenditures: their components, evolution over time, and construction and operating costs

Size and how much is federally funded

Size and composition of research groups

Time commitments of researchers

Patterns of communication among researchers

Equipment needs across fields (including

the fate of old equipment)

Requirements for new hires in research

positions

Citation impacts for institutions and sets

of institutions

International collaborations in research

areas

Research-technology interface, e.g., university/industry collaboration

New production functions and quantitative project selection measures

Comparison between earmarked and

peer-reviewed project outcomes

Evaluation of research projects/programs

Proposal success rate, PI success rate,

proposal pressure rates, flexibility and

continuity of support rates, project

award and duration rate, active research community and production

unit indices

Agency data col- X x x

Iection (and

FCCSET)

Agency data

collection

Lead agency

survey

Lead agency

survey; onsite

studies

Bibliometrics;

surveys of

industry and

academia

Agency analysis

x

x

x

x

x x

x

x

x

x

x

x

x

KEY: FCCSET=Federal Coordinating Council on Science, Engineering, and Technology; OMB=Office of Management and Budget; OSTP-Office of Science

and Technology Policy; Pl=principal investigator.

SOURCE: Office of Technology Assessment, 1991.

tive research capability in the world. For many system. In such an environment, the prospects of

decades, scientific research has contributed in

portant ways to the cultural, technological,

economic base of the Nation.

In the 1990s, changing funding pattern

im- fashioning a system that is responsive to national

and needs through selective, yet generous research

funding will demand well-informed, coherent policies. 123

and

various pressures from both outside and within the The system will face many challenges, but four

scientific community will test the Federal research are clear: First, new methods of setting priorities and

123As BrookstMSobserved: “l”hemxarchtmkrpnse is more like an organism than like a collection of objects. ‘he removal of onepart maY degrade

the functioning of the whole organism and not just the particular function ostensibly served by the part removed. ” Harvey Brooks, “Models for Science

Planning,” Public Administration Review, vol. 31, May/June 1971, p. 364. Policies must respond to, and in some ways, anticipate, the consequences

of funding decisions on the research system. Indeed, this report has tried to warn about extrapolating the past to manage the future of the system.
Chapter l—Summary and Issues for Congress ● 43

increased use of existing methods are required at all

levels of decisionmaking. Second, Federal expenditures for individual components of research projects

have increased faster than inflation. Understanding

and coping with these increases is imperative in

research decisionmaking. Third, the development of

human resources for the science and engineering

work force must occur through Federal incentives

and institutional programs that act on the educational pipeline (K-12 through graduate study). Finally, gaps and uncertainties in the data used to

describe the Federal research system must be

reduced, and be replaced by more routine provision

of policy-relevant information.

OTA finds that Congress, the executive

branch, and research performers must converge

on these issues. Potential congressional actions

fall into three categories. Congress can: 1) retain

primary responsibility for decisions and initiating actions; 2) place some of the responsibility for

coordination and decisions on the executive

branch; and 3) encourage research performers

(especially universities, as well as Federal and

industrial laboratories) to address components of

these issues. (For a summary of possible actions, see

table 1-6.)

At the congressional level, hearings, legislation,

and oversight should first address crosscutting and

within-agency priority setting at the national level.

OTA suggests that one or more committees of

Congress routinely (preferably biennially) hold

hearings that require the research agencies, OSTP,

and OMB to present coordinated budget plans with

analyses that cut across scientific disciplines and

research areas. Coordination among relevant committees of Congress would make this most productive. These hearings could also focus on crosscutting

criteria for research decisionmaking within and

across agencies. Emphasis must be placed on criteria

to expand the future capabilities of the research

system, such as strengthening education and human

resources. A second set of congressional actions

could explore cost-accountability efforts at the

research agencies and throughout the research system. A final set of hearings ought to examine the

state of data on the research system and improvements to inform congressional decisionmaking.l24

Table 1-6-Summary of Possible Congressional,

Executive Branch, and Research Performer Actions

Congressional hearings, ●

Iegislatlve efforts,

and oversight to:

●

Executive branch

actions to:

●

Research performer ●

actions to:

●

Set priorities across and within

agencies, and develop appropriate agency missions.

Evaluate the total portfolio to see

if it fulfills national research goals,

human resources needs, scientific infrastructure development,

and balance.

Initiate greater cost-accountability throughout the Federal research system.

Expand programs that fortify the

educational pipeline for science

and engineering, and monitor the

combined contributions of

agency programs to achieve education and human resources

goals.

Augment data and analysis on the

Federal research system for congressional decisionmaking.

Enhance cross-agency priority

setting in the Federal budget and

increase research agency flexibility to address new priorities.

Institute better cost-accountability and cost-containment

measures by agencies and research performers.

Expand agency programs to promote participation in the educational pipeline for science and

engineering, and require agencies to report progress toward

these goals.

Monitor and analyze policyrelevant trends on the research

system, especially as related to

the changing organization and

productivity of research groups

and institutions.

Contain and account for research

expenditures.

Revise education and research

policies as they affect: a) recruitment and retention in the educational pipeline for science and

engineering, and b) faculty promotion, tenure, and laboratory

practices.

SOURCE: Office of Technology Assessment, 1991.

Hearings could be followed with congressional

oversight-on agency progress toward their research missions, implementing the criteria chosen

by Congress to enhance research decisionmaking,

lmnere is a role for tie conw~sio~ support ~encies, as well as other sources of expert advice. For other proposals, see tinegie CO* sion on

Science, Technology, and Governmen~ Science, Technology, and Congress: Expert Advice and the Decisionmaking Process (New York NY: February

1991).
44 ● Federally Funded Research: Decisions for a Decade

instituting greater cost-accountability, and providing useful data and analysis on an ongoing basis to

Congress.

Some of these hearings and oversight efforts

already take place in committees of Congress. While

they have been very useful, OTA finds that to effect

change in the research system, congressional action

must be comprehensive and sustained. Posture

hearings with the Science Advisor and agency

directors will not suffice.

In its role as the prime sponsor of Federal research,

the executive branch (especially OSTP, OMB, and

the research agencies) could provide more flexibility

in response to changing research priorities. For

instance, the executive branch could systematically

initiate tradeoffs among agency research programs

including, with the cooperation of Congress, the

termination of programs. This would help to create

more coordinated research policies. Similarly, the

research agencies could institute greater costaccountability measures, and include costs as explicit factors in decisionmaking at the project level.

This would provide a more realistic assessment of

future capabilities with respect to projected funding

levels. On human resources issues, the executive

branch could implement or expand agency programs

and reporting requirements to: 1) encourage recruitment and retention of women, U.S. minorities, and

other underparticipating groups in the educational

pipeline for science and engineering; and 2) monitor

the changing structure of research performance,

especially forms of research organization, and devise funding allocation methods that accommodate

both the needs of the PI and research teams. Finally,

each of the research agencies (with NSF as the lead

agency) could conduct routine data collection and

analysis on policy-relevant aspects of their programmatic contributions to the research system.

Not all problems in the research system, however,

can be addressed in Congress or by the executive

branch. Universities and laboratories (both Federal

and industrial) are key components of the system,

and many policies are dictated by the practices

within these institutions. Containin g research expenditures and expanding the educational pipeline

through institutional programs and requirements are

examples of policy areas in which research performers must fulfill their role in the social contract

implied by the Federal patronage of research. The

Federal Government can only encourage universiPhoto credit: Jay Mangum Photography

Communication among scientists and engineers is an

essential part-of the research process.

ties and laboratories to follow new paths; few direct

Federal incentives are available to initiate change.

Greater delineation of government and research

performer responsibilities would help to sanction

congressional and executive branch action on problems in the research system.

In addition to specifying at which level (congressional, executive branch, or research performer)

issues could be appropriately addressed, responses

to the four challenges outlined above must also

recognize many inherent tensions in the research

system. They include the merits of more centralized

decisionmaking juxtaposed against the advantages

(and realities) of a decentralized Federal research

system. Other tensions arise between the funding of

mainstream individual investigator programs and

set-aside or more specialized programs (see again

table l-l). Inevitably, policies that relieve some

tensions will engender others.

In Summary, decisionmaking in the Federal research system concerns many laudable goals, and

the options are clearly competing “goods.” Thus,

the Federal Government must make tough choices,

even beyond issues of merit and constricted budgets,

in guiding the research system. A quarter-century

ago, a chapter on ‘‘Science and the Federal Government’ concluded with these words:
Chapter l—Summary and Issues for Congress . 45

As never before in history, the status of science patron and has fashioned a host of institutions to

and technology has become an important hallmark ofadminister vastly increased commitments to sciena nation’s greatness; and the United States clearly tific and technological excellence.l25

has perceived and acted upon this fact. In the process,

the Federal Government has displaced the univer- Sustaining and managing this system is the chalsity, industry, and the private foundation as chief lenge of the decade ahead.

IzSCited inwphs~ders andl%ed R. Brown (eds.), Science and Technology: VitalNationaZAssers (WaShingtorL ~: ~dustri~ college of the ~ed

Forces, 1966), p. 86.
CHAPTER 2

The Value of Science and the

Changing Research Economy
Contents

Introduction +? . . . . . . . . . . +.’......’.+o.......+ .. .. +. do+*”...-..*+. ● +. 0-. ””. . ””’” 5249

Research Funding in the united states .- .. ... =. ”..++... ..0..0” .00S0=**”O

+

0

9+-

.

Documenting Perspectives on the Future of Research . .. . .. . .. .......+. ... ..+..... 55

Historical and Current Federal Roles in the Research System .................~~~.+” . 60.

Prospects for the 1990s .. .. .+. . . . . . . . . . . . ...s”......+.--.....*.+... . .. -=**+* .*-+.-*

Boxes

Page Box

2-A. History of Superconductivity: Scientific Progress Then and Now . . . . . . . . . . . . . . . . . ~~

2-B. Public Interest in Science, ..”..... ..”...Q+.++..”.o.” .+oo.o.’+=*+..”**.*.*** *

2-C, Calculating Constant Dollar Trends for Research .. .. .. .. .. .. ... ++. ..~+~”++ .+... 56

2-D+ An Interpretation of Researchers’ Distress by Leon M. Lederman........~......... 58

2-E. The Perils of Being a Young Investigator .....”.....”. ..o+o.”+.ow++....* ● + 65
CHAPTER 2

The Value of Science and the Changing Research Economy

This is a golden age of scientific discovery with great potential to improve our

performance as a Nation. This is the rationale we use in our requests for increased

funding. But even a country as rich as the United States cannot write a blank check for

science. We need to discipline ourselves in how we request support and in how much we

ask for. Otherwise we will lose our credibility.

Frank Pressl

Introduction

Research advances the world stock of scientific

knowledge and the countries that finance its pursuit.

The United States, in particular, has a history of

strong support of research and belief in its inherent

worth. Scientific discoveries have spurred technological and other kinds of developments since the

beginning of the industrial age, and thus have shaped

much of Western culture. Cures to diseases have

been found, better automobiles and space probes

have been developed, the Earth and its environments

more fully understood, and the foundations of

atomic matter explored.

The importance of science to progress in most

Western societies is indisputable. In the words of

two economic historians:

Science. . . is pushing back the frontiers of

knowledge at what seems an accelerating pace.

Because knowledge creates economic resources and

because knowledge generally grows at an exponential rate, future advances in human welfare can be at

least as striking as those of the past two hundred

years.

2

In the United States, scientific and engineering

research has a significant impact on the products and

processes that fuel U.S. economic growth and

productivity.3 There is also ample recognition of the

significant role played by the Federal Government in

legitimizing and financing research as a public

good.4 (This is epitomized by the case of superconductivity, see box 2-A.) Such findings are reassuring

that, in the words of science policy statesman

1’ ‘NAS AIUNMI M@ting: Kudos From George BUSIL Challenges From Frank Ress, ’ NewsReport of the National Research Council, vol. 40, June

1990, p. 8.

~athan Rosenberg and L.E. Birdzell, Jr., “Science, Technology and the Western Miracle, ” Scient@”c American, vol. 263, No. 5, November 1990,

p. 54.

sRepofig tie res~ts of a new emp~c~ investigation economist EdtiMansfield fiuds: “. . . that about one-tenth of the new products and

pmcessescommercialized during 1975-85 in. . . [seven] industries could not have been developed (without substantird delay) without recent academic

research, The average time lag between the conclusion of the relevant academic research and the fwst commercial introduction of the innovations based

on this research was about seven years. . . . A very tentative estimate of the social rate of return ffom academic research during 1975-78 is 28 percen~

a figure that is based on crude (but seemingly conservative) calculations and that is presented only for exploratory and discussion purposes. It is important

that this figure be treated with proper caution. . . . Our results . . . indicate that, without recent academic research, there would have been a substantial

reduction in social benefits.” See Edwin Mansfield, “The Social Rate of Return From Academic Research” Research Policy, forthcoming 1991.

Another analysis, using different measures, supplemen~ Mansfield’s finding. While knowledge is found to be a major contributor to productivity

growth, there is roughly a 20-year lag between the appearance of research in the academic community and its effect on productivity as measured by

industry-absorbed knowledge. See James D. Adams,‘‘Fundamental Stocks of Knowledge and Productivity Growth,” Journal of Political Economy,

vol. 98, No. 4, 1990, pp. 673-702. Of course, during the 20-year gestation period, much applied research and development must occur before the effects

on industrial productivity are realized. Economists fmd Mansfield’s empirical approach the most direct evidence of economic returns to date. Summary

of reactions at American Economic Association and National Science Foundation seminam in 1989 and 1990 provided by honard hale- personal

communicatio~ January 1991. For a discussion of measurement techniques, see ch. 8.

atid~d, the U.S. rese~ch system is desi~ed so that returns on Federal investment will accrue to the pfivate SeCtor and Other f@iOm. me res~ts

of publicly funded research are for the most part openly disseminated with little or no copyright protection or patent exclusivity. For how this situation

is changing, see U.S. Congress, Office of Technology Assessment, Intellectual Propeny Rights in an Age ofElectronics and information, O’IA-CIT-302

(Washington DC: U.S. Government Printing OffIce, April 1986). Also see Congressional Budget Office, ‘‘Federal Investment in Intangible Assets:

Research and Development,” unpublished documen~ Februaty 1991.

-49 -
50 ● Federally Funded Research: Decisions for a Decade

Box 2-A—History of Superconductivity: Scientific Progress Then and Now

The history of superconductivity illustrates the episodic nature of progress in scientific research and the

limitations of predictions for scientific advancement in a specific research area. Due to resistance, normal

conductors will lose energy in the form of light or heat when a current is passed through them. While this is not

a wholly undesirable effect (e.g., in heaters and light bulbs), in most electric applications, resistance wastes energy.

Successfully harnessing the resistance-free currents of superconductors could be revolutionary: energy could be

transmitted with perfect efficiency; electronic devices could be made faster and smaller; and the power of

superconducting magnets (many of which are much stronger than traditional electromagnets) could transform

traditional transportation methods both on land and at sea.1 The first superconductor was discovered in 1911 by

Kammerlingh Onnes, a Dutch scientist. Using liquid helium, Onnes cooled mercury to 4 degrees Kelvin (K) above

absolute zero,2 at which point an electric current flowing through the mercury suddenly lost all resistance (for a

chronology of subsequent progress, see figure 2A-l).

The Science of Superconducting Materials

Limitations on the physical properties required

for a material to superconduct have hindered widespread applications. For every superconducting material there is a threshold for its physical properties

(temperature, magnetic field level, and current density)

above which it will not superconduct, By the 1950s,

researchers had discovered many materials that would

superconduct, but at temperatures no higher than about

20 K.

The 1950s brought two separate breakthroughs

that moved superconductivity closer to applicability,

First, researchers in the Soviet Union discovered a new

class of superconductors that would remain superconducting in high magnetic fields, and that could

eventually be used in superconducting magnets. Second, in 1957, the American researc h team of Bardeen,

Cooper, and Schreiffer received the Nobel prize and

recognition for a theory explaining superconductivity.

From the 1950s to the early 1980s, progress

toward higher temperature superconductors was slow.

Then, a surprising breakthrough occurred in late 1986

that transformed superconductivity research and drew

widespread public attention. In Zurich, the IBM

research team of Bednorz and Mueller discovered a

new ceramic material that remained superconducting

at temperatures as high as 35 K. A few months later in

1987, a research team at the University of Houston

developed a similar ceramic material that could

superconduct at 92 K. Not only did these discoveries

provide the long-awaited ability to use liquid nitrogen

instead of helium as a coolant, the discoveries were

made at such an’ incredible pace, considering the

history of superconductivity research, that the goal of

room-temperature superconductivity (at roughly 300

K) suddenly appeared to be within reach.

Figure 2A-l-Superconducting Critical Transition

Temperature v. Year

I

250

TIBaCaCuO

BiCaSrCuO

YBa 2Cu30 7

l

For a HCOUI-VC ddpti~ of @k8tkMM for mlpcmmdnctivity * Us. -, Omcc of IkCImology AwSanw@

High-T~ature Superconciuctivhy in Perspective, 0TA4M40 (WadngtoQ DC: us. CkwCmmMnt -~wApril 1990).

2One &#ec Kelvin (K) is oqtud to OXE de&w &kdus (~, except that Kehrn is mcaaurcd fium ablate zero (-273 W). Room

tunPcmWc (about 75 ‘F, or 25 W) k about 300 K.
Chapter 2—The Value of Science and the Changing Research Economy ● 51

The Federal Response

The response to these discoveries was enormous.

The popular press lauded high-temperature superconductivity as ‘‘. . . the startling breakthrough that

could change our world. Scientific meetings where

superconductivity results were rumored to be released

became standing-room-only events.4 While the temperature barrier still frustrates researchers, work continues in other areas that are key to useful applications

of superconductivity, like current densities and magnetic fields. Success has been attained in many areas,

but much more research needs to be done.

Fortunately, the Federal Government has maintained its commitment: in 1987, President Reagan

presented an 11-point agenda to increase superconductivity research and development (R&D) in the United

States, and in 1988, Congress enacted several laws

pe rtaining to superconductivity R&D, mostly aimed at

spurrin g commercial development of superconducting

technologies. The Federal superconductivity budget

rose from $85 million in fiscal year 1987 to $228

million in fiscal year 1990, with most of the increase

going to high-temperature researches Funding is spread

among several different agencies, primarily the Departments of Defense (DOD), Energy, and Commerce,

the National Science Foundation, and the National

Aeronautics and Space Administration. Programs at

different Federal agencies have aided scientists in the

exchange of research information.6

Photo credit: U.S. Department of Energy

A magnet is levitated by high-temperature

superconducting materials that are cooled in liquid

nitrogen. Superconducting materials may eventually

levitate much larger bodies, such as magnetically levitated

trains. Superconductivity is a research area that may yield

many fruitful applications.

Congress has made several attempts to coordinate superconductivity research. Part of the 1988 Omnibus Trade

and Competitiveness Act created the National Commission on Superconductivity (NCS). The Trade Act also

mandated an increase in staff for the National Critical Materials Council (NCMC). Finally, the National

Superconductivity and Competitiveness Act of 1988 called for cooperation among the Office of Science and

Technology Policy (OSTP), NCMC, and NCS in order to produce a 5-year National Action Plan for

Superconductivity to be accompanied by annual reports. The success of these initiatives has been limited. The 5-year

National Action Plan was published in December of 1989, but the formation of NCS was delayed. Although the

plan itself acknowledged the need for better Federal coordination, it lacked both the budget recommendations and

the long-term perspective Congress had requested.7 In addition, OSTP’s Federal Coordinating Council on Science,

Engineering, and Technology Committee on Superconductivity report of March 1989 did little more than assemble

agency superconductivity budget data and list programs in the agencies. 8

Questions remain, such as whether DOD funds too high a percentage of superconductivity research and

whether the Federal laboratories are doing too much of the research relative to other performers. Progress in the

development of high-temperature superconductivity is likely to unfold slowly—with substantial assistance from the

Federal Government.

3

Michael D. Time, 

4

Phil Adamsak “A Super Year in Science, ” Visions, fall 1987, p. 20. 5

Office of Technology As sessment op. cit., footnote 1, p. 63. 6

The Arnes laboratory distributes the “High-Tc Update,” a widely read the national laboratories have broadcast nationally

several and the Department of Energy has established a computer database that shares 

 with industry. The National Aeronautics and Space also maintains a Space Systems Committee, a

group with representatives from industry, universities, government 7

Office of Technology Assessment, op. cit., footnote 1, p. 63.

8

Ibid., p. 69.
52 ● Federally Funded Research: Decisions for a Decade

Harvey Brooks: “A strong basic science is a

necessary condition for a strong economy, a livable

environment, and a tolerable society. ’

Survey results indicate that since the mid-1970s

public confidence in the scientific community

ranked second only to medicine and ahead of 11

other social institutions, including education, the

press, and Congress.6 Furthermore, the expectations

of the American public about science and technology during the next 25 years include cures for cancer

and AIDS, safe long-term storage or disposal of

wastes from nuclear powerplants, establishment of

a colony on the Moon, and development of genetically engineered bacteria to destroy toxic chemicals.

But among the same sample of adults, realism about

the possible negative consequences of science and

technology is clearly evident. More than two in five

respondents considered another Three Mile Islandtype accident and the accidental release of a toxic

chemical that results in numerous deaths of AmeriCans ‘‘very likely. ’ Finally, when asked their

preference for problems that should receive more

Federal funding, three of four Americans responded

“helping older people,” “improving education,”

and “reducing pollution, ” two of three noted

“improving health,” one in two favored ‘‘helping

low income people,” and one of three responded

‘‘scientiilc research’ (which was well ahead of

‘‘exploring space’ and ‘improving defense’ ‘).8 For

further discussion of Federal funding in the “public

interest, see box 2-B.

Taken together, the investments and expectations

of the Federal Government in research have contributed to a shining history of scientific advance in the

United States. Universities, Federal laboratories,

and industrial research centers have discovered

many new phenomena and developed theories and

techniques for their continued exploration and use.

In the 1990s, preserving quality in research, while

understanding changes in the political and economic

environment in which it has grown, will require

planning and adaptation by research sponsors and

performers alike.

Research Funding in the United States

Focusing on research (not development), as OTA

does in this report, reduces the scope, but not the

complexity of the Federal research system.9 The

Federal Government spent over $11 billion in fiscal

year 1990 on basic research and over $10 billion on

applied research. Research thus represents 1.8 percent of the total Federal budget (at $1.2 trillion). This

1.8 percent, or roughly $21 billion, is an abstraction

referred to as the “Federal research budget. ”10

Funding for research in the United States is led by

the Federal Government (47 percent of the national

total). Industry is a close second at 42 percent;

universities and colleges (the category that includes

State and local government funds) follow at 7

percent; nonprofit institutions and others fund the

remaining 4 percent. Industrial support of basic and

applied research has grown dramatically over the

5Harvey Brooks, “Can Science Survive in the Modem Age?” Science, vol. 174, Oct. 1, 1971, p, 29. Brooks goes onto caution that a strong basic

science is not a sujj%ient condition. For a recent postscrip~ see Harvey Brooks, “Can Science Survive in the Modem Age? A Revisit After Wenty

Years,” National Forum, vol. 71, No. 4, fall 1990, pp. 31-33.

me question asked was: “AS far as the people running these institutions are concerned, would you say you have a great deal of confidence, only

some conildence, or hardly any contldence at all in them?’ Since 1973, fkom 37 to 45 percent of the respondents indicated “. . . a great deal of

contldence.” See National Science Board, Science and Engineenng Indicators-1989, NSB 89-1 (Washingto~ DC: 1989), p. 172 and app. table 8-11.

~espondents in 1985 were asked: “Do you think it is very likely, possible but not too likely, or not at all likely that this result will occur in the next

25 years?” National Science Board, Science and Engineering Indicator~1987, NSB 87-1 (Washington DC: 1987), p. 150 and app. table 8-10,

8~erespondents were asked to tell, foreachproblem,“. . . if you think that the government is spending too little money on i~ about the right amount,

or too much. ” See National Science Board, op. cit., footnote 6, p. 174 and app. table 8-13. A sample of British respondents were asked the same question

in 1988. Improving health care and helping older people topped their list while 47 percent (v. 34 percent of the U.S. sample) expressed a desire for

increased government funding of scientitlc research.

~ empirical terms, “research” has changing referents in the report. Sometimes a measure refers to “academic” or “university” research other

times to ‘‘basic’ research. The reader is alerted to these different performers or activities as OTA reviews them and the sources of information used to

characterize scientific research.

lo~e resemch fiWes~ current dou~ estimates. See AIbert H. lkich et al., Congressional Action on Research and Development in the FY 1991

Budget (Washington DC: Americrm Association for the Advancement of Science, 1990). Other figures are computed from various sources cited in table

1-2.
Chapter 2—The Value of Science and the Changing Research Economy ● 53

Box 2-B-Public Interest in Science

At a time when U.S. society has embarked on more technological adventures than ever before, Americans

apparently understand less about science and technology than citizens in other western countries. But understanding

alone is not the issue; rather, it is the complex relationship among public understanding, public confidence in science

and technology, and the public interest.’ From the turn of the century through World War II, American technology

and science came into its own. New inventions for the benefit of consumers were talked about everywhere from the

Sears and Montgomery Ward catalogs to popular magazines; stories about the new invention, the telephone, were

plentiful; and even if not everyone understood the new technology, they had confidence in it. 2

Military technology, given its lasting impact on everyone’s lives during wartime, seemed easier to fathom

“back then. ” Soldiers understood how a gun worked; stories abound about how American GI’s were able to fix

things on the spot, using whatever spare parts they could lay their hands on. People thought they understood the

technology that surrounded them and that it was essentially beneficial.3

With the development of the atomic bomb (necessarily shrouded in secrecy) came the end of innocence. The

shattering of Hiroshima and Nagasaki was accompanied, for many, by a shattering of faith in science and technology

as forever benign and helpful. In ways that we have only now begun to understand, the image of destruction

associated with the atom bomb has affected all technology, certainly all technology associated with nuclear power

and nuclear waste, With Three Mile Island, Bhopal, the Challenger accident, and Chernobyl, this image of

destruction has become the paradigm, for many, of all science and technology. 4

The discovery of restriction enzymes that slice strands of DNA into separate pieces, and that DNA pieces from

different species will connect with each other, has given rise to the great hope of understanding and curing genetic

diseases. Yet it also has raised fears of somehow disturbing the natural universe, changing things that ought not be

tinkered with. To know more sometimes is to fear more: ‘‘unintended consequences “ is today a familiar refrain;

even good intentions have side effects.

The very advance of biological and medical knowledge itself leads to frustrations and contradictions, further

undermining confidence in science. If we can perform the miracle of organ transplants, why can we not cure multiple

sclerosis? If we can cure childhood leukemia, why not lung cancer? Science editor Daniel E. Koshland writes:

But as architects of change, we [scientists] have occasionally oversold the product, implying that it will bring

unmixed good, not acknowledging that a scientific advance is a Pandora’s box with detriments or abuses as well as

benefits. By confessing that we are not omnisc ient we may lose some awe and admiration, but we will gain in

understanding and rapport.5

What can the scientific community do? Despite some negative feeling about science, or some aspects of it, there

are indications that the public is more interested in it and more willing to make the effort to learn than they are given

credit for. Although 20 percent of college graduates earn science and engineering degrees, many more enter college

eager to learn science,6 The television program ‘‘NOVA’ which covers all aspects of science, is consistently among

the more highly watched programs on public television. And 95 daily newspapers across the Nation have weekly

l

This box is a@t~ from AI~ H, M&ow~ president Scientists’ Institute for Publk hlfomtiOrL who wrote it expredy for ~s Ow

report under the title “Public Understanding of Science’ For an overview of the relationship between public interest, understanding, and

cotidence, see Kenneth Prewitt, ‘ ‘The Public and Science Policy, ” Science, Technology, d Human Wues, vol. 7, spring 1982, pp. 5-14.

2One of the best descriptions of this phenomenon is to be found in Daniel J. Boorst@ The Americans: The Democratic E~erience (lJew

York NY: Vintage Books, 1974).

3

There is a diffem~e ~~wn ~ders~ding tie scientMc principles behind an invention or technology ~d bvtig a gemxd id= of ~w

the parts fit together or what sequence of events must occur to make the technology work.

4See Dq] E. Chub@ “progress, Culture, and the Cleavage of Scienee From s~iety.” Science, Technology, and Social Progress, S.L.

Goldman (cd.) (Bethleherq PA: Lehigh University Press, 1989), pp. 177-195; and “Is Knowledge a Dangerous Thing?’ The Economisf, vol.

318, Feb. 16, 1991, pp. 21-22.

5Daniel E. Kos~@ $ ‘To See Ourselves As Others SIX us, ’ Science, vol. 247, Jan. 5, 1990, p. 9. For a content analysis of how popular

magazines portrayed science in the fmt half of the 20th century, see Marcel C. LaFollene, Making Science Our Own.” Public Images of Science

1910-1955 (Chicago, IL University of Chicago Press, 1990),

6

u.s. ConF&.s, OffIce of Technology Assessment, Elementary and Secondary Education for Science and Engineen”ng, OTA-TM-sET41 (Washington DC: U.S. Government Printing Office, December 1988), ch. 1.

Continued on next page
54 ● Federally Funded Research: Decisions for a Decade

Box 2-B—Public Interest in Science-Continued

science sections which, according to their editors, are among the most highly regarded sections in the paper. This

represents a growth from 66 such sections in 1986 and 19 in 1984.7

The attitude in the scientific community has also changed. Fifteen years ago, most scientists avoided the

popular press. Now, many scientists and engineers relish being quoted. 8 Still, working to improve public

understanding is not rewarded in many ways within the scientific community; the time is taken from other pursuits,

and therefore can be costly to one’s career. 9

What mechanisms would encourage more involvement by scientists and engineers in raising public interest

in and understanding of science efforts? Congress might include required spending of a portion of research grants

on public understanding efforts, designating a fraction of each agency’s budget for an office devoted to help grantees

develop public understanding efforts, and giving awards to scientists who have made substantial contributions to

public understanding. 10 At a time when more and more of American life is rooted in science and technology, and

when the Nation’s economic well-being depends as never before on its understanding and utilization, the Federal

Government cannot be complacent about the public’s interest and confidence in science.ll

7 "Newspaper sc~~c Sections Still on the N*, ’ WPlscope, vol. 18, spring 1990, p. 1. As one scienee policy statesman writes: “I have

come to believe . . . that the way things will work out for American science is vexy much in the hands of communicators--+f scienee writers

and reporters. They are a brtxxt of science watchers, and the last thing in science’s interews is to patronize or condescend to them. ’ William D.

Carey, “Scientists and %ndtwxes: Regions of the Mix@” American Scientist, vol. 76, March-April 1988, p. 144. Also see Maurice Goldsmit4

The Science Critic (LondorL England: Routkdge & Kegan Paul+ 1986).

g~ additioq ~ ~lol%ist ~ro~y Neti puts ic “Dependent more on political choices *~ review, my Scientits in ~1980s

became convinced that scholarly communication was no longer sufficient to assure support for their costly enterprise, that national visibility

through the ma9s media was strategically csential. They greatly expanded efforts to work the medi% trying to shape the images conveyed. ’

Dmmthy Nelkin+ “Selling science,” Phym”cs Toalzy, November 1990, p. 45. Also see the special issue in which this article appears,

‘‘Communicating Physics to the Public,’ Physics Today, November 1990, pp. 23-56.

9SCC Neal E. Mler, The Scientist’s Respom”ti[ity for Public Info~”on (New Yok NY: scientists’ hMtitUte fOr ~bk ~o~tio~

Media Resource Semicq 1990); and John P. Donnelly, ‘‘Rese-archers Must Join Forces to Bolster Public Confidence and Funding SupporL”

The Scientist, vol. 4, No. 20, Oct. 15, 1990, p. 16.

lORwdmts for Swh ~tiviti~ include a l-percent set-aside in the budgets of the National Institutes of Health ~for ev~@on of

W resuu-ch and the annual “public Understanding of Science and T&%nology’ awards given to science journalists by the Americarl

Association for the Advancement of Sciencs and Westinghouse.

1 l~er public understamling of science will not necessarily lead to ~eater Federal funding of r~k As one commentator obsenmd

a generation ago: “AJthough there is no question that the public has demonstrated its willingnas to provide . . . suppofi I doubt whethez the

intrinsic cultural value could be used to justifjf to the pubIic or to politicians more than a smrdl tiction of the present support for basic seienee

in the United States, or indeed in any other major country of the world. ’ Haney Brook, “Are Scientists Obsolele?” Science, vol. 186, Nov.

8, 1974, p. 508.

last 20 years, especially in the early and mid- While questions of relative funding can be gauged

1980s. 11 For basic research alone, the Federal with funding data (e.g., comparisons between FedGovernment funds 62 percent of the total, followed eral and industrial support), it is not easy to compare

by industry (21 percent), universities and colleges expenditures in one year to those in another.

(12 percent), and nonprofit institutions and others Economic change affects the “value” of a dollar

(5 percent) .12 over time. Because some goods (foodstuffs, automo11The Mtioml R&D effort is funded p~“ y by the Federal Government, industry, and academic institutions. In 1990, industry and the Federal

Government together accounted for nearly 96 percent of total suppofi with universities and colleges contributing 3 percent, and other nonprofit

institutions funding 1 percent. Industry is the largest single source of R&D funds, providing $74 billi-m compared to the Federal Government’s $69

billioq and the past decade represents a period of great growth in industrird R&D spending. National Science Foundation, National Patterns of R&.D

Resources: 1990, NSF 90-316 (Washington DC: May 1990), table B-5.

lzFor &ese aggregate figllres, the National Science Foundation estimates of basic/applied/development breakdowns-despite some fUzziness in

labeling-are thought to be reliable. See ibid.
Chapter 2—The Value of Science and the Changing Research Economy ● 55

Photo reedit: Jay Mangum Photography

This research is part of an acid rain study in the Duke Forest Project, NC. Research can take many forms, from space exploration

to the study of microbes, and almost all are represented in the Federal research portfolio.

biles, housing) change slowly over time, economists Documenting Perspectives on the

have developed so-called constant dollars or ‘‘defla- Future of Research tors’ to use in comparing economic activity in two

or more years. Constant dollars work less well for

goods that change rapidly (e.g., computers, con- The American public holds scientific research in

sumer electronics, and defense technologies), and high esteem, but does not see it as the Nation’s top

not at all for products that, by definition, are priority. This contrasts with survey findings of the

dissimilar from one year to the next.13 The use of any late 1980s and 1990 reflecting the perceptions of

two deflators can also lead to very large differences, scientists and engineers. Biomedical researchers in

especially as the timeframe lengthens. Taking into

account these difficulties in the use of deflators for

comparing research funding over time, OTA has

chosen to use the ‘‘Gross National Product Implicit

Price Deflator. ” This deflator reflects changes in

total public and Federal expenditures. Thus, OTA’s

figures can be easily compared, as Congress routinely does, with trends in other public expenses.14

(Box 2-C discusses different deflators and their use

in interpreting trends in research funding.)

academia and industry, recombinant DNA researchers, young faculty researchers in physics, and a

cross-section of Sigma Xi (The Scientific Research

Society) members all report difficulty in establishing or sustaining research programs and fear reductions in Federal funding for individual-investigator

research (which they see as a top funding priority) .15

Perhaps the most forceful recent advocate of increased research funding is Nobel laureate physicist

 ‘ ‘product’ i.e., has measurable outputs. But the value of the output is not determined by market prices. It 

be more accurate perhaps to treat research as a ‘‘process, “ i.e., an activity or to the economy.

 is contained in the Economic Report of the President (Washington ent Printing Office, 1990).

 Gallup Losing Ground Biomedical

Research: The Shortage of American Scientists DC: February 1991); Isaac “The Impact of Activist Pressures on Recombinant

DNA Science, Technology, & Human vol. 16, No. 1, winter 1991, pp. 70-87; American Physical Society survey results reported 

 et al., ‘‘Their Most Productive Years: Young Physics Faculty in 1990,” Physics February 1991, pp. 37-42; and Political Economy

Research Institute, “Researcher Perspectives on the Federal Research System,” contractor July 1990 (available through the National

 Information Service, see app. F).
56 ● Federally Funded Research: Decisions for a Decade

Box 2-C--Calculating Constant Dollar Trends for Research

While seeming a trivial problem at first glance, calculating funding trends for research in constant dollars (i.e.,

units that have the same spending power in each year) can be full of pitfalls. Different methods can lead to quite

different trends and, therefore, policy conclusions. For example, the constant dollar values calculated using a

method developed at the Department of Commerce (and used by the National Science Foundation) imply that

research expenditures in the United States have grown by roughly 40 percent in the period 1%9 to 1990, Similar

calculations based on a method developed by the Office of Management and Budget (and used by the American

Association for the Advancement of Science) imply that research expenditures have grown by less than 15 percent

(see figure 2C-1).1

Figure 2C-1—Federal Research Spending in Constant Dollars Using Two Different Deflators:

Fiscal Years 1960-90 (In billions of 1982 dollars)

10

GNP deflator used by

5

\

!

I

NSF

1 1 , L ,

1960 1965 1970 1975 1980 1985 1990

KEY: R&D. Research and Development; OMB - Office of Management and Budget; AAAS. American Association for the Advancement of Science;

GNP . Gross National Product; NSF. National Science Foundation.

SOURCES: Current dollar data came from National Science Foundation, Federal Funds for Research and Development, Detailed Historical Tables:

Fiscal Years 1955-1990 (Washington, DC: 1990), table A; and National Science Foundation, Selected Data on Federal Funds for

Reseach and Development: Fiscal Year 1989, 1990 and 1991 (Washington, DC: December 1990), table 1. Deflator data came from the

Office of Management and Budget, Budget Analysis and Systems Division, unpublished data; and National Science Board, Science and

Engineering Indicators--1989, NSB 89-1 (Washington, DC: 1989), app. table 4-1.

So how does one calcuIate a constant dollar trend? The object is to translate dollars from one year to the next,

i.e., to find the price of a market basket of commodities. The deflator is the ratio of the purchasing power of a dollar

for a particular year to that of a reference year. A change in the index means that purchasing power has changed

with respect to the same market basket. This change can also be expressed as ‘‘constant dollars, such as ‘‘1982’

or 1988’ dollars. These ratios can then adjust any dollar amount for a given year to get a value in constant dollars.

A set of ratios or indices for a series of years is called a‘ ‘deflator. ” To calculate a deflator, a comparison must

be made between how much a specific thing costs in the year in question and in the constant dollar year. The

differences between methods used to calculate constant dollar trends depend on what goods or services are tracked

to make up the deflator. For instance, increasing salaries are very different from increasing (or decreasing) prices

of computers.

Congress is most interested in comparing research expenditures to other elements of the Federal budget. Thus,

a deflator that represents expenditures on products and services that are often bought throughout the United

l

Informal meeting on deflators, hosted by the American Association for the Advancement of Science, Dec. 5, 1990. OTA notes that the

National Institutes of Health uses its own deflator, called the Biomedical Research and Development Price Index, which is discussed in ch. 6.
Chapter 2—The Value of Science and the Changing Research Economy ● 57

States-a constant dollar in the most general sense-is often the most useful for congressional policy analysis.

Using the Gross National Product (GNP) Implicit Price Deflator developed by the Department of Commerce is

usually acceptable, since it employs a large market basket of goods to calculate its constant dollar ratios.2 Constant

dollar trends for research calculated with this deflator compare research expenditures to other expenditures

throughout the economy.

In other contexts, a deflator that specifies indices relating only to research (salaries, facilities, and

instrumentation) could be preferable, In such a deflator, if 45 percent of total expenditures for research goes to

salaries, 3 45 percent of the deflator would reflect the changes in these salaries. When other components of the

deflator are similarly adjusted-equipment, facilities, and indirect and other costs-a new index is derived. Use of

such an index to adjust total research expenditures would approximate how much scientists were spending in one

year as if the prices and contents of the market basket of goods and services were unchanged (i.e., the effect of

increasing salaries and cost of equipment and other items would have been removed).4 Deflators are difficult to

calculate for science and engineering research, because the items and mix of the market basket cart change rapidly

and they may be quite different in separate fields of inquiry. In addition, even a ‘‘correct’ deflator of this type can

be misleading because it only concerns inputs and not the changing character of research outputs, i.e., one is not

buying the same science and engineering “product.”

Given the problems with research-specific deflators and the advantage of a general GNP deflator to compare

expenditures across the economy, all constant dollar figures and tables in this report were calculated with the GNP

Implicit Price Deflator for 1982 dollars (unless noted otherwise). However, OTA does not make any specific policy

assumptions based exclusively on constant dollar trends.

2Economic Repoti ofrhe President, Transmitted to Congress February 1990 (W@@tU% w: U.S. @v~ent ~% ~lce> 1~)~

pp. 298-299, table C-3.

3s= ch. 6 of this Rpofl.

4N0 &flator has been created using this method. Bmce Bakcz, OfEee of Management fd Budget+ PCTSOIMJ communicatio~ Nov. 26

1990. But see a pair of working papers by John E. Jankowski, Jr., Nationat Seieuce Foundatio~” ‘Do We Need aPriee Index for Industrial R&D?’

n.d.; and ‘‘Construction of a Price Index for Industrial R&D Inputs, ” Aug. I, 1990. Among the approximah “Ons used is the Office of Management

and Budget noncapital Federal expenditures deflator developed to normahze“ tdl expenditures Of b Federal GOV~ that do not illvOh’c the

specific procuremrmt of large, capital ite nw--dviously a much larger set of expmditnrcs than those involved in nxearch. AS stated by Bruce

Baker, Ofilce of Management and Budget: “This is nor an R&D deflator, it is a deflator used to deflate R&D. ” American Association for the

Advan cement of Science, op. cit., footnote 1. The problem with the use of this deflators that even though it excludes many expenditures unrelated

to research, the expenditures that are reflected in the deflator are not guaranteed in any way to mimic rescareh expenses over time. Consequently

such a deflator may be just as ‘‘wrong’ as any other deflator to calculate researeh productivity.

Leon Lederman, who also relies on a survey of system and the place of U.S. science in the world has

active researchers in major universities (see ‘box

2-D).

Such surveys can take the pulse of a population,

tapping respondents’ perceptions, experiences, and

feelings. Other data, however, must be assembled

and analyzed to provide a more systematic, wellrounded characterization of the state of affairs-and

general health---of the Federal research system. That

is OTA’s objective in this report.

remained strong. Other countries support research

infrastructures at the forefront of many fields—

which is expected in an internationally competitive

economy—but U.S. science still ranks at or near the

top in most fields. This is a testament to the strength

and scale of federally funded research. l6

This system will face many challenges in the

1990s, including living with tight fiscal conditions.

In the 1980s, four categories of Federal spending

Although scientists may now feel engulfed by the consistently increased in constant dollars: defense,

stress of research competition, the Federal research entitlements (Social Security, Federal retirement,

1 6There is evidence that the United States is a latecomer to the stresses beleaguering other mtions. See Susan E. Coz..zens et d. (edO The Research

System in Transition, proceedings of a NATO Advanced Study Institute, 11 Ciocco, Italy, Oct. 1-13, 1989 (Dordrecht, Holland: Kluwer, 1990). The

question of whether the United States is ‘‘losing ground” to other mtions very much depends on which fields or research areas are of concerq and which

indicators of research productivity one chooses to embrace. For evidence to the contrary, see Gina Kolata, “Who’s No. 1 in Science? Footnotes Say U.S.’

New York Times, Feb. 12, 1991, pp. Cl, C9; and “No Slippage Yet Seen in Strength of U.S. Science, ” Science Watch, vol. 2, No. 1, January/Februrq

1991, pp. 1-2.
58 ● Federally Funded Research: Decisions for a Decade

Box 2-D—An Interpretation of Researchers’ Distress by Leon M. Lederman

On January 7, 1991, Leon M. Lederman, Nobel laureate physicist and President-Elect of the American

Association for the Advancement of Science (AA/M), sounded “a cry of alarm” for academic science. He released

a report to the AAAS membership expressing concern ‘‘. . . far the future of science m the United States and for

the profound cultural and economic benefits that science brings. ”l The following are excerpts from the report,

which was based on an informal survey of natural sciences faculty in 50 U.S. universities, including the top 30

institutions in Federal R&D funds received. The survey yielded letters from 250 scientists. The text below is an

excerpt from Lederman’s report and is followed by a postscript written by him expressly for this OTA report. 2

... The responses paint a picture of an academic research community beset by flagging morale, diminishing

expectations, and constricting horlzons. . . 

(There were) three incidents where we had to stand by while competitors from abroad moved forward on research based

on our ideas. ., . The history of the past decade is one of continued harasssment over roomy, lost opportunities due to

inadequate support, and a stifling of Imagination due to money worries If U.S. scientists must continue to stand by and watch

as our best ideas are carried f orward by groups from abroad, our nation cannot hope to escape a rapid decline.

-Professor of Physics,

Massachusetts Institute of Technology

. Academic science has not arrved at its present state through a conscious decision by the Administration or

Congress, No political leader has advocated starving science-- indeed, most feel that they support it strongly,

Presidents Reagan and Bush have both promised to double the size of the National Science Foundation’s budget

within five years, and Congress, almost every year, appropriates more for the National Institutes of Health than the

Administration requests, . . .

However, recent growth has been insufficient to compensate for the effects of the long drought that preceded

it, Thus, In the view of those in the Iaboratorles, there has been a gradual year-by-year erosion in the availability of

funding and h the health of academic science over nearly two decades, . . .

I suspect that if I were twenty years younger I would not choose an academic research career. Even now I find myself

considering other options. I’m Wed of writing ‘excellent’ proposals that aren’t funded.

-Professor of Chemistry,

Duke University

The (funding) problem is compounded , . . by a number of other factors that, taken together, further restrict the

results that can be obtained from each research dollar, One factor Is complexity--or what same observers have

called “sophisticated inflation, ” As our understanding of nature Increases, the questions we need to answer

became more complex. There is a corresponding Increase in the sophistication (and cost) of the equipment

needed to do research, both for small, “table tap” experiments and large facilities such as telescopes and

accelerators .... The cost of regulation is a second factor. In many fields, particularly in the Iife sciences, increased

regulation absorbs significant funds and research time. . . . A third factor is institutional overhead. According to the

National Science Foundation, indirect costs at universities (including admlnistration, maintenance of buildings,

utilities, etc.) have risen from 16 percent of the national academic R&D budget In 1966 to about 28 percent in

1986... .. . (and this) means that less money is available to the Iaboratory scientist for the direct costs of research. . . .

The problem is more serious than average grant size or proposal success rates (at the National Science

Foundation and the National Institute of Health), however. The letters reveal potentially important changes in the

way scientists as individuals pursue their craft. As a consequence of the increasingly difficult search for funding,

academic scientists are less willing to take chances on high risk areas with potentially big payoffs, Instead, they

prefer to play it safe, sticking to research in which an end product is assured, or worse, working in fields that they

believe are favored by funding agency officials, These scientists are also increasingly viewing their fellows as

cornpetitors, rather than colleagues, leading to an increasingly corrosive atmosphere. The manifestations of this

attitude range from a reluctance to share new results with other scientists to public bickering about relatlve priorities

in funding different fields,

We are tending to do "safer" projects, avoiding the high risk, but high payoff projects. In the present climate we cannot

afford to have experiments not work .. . . Undergr aduates, graduate students and postdocs continually ask about the benefits

of pursuing an academic career when funding is so tight.

—Assistant Professor of Biology,

Carnegie-Mellon University

1Science: The End of the Frontier? a report from Leon M. Lederman, president-elect, to the Board of Directors of the American

Association for the Advancement of Science (Washington, DC; American Association for the Advancement of Science, January 1991).

2OTA does not necessarily agree with the conclusions either m the report or the postscript.
Chapter 2—The Value of Science and the Changing Research Economy ● 5 9

...(in addition) respondents reported that they are cuttinng back on the number of students they are trainlng, and

that students now In the laboratories are opting out of research careers.

While the current loss of productive groups is serlous, even more disturbing is the negative influence the present difficulties

are having on the next generation. On a recent visit to MIT I had an informal lunch with about twenty graduate students i n

organic chemistry and asked how many of them were going into academic science. One person raised his hand and he was

returning to a small Iiberal arts college where he hod been a student. This group agreed that thelr lack of interest in unlversity

level positions is thelr perception that the challenge of gaining funding is now dominant over the challenge of the science.

-Professor of Chemistry,

University of Illinois

What would It take to relieve the acute problems In academic research and restore U.S. science to its pre-1968

excellence? Let us consider this question Independently of “practical" constraints dictated by current events. M y

analysis.... indicates that we should be spending at least twice as much as we were In 1968 (In constant dollars)

if we are to approach the conditions of [this era). Indications from NSF, NIH and DOE tend to confirm the pressure

for a doubling of the current Ievel of funding for academic science, which amounts to about $10 billion a year This

huge sum could, I believe, be effectively deployed In two or three fiscal years.

Beyond this, in future years, I would argue that the growth of four percent per year In the number of academic

scientists and the complexity factor growth estimate of five percent per year imply that a sustained flourishing of

academic research requires annual real growth of eight to ten percent.... Such an Increment may sound

substantial in our current climate, but as the economy responds, academic research would remain only a tiny

fraction of total federal spending for many decades. Furthermore, even with such Increases, it would be a decade

or two before our level of nondefense research expenditure proportional to GNP would equal the 1989 levels of

Japan or West Germany.

February 1991 Postscript

In his budget for FY 1992, the President requested significant increases for science, averaging 5-10 percent

above inflation. In view of the fiscal constraints, scientists must stand In awe at the respect their work has earned.

This is the eighth year of real increases initiated by the Administration and passed by Congress. Nevertheless, the

AAAS Inquiry has dramatically confirmed indications of serious troubles at the Iaboratory bench.

There are several reasons for believing that, In spite of these Increases, the Nation is seriously underinvesting in

research. One is the comparison with what our economic competitors are doing. Another is the comparison of our

relative research capability today with what It was in the late 1960s.

International prizes (identifying when the work was done) as well as patents and a hard-to-quantify loss of

scientific and technological self-confidence paint in the same direction, The unprecedented stress within the

scientific community described above is another indicator.

The crisis documented in the AAAS survey must be viewed as part of a larger pattern of national decisions. My

analysis indicates that a continuation of the kind of investment we were making in the 1960s would have brought

us today to somewhere near $30-40 billion for academic research. This is what motivated the “unrealistic” proposal

for a doubling of the budget with subsequent 8-10 percent annual Increases for at least a decade,

We are keenly aware that we have concentrated on only one important element of a problem that must

include many other components, such os non-military R&D in industry and the national Iaboratories, and the overall

scientific literacy of the work force, Research and education are so intimately entwined that they must be treated

together. Only very briefly mentloned in the report are the human resources devoted to what economist Robert

Reich calls “strategic brokers,” those who translate R&D results into economic products. The record of U.S.

investment in research and education, even given the increases, is one of decline relative to the GNP and relative

to other industrialized societies. Whereas it is surely true that sums allocated by the Federal Government could

always be spent more efficlently (especially in education), the problem is clearly underinvestment. Yet the primary

asset of a modern industrial nation in the 21st century is its brainpower: a skilled, educated workforce.

The vision to recognize this os a salient feature of our times resides in many of our leaders. No doubt some such

perception explains the favoring of science in tough times. However, the resources that are really demanded are

far greater, as has been “unrealistically’ proposed In the AAAS report. Nevertheless, if these human capital

investments are judged in the context of a $5 trillion GNP or a $1.4 trillion Federal budget. It becomes clear that the

issue isn’t cost---it iS a matter of choice, The choice is to treat the human resources of the Nation------an educated,

capable work force--as the key to a successful society. If we choose wisely, and I let my imagination soar, the

expenditure for academlc scientific research will one day reach $50-100 billion (In 1991 dollars). With comm ensurate

Investment in education and infrastructu re, we can restore not the world Ieadershlp we once enjoyed, but the

position of the Nation as a dynamic and resourceful society, a leading participant in the new global economy of

the 21st century. If we fail to see this long term issue, if we are dominated by our "third quarter” crises, if we hesitate

because we have lost faith In the power of the human mind, our long term prospects wiII be dismal Indeed.

292-863 0 - 91 - 3
Chapter 2—The Value of Science and the Changing Research Economy ● 5 9

...(in addition) respondents reported that they are cuttinng back on the number of students they are trainlng, and

that students now In the laboratories are opting out of research careers.

While the current loss of productive groups is serlous, even more disturbing is the negative influence the present difficulties

are having on the next generation. On a recent visit to MIT I had an informal lunch with about twenty graduate students i n

organic chemistry and asked how many of them were going into academic science. One person raised his hand and he was

returning to a small Iiberal arts college where he hod been a student. This group agreed that thelr lack of interest in unlversity

level positions is thelr perception that the challenge of gaining funding is now dominant over the challenge of the science.

-Professor of Chemistry,

University of Illinois

What would It take to relieve the acute problems In academic research and restore U.S. science to its pre-1968

excellence? Let us consider this question Independently of “practical" constraints dictated by current events. M y

analysis.... indicates that we should be spending at least twice as much as we were In 1968 (In constant dollars)

if we are to approach the conditions of [this era). Indications from NSF, NIH and DOE tend to confirm the pressure

for a doubling of the current Ievel of funding for academic science, which amounts to about $10 billion a year This

huge sum could, I believe, be effectively deployed In two or three fiscal years.

Beyond this, in future years, I would argue that the growth of four percent per year In the number of academic

scientists and the complexity factor growth estimate of five percent per year imply that a sustained flourishing of

academic research requires annual real growth of eight to ten percent.... Such an Increment may sound

substantial in our current climate, but as the economy responds, academic research would remain only a tiny

fraction of total federal spending for many decades. Furthermore, even with such Increases, it would be a decade

or two before our level of nondefense research expenditure proportional to GNP would equal the 1989 levels of

Japan or West Germany.

February 1991 Postscript

In his budget for FY 1992, the President requested significant increases for science, averaging 5-10 percent

above inflation. In view of the fiscal constraints, scientists must stand In awe at the respect their work has earned.

This is the eighth year of real increases initiated by the Administration and passed by Congress. Nevertheless, the

AAAS Inquiry has dramatically confirmed indications of serious troubles at the Iaboratory bench.

There are several reasons for believing that, In spite of these Increases, the Nation is seriously underinvesting in

research. One is the comparison with what our economic competitors are doing. Another is the comparison of our

relative research capability today with what It was in the late 1960s.

International prizes (identifying when the work was done) as well as patents and a hard-to-quantify loss of

scientific and technological self-confidence paint in the same direction, The unprecedented stress within the

scientific community described above is another indicator.

The crisis documented in the AAAS survey must be viewed as part of a larger pattern of national decisions. My

analysis indicates that a continuation of the kind of investment we were making in the 1960s would have brought

us today to somewhere near $30-40 billion for academic research. This is what motivated the “unrealistic” proposal

for a doubling of the budget with subsequent 8-10 percent annual Increases for at least a decade,

We are keenly aware that we have concentrated on only one important element of a problem that must

include many other components, such os non-military R&D in industry and the national Iaboratories, and the overall

scientific literacy of the work force, Research and education are so intimately entwined that they must be treated

together. Only very briefly mentloned in the report are the human resources devoted to what economist Robert

Reich calls “strategic brokers,” those who translate R&D results into economic products. The record of U.S.

investment in research and education, even given the increases, is one of decline relative to the GNP and relative

to other industrialized societies. Whereas it is surely true that sums allocated by the Federal Government could

always be spent more efficlently (especially in education), the problem is clearly underinvestment. Yet the primary

asset of a modern industrial nation in the 21st century is its brainpower: a skilled, educated workforce.

The vision to recognize this os a salient feature of our times resides in many of our leaders. No doubt some such

perception explains the favoring of science in tough times. However, the resources that are really demanded are

far greater, as has been “unrealistically’ proposed In the AAAS report. Nevertheless, if these human capital

investments are judged in the context of a $5 trillion GNP or a $1.4 trillion Federal budget. It becomes clear that the

issue isn’t cost---it iS a matter of choice, The choice is to treat the human resources of the Nation------an educated,

capable work force--as the key to a successful society. If we choose wisely, and I let my imagination soar, the

expenditure for academlc scientific research will one day reach $50-100 billion (In 1991 dollars). With comm ensurate

Investment in education and infrastructu re, we can restore not the world Ieadershlp we once enjoyed, but the

position of the Nation as a dynamic and resourceful society, a leading participant in the new global economy of

the 21st century. If we fail to see this long term issue, if we are dominated by our "third quarter” crises, if we hesitate

because we have lost faith In the power of the human mind, our long term prospects wiII be dismal Indeed.

292-863 0 - 91 - 3
60 ● Federally Funded Research: Decisions for a Decade

Medicare, and Medicaid), net interest on the Federal

debt, and Federal spending on research.17 While the

deficit continues at record levels, the Omnibus

Budget Reconciliation Act of 1990 will temper

Federal spending, including possible modifications

and further priority setting in expenditures for

research. 18

In addition, the scientific community has grown in

size since the 1960s, reflecting a rising research

economy that supported the pursuit of many spectacular opportunities. However, as more knowledge is

gained, expenditures for cutting-edge research have

also increased. These factors have combined to

magnify the burdens on research performers and

institutions, and on the Federal sponsors that fund

them.19 Many in the research system also wonder, as

the uncertainty increases over enrollments by U.S.

students in science, whether the next generation of

scientists and engineers will sustain the research

enterprise. 20 The pressures mount on public policy

to decide which opportunities are most urgent,

which agency programs to favor, and the rationale

for supporting a diversity of fields, sectors, and

research personnel. In the words of Yale Medical

School Dean Leon Rosenberg:

The scientific community is responsible in a

major way for the paradoxes and dilemmas in which

we find ourselves. . . . There are more opportunities

than ever to ferret out the secrets of human biology

and apply those secrets to the reduction of human

suffering. The dilemma is that we must obtain more

funding for the support of this effort in order to

capitalize on those opportunities and improve the

morale of the scientific community, while at the

same time acknowledging that we have been generously supported for the past 40 years.21

This report explores the ‘paradoxes and dilemmas’

of supporting U.S. science in the 1990s, while this

chapter introduces the history of the Federal research

system and current challenges that demand Federal

policy attention.

Historical and Current Federal Roles

in the Research System

The Federal research system has many participants. They include Congress, the Federal research

agencies, the Office of Management and Budget

(OMB), the Office of Science and Technology

Policy (OSTP), academic research institutions, Federal and industrial laboratories, the National Academy of Sciences complex, professional societies,

think tanks, and others.22 Together these components sponsor, per-form, and guide the activity called

‘‘research.

Recognition of the role of the Federal Government in the support of research grew during the early

parts of the 20th century, especially before and

immediately after World War II. During the 1930s

and 1940s, the Departments of Defense (DOD) and

Agriculture (USDA), the Public Health Service

(largely through the National Institutes of Health,

NIH), and the Atomic Energy Commission (then, the

Energy Research and Development Administration,

and now the Department of Energy, DOE) collectively funded a diverse Federal research portfolio.23

In the 1950s, the National Aeronautics and Space

Administration (NASA) began to sponsor space

exploration projects, and in the 1960s, it launched a

celebrated and successful effort to safely land

humans on the Moon and to gather data on the solar

176 ‘Ouflays by Category, “ Government Executive, vol. 22, September 1990, p. 44.

1

8 See Jeffrey Mervis, ‘‘Science Budget: A Zero-Sum Game,’The Scientist, vol. 4, No. 24, Dec. 10, 1990, pp. 1,6; and David C. Morrison, ‘‘Pinching

the Research Budgeq” National Journal, vol. 22, No. 49, Dec. 8, 1990, p. 2996.

lgseewi~~ D. caey, “R&D ~ tie Fe&rd Budget: 1976- 1990,” and Rodney W. Nichols, “Mae West at Olympus: Five Puzzles for R&D,” both

in Science and Technology and the Changing World Order, colloquium proceedings, Apr. 12-13, 1990, S.D. Sauer (cd.) (Washingto~ DC: American

Association for the Advancement of Science, 1990), pp. 43-51, 53-69.

me gap between current rhetoric and current problems in science education as they relate to the Nation’s research capability is examined in Iris

Rotberg, ‘‘I Never Promised You First Place,’ Phi Delta Kappan, vol. 72, December 1990, pp. 296-303.

ZIQuoted in Dick Thompson, “The Growing Crisis in Medical Science,” Time, Dec. 17, 1990, p. 21.

zzB=ause ~vemitiesp~om tie Prepondmace of basic ad appli~ rese~h and train most of tie res~ch work force, md because much Of the

data on research performance has been collected on academia, this report often focuses on academic research performers. However, when relevan~ and

especially where data are available, other performers are discussed.

~See Margaret W. Rossiter, ‘‘Science and Public Policy Since World War II, ’ Historical Writing on American Science: Perspectives andProspects,

S.G. Kohlstedt and M.W. Rossiter (eds.) (Baltimore, MD: Jolms Hopkins University Press, 1986), pp. 273-294; and Julius H. Comroe, Jr.,

RetroSpectroScope: Insights Into Medical Discovery (Menlo Park CA: Von Gehr Press, 1977).
Chapter 2—The Value of Science and the Changing Research Economy ● 61

system. Federal research was supported and selected

in partnership with the scientific community and

with little constraint to adhere to formal agency

missions.24

For many years, the core of the national effort in

science was increasingly understood to reside in and

be expressed through the National Science Foundation (NSF) .25 A 1965 National Academy of Sciences

report, Basic Research and National Goals, went so

far as to state that:

. . . the National Science Foundation is viewed

. . . as being responsible for. . . “intrinsic basic

science,’ the motives for which are relatively remote

from politically defined missions. Since this is a

social overhead whose connection with specific

applied objectives of the society is distant and

undefined, it would seem. . . that allocation of

resources to this activity would be even more

difficult than the allocation to mission-related research.26

Since NSF primarily funded research in universities,

science policy was generally equated with the

provision of resources for research, principally

through the university-based research system.

Although DOD, NASA, DOE, and USDA had

significant basic and applied research budgets in the

1960s and 1970s, and NIH finding soared with the

War on Cancer in the early 1970s, it was not until the

1980s that infusions in defense research and development (R&D) and the debates over the importance

of federally sponsored applied research once again

highlighted the pluralistic Federal role.27 “The

fragmented, mission-oriented structure that emerged

after World War II went a long way toward realizing

Vannevar Bush’s vision of a Federal system for the

support of science and engineering. In large measure, it was responsible for the emergence of the great

American research universities and the ‘golden age’

of science. ’28 Today, research is understood to be an

activity pursued in many agencies of the Federal

Government and sectors of the U.S. economy .29

The wisdom of the compact between science and

the Federal Government has been demonstrated

repeatedly in the last half of the 20th century. As

more and more has been explicitly demanded of

scientific and technological institutions in U.S.

~See U.S. Con=ess, House COmmittw on Science, Space, and Technology, Task Force on Science Policy, A History of Science poli~ in’ the Unitd

States, 1940-1985, 99th Cong. (Washington DC: U.S. Governrnent Printing Office, 1986), especially pp. 1540; also see Alan T. Wate- “Basic

Research in the United States,” Symposium on Basic Research, Dael Wolfle (cd.) (Washington DC: American Association for the Advancement of

Science, 1959), pp. 17-40. The celebrated Mansfield amendment, passed as part of the fiscal year 1970 Military Authorization Act (Public Law 91-121),

prohibited military funding of research that lacked a direct or apparent relationship to a specitic military function. Through subsequent modiilcatiom

the Mansfield amendment moved the Department of Defense toward the support of more short-term applied research in universities. For a discussion

see Genevieve J. Knezo, “Defense Basic Research Priorities: Funding and Policy Issues,’ CRS Report for Congress (Washington DC: Congressional

Research Service, Oct. 24, 1990), pp. 5-9.

fiwhile the BushReportand~e WMmanllepcmt(intr ochud inch. 1) were both effusive intheirpraiseof the WcialbenefitS emanntingfrom scientific

advance and the underlying rationale for the Federal support of science, each took a different approach to the administration of a mtional science

foundation. OTA points out that “. .. the Steehnan report regarded science as a special interest, Although large-scale government support for science

was a new phenomenon, science was not considered to be suftlciently different ftom other policy areas to warrant any special political relationships. ”

Bush supporters were”. . . convinced that science was distinct from other types of government programs, that it must be free from political control, and

that to be successful, scientists should be able to direct their own affairs. . . . Scientists, . . . through advisory groups and a system of review by scientific

peers, would decide how research should be conducted and would influence the research agenda.” See U.S. Congress, Office of Rchnology Assessment

The Regulatory Environmentfor Science, OTA-TM-SET-34 (Springlleld, VA: National lkchnical Information Sewice, February 1986), pp. 15-16.

~eorge Be Kistiakows~, “Summary,’ in National Academy of Sciences, Committee on Science and Public Policy, Basic Research andNational

Goals, A Report to the Committee on Science and Aslromutics, U.S. House of Representatives (Washington DC: March 1965), p. 11. This collection

of essays evolved, in the words of Committee ChairmanGeorge P. Miller, into “. . . the production of a comprehensive study designed to throw into

bold relief some of the more serious phases of policy which Government must consider in its decisions to support or otherwise foster research in

America.” (p. v).

zTFrom the msearch~’s pers~ctive, multiple sources of Federal support provide funding flexibility, i.e., choice among agencies. From a Fede~

perspective, flexibility allows choice among alternative research initiatives and performers. New programs can be started or old ones refocused.

‘Joseph G. Morone, ‘‘Federal R&D Structure: The Need for Change,’ The Bridge, vol. 19, fall 1989, pp. 3-13. For a discussion of the “university

research economy, ” see Roger L. Geiger, “The American University and Researcm” in Government-University-Industry Research Roundtable, The

Academic Research Enterprise Within the Industrialized Nations: Comparative Perspectives, report of a symposium (Washington, DC: National

Academy Press, March 1990), pp. 15-35.

?f’he importance of nonprofit foundations and the private sector in supporting, defining, and utilizing basic research is also indisputable (though the

extent of their participation differs greatly by field, industry, and measures of contribution). See National Science Foun&tio~ op. cit., footnote 11.
62 ● Federally Funded Research: Decisions for a Decade

society, the social contract has changed.30 A new

relationship may be evolving, but the trusteeship

remains intact.31 Today, with the expectation of

sustained Federal support of science, concern has

shifted to ‘how much growth’ and ‘how to manage

expansion.” With acute and widespread awareness

of the dependency of research institutions on Federal

support, money has become the lightning rod of

debates over science and other institutional domains. While this is apparent to most decisionmakers, equally important but less visible is the

issue of the organization for making policy choices,

i.e., how to distribute whatever monies are allocated

for research.

Differing conceptions of urgency, time-scale, and

level of investment feed tensions within the scientific community as Federal priorities change. In a

dynamic, pluralistic system, discontinuities in funding can be expected. The Federal Government is

accused of supporting faddish research on the one

hand, and of sluggishness in responding to new

research opportunities on the other. What is often

seen as a choice between big science and little

science, or between high-energy physics and molecular genetics, is often more apparent than real.

Overall funding decisions are often shaped more by

funding allocations between research and other

national objectives.32 As symbolized in the debates

over the Superconducting Super Collider and the

Human Genome Project, there is a sense of congressional urgency, frustration, and ambivalence over

research goals.

While representative democracy ultimately invests the power of decisionmaking in elected officials of the Federal Government (who judge political

and national needs), these decisions are tempered by

expert advice. Such judgments have consequences

for decisionmaking and accountability, especially at

the research agencies.33 More than the other

branches of government, Congress-the representative of the public interest-is at the nexus of the

trusteeship for research. Congress plays an increasingly active role, both in determiningg the Federal

research budget and in stewarding the Federal

research system in directions that serve the public

good (see chapter 3).

Prospects for the 1990s

Science and engineering are increasingly vital

parts of the Nation’s culture; research contributes in

many ways to the technological and economic base.

Since the post-Sputnik era, both the capacity to

perform research and the demand for funds to sustain

scientific progress have grown. As the research

enterprise moves into the 1990s, the Federal research system will experience changing funding

patterns and various pressures from both outside and

within the scientific community. How, in the face of

changing funds and goals, can Congress ensure that

the research system satisfies national needs, while

retaining the diversity, flexibility, and creativity that

have characterized U.S. contributions to scientific

knowledge and its payoffs? Four challenges are

clear.

First, new methods for setting priorities in research funding will be required. Looking across

fields and at objectives that build on, but are not

limited to, scientific merit is the responsibility of

OSTP, OMB, the research agencies, and the scientific community, as well as Congress. Each may

weigh funding criteria differently, but each has a role

in preparing the enterprise for tomorrow’s research

opportunities as well as today ’s.

Concern over the amount and distribution of

Federal research funding is voiced increasingly

throughout Congress. As one former member put it:

At present we have no well-defined process

. . . for systematically evaluating the balance of the

overall Federal investment in research and develop-

~or commentary on how 40 years of Federal funding policy strayed from the letter, and perhaps even the spirit, of Wnnevar Bush’s vision of a

centralized system see Deborah Shapley and Rustum Roy, Lost at the Frontier (Philadelphi~ PA: 1S1 Press, 1985). The House Committee’s Science

Policy Task Force concurred with this appraisal in 1986, observing that: “The National Science Foundation, originally conceived as a central

coordinating body, was left with a restricted jurisdiction over uuclassiiled, basic research. ” House Committee on Science and ‘IiAmology, op. cit.,

footnote 24. As Morone, op. cit., footnote 28, p. 4, puts it: ‘‘In effect Bush called for a Department of Science, which would fund research as well as

educatio~ natural sciences as well as life sciences, and mission-oriented research as well as general, or ‘pure,’ science. ”

slKenneth Prewitt, “The Public ad Science pOhCy, ” Science, Technology, & Human Values, vol. 7, No. 39, spring 1982, pp. 5-14.

szFor a discussio~ see Genevieve J. Knezo and Richard E. Rowberg, “Big and Little Science,” CRS Review, February 1988, pp. 6-8; and ‘‘Money

for the Boffis,” The Economist, vol. 318, Feb. 16, 1991, pp. 15-16.

ss~ee OTA contrmtorrepo~, featur~ later in this report, provide data on the rhetoric of accountability used by vtious participants in the Fedeml

research system. But see Office of Technology Assessment, op. cit., footnote 25. On the role of the media in promoting accountability, see Marcel C.

LaFollette, “Scientists and the Media: In Seamh of a Healthier Symbiosis, ” The Scientist, vol. 4, No. 14, July 9, 1990, pp. 13-15.
Chapter 2—The Value of Science and the Changing Research Economy ● 63

ment and in the variety of fields that we try to serve.

The R&D budgets of the different Federal agencies

are evaluated separately and largely independently,

both within the executive branch and certainly here

in the House and Senate. . . . Of particular interest

are the criteria for evaluating competing research

development projects in different fields and the

organizational arrangements for helping us to do a

better job of allocating scarce resources.34

Since the support of science and engineering

research is vital for the future of the United States,

the Federal Government attempts to maintain a

strong ‘‘science base, ‘‘ i.e., research across a wide

range of science and engineering fields.35 To the

extent that specific areas, problems, and projects

may be singled out for enhanced finding, debate

within the scientific community centers on the

adverse impacts of funding large new initiatives, or

‘‘megaprojects, on the science base. The criteria

and information to inform priority setting are thus

paramount issues, as decisions must be made

between competing goals.36

A second challenge is that, because demands for

research funds are likely to continue to outpace

funding in most parts of the research budget,

strategies for coping--devised by sponsors and

performers alike-will be needed. Congress is

especially concerned about the question of costs,

because the Federal Government supports research

expenditures (e.g., salaries, indirect costs, equipment, and facilities) that have increased over the

general rate of inflation. In addition, more researchers are performing federally funded research and, in

the aggregate, are spending more across-the-board

on their research projects.37

Photo credit: U.S. Department of Energy

This is a cross section of cable destined for the

Superconducting Super Collider. Capital expenditures,

especially for equipment, are an integral part of most

megaprojects.

Recently, the Federal Government has experimented with ways to cope with the rising demands

of research, i.e., the expectations that spending will

increase in the performance of research. First,

Congress imposed salary caps on NIH- and NSFfunded research grants. In fiscal year 1991, legislation relaxed these constrictions. Second, Congress

and USDA recently placed a ceiling on the proportion of indirect costs allowable on research grants.

This experiment has yet to be fully implemented, but

it is expected that universities will attempt to recover

an of the House Subcommittee on Science, and in U.S. Congress, House Committee on Science,

Space, and Technology, The Hearings on Adequacy, Direction, Priorities for the American Science and Technology Effort, Feb.

28-Mar. 1, 1989 (Washington DC: U.S. Government Printing Office, 1989), pp. 1-2.

 see David Baltimore, ‘‘The Worsening Climate for Biological Research,’Technology Review, vol. 92, No. 4, May-June 1989, p. 22.

 agency tradeoffs routinely within and “peerreview’ informs the project of Programs, 

them accountable to specialized research communities. When criteria in addition to merit included peer reviews, however, selection

mechanisms come under duress, See Margaret of Review, ’ Vantage Point, spring 1990, pp. 12-13.

For recent appraisals of selection mechanisms and agency accountability for see U.S. Congress, House Committee on Science and 

 on Science Policy, Project Selection, vol. 17, hearings, 99th Apr. 8-10, 1986 DC: U.S. Gov Printing

Office, 1986); and National Science Foundation of the Inspector General, Semiannual Report to Congress, No. 2, 1, 1989-Mar. 31, 1990

(Washington DC: March 1990).

 Research Science and Technology in the Status, Trends, and Issues

(Washington DC: National Academy Press, October 1989), p. 2-32. More qualitative information is needed to understand the contexts of research

ce and to interpret the quantitative estimates of time and expenditures reported in various National Science Foundation surveys. For example,

see National Science Scientific Research Facilities at Universities and Colleges: 1990 DC: September 1990).
64 ● Federally Funded Research: Decisions for a Decade

their costs from the Federal Government by charging

more items to direct costs that were formerly part of

indirect costs.38

Third, addressing the changing demands on the

educational pipeline (K-12 through graduate study)

for science and engineering will be vital for maintaining strength in the performance of research.

Through the direct support of graduate students and

the indirect support of research institutions, the

Federal Government is pivotal in the creation of a

robust research work force. OTA has documented

the initiatives needed to maintain the readiness of the

educational pipeline. Recruitment and retention

programs can respond to changing demands for

researchers and enhance preparation for diverse

career opportunities for graduates with science and

engineering Ph. D.s.39

Human resources are the principal component of

the research system. Increasing participation in

research by those groups chronically underrepresented in science and engineering (women, ethnic/

racial minorities, and the physically disabled) and

those acutely affected by resource constraints (e.g.,

young investigators, see box 2-E) is a challenge to

the goal of enlarging capacity in the Federal research

system. The Nation (not just science and engineering) gains from the flow of new Ph.D.s into this work

force. The character of the flow (not just its

intensity) will determine the robustness of the

research system in the 1990s.

Finally, filling gaps and reducing uncertainties in

policy-relevant information is essential for better

informed decisionmaking. NSF is defined as the

Federal agency “. . . to make comprehensive studies

and recommendations regarding the Nation’s scientific research effort and its resources for scientific

activities. ’40 Empirical knowledge about the Federal research system has grown immensely, yet each

Photo credit: U.S. Department of Agriculture

A researcher studies the growth of a plant. Increasing

the participation of traditionally underrepresented

groups in science and engineering will continue

to be a focus in federally funded research.

of the three issue areas outlined above suffers from

a lack of some appropriate data on which to base

Federal policy.

New research indicators are needed as a means of

monitoring change in the Federal research system.41

OTA has also found (see chapter 8) that the

evaluation of research projects would add to the

investment decisions of policymakers and program

 see ‘‘Universities Fear That U.S. Will Limit Payments for Overhead Costs Incurred by Researchers,’ The Chronicle

of Higher vol. 37, No. 12, Nov. 21, 1990, pp. A19, A21. For a university perspective, see Association of American Universities, Indirect

Costs Associated With Federal Support of Research on University Campuses: Suggestions for Change (Washington DC: December 1988).

 repro by Assessment: Educating Scientists Engineers: Grad School,

 DC: U.S. Government Printing Office, June 1988); Elementary and Secondary Education for Science and Engineering,

 (Washington, DC: U.S. Government Printing Office, December 1988); and Higher Education for Science and Engineering,

 (Washington, DC: U.S. Government Printing March 1989).

 National Science Foundation was the agency data liaison and monitor. For the scope of these responsibilities, see especially sections

2-3 and 5-8 of Executive Order 10521, reproduced A Patron for Pure Science: The Formative Years,

1945-57 (Washington DC: Science Foundation 1982), app. 1, quote from p. 353.

 Lawrence ‘ ‘The Search fOr ’ Knowledge: Creation, Utilization, vol.

9, December 1987, pp. 168-172.
Chapter 2—The Value of Science and the Changing Research Economy ● 65

Box 2-E—The Perils of Being a Young Investigator

“The next generation. ” “The seed corn. “ “The future of scientific research. ” These are some of the words

used to describe young investigators. Current commentary on the funding of research grants, especially in

biomedicine and by the National Institutes of Health (NIH), centers on the fate of young investigators.l This

commentary underscores the unity of training and research, yet suggests the strain experienced by a growing

segment of the research work force.

Many see the problems of young investigators as a natural adjustment of the research labor market to greater

competition in funding or to changes in the structure of research teams. In the words of Rockefeller University

President David Baltimore: “How much growth in biomedical research personnel is needed and how much is

healthy?” 2 Others see the plight of young investigators as stemming from problems in funding allocation

mechanisms. Recognizing that the young investigator with little or no track record is at a disadvantage in

head-to-head competition with senior investigators for Federal research funds, both NIH and the National Science

Foundation (NSF) have established mechanisms that narrow the pool of eligibles. NIH’s First Independent Research

Support and Transition (FIRST) awards grant 5 years of support, not to exceed a total of $350,000, to successful

first-time applicants to NIH.3 Begun in 1987, recipients of FIRST awards (R-29s) have indeed fared better than other

young investigators in competing for traditional individual-investigator (ROl) funds. In fiscal year 1988, one-half

of the R29 awardees were under 36 years of age, compared to 14 percent of RO1 recipients, and 23 percent of the

young investigators were female compared to the 14 percent of traditional NIH grant recipients. 4 Perhaps the best

news for those who monitor award trends is that once young investigators get an NIH grant, they win renewals as

often as senior investigators. 5

At NSF, the much-heralded (now 7-year-old) Presidential Young Investigator (PYI) program awards 5 years

of funding.6 PYIs are augmented in two directorates by Research Initiation Awards. These provide up to $100,000

for 2 years, including an institutional matching incentive to help defray equipment costs. In 1989,726 applications

were received; 17 percent were funded. This constituted mild relief from the slim success rates, roughly one in five,

that first-time applicants have experienced since 1984 throughout most NSF programs. (More seasoned

investigators have succeeded during that period at a rate of one in three.)7

New PhDs ‘‘itch,’ in the words of one, to establish their own laboratory, attract graduate students, and produce

experimental results. The goal is to replicate the career pattern of one’s mentor. But, can every young investigator

become a PI? This will bring more proposals, more competition, more demands for research funds. A young

investigator with an excellent NIH priority score for her proposal but no money says: “When we slam up against

this problem, we have self-confidence to say ‘this is unjust,’ not ‘I am unworthy. ’ In a way, it takes an egoist to

persevere. "8

l~um wm ~~~mbte~ for ~arnple, at the National Academy of Sckncedlnstitute of Mdic&, ‘‘FOIXUII on SuPPo~

Biomedical Reseamh Near -Probkzn.s and options for ACtiO%’ ‘ Washington DC, June 27, 1990. In addition+ the Nationat Reseam.h

Council’s Commission on Life Sciences is studying the funding of young investigators. A rqort is due in fall 1991. See “Scientists Explore

Ways lb Help Young Researc hers,” NewsReporr of the National Research Council, vol. 40, August-September 1990, pp. 6-8.

-ted m “NTHCrowd SeekS New Ways Out of Money Cninc4° Science & GowwmtentRcport, vol. 20, No. 13, Aug. 1,1990, p. 2.

3sm JOC mm “NSF, NIH Apply Band-Aids,” Science, VOI, 249, July 27, 1990, p. 352.

4Natio~ IIMtiWM of Heal@ Division of Rc=h -, “Brkfing on NIH FIRST Activity,” spring 1989, pp. 6, 15, 18.

5p~c4 op. cit., footnote 3.

%s program awards about 200 grants per year with the exptxtation that during the 5-year period indus~ funding will be secured to

solidify the investigator’s research program and its impact. Even with industrial funding, however, the researcher is likeIy to apply for regular

grant suppofl. A National Science Foundation task force has recently recornm emted cutting the number of Presidential Young Investigator

awarded by ondxd.f, increasing the award amount and dropping tbe matching fund rcquiremen~ as well m amcmling the application process

to include a full-blown proposal instead of nomhwhg and endorsing lettczs from mentors and other senior inve@@rs. See Pamela Zurer,

“NSF Young Investigator Program May Be S-” Chen”caZ & Engineen.ng News, vol. 68, No. 50, Dec. 10, 1990, p. 7; and “Presidential

Young Investigators” letter, Chefi<cal & Engineering News, vol. 68, No. 50, Dec. 10, 1990, p. 5.

7Joe p~~ “Yo~g kvestiga~ at Ri~” .$cience, vol. 249, July 27, 1990, p. 353; the National Scicncc Foundation *o V* “~w

investigator awards, ’ i.e., awards to applicants not funded by NSF in the previous 5 f~cd years. Since 1984, 20 to 25 percent of total awards

were made to new investigations. See Manpower Comments, vol. 27, No. 5, June 1990, p. 31.

8P~c4 op. cit., f~tnote 7. A j-rfaculty member at the Salk Institute adds: ‘ ‘I WOITY ti~ @ ~-’t ~ ~~.~ti@t~ ~

funding at w the greater the chance your grant will be killed by bad luck--not because it isu’t good scieme. ” Arm Gibbons, ‘The SaIk Institute

at a Crossroads,” Science, VOI. 249, July 27, 1990, p. 361.

Continued on next page
66 ● Federally Funded Research: Decisions for a Decade

Box 2-E—The Perils of Being a Young Investigator-Continued

Another tack is to be (reluctantly) pragmatic, “. . . buttering up senior researchers and NIH review panel

members who could help their chances of getting funded. , . . When good science could get you a grant, you didn’t

need to do it. Now you have to, and that’s turning many people into cynics."9 Is the next generation to be the ones

who feel deceived when the system does not work for them the way it was “supposed” to? This is a question of

expectations. A recent survey of young physics faculty at all 175 physics Ph.D.-granting universities in the United

States (conducted by the American Physical Society) adds another perspective to gauging the plight of the young

investigator. 10 In 1990,70 percent of the young physics faculty reported that research funding is inadequate, whereas

in 1977 less than 25 percent responded similarly. Of the 1990 young Ph.D. faculty who submitted ‘‘start-up’ (i.e.,

their first) proposals, condensed matter physicists submitted the largest average number of proposals (over five),

and experienced the lowest success rates (25 percent). All other subfields had success rates from 38 to 55 percent. 1 1

The report concludes that “. . . . there has been a major change for the worse in the research climate. ” For

condensed matter physicists, most of whom consider NSF the dominant source of support, this may be true. But

the perceptions do not generalize across all subfields. Indeed, both 1977 and 1990 young physics faculty

overwhelmingly ‘‘would recommend physics’ and would choose to pursue a career in physics again. In addition,

twice the proportion of 1977 young faculty claimed that the ‘‘job market was worse than expected’ than reported

by the 1990 young faculty (61 percent to 31 percent) .12

The merits of additional support to young investigators cannot be overstated. How this is to be achieved poses

formidable challenges to research agencies and program managers, as well as to the scientific community. All

contribute to the expectations and the standards for measuring the research performance of new Ph.D. s. For those

young investigators who embark on academic research careers, the prospect of a FIRST, PYI, or Research Initiation

award is vital if they are to become senior researchers. NIH and NSF face choices, too, in shaping researchers’

expectations. These choices might include:

● limiting the amount of Federal funding that goes to one principal investigator, taking into account all sources

of Federal research funds and cost differences among fields;

● addressing policies at some universities that prohibit nonfaculty personnel from applying for Federal

research funds as principal investigators, and encouraging these universities to lift such bans;

● requiring the sharing of doctoral students and instrumentation; and

● encouraging universities to restrict the number of refereed publications considered for promotion, tenure,

and other awards (to decrease the amounts of Federal funding required to publish longer lists of research

papers) .13

%lm op. cit., footnote 7, pp. 352-353.

lWhequesao make was circulated to 939 physicists who earned a Ph.D. degree in 1980 orlatcr and then received acaderm “c appointments.

lEc response rate was 71 percent. See Roman Cm~o et al., Their Most Productive Years, Report on the 1990 Survey of~oung Physics Faculty

(Washington, DC: American Physical Society, 1991) (reprinted in Physics Zlxky, Fdxnary 1991, pp. 37-42).

Ilcondmd mattm p~icists mpresentti the largest subfield (one-third of the total respondents) in the 1990 sample. Md., Uble 3.

121bid., table 5.

13 For diXWslon of ~X ~otier id-, scc wtitute of Mxticine, Futi”ng Health Sciences Research: A Stiutegy TO Restore Babce

(Wud@pon. DC: National kadcmy Press, November 1990). For insight into the contentiousrms that greeted the institute of Medicine report,

see Peter G. Gossel@ ‘‘A Clash of Scientific Titans: Key Groups Battle Ovez Funds for Medical Rojccts, ’ The Washington Posr, Health sectioQ

Dec. 18/25, 1990, p. 6.

managers and would further serve to keep agencies support structure and creating policy-useful indicaalert to problems in the process of research perform- tors and evaluations could assist policy formulation

ance.

42 Filling information gaps in the Federal by both the legislative and executive branches and

dz’1’rend dah we desirable because they revd the early signals of flagging or surging health in one area or another. Because Wkt is ~~g ~e~~~

is changing over time, such trends are open to interpretation. In sho~ interpretation must keep apace of growing sophistication in measurement. This

and not the data alone becomes information for decisionrnaking. See, for example, Ciba Foundatio~ The Evaluation ofkientific Research (FJew Yorlq

NY: JohrI Wiley & SonS, 1989); Computer Horizons, Inc., “An Assessment of the Factors Affecting Critical Cancer Research Findings, ” executive

summary, NIH Evaluation Project No. 83-304, Sept. 30, 1987; and U.S. Congress, office of lkchnology Assessmen4 Research Funding as an

Investment: Can We Measure the Returns? OTA-SET-TM-36 (Washingto~ DC: U.S. Government Printing Office, April 1986).
Chapter 2—The Value of Science and the Changing Research Economy ● 67

help to inform decisionmakers about the effects of a After introducing this decentralized system—how

changing research economy on research priorities, the executive and legislative branches negotiate

expenditures, and performers. Information, how- national goals and the Federal budget, and how the

ever, is not cost-free. Additional funding both for agencies determine the allocation of research

agency data collection and analysis, and extramural funds-OTA assesses the challenges to managing

‘‘research on research, ’ may be a necessary invest- federally funded research. ment in the Federal research system of the 1990s.

In the chapters that follow, OTA delineates the

participants and their roles in the research system.
CHAPTER 3

The Federal Research System:

The Executive and

Legislative Branches
Contents

Page

Introduction . . . . . . . . . . . . .. . . . ........ . . . . . . . . . . . . . . . . . . . . . . . . . .. + .. . . . . .+ . . . . . . . .

The Executive Branch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The Science Advisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .

Office of Management and Budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

External Advice and Interest Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The Legislative Branch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The Congressional Committee Structure and the Budget Process . . . . . . . . . . . . . . . . . . .

Congressional

Congressional

Conclusions . . . .

Box

Oversight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Earmarking . . . . . . . . . . . . . . . . . . . . . . . . . . .. .....................+.. . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Boxes

71

71

74

77

78

81

82

86

87

93

Page

3-A. OMB and the Research Budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

3-B. New Layers of Complexity for the Federal Budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Figure

3-1.

3-2.

Advisors to the

Figures

Page

President 1932-90 . . . . . . . . .....,, . . . . . . . . . . . . . . . . . . . . . . 75

Apparent Academic Earmarks by State and at Universities and Colleges:

Fiscal Years 1980-89 . . . . . . . . . . .,...., . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .93

Tables

Table

3-1. Congressional Authorization Committees and Appropriations

With Significant Legislative Authority Over R&D . . . . . . . . . .

3-2. Apparent Academic Earmarks: Fiscal Years 1980-89 . . . . . . . .

Page

Subcommittees

. . . . . . . . . . . . . . . . . . . . . 83

. . . . . . . . . . . . . . . . . . . . 90

3-3. Apparent Academic Earmarks, by Appropriations Subcommittee: Fiscal Years

1980-90 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

3-4. Apparent Academic Earmarks Contained in the Fisca1 Years 1980-89

Appropriations, Ranked by State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
CHAPTER 3

The Federal Research System:

The Executive and Legislative Branches

In the final analysis, after science and technology decisions have been subject to the

judgment of conflicting objectives, . . . they are then subject to the reality of the Federal

budget process. First research and development programs must compete with other

Federal programs for the availability of limited Federal dollars . . . for there will always

be more programs and projects than there will be funds to implement them. Thus another

set of choices in how to allocate the funds to gain the greatest benefits must be faced.

Introduction

It is often said that the best scientists not only

know how to solve problems, but how to pick them.

Choosing where to put valuable time and resources

is central to the success of any scientist, laboratory,

or university. The same is true for the Federal

Government.

Decisionmaking occurs on many levels within the

Federal research system. The most macroscopic

level for research decisionmaking concerns a spectrum of general research problems such as space

exploration, aging, or AIDS (acquired immunodeficiency syndrome). The President and Congress are

ultimately responsible for decisions made at this

level. At mid-levels, the focus shifts to fields such as

astrophysics, virology, or artificial intelligence.

Most often Federal agencies and specific congressional committees take the lead in these decisions.

Priorities within a single field of science or technology usually involve specific government programs

and congressional subcommittees. And, finally, at

the most microscopic level, the focus is on areas of

research specialization and often involves specific

processes of funding allocation.2

A focus of this report is the tremendous diversity

within the Federal Government in the selection of

priorities for research. Every Federal agency and

congressional committee seems to do it differently.3

If the government is to respond to changing fiscal

Don Fuqual

conditions, many choices within the organization

and management of the research budgets must be

made.

This chapter discusses the highest level of decisionmakers-the President, the executive branch,

and Congress. (Chapter 4 introduces the Federal

agencies and other participating bodies.) Although

in this discussion the executive and legislative

branch% are treated separately, there is important

interaction between them, both formally at congressional hearings and executive branch briefings and

informally among staff.

The Executive Branch

When President Bush awarded the National

Medal of Science and the National Medal of

Technology to 30 scientists and engineers in November 1990, he remarked: ‘‘More and more our

Nation depends on basic, scientific research to spur

economic growth, longer and healthier lives, a more

secure world and indeed a safer environment. ”4

Traditionally, Presidents have been very supportive of science and engineering, or what is categorically known as research and development (R&D).

However:

Every administration refers each year to its

“R&D budget,” which is described in various

documents—most notably, Special Analysis J, produced by the Office of Management and Budget. In

actuality, there is no Federal R&D budget, if by

IDon Fuqu% “science policy The Evolution of Anticipation, “ Technology in Socie~, vol. 2, 1980, p. 372.

?For overviews, see Bruce L.R. SmitlL American Science Policy Since World WarZZ (Washingto% DC: The Brookings Institution 1990); and David

Dicksou The New Politics of Science (New York NY: Pantheon, 1984).

s~dviewed ~ a ~ross.mtio~fiwewor~ tie U.S. rese~hsyst~ is distinctive. See app. D for a discussion of pliofity setting in Other COUll~~.

Quoted in “National Medals Are Pinned on 30 Scientists,” The Washington Post, Nov. 15, 1990, p. A23.

–71–
72 ● Federally Funded Research: Decisions for a Decade

Photo credit: Michael Jenkins

The House Committee on Agriculture, which has jurisdiction over the Department of Agriculture and its research programs, votes.

“budget” is meant a plan for matching priorities

with spending. What each administration presents to

the public is an after-the-fact compilation of the

R&D spending plans of the individual mission

agencies and NSF, plans that were developed

through a complex and fragmented sequence of local

interactions among individual groups with the agencies, the White House Office of Science and

Technology Policy, and a slew of congressional

committees. 5

The most consistent indicator of Presidential

priorities over the last 30 years has been the

Presidential Budget Message, presented to Congress

every year, which accompanies the Presidential

budget. A review of these documents, extending

back to the Kennedy Administration, gives an

interpretation of Presidential direction, at least

rhetorically, of the Federal research system.6

During the 1960s, the mastery of space and space

science, as symbolized by a manned lunar landing,

was a central mission. Competition with the Soviets

both in research and economically was the center of

the debates. Domestic research needs received

increasing emphasis from 1964 through 1968, linked

to the programs and aspirations of the Great Society,

but tempered by economic constraints stemming

from increasing involvement in Vietnam. Pollution

also became a major item of concern from 1964

onward. Specific research emphases included:

National Aeronautics and Space Administration

 G. “Federal R&D Structure: The Need for Change, ” The Bridge, vol. 19, fall 1989, p. 5. Special Analysis J was discontinued in 1990, but is discussed below.

 following is based on Mark “Basic Research Goals: Perceptions of Key Political Figures,” OTA contractor June 1990.

Available through the National Information Service, see app. F. Readers will note below the lumping of “R” and “D,” as well as the lack of distinction between “basic” and “applied” research. The macro view seeks the big picture, e.g., R&D relative to veterans’ affairs,

and other national Refinement.s come in later chapters,
Chapter 3—The Federal Research System: The Executive and Legislative Branches ● 73

(NASA) pursuit of manned flight, planetary probes,

and scientific satellites; National Science Foundation (NSF) support of facilities at universities and

colleges to strengthen science education; health

research, including the prevention of cancer, heart

disease, strokes, mental illness, mental retardation,

and environmental health problems; environmental

research, including resource conservation and development, oceanographic studies, and water and air

pollution abatement; transportation research; and

defense research.

During the 1970s, as space flight and research

were scaled back, energy research issues became

increasingly prominent, emphasizing the development of energy alternatives and the improvement of

existing ones. These issues were linked to growing

concern about dependence on foreign oil, and also to

environmental concerns of pollution and conservation of natural resources. Specific energy research

programs were emphasized by President Nixon and

others, including fusion power and geothermal and

solar energy. President Carter stressed conservation

and alternative energy sources and advancement in

nuclear power technology. Defense research was

consistently supported, and preservation of national

economic preeminence remained a strong goal on all

fronts. 7

During the 1980s, economic recovery, competitiveness, and leadership were the rhetorical focal

points of discussions of the goals and justifications

for research. Specific attention to the category of

“basic research,” begun in Presidential addresses

during 1978, was linked to goals of economic,

military, and technological leadership (although

these goals were not necessarily reflected in the

distribution of research finds, e.g., defense basic

research funding did not increase markedly in the

1980s). In the Presidential messages of 1982 to

1986, the shift of Federal aid to scientific research

and away from application and development became

explicit. Cuts in applied energy research and agricultural sciences were made, while basic energy,

defense, and biomedical research were augmented.

In the late 1980s, as in the early 1960s, big science

research projects were featured on the Presidential

agenda. The Space Station, the Strategic Defense

Photo credit: Jamie Netter, OTA staff

The President can be a major architect of the research

system, and some Presidents have shown more interest

in research and development issues than others.

Initiative, AIDS, the Human Genome Project, and

the Superconducting Super Collider (SSC) all figured prominently.

American Presidents of the last three decades

have paid heed to maintaining the science base-the

broad spectrum of researchers and research supported by the Federal Government-but have also

felt the need to concentrate resources toward achieving stated research goals. During the 1960s, when

research budgets were increasing rapidly, the President could add new objectives to the system while

maintaining other research programs. Now, Presidents must make more choices in fiscal allocation.

For example, President Reagan distinguished between basic and applied research, favoring the

former with budget increases and decreasing the

latter in specific areas such as energy. (Under the

Bush Administration, this distinction faded and

several applied energy projects have been pursued.)

However, Presidents have generally been less

involved in decisions about research policy than in

areas such as economic, space, or defense policy

(with the possible exception of decisions about

particle accelerators). Until recently, Presidents

often viewed research as within the purview of

specific agencies, intertwined with the development

of technologies and the procurement of certain

goods or services, but rarely a policy objective per

se. To keep abreast of research issues, the President

 and Herbert Simons, “Basic Research Goals: A Comparison of Political Ideologies,” contractor June 1990. Available

through the National Information see app. F.
74 ● Federally Funded Research: Decisions for a Decade

relies on many groups including the Office of the

Science and Technology Policy (OSTP) and the

Office of Management and Budget (OMB).

The Science Advisor

Science Advisors most often have impeccable

technical credentials and extensive experience

within the scientific community8 (see figure 3-l).

President Eisenhower appointed James Killian the

first titled Science Advisor in 1958. At present, D.

Allan Bromley holds that position. He is typical of

past science advisors: physicists with outstanding

research records and a history of participating in

government advisory committees on science and

technology. 9 Advisors over the last 30 years have

come from industry and university settings.

One criticism of Science Advisors has been that

they favor the physical sciences, while Presidential

goals have included life and social science objectives as well.10 Another criticism of the position is

that, while acting as the representative of the

President, advisors are also seen as allies of the

science community from which they were recruited,

expected to give advice on all scientific matters as a

“scientist.” This dual role can be difficult. Some

advisors, notably Keyworth and Graham in the

Reagan Administration, were regarded as outsiders

by the scientific community. They were less trusted

and seen more as voices articulating the President’s

ideological agenda.

Since the Office of Science and Technology

Policy Act in 1976, the Science Advisor has also

been the director of OSTP.ll OSTP was created by

Congress to strengthen the role of the Science

Advisor by creating a position that was parallel to

the Director of OMB and the Chairman of the

Council of Economic Advisors.12 OSTP currently

includes a small staff-less than 75—with a portion

of the personnel detailed from various Federal

agencies. With the confirmation of a new advisor

(which usually coincides with the beginning of a

Presidential administration), a new OSTP staff is

assembled. Consequently, few senior OSTP staff

will serve in their positions for longer than 4 to 5

years. However, many have extensive experience

within the executive branch, Congress, or the

scientific community. While this staff turnover

requires that the Science Advisor and OSTP “start

from scratch” and provides limited institutional

memory, it also allows OSTP to construct a new

agenda with each advisor.

In addition to providing a resource for scientific

and technical information for the President, the

responsibilities of OSTP include coordination of

R&D activities throughout the agencies. The Federal

Coordinating Council on Science, Engineering, and

Technology (FCCSET), under the chairmanship of

Science Advisor Bromley, provides a forum for

coordination. 13 Bromley has paid special attention to

FCCSET during his tenure, increasing the participation of senior agency personnel. In 1989, there were

nine active FCCSET committees.14

The Science Advisor also chairs the President’s

Council of Advisors in Science and Technology

(PCAST), which provides independent expert advice to the President. PCAST was created in 1989 in

the image of the President’s Science Advisory

awi~m GoldeU Science ati Technology Advice to the President, Congress, and Judiciary (New York NY: pergamon PresS, 19*8).

9James Killian Was a notable exception. He was trained as a humani st who rose through the ranks at the Massachusetts Institute of T&hnology as

an administrator. He was accepted into the scientific community and treated as an equal member. Harvey Brooks, Harvard University, personal

communieatio~ February 1991.

%m interview soon after his appointmen~ Bromley admitted the overrepresentation of physical scientists on such bodies as the president’s Scien@

Advisory Committee, pointing out that “. . .the life sciences must be brought in more strongly than they are now.” See Jeffrey Mervis, “New Science

Advisor Sees Strong Ties to Bush Public Support as Keys to Job,” The Scientist, vol. 3, No. 11, May 29, 1989, p. 3.

1 l~e foreuer of the OffIce of Scienw and TwhnoIogy Policy was the Office of Science and ‘lkchnology (OST). The position of OST dir~tor was

created by President Kennedy in 1961.

12Ric~d c. A&&on, “scienw Advice at the Cabinet Level,” in Goldeq Op. cit., footnote 8, P. 12.

IsFor a histow of the Federal Coordinafig Council for Science, Engineering, and Technology, see congressional Research Service, Interagency

Coordination of Federal Scientl~”c Research and Development: The Federal Council for Science and Technology, Report to the Subcommittee on

Domestic and InternatioM Scientilc Planning and Analysis, committee on Scienm and lkchnology, U.S. House of Representatives, 94th Cong.

(Washington DC: U.S. Government Printing Office, July 1976).

14Genevieve J. Knezo, “White House OffIce of Science and Technology Policy: An Analysis,” CRS Report for Congress @4@ingtou DC:

Congressional Research Service, Nov. 20, 1989), pp. 61-62.
Figure 3-l—Science Advisors to the President, 1932-90

Science Bush Steelman

Advisors:

Kistiakowsky Dubridge

I Hornig I David Stever Press Keyworth Graham

I I [

Killian Wiesner McTague a Bromley

Presidents: Roosevelt Truman Eisenhower Kennedy Johnson Nixon Ford Carter Reagan Bush

1932 1936 1940 1944 1948 1952 1956 1960

aMCTague was acting wien~ Advisor between KeyWorth and Graham.

SOURCE: Adapted from William G. Wells, School of Government and Business Administration,

dissertation, 1977, p. 18.

1964 1968 1972 1976 1980 1984 1988 1992

George Washington University, “Science Advice and the Presidency, 1933-76,” unpublished
76 ● Federally Funded Research: Decisions for a Decade

Committee (PSAC), which had been disbanded by

President Nixon in 1973.15 (Although the authorization to constitute a new PSAC was included in the

1976 legislation which created OSTP, no action was

taken until 13 years later.) Members of PCAST,

appointed by the President, are distinguished leaders

in science and engineering from industry, philanthropy, and academia. Although PCAST can be

asked to comment on specific scientific and technological matters, it can also offer opinions on other

issues and solicit its own outside analysis. (PCAST

has only been in operation for a little over a year, so

it is difficult to determine the role that it may play in

the 1990s.16)

PCAST, OSTP, and the Science Advisor are

advisory to the President. As such, they have not

been given much power. As a former staff member

in the George Keyworth-led OSTP writes:

The position of the President’s Science Advisor

(and director of OSTP) is strictly a staff function,

with no line authority and no control over budgets.

The primary tool available to the President Science

Advisor is persuasion. How effective he is in

convincing agencies to shape or modify their R&D

budgets depends largely on the strength of his

personal relationship with inner circles of the White

House.17

Two general comments can be made about the

roles of these advisory bodies for the next decade.

First, global problems such as climate change and

pollution involve issues of cooperation and planning, while President Bush’s goals for science and

mathematics achievement by the year 2000 highlight the urgency of education and human resources

for the Nation’s vitality. Both will require domestic

policy coordination, and tough research funding

tradeoffs may be needed. Second, while the role of

Photo credit: Jamie Netter, OTA staff

The Old Executive Office Building is home to much of the

Office of Science and Technology Policy and the Office

of Management and Budget.

science advice will not wane in the 1990s, it is

unlikely that the role of the Science Advisor, OSTP,

or PCAST will be strengthened legislatively. To

some, OSTP has to “. . . brighten its image on the

White House political screen. ”18 Yet, in the executive branch where influence is often equated with

budgetary control, the advantage resides primarily

with the research agencies and OMB.

 in 1957 the President’s Science Advisory Committee science advice from 

 of Defense Mobilization to the White House. Among its actions, proposed establishing a counterpart group of representatives from

the Federal research agencies to improve coordination of the Nation’s R&D effort. The result was the founding, in 1959, of the Federal Council for

Science and See Ralph Sanders and Fred R. Brown Science Technology: National DC: Industrial

College of the Armed Forces, 1966), ch. 5, especially p. 76.

16 See Jeffrey ‘‘PCASTMembersReady to President Seems Ready to The Scientist, vol. 4, No. 10, May 14,1990, pp. 1,14-15.

One role tbat the President’s Council of Advisors in Science and has already played is defining a pool of eligibles for key agency posts.

From among its 12 members, President Bush ted physicist Walter Massey to head the National Science Foundation and cardiologist Bernadine

 as director of the National Institutes of Health.

 cit., footnote 5, p. 6.

 say is Advisor who is the frost Science Advisor elevated to the title of Assistant President.

Robert Rosenzweigquoted in ‘A Good Budget for Science, But Troubles Lie Ahead,’Science and Government Report, vol. 20, No. 18, Nov. 15,1990,

p. 2.
Chapter 3—The federal Research System: The Executive and Legislative Branches ● 77

Office of Management and Budget

Research budgets are strongly influenced within

the Executive Office of the President by OMB. As

Science Advisor Bromley has remarked: ‘‘It became

evident a long time ago that if you control the

budget, you control public policy. This is one of the

facts of life that a science advisor must learn, that

OMB is a tough player and not necessarily sympathetic. ”19 OMB crafts the budgets of research

programs to reflect the priorities of the President,20

and helps to set realistic targets for the next year’s

budget in all research programs while attempting to

balance the competing needs of the Federal departments and agencies.

The manner in which these concerns are negotiated with the agencies is left to the discretion of the

OMB budget examiners. Budget examiners concerned with research are located in at least three of

its six divisions: Natural Resources, Energy, and

Science (which includes NSF, agriculture, and

space); Human Resources, Veterans, and Labor

(which includes health and education); and National Security and International Affairs (defense).

Through Special Analysis J, OMB traditionally

presented proposed R&D agency budgets for the

new fiscal year. The publication of this analysis was

discontinued after the fiscal year 1990 budget, but

since that time, R&D has been discussed (with the

information traditionally presented in Special Analysis J) in a separate introductory chapter to the

President’s budget.21

OMB’s role in research priority setting and fiscal

allocation is not public. The deliberations of the

agency are internal, building on agency submissions

preliminary to OMB decisions (see box 3-A).

Behind-the-scenes negotiation between OMB examiners and agency budgeters is common. This closeddoor policy minimizes contention, and perhaps

stifles controversy, both within and without the

government on specific funding issues.

A strong perception of OMB standards and

policies on research has grownup outside of OMB

and the executive branch. Most importantly, many

observers state that OMB has an active role in

deliberations over research agendas, particularly in

support of projects such as the Space Station and the

SSC. New programs, especially Presidential initiatives such as the Moon/Mars mission, require that

OMB be involved early in the fiscal process.22 But

because these deliberations are shielded from public

view, critics claim that these policies are not

sufficiently debated.

The fiscal 1991 budget act placed a separate cap

on discretionary spending in three budget categories: defense, domestic, and international programs.

These caps limit spending for each of fiscal years

1991 through 1995 and specify methods of enforcing

deficit targets. 23 The caps will force tradeoffs within

each category of the budget; this will effectively

reduce flexibility and foster more negotiation within

the executive branch (i.e., among OMB, OSTP, and

the agencies) in the allocations for specific programs.

24 Members of OMB staff also stress, however, that these caps will force greater priority

setting based on the research issues, because the

overall spending levels will be set.25 Observers

agree that the Omnibus Budget Reconciliation Act

of 1990 has enhanced OMB’s authority relative to

Congress because OMB “. . . will have a final say

on cost estimates for all programs. ’ ‘26 (For further

information on the budget act, see box 3-B.)

19M~is, op. cit., footnote 10, p. 3.

~nder President Bus~ the Federal Coordinating Council for Science, Engineering, and ‘Ikchnology and the Science Advisor have participated in

the implementation of several presidential priorities. For the fiscal year 1991 budget, they included global climate change, high-performance computing,

and mathematics and science education.

zl~ese Wyses, in turn, form the basis for an analysis and spring colloquium on the R&D budge~ held in Washingto% DC, ~dpr~ented by the

American Association for the Advancement of Science. These proceedings, edited and published the following fall, serve an important interpretive

function for the scientific community, relating the budget to topical issues in science and technology. For the 15th annual proceedings, see Susan L. Sauer

(cd.), Science and Technology and the Changing World Order (Washingto~ DC: American Association for the Advancement of Science, 1990).

22Hugh~we@ c ‘scienceAdvising ~d OMB,’” The Presidency and Science Advising, VO1. 3, Kenneth w. Thompson (~.)~

MD: University

Press of America, 1987).

~See “Tifle ~—Budget Enforcemen~” Congressional Recor~ouse, Oct. 26, 1990, pp. H 12743-H 127M.

~Karl Erb, Oftlce of Science and ‘Ikchnology Policy, personal communication Novmber 19W.

~Ro&fi (&ady, aSSWiate dfi=tor, Na~~ Resoues, Ener~, ad science, Office of M~gement md Budget person~ cOIUUll,micatio@ Feb. 7,

1991.

Usee Thomm J. De~u@, ‘ ‘Deficit-Reduction pl~ Cotid Tighten Budgets for Student Aid ~d Rese~@”The Chronicle ofl-ligher Education,

VO1. 37, No. 10, NOV. 7, 1990, pp. 1418, A28.
78 ● Federally Funded Research: Decisions for Decade

Box 3-A-OMB and the Research Budget

The Office of Management and Budget (OMB) reviews the budgets of all Federal agencies before submission

of the President's budget to Congress arid perform crosscutting budget analyses, espcially for topics of partieuiar

interest to the President. Traditionally, OMB has been very supportive of research in the Federal budget. This

reflects the importance attached to research and development (R&D) in the budgeting process and the overall real

and symbolic value of Federal research support as an indicator of future planning and direction in government

investments.

Under President Bush and OMB Director Darman, the process of planning the research budget has changed.

Before the budget is collated, R&D is the subject of several separate briefings and detailed analyses of issues

concerning research initiatives (e.g., funding for individual investigators and big science projects on a case-by+ase

basis). The Federal Coordinating Council on Science, Engineering, and Technology (FCCSET) committees and the

Science Advisor have also participated extensively in the implementation of programs in severa1 Presidential

priority areas, especially global climate change, high-performance computing, and mathematics and science

education. Important criteria for R&D investment used by OMB include the support of excellent science and

engineering, long-term competitiveness and economic concerns, commercial spinoffs, national prestige, and

“national security”—in the broadest military and economic sense.l

In the budget process for fiscal years 1991 and 1992, OMB asked the research agencies to submit budgets at

five levels of funding, which include scenarios with real cuts as well as augmented funding, In addition, OMB

requested that, for areas of particular Presidential interest, agency budget requests be ”... described and justified

relative to the goals, objectives, and research priorities. . .“ outlined in various framework documents, such as the

U.S. Global Change Research Program.2 In areas not of highest priority, less crosscutting analysis is performed, and

the manner in which these concerns are negotiated with the agencies is left more to the discretion of OMB budget

examiners,

Tradeoffs are made among agency programs, and between the “research budget” and other areas of domestic

discretionary finding. Under the new budget agreement, when OMB “passes back” the agency budgets after the

first review, OMB has budgeted up to the caps determined for the agencies. If an agency wishes to increase specific

levels of funding, decreases to the agency budget must also be specific to allow the total budget to remain under

the spending cap. Tradeoffs are then made explicitly among agency programs.3

In summary, OMB provides a unique crosscutting function in research budgeting within the executive branch,

Under President Bush, the implementation of research priorities has been accompaniedby an increased roll for the

Science Advisor and the FCCSET committees.4 In addition, general research priority setting has been elevated in

the presentation of the President’s budget. However, priority setting unrelated to targeted Presidential concern

remains primarily at the discretion of the budget examiners for the specific agencies, and tradeoffs are within agency

budgets.

@~A IW@@ widI Robert Grady, 3oseph Hezir, and kuAcF@ows, Ufficeof -t%tltlnt~dIllldget, Feb. v, 1991.

~WMtxt3L Grady, AssoGiate Direetor, Natural Resourees, Energy, and S@mee,OffkX Of hMtl@ll@ ~d~udg~~”-of~@f_~

Memoramfam on the FY 1992 U.S. Global Change Research PrcJ~” unpublished doeumemt, JnrM 18,1990.

3~ yJ arman CMficeof Management and Bud@ (UMB) was saidtu ccMskierR&13 asykdding especially high futtwemtums on Fmk3@

imwstmemt. ‘lb this ex die process instituted by CM@3 encmrages the research agedesto develop coherent proposals and discourages lioth

within-agency disagreermmfs and mresponsiveness of an agency to the requests of either 0M3 or ti Officoof f?tience W IMmOlogyPoiicy. @oundrales and deadlines am spelled out in the “terms of reference” issued by OMB in every priority areadesignmd for an agency mow%uk

4Part af this incr-Clft3ee of Scieriee snd ‘fkclmology Pcdicy (and Office of xewont and Bwlge~-relnfore@ rol~ m l@d @

requests of th~ agencitw for evaluative dst,a. Of speeial relovanee to W f3TA report is the information gatlwring in progress @ the “structure

of science,” an activity of the l%deral Coordinating C!our@ on Science, Engineering, md ‘Ibchnology CcmitxdttW cm Fhysieal, Mathematical,

and Ik@neering Sciences.

External Advice and Interest Groups policy structure that takes into account a range of

views on many decisions. Active debate occurs both

Much scientific and technical advice is solicited informally and within the scientific literature on

from the scientific community by the executive programs, policies, and projects initiated by the

branch. This partnership between the scientific Federal Government, and this debate often influcommunity and government has led to a complex ences government decisions.
Chapter 3—The Federal Research System: The Executive and Legislative Branches ● 79

Box 3-B—New Layers of Complexity for the Federal Budget

A budget process many critics had said was too complicated has become even more so, thanks to sweeping

changes adopted by Congress and approved by President Bush.l Here are the major revisions to the 1985 Balanced

Budget Act that will dramatically alter the way the budget will be drafted through fiscal 1995.

Discretionary Spending

For fiscal years 1991 to 1993, the law establishes separate ceilings for each of three categories of discretionary

spending: defense, international aid, and domestic programs. If Congress chooses to increase spending for any

discretionary program, it must offset the increase by cutting spending within the same category.2 If it fails to make

an offsetting reduction, an automatic spending cut--a sequester, in budget jargon-would slice enough from all

other programs in that category to bring spending to below the ceiling.

The spending caps, in billions of dollars, are as follows (BA is budget authority, the amount Congress

authorizes the government to spend in current or future years; O is outlays, the actual spending expected in each

year):

The

1991 1992 1993

Defense

BA . . . . . . . . . . . . . . . . . . . . . $288.9 $291.6 $291.8

0 . . . . . . . . . . . . . . . . . . . . . 297.7 295.7 292.7

International

BA . . . . . . . . . . . . . . . . . . . . . 20.1 20.5 21.4

0 . . . . . . . . . . . . . . . . . . . . . 18.6 19.1 19.6

Domestic

BA . . . . . . . . . . . . . . . . . . . . . 182.7 191.3 198.3

0 . . . . . . . . . . . . . . . . . . . . . 198.1 210.1 221.7

For fiscal years 1994 to 1995, the new law establishes a single pot of money for all discretionary spending.

White House and Congress will have to decide how to allocate that money between the three spending

categories. Spending above that overall limit would trigger a sequester to bring spending down to that ceiling. Total

discretionary funds for those two years, in billions of dollars, are as follows:

1994 1995

BA . . . . . . . . $510.8 $517.7

0 . . . . . . . . . 534.8 540.8

Pay-as-You-Go Spending

Under the new law, Congress is required to offset the costs of any new entitlement spending programs and any

tax reduction legislation. If Congress creates an entitlement program or tax benefit that is not ‘‘revenue

neutral" ‘--not financed by an offsetting tax increase or spending cut—it would then have to adopt a deficit cutting

‘‘reconciliation bill to find the needed savings. Failing that, a sequester would cut enough from all other

entitlements (except those, such as social security, that are already exempt from the ‘‘sequester’ under the 1985

Balanced Budget Act) to make up the difference.

Adhering to broad sentiment in Congress, the new law takes the social security trust funds out of the deficit

calculations. As a result, the funds’ growing surpluses will not be used in determining whether the government has

met its annual deficit targets. That is a victory for those who complained that the trust fund surpluses were masking

the budget deficit’s true size.

Sequesters

Unlike the 1985 law, which called for a sequester in October of each year in which Congress failed to meet

specific deficit targets, the new law creates a schedule under which sequesters can occur several times a year for

discretionary programs and once a year for entitlements and tax cuts.

1The following is an edited version of "Adding New Layers of Complexity to Budget," a box appearing in Lawrence J.Haas, “New

Rules of the Game, ” National Journal, vol. 22, No. 46, Nov. 17, 1990, p. 2796.

2

The defense category, however, does not include the costs of Operation Desert Storm in the Persian Gulf, which the law assumes

Congress will finance separately.

Continued on next page
80 ● Federally Funded Research: Decisions for a Decade

Box 3-B—New Layers of Complexity for the Federal Budget-Continued

First, a sequester directed at budget-busting appropriations bills can be triggered, if required, within 15 days

of the end of a session of Congress. Second, a sequester can occur within 15 days of the enactment of such

appropriations bills if the enactment takes place before July 1. Third, if those appropriations bills are enacted after

July 1, a sequester would be applied to spending bills for the next fiscal year, which begins on October 1.

For entitlements and tax cuts, the law calls for a one-time review of all bills to determine whether they will,

in total, increase the deficit. If they will, nonexempt entitlements would be sequestered at the same time as the

end-of-session appropriations.

Deficit Targets

The new law sets deficit targets for the next 5 fiscal years. They are (in billions of dollars):

1991 . . . . . . . . . $327

1992 . . . . . . . . . 317

1993 . . . . . . . . . 236

1994 . . . . . . . . . 102

1995 . . . . . . . . . 8 3

For fiscal years 1991 to 1993, the targets are not binding on the White House and Congress. Along with the

spending caps for defense, international aid, and domestic programs, they will be adjusted to account for changes

in economic and technical assumptions. For fiscal years 1994 to 1995, the President may adjust the deficit targets,

if he chooses, for economic or technical reasons. If he does not adjust them, failure to reach those targets will trigger

a sequester like that required under the 1985 budget law.

Scorekeeping

Furthering a trend that began when the Balanced Budget Act was revised in 1987, the Office of Management

and Budget (OMB) has been given additional authority to tabulate the cost of tax and spending legislation.

Previously, OMB had the power to decide whether, based on the costs of all such legislation and other factors, a

sequester was required. But the Congressional Budget Office and Congress’s Joint Committee on Taxation had the

duty of tallying the costs of each tax and spending bill as it moved through Congress. Now, OMB’s cost calculations

will be binding on Congress.

While the Science Advisor and numerous advi- bership of eminent specialists, working through

sory committees allow the scientific community, or

more accurately, the various research constituencies

within it, a voice in government decisions, other

channels also exist to influence Federal policy. An

unrivaled source of authority is the independent,

congressionally chartered (in 1863) National Academy of Sciences (NAS). The presidents of NAS, the

National Academy of Engineering, and the Institute

of Medicine act as opinion leaders and buffers

between the science community and the Federal

Government (discussed further in chapter 5). Both

the executive branch agencies and Congress call on

(and pay for) NAS to conduct studies on issues of

some urgency and importance in science, technology, and medicine. 27 The academies’ elected mempanels and commissions, lends credibility to the

reports they issue.28

Various interest groups have also traditionally

played major roles in the formulation of Federal

research funding and regulatory policy. Of an

estimated 6,000 public and special interest groups

active in Washington, many have a stake in some

aspect of the diffuse Federal research activities.29

Prominent interest groups that lobby on behalf of

science include many industrial groups, professional

societies, the higher education associations, and

other more specialized groups that encourage research in targeted areas, such as the environment or

health.

27And tie con~essio~ appetite has grown from 9 National Academy reports mandated by the 95th Congress (1979 to 1980) to 24by tie lolst. See

“Congress Hungry for NAS Advice,’ Science, vol. 250, Dec. 7, 1990, p. 1334.

28For a definitive look at the National Academy of Sciences as a social institution see PhilLip Boffey, The Brain Bank of America (New Yo*, NY:

McGraw-Hill, 1975). The NationaJ Academy Press also publishes a quarterly journal, ZssUes in Science& Technology, which provides a policy forum

for an array of opinion leaders in and out of government. The National Research Council’s (NRC’s) NewsReport also provides a record of National

Academy of Sciences’ studies undertaken by NRC.

29Deborah M. Burek et al. (eds.), Encyclopedia ojAssociations, vol. 2 (Detroit, MI: Gale Researc4 kc., 19*9).
Chapter 3—The Federal Research System: The Executive and Legislative Branches ● 81

As discussed below, interest group lobbying is

most often associated with the legislative branch

since the congressional decisionmaking process is

more open and decentralized. However, lobbying of

executive agencies also occurs and can sometimes

have a significant effect on specific research programs. For example, program managers at the

Agricultural Research Service in the U.S. Department of Agriculture (USDA) and at the Conservation and Renewable Office in the U.S. Department

of Energy (DOE) state that agribusiness and energy

industry lobbies, respectively, play a large role in

setting agency priorities. In the current system, this

involvement is important, because agribusiness and

the energy industries are considered the eventual

clients of these programs. Interest groups can also

provide additional technical information (which

may not be available to agency personnel) that can

be used for decisionmaking, and they can influence

the development of debate on specific programs.

Outside interest groups can be seen as an informal

extension of the advisory committee system and can

be very beneficial to agency operations. However, in

a more ideal system, the influence of interest groups

and their interactions with the government would be

made more public.30

After the executive branch agencies, OMB, and

others produce the President’s budget, it goes to

Congress. Research program budgets and their

accompanying support documentation are subsequently reinterpreted by congressional committees

to determine agency priorities (e.g., increases over

inflation or predicted spending targets in specific

programs are interpreted as strong executive branch

support, while corresponding decreases are interpreted more negatively). Congress then has an

opportunity to comment on and change these priorities.

The Legislative Branch

Congress has traditionally been very supportive

of the research enterprise in the United States, and

rarely do debates over research issues divide along

partisan lines. In particular, there has existed over at

least the last 30 years a broadly shared supportive

ideology covering goals, values, programmatic priorities, and rationales. The same arguments about

health, economic competitiveness, and national

prestige are part of members’ arguments about

science policy from all ideological perspectives.31

Nevertheless, emphases given to specific programs

have varied over the years.

During the 1960s, research was perceived as a

means of increasing national prestige, enhancing

security, and providing benefits. The goals of

outdistancing the Soviets and maintainingg a leadership position in the world through research were

supported by Democrats and Republicans, liberals

and conservatives, hawks and doves. However,

while some members were convinced that research

monies for defense could be better spent on domestic

problems, others believed that direct expenditures

on defense research would be more effective for

boosting the economy and national defense.

During the late 1960s and early 1970s, science

was burdened with greater material expectations,

especially after the success of the Apollo Moon

program. 32 Preservation of national preeminence

remained a strong goal on all fronts, but different

groups stressed different tangible rewards. The

Democratic party platform in 1972 argued that

research should protect the environment and improve employment for scientists .33 In the same year,

Republicans sought a science that would improve

U.S. economic competitiveness internationally.34

Liberals and conservatives clashed over the pace and

extent of environmental initiatives. However, these

disagreements were most often expressed over

specific programming rather than the importance of

a clean environment.

During the mid- to late 1970s, Democratic and

Republican priorities diverged, despite agreement

on some specific program areas. For example, 1976

Democrat and Republican party platforms supported

energy research. However, the Democrats made a

case for government investment, calling for”. . . major

~See tie Byd Anti-Lobbying Provision (Public Law 101-121).

slBkdsell and Simons, op. Cit., footnote 7.

Szsee J.W. F~bright, ‘1s the ~oject Apol10 Rogram TO Land Astronauts on the Moon by 1970 a Sound National Objective?’ CongressionuZDigest,

vol. 44, February 1965, pp. 47, 49, 51, 53; and Barry M. Goldwater, “Is the Project Apollo Program ‘lb Land Astronauts on the Moon by 1970 a Sound

Nationrd Objective?’ Congressional Digest, vol. 44, February 1965, pp. 53,55.

ssBmce D. Johnsou Natio~/ paw pla~or~: VO/UnW H, 1960-76 (Urbana, IL: University of Illinois ~ess, 1978), pp. 802-803.

‘Ibid., pp. 876-877.
82 ● Federally Funded Research: Decisions for a Decade

Photo credit: Michael Jenkins

Two members of the Senate Committee on Energy and

Natural Resources, which has jurisdiction over the

research programs at the Department of Energy and

Interior, confer.

initiatives, including major governmental participation in early high-risk development projects. . . .“35

The 1976 Republican party platform detailed the

importance of maintainingg a balance among private,

university, and government efforts at scientific

research. It pledged to “. . . support a national

science policy that will foster the public-private

partnership to insure that we maintain our leadership

role. ’36 This position in the Republican platform

was deepened considerably in OMB’s Issues ’78,

which accompanied President Ford’s final budget.37

Issues ’78 stressed the importance of leaving a role

for the private sector and avoiding government

involvement in readying technologies for commercial development.

During the 1980s, Republicans took the ‘Issues’

agenda a step further, arguing that ‘‘partnerships”

among government, universities, and industry were

the best way to promote research, leaving all

development issues to industry except in cases of a

pressing defense interest. Democrats contested this

rationale, arguing that a massive increase in the

research funds oriented toward defense tarnished

relations between the government and the scientific

community .38

In general, Congress is empowered to be an

architect of the research system. To implement or

guide initiatives in the U.S. research system, Congress can adjust the research budget, craft legislation, or monitor and influence Federal agencies

through the oversight function. (See appendix A for

a summary of major legislation passed by Congress

since 1975 affecting U.S. R&D.) Unfortunately,

because it must consider the priorities set by the

Federal agencies after they have been codified in the

President’s budget or after they have been acted on

in a program, Congress’ position has often been

reactive rather than proactive.

The following describes the congressional committee structure, budget process, and the oversight

function. These processes are well understood. The

relatively new phenomenon of earmarking appropriations to universities (for eventual use in the

conduct of research) is described in more detail.

The Congressional Committee Structure and

the Budget Process

Almost one-half of the 303 committees and

subcommittees of the 10lst Congress claimed jurisdiction over some aspect of research. 39 While

inhibiting development of coordinated public policy, this fragmentation has characterized the long

history of Federal involvement in research. Furthermore, congressional history shows that Congress has

generally chosen to decentralize decisionmaking

further rather than to consolidate and coordinate the

Federal legislative process.

The Committee System

Congress’ internal party organizations in each

house assign members to committees, considering

their preferences, party needs, and the geographical

 p. 934.

 p. 984.

 of and ’78 DC: U.S. Government 

 of more active Federal intervention in R&D that affects the civilian economy. to

the v. Bush debate of the late 1940s over the role of a national science foundation. Brooks, op. cit., footnote 9.

 following is breed on U.S. Congress, of Assessment, the Goods: Public Technologies

and Management, OTA-SET-477 (Washington DC: U.S. Government Printing April 1991). Also see Morris P. Congress: Keystone

of the Washington Establishment, 2d ed. (New CT: Yale University Press, 1989).
Chapter 3—The Federal Research System: The Executive and Legislative Branches ● 83

Table 3-l—Congressional Authorization Committees and Appropriations

Subcommittees With Significant Legislative Authority Over R&D

Jurisdictions of authorization committees: a Agency

House:

Agriculture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Armed Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Energy and Commerce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Interior and insular Affairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Science, Space, and Technology . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Public Works and Transportation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Merchant Marine and Fisheries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Veterans Affairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Foreign Affairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Senate:

Agriculture, Nutrition, and Forestry. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Armed Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Commerce, Science, and Transportation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Energy and Natural Resources , . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Labor and Human Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Environment and Public Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Veterans Affairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Foreign Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

USDA

DOD,DOE

DOE, ADAMHA, NIH,CDC,DOT

DOI

NASA, NSF,DOE,EPA,NOAA, DOT,NIST,DOl

NOAA,DOT

USDA,NOAA,DOT

VA

A.I.D.

USDA

DOD,DOE

NSF, NASA,DOT,NOAA,NIST

DOE,DOI

NIH,ADAMHA, CDC,NSF

EPA

VA

A.I.D.

Jurisdictions of appropriations committees:a Agency

Labor, Health and Human Services, Education

and Related Agencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . NIH,ADAMHA,CDC

HUD and lndependent Agencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . NASA,NSF, EPA,VA

Energy and Water Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DOE

interior and Related Agencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DOE,USDA, DOI

Agriculture, Rural Development and Related Agenciesb . . . . . . . . . . . . . . . . . . . . . . . . USDA

Commerce,Justice, State, the Judiciary, and Related Agencies . . . . . . . . . . . . . . . . . NOAA,NIST

Transportation and Related Agencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DOT

Foreign Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.I.D.

Defense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DOD

KEY: ADAMHA=Alcoho~ Drug Abuse, and Mental Health Administration; A.1.D.=Agency for international Development; CDC=CentersforDisease Contro~

DOD=U.S. Departmentof Defense; DOE=JJ.S. DepartmentofEnergy; DOi=JJ.S. Departmentofthe anterior; DOT=dJ,S. DepartmentofTransportation;

EPA=JJ.S. Environmental Protection Agency; HUD=U.S. Department of Housing and Urban Developmen~ NASA=National Aeronautics and Space

Administration; NIH=National Institutes of Health; NIST=National Institute of Standards and Technology; NOAA-National Oceanographic and

Atmospheric Administration; NSF= National Science Foundation; USDA=4J.S. Department of Agriculture; VA=U.S. Department of Veterans Affairs.

aThe jurisdictions of the authorizing ~mmittees are not exclusive. For this table, repeated authorization of a number of R& D-reiated Pro9rams was r~uired

to establish jurisdiction.

bhe corresponding subcommittees of the Senate and House Committees on Appropriations have the same name with one exception: the Senate

Subcommittee on Agriculture, Rural Development, and Related Agencies and the House Subcommittee on Rural Development, Agriculture, and Related

Agencies.

SOURCES: Office of Technology Assessment, 1991; and Elizabeth Baldwin and Christopher T. Hill, “The Budget Process and Large-Scale Science Funding,”

CRS Review, February 1988, p. 15.

and ideological balance of each committee.40 Most committees with important legislative jurisdiction

bills are referred to one standing committee, but the over research.

complexity of public policy issues means that major Overlapping committee jurisdictions can slow

bills are often sent to multiple committees with and even stall policy development and send mixed

overlapping jurisdictions. Individual committee signals to the executive branch and lower levels of

rules determine a bill’s subcommittee assignments, government. Committees that try to develop comwhich also can overlap. Table 3-1 shows the prehensive research policies are often frustrated by

~ the IOlst Congress, the Senate had 16 standing committees and 87 subcommittees; the House operated with 22 committees aud 146

subcommittees. In additio~ the IOlst Congress has 9 special or select (with 11 subcommittees) and 4 joint committees (with 8 subcommittees) whose

functions are primarily investigative. The average Senate committee had five subcommittees, compared to seven in the House. Every House member,

except top party leaders, sewed on at least one standing committee. Semtors served on at least two committees.
84 ● Federally Funded Research: Decisions for a Decade

the vested interests of their sister committees,

executive branch agencies, and various research

communities.

Congressional committees have evolved into

permanent bodies with authority to propose legislation, an independence that has given committees

almost unassailable influence over legislation in

their specialized areas.41 Committee chairmen con- 42 They tend to be sequently wield enormous power.

long-lived in their positions, holding them much

longer than the terms of most presidents or Federal

agency executives. This longevity allows committee

chairmen to influence the long-term course of events

in a particular area and to implement detailed

agendas. However, some committees are better

positioned on certain issues than others.

The Budget Process and the Authorization and

Appropriations Committees

Authorizing committees in both houses report

annual or multiyear authorization bills for Federal

programs under the jurisdiction, thereby setting the

maximum amount of money an agency may spend

on a specific program. The exceptions are entitlement programs, such as social security and Medicaid, which operate under permanent authorization

and are effectively removed from the authorizing

process. Authorizing (or legislative) committees and

subcommittees are influential through their oversight functions when major new legislation is first

passed, when an agency is created or its program

substantially modified, and when setting funding

authorizations to initiate, enhance, or terminate a

program. During the 1980s, deficit reduction laws

and trends restricting spending, shortcomings in the

budget process, and new programs greatly expanded

the roles of the “money” committees-Appropriations, Budget, and Ways and Means on the House

side, and Appropriations, Budget, and Finance in the

Senate—at the expense of authorizing committees.

After the Presidential budget reaches Congress,

the Budget committees in the House and Senate

provide a concurrent resolution that sets an overall

ceiling and limits for major spending areas, like

health or transportation. Appropriation bills, origiPhoto credit: Michael Jenkins

Members of the House Committee on Agriculture debate the

1990 Farm Bill, which affected many research programs at

the Department of Agriculture.

nating within the House Committee on Appropriations and its 13 subcommittees, effectively

control spending since authorized funds may not be

spent unless they are also appropriated.

No less than nine subcommittees of Appropriations have jurisdiction over research. While these

nine subcommittees will decide what monies are

appropriated for research, the initial distribution of

funds by the full committee among the subcommittees can have serious implications for research

funding. For example, the Veterans Affairs, Housing

and Urban Development, and Independent Agencies

Appropriations Subcommittee is responsible for the

budgets of NASA and NSF, or 35 percent of the

civilian R&D budget. If this subcommittee is for

some reason “left short,” then science funding

could suffer significantly as it competes with housing, veterans’ affairs, and other programs. Furthermore, research budgets will be largely negotiated

within the new “domestic” spending category,

making decisions all the more difficult. As noted

earlier, the Budget Reconciliation Act of 1990

establishes limits on discretionary spending by

category (a new Title VI of the Congressional

Budget Act of 1974). It also states:

As soon as possible after Congress completes

action on a discretionary spending . . . bill, and after

consultation with the [House and Senate] budget

 updated by Carol Hardy, The Congressional Standing Committee System--An Introductory Guide 

Research Service, May 1989), p. 2.

 power revolution of 1910, ” limiting the role of the Speaker by establishing seniority the 

determiningg committee chairmanship and moving up in its ranks. p. 3. In battles of information where larger support staffs can determine the victor,

committee chairmen have a distinct advantage with additional committee personnel.
Chapter 3—The Federal Research System: The Executive and Legislative Branches ● 85

committees, the Congressional Budget Office

(CBO) is to provide the Office of Management and

Budget (OMB) with an estimate of the bill’s effect

on spending and revenues. . . . OMB is required to

explain differences between its estimates and those

of CB0.43

Some research agencies fared very well in the

congressional appropriations process during the

1980s. For example, even under tight budgetary

constraints, the National Institutes of Health was

routinely given more money by the Labor, Health

and Human Services, Education, and Related Agencies Subcommittee than the Administration had

originally proposed.44 USDA, and to some extent

DOE, have also consistently received more in actual

budget authority than allocated in the President’s

budget. Although in theory, policy and oversight is

reserved for authorizing committees, appropriations

committees frequently insert legislative provisions

and funding for special projects into bills (see the

discussion below on congressional earmarking). The

appropriations committees’ control over spending

and the tendency to modify authorizing legislation

creates tensions and intensifies intercommittee rivalries, particularly in the House where a smaller

proportion of members serve on the Committee on

Appropriations.

In Congress, jurisdiction or turf can mean additional staff, publicity, and power, prompting committees to seek broad jurisdictions and resist moves

to narrow them, perpetuating conflicts and overlaps.

Research issues are particularly susceptible to fragmentation and competition, because they cut abroad

swath across national life. Historically, each issue

has developed independently based on different

goals and objectives, establishing supportive committee connections and constituencies that are hard

to alter. Larger jurisdictional areas allow greater

flexibility in linking issues within comprehensive

legislation. However, they can also pit unrelated

issues against each other for attention on a committee’s agenda.

External Advice and Scientific Interest Groups

The congressional process is open and decentralized and is designed to incorporate public opinion.

Like the executive branch, Congress solicits advice

from scientific experts on many issues. This partnership and the active open involvement of the scientific community has lent strength to government

decisionmaking on research.

In addition to solicited advice, scientific information is also offered by the thousands of public and

special interest groups that actively lobby the

Federal Government. These groups organize the

opinions of their constituents. They employ technical experts to press their cases to Congress, testifying at hearings, providing privileged information,

drafting model legislation, publishing and distributing reports, and meeting with members and staff. For

example, in the global climate change debates in

Congress, various environmental groups (both for

and against action on global climate change) have

presented comprehensive technical analyses detailing the current state of scientific knowledge and the

most notable gaps. These analyses have influenced

the allocation of monies for research in these areas.

The number of interest groups and politically

active professional organizations increased dramatically during the 1970s and 1980s. This proliferation

coincided with an expansion of congressional subcommittees, which provided more opportunities for

lobbying and greater public participation in executive agency rulemaking. 45 While often seen a s

detrimental to the process, interest groups can

furnish valuable information to debates and can

present important arguments. Nevertheless, as with

executive branch lobbying, because of the informal

nature of the relationship of interest groups to

43’’ Title ~,” Op. Cit., footnote 23, p. H12745. A later section on “scorekeeping” underscores the point: “Section 251(a)(7) and 252(d) of

Gramrn-Rudman-Hollings as amended by this conference agreement provides that the Office of Management and Budget must make its estimates in

Conforrnance with scorekeeping guidelines determined for consultation among the Semte and House Committees on the Budget the Congressional

Budget OffIce, and the Office of Management and Budget” (p. H12749), See Lawrence J. Haas, “New Rules of the Game,’ National Journal, vol. 22,

No. 46, NOV. 17, 1990, pp. 2793-2797.

~~en= Association for me Advmcement of Science, Congressional Action on Research and Development in the Fy 1991 Budget (was~gto~

DC: 1990), p. 7.

dsne research lobbies are a heterogeneous lot, ranging from for example, the Industrial Research Institute, tie P-ceutical ~~ac~ers

Association, and Research! America to the education lobbies, such as the National Association of State Universities and Land-Grant Colleges and the

Association of American Universities, and the Federal liaisons for the research universities who work closely with State congressional delegations.
86 Federally Funded Research: Decisions for a Decade

Congress, interactions can appear unseemly. Every

issue, including research funding, has a constituency

and, therefore, special interests.46

Congressional oversight

Congress has invested the executive branch with

broad authority over the multitude of Federal

agencies and programs. However, in 1946, Congress

officially reaffirmed its responsibility for oversight

in the Legislative Reorganization Act. In 1970,

Congress required that House and Senate committees publish oversight reports every 2 years, and

increased committee staff size. Congress further

acted in the 1974 Congressional Budget and Impoundment Act to strengthen the role of the General

Accounting Office (GAO—a congressional support

agency) to acquire fiscal and program-related information. 47

In addition, House committee rules adopted in

1974 stipulated that committees with more than 15

members (raised to 20 members in 1975) create

oversight subcommittees or require that legislative

subcommittees provide oversight. Legislative subcommittees can only carry out oversight within their

jurisdiction, while oversight subcommittees operate

within the full committee’s jurisdiction.48

Oversight can be exercised through: 1) hearings

and investigations; 2) the authorization and appropriations processes; 3) GAO audits and investigations; 4) other studies by congressional support

agencies; 5) legislatively mandated periodic reporting from executive branch agencies to Congress; 6)

the Senate confirmation process of high-level political appointees; 7) casework and constituent questions about Federal agencies; 8) creation of special

task forces; and 9) informal, nonstatutory controls,

such as informal contacts between agency personnel

and congressional staff. Groups outside of Congress

and the executive agencies aid these processes by

providing information to Congress about potential

and existing problems in the executive agencies.

Congressional oversight has been important in

determining the budgets of specific research programs and encouraging coordination between the

research agencies.

49 Congressional oversight addresses the problems of research management and

priority setting. Recently, fraud and misconduct by

scientists in federally sponsored research projects

have also been a focus of congressional investigations. 50 Combined with the power of the purse,

Congress has effective tools to initiate change within

the Federal research system.51

One tool that has been increasingly used by

Congress in the last decade is academic earmarking-the provision of funds as line items in the

budget for specific research facilities and projects.

Because this practice is seen as circumventing

normal procedures, it has been a subject of heated

debate within the scientific community and Congress.

~Some~es, tie beststrate~ for serving that interest is disputed among the lobbyists themselves. All work behind the scenes; some also place

advertisements in the The Washington Post and The New York Times. For example, see Joseph Palca, ‘‘Grants Squeeze Stirs Up Lobbyists,” Science,

vol. 248, May 18, 1990, pp. 803-804.

dTCongressioti Quarterly, CongressioM/ Quarterly’s Guide to Congress, Michael D. Wormer (cd.) (Washington+ DC: Confessional @arterlY ~c.,

1982), pp. 459-462.

~k1990, there were 11 House committees with oversight subcommittees: Armed Services; Banking, F_a and Urb~ Affairs; Energy ~d

Commerce; Interior and Insular Affairs; Merchant Marine and Fisheries; Post office and Civil Service; Public Works and Transportatio~ Science, Space

and Technology; Veterans Affairs; Ways and Means; and Select Intelligence. In addition, there are four committees whose implicit function is oversight:

Appropriations, Budget District of Columbia, and Government operations. The House rules alSO give seven committees special oversight abilities to

cross jurisdictional lines: Armed Services; Budge~ Education and Labor; Foreign Affairs; Interior and Insular Affairs; Science, Space and lkchnology;

and Small Business. Three Senate committees had oversight subcommittees: Agriculture, Nutritiou and Forestry; Finance; and Government Operations.

Two Semte committees have implicit oversight responsibilities: Appropriations and Budget. Together, the House and Semte committees have oversight

over all of the R&D programs in the Federal agencies.

@See NIorris S. Ogul ~d Bert A. Rockm~, “Overseeing Oversight: New Departures and Old Problems,”Legislative Studies Quarterly, vol. 15,

February 1990, pp. 5-24.

~see Mari@nJ. Litiejolmand Christine M. Matthews, ‘‘Scientillc Misconduct iuAcademia: Efforts to Address the Issue,’ CRSReportforCongress,

89-392 SPR (Washington, DC: Congressional Research Service, June 30, 1989); Rosemary Chalk and Patricia Woolf, “Regulating a ‘Knowledge

Business ’,” Issues in Science & Technology, vol. 5, No. 2, winter 1988-89, pp. 33-37; U.S. Congress, House Committee on Science, Space, and

lkchnology, Subcommittee on Investigations and Oversight, Maintaining the Integrity of Scientific Research, IOlst Cong. (Washingto~ DC: U.S.

Government Printing Office, January 1990); and U.S. Congress, House Committee on Government operations, Are Scientific Misconduct and Conj7icts

of Interest Hazardous to Our Health? IOlst Cong. (Washington, DC: U.S. Government Printing Offke, September 1990).

slFor an @ysiS, see Marcel C. LaFollette, “Congressional Oversight of Science and lkchnology Programs,’ paper prepared for the Committee on

Science, Ikchnology, and Congress, Carnegie Commission on Science, lkchnology, and Governmen~ New York, NY, September 1990.
Chapter 3—The Federal Research System: The Executive and Legislative Branches ● 87

Congressional Earmarking

Since the early decades of this century, powerful

legislators, especially committee chairmen and

ranking members, have made the congressional

earmark (a specific project funded directly by

congressional appropriation) a routine, albeit small,

part of the process by which the Nation’s budget is

disbursed to regions, States, and districts. Through

earmarks a range of goods and services are procured.

The practice of congressional earmarking is now a

well-entrenched and important component of this

political system, and it has historically been re -

garded as a redistributive device that addresses fiscal

inequities through legislative power.52

While earmarking has been a traditional funding

mechanism in many areas of government spending,

explicit ‘‘academic earmarks’ appear to be a

relatively new phenomenon, dating to the early

1980s. 53 That this funding mechanism has been

extended to academic research is not surprising,

given the geographical and other inequities in

research funding. However, for the scientific community in which the ethic of peer review is so strong,

earmarking is contrary to the established mentality

of ‘‘fair’ funding allocation. It signals a departure

from the old social contract that delegated authority

to representatives of the scientific community to

judge technical merit and advise the Federal Government on research investments.

5 4

What Is an Academic Earmark?

For the purposes of this discussion, OTA defines

a congressional academic earmark as a project,

facility, instrument, or other academic or researchrelated expense that is directly funded by Congress,

which has not been subjected to peer review and will

not be competitively awarded.55 Among the largest

examples of 1990 earmarks under this definition are

the Soybean Laboratory at the University of IllinoisUrbana, the Waste Management Center at the

University of New Orleans, a medical facility at the

Oregon Health Sciences University, and a geology

research project awarded to the University of Nevada system.

There are other definitions of academic earmarks. 56 One states that an earmark is any research

project or facility directly funded by Congress. This

definition implies that the executive branch role in

setting budgets and priorities and administering the

Federal Government’s research programs is more

valid than decisions made by Congress. Not surprisingly, some members consider this definition an

insult to Congress. Another definition stresses that

earmarks are projects that are initiated by Congress

and receive appropriations, but not approved by

authorizing committees. This definition reflects

some members’ view that the legislative process

should work as is formally intended, i.e., authorizations should always precede appropriations. Consequently, this definition is sometimes used within

Congress to oppose earmarking, as earmarkers

violate the norms of the budget process .57 Still other

definitions seek to make exceptions for direct

appropriations for projects in the Agriculture appropriations bill, because agricultural research is said to

have a distinct culture where such projects are the

norm. Finally, other definitions make a distinction

52Jo~A- Ferejo@ fJork Bame/PoZitic~ (sword, CA: Stiord University Hess, 1974), p. 252. One story has it that the word “e~ k’ ‘ derives

horn a practice as old as the Republic itself. Pigs’ ears were cut off prior to the animals grazing in a common area with the pigs owned by others. Credit

for a stolen or slaughtered pig could be established by possession of the physical evidence-the ‘‘mark’ of the ear.

sqHugh~we~ who retied titer 35 yWs from tie offIce of -gement ~d Budget (OMB) in 1986 as deputy associate dkector for energy ~d

science, cites as the origin of the current wave of academic earmarks Science Advisor George KeyWorth. In 1982, without consulting either with OMB

or the materials research community, Keyworth attempted to insert $140 million in the Department of Energy budget as a ‘‘Presidential initiative” for

a National Center for Advanced Materials at Lawrence Berkeley Laboratory in California. A storm of protest led, depending on the source, to a scaling

back of the project or a temporary deferral by Congress. See Wil Lepkowski, “Hugh Lowe@ Key Science Policy Official, Retires,” Chemical &

Engineering News, vol. 64, No. 29, July 21, 1986, pp. 16-18; and Robert P. Crease and Nicholas P. SamiOS, ‘ ‘Managing the Unmanageable, ” TheAtlantic

Monthly, January 1991, p. 88. Crease and Samios interpret the significance of this event this way:‘‘After that episode Congress lost the restraint with

which it had traditionally approached the basic research budget. If Presidential initiatives were possible, it was argued, so were congressional initiatives,

and universities began to lobby Congress directly for them. ”

~See ~c~dc. Atkinson ~dWilliw A. Blapied (~s.), Science, Technology, ad Government:A cri~i~ o~f’~rpo~e,proceefigs Of a ,Sy’mpOSiUm,

March 1988 (La Jolla, CA: University of California, San Diego, 1989), pp. 53-61.

55~ ~s contex~ ~ti review refers t. tie Comvtitionof props~s for fids, Wfich me mted by independent scientific experts selected to advise ~

agency. See ch. 4 for a more complete deftition of peer review.

sGWhat follows is based on James Savage, University of Virginia, “Academic Earmarks and the Distribution of Federal Research Funds: A Policy

Interpretation,” OIA contractor report, July 1990. Available through the Nationalkchnical Information Service, see app. F.

57CoWess is not of one ~d on mmking, and while many congressional mpresen~tives e armar~ others are steadfastly opposed to it. See Dan

Morgm “Nunn Says He’ll Investigate Some Defense Bill Projects, ” The Washington Post, Oct. 10, 1990, p. A4.
88 ● Federally Funded Research: Decisions for a Decade

between earmarks and direct appropriations for

historically Black colleges and for other traditionally federally funded institutions such as Gallaudet

University in Washington, DC.

The Debate Over Congressional

Earmarking for Research

Within much of the scientific community, academic research earmarking is disdained: it is seen as

circumventing peer review, politicizing science, and

reducing the quality of research by diverting funds

that otherwise would be awarded competitively for

facilities and projects.58 However, no one claims that

simply because a project was funded through

earmarking that ipso facto it would produce bad

science. There is in fact evidence that earmarks can

produce well-respected research;59 some universities have defended their earmarks by pointing to

positive evaluations of the earmarked projects by the

relevant Federal agency (after the project has been

initiated). Opponents to earmarking state that, given

limited Federal resources, many worthy projects are

likely to be denied funding, and thus some means of

evaluating and ranking all research proposals are

desirable. Some appropriations subcommittees may

seek the advice of the cognizant agency on the merits

of an earmark before funding it, particularly on

facilities projects, but there is no evidence that such

advice is sought systematically.

Earmarks often originate with legislation proposed by powerful members of Congress and

strategically placed members on specific committees. There is much benefit to obtaining an earmark,

especially since such projects are a relatively inexpensive way to help ensure reelection by bringing

Federal funds to the member’s district. These

members are thought to be able to stifle debate on the

merits of these projects, or cooperation between

members is thought to circumvent it. This lack of

open debate is seen as potentially jeopardizing the

quality of the projects funded by earmarks and

contributes to the perceived waste of national

resources.

60

On the other hand, many support earmarking,

claiming it as legitimate political decisionmaking

without which fair distribution of Federal funds

would never take place. Proponents contend that

there must be a tradeoff between efficiency and

distribution, and that policymakers must work so

that a portion of the wealth can be distributed to poor

areas of the country.61

Congressional earmarking must also be viewed in

relation to the almost absolute power of executive

agencies to disburse Federal monies (subject to

oversight by Congress). By seeking support for a

specific program or project, executive agencies can

designate monies for specific geographical areas or

institutions-much like an earmark. For example,

the SSC is to be built in Texas. DOE is thus supporting

research in a specific geographical area and in the

institutions and groups that will participate in the

creation of the SSC.62 Congress wonders whether it

is responsible democratic government to confine all

direct spending power in the executive branch. If

agency processes do not meet desired ends, many

claim that there must be some method for Congress

to directly correct inequities. Earmarks thus are seen

by many-both inside Congress and out—as expenditures having merit in furthering socially justifiable goals.

As congressional earmarking is currently practiced, it can disrupt agency budgeting. If additional

money is not set aside for earmarks, then funds that

were planned by the agency for their new or

58~e5e a~ents me reviewed in Daryl E. Chub@ “ScienWlc Malpractice and the Contemporary Politics of KIIowkdge,” TheO~~e~ Ofkie~Ce in

Sociery, S.E. Cozzens and TF. Gieryn (eds.) (Bloomington LN: Indiana University Press, 1990), pp. 149-167. Also see Ken Schlossberg, “Earmarking

by Congress Can Help Rebuild the Country’s Research Infrastructure,” The Chronicle of Higher Education, vol. 36, No. 19, Jan. 24, 1990, p. A48; and

Bob Davis, “Federal Budget Pinch May Cut Amount of ‘Pork’ to Colleges Living Off of the Fat of the Land,” The Wa12 Street Journal, May 2, 1990,

p. A18.

590TA bterView5 in the qn-ing of 1990 at the Department of Energy (DOE) found tit mmy e~ ks of the early 1980s produced research centers

highly regarded by some DOE program managers who had originally opposed them.

mU.S. Gener~ Accounting Otllce, Budget Issues: Earmarking in the Federal Government (WiiSh@tOm m: Jmum 1990), P. 1.

61Earmarking can be in conflict with peer review, and perhaps should be. The two processes are designed to achieve different goals.

G@M K~tz, “pie ~ fie S@: Big Science is Ready for Blastoff,” Congressional Quarterly Weekly Report, VO1. 48, Apr. 28, 1990, p. 1254.
Chapter 3—The Federal Research System: The Executive and Legislative Branches ● 89

continuing programs must be reallocated to cover

the congressionally mandated expenditures.63” For

example, to cover earmarked projects in fiscal year

1989, DOE’s Office of Basic Energy Sciences

reallocated $20 million from programs it had

planned. 64 To the extent that this is undesirable, it

could be remedied if Congress would increase

appropriations to cover the expense of earmarked

programs or facilities. In addition, an earmark

includes permission for the agency to spend less in

another area (e.g., when Congress designates money

for equipment at a specific university while also

appropriating monies for programs that disburse

funds to universities for the same type of equipment), and this tradeoff could be made explicitly in

the congressional budget.

There are few sources of academic earmarking

information, and longitudinal data are even harder to

compile.65 Table 3-2 shows that for fiscal years 1980

to 1989 over 300 earmarks in appropriations bills for

academic facilities and projects represented a total

dollar value exceeding $900 million. In the fiscal

year 1991 budget, at least $270 million was designated for earmarks.66 The data focus on appropriations, reflecting the fact that most earmarks originate

in appropriations rather than authorization bills.

Eventually the focus of data collection and analysis

will have to expand, however, because academic

earmarks have appeared in authorizing legislation,

and some are added in amendments to legislation on

the House and Senate floors.

Academic Earmarks: Increasing Research

Capacity and Equity?

Two issues have been linked to earmarking. The

first is that the Federal Government has decreased its

funding for facilities since the 1960s. Because many

earmarks are specified for facilities construction,

some argue that a Federal facilities program would

decrease the frequency of earmarking in Congress.67

However, since the potential demand for new

facilities is so large, no Federal facilities program

could immediately address all of the need, and

earmarking would still be important to allow some

institutions to receive facilities monies in advance of

others. A similar argument holds for the earmarking

of equipment.

The second issue addresses the most commonly

stated reason for pursuing earmarked funds-that

existing proposal review systems are biased in favor

of certain institutions over others for the distribution

of Federal funds. Academic institutions that are not

research intensive (the so-called have-nets) seek

earmarks to acquire the scientific infrastructure that

gives research universities (the so-called haves) a

competitive edge in winning research awards. Earmarking thus is seen as a means of reducing inequities

@The Office of Science ad Technology Policy identified over $800 million incon~ssiondearrnarks for R&D projects inthefiscal year 1991 budget.

Over one-third of these came from accounts”. . . that were either cut or held constant by Congress-which means that the money had to be taken directly

horn other projects.’ See Colin Norman, “Science Budget: Growth Amid Red Inlq” Science, vol. 251, Feb. 8, 1991, pp. 616-618, quote from 617.

‘Corey S. Powell, ‘‘Universities Reach Into Pork Barrel With Help From Friends in Congress, “ Physics Today, vol. 42, No. 4, April 1989, pp. 43-45.

GsSoWces include systematic listings in The Chronicle of Higher Education of institutions receivm earmarks, occasional stories in Science &

Government Report, in the newsletter Higher Education Daily, and a recent study for the House Committee on Armed Services. A General Accounting

Office (GAO) study of 17 Department of Defense projects “. . . either congressionally mandated or established by the Army . . .“ included four projects

that were “. . . established non-competitively. ’ However, the focus of the GAO report is oversight of university research and is not a comprehensive

assessment of academic earmarks. See U.S. General Accounting Office, International Security and International Affairs Divisio~ Defense Research:

Information on Selected Universi~ Research Projects, GA()/NSIAD-90-223FS ~ashingto~ w: August 1990). The most comprehensive data on

“apparent academic earmarks” have been assembled by James Savage for the Oftlce of the President, University of California. They are based in part

on Susan Bore% ‘‘Appropriations Enacted for Specific Colleges and Universities by the 9&h Through the 1O(MI Congress,” CRS Report 89-82 EPW,

Feb. 6, 1989. See James Savage, Office of the President, University of California ‘‘The Distribution of Academic Earmarks tithe Federal Government’s

Appropriations Bills, FY 1980- 1989,” rnimeo, Mar. 7, 1989; and James Savage, office of the President, University of CalifOrni% “Apparent Academic

Earmarks in the FY 1990 Federal Appropriations Bills,’ mirmm, Dec. 19, 1989. Michael Crow, Iowa State University, personal communication August

1990, insists “. . . that there is currently no reliable data source from which to draw mermin~”ti conclusions about the impact of academic~wA comprehensive, well-defined effort is needed.’ O’lA endorses the cdl for more systematic study.

66Eliot Mars~l and David p. Hamilton, ‘A Glut of Academic Por~’ Science, vol. 250, Nov. 23, 1990, pp. 1072-1073. Another estimate coma ~m

Colleen Cordes, “Congress Earmarked $493 Million for Specific Universities; Critics Deride Much of the Total as ‘Pork Barrel’ Spending,” The

Chronicle of Higher Education, vol. 37, No. 24, Feb. 27, 1991, pp. Al, A21.

GTFormoreon the issue of researchfacilities, originated thrOU@ e armarks orno~ see Congressional Research Service, BncksandMortar: A Summary

Analysis of Proposals TO Meet Research Facilities Neeh on College Campuses, report to the Subcommittee on Science, Research and ‘lixhIIoIogy,

Committee on Science, Space, and Technology, U.S. House of Represenhtives, IOOth Cong. (Washington DC: U.S. Government Printing oflice,

September 1987).
90 ● Federally Funded Research: Decisions for a Decade

Table 3-2—Apparent Academic Earmarks:

Fiscal Years 1980-89

Fiscal year Dollar value Number

1980 . . . . . . . . . $ 10,740,000 7

1981 . . . . . . . . . —a —

1982 . . . . . . . . . 9,370,999 9

1983 . . . . . . . . . 77,400,000 13

1984 . . . . . . . . . 39,320,000 6

1985 . . . . . . . . . 104,085,000 36

1986 . . . . . . . . . 115,885,000 39

1987 . . . . . . . . . 113,800,000 41

1988 . . . . . . . . . 232,292,000 72

1989 . . . . . . . . . 202,537,000 87

1990 . . . . . . . . . 132,381,087 94

Total . . . . . . . $1,037,811,086 407

aTheon[ydir@ appropriationsin 1981 were to historically Blackuniversities(three)andtwootherinstitutions withintheDistrictofColumbia, which

Savage does notcount as academic earmarks.

SOURCE: James Savage, Office of the President, Universit y of California,

“Apparent Academic Earmarks in the Fiscal Year 1990 Federal

Appropriations Bills,” mimeo, December 1989, table 1.

or leveling the playing field.68 (See table 3-3 for a

10-year breakdown of earmarking by appropriations

subcommittee.)

Congress is concerned with equity in all of the

funds that it disburses. The historical concern within

Congress over equity in science funding has centered on geographical equity, not the contemporary

emphasis on institutional development. Geographical distribution suggests that certain institutions

from each major region should be competitive in

receiving Federal funding. In this manner, each

region would have some opportunity to develop

centers of excellence.@ Institutional equity refers to

the ability of each institution to be able to rise to

prominence through Federal research funding. However, since there are 3,400 colleges and universities

in the United States (1,300 that award science and

engineering degrees, and 100 institutions that already command the largest share of Federal resources and produce most new Ph.D. researchers),

the Federal Government faces a daunting task.70

The present situation is this: much like the

stratified distribution of competitively awarde d

Federal research funds, academic earmarking over

the last decade has primarily benefited a handful of

States and academic institutions, although the total

amount of earmarked dollars has been relatively

small. Table 3-4 ranks the recipients of earmarked

funds by States for fiscal years 1980 to 1989. More

than 40 percent of these funds went to just 5 States,

while two-thirds were awarded to only 10. The

bottom 10 States received less than 10 percent of the

earmarks.

. . . 3 of the top 10 earmarkers include Massachusetts, New York, and Illinois, which rank in the

National Science Foundation’s list of top 10 State

recipients of Federal research funds. . . . NSF’s top

10 research States received more than a third of all

earmarks. 71

This same pattern of concentration is evident at the

institutional level: 10 universities received nearly 40

percent of the earmarks during the last decade. (For

the full distributions by State and by institution, see

figure 3-2. For a ranking of institutions by Federal

R&D funds awarded, see appendix B.)

A question for analysis might be: How have

earmarked funds affected the research capability of

the institutions receiving them? Between fiscal years

1980 and 1989, 20 academic institutions each

received roughly $14 million in earmarked funds, or

collectively 60 percent of the total earmarked

dollars. But to determine the relationship between

earmarking and research capability, other questions

need to be addressed empirically: How have institutions used their earmarked funds? And if an institution improves its research capabilities and performance, as indicated by a change in its ranking of

Federal research funds received or by its publication

and citation output, is this due to the earmarks it

receives? Or could it be that a university adminis6fJsee WiIlim c. Boesman and Christine Matthews Rose, ‘‘Equity, Excellence, and the Distribution of Federal Research and Development Funds,”

CRS Report fir Congress (Washingto~ DC: Congressional Research Service, Apr. 25, 1989). But evidence shows that even research-intensive

institutions pursue earmarks.

6~or evalmtion of ~emly Natio~ Science Foundation program dedicated to this proposition, see David E. Dmw, Science Development: ‘n

Evaluation Study, technical report presentti to the National Board on Graduate Education (Washingto~ DC: National Academy of Sciences, June 1975).

70 Science ad eng~eenng resemch ~q~es a huge ~~pi~ ~ves~ent in facilities and imtrumentatio~ followed by sustained operating SUppofi Of

that infrastructure, to attract and retain cutting-edge researchers. This capability in turn breeds success in the competition for Federal research funds.

See Norman M. Bradb~ “The Rankin“ g of Universities in the United States and Its Effect on Their Achievement” Miner-vu, vol. 26, No. 1, spring

1988, pp. 91-100. In fiscal year 1989, 100 institutions received 85 percent of Federal academic R8zD funds. See National Science Foundation, Se2ected

Data on Academic Science/Engineering R&D Expenditures: FY 1989, NSF 90-321 (Washington DC: October 1990), and CASPARdatabase, table B-35.

71A list of 74 Universities ad colleges receiving more than $1 million in earmarked funds hfkd Years 1980 to 1989 cm also be found iII Savage,

“The Distribution of Academic Earmarks in the Federal Government’s Appropriation Bills,” op. cit., footnote 65, pp. 6-7, 20-22.
Chapter 3—The Federal Research System: The Executive and Legislative Branches ● 91

I

I

I

I

I

I

I

I

I

mm

&o I

- -

c=

1111

mcoolm

Otimicd

--- -

C9C9 U) * --- -

ml

0

z

0

a

1- - -

-0) 

-

hb-l------- -~-au).cx)o)o

T--3-t3Jw*~

----- -

. .

. .

. .

. .

. .

. .

. .

. .

. .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

I=alcno alcococn Cncno)u)

-1--vy

:’6 :

292-863 0 - 91 - 4
92 ● Federally Funded Research: Decisions for a Decade

Table 3-4-Apparent Academic Earmarks Contained in the Fiscal Years 1980-89

Appropriations, Ranked by State (includes District of Columbia)

Percent of Fiscal year

funds 1988 Federal

Earmark ranka Earmarked funds (cumulative) research rankb

. . --- . . - --- 1. Massachusetts . . . . . . . . . . . . . . . . . .

2. New York . . . . . . . . . . . . . . . . . . . . . .

3. Oregon . . . . . . . . . . . . . . . . . . . . . . . .

4. Florida . . . . . . . . . . . . . . . . . . . . . . . .

5. Illinois . . . . . . . . . . . . . . . . . . . . . . . . .

6. Louisiana . . . . . . . . . . . . . . . . . . . . . .

7. South Carolina . . . . . . . . . . . . . . . . .

8. West Virginia . . . . . . . . . . . . . . . . . . .

9. Alabama . . . . . . . . . . . . . . . . . . . . . .

10. New Hampshire . . . . . . . . . . . . . . . . .

11. Pennsylvania . . . . . . . . . . . . . . . . . . .

12. Hawaii . . . . . . . . . . . . . . . . . . . . . . . .

13. Iowa . . . . . . . . . . . . . . . . . . . . . . . . . . .

14. Mississippi . . . . . . . . . . . . . . . . . . . . .

15. Arizona. . . . . . . . . . . . . . . . . . . . . . . .

16. California . . . . . . . . . . . . . . . . . . . . . .

17. North Dakota . . . . . . . . . . . . . . . . . . .

18. Oklahoma . . . . . . . . . . . . . . . . . . . . . .

19.Washington . . . . . . . . . . . . . . . . . . . .

20. District of Columbia . . . . . . . . . . . . . .

21. Indiana . . . . . . . . . . . . . . . . . . . . . . . .

22. Texas . . . . . . . . . . . . . . . . . . . . . . . . .

23. Nevada . . . . . . . . . . . . . . . . . . . . . . .

24. Utah . . . . . . . . . . . . . . . . . . . . . . . . . . .

25. North Carolina . . . . . . . . . . . . . . . . . .

26. Kansas . . . . . . . . . . . . . . . . . . . . . . . .

27. New Jersey . . . . . . . . . . . . . . . . . . . .

28.New Mexico . . . . . . . . . . . . . . . . . . . .

29. Ohio . . . . . . . . . . . . . . . . . . . . . . . . . . .

30. Kentucky . . . . . . . . . . . . . . . . . . . . . . .

31. Rhode Island . . . . . . . . . . . . . . . . . . .

32. Maryland. . . . . . . . . . . . . . . . . . . . . . .

33. Connecticut. . . . . . . . . . . . . . . . . . . .

34. Idaho . . . . . . . . . . . . . . . . . . . . . . . . . .

35.Wisconsin . . . . . . . . . . . . . . . . . . . . .

36. Michigan . . . . . . . . . . . . . . . . . . . . . .

37. Minnesota.. . . . . . . . . . . . . . . . . . . . .

38. Vermont . . . . . . . . . . . . . . . . . . . . . . .

39. Arkansas . . . . . . . . . . . . . . . . . . . . . . .

40. Georgia . . . . . . . . . . . . . . . . . . . . . . . .

41. Virginia . . . . . . . . . . . . . . . . . . . . . . . .

42. Nebraska . . . . . . . . . . . . . . . . . . . . . .

43. Alaska . . . . . . . . . . . . . . . . . . . . . . . .

44. Missouri . . . . . . . . . . . . . . . . . . . . . . .

45. South Dakota. . . . . . . . . . . . . . . . . . .

46. Montana . . . . . . . . . . . . . . . . . . . . . . .

Total . . . . . . . . . . . . . . . . . . . . . . . . . . .

$92,416,000

82,901,333

78,150,000

62,377,000

60,819,000

56,700,000

37,700,000

34,423,000

34,400,000

30,045,000

26,800,000

25,317,500

24,858,000

23,426,000

22,575,000

21,740,000

20,259,333

17,515,000

15,920,000

15,020,000

13,050,000

11,700,000

11,100,000

11,000,000

9,900,000

9,321,000

8,810,000

6,100,000

5,700,000

5,304,000

5,000,000

4,350,000

3,750,000

3,690,000

3,550,000

3,250,000

1,800,000

1,450,000

1,200,000

517,000

450,000

315,000

290,000

225,000

195,333

50,000

$905,429,999

62.9

76.5

86.5

92.8

96.7

4

2

26

16

41.6%. 7

30

33

43

22

34

6

35

27

40

25

1

44

38

12

29

17

5

41

23

9

32

21

28

10

37

31

3

14

47

98.9 11

8

20

39

45

99.8 13

18

36

42

19

99.9 51

100.0% 49

aNoearmarks were identified forthe Statesof Maine,Tennessee, Delaware, Colorado, orwyoming.

bRanKngin termsof Federal R&Dexpend~ures at doctorate-granting institutions.

SOURCEOFFEDERALRANKING: NationalMe~eFoutiation,~&mic~kn~/EngjneetingR&D~un~s, I%cal

Year 1988, NSF 89-326 (Washington, DC: 1989), p.32,tableB-24.

SOURCE: JamesSavage,Officeofthe President, UniversityofCalifornia, ‘7he Distribution ofAcademic Earmarksin

the Federal Government’s Appropriations Bills, Fiscal Years 1980-1990:’ mimeo, Mar. 7,1989, table2.
Chapter 3—The Federal Research System: The Executive and Legislative Branches ● 93

Figure 3-2—Apparent Academic Earmarks by

State and at Universities and Colleges:

Fiscal Years 1980-89

Cumulative distribution of academic

earmarks, by State: FY 1980-89

Millions of dollars

800 -

600 -

400 -

200 -

0

94%

—

97% 99% ‘ 10 0%

1 1

‘o 5 10 15 20 25 30 35 40 45 50

Number of States

Cumulative distribution of academic

earmarks at universities and colleges: FY 1980-89

Millions of dollars

“ooo~

600

K

59%

400

200

0

82%

92% 96% 97% I

1

98% 100%

1 ,

0 20 40 60 80 100 120 140 160 180 200

Number of universities

SOURCE: James Savage, University of California, Office of the President,

“The Distribution of Academic Earmarks in the Federal Government’s Appropriations Bills, Fiscal Years 1980-1989,” mimeo,

Mar. 7, 1989, tables 3 and 6.

tration that seeks earmarks is also engaged in a

broader campaign to strengthen the research mission

of the institution?

At present, there are no answers to these questions. 72 Nevertheless, data could be collected and the

effect of earmarking evaluated over time.73 If the

results of these studies show that science performed

in earmarked projects or with earmarked facilities or

equipment is markedly inferior to other research

projects supported under other agency programs,

then steps could be taken to isolate problems

inherent in earmarked projects. On the other hand, if

these studies show that earmarked projects have an

impact on research that is equal to (or even greater

than) other projects supported through executive

branch programs, then perhaps some of the concern

over congressional earmarking is misplaced.74

Conclusions

This chapter has presented an overview of the

highest level of decisionmaking for research in the

Federal Government. Both the executive branch and

Congress attempt to respond to changing national

needs and potential research opportunities. However, due to their respective political agendas, modes

of organization, and spheres of responsibility, they

often disagree about the appropriate Federal role to

pursue them.

The President, OMB, OSTP, Congress, and interest groups have separate roles in the decisionmaking

process. They differ primarily in their concerns and

priorities. For example, OMB is mostly concerned

with fiscal issues, whereas OSTP is more concerned

with coordination and comprehensiveness. Thus,

long-term budgetary planning is very difficult.

In particular, the “research budget” is rarely

considered as a whole in the Federal budget process.

Separate parts of what might be considered the

research budget are contained in many different

budgets. Consequently, issues of concern to many

parts of the research system are not considered

across-the-board. “Nowhere in government is the

7%stitutions that have won earmarked money say its biggest impact is on their ability to recruit talented scientists. But it may also help to relieve

the squeeze on research space, and in generat upgrades the technicat capabilities of the researchers involved. See Colleen Cordes, “Congressional

Practice of Earmarking Federal Funds for Universities Offers Both Promise and Peril,”The Chronicle of Higher Education, vol. 36, No. 17, Jan. 10,

1990, pp. A19, A24. It is too soon to tell how the quality of research produced at facilities created through earmarked funds compares with the research

emanating from exclusively peer-reviewed, project-based academic centers.

TsCrow, Op. cit., footnote 65, points out that: “A smle earmarkcanprovide auniversitywiththe opportunity. . .for the development of relationships

and personal acquaintances that might yield nonesrmarked collaboration with that Federal agency in the future. A singleearmark might provide the

opportunity to develop new and continuing relationships with business and industry or State government. . . . Thus, while a university’s total research

funding may increase only marginally over a 5- to lo-year period (less than 5 percent) as a result of an earmark, the earmark might still have had

substantial impact because of its impact on a specific program (e.g., a 50-percent increase in competitive fimding over a 5- to 10-year period).” Also

see “How Iowa State University Wins Millions in Earmarked Funds,” The Chronicle of Higher Education, vol. 37, No. 24, Feb. 27, 1991, p, A21.

T4HOWeVer, the problem of regional inequity will remain regardless of e~ks. This could be addressed by both the executive branch and Congress.
94 ● Federally Funded Research: Decisions for a Decade

Photo credit: Michael Jenkins

The Senate Committee on Energy and Natural Resources holds a public hearing.

big picture considered; nowhere is the overall health process. Earmarking is also one visible, albeit minor,

of U.S. science and technology a primary mission or means of congressional budget negotiation. In the

responsibility. ’75

next chapter, OTA introduces the major research

An important aggregation of the research budgets agencies, their priority setting for research, and

could occur in the congressional appropriations funding allocation mechanisms.

 op. cit., footnote p. 
CHAPTER 4

The Federal Research System:

The Research Agencies
Contents

Page

Introduction . . . . . . . . . . ... ... ..*. ... ... ...............*+.....+.....""`"'*'""'""''"""`""" 97

Priority Setting in the Federal Agencies............++..”” .. .. .. ++ .9 ’. .” ”++OO+SO”. S’” +“-”. 99

National Institutes of Health ... .......+.! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...+. . . . . . ..... 99

Department of Defense.. . . ., .+ +.. . . . . . . . . . . .. . .. .. ... .......+. .“....-...++*Q’””. ...’” ~~~

National Aeronautics and Space Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...+.,

Department of Energy .. .. .. +. .. .+ +.. . . . . . . . . . . . .t. .. .. .. .. .. do+++...-...-’++ “+.+..”” ~~

National Science Foundation .. .. .. ... . .......4. .+ .. .. ... . ”to ..-..bo++gm ..’.+..

Department of Agriculture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

Other Agencies .. .. .. . ... .++.... .+ . . . . . . . . . . . . . . . . . . . . .......’”+....-”. “-.~”++.+..’. 116

Crosscutting Descriptions of Agency Priority Setting . . . . . . . . ,,.,,,,.. . . . . . . . . . . . . . . . . . . ..*116

Division of Labor . . . . . . . . . . . . . . . . . . . . . . . . . ..............-.++.+T”... “-.s++”o..+==.+== 117

Coordination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......... ..*..+.** ● ****.*’. 7“..0

”””

Bottom-Up and Top-Down Management and the Use of External Advice .... ... ++.. . . . . . . 117

Intramural and Extramural Research .. .. .. .. .. .. ... ..+. . . . . . . .........~....~..-,.”=~”+ + 119

Issues of Agency Priority Setting .. .. ..+. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

Risk-Taking and Conservatism .. ...............................4.. ~..,.........~cs~~ ● + 121

Flexibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . +..+....+= ...*.*.*.* ● ++=..**.* * 121

Strategic Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..+++.. ●.* 123

Redirecting the Agencies and Addressing New Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Funding Allocation in the Federal Agencies .. . .+ . . . . . . . . . . . . . . .. ...+.........’.+++. . . . . . . 124

Peer Review. ..+. . . . . . . . . . . . . . . . . . . .. .+ . . . . . . . . . . . . . . .. .. +. +. . . ......++” m”... ~ 125

Manager Discretion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....*.+. . . . . . . . . . . . . . . . . *.....*. . . . 125

Agency Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .............-.~’++-.””.. ----- 126

Blurring of Peer Review and Manager Discretion . . . . . . . . . . . . . . . . . . . . . . . .**..... ...***.. 127

Set-Asides and Formula Funding .. .. .. .. .+ +. .. ... ... +...... ,. .. .. .. .. ~. ”., ++. ... .-~.”’ ~~~

summary . . . . . . . . . . . . . . . . . . . . . . .+.....+ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Boxes

Box Page

4-A+ OTA Interviews at the Federal Agencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

4-B. The Defense Advanced Research Projects Agency . . . . . . . . . . . . . . . . . . . . . . . . . 106

4-C. Coordination of Superconductivity R&D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .+.* 118

4-D. Small Grants for ExploratoryResearch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

4-E. Fine-Tuning Project Selection at NIGMS . . . . . . . . . . . . . . . . . . . . . . . . . .....4* * 128

4-F. Small Business innovation Research Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

4-G+ The NSF ESPCoR Program: Geography and Research Capability . . . . . . . . . . . . . . . . . . . . . . 131

Figures

Figure P

4-1.ResearchObligations in the Major Research Agencies: Fiscal Years 1960-90...,

&2. Basic and Applied Research Funds for DOD: FiscalYears 1960-90 ... .. .. .,. ... +..+.... ~~

4-3. Basic Research in DOD by Service: Fiscal Years l969-90 . . . . . . . ......”* ....+.., . . . . . . . 103

4-4. Applied Research in DOD by Service: Fiscal Years 1969-90 . . . . . . . . . . . . . . . . . . . . . . . ..++ 103

4-5. Basic and Applied Research Funds at NASA: Fiscal Years 1960-90 . . . . . . . . . . . . . . . . . . . . . 107

&6, Bacic and Applied Research at the Department of Energy: Fiscal Years 1980-90. . . . . . . . . . 109

Tables

Table Page

4-l. National Institutes of Health History of Congressional Appropriations: Fiscal Years

1970-90 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

4-2 National Science Foundation Obligations: Fiscal Years 1952-90 ... .. .. .. ... ... .......+ + 112

43. Trends in Federal Obligations for Total Research and Development by Major Agency:

Fiscal Years 1955-88 ...+.,. .,..+... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..**...* ..*..** 114

.+....,. . . .

..,.+... . . .

. . . . . . . . . . .

.+....,. . . .

..,,.... . . .
CHAPTER 4

The Federal Research System: The Research Agencies

Introduction

The effective management of the Federal research

system depends on the quality of the research

agencies and their staff. Over the last 30 years, as

research budgets and the system have grown in size,

the importance of these agencies in decisionmaking

has increased.

Each agency has its own culture, which contributes not only to its success, but also embodies

historically the ‘‘way things are done. ’ Agency

culture is thus a powerful determinant of future

directions, with specific goals reflected in the

collective knowledge of agency personnel. Pluralism and decentralization characterize each of the

research agencies, with many separate programs

pursuing diverse objectives. In particular, the lines

of decisionmaking within an agency are more

complicated than any organizational chart would

suggest.

In preparing this report, OTA selected the six

Federal agencies that fund most of the Nation’s

research. They are, in the order of their fiscal support

of research (including basic and applied): the

National Institutes of Health (NIH), the U.S. Department of Defense (DOD), the National Aeronautics

and Space Administration (NASA), the U.S. Department of Energy (DOE), the National Science Foundation (NSF), and the U.S. Department of Agriculture (USDA)l (see figure 4-l). OTA reviewed

historical budget figures for these agencies and

conducted inperson interviews with agency personnel, ranging from top administrators, who interpret

and set annual research priorities, to program

managers, who disburse the funds. The interviews,

125 in all, yielded information on goal setting in the

agency, proposal review, and methods of allocating

funds. Interviews with National Academy of Sciences (NAS) staff (who are commissioned by

research agencies to perform studies that will

enhance decisionmaking) augmented the agency

descriptions (see box 4-A).

OTA found that the research agencies generally

attempt to follow their missions, as outlined in their

founding charters and in subsequent legislation.

However, congressional and executive views diverge on what is included in missions. There is also

disagreement at many agencies over what constitutes a thoughtful, fiscally prudent, and expeditious

Figure 4-l-Research Obligations in the Major

Research Agencies: Fiscal Years 1960-90

(in billions of 1982 dollars)

Billions of dollars

6

5

4

3

2

1

0

1

t

)

1960 1965 1970 1975 1980 1985 1990

— N I H + DO D --%-- NASA

+ DOE + NS F + USDA

KEY: DOD - U.S. Department of Defense; DOE = U.S. Department of

Energy; NASA = National Aeronautics and Space Administration;

NIH = National Institutes of Health; NSF = National Science

Foundation; USDA = U.S. Department of Agriculture.

NOTE: Research includes both basieandapplied. Before 1989, obligations

for NIH were not broken out in this source. Figures were converted

toeonstant 1982 dollars using the GNP Implicit Price Deflator. 1990

figures are estimates.

SOURCE: National Sderwe Foundation, Federa/ Funds for Research and

Development, Detailed Historkal Tables: fiscal Years 1955-

1990 (Washington, DC: 1990), table A; and National Seienee

Foundation, Selected Data on Federal Funds for Research and

Development: fiscal Years 1989, 1990 and 1991 (Washington,

DC: Deeember 1990), tables 4 and 5.

IToge~er mew agencies WIpply roughly 95 permnt of the Federal research budget. See Albert H. ‘Ikich ~d Kathleen G~P, R= in f~e J980s:

A Special Report (Washington, DC: American Association for the Advancement of Science, September 1988).

-97 -
98 ● Federally Funded Research: Decisions for a Decade

Box 4-A-OTA Interviews at the Federal Agencies

For this study, OTA sought data on the research goals of the six major Federal agencies that fund basic research:

the U.S. Department of Defense (DOD), the U.S. Department of Energy (DOE), the National Institutes of Health,

the National Aeronautics and Space Administration, the National Science Foundation, and the U.S. Department of

Agriculture. In addition to collecting budget data, OTA performed 125 interviews with agency personnel ranging

from top administrators who interpret annual budget priorities to program managers who disburse the funds.

Interviews at the National Academy of Sciences and the Office of Management and Budget augmented the agency

interviews.

Discussions centered on decisionmaking, priority setting, and funding allocation mechanisms. Typical

questions that were asked included:

1. What are the stated goals for agency research monies and programs? What goals are not stated, but are

implicit in the agency’s mission? How have these goals changed since the 1960s? the 1970s? the early

1980s?

2. What processes (both formal and informal) are used in the agency to set priorities and goals for research

monies? How has this process changed in the last 20 years?

3. How do new directions in research that are not anticipated get funded?

4. Does Congress set goals for the money that the agency allocates? Has this changed over time?

5. How do agency divisions coordinate with other parts of government?

6. What mechanisms are used to allocate funds? How do these mechanisms differ for extramural and

intramural funding?

To illustrate the scope and depth of the interviews at the agencies, the interviews conducted at DOE can be used

as an example. Interviews were conducted in five offices under the Secretary of Energy, and one laboratory was

chosen as a case study. Excluding those interviewed at Los Alamos National Laboratory (LANL), four directors

of offices at DOE headquarters, eight division directors, and five program managers were interviewed. Since the

Office of Energy Research (OER) is the primary supporter of basic research at DOE, the Executive Director of OER,

six division directors, and four program managers were interviewed. The Director of the Office of Weapons

Research, Development and Testing; the Deputy Assistant Secretary for Renewable Energy; and one program

manager in the Office of Conservation and Renewable Energy were interviewed. In addition, departmentwide

priority setting was discussed with the Under Secretary for Policy, Planning, and Analysis, and the chief planner

for research. Finally, budget data were discussed with the Deputy Director for Research in the Office of the Budget.

At LANL, OTA staff toured the facility and interviewed the Deputy Director of LANL, the Director of the Meson

Physics Facility, and the Deputy Director of the Health Research Laboratory, as well as a number of scientists and

other members of the staff.

In all agencies, the offices that support research were identified, as well as those that participate in

departmentwide planning. In addition, one or more inhouse laboratories were chosen for site visits. Summaries of

the interview results were prepared and distributed to all interviewees for comment (with the exception of DOD,

where a smaller set of reviewers was selected). Because the number of people interviewed had to be limited, the

analysis sought only to illu minate the structure and diversity that characterizes executive branch decisionmaking

in research. The table of organization was sampled to capture various perspectives on decisionmaking within and

across the research agencies.

SOURCE: Office of Technology Asscssment, 1991.

strategic plan to attain specific goals. Goals at the funds), and of what their future directions and needs

‘‘macro level’ (e.g., a Presidential call for more will be. However, programs managers must often

research and development (R&D) in a specific area) make tough decisions within limited budgets about

do not necessarily map neatly into agency missions, who to fund, whether to provide money for instruand some macro level goals cannot be addressed mentation or personnel, and whether to favor disadthrough current agency structures. vantaged groups such as women, minorities, and

young investigators. Competing goals of education,

Agencies also have a good sense of their research equity, and economic activity must be weighed in

constituencies (i.e., the scientists that receive agency every program.
Chapter 4--The Federal Research System: The Research Agencies ● 99

OTA found that peer review, manager discretion,

and combinations of these methods are used by the

research agencies to distribute funds. Since the

beginning of the 1980s, the distribution patterns of

research funding have been under great scrutiny. It

is not only a matter of who should receive the funds,

but how they are allocated (e.g., individual investigator grants or block grants to centers, short-term or

long-term projects).

In this chapter, the major research agencies are

described, their priority-setting mechanisms outlined and compared, and their funding allocation

mechanisms discussed. Agency planning efforts and

direction from other agencies of the Federal Government and the scientific community are analyzed.

Priority Setting in the

Federal Agencies

Federal agencies initiate, manage, and terminate

programs. At each step in the process, agency

personnel must decide which program, or component of a program, will take precedence. What

follows is a brief introduction to each of the major

research agencies and to their priority-setting

mechanisms for research. The agencies are presented in descending order of their annual research

budgets.

National Institutes of Health

NIH is the largest research agency in the Federal

Government in terms of dollars awarded to basic and

applied research. It is the principal biomedical

research arm of the U.S. Department of Health and

Human Services (HHS), funding biomedical and

basic research related to a broad spectrum of

diseases and health problems both in its own

research facilities (the NIH laboratories) and in

external organizations.

The missions of the institutes are reflected in their

titles. There are categorical, or disease-oriented,

institutes, such as the National Cancer Institute

(NCI) or the National Heart, Lung, and Blood

Institute (NHLBI). And there are institutes with a

population-based research focus that is population

based, such the National Institute on Aging. The

exception to these categorizations is the National

Institute of General Medical Sciences (NIGMS),

which has no targeted responsibility other than

general basic research.

Photo credit: Research Triangle Institute

Researcher trains hospital staff for a National Institutes

of Health (NIH)-sponsored clinical trial of medical

therapies. Clinical trials are an integral part of

NIH applied research.

NIH is part of the Public Health Service, so its

work is very much tied to public health issues.

Although NIGMS is devoted primarily to basic

research, the categorical institutes conduct a range of

research from basic to applied to development. For

example, NCI’s mission is to implement programs

on the cause, diagnosis, prevention, and treatment of

cancer. Often, the missions of the institutes overlap

with each other or with other agencies. In these

cases, a lead institute will coordinate. Because many

of the institutes are categorical, NIH and Congress

tend to set their research agendas epidemiologically,

focusing their mission on diseases of highest prevalence. Critics of NIH question this approach, saying

that it focuses the research agendas too much on the

diseases of the majority, skewing research that could

lead to health improvements in other areas.

Since the 1960s, the goals and justifications for

health research have been fairly constant-improving the health of the American people, curing

particular chronic diseases, and contributing to the

economic well-being of the Nation by producing a

healthier work force. However, particular emphases

have shifted. During the early 1960s, mental retardation was emphasized by President Kennedy. In the

1970s, cancer and heart disease, which had been

prominent research areas for decades, became even

more important as President Nixon declared the War

on Cancer in 1971. Vast sums of money were
100 ● Federally Funded Research: Decisions for a Decade

dedicated to attempts to eradicate these families of

disease with mixed success. Although levels of

funding remained high, by the late 1970s, the role of

the environment in creating and reducing cancer risk

replaced the earlier research focus on viral etiology

and understanding cellular mechanisms.2

During the mid- 1970s, the discovery of recombinant DNA shifted the emphasis of research once

again, this time to biotechnology, which received

increasing attention throughout the Reagan Administration. Most recently, treating and curing AIDS

has been a dominant goal of NIH research. It first

appears in the 1983 NIH budget authorization

testimony,3 and every year since then AIDS has

received the largest increases in research funding

within the NIH budget.

The fiscal year 1991 appropriation to NIH was

just over $7.4 billion. NCI has the largest appropriation at $1.7 billion, followed by NHLBI at $1.1

billion. The National Institute on Deafness and

Other Communication Disorders had the smallest

appropriation at $135 million. See table 4-1 for budget

histories of the various institutes from 1970 to 1990.

Each institute has an advisory council, which is

appointed through HHS and is made up of scientists

and lay people. Program officers must go before the

council to present ideas for new programs, and

councils review program balance. Each institute

may also form advisory committees with programmatic foci; for example, NHBLI has six committees

to assist in specific fields. Committees help develop

new initiatives. It should be noted, however, that the

council is only advisory, except for its ability to

approve or disapprove grant applications.

When institute staff notice evidence of an emerging area of research, they assess the importance of

the new field and gauge interest and capabilities.

They can then convene a meeting or workshop, write

up a proposal for a new program, and go to their

council for approval. If the program does not have a

known constituency, an institute will often issue a

request for applications.

Some observers have criticized NIH in its response time to new research needs, such as AIDS

and the Human Genome project. On the other hand,

some scientists said NIH responded too quickly with

its AIDS agenda. Interestingly, AIDS was incorporated into the existing NIH structure, with the

National Institute of Allergy and Infectious Diseases

(NIAID) taking the lead. The Human Genome

project, which some argued belonged in NIGMS,

was placed in the Office of the Director. Both

approaches have been simultaneously criticized and

hailed.4 To date, there has been no mechanism for

centralized planning at NIH. However, NIH, for the

first time, is developing a strategic plan that cuts

across all institutes. In addition, each institute

submits its annual plan to the Office of the Director

along with the budget. However, the Director has

little authority to redirect the agenda of any institute.

Through the budget process, Congress provides a

coordinating function.

Despite growth in funding over the last decade,

NIH views itself as being in a “steady state” and

under enormous strain. After experiencing phenomenal growth (virtually a doubling of the budget in

real terms during the 1980s), including an intramural

budget that exceeds $900 million today, managers

still feel they must juggle priorities, reorient existing

programs, and make small, incremental changes in

other programs-both intramural and extramural.5

But there are exceptions: NIAID rose from seventh

place among institute budgets to third place (from

%icluud A. Rettig, Cancer Crusade: The Story of the National Cancer Act of1971 (Princeto@ NJ: Princeton University Press, 1977); and Stephen

Strickland, Politics, Science, and Dread Disease (Cambridge, MA: Harvard University Press, 1972).

JwkPollack, “Basic ResWch Goals: Perceptions of Key Pditicd Figures, ” OTA contractor repo~ June 1990. Available through the National

‘lkchnical Information Service, see app. F,

4With matrix managemen~ some National Institutes of Health staffers said (in OTA interviews during the spring of 1990), the best way to respond

to a new research initiative is to create a new associate director in the OffIce of the Director to coordinate efforts among the institutes. Also see Institute

of Medicine, The AIDS Research Program of the National Institutes of Health (WaShingtO~ DC: National Academy Press, 1991); and Janice Long,

“AIDS Research: More Funds, Coherent Strategy Needed, ” Chemical & Engineering News, vol. 69, Mar. 11, 1991, p. 4.

s~oblas cited as ~se~ the Natio~ ~ti~tes of He~th (N@ me: noncompetitive wages (espec~y for young researchers), hlCNMed

politicization (notably over fetal tissue research and the use of animals for experimentation), “accountability fever” (centering on congressional

investigations of purported misconduct in rese~ch and complaints about NIH’s own process of inquiry), excessive paperwork to document

research-related decisions, and lack of direction (the difference between an “acting’ and apnxidentially nomina ted, Semte-approved director). See Rick

Weiss, “NIH: The Price of Neglec~” Science, vol. 251, Feb. 1, 1991, pp. 508-511. The confiiation in March 1991 of Bernadine Healyas NIHDirector

filled a vacancy that existed since August 1989. See Larry ThompsoU “NIH Gets Its First Woman Director,” The Washington Post, Health sectio~

Mar. 26, 1991, p. 8.
Chapter 4-The Federal Research System: The Research Agencies ● 101

tY)cn T-o *

C6aib o-i

C9. mab to m(r). .

/3

0

aJ

tmolc90lU5ti(o=rk a)03m*co CUC9N..

77375% Cc cc c Q .~.Q .Q.Q ....+ (ula(u(um Zzzzz

~1.Qb.Y.y

Lolncncoml tol-f=cno2 l-co C9ml

.

&

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

.

.

.

El

z
102 ● Federally Funded Research: Decisions for a Decade

almost $375 million to over $907 million) between

fiscal years 1984 and 1991. AIDS research funds

account for this increase, with one-half of the current

NIAID budget devoted to research on the disease.

AIDS funds are new money, and are not taken from

other biomedical budgets.

NIH is perpetually struggling to balance its

research and public health missions. Some institutes

are more responsive to one or the other, and therefore

are oriented to individual investigators (and basic

research) or centers (and clinical, disease-focused

work), but all seek to serve both missions. What has

been especially inconsistent is the NIH approach to

public health or research crises. As noted above,

while the AIDS initiative was assigned to one

institute, Human Genome, originally assigned to the

Office of the Director, is now a separate National

Center for Human Genome Research that is not part

of the Office of the Director. Many argue that one

approach or the other responds better to crises while

remaining supportive of, and responsive to, developments in basic research. Perhaps new efforts aimed

toward a strategic plan could be used to better

address public health and research crises.6

Department of Defense

DOD is the second largest source of basic and

applied research funds in the Federal Government.

Justification for defense R&D throughout the last

three decades has been to stay ahead of the Soviet

Union in the development of new military technologies. However, defense research is also inextricably

linked to the expansion of knowledge and bolstering

the overall U.S. technological base.

The military funds research through three categories: 6. l—research of the most fundamental nature;

6.2—applied research and exploratory development; and 6.3A—the initial stages of advanced

development.7 Research within DOD can be characterized by two phrases: “technology-push” and

‘‘requirements-pull. Knowledge gained from research creates areas for potential advancement, some

of which were unforeseen when the research began.

Figure 4-2—Basic and Applied Research Funds for

DOD: Fiscal Years 1960-90 (in billions of 1982 dollars)

Billions of dollars

‘~

1 -

01 I I I I I I I I 1 I 1 I I i I I 1 I 1 I I I I I I I I I I 1960 1965 1970 1975 1980 1985 1990

+ Basic research + Applied research

NOTE: Figures were converted to constant 1982 dollars using the GNP

Implicit Price Deflator. 1990 figures are estimates.

SOURCE: National .Seience Foundation, FedWa/FunA for Research and

Development, Dehiled Historka/ Tables: 17scal Years 1955-

1990 (Washington, DC: 1990), table A; and National Seienee

Foundation, Selected Data on Fedetal Funds for Researctr and

Development: t%cal Years 1989, 1990 and 1991 (Washington,

DC: Deeember 1990), tables 4 and 5.

This new knowledge nudges the system to incorporate new ideas and thereby gain a greater level of

capability (technology-push). At the same time,

identified needs define areas for research and

technological results to enhance the military. These

requirements shape the directions of research and set

the level of effort to be pursued (requirements-pull).

Figure 4-2 presents the basic and applied (corresponding roughly to 6.1 and 6.2) research funds

authorized for DOD from 1960 to the present.8

Figure 4-3 graphs basic research funding in constant

%ee ktitute of Medicine, Futiing Health Sciences Research: A Strategy to Restore Balance, Floyd E. Bloom ~dw A. Rmdolph (eds.)

(WashingtorL DC: National Academy Press, November 1990).

TThe rest of the 6.3 categow is devot~ to more advanced development. U.S. Congress, Oftlce of lkchnology Assessment, Holding the Edge:

Maintaining the Defense Technology Base, OTA-ISC-42C) (Washingto~ DC: U.S. Government Printing OffIce, March 1989).

S’rhese dab were collect~ by the Natioti Science Foundation (NSF). Although some of the research agencies report problems with the NSF s~ey

(discussed below), the Department of Defense already categorizes its research with 6.1 and 6.2 budget designation. Funds reporkxi as “basic” and

“applied” correspond to 6.1 and 6.2 funds, and their interpretation is fairly straightforward.
Chapter 4-The Federal Research System: The Research Agencies ● 103

Figure 4-3-Basic Research in DOD by Service:

Fiscal Years 1969-90 (in millions of 1982 dollars)

Millions of dollars 350

300

250

.

200

150

100

50

0, I I I I I I I I I I I I I I I I I I I I

Figure 4-4—Applied Research in DOD by Service:

Fiscal Years 1969-90 (in millions of 1982 dollars)

Millions of dollars 1200 f

>

1000

800

600

(

400

200

0!

\

I I 1 I I I I I I I I I 1 I 1 I I I I

1969 1974 1979 1984 1989

— Arm y + Nav y + Air Force

KEY: DOD E U.S. Department of Defense.

NOTE: Figures were converted to constant 1982 dollars using the GNP

Implicit Price Deflator. 1990 figures are estimates.

SOURCES: National Science Foundation, Federa/Funb forResearch and

Development, Detailed Historical Tables: fiscal Yeafs 1955-

1990 (Washington, DC: 1990), table A; and National Science

Foundation, Selected Data on Federal Funds for Research

and Development: Fisca/ Years 1989, 1990 and 1991 (Washington, DC: December 1990), tables 4 and 5.

dollars for the three services, and figure 4-4 presents

applied research funds by service from 1969 to the

present. Compared with basic research funding for

other research agencies, the services show remarkably little fluctuation in allocated finds, adjusted for

inflation. The Navy has been consistently awarded

more funds than either of the other services, roughly

twice that of the Army or Air Force. Although

applied research funding decreased in the late 1960s,

it has remained even more constant than basic

research in the 1970s and 1980s, and the three

services have received almost identical levels of

funding for applied research since 1970. Basic and

applied research is also supported by the Strategic

Defense Initiative Organization (SDIO-although

SDIO funds are technically categorized as 6.3A) and

the Defense Advanced Research Projects Agency

1969 1974 1979 1984 1989

— Arm y + Nav y + Air Force

KEY: DOD = U.S. Department of Defense.

NOTE: Figures were converted to constant 1982 dollars using the GNP

Implicit Price Deflator. 1990 figures are estimates.

SOURCES: National Science Foundation, Fedem/FundsforResearch and

Development, Detailed Historical Tables: I%sca/ Yeats 1955-

1990 (Washington, DC: 1990), table A; and National Science

Foundation, Selected Data on Federal Funk for Research

and Development: Fiscai Years 1989, 1990 and 1991 (Washington, DC: December 1990), tables 4 and 5.

(DARPA). These two organizations often cooperate

with the three services to fund and operate research

projects.

As recently stated by the Congressional Research

Service:

Although military basic research funding totals

almost $1 billion annually, it (together with military

applied research funding) has decreased since the

mid- 1960s in real dollar terms and relative to

increases in total research, development, testing, and

evaluation. Despite recent congressional action to

increase military research budgets, executive branch

decisionmakers have not sought large increases for

research funding. As a result, critics say, too much

attention goes to weapons development and too little

to “creative” science needed to produce knowledge
104 . Federally Funded Research: Decisions for a Decade

vital to U.S. national security. Some allege that

because of funding cutbacks the quantity and quality

of military research may be decreasing.9

Recently each of the services concluded that a

published strategic plan, explicitly covering their

individual projections for the future technology

base, would both aid in policy formulation and

positively influence the budget for research. In the

Air Force, it was called ‘Project Forecast II;’ in the

Navy, “Navy 21;” and, in the Army, “Army 21.”

Some of these plans take into account the “new

reality” for the future: the decreasing likelihood of

a European war, the increasing likelihood of lowintensity conflict (especially in the Third World

and/or connected with drugs), increasing global

economic and technological competition, the decreasing U.S. defense industry and R&D base, the

decreasing supply of U.S. citizen scientists and

engineers, and finally, decreasing defense budgets.

Based on this future scenario, the plans identify key

emerging technologies and areas for enhanced

research.

The three services differ in the degree of centralization in the dispersal of 6.1 money. In the Air Force

and the Navy, almost all 6.1 research monies flow

through the Air Force Office of Scientific Research

(AFOSR) or the Office of Naval Research (ONR),

respectively. In the Army, 6.1 funds are more

decentralized.

Army

Much of Army research is closely linked to

priority setting for all of the R&D funds in the

Army’s laboratories and institutes. Laboratories in

the Army act independently, although they determine priorities in relation to overall directives from

Laboratory Command. With this independence

comes requirements for a high level of accountability, and laboratories are reviewed regularly. Most are

‘‘industrially funded’ ‘-competing for funds from

sources within and without the Army.

In addition, the mission of the Army Research

Office (ARO) is to “. . . develop the Army Materiel

Command research program for mathematics, and

the physical, engineering, atmospheric, terrestrial,

and biological sciences according to Army-wide

requirements. 10 Eighty-three percent of the research contract program monies go to universities,

10 percent to industrial laboratories, and 7 percent to

Federally Funded Research and Development Centers and not-for-profits.ll ARO receives guidance

from its parent Army Materiel Command, which

provides focus to its research programs. ARO has

also come to rely on informal types of outside input,

especially from the scientists that it supports. The

Medical Research and Development Command

recently developed an Army Medical Technology

Base Plan, which provides guidance to the medical

research community within the Army. Finally, the

mission of the Army Research Institute for the

Behavioral and Social Sciences is to focus on

"

. . . the acquisition, training, development, utilization and retention of the Army’s personnel resources. 12 Three laboratories and many university

contracts support this goal.

Navy

Almost all of the 6.1 research dollars in the Navy

are disbursed by the Office of Naval Research. Over

one-half of ONR funds go to universities, one-fifth

to ONR laboratories, over 10 percent to other Navy

laboratories, and the final 10 percent to industry and

other government research organizations. ONR

funding is spread among disciplines, with a little less

than one-half devoted to areas of explicit Navy

emphasis, such as ocean and atmospheric sciences,

computers, and materials. Other areas of support are

linked closely to broader defense interests: astronomy and astrophysics; biological, medical, cognitive, and neural sciences; general physics, chemistry, and mathematics; and energy conversion, radiation sciences, and electronics.

In addition to Navy 21, ONR relies on inhouse

personnel (including personnel from the ONR laboratories), foreign field offices, and outside experts

and panels (including NAS) to help set priorities.

This type of planning is relatively new for the Navy.

Before 1970, a primary research criterion was the

quality of the science. Most of the research was not

sGenevieve J. ~e~, “Defense Basic Research Priorities: F~ding ~dpoficy ~slles,’ CRS Repotifor Congress (Wasl@gtoU DC: Congressional

Research Semice, Oct. 24, 1990), p. 38.

lo~y Resemch Mice brief~ materials prepared for om, my 1990.

‘lIbid.

121bid.
Chapter 4-The Federal Research System: The Research Agencies ● 105

multidisciplinary and minimal advice was requested

from the external scientific community. Now, due in

part to the 1970 Mansfield Amendment, mission

relevance is a strong criterion; multidisciplinary

programs are enhanced and are greater in number;

some programs can be put on a “fast-track”; and

substantial input is sought from the external scientific Community.

Air Force

Before 1974, inhouse Air Force laboratories

controlled most 6.1 monies. After 1974, the Air

Force consolidated the disbursal of 6.1 monies into

one unit, the Air Force Office of Scientific Research.

Each laboratory still has a portion of 6.1 monies, but

the bulk are distributed by AFOSR. Air Force

laboratories compete for these funds along with

universities and other performers. In addition to

Project Forecast II, each year key personnel in the

Air Force research system and managers of the

science and technology areas discuss the “macro

strategy” for the next year. A report is then sent to

the separate parts of the Air Force research system,

such as AFOSR, which reinterprets its programs in

terms of these goals.

Even though there is a significant amount of

‘‘top-down’ direction in the distribution of Air

Force 6.1 money, it is still primarily a bottom-up

process. The influence of top-down management is

viewed as adding discipline to the management of

research programs, which still respond primarily to

scientific community concerns about the direction of

research. The balance of top-down and bottom-up

management seems intermediate to that in the Army

where the management is more decentralized and

bottom up, and to the Navy where ONR provides

greater top-down management.

DARPA and SDIO

Project selection in research at DARPA is very

different from that at other DOD research agencies.

Project managers state that they are not attempting

to maintain strength across a field, rather they are

funding good ideas that are on the forefront of

technology development to meet desired objectives

(see box 4-B).

In the Strategic Defense Initiative Organization,

the Innovative Science and Technology Office

(ISTO) is the core unit that funds basic and applied

research for SDIO. Within the overall mission of

developing space surveillance, weapons, and communications technologies, ISTO determines future

directions for research. ISTO includes an eightperson research management team, which sets goals

and works to see that these goals are achieved. The

measure of success is ISTO’s impact on SDIO.

At present the three services, SDIO, and DARPA

set their own research agendas, gaining the usual

advantages of pluralism. In a previous report, OTA

also found disadvantages to pluralism, which included ‘‘. . . wasteful duplication of efforts, lack of

critical mass to solve common problems, fractionated efforts, and inattention to areas that are on no

component’s agenda. It also risks failing to identify

areas of common or overarching significance. ’ ’13 In

a mission-oriented organization like DOD, these

disadvantages seem too large to ignore. OTA also

found previously that the inability to define the

products of research has limited DOD’s use of

quantitative decision support and evaluation methods like those used in industry .14

From 1989 to 1990, DOD prepared for a downturn

in funding. After a period of phenomenal growth in

the 1980s, DOD projected that such funding could

not continue, and that a real decline in funds was

therefore likely. DOD set in motion planning

activities to construct useful options in such a

funding scenario, such as the consolidation of

several research laboratories, and many of the

priorities embodied in these plans have been implemented in the DOD budget. The consequences of

these decisions have yet to be evaluated.

National Aeronautics and Space

Administration

NASA is now the third largest source of research

funds (both basic and applied) in the Federal

Government. NASA was created in 1958, 1 year

after the launch of Sputnik, and took over the

National Advisory Committee on Aeronautics’ laboratories. In fiscal year 1960, the research budget

was slightly over $0.5 billion (in constant 1982

ls~lce Of TWhnOIOgy Assessmen~ op. Cit., footnote 7.

WJ.S. Congress, Officeof Technology Assessmen~ ‘Evaluating Defense Department Researc~ background paper of tie tite~tiotiSectity ~d

Commerce Progrm June 1990.
106 . Federally Funded Research: Decisions for a Decade

Box 4-B—The Defense Advanced Research Projects Agency

Because the creation of new technologies is often interdisciplinary and involves risky research ventures,

President Eisenhower felt that “. . . a different type of organization was needed with unique business practices.”1

The mission of the Defense Advanced Research Projects Agency (DARPA), created m 1958, is to “. . . develop

‘revolutionary’ technologies that can make a significant impact on the future of the United States’ defense posture,

and to ensure that those technologies effectively enter the appropriate forces and supporting industry base. ’

The “unique business practices” that now prevail at DARPA involve program managers directly with the

projects that they fund and manage. Managers typically create a portfolio of research projects seeking particular

objectives, such as the use of Gallium-Arsenide in new micro circuitry developments, and follow them closely.

Programs are expected to last 3 to 5 years; the manager is given almost total discretion over funding allocation; and

the success of the manager and the program are judged by the results produced

Managers are also given discretionary money to pursue ideas for future programs, and every year new programs

compete for funding. DARPA stresses that this competition is based almost exclusively on the worthiness of a

particular idea, not on external considerations such as maintaining U.S. strength in a particular research field. Also,

DARPA’s contribution must be unique. An “inhouse rule” stresses that 80 percent of the funding in a particular

research area must come from DARPA. This targets DARPA’s investment in emerging research topics.

DARPA further stresses the importance of allocating enough funds for a project to see it through to completion.

Because of funding shifts, many agencies must compromise their programs and projects by allowing only partial

funding. At DARPA, programs and projects are routinely terminated to make way for others.

Among the agencies where OTA conducted interviews, DARPA is applauded as the only organization that can

effectively trade off agency programs and, if needed stop a project. DARPA allows less than 1 year to switch

program direction, whereas research managers in many other agencies state that it takes at least 2 years, and often

much longer, to achieve such redirection. DARPA relies foremost on program managers to determine when to halt

a program, which is hailed as a key to DARPA’s success.3

DARPA’s accomplishments in high-performance computing, solid state devices, advanced materials, and

many other areas have sparked much congressional interest. Attempts to model other agencies after DARPA,

particularly a “Civilian Advanced Research Projects Agency,” have concentrated on DARPA’s novel

organizational style.4 Congress could also consider instructing the Federal research agencies that do not already

have programs specializing in high-risk research to adopt select DARPA management techniques.

1Craig I. Fields, testimony at hearings before the House Committee On Armed Services, Subcommittee on Research and Development,

Mar. 1, 1990, p. 1. AiSO 5&! “’_fhc GoVtXlllll cut’s Guiding Hand: An Interview With E!x-DARPA Direetor Craig Fields, ” Technology Review,

February-March 1991, pp. 3340.

%bid., p. 1.

3S& m ~, 4’DMA Bets on Hi@-R.igk R&D,” Research & Developwwu, November 1989, pp. 3942.

4SM Semte md Howe bilk: S. 1978 and H. 3833; d Senate Comm.ittw on Gov crnmcnt Affairs, hearing on S. 1978, Trade ‘Ikdmology

and Promotion Act of 1989, June 12, 1990.

dollars) and, by fiscal year 1990, it had surpassed $2 ogy, and world affairs, and of expanding knowledge

billion (in constant 1982 dollars, see again figure

4-l).

Not unexpectedly, the primary focus of NASA

research from 1961 to 1969 was directed at achieving President Kennedy’s announced goal of landing

men safely on the Moon by the end of the decade.

The use of satellites for communications, meteorological observations and research, and Earth resource surveys were also persistent emphases. The

investment in space was justified on the basis of

perceptions of U.S. leadership in science, technolof the universe. As the economy tightened and the

lunar landing neared, the ostensible practical benefits of space research and of space-related technology received increasing emphasis. The end of the

Apollo program produced a need for new priorities

both to guide the agency’s activities and justify

continued high levels of funding. In the mid- 1970s,

the Space Shuttle began to move to center stage.

During the 1980s, research priorities at NASA

diversified. NASA began to emphasize commercial

uses of space (including industrial research), as well
Chapter #-The Federal Research System: The Research Agencies . 107

Photo credit: National Aeronauts and Space Administration

Scientists work on a mirror for the Hubble Space Telescope

(HST). Building the apparatus for any mission in space,

such as the HST, is complex and involves many

different components.

as the use of space for defense. In addition, NASA

initiated work on the Earth Observing System to

collect much more environmental data than had

previously been collected from space. Recently,

President Bush has also set a goal to return humans

to the Moon and explore Mars.

Basic and applied research management at NASA

is split between the Office of Space Science and

Applications (OSSA) and the Office of Aeronautics,

Exploration, and Technology (OAET). Data on

basic and applied research funding at NASA are

presented in figure 4-5.15 Over the last three decades,

basic research funding has oscillated slowly, between $600 and $800 million (in constant 1982

dollars). Applied research shows a more active

Figure 4-5-Basic and Applied Research Funds at

NASA: Fiscal Years 1960-90 (in millions of 1982 dollars)

Millions of dollars

1500 -

1000 “

500 -

0 I I I I I I I I I III I I I I

1960 1965 1970 1975 1980 1985 1990

 Basic research Applied research

KEY: NASA National Aeronautics and Space Administration.

NOTE: Figures were converted to constant 1982 dollars using the GNP

Implicit Price Deflator. 1990 figures are estimates.

SOURCE: National Foundation, Federal for Research and

Development, Detailed Historical Tables: Years 1955-

1990 (Washington, DC: 1990), table A; and National 

Foundation, Selected Data on Funds for Research and

Development: Years 1989, 1990 and 1991 (Washington,

DC: December 1990), tables 4 and 5.

history, ranging from $0.8 to $1.6 billion (in

constant 1982 dollars), but in the 1980s applied

research has held fairly constant at nearly $1.0

billion.

OSSA sets priorities in conjunction with the

budget process and by selecting specific projects.

The process is essentially bottom up with project

managers proposing new initiatives. However, when

large missions are proposed, such as Space Station

Freedom, top-down direction will determine the

parameters of the effort. OSSA recently produced its

first strategic plan, which emphasized a commitment

 notes agencies do not the division into basic, applied, and development useful. Consequently, Offices 

that the data that they report to the National Science Foundation (NSF) is and prone to errors, The NSF figures also remove the funds for

equipment purchase from the research and development budget line items and add the support funds from the Research and Program Management

appropriation associated with R&D. OTA uses these data only as a general indicator of level of effort particular areas.
108 . Federally Funded Research: Decisions for a Decade

to the Space Station Freedom and the Earth Observing System and to flying a mixture of small and large

missions.

The National Research Council (NRC) plays a

particularly strong advisory role for OSSA, and the

Space Studies Board provides input for most NASA

basic research programs. The board is unique at

NRC because it has an institutional relationship with

NASA, i.e., NASA funds the board and requests

many studies, but the board can use these resources

to initiate studies independently. In fact, the board

has been able to preserve its credibility because it

has not always agreed with NASA, and has openly

disputed it on some occasions. Roughly every 10

years if events do not call for an earlier revision, the

board writes a strategic plan for every discipline in

OSSA. The Space Studies Board also conducts

periodic reviews of the programs and every new

mission, and other larger topics such as ‘‘manned’

v. ‘‘unmanned’ flight are routinely studied.

In addition, OSSA has an internal structure of

advisory panels. The panels are usually made up of

representatives from academia, industry, Federal

laboratories, and other interested groups such as

program managers from other agencies. They are

consulted at least once or twice each year (sometime

quarterly) about future directions for research programs. However, as with NRC, their findings are

never binding.

In early 1990, “exploration” was added to the

Office of Aeronautics, Exploration, and Technology, formerly the Office of Aeronautics and Space

Technology. The new program participates fully in

the Administrator’s Moon/Mars Initiative, which

gives it a new and higher profile within the agency.

The aeronautics work in OAET is almost all basic

and applied research, and OAET views its role as the

basic research provider in aeronautics for the country. Consequently OAET’s advisory committees are

primarily composed of and almost always chaired by

industry representatives. Generally the decisions of

research direction are made by the associate administrator. It is a somewhat open process, in which

there is ample chance for those outside NASA to

comment.

In the 1970s and into the 1980s, OAET’s space

technology component asked of project directors:

‘‘what will they need for the future?’ In 1986 and

1987, the program changed its philosophy. It focused on short-term problems and attempted to

promise system delivery by specific dates. In 1989,

the deputy administrator questioned this approach.

Now 60 percent of the funding goes to near-term

solutions to mission problems; 30 percent to longterm solutions; and 10 percent to high-risk research.

The frost 90 percent is developed in conjunction with

mission managers, and the rest is decided within the

space technology group, and can be used to support

risky research, such as studies on ‘‘ wormholes’ ‘—

shortcuts between distant points in space.

Recent problems have plagued many NASA

programs, such as a flaw discovered in the Hubble

Space Telescope, the halt of space shuttle flights due

to hydrogen leaks, and nagging questions about the

Space Station. A reflective look at NASA programs

by Congress has been urged, and calls for an

overhaul of NASA’s management structure have

grown louder. 16 Director Truly has cited the need ‘or

a better match between agency programs and its

resources. In addition, many have pointed to the

failure of NASA programs to encourage a civilian

space industry that also supports research. While

NASA has been charged (since 1960) to promote a

civilian space capability, it has been successful to a

lesser extent than predicted one, two, and three

decades ago.17 An Advisory Committee on the

Future of the U.S. Space Program has reviewed

NASA’s programs and has suggested such goals as

building a reliable space transport system, improving NASA’s civilian pay structure, and augmenting

IGFOreX~ple, SeeIlaVidc. Morn.so~ ‘ ‘Hill to NASA: Come Dow” National Journal, vol. 22, No. 18, my 5, 1990, pp. 1077-1081; fi~y Sawyer.

“Truly: NASA Needs More Flexibility,” The Washington Post, Sept. 14, 1990, p. A17; and Katby Sawyer, “NASA: Mission Implausible,” The

Washington Post, Nov. 4, 1990, p. C3.

17see Mmk R. Od~, “A Viewpoint on Commercial Space Activities: Realities and Options for the 1990s,” Science, Technology, and the

Changing World Order, colloquium proceedings, Apr. 12-13, 1990, S.D. Sauer (cd.) (M%shingtoq DC: American Association for the Advancement of

Science, 1990), pp. 253-264.
Chapter 4-The Federal Research System: The Research Agencies . 109

NASA facilities.18 OMB and the National Space

Council have been directed to create an implementation plan based on its suggestions.l9

Department of Energy

DOE is the fourth largest source of basic and

applied research funds in the Federal Government.

DOE is also the youngest of the six major research

agencies. Created by the Carter Administration to

succeed the Energy Research and Development

Administration, DOE inherited a strong research

base from another predecessor, the Atomic Energy

Commission, including the national laboratories and

a network of university researchers.

When DOE was founded, in the wake of the

formation of OPEC and the subsequent Arab oil

embargo, its top priority was to lessen U.S. dependency on foreign countries for meeting its energy

needs. At the same time, rising concern with

environmental issues such as water and air pollution

spumed research on developing cleaner, more efficient energy sources. Nuclear power was an avenue

frequently stressed, although the accident at Three

Mile Island in 1979, compounded by cost concerns,

seemed to slow work on fast-breeder reactors. The

Carter Administration also placed particular emphasis on achieving short-term results through work on

conservation, cleaner burning coal, solar electrical

power, and other sources.

The 1980s saw a marked shift in the priorities of

DOE, emphasizing long-term rather than short-term

research and stressing the role of the Federal

Government as a risk-taker, pursuing research projects that, if potentially profitable, are to be turned

over to the private sector for demonstration and

commercial development. The Reagan Administration emphasized basic research over applied research, cutting the latter in the mid-1980s while

increasing basic research markedly over the same

period (see figure 4-6).

Figure 4-6-Basic and Applied Research at the

Department of Energy: Fiscal Years 1980-90

(In millions of 1982 dollars)

Millions of dollars

““”~

2000 “ Q/

1500>: .<

1000 -

500

t

1980 1982 1984 1986 1988 1990

— Basic research + Applied research

~ Total research

NOTE: Figures were converted into constant 1982 dollars using the GNP

Implicit Price Deflator. 1990 figures are estimates.

SOURCES: National Science Foundation, FedemlFunbforResearch and

Development, Detailed Historical Tables: fiscal Years 1955-

19W (Washington, DC: 1990), table A; and National Science

Foundation, Selected Data on Federal Funk for Research

and Development: Fiscal Years 1989, 1990 end 1991 (Washington, DC: Deeember 1990), tables 4 and 5.

In general, priority-setting mechanisms for research at DOE appear to be very much like those at

DOD and NASA in the 1960s.20 However, compared

with other agencies, less accountability is required

from project to project. This is not to say that

accountability does not exist. DOE is responsive to

the scientific community and to the rest of the

government. Research managers outside of DOE

envy DOE’s flexibility, but see the tradeoff as a loss

of excitement in working toward a defined goal.

18NX AdviSov COIIMOWX on the Fume of the U.S. Space progr~ Report of the Advisory Committee on the Future of the U.S. Space program

(Washi.ngtoIL DC: U.S. Government Printing Wlce, December 1990); and Philip C. Abelsoq “Future of the U.S. Space Program,” Science, vol. 251,

Feb. 25, 1991, p. 357.

lg~ne Koprows~, “OMB to JOiII S- t on Space Report,” Washington Technology, vol. 5, Dec. 20, 1990, p. 1.

% the defense programs, although nuclear weapons research occurs within both the Department of Energy (DOE) and the Department of Defense

(DOD), there is a clear division of labor. DOD builds the delivery systems and DOE produces the nuclear weapons to go inside them. Tb set goals, every

2 years a document comes from the Pentagon called ‘ ‘Nuclear Weapons Development Guidance.” It outlines the requirernents of future systems. Based

on this document, supplemented with threat assessments and other analyses, the DOE defense group decides the future direction of their research

programs. Generally no large redirection is required.
110 . Federally Funded Research: Decisions for a Decade

They wonder, too, about DOE’s accountability to

basic and applied research missions.21

Recently, the Secretary of Energy attempted to

institute more strategic planning within DOE. In

particular, he created a National Energy Strategy

(NES) with input from the offices within DOE and

from external advisors. The planning process for the

NES required planning at all levels of DOE, and

Secretary Watkins has sought to maintain and

further this planning function at DOE. As it is too

early to observe the changes in response to these

initiatives, OTA cannot judge their effectiveness,

but such planning is reportedly beneficial at other

agencies.

In the Office of Energy Research, programs such

as Basic Energy Sciences and High Energy and

Nuclear Physics use an “iterative” process of

priority setting-where ideas are proposed (with

origins both within and without DOE), feedback

from the scientific community and other parts of

government are received, and the proposal is revised—to determine goals. In particular, as national

goals are defined and new ideas arise from either

within DOE or without, the program will frost

consider them internally. If the new initiative would

fit into the existing program or complement it, then

the idea will be fielded to a wider audience.

Sometimes this audience includes only other parts of

the agency. DOE may, however, hold public workshops and/or panel meetings to devise a plan of

action.

If a plan is codified by the Office of Energy

Research (OER) or within one division, it is sent out

for review to DOE personnel, academic and industrial representatives, and other interested parties.

This method of fielding new ideas requires much

responsiveness on the part of DOE to groups outside

of the agency, including the scientific and industrial

communities. This method also develops strong

working relationships with these communities, but it

can have its drawbacks. For instance, some managers complain that the scientific community tries to

dictate on occasion (and more than at other agencies)

Photo credit: U.S. Department of Energy

Researcher at Oak Ridge National Laboratory studies

the health and environmental effects resulting from

synthetic fuels conversion processes. The Department

of Energy sponsors research in many broad

scientific areas.

plans that could never gain either the finances or the

political support necessary to emerge as new programs.

Complicating decisions on priorities is the fact

that DOE has a very broad research base. For

example, under OER, the Basic Energy Sciences

program considers its mission similar to NSF. As

one manager put it: ‘‘. . . the research that we

support is as broad as NSF’s, but with a different

emphasis. Also, with the major cuts in applied

energy research funding during the Reagan Administration, the applied programs in offices outside of

OER lost much of their research function. They now

may look to OER to develop needed research

programs.

In addition, each program has an advisory panel,

such as the High Energy Physics Advisory Panel.

Until recently there was also an overall advisory

committee, the Energy Research Advisory Board

(ERAB), which reported to the Director of Energy

Research and the Secretary of Energy. This group

 for the ambivalence over and checkered funding history of the fusion research program. is by 

as the best long-term alternative to fossil fuel energy dependence. U.S. participation in a major multinational effort to design a fusion energy test reactor,

the International Thermonuclear Experimental Reactor, marks a renewed commitment that is reflected in the Administration’s fiscal year 1991 budget.

See Mark Crawford, “U.S. Backing for Fusion Project Seen,” Science, vol. 251, Jan. 25, 1991, p. 371; and Christopher “DOE Rallies to

Save U.S. Fusion Research Program,” vol. 349, Jan. 24, 1991, p. 269.
Chapter 4--The Federal Research System: The Research Agencies . 111

was disbanded in early 1990, and the Secretary has

formed the Secretary of Energy’s Advisory Board

(SEAB), which is already in operation. SEAB’s

charter has been expanded beyond the scope of

ERAB, to include advice on the National Energy

Strategy and on the role of the national laboratories.

In the late 1970s and early 1980s, while the

Reagan Administration was entertaining notions of

abolishing DOE, many interviewees said that the

planning for DOE’s research was more external to

the agency, with NAS and other organizations

playing key roles. Program shifts were primarily

budget controlled and long-term goals often suffered. Furthermore, many decisions on specific

programs were dictated by the scientific community

they served. The interviewees state further that the

system has now evolved so that DOE can make

decisions that balance external as well as internal

forces. This is accomplished primarily through the

iterative process described above. Differences between programs are due primarily to the constituencies and the types of problems addressed, but

differences are also due to historical tradition.

In the applied research offices of DOE, processes

of goal setting are also iterative. Most ideas are first

taken up internally, and then may be augmented by

contractor reports. After much deliberation, they are

taken to the public. Five-year plans are written for all

new programs and receive extensive review.

In the late 1970s and early 1980s, the process of

priority setting for applied research was inward

looking. Now with the large consulting process

described above and participation by industry, DOE

perceives that it is better serving its “client’’—the

energy industry-and thereby the public. During the

Reagan Administration, the emphasis was on funding research that was too risky for industry. Under

the Bush Administration, the emphasis seems to

have shifted toward projects perceived to have the

highest payoff for industry and DOE.22

National Science Foundation

NSF is the fifth largest source of basic and applied

research funds in the Federal Government. Established in 1951, it has evolved into an agency

composed of eight directorates in addition to the

Director’s Office. The Director and Deputy Director

are appointed by the President and serve 6-year

terms. Five of the directorates fund basic and applied

science and engineering; another two focus on

education, human resources, data, and policy; and

one handles NSF’s administrative matters. On average, each directorate has five divisions. Each division has several programs.

The primary role of NSF is to support basic

research and science education across broad categories of science and engineering. This is done

primarily through support for university-based individual investigators, who absorb over 60 percent of

the research budget. Aggregate support to groups

and centers represents a small portion of the budget

(less than 10 percent) and is more sensitive to budget

fluctuations. 23 Support for individual investigators

is considered the primary mission, even by those

managers with portfolios covering group and center

support.

A number of research administrators at NSF

prefer to use the terminology “fundamental v.

directed” rather than “basic v. applied” in making

distinctions between categories of research funding.24 In using the former terminology, they are

likely to respond that they fund both (but much more

fundamental than directed). In using the latter

terminology, they are more likely to say they fund

only basic research. Most administrators say that

they never give a grant with applications in mind,

but they are pleased when grantees cite NSF-funded

work when seeking patent applications.

In its first operating year, the NSF budget was

$151,000. In constant dollars, the budget has grown

over NSF’s history, although not consistently (see

table 4-2). The NSF budget authority for fiscal year

1991 is $2.2 billion. Currently, NSF funding is

provided in six separate appropriations: Research

and Related Activities (R&RA); Education and

Human Resources (EHR, formerly Science and

Engineering Education); U.S. Antarctic Program;

Facilities, Program Development and Management;

and the Office of Inspector General. R&RA has

22For ex~ple, see Alan Sctiesheti, “Toward a Golden Age for ‘lkchnology Transfer, ” Issues in Science& Technology, vol. 7, winter 1990-91,

pp. 52-58.

23Natio~ Sciaw Fo*&tiou Report on Fu&ing Tre&s ~~Balance of Activities: National Science Foun~tion ]95]-]988, special report, NSF

88-3 (Washington DC: 1988).

~OTA ~temlews at the National Science Foundation, spfig 1~.
112 . Federally Funded Research: Decisions for a Decade

Table 4-2—National Science Foundation Obligations:

Fiscal Years 1952-90

(in millions of constant 1982 dollars)

Research and

Year Total obligations related activities

1952 . . . . . . . . . . . . . . . $ 13.6

1954 . . . . . . . . . . . . . . . 28.6

1956 . . . . . . . . . . . . . . . 54.8

1958 . . . . . . . . . . . . . . . 159.7

1960 . . . . . . . . . . . . . . . 494.1

1962 . . . . . . . . . . . . . . . 795.2

1964 . . . . . . . . . . . . . . . 1,049.1

1966 . . . . . . . . . . . . . . . 1,316.5

1968 . . . . . . . . . . . . . . . 1,319.2

1970 . . . . . . . . . . . . . . . 1,100.2

1972 . . . . . . . . . . . . . . . 1,292.9

1974 . . . . . . . . . . . . . . . 1,195.7

1976 . . . . . . . . . . . . . . . 1,148.1

1978 . . . . . . . . . . . . . . . 1,187.3

1980 . . . . . . . . . . . . . . . 1,138.8

1982 . . . . . . . . . . . . . . . 999.1

1984 . . . . . . . . . . . . . . . 1,213.4

1986 . . . . . . . . . . . . . . . 1,311.9

1988 . . . . . . . . . . . . . . . 1,420.1

1990 . . . . . . . . . . . . . . . 1,586.7

$ 7.6

23.1

44.4

102.1

287.0

548.8

724.3

942.7

942.0

783.6

1,032.2

1,000.4

972.0

1,017.9

983.6

909.7

1,065.0

1,140.0

1,202.2

1,312.9

NOTE: Fiscalyear 1990figures areestimates.

SOURCE: National Science Foundation, Report on Funding Trends and

Balance ofActivities:Nationa/ Scierrce Foundation, 1951-1988,

special report,NSF88-3 (Washington DC: 1988);andNational

Science Foundation, pressrelease, PR90-05, Jan.29,1990.

accounted for more than 70 percent of the budget

since 1967, and 80 percent or more since 1982. EHR

has been the most variable, ranging from 46 percent

in 1959 to a low of l.5 percent in 1983. It is also the

target of recent increases, approaching an all-time

high of $322 million of NSF’s $2.2 billion fiscal year

1991 budget.

Within directorates, research funding is very

much a bottom-up process. Goals are set by scientific opportunity and the proposal process, as well as

in special initiatives from advisory panels. Through

its grants program, NSF receives proposals for

research spanning the fullest range of science and

engineering. The scientific community is NSF’s

constituency, and program staff project a strong

sense of obligation and commitment to that community. There is an explicit ethic pervading the

directorates that discourages heavy-handedness in

the setting of priorities. Staff serve as interpreters,

advocates, and jurors throughout the priority-setting

and plannin‘ g process.

An exception to the above lies in the Engineering

Directorate. Created as a separate unit in 1983,

Engineering tends to set its priorities around national

needs. For example, a recent initiative involved a

Request for Proposals in design and manufacturing

systems. It was the sense of NSF staff and its

advisory committees that there was a need for

research in those areas. In addition, the Engineering

Directorate tends to address problems more centrally, and many areas of engineering are crossdisciplinary. To this extent, the divisions of Engineering, and the methods by which they set priorities, differ somewhat from the way other directorates

operate.

The agency primarily sets priorities and plans

through a process described by many as”. . . continuous, open, and decentralized. ” The decision cycle

is keyed to the annual Federal budget and annual

appropriation cycles. Eight populations provide

formal and informal input into the planning process.

They are: 1) the National Science Board (NSB); 2)

advisory committees; 3) professional societies; 4)

NRC; 5) Visiting Scientists, Engineers, and Educators (also known as “rotators’ ‘); 6) NSF staff; 7) the

Inter-Directorate Task Force; and 8) Congress.

Each spring, the advisory committees meet with

program managers and division directors to recommend priorities for the current year and years to

come. Besides scientific opportunity, staff usually

recommend that NSF not fund research already

well-supported by other agencies.

Plans are eventually forwarded to NSB for consideration at their June meeting. A strategic plan is

developed that must be set against the general

recommendations of NSB. For example, in 1989,

NSB decided on four general priorities for NSF to

pursue-international cooperation in research, education, economic competitiveness, and better methods for leveraging Federal dollars (i.e., to share

funding with other-typically State or privatesources). If an organizational unit within the agency

proposes a new program that covers all or most of

these priorities, it has a very good chance of getting

a proportional increase in its budget. For example, in

the late 1970s it was decided that there should be

more funds for the physical sciences in the 1980s; in

the 1980s it was decided that in the 1990s NSF

should focus on building strength in engineering and

computer sciences. The mid- to late 1990s should

bring more funds to environmental sciences and

geosciences. National needs are very much a part of

the planning process.

In addition to planning conducted on a program

basis, there has been increasing attention paid to
Chapter 4--The Federal Research System: The Research Agencies . 113

planning on an activity basis: by whom and how will

research be conducted? This has resulted in more

support for women and minorities and broader

geographic distribution of funds. Between 1985 and

1990, support for the individual investigator went up

25 percent as compared to other research funding

modes, such as groups and centers.

NSF faces a daunting task-being all things to all

people. The organic act entrusts it with the support

of the Nation’s basic research and science education.

(Thus, every research program at NSF has an impact

on human resources.25) Within the scientific community, however, there is growing concern that NSF

has reduced its flexibility by relying too strongly on

traditional mechanisms to set priorities and allocate

funds. While not wishing to abandon peer review,

NSF has sought some alternatives. A recent report,

which addresses these issues of emphasis and

process from the perspective of senior staff, stresses

that NSF must serve all research performers, streamline the proposal process, and better integrate human

resources with research funding considerations.26

Department of Agriculture

USDA is now the sixth largest source of basic and

applied research funds in the Federal Government.

USDA has a long history of support for research,

especially when compared with other government

agencies. In 1862, the Merrill Land-Grant College

Act recognized the importance of agricultural research and education by setting aside Federal land

for agricultural colleges. In 1887, the Hatch Act

created the State Agricultural Experiment Stations

and assigned administrative responsibility for them

to the land-grant institutions. During this time,

USDA also grew in power as a research provider,

creating an expanding research network.27

In the late 1960s, environmental problems began

to dominate discussions of research in agriculture,

with particular concern expressed for finding alternatives to the use of chemicals as pesticides and in

fighting plant and animal disease. Throughout the

last three decades, research on human nutrition has

been stressed, as well as with finding means for

improving the productivity of American farms.

R&D funding levels for USDA since 1955 are

tabulated with the other agencies in table 4-3. In

constant dollars, USDA R&D funds have hardly

grown since 1965. For basic and applied research,

the figures are similar. In 1960, USDA research

funds totaled just under $0.5 billion. Throughout the

1960s, 1970s, and 1980s, their total grew steadily,

but declined from 1985 to slightly under $0.8 billion

in fiscal year 1988.

USDA is advised by many groups. Most important is the Joint Council on Food and Agricultural

Sciences (JCFAS), created by an act of Congress in

1977 to coordinate and encourage research, extension, and higher education in agriculture. Its members include influential representatives from public

and private sectors, producers, industry, and government; as well as directors of research, extension, and

higher education activities in universities, agricultural experiment stations, and other centers. While

JCFAS has the mandate to evaluate and recommend

changes to USDA programs, it cannot direct USDA

to institute them. Another advisory body is the Users

Advisory Board on Research and Education, with

membership selected from those who benefit from

research and education. These and other groups

advise the various research components of USDA.

Agricultural Research Service (ARS)

ARS was established in 1953 as USDA’s inhouse

agricultural research agency.28 The National Program Staff (NPS) is a core component of ARS

headquarters and is responsible to the administrator

for planning, developing, and coordinating the ARS

ti~~ough tie Natio~ ScienceFo~&tion*~ (NSF) ~~e of fie toM Feder~ rese~ch ~d d~cloprn~tbudget r~uestedfor fiscd year 1992 is Oldy

3 percen~ its education and human resources programs represent 23 percent of the total proposed Federal agency effort. Programs such as Research

Experiences for Undergraduates (which is slated to support ahnost 12,000 students) are increasingly visible. See Frederick M. Bernthal, acting director,

National Science Foundation testimony at hearings before the House Committee on Science, Space, and lkchnology, Subcommittee on Science, Feb.

20, 1991, pp. 7-8, 11.

~See Natio@ Science Foundation Report o~the Merit Review Task Force, NSF 90-113 (Washington DC: Aug. 23, 1990); ad Jeffrey Mervis,

“Panel Weighs Overhaul of NSF’s Grant System,” The Scientist, vol. 5, No. 1, Jan. 7, 1991, pp. 1,6-7, 12.

zT~wence Busch and Wifi= B. IACY, Science, Agriculture, and the Politics of Research (Boulder, CO: WesWiew press, 1983).

%cenm~ OffIces of me Agric~~~ Rese~ch Service (ARS) are in Washington, DC, md Beltsvfile, MD. ~~e me aPPro ximately 7,000 full-time

employees (of which 2,350 are scientists) scattered across the United States, Puerto Rico, the Virgin Islands, and several foreign countries. Research

is conducted at 122 domestic and 6 overseas locations by civil service scientists. Last year there were about 1,700 projects ongoing with budgets ranging

from $100,000 to $1 million. Much of the work of ARS is conducted in direct cooperation with the State agricultural experiment stations, other State

and Federal agencies, and private organizations.
114 . Federally Funded Research: Decisions for a Decade

Table 4-3—Trends in Federal Obligations for Total Research and Development,

by Major Agency: Fiscal Years 1955-88 (in millions of constant 1982 dollars)

Ail Total

other nondefense

Year USDA HHS NSF DOE NASA agencies agencies DOD

1955 . . . . . . . . . . . . . . . . . .

1960 . . . . . . . . . . . . . . . . . .

1965 . . . . . . . . . . . . . . . . . .

1970 . . . . . . . . . . . . . . . . . .

1975 . . . . . . . . . . . . . . . . . .

1980 . . . . . . . . . . . . . . . . . .

1985 . . . . . . . . . . . . . . . . . .

1988 . . . . . . . . . . . . . . . . . .

1955 . . . . . . . . . . . . . . . . . .

1960 . . . . . . . . . . . . . . . . . .

1965 . . . . . . . . . . . . . . . . . .

1970 . . . . . . . . . . . . . . . . . .

1975 . . . . . . . . . . . . . . . . . .

1980 . . . . . . . . . . . . . . . . . .

1985 . . . . . . . . . . . . . . . . . .

1988 . . . . . . . . . . . . . . . . . .

$347

505

788

738

728

804

837

778

$ 327

1,284

3,050

3,205

4,155

4,421

4,865

5,079

$ 46

300

657

758

1,031

1,031

1,195

1,379

$1,574

3,059

4,353

3,533

3,548

5,560

4,410

4,027

$ 207

1,483

17,374

9,974

5,311

3,783

2,955

3,636

$ 325

759

1,156

2,410

2,603

2,938

2,227

1,862

Agency Percentage of Total Annual Nondefense R&D Funding

13.4

6.8

2.9

3.5

4.2

4.4

5.1

4.6

12.6

17.4

11.1

15.3

23.9

24.1

29.5

30.3

1.8

4.1

2.4

3.6

5.9

5.6

7.2

8.2

60.9

41.4

15.8

16.9

20.4

30.3

26.7

24.0

8.0

20.1

63.1

47.6

30.6

20.6

17.9

21.7

12.6

10.3

4.2

11.5

15.0

16.0

13.5

11.1

$2,58 6

7,390

27,525

20,941

17,376

18,357

16,490

16,761

100

100

100

100

100

100

100

100

$9,591

22,938

23,753

19,319

15,620

16,352

26,458

34,489

KEY: USDA=U.S. Department of Agriculture; HHS=U.S. Department of Health and Human Serviees; NSF=National Science Foundation; DOEdJ.S.

Department ofEnergy; NASA=National Aeronautics and Space Administration; DOD-U.S. Departmentof Defense.

NOTE: Totals arenot exact due torounding.

SOURCE: National Research Council, lnvesfirrgin Research:A Proposa/ toStrengthen theAgriwitura~ Food, andEnvironmenta/System(Washington, DC:

National Academy Press, 1989),tableA.l, p.96.

national research program. There are about 30 NPS

employees with expertise in a discipline, commodity, or problem. Their role is individually and

collectively to plan research programs, set priorities,

allocate resources, review and evaluate research

progress, and provide coordination.

ARS has a long-range Program Plan-designed in

the 1980s—and an Implementation Plan, which

describe how the Program Plan is to be operated over

a 6-year period. The Program Plan focuses on the

goals, objectives, and broad research approaches

that ARS will pursue. The current Implementation

Plan covers 1986 through 1992 and considers,

among other things, how the budget and shifts in

research needs relate to the goals and mission of the

agency. This strategic planning is relatively new,

having started in 1983. Administrators of NPS feel

that the development of the Implementation Plan has

enabled them to set priorities, helped in redirection

of finds, and has increased communication between

ARS and groups such as other USDA agencies,

Congress, user groups, and scientists.

The Implementation Plan was put together by

NPS and the ARS laboratories with input from

industry, academia, and regulatory agencies. Because program areas often overlap, NPS works

together in planning for the entire research program.

NPS, therefore, is very centralized and not only does

planning and priority setting, but also makes allocation decisions and performs program reviews.

Cooperative State Research Service (CSRS)

CSRS is USDA’s “. . . principal entree into the

university system of the United States for the

purpose of conducting agricultural research.” It "

. . . participates in a nationwide system of agricultural research program planning and coordination

among the State institutions, USDA, and the agricultural industry of America.”29 Programs of research

are jointly developed with the State Agricultural

Experiment Stations, forestry schools, 1890 LandGrant Universities, and other cooperating institutions. The most recent planning exercise resulted in

the strategic plan entitled “A Research Agenda for

the 1990s. ’ This is the first time that such a strategic

plan has been developed. It outlines current research

efforts and areas of proposed enhancement, including the safety and stability of consumer foods, and

the protection of water quality.

Zgcmperative State Research Service, “Budget Submission for 1990,” hearing before the House Subcommittee on Rural Development Agriculture,

and Related Agencies Appropriations of the Committee on Appropriations, Part 2, p. 444, 1989.
Chapter 4-The Federal Research System: The Research Agencies . 115

Photo credit: U.S. Department of Agriculture

An Agricultural Research Service scientist (rear) and a

graduate student (front) transplant seedless grape

varieties. Research and education are intertwined

in many research areas.

The majority of CSRS Federal funds (approximately $200 million out of the $340 million in fiscal

year 1989) comprise formula funds, which are

directly appropriated by specific acts of Congress.

Special Research Grants amount to another $61

million (fiscal year 1989), and consist mostly of line

item appropriations (which many liken to earmarks)

requiring oversight from CSRS. Priority setting is

negotiated between the cooperating institutions and

CSRS. In addition, the Competitive Research Grants

Office (CRGO) conducts a nationwide competition

for basic research funds in specific fields. CRGO

began in 1978 with programs in plant science and

nutrition and, by 1985, it had expanded to include

animal and biotechnology research. NRC, with

strong support from USDA, has proposed a National

Agriculture Research Initiative, which would enlarge the USDA Grants Program from $45 million to

over $500 million. The program would increase

research funds in areas not presently supported at

USDA, such as global climate change.3 0

Forest Service

The research mission of the Forest Service is to " . . . serve society by developing and communicating scientific information and technology needed to

protect, manage, and use the renewable natural

resources of the Nation’s 1.6 billion acres of forest

and related range lands. ”31 Within the structure of

USDA, the Forest Service is quite separate from

CSRS and ARS, as it reports to the Office of the

Assistant Secretary for Natural Resources and Environment rather than the Assistant Secretary for

Science and Education. Furthermore, its budget is

not considered by the congressional agriculture

committees in Congress, but by the interior committees.3 2

The decentralized nature of the Forest Service

research work force encourages bottom-up planning. 33 Reently the Forest Service stations have

been required to submit budgets at four different

funding levels, ranging from 90 percent of the

funding level 2 years before to 10 percent over the

agency request from the prior year. An iterative

process between Washington and the stations adjusts what work will be done at different budget

levels. Perhaps the most important trend is that in the

early 1980s, and before, the budget process was

tightly controlled by the Deputy Chief for Forestry

Research. Now the process is much more open, and

the stations are more responsive to national problems, such as global change, water quality, and

endangered species. For instance, the percentage of

funding devoted to national problems rose from 28

percent in fiscal year 1989 to 42 percent in the fiscal

year 1991 Presidential request, with new developments funded as special initiatives.

 U.S. of Agriculture, Cooperative State Research Service, National Research Initiative Competitive Grants Program: 

Description (Washington DC: 1990).

 Department of Agriculture, “Strategy for the ’90s for USDA Forest Service Research,” review copy, February 1990.

 where researches conducted. Within the 8 

and FPL, 190 Research Work Units are gathered at 74 locations. Over 700 scientists work in these units with a total budget of nearly $150

million. Extramural research is supported at a low level—approximately $14 million per year, although this is deceptive since many of the are

located on college campuses.

 Work Unit Descriptions a duration

and often will build directly on previous work. The Station Director has a large amount of discretion to choose projects at the RWU level, and the RWUDS

are reviewed inhouse in the Washington Office to provide balance in a nationally coordinated program.
116 . Federal/y Funded Research: Decisions for a Decade

OTA found in this and earlier studies tha t

investment in research at USDA has lagged behind

other agencies, and that USDA has difficulty in

clearly stating its mission, planning for the future, or

setting priorities in research.34 Consequently, much

of the new agriculture-related science (e.g., biotechnology) is performed by scientists who are not

trained in the agricultural sciences and who do not

pursue agricultural problems. Many blame the lack

of growth in research funding at the agency to the

lack of a comprehensive strategic plan.35

Other Agencies

The six agencies described above together devote

over $11 billion annually to basic research. Also

contributing to the research base, but on a much

smaller scale, are the following 10 agencies: the

Alcohol, Drug Abuse, and Mental Health Administration (in HHS); the U.S. Geological Survey (in the

Department of the Interior); the Smithsonian Institution; the Environmental Protection Agency; the

National Institute of Standards and Technology and

the National Oceanic and Atmospheric Administration (both in the Department of Commerce); the

Department of Veterans Affairs; the Department of

Education; the Agency for International Development (in the Department of State); and the Department of Transportation. This group of 10 agencies

represents approximately 5 percent of the total

Federal expenditure on basic research.36 Although

their contribution is comparatively small, these

agencies lend breadth and flexibility to the Nation’s

research capacity.

Crosscutting Descriptions of Agency

Priority Setting

Comparisons of the research agencies reveal the

variation and complexity in the Federal research

system. While agency cultures are very different, the

prospect of transferring methods and standards

across agency boundaries deserves consideration.

OTA first examines various characteristics of the

organization and management of the Federal research system.

Division of Labor

The Federal research system can be thought of as

a composite of the various agencies that support

research. Each agency has a mission and therefore a

purview of research responsibility. NSF and NIH,

for example, have the broadest scope in research

areas funded. Any project within a discipline that is

of high quality and does not clearly fall under any

other agency’s jurisdiction can be a candidate for

funds.37

NASA, DOE, DOD, and USDA have more

restrictions (than NIH or NSF) on the research areas

that they support. NASA supports science that can

make use of space (and most often seeks information

about space), either through satellites, experiments

above the atmosphere, or human exposure to zero

gravity. DOE funds research relating to nuclear

weapons and all forms of energy and its effects on

humans and the environment, which is interpreted

broadly in the Department.

Although some claim that because the research

areas supported by DOD and USDA are closely tied

to their technical missions, the research by definition

cannot be basic or fundamental in nature. Indeed,

OTA finds that the research supported by these

agencies can be as fundamental as that supported by

other agencies, such as NIH or NSF. In addition, the

amount of funds spent on basic research at these

agencies is comparable in size to that disbursed by

NSF. Nonetheless, these agencies’ priorities shape

research goals.

Areas of support among the agencies allow a

multitude of questions to be posed and investigated

differently within the research system. This also

provides some measure of pluralism in research

opportunities, i.e., many researchers have two or

three agencies (and even more programs) within the

~U.S. Cowess, office Of TMmoIogy Assessmen4 Agricultural Research and Technology Transfer Policies for the 1990s, speci~ repo~

OTA-F-448 (Washington DC: U.S. Government Printing OffIce, March 1990).

351bid.

MAl budget &I@ reported beIOW are based on National Science Foundation Division of Science Resources Studies, Federal Fundsfor Research and

Development: DetailedHistoncal Tables, Fiscal Years 195.5-1990 (Washington DC: 1990); and National Science Foundatio~ SelectedData on Federal

Funds for Research and Development: Fiscal Years 1989, 1990, and 1991 (Washington DC: December 1990).

sTResearchersq~c~y learnwhatresemch is and is not eligible for funding at an agency. Program ~o~cements, Conversations Withprogrm officers,

and the fate of other submitted proposals convey to the researcher which agency (and program within it) is an appropriate source of funding. This, too,

is part of the agency culture, which forms a constituency of extramural research performers.
Chapter 4-The Federal Research System: The Research Agencies ● 117

Federal Government to apply for funds. Pluralism

has long been hailed by the scientific community as

a strength of the research system in the United

States.

38 However, the pursuit of agency missions is

not without legislative contention, as Congress

consistently asks agency managers, in authorization

and appropriations hearings, how specific research

programs support the agency mission. 3 9

Coordination

The division of labor among the agencies does not

seek to eliminate overlap; indeed, agencies cooperate to fund some areas of mutual interest. Agencies

with broad research agendas, such as NSF and NIH,

coordinate more routinely with other agencies—

more than those, such as USDA, with a more narrow

scope.

In addition, because of the size of agencies and

departments, coordination within the agency or

department can be important as well. For example,

the services in DOD sometimes attempt to find a

niche in a scientific area so there is no overlap with

another service. In supercomputers or artificial

intelligence, for instance, the Air Force has chosen

to rely on the other services. The Air Force in turn

takes the lead in other areas, such as mathematical

control theory. In areas that require overlap, however, agencywide committees are often employed to

coordinate the activities of the services, DARPA,

and SDI0.40

Coordination among and within Federal agencies

occurs at two levels, at the agency program level and

at the research performer level. Agency-level coordination generally occurs through committees. One

standing coordinating mechanism is the Federal

Coordinating Council on Science, Engineering, and

Technology in the Office of Science and Technology

Policy. It has several subcommittees, such as Global

Change and High-Performance Computing, which

provide a forum in which agencies can communicate

(see chapter 5). Other committees that are not

governmentwide also exist, which may coordinate

two or more agencies on a specific topic.

Researcher and program manager level coordination occurs through meetings and other communication that is a normal part of the discourse of the

scientific community. It is at this level that the

separate roles of agencies are most apparent and that

researchers accommodate to changing funding levels in the cooperating agencies. An illustration of

agency and performer interaction can be found in

superconductivity research (see box 4-C).

Bottom-Up and Top-Down Management,

and the Use of External Advice

One of the most prevalent styles of management

of research has been loosely titled bottom up, which

implies that research ideas and priorities originate

with researchers who communicate these ideas to

their sponsors (agency program managers, for example). These managers in turn talk to their superiors.

As ideas percolate, their relative importance is set.

Bottom-up management contrasts with top-down

management, where the most senior decisionmakers

in an agency decide the priorities for the system, or

their part of it. These directives are then transmitted

down the organizational ladder in consultation with

managers, eventually to researchers.

OTA finds that in the research agencies both kinds

of management are prevalent and are often mixed. In

short, decisionmaking is more complicated. Some

agencies employ much stronger top-down direction.

In the Agricultural Research Service of USDA,

priorities are set by the National Program Staff, and

at DOD, managers at all levels exert a great deal of

influence over the areas in which they support

projects. On the other hand, agencies such as NSF

and NIH employ mostly bottom-up management. At

NSF, this means that only priorities among areas of

support are set at the top (by the Director, the

Assistant Directors, and NSB). For example, deci38AS H~terDupree observed: ‘‘A plural set of government agencies went to a plural set of congressional committees to ask for appropriations, which

were then distributed by grant and contract to investigators in a plural set of universities. ’ Quoted in U.S. Congress, House Committee on Science and

lkchnology, Task Force on Science Policy, A History of Science Policy in the United States, 1940-1985, 99th Cong. (WashingtorL DC: U.S. Government

Printing Office, September 1986), p. 40.

sgover~p in supprt respomibfiity ~ong agencies for certain areas of research ensures better diffusion of results into multiple applications, a ~d

of inadvertent diffusion policy. Harvey Brooks, Harvard University, personal communicatio~ February 1991.

4A Vew g~ exmple is he Jo~t Senices El~@ofics progr~, ope~ted con~uously sb~ 1945, ~ but 20 or so universities. The the mili~

services provide equal contributions to each university group, but delegate administration to one service. For example, Harvard University and the

Massachusetts Institute of ‘lkchnology each have such a program administered by the Office of Naval Research. This is a fairly successful

interdisciplinary program. Ibid.
118 ● Federally Funded Research: Decisions for a Decade

Box 4-C—Coordination of Superconductivity R&D

Major research initiatives are usually executed by one Federal agency, based on the scope of the research and

the mission of that agency. There are exceptions, of course, where research on one area is done in several different

agencies, and these research areas bring with them the added burden of coordination. In the case of

superconductivity, coordination becomes especially important since research is spread among several different

agencies, primarily the Departments of Defense (DOD), Energy (DOE), and Commerce, the National Science

Foundation, and the National Aeronautics and Space Administration (NASA).1

There are two aspects of coordination that require quite different approaches. First, coordination is required

to monitor the different programs and make appropriate decisions to ensure an efficient allocation of research funds

to all of the agencies. Second, at the researcher and program level, there must be an adequate flow of information

between researchers to avoid overlap or duplication of research. Effective coordination at both the national level

and the researcher level is vital to a successful research program.

Congress has made several attempts to encourage coordination of superconductivity research and development

by the executive branch. Part of the 1988 Omnibus Trade and Competitiveness Act created the National

Commission on Superconductivity (NCS) which was to meet, produce a report and then dissolve by December

1989. The Trade Act also mandated an increase in staff for the National Critical Materials Council (NCMC), which

at the time had no active members. Finally, the National Superconductivity and Competitiveness Act of 1988 called

for cooperation between the Office of Science and Technology Policy (OSTP), NCMC, and NCS in order to produce

a 5-year National Action Plan for Superconductivity to be accompanied by annual reports.

The success of these initiatives has been limited. The 5-year National Action Plan was published in December

of 1989, but the formation of NCS was delayed, so it did not take part in the plan’s formation. Although the plan

itself acknowledged the need for better Federal coordination, it lacked both the budget recommendations and the

long-term perspective Congress had requested.2 In addition, the Federal Coordinating Council on Science,

Engineering, and Technology Committee on Superconductivity report of March 1989 did little more than assemble

agency superconductivity budget data and list various prograrns.3

Fortunately, at the researcher and agency program level, the exchange of information has successfully

protected superconductivity research from overlap and duplication. Programs at different Federal agencies have

aided scientists in the exchange of research information, if not actual coordination of effort. The Ames laboratory

distributes the “High-Tc Update,” a widely read newsletter; the national laboratories have broadcast nationally

several high-temperature superconductivity conferences; and DOE has established a computer database that shares

research results with industry. NASA also maintains communication through the Space Systems Technical

Advisory Committee, a group with representatives from industry, universities, and government organizations.

The success of the ground-level coordination efforts is promising, but the resistance to priority setting from

the administration may inhibit the progress of superconductivity research. In particular, such questions as whether

DOD funds too high a percentage of superconductivity research, and whether the Federal laboratories are doing too

much of the research relative to other performers are important to the future success of the development of

superconductivity. These questions must be addressed through agency-level coordination,

lF-g leve]s at wh ~emey, of course, ~ a =pme ~d * ‘ore prewing issue. See Kim A. McDonal& ‘I%nel Urges Imxeamd

Support for Superconductivity, Re.comxneti SpeCW]C Goals for Research in Fie14° The Chrom”cle of Higher Educ&”on, VOL 36, No. 48, Aag.

15, 1990, pp. &?i, A7.

2U.S. CO~SS, office of ‘Rximology Assessme@ High-Temperature Supercondti”vity in Perspective, 0’C4-E-4LW (wWgtom ~:

U.S. Government Printing Office, April 1990), p. 63.

31bid., p. 69.

sions are made to support the physics program at one Agencies that are more bottom up also tend to

funding level and the chemistry program at another. employ more panels or to commission more studies

Bottom-up management at NSF and NIH leads to a from outside of the agency to help set priorities. At

different selection mentality than that in other NSF, NIH, and NASA, standing external commitagencies, specifically to reliance on peer review as tees, NAS, the National Academy of Engineering,

a formal mechanism for incorporating advice from and the Institute of Medicine play significant roles

the scientific community. in deciding agency priorities. However, agencies
Chapter 4---The Federal Research System: The Research Agencies ● 119

such as DOE, USDA, and DOD employ outside

panels to assist in determiningg research directions to

a lesser extent.

Another difference between top-down and bottom-up management is the degree to which the

agency becomes invested in the success or failure of

a project. For example, DOD has a large operational

investment in the results of the research it supports.

This provides an atmosphere that reminds researchers that DOD has a stake in their success. Consequently, these researchers report favorably that DOD

is more realistic about the funds and time needed to

complete a project, and program managers are more

available during the course of the project to aid with

difficulties that may arise.41 This contrasts with the

experience of NSF and NIH researchers where the

agency does not have a stake in the success of any

one project, because there is no expectation of direct

‘‘use’ and no timetable for making ‘‘progress.

intramural and Extramural Research

Five of the agencies that OTA studied support

intramural research (NIH, DOE, DOD, NASA, and

USDA). Intramural research facilities are most often

within laboratories either run directly by the agency

or by an outside contractor. Together the Federal

research agencies are the primary sponsors of

hundreds of laboratories. Some are administered

directly by the Federal Government, such as

NASA’s Goddard Space Flight Center, while others

are administered by a university or corporation, such

as DOE’s national laboratories. Laboratories can be

funded institutionally, where monies flow from the

research agency to support all activities at the

laboratory, or industrially, where the laboratory

competes with other laboratories and research organizations to perform research for clients in number

of research agencies. Often there is a mixture

between institutional and industrial support within a

laboratory, as in many of the DOE national laboratories.

Intramural Research

Research in intramural laboratories has many

distinct advantages for the Federal Government.

Photo credit: U.S. Department of Energy

Los Alamos National Laboratory (LANL) is one of the oldest

intramural laboratories of the Department of Energy. LANL

was established in 1943 to develop World War II atomic

bombs, and today retains responsibility for conducting

defense-related research programs. Research at LANL

has also diversified into other fields such as fossil

and geothermal energy.

First, laboratories can maintain a research effort over

one or more decades. Second, laboratories can easily

incorporate a multidisciplinary approach to problems. Third, DOE managers report that they more

often fund ‘‘risky’ research-research that has a

very good chance of abject failure, but also a good

chance of resounding success—at the laboratories,

because the laboratories can absorb a setback

without jeopardizing graduate students or young

faculty. Fourth, project managers can easily maintain their involvement in the projects at a laboratory.

Fifth, the research at the laboratories can be put “on

the fast track,” in the words of one manager, when

the results are needed on a timetable. Sixth, there is

ample evidence that the laboratories can often

perform research at a reduced cost to that performed

extramurally. Finally, the laboratories are often the

only sensible place to site facilities needed for a

project, because access and maintenance can be

assured.42

Disadvantages of intramural research include

problems in recruiting and retaining personnel. The

 in project monitoring more or less create the need for ex post accountability-and the for evaluation of 

outcomes (see 8).

 laboratories sites of science education. The role of mission agencies and their laboratories science education 

noticeably, especially at the Department of Energy. See U.S. Congress, House Committee on Science, Space, and Subcommittee on Energy

Research and Development, Role of the Department in Science, Engineering 

(Washington DC: U.S. Government Printing June 13, 1990).
120 ● Federally Funded Research: Decisions for a Decade

government laboratories must pay on the Federal.

pay scale, but while salaries have risen in academia

and industry for scientists, growth in salaries in the

Federal Government has been very sl0W.

43 Consequently, although research problems addressed at

the laboratories and the research environment can be

very exciting, researchers are often attracted by

much higher paying and more flexible jobs elsewhere. 44 Some critics claim that the pay scale and

government cutbacks have limited the quantity and

quality of research in the Federal laboratories.45 In

addition, intramural laboratories often do not support graduate students, which are an invigorating

part of research.

Another disadvantage is that many laboratories

are large organizations. While they were built with

a mission (or a set of missions) in mind, the mission

may have been achieved or abandoned.% The

laboratory must then find a new mission or face the

prospect of downsizing or phaseout. Laboratories

have sometimes moved from mission to mission, but

this can lead to great stress in the organization.47

Extramural Research

The advantages of extramural research include

competition on the ‘‘open market” for the best

research teams for a particular problem. (Note that

laboratory teams can often compete for these funds

as well.) Extramural researchers are paid competitively and can be solicited for one project. The

research performers are top scientists in an area and

enjoy access to state-of-the-art equipment.

Disadvantages of relying on extramural research

are, frost, that extramural researchers must bid for the

project. If a new project is not associated with

enough gain (either in money, equipment, or publications), it is difficult to find extramural researchers

willing to apply (although this may also be true in

some cases for intramural laboratories) .48 In academia, government-sponsored research is also constrained by the academic environment. For example,

although DARPA funds university research, on

some projects DARPA finds it difficult to work

solely with universities. The university structure of

research, with a professor and his or her graduate

students, often operates too slowly for DARPA’s

purposes. Also, DARPA reserves the right to terminate at any point, which can be disastrous for a

professor and especially for a graduate student.

OTA finds that both intramural and extramural

capabilities are important for the advantages they

provide the agencies; both should be supported by

the Federal Government.49 At present roughly onequarter of all research funds are spent intramurally,

slightly under one-half extramurally in universities

or colleges, one-quarter in industry, and onetwentieth by other performers.50

Issues of Agency Priority Setting

Some priority-setting issues are of particular

concern across all of the research agencies. OTA

identifies four in particular: 1) risk-taking and

conservatism, 2) flexibility, 3) strategic planning,

and 4) redirecting the agencies.

43u.s. Dep~ent of commerce, Bureau of Economic Analysis Government Division,‘‘Biomedical Research artdDevelopment Price Inde% report

to the National Institutes of Heal@ Mar. 30, 1990,

‘Pepper Leeper, ‘‘NIH Intramural Program: No Radical Changes Needed,’ NewsReport of the NationaJResearch Council, December 1988-January

1989, pp. 2-5.

45~or exmple, see ~ezo, op. ~it., fm~ote 9, p. 38+ Simila s~tements were made in sevm~ OW interviews at the research agenCieS.

46f~~so big labs tend t. ~ome b~aucrat~ed as Feder~ pay policy leads to a gerontocracy, With tie lemt en~Prene@P~PIe ‘m~g ‘n” A

big disadv~tage of intramural research is that it is much easier to do relatively ‘boring’ but vitally usefid research e.g., spectroscopic tables, and

systematic physical and chemical property measurements. ’ Brooks, op. cit., footnote 39.

ATEvidence of fement ~&en most appment at me Dep~ent of Enqy. SW Cowil on Competitiveness, ‘‘National hbs Meet with DOE,”

Legislative and Policy Update, vol. 3, No. 2, Jan. 28, 1991; Mark Crawford, “Domenici Bill to Broaden Labs’ Missions,” The Energy Daily, vol. 19,

No. 14, Jan. 22, 1991, pp. 1-2; and “Roundtable: New Challenges for the National Labs,” Physics Tow, February 1991, pp. 24-35.

4S’r’his Probla co~d ~~mwuded by tie re~nt Bush Atis@ationpropos~ to c~ge WKTS of Feder~ facilities, such u Brookhaven National

Laboratory’s synchrotrons light source, a fee to cover operating costs. See Mark Crawford, ‘‘Researchers Protest User Fees at National Labs,” Science,

vol. 251, Mar. 1, 1991, p. 1016.

4% ~spome t. outside Pmel ~omen~tiom, ~=agencies with ~bs~ti~ inho~~semc~e Dep-ent of Agriculture, the National

oCeiUliC and Atmospheric AdmMstrationi and the Environmental Protection Agency—me seeking to expand their extramural programs, if Congress

approves. See Elizabeth Pennisi, ‘‘New Policies at Three Federal Agencies Lend More SupporI to Outside ResearclL’ The Scientist, vol. 5, No. 2, Jan.

21, 1991, pp. 3,5.

~S~ Bette Hileq“Facts and Figures for chemical R&D,’ Chernkal & Engineering News, vol. 68, No. 34, Aug. 20, 1990, pp. 28-30.
Chapter 4-The Federal Research System: The Research Agencies . 121

Risk-Taking and Conservatism

Agency cultures promote differences in the kinds

of projects selected for funding. In particular,

agencies differ to a large extent in the amount of

risk-taking in research that they encourage in

scientific programs and by research managers.

Risk-taking can be defined in a number of ways.

Perhaps most important is that risky research is not

considered in the mainstream, no specific outcome

of the project is assured, and a large chance of failure

exists (i.e., of not reaching the objectives set out in

the proposal). However, the definition of risky

research changes depending on the agency and field

of inquiry, and to some extent, every research project

is inherently ‘‘risky. ’

Within agency cultures some programs can assume more risks, because in the pursuit of a specific

objective it is often wise to try some conservative,

yet slower means, along with more experimental,

less certain paths that might yield large payoffs.

DOD claims to take the most risks in the course of

its research program. DOD expects that most basic

research will not attain the results that it originally

proposed. Within the 6.1 category, DOD research

managers assume that less than 30 percent of the

projects will succeed; within 6.2, roughly 30 to 60

percent; and, within 6.3A, roughly 60 to 90 percent.

Supporting unsuccessful projects is viewed as part

of the business of finding projects that do pay off.51

in contrast, when there is no defined goal, how is

risk defined-proposals that earn diverse ratings

from peer reviewers? that are submitted by researchers with no track record? or that appear to be

of marginal interest to the particular program

weighing its merits? Attributes of risk are not clear

cut. Further, the pace and impact of results emerging

from federally funded research projects are no guide

to their ‘‘riskiness, ‘‘ judged retrospectively.

Yet NSF and NIH managers claim that 90 percent

or more of the projects that the agencies support are

“successful.’ Success in the NSF and NIH context

may mean that refereed publications were produced

from the project. This satisfies the criterion of

adding to the archive of knowledge, without measuring how that research was received by the community .52 NSF has recognized a need to support more

“high-risk” projects, instituting the Small Grants

for Exploratory Research program to engender more

risk-taking (see box 4-D).

Since it is obvious that not all scientific advances

are made through slowly evolving research (epitomized by DOE’s fusion energy program), but often

with new and exciting projects, it is important for

each Federal agency (and probably most research

programs within it) to support both kinds of projects.

This point is recognized by the scientific community

when it simultaneously urges funds for new avenues

of science as well as for the “science base, ” by

which is meant the protection of evolutionary (and

usually individual investigator, small team) research. While DOD addresses risk-taking through

expectations for project outcomes and NSF has

created a separate program, most agencies rely on

program manager discretion to incorporate risktaking. As priorities are set in new areas, it is very

important to continue, and even augment, risktaking in individual investigator research, and agencies should be encouraged to increase their efforts to

fund risky projects.

Flexibility

When new priorities are introduced at a research

agency, it must be flexible enough to reorient and

develop relevant programs. Flexibility can be defined in a number of ways. But the most critical

aspect of flexibility for funding scenarios in the

Federal Government is the ability to make tradeoffs

among scientific programs and to pursue growth by

substitution-to start and stop programs, and to

encourage new ideas without allowing fiscal constraints to hinder (or undermine altogether) their

pursuit.

At the program level, flexibility is already provided in several ways. First, many agencies budget

discretionary monies for managers to pursue new

ideas. For example, some agencies (e.g., the Office

of Naval Research) divide their pools of money into

‘‘core’ and ‘accelerated’ research initiatives. Core

programs maintain expertise in certain areas and rely

on principal investigators to propose goals for their

research. Accelerated initiatives allow significant

amounts of money to be quickly infused into a

specific project area. Second, programs that disburse

Slnese fiWes are a consensus among the Department of Defense managers whom OTA interviewed.

5ZFor euple, see Malcolm Gladwell, “Are Nobel Prizes for U.S. Vestiges of ‘Golden Age’?” The Washington Post, Nov. 16, 1990, p. A6.
122 ● Federally Funded Research: Decisions for a Decade

Box 4-D—Small Grants for Exploratory Research

Risk-taking is an important part of scientific research. In particular, the rate of scientific advancement

witnessed this century could not have been achieved had the Nation not invested in some high-risk research along

the way. However, the National Science Foundation (NSF) has been criticized that its funding decisions, based on

a system of external peer review system, has become too conservative.1 In response, NSF instituted in 1989 the

Small Grants for Exploratory Research (SGER) program, which funds only small, high-risk research projects. In

the words of a former NSF assistant director, who spearheaded the pilot program that led to SGER: ‘‘With the small

amount of money, relatively, that NSF can give out, we cannot take care of all research needs. On the other hand,

we have sufficient amounts of money to stimulate more creative and innovative research by playing a catalytic

role.

SGER grants are funded differently from ordinary NSF grants in several important ways. ‘he definitive

difference is that SGER grants go exclusively to researchers who are exploring “novel ideas” or “emerging

research areas. In addition, NSF eliminated formal, external peer review. Final recommendations on funding are

left entirely up to the program manager, although the manager may certainly seek as much advice as he or she

desires. More than in other NSF programs, grant applicants are encouraged to discuss their proposals with the pro@

manager before submitting them in order to ascertain the proposal’s chance for success. This reduces the number

of unsuccessful proposals submitted, thus increasing the efficiency of the process and saving time. Also, this

practice helps to foster a favorable working relationship between the researcher and the program manager. However,

critics fear that this interaction might ‘‘. . . work against faculty who are not comfortable with selling themselves

to others.

Processing speed is another important aspect of the SGER program, as high-risk research often implies

fast-paced. SGER grants are limited to $50,000, and the duration is no more than 2 years, usually only 1. Keeping

grants on this smaller scale can make them easier to process.

In 1990, NSF funded 244 SGER proposals (while declining 210) at an average award of $34,254.5 Also

encouraging is the amount of activity in divisions, such as the biological sciences and Earth sciences, where the

SGER prograrn is being instituted for the first time.6 The SGER program appears to provide an outlet for NSF to

fund cutting-edge, high-risk research that the traditional NSF peer review system might not be equipped or inclined

to support. The genre most served seems to be “cross-disciplinary’ research, such as studies of natural disasters.7

In addition, an Expedited Awards for Novel Research program (forerunner of SGER) survey of recipients found

that 90 percent of SGER-funded researchers go on to apply for a regular NSF grant.8

SGER program spending is limited to 5 percent of each program. However, it appears to provide access to NSF

funding for new researchers and, as one researcher put it, SGER support might result in “. . . fewer publications per

dollar, but more chances for quantum leaps in advancing science. ”9

lm Natiorjal Mence Foundations own survey of 14,000 applicants who had h awarded or declined fund@ dur@ N Y= 1985

found that two out of three agreed with thh statement: “NSF is not likely to fund high-risk exploratory research became the like W of

obtaining favorable reviews is slim,” !% National Science Fmndatiou Pmgmrn Evaluation Staff, Proposal Review at NSF: Perceptions@

PrincipaJlnvestzgarom, NSF 88-4 (WaahingtorL DC: February 1988), p. 18.

2Nm sfi quoted in David Bjerkiie, “Fast-Track Grants,” Technology Review. vol. 93, No. 6, Auguat/Scpternbez 1990, p. 19.

J-t follows, unless o~“ irxticatex$ is based on National Science FotmdatiorL Small Grants for Exploratory Research brochure,

1989.

4JmH M. MfiLI~ tire, progrm Evdw.iozI SW. National Science FoundatiorL “Responses to Bulletin Board M=Sage mut

Qoick-Respome, Non-Rcvicwcd Gin@” Mar. 10, 1989, p. 5.

5~”_ statistics on the Small @ants for Explorato~ Resead p~ PrOVidCd by James MccullouglL &r@or, Program

Evaluation Staff, National Science Rxmdatim persomd cornmunicati~ Jan. 23, 1990,

6Natio~ WeIMX FO~“ L unpublished dat% Aug. 8, 1990.

7W f~ of SUCb ~O@ at the National Science F~u at least before the scope of M E.t@ncering Directorate wa9 enlar@,

was problematic. See Alan L. Portcx and Frederick A. Ro~ ‘‘Peer Review of Intcxdiaciplinary Resc.amh

& Human %/lUS, vol. 10, No. 3, ~

ProPO*” Science, Technology,

198S, pp. 33-38.

81nvcs@tom without prior National Science Foundation SUpport are~to apply to the Small Grants for Explomtmy Rcwamh

P-, ~d b pm- indeed seems to attrsct many fret-time applicants. SCC Bjcrkliq op. cit., footnote 2, p. 19.

%sponses to McCuUOU@ op. cit., footnote 4, p. 7.
Chapter 4-The Federal Research System: The Research Agencies ● 123

money by manager discretion have inherent flexibility. Specifically, program managers have the ability

to continue or disband a research project, whereas in

competitive peer reviewed grant programs many

more persons—the ad hoc or standing peer review

panel and the program manager-are asked to

concur on a specific decision. Finally, in some

programs, such as the Office of Space Science and

Applications at NASA, managers are allowed some

portion of their budget as discretionary. Discretionary money is important to foster new ideas within a

‘‘zero-sum’ climate where money spent on one

research project detracts from another.

Agencywide tradeoffs in budgeting and resource

allocation are very important, but once a program

has been initiated it is hard to end. This happens for

a number of reasons. First, many people have

become invested in working and supporting the

program. Second, the program’s political constituency may wish to see it pursued, and may lobby both

Congress and the agency. Finally and most likely,

given funding constraints, it maybe very difficult to

start a new program. It tends to be easier to redefine

an old program to meet new goals, though it may not

be immediately as effective as a completely new

program.

OTA finds that agencies and managers throughout

the Federal research system could be provided with

the means, and perhaps incentives, to be more

flexible. Ending one program and starting another

could be made easier. More discretionary money

could be provided, incentives for managers could be

increased, and manager discretion accompanied by

accountability and attention to success could be

encouraged. Flexibility to adapt to research developments within a changing budget envelope is imperative. The production of excellence in research and

the reduction of stress if funding does not keep pace

with demand by the research community must go

hand-in-hand.

Strategic Planning

Strategic plans have recently been employed by

many of the research agencies as an important

component of the research portfolio.53 These agencies include the three services in DOD, CSRS in

USDA, the NSF research programs (through NSB),

and the Office of Space Science and Applications in

NASA. While these agencies have always planned

their near-term activities, many agencies have begun

to codify the plans and publicly distribute them for

comment.54

Strategic plans are very useful because they

communicate within the organization, the Federal

Government, and the research community the intentions of the research program over the next 5 to 20

years. They articulate the mission of the research

investment and outline the steps necessary to attain

intermediate and long-term goals. The mission may

be as general as supporting research in abroad area,

or as specific as solving a particular problem or

developing the foundation for a specific technology.

For instance, in the strategic plan for OSSA, NASA

states that it will attempt to launch a combination of

small and big satellite missions every year, thus

showing a commitment to small science missions in

space. If a program already has a clear idea of its

mission and the means of attaining its goals, then the

construction of a strategic plan is relatively easy. If

that understanding does not exist, then the creation

of a plan can be very useful in defining and pursuing

those objectives.

Often the formation of a strategic plan is resisted

within a program for fear of perpetuating relative

funding differences and forcing decisions prematurely to pursue specific objectives. Judicious and

regular revision of plans has led to a more realistic

allocation of funds and allowed oversight by the

executive branch and Congress to proceed smoothly.

Rather than arbitrarily freezing the program, its

potential can be highlighted and new options entertained within and without the current program

structure.

While strategic plans are not the solution to all of

the problems presented by the changing research

economy, and can be used to justify decisions rather

than to improve on them, OTA finds that strategic

and contingency plans (especially when accompanied by ex post evaluation—see chapter 8) are

elements that can be employed by the research

s3nere is a school of thought (to which OTA substantially subscribes) that strategic plans are primarily usehd for commtiCation ~d ~ve ~o

negative potentials: 1) they may be put on the shelf and ignored, and 2) they may be implemented blindly. Strategic plarming needs to be an

institutiomlized ongoing process, and plans need to be working documents that are constantly revised so the plarming horizon rolls forward.

s4~oW~ ~dagencies withtie Dep~~ent of DCfe~e, tie National Aeromutics and Space Administration, the National Science Fo~~tioIL

and the U.S. Department of Agriculture have recently produced strategic plans. The National Institutes of Health is in the process of developing one.

292-863 0 - 91 0 5
124 ● Federally Funded Research: Decisions for a Decade

agencies to plan for the future, increase communication, and accommodate new and continuing developments within the Federal system.

Redirecting the Agencies and Addressing

New Problems

What happens when the current mission of the

agencies is no longer well formulated or appropriate? Many agencies have been chastised that their

mission is either out of date or lost amid a multitude

of programs. Observers further claim that th e

agencies are calcifying, pursuing programs an d

setting priorities because of tradition rather than

national need.

5 5

For example, DOE has been challenged that it

does not support research that primarily seeks to

solve the Nation’s energy problems, but instead

supports a broad array of research programs from

high-energy colliders to radiation exposure in humans. Some claim that these problems would be

better pursued in an agency like NSF and that DOE

should concentrate on energy research. In a contrasting example, USDA has been repeatedly criticized

for supporting a narrow research agenda. Biotechnology and other fields related to agriculture do not

easily gain support within the USDA system. Critics

point out that the USDA system does not coordinate

well and many research opportunities fall through

the cracks between ARS, CSRS, the Forest Service,

and other programs.

56 In a third example, after a

series of large programs had not lived up t o

expectations (e.g., the Space Shuttle and the Hubble

Space Telescope), a Presidential commission was

created to conduct a comprehensive review o f

NASA priorities and procedures.

5 7

Many agency problems result from Federal attempts to cope with tighter budgets and setting of

priorities. OTA finds that the Federal agencies are

responsive to changing national needs, but ar e

limited by the program structure and budget. Agency

missions were defined many decades ago, ofte n

when budgets were expanding, and these mission

statements were ambitious. Agencies always seek

growth as an overall objective. But decisionmaking

structures do not serve as well when tradeoffs

between agency programs must be made and managers have little incentive to termin ate programs .58 For

the research system to thrive in the 1990s, the

t ermination of some programs in favor of others may

be required.

Some also question whether the scope of many

agencies’ programs should be reduced so tha t

whatever they decide to do they can do well. Perhaps

lessons learned at DARPA are instructive. DARPA

rarely pursues a problem without the required funds,

and attempts not to start programs at low levels

(which implies that the budget must free up t o

accommodate the program sometime in the future).

DARPA personnel regard this philosophy as crucial

to their success.

I n summary, crafting goals and missions for the

Federal agencies as the research economy changes is

not just a matter of the scientific objectives, but also

of management. The agencies were created a t

different times over the last half century and carry

with them cultural traditions and organizational

structures. As new goals are assigned to the research

system, Congress and the executive branch must pay

special attention to the capabilities and decisionmaking mechanisms of each agency. his includes

the methods by which priorities are implemented in

the selection of researchers and projects for support.

OTA considers these methods next.

Funding Allocation in the

Federal Agencies

When applying for Federal research funds, researchers submit a proposal. In general, a proposal

requests support for an individual, a specific project,

or a center,59 and is submitted to a particular program

in an agency for review. The process of review can

be thought of as a continuum of methods ranging

ssRemarks at “OTA Workshop on Costs of Research and Federal Decisionmaking, ” July 9, 1990-

fioffice of RChUOIOw Assessmen~ op. cit., footnote 34; U.S. Congress, OffIce of ‘Ikchnology Assessment, U.S. Investment in Biotechnology,

OTA-BA-360 (Washingto~ DC: U.S. Government Printing OffIce, 1988); and U.S. General Accounting OffIce, Biotechnology :Analysis of Federally

Funded Research (Washingto~ DC: U.S. Government Printing OffIce, 1986).

STAdvisov Committee on the Future of the U.S. Space Progrw op. cit., footnote 18.

58~deed hey we encomaged t. keep Progm ~ Owration. See ~ey Averc~ “policy uses of ‘Ev~uation of Research’ Literate, ” OTA

contractor repofi July 1990. Available through the National Wchnical Information Service, see app. F.

s~or a case smdy ~~ysis of Federal mwhanisms used to fund university researc4 see U.S. General Accounting OffIce, University Funding:

Assessing Federal Funding Mechanisms for University Research, GAO/RCED-86-75 (Washington, DC: February 1986).
Chapter #-The FederalResearch System: The Research Agencies . 125

from soliciting advice from experts outside the

agency, or peer review, to relying solely on the

judgment of the research officer who must defend

decisions to award or decline funding; this might be

called manager discretion. In practice, a mix of

these methods, even within the same agency, is

common.

Peer Review

“Peer review” describes a family of methods

used to make funding decisions about research

projects. It usually comprises a multistage process,

where reviews of the proposal are solicited from

experts in the scientific subdiscipline of the proposal. Reviewers are most often asked about the

technical excellence of the proposal, the competence

of the researchers, and the potential impact of the

proposed project results on a scientific discipline or

interdisciplinary research area. Peers may also be

asked about the project’s relevance to the objectives

of the funding program. The proposals and reviews

may then be considered by a panel of experts, and

competing proposals compared. The panel eventually ranks the proposals in the order in which they

think the proposed projects should be funded.

There are distinct advantages to this form of

proposal review: the participants are acknowledged

experts who make absolute and relative judgments

of proposal quality, or ‘‘scientific merit, ’ and who

offer their time on a largely volunteer basis. The

process is expected to operate according to values of

fairness and expediency. However, at the two

agencies that depend most on external peers, NSF

and NIH, problems with and suspicions about

systematic biases in proposal review have produced

a series of studies and self-studies.60 Such studies

raised questions about the composition of review

panels and the fate of proposals submitted by

investigators at research universities.61

Probably the most predominant criticism of peer

review, and the one that has troubled Congress the

most, has been the allegation that it is controlled by

an “old boys network,” which informally favors

those like themselves, and that decisions are made

behind closed doors where aspersions can be cast

against a researcher without providing a forum for

refuting them. Attempts have been made at both

NSF and NIH to correct faults found in peer review,

but neither agency would suggest that all of the

problems have been freed. Rather, given the strength

of peer review in soliciting expert opinion, they ask

‘‘what method is better?’ ’62

Manager Discretion

Manager discretion as a project selection method

refers to agency investment in the expert judgment

of a single decisionmaker or administrator-the

program manager.

63 This is not only the technical

judgment of the manager, but also his or her ability

to put together the best portfolio of research to

achieve the goals of the program. Manager success

is therefore seldom evaluated on the basis of one

project or before a series of projects are complete.

Rather, it is based on the success of an entire

research program. In agencies that rely heavily on

manager discretion, there is strict accountability of

managers for program decisions. But managers do

not work in isolation; there is oversight from

superiors, and inhouse advice is readily available. In

%ese am reviewed in D.E. Chubin and E.J. Hackett, Peerless Science: Peer Review and U.S. Science Policy (Albany, NY: SUNY Press, 1990),

chs. 2 and 3; U.S. General Accounting Office, University Funding: I@ormation on the Role of Peer Revl”ew at NSF and NIH, GAO/RCED-87-87FS

(Washington DC: March 1987); and NIH Peer Review Committee, “Sustaining the Quality of Peer Review: A Report of the Ad Hoc Pane~”

unpublished report, December 1989. Early studies of note include: NIH Grants Peer Review Study ~Grants Peer Review: Report to the Director,

NZH Phase Z (Washington DC: December 1976); Grace M. Carter, What We Know and Do Not Know About the Peer Review System, report

N-1878-RC/NIH (Santa Mollic& CA: RAND Corp., June 1982); Stephen Cole et al., PeerReview in the National Science Foundiztion: PhaseIof a Study

(Washingto~ DC: National Academy of Sciences, 1978); and Jonathan R. Cole and Stephen Cole, Peer Review in the National Science Founahion:

Phase II of a Study (Washingto~ DC: National Academy of Sciences, 1981).

GIForarecent example of such ~ysis at the National Science Foundation, see James McCulloug@ ‘First Comprehensive Survey of NSF APPlic~ts

Focuses on Their Concerns About Proposal Review,’ Science, Technology, & Human Values, vol. 14, No. 1, winter 1989, pp. 78-88, and associated

commentaries that follow the article.

GZSee Jon~v, ‘End of the Peer Show,” New Sa”entist, vol. 127, Sept. 22, 1990; ~d Jeremy merf~, ‘‘PeerReview: Software for Hard Choices,”

Science, vol. 250, Oct. 19, 1990, pp. 367-368.

Gs~t OTA is calling ‘‘manager discretion” is discussed in the organizations literature as a management tool or approach that springs, for example

in the case of the National Aeromutics and Space AdmILU0 “stration space program, from ‘‘. . . the complex conceptual, planning, administrative, and

evaluative tasks facing the agency and its contractors. ’ See Karl G. Harr, Jr. and Virginia C. Lopez, “The Nationrd Aeromutics and Space

Administration: Its Social Genesis, Development and Impacc” Managing Innovation: The Social Dimensions of Creativity, Innovation and Technology,

S.B. Lundstedt and E.W. Colglazier (eds.) (New York NY: Pergamon Press, 1982), p. 181.
126 . Federally Funded Research: Decisions for a Decade

particular, technical judgments of inhouse staff are

commonly solicited, and additional reviewers can be

tapped from outside the agency.

Manager discretion has many advantages as a

funding allocation mechanism. First, because program managers are intricately involved in th e

development of a program, they can best gauge the

relevance of projects selected for funding to program

objectives. Second, manager discretion allows an

agency to implement new goals quickly, since it is

easier to instruct managers to alter selection criteria

or allocation methods (and hold them accountable

for doing so) than to convince external peer reviewers to weigh factors others than technical merit in the

rating of proposals. Finally, the ethos of manager

discretion can result in the funding of proposals that

do not reflect the collective wisdom in vogue. As put

by one manager at the Office of Naval Research,

where manager discretion is the rule: ‘‘We don’t

take votes in the science Community. ”6 4

However, manager discretion can also suffer from

isolation—soliciting too little opinion from outside

of the agency, as well as relying foremost, and

sometime solely, on the technical judgment of the

program manager. Manager decisions can also be

seen as capricious, since they are not based on a

consensus among peers, Also, wherever manager

discretion is used as a decisionmaking device, it

assumes an organizational structure that recognizes

managerial responsibility for activities and objectives within time and cost limits.

Although on the surface peer review and manager

discretion seem very different, many agencies use a

combination of the two in their decisionmaking.

What follows is a brief description of the funding

allocation methods in the major research agencies.

For a more detailed discussion, see appendix C.

Agency Overview

NIH can be considered the original site of peer

review in the Federal Government, beginning with

the National Advisory Cancer Council in 1937.65

Today, NIH has an elaborate “study section”

system for soliciting and reviewing proposals from

extramural researchers. Section “secretaries’ are

pivotal in proposal processing. Study section recommendations are directed to 1 of 13 institutes and

must ultimately be approved as funded projects by

the appropriate advisory council. NIH intramural

researchers located in NIH laboratories around the

country compete for separate support. NIH uses

almost 100 chartered panels to recommend decisions about the relative merits of proposals.66

DOD research agencies rely primarily on inhouse

review and manager discretion. DARPA in particular is known for its strong program managers.

DARPA solicits proposals tailored to a field of

interest and specific research objectives. Funding is

awarded (and withdrawn) almost exclusively at the

discretion of the project manager. The Office of

Naval Research controls most of the 6.1 funding for

the Navy and is also noted for the independence of

its program managers who are often referred to as

‘ ‘czars.‘‘ The Air Force Office of Scientific Research disburses all of the Air Force’s 6.1 budget,

both to its own laboratories and to universities, with

inhouse review and manager discretion decisive in

project selection. Army research programs are decentralized, and inhouse review is used to allocate

monies to universities and numerous DOD laboratories. The University Research Initiative in the Office

of the Secretary of Defense supports additional

research at universities. The funding, however, is

allocated through the services and DARPA.

Research proposals at NASA are processed differently by the Office of Space Science and Applications and by the Office of Aeronautics, Exploration,

and Technology. At OSSA, proposals relating to

future flight missions are solicited through Announcements of Opportunity (AOs). Research Announcements are more modest in scope than AOs,

and can solicit ‘‘guest’ observers (who will participate in a mission after the original investigators) and

support theoretical work. Unsolicited proposals are

also considered. Funding is based primarily on

technical merit reviewed by an expert panel selected

by the program manager, an inhouse group, or an

outside contractor. NASA staff provide further

640TA ~temiews at the Office of Naval ResearcL sp@ 1990.

bs~e Natio~ ~ti~tes of Healti epitomizes how much project selection can be influenced, in the long rum by the very scientists who receive me

funds. See Nicholas C. Mullins, “The Structure of an Elite: The Advising Structure of the U.S. Public Health Service,” Science Studies, vol. 2, 1972,

pp. 3-29.

fine Natioml ~titutes of Health is the largest part of the Department of Health and Human Services; only one other Component of M department,

the Alcohol, Drug Abuse, and Mental Health Administratio~ supports extramural research.
Chapter 4--The Federal Research System: The Research Agencies . 127

review to determine feasibility and mission relevance. Proposals are ranked and the program manager selects from the top 20 to 40 percent. Division

managers must approve these selections.

67 A t

OAET, proposals are solicited through Requests for

Proposals. All responses are reviewed inhouse, and

most grants and contracts are administered b y

NASA laboratories. In all, one-half of OAET’s total

R&D funds is disbursed to the laboratories (chiefly

Ames, Langley, and Lewis), while 30 percent goes

to industry, and 20 percent to universities.

DOE’s civilian science programs use many of the

same proposal review techniques as NASA, with

peer review of scientific merit and final judgment by

the program manager. All proposals are solicited

through Broad Agency Announcements. The majority of research funds are awarded to the laboratories,

and these expenditures are estimated in the budget

request for DOE. Almost all of the agency’s defense

research is done at the laboratories; funding is

competed among them and distributed on the basis

of inhouse reviews.

NSF funds only extramural research .

68 It uses

program announcements and, through its system of

‘‘rotating’ program managers, routinely circulates

members of the research community into th e

agency’s decisionmaking apparatus. Although peer

review is the guiding principle of NSF proposal

review, its form varies greatly within and across

agency directorates, divisions, and programs.

USDA is a multilimbed agency. The fundin g

procedures of the Agricultural Research Service are

highly centralized and totally inhouse. Proposals are

received in response to an annually revised 5-year

National Program. They are sent for external review

only after the decision to fund has been made and

only to approve the dollar amount of support. The

Agriculture Grants Program of the Cooperative State

Research Service is a separate arm of USDA.

Outside panels rank proposals and, along with

program managers, determine funding levels. CSRS

also has “nationally targeted programs” and “special programs, ’ the latter being congressionally

earmarked funds. Both categories are supervised

inhouse. The Forest Service is another arm of

USDA, with stations scattered around the United

States competing for funds nom the National

Program. Research work unit descriptions are solicited from all of the laboratories and are competed at

the national level; outside review is rarely solicited.

Blurring of Peer Review and Manager

Discretion

This overview illustrates the various combinations of peer advice and manager discretion used in

the research agencies. Some research agencies have

always used a particular method-DOD has consistently relied on manager discretion augmented by

informal reviews. Some agencies have recently

altered their methods.

For instance, NSF renamed its proposal review

process ‘‘merit review’ in 1986 to reiterate that

‘‘merit consists of more than peer judgments,

especially relevance to agency missions.69 Likewise,

NIH stresses the role of institute advisory councils

in weighing priority scores against program relevance

70 (for an example see box 4-E). While it has

always been the case that technical merit is a

necessary but not sufficient condition for research

funding at agencies other than NSF and NIH, the

exercise of manager discretion in the selection

processes of these peer review-based agencies has

become more explicit.

The issue is not which method, peer review or

manager discretion, is better, but that either one, or

a combination, can be used effectively to address

670fierdiscretio~ money (representing about lopercent of the budget) is available to the division director and the progmmmamger. It is disbused

for projects of higher risk, or for specific needs not addressed through the procedures described above, using a less formal procedure (sometimes only

with internal review).

~For e~ple,in fiscal year 1989, the National Science Foundation (NSF) received 44,300 proposals and made 16,700 awwds. The agencY suPPorts

the research of 18,900 scientists (including salary for an average of 2 months each year), 3,600 postdoctoral researchers, and 15,600 graduate students.

The average award amount to individual investigators ranges across directorates from $50,000 to $150,000. Comparable information in all of these

categories, over the last decade, is lacking for the other agencies except the National Institutes of Health and the Department of Veterans Affairs. See

U.S. Congress, Office of ‘Ikdnology Assessment, ‘‘Proposal Pressure in the 1980s: An Indicator of Stress on the Federal Research System,” staff paper

of the Science, Education, and Transportation Program, April 1990, pp. 4-7, and table 1. Since publication of the O’IA staff paper, NSF has developed

revised numbers for competitively reviewed proposals: 27,300 received and 8,400 awarded with a median annual award of $55,000. Linda Pwker,

National Science Foundation personal communication, Jan. 23, 1990.

@National Science Foundation, Advisory Committee on Merit Review, Final Report, NSF 86-93 (Wm~gto~ DC: 1986).

ToFor a ~storic~ perspective, see Stephen P. Strickland, The StOrY of the NZH Grants Program (Lanbam, MD: University Press of America, 1989).
128 . Federally Funded Research: Decisions for a Decade

.

BOX 4-E-Fine-Tuning Project Selection at NIGMS

The mission of the National Institute of General Medical Sciences (NIGMS), one of the 13 National

Institutes of Health, is the support of basic research in the life sciences. Early in 1990, the National Advisory

General Medical SCience (NAGMS) Council issued new guidelines that expand the factors taken into

consideration by the scientific staff in making project funding decisions.l “In times of extremely

constrained funding,” the NAGMS Council stated that ". . . the Institute [must] promote the broadest

possible diversity of ideas and approaches,.. ." and " . . . encourage the ideas and talents of established

investigators and of the young or new investigators who will provide the next generation of research

accomplishments." The NAGMS Council recommended a policy authorizing that special consideration be

given to a highly rated application from an investigator". . . who has no other significant source of research

support . . ." as opposed to such applications from investigator “. . l whose total research support from

all sources, including the pending “ award exceeds $500,000 (direct costs).”

Under this policy, the advisory council chose to free up funds by: 1) reducing the amount of funding

received by some investigators in the $500,000 plus category, 2) not approving two awards for projects that

were within the “theoretical” NIGMS payline, and 3) cutting 30 percent of the competing continuation

grants (60 out of 200) beyond the 12 percent across-the-board reduction. As a result of this shift of funds,

6 percent of institute awards (n=21) were made to "’ . . . grantees who had no other significant source of

research support and who also had percentilesw that were beyond the theoretical“ Institute payline,... " i.e.,

who would not have been funded under the traditional NAGMS guidelines.

What are the lessons derived from this advisory council action?There are at least two appraisals. The

positive one is that an NIH advisory council is searching for ways to support investigators without

compromising the integrity of either the peer review system or the research to be funded. Priority scores

were intended as the chief input to, but not the sole determinant of, award decisions. The NAGMS Council

recognizcs the imprecision of priority scores at the margin, and does not embrace their use as the sole

criterion for funding,

An appraisal that is more negative is most clearly stated in a letter sent in June 1990 to Acting NIH

Director William F. Raub.2 Citing “... little comfort in the idea that the change is only temporary,

the author notes that ".... there will always be a case to be made for redis

..."

tributionist policies, because there are always more losers than winners and many of the losers are quite meritorious.” He protests that peer

judgements about the quality of science will be secondary to consideration of the financial condition of the

applicants.

Another criticism is that the new NAGMS policy is merely another in a series of ad hoc responses to

the problem Caused by the’ insufficient number of new and competiting.grants,"... rather than looking

broadly at NIH's total research and training portfolio and the adequacy of its budget to support it...NIH

has other programs to achieve other purposes: for example, the special programfor young investigators. We

support those programs and want them to be adequately funded. But the core NIH research grant programs

should not be used to solve problems extraneous to their proper goals.”3

Expanding the pool of supported investigators, especially the "next generation,” and diversifying the

approaches to research that fail within the NIGMS mandane is part of the NIH mission. On the other hand,

the NAGMS policy is seen by some as tampering l with the traditional NIH review system. This use of

discretionary action by program “ officers and advisors should be applauded, but continues to be a source of

debate in government and the scientific community.

1

The iWlOw@ is

-.

m quo@d m National A&Wry Gmcrai MQdical~

 - CuudL “J- 1990 Motion and

Guiddincs Regarding Funding ~“ WnMfm%i Wn18dj% Feb. 8,1990.

%JKqtsti the letter arc used ww to illustrate gcmzicpoints probably held by o?hcrs. w WM@IY of * ah~ pd.

3fi&
Chapter 4-The Federal Research System: The Research Agencies ● 129

programmatic goals (and the choice of which

method depends on the goal) .71 In general, OTA has

found that the agencies will often adapt funding

allocation strategies to new goals.

Set-Asides and Formula Funding

In addition to the mainstream disbursal of funds,

agencies often allocate funds using other types of

programs. The two prominent categories of such

programs are set-asides and formula funding. While

their origins differ, each method of funding clearly

allows the Federal research agencies the discretion

to pursue certain national needs by applying a

different or reordered set of criteria to the selection

of research performers.

Set-aside programs are agencywide discretionary

actions. They select one characteristic that captures

a need not served by mainstream proposal review

and restricts competition for research funding to a

pool of eligibles who qualify by virtue of that

characteristic. Thus, there are set-asides for women,

ethnic minorities, young investigators, investigators

located at traditionally nonresearch institutions, and

investigators residing in States that have been

underrepresented in the amount of Federal research

funds they receive relative to their share of the

general population or the number of undergraduates

they enroll. (There are set-asides in other agencies as

well. See box 4-F.)

The assumption underlying set-aside programs is

that there are capable researchers everywhere whofor lack of opportunity or obvious disparities in

experience-are disadvantaged in the ordinary competitive proposal process. The solution is a separate

competition, still organized around the criterion of

technical merit, that pits like against like. (For a

model of an NSF set-aside that attempts simultaneously to strengthen institutional research capability and geographic diversity, see box 4-G.) For some

researchers, set-asides are the only way into the

Photo credit: U.S. Department of Energy

Scientists study the results of a nuclear magnetic

resonance experiment. Several agency set-aside

programs address the recruitment and retention of

women in scientific fields.

Federal grants system; for others it is a springboard

to continued competition in regular agency programs.

Formula funding can be traced to the Hatch Act

(1887), which authorized the allocation of Federal

funds to land-grant universities for the conduct of

research. 72 These funds are a kind of categorical or

block grant disbursed to the States, which enjoy

considerable discretion in their use. Typically, the

subject areas to be addressed by formula-supported

research are selected by directors, deans, department

heads, and faculty in the land-g-rant institutions,

within the broad guidelines of the enabling legislative acts. Peer review methods may be employed at

this decentralized level.73 In agriculture, competitive

grant funding is used to augment formula funding

that expands the science base, e.g., new research in

agricultural biotechnology.

71 t. in for of probably

value in a variety of blends between managerial discretion and peer review in different agencies and in different programs of a single agency. ’ Brooks,

op. cit., footnote 39.

 Hatch (the 

agricultural extension services) prescribed allocations to each State proportional to the magnitude of its agricultural enterprise. proportions are

indexed roughly to annual cash sales of agricultural products in the States and the investment of State funds in the State Agricultural Experiment Stations.

For details, see Don Illinois Agricultural Experiment “Recapturing the Vision: The Case for Formula Funds,” proceedings of the 1989

Annual Meetings of the Agricultural Research Institute, MD, May 1990.

 for project in include: potential economic and social importance of the research activity 

 and potential for the activity to generate other research need to fill gaps in agricultural knowledge; and need to provide continuity

in long-term research programs. See Don Illinois Agricultural Experiment ‘‘Mechanisms for Federal Funding of Agricultural Research

and Development” August 1988, p. 4.
130 . Federally Funded Research: Decisions for a Decade

Box 4-F—Small Business Innovation Research Program (SBIR)

The Small Business Development Act of 1986 (Public Law 99-443) requires Federal agencies that spend more

than $100 million annually on extramural research or research and development (R&D) to set aside 1.25 percent

(when fully operational) of those funds for a Small Business Innovation Research (SBIR) program.1 These programs

are intended to encoura ge innovation by allocating grants or contracts specifically to small businesses conducting

research on relevant topics. Minority firms are also encouraged to compete.

The notion of a set-aside program for small businesses, initiated by the National Science Foundation in 1977,

was initially disparaged by the academic research community, who viewed the program as a drain on available

funds. It was instituted governmentwide in 1982, and now provides substantial funds for science and

technology-intensive firms conducting research on agency objectives considered too risky to interest financial

investors. The seed money supplied by the Federal Government for the initial phases of research is leveraged in later

phases by private capital. The receipt of SBIR funds is considered an asset by some investors, who feel that it reflects

a measure of endorsement by Federal granting agencies.

The program has three phases. In phase I, projects are tested for scientific merit and feasibility. In phase II, the

principal research effort, successful phase-I projects are supported for up to 2 years. Products or services that reach

phase III are developed for private or government use. Before a project can enter phase III, it must secure additional

sources of support because SBIR finding ceases after phase II.

In fiscal year 1990, the U.S. Department of Agriculture funded 32 phase-I and 13 phase-II projects at a total

of $14.1 million. The award rate was 10 percent. In the same year, the U.S. Environmental Protection Agency funded

over 15 phase-I and nearly 20 phase-II projects at no more than $25,000 each. The Department of Commerce spent

$1 million on 9 phase-I and 2 phase-II grants in fiscal year 1990. One of the largest contributors, by virtue of the

size of its budget, is the National Institutes of Health, which spent $73 million on SBIR in fiscal year 1990.

Biotechnology companies have fared well under the NIH SBIR program and praise the program for giving them

the boost needed to conduct high-risk research.2

SBIR was reauthorized in 1987 for an additional 5 y ears-until 1993. It continues to be one of the few sources

of direct Federal support for applied R&D conducted by small companies.

l~s Wt isuon a 19i12 act (PublicIAW 97-219) and a successful experimental program of the Natiomd Science Fo@ation (NSF’).

The sources for what appears below are program solicitations of the Small Business A.&rums tration’s and NSF’S Small Business innovation

Research I%ograms, Washington Ffl, Sept. 24, 19%3, and National Science Foundation staff, personal communications, December 1990.

2B U( see Jeffrey Mervis, “Scienti]c Conflict of Interest Regulations Offer I.xmphole to Small Business Program, ” The Scientist, vol. 5,

No. 6, Mar. 18, 1991, pp. 1, 8-9.

Advocates of formula funding state that: ment. Critics of formula funds focus on the need for

Formula funds created the public institutional

structure of U.S. agriculture and remain essential to

preserving the unique strengths of key institutions.

Formula funds leverage much State and private

support for agricultural research. They distribute

costs in proportion to producer, consumer, and

spillover benefits. Formula funds provide much

needed continuity to programs that are otherwise

fragmented by the short-term, unpredictable nature

of gifts, grants, and contracts. They are needed to

offset unrecoverable indirect costs of projects, including . . . depreciation on buildings and equipment. . . . By decentralizing scientific priority setting and operational management, they avoid capricious top-down decisions and overcome the deleterious averaging effect of consensus-based managepeer review, incorrectly implying that formula funds

are not allocated competitively. The peer review

issue clouds other important issues, including. . . the

inability of typical peer review panels to apply siteand situation-specific criteria.74

Yet many still question the review received for

formula funded projects, and favor funds awarded

through openly competitive programs as “better

spent.

Both set-asides and formula finding represent a

form of legislated and/or within-agency recognition

that certain research goals cannot be achieved via

conventional proposal review. Thus, agency programs are created to direct funding that satisfies

74Holt, op. cit., footnote 72, p. 1.
Chapter 4--The Federal Research System: The Research Agencies . 131

Box 4-G—The NSF EPSCoR Program: Geography and Research Capability

Nowhere has the concern for regional distribution of Federal research funds been better institutionalized than

in the National Science Foundation’s (NSF) Experimental program to Stimulate Competitive Research (EPSCOR).l

Established in 1978, EPSCoR awards “. . . small amounts of money to 16 have-not States and Puerto Rico to use

as a magnet to help their universities and local industries excel in one or more areas of science and engineering.

The States are Alabama, Arkansas, Idaho, Kentucky, Louisiana, Maine, Mississippi, Montana Nevada, North

Dakota, Oklahoma, South Carolina, South Dakota, Vermont, West Virginia, Wyoming, and the Commonwealth of

Puerto Rico.

The EPSCoR States have formed a nonprofit organization, the Coalition of EPSCoR States, that argues for

greater Federal investment in the development of science and engineering capability nationwide. Relative to the

Nation as a whole, the Coalition points out, EPSCoR States”. . . have low per-capita incomes, high unemployment,

poor schools, retarded economic development, and low levels of science education attainment and scientific

manpower production. The EPSCoR States received 5.4 percent of Federal R&D funds in 1980, and 5,6 percent

in 1987. By improving the competitive position of States with underdeveloped science and engineering fundamental

research infrastructures, EPSCoR hopes to contribute to the health of all research and development (R&D) within

the United States.4

EPSCoR as Antidote

Selection as an EPSCoR-eligible State allows the State to compete for a research enhancement award of

between $3 and $5 million over 3 to 5 years. The money is awarded to a lead institution within a State to implement

the proposed State R&D plan, to stimulate academic research activity, and to enhance the competitive stature of

institutions in select research areas.5 The size of a State’s EPSCoR award is determined by the quality, number, and

type of projects; the current status of its research environment; the scope and magnitude of the proposed

improvements; and the potential to demonstrate significant change as judged by merit reviews

The objectives of EPSCoR are to increase the competitiveness of participant scientists and engineers-working

as individual investigators, in research groups, or in a research center—to obtain other R&D funds; to effect

permanent improvements in the quality of science and engineering research and education programs; and to ensure

that improvements achieved through EPSCoR-initiated activities continue beyond the end of the EPSCoR grant

period?

EPSCoR can also leverage investment from other sources; and, for every Federal dollar, three local dollars are

being invested in support of EPSCoR from industry and other sectors. 8 In Montana, for example, about 220

researchers have received aid and about one-half of them have gone on to win Federal grants through NSF’s regular

merit review system. Another 20 percent have won support from non-Federal sources. South Carolina has enjoyed

similar success: the mathematics departments at both Clemson University and the University of South Carolina

ranked 47th and 62d, respectively, in outside support after participating in EPSCoR. Previously, neither had been

among the top 100. 9

lNatioml SCICUCC FOUII&tiOL Division of Research Initiation and Improvement, “Experunental Program To Stimulate Competitive

Research Program Plan FY 1989- 1995,’ unpublished repo~, n.d.

‘Jeffrey Mervls, ‘‘When There’s N’ot Enough Money To Go Around,’ The Scientist, vol. 4, No. 8, Apr. 16, 1990, pp. 1, 8.

3Coalition of EPSCoR States, ‘ ‘EPSCOR: A State-Based Approach to Expanding American Research Capacity, ’ a congressional briefing

paper, Feb. 20, 1990.

4Joseph G. Daneh ‘‘A Mtiel Program for Expanding the Nation’s Science and Engineering Infrastructure, ” summary for the armud

meeting of the Arnertcan Assoeiatlon for the Advancement of Science, New Orleans, LA, Feb. 20, 1990.

~Joseph G. Danek, National Scienw Foun&itio~ personal communicatio~ DeCemkr 1990. BY itS descriptive I anguage, the National

Science Foundation apparently does not like to emphasize thatEPSCORisan‘‘equity’ program; rather it refers to EPSCOR as a capacity building

program.

~Na~o~ Science Foundation, Op cit., footnote 1, p. 2.

71bid., p. 1

8Dmek, op. cit., foomote 4

9Colleen Cordes, ‘ ‘Troy NSF Program Hailed as Model for Broader Distribution of U.S. Funds, ’ The Chrom”cle of Hi~her E&cation,

vol. 36, No, 45, July 25, 1990, p. A17.

Continued on next page
132 ● Federally Funded Research: Decisions for a Decade

Box 4-G—The NSF EPSCoR Program: Geography and Research Capability--Continued

Prospects for Emulation

Several statewide EPSCoR initiatives have created ongoing organizations dedicated to the long-term support

of science and engineering research. Included are the following: Montanans on a New Track for Science; Louisiana

Stimulus for Excellence in Research; Oklahoma Center for the Advancement of Science and Technology; and

Arkansas Science and Technology Authority. 10 Through participation in the EPSCoR program, these and other

States have been able to target their weaknesses and make significant strides in meeting the needs and improving

the quality of their research communities in select areas.l l

EPSCoR was funded at roughly $11 million in fiscal year 1991. The EPSCoR Coalition is seeking additional

funds from NSF and for the establishment of similar programs in other agencies. The National Aeronautics and

Space Administration (NASA), for example, is embarking on a program to help academic researchers compete for

NASA funds and to improve overall scientific literacy in underfunded States. The U.S. Department of Agriculture

is considering the provision of seed grants to scientists who have not received competitive grants from them in

5 or more years. 12

The prospect of redistribution worries critics of EPSCoR who fear a dilution of research capability. They claim

that EPSCoR undercuts peer review. But because the program aims to make States more competitive at a national

level, it pits them against one another for limited funds.13 Acting Director of the National Institutes of Health,

William Raub, also suggests that an EPSCoR-type program may not be transferable to the health care arena. Many

poorly funded colleges simply do not possess an adequate research infrastructure; there is no clinical program or

animal facility in which such research might be supported. 14

Thus, if expanding the EPSCoR model across Federal agencies is to be seen as a serious intervention, then

several questions remain. In the face of a tight Federal budget, how much money should be devoted to assisting

scientists and engineers in some States to become more competitive? (Would doubling or tripling the amount of

the annual EPSCoR award multiply or hasten returns?) At what level has a State made enough progress to graduate

from EPSCoR, or fallen behind enough to be added to the list?15

Quantitative measures of success must also be developed. Areas that might be examined include: the extent

of increased competitiveness for Federal R&D funding among individual investigators and research groups, the

scope and effectiveness of departmental and institutional enhancements of the research environment, and the

demonstration of long-term State financial support of EPSCoR to advance the cause of education and human

resources for science and engineering. 16 If broader geographic distribution of Federal research funding is sought,

the EPSCoR model could be emulated.

l~~e~ op. cit., footnote 4.

llNatio~ Sciewe Foti(iou op. cit., foomote 1! p. 2.

12&fe~k, op. cl~,, foomotc 2, p. 12. me Department of Enexgy, the Department of Ne=, A tic fi~~~~‘t=ti~ ‘ge~

were all directed by the 10lst CongreSS to introduce EPSCOR programs. See Audrey T ba~ ‘‘Congress Heaps Funds on EFSCOR for Research

in ‘Have-Not’ States, ” Physics Tb&zy, February 1991, pp. T7-78.

lqcordes, c)p. cit., foomoti 9, P. ’17

141bid., p. 12.

151bid., p. A17.

]~iven ~ coWenwtion of eI.MC tiotities b IXMMY mCOR states, ~ hUIIMU

resources potential of the program to inmease

participation in scientific careers has yet to be emphasized, except in Fuerto Rico. Established in 1980 with EPSCOR and University of Puerto

Rico support (and Subsequent.Iy fi-om the National Science Foundation’s R-ch Centers of Excellence Fmgram in 1988), the Resource Center

for Science and Engineering offers programs at every stage of the educational pipeline. The university has awarded 91 I%D.s in the scienecs

in the last decade, rnabng it the leading grantor of doctoral degrees to minority scientists. See Manuel GomeL “A Comprehensive Regional

Center to Develop Human Resowes in Science and mathematics in Puerto Rico,” presented at the Fifth EPSCOR Conference Aug. 15, 1990.

longstanding or emerging needs in novel ways. Such the agency mission, or address multiple deficiencies

departures are almost always seen as diluting in the distribution of research funds and the execuquality, i.e., trading off excellence in research for the tion of research?

fulfillment of “subsidiary” agency objectives. But This question cuts to the core of this study: What

at what point do these objectives become central to does the Federal Government expect research fund-
Chapter 4-The Federal Research System: The Research Agencies . 133

ing to accomplish? Entering the 1990s, OTA foresees agency funding criteria and methods, on the one

hand, and researcher expectations, on the other,

changing to accommodate a wider range of demands

imposed on the Federal research system.

Summary

In this chapter,

research agencies,

OTA has introduced the Federal

and outlined their priority setting

and funding allocation mechanisms. In general, the

Federal agencies are characterized by diversity,

pluralism, decentralization, and a division of labor,

but together they form a comprehensive research

system.

Each agency follows its research mission, but

there is much disagreement, both within the agencies

and in various research communities, over what

constitutes that mission. Agency programs and

research foci change in response to shifting priorities, but as with all large organizations, this change

occurs slowly. The pace of change is especially

hampered in research by the long-term nature of the

work and by the inability to reorient programs

quickly. Risk-taking, flexibility, strategic planning,

and redirecting agencies are longstanding challenges.

Agencies use a combination of peer review and

manager discretion to allocate funds. In addition to

the mainstream programs, agencies also create

set-aside programs to foster the development of

underprivileged parts of the research community. In

another type of funding, some agencies (especially

USDA) disburse funds by formula, which are

allocated as block grants to specific institutions.

Agencies have a good sense of their research

constituencies and attempt to cultivate both their

development and long-term responsiveness. Nevertheless, much of the brunt of the pressure on the

scientific community is reflected in agency programs. Program managers must make tough decisions about where to allocate funds and how to

support personnel, facilities, and equipment.

In summary, agencies have the resources to adapt

to changing internal and external priorities. However, Congress may wish to increase agencies’

ability to set and coordinate goals and to address

other issues. These issues—priority setting at several levels of decisionmaking, costs of research,

human resources for the research work force, and

data collection and analysis on the Federal research

system—are discussed in the following chapters.
CHAPTER 5

Priority Setting in Science
Contents

Page

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

Historical Justification for Priority Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

The Funding Climate and Research Priorities . . . . . . . . . . . . . . . . . . .. .. ... . .,. +....,, 140

The Science Base ● +*.... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ,,,... 146

Peer Review and Priority Setting Across Broad Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

The Dilemma of AgencyPriority Setting . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . ,.,,, 149

Science Megaprojects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

What Constitutes a Science Megaproject? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

The Process of Megaproject Selection ... .+. ...+,.., . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

Megaproject Costs and Management, . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . 162

Research Priorities and the ’’Big Picture” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

Conclusions . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . .+.*... . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Boxes

Box Page

5-A. A Statement From the Scientific Community on the Evaluation of Competing

Scientific Initiatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

5-B. Criteria for Awarding a Magnet Research Laboratory: NSF, Florida State,

and MIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . 145

5-C. Peer Review in Changing Environments: Remarks at a Roundtable Discussion . . . 148

5-D. Priority Setting by the Ecological Research Community . . . . . . . . . . . . . . . . . . . . ..,. 150

5-E. Federal Investment in Computer Science . ............,...,...,.,.,,,4,..,.,,. . 152

5-F. Behavioral and Social Sciences: Organization and Federal Funding . . . . . . . . . . . . . . 155

Figures

Figure Page

5-1. Federally Funded Research by Broad Field: Fiscal Years 1969.90. . . . . . . . . . . . . . . 147

5-2. Distribution of Federal Support of Neuroscience Research: Fiscal Year 1990 . . . . . 151

5-3. Where the Money Goes for Three Megaprojects

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

5-4. Cost Scenarios for the Science Base and Select Megaprojects: Fiscal Years

1990-2005., ... +... . . . . . . . . . . . . . . . . . ...,.....,., . *....+**......*.......*,,.. 165

Tables

Table

5-1. s

Page

ummary of President Bush’s $1.4 Trillion Fiscal Year 1991 Budget, Items Listed

Under Theme I: Investing in the Future . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

5-2. Favorable Senate and House Votes on Science Issues in the 101st Congress . . . . . . 142

5-3. Two Statements on Research Priorities ... ... ... ... .,. ... ... ... .+. .,. + .,,.,4++, 143

5-4. Framework for Assessing Science and Technology Budgets . . . . . . . . . . . . . . . . . . . . . 144

5-5. Inhouse Evaluation for the NSF Strategic Plan: Research Advances and

Opportunities Lost or Postponed, Research and Related Directorates,

Fiscal Years 1987-90. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

5-6. A Comparison of Science Megaprojects . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 5-7. Four ‘‘Big Science” Initiatives in the Fiscal Year 1991 Budget . . . . . . . . . . . . . . . . . 163
CHAPTER 5

Priority Setting in Science

Even if we could double the science budget tomorrow, we would not escape the need

to establish. priorities. . . . At present we have no well-de fined process. . . for systematically evaluating the balance of the overall Federal investment in research and

development and in the variety of fields that we try 10 serve” . . . .

Introduction

At every level of decisionmaking in the Federal

research system, goals are outlined and translated

into plans for their achievement. For the system to

provide both continuity and flexibility in research

funding, priorities are set, chiefly through the budget

process. Both the executive and legislative branches

have mechanisms to set priorities, many of which

were detailed in the two previous chapters. However, broad priority setting is generally resisted by

the recipients of Federal funding because it orders

the importance of research investments, often in

ways that groups within the scientific community do

not support. This problem is especially perplexing,

because there are few mechanisms and no tradition

of ranking research topics across fields and subfields

of inquiry.

Priority setting can help to allocate Federal resources both when they are plentiful, as they were in

the 1960s, and when they are scarce, as is expected

in the early 1990s. Governance requires that choices

be made ultimately to increase the benefits and

decrease the risks to the Nation. For example,

decisionmakers in the Office of Management and

Budget (OMB) routinely compare the projected

costs, benefits, and risks of certain programs. The

benefits of research increase technological capability, national security, health, economic activity, and

educational resources. Setting priorities is a way the

government achieves national goals.

Doug Walgrenl

In the grand scheme of things, research is one

Federal concern among many, routinely costing less

than 2 percent of the domestic and defense budgets.

Research has traditionally been a favored part of the

budget-only four budget areas have consistently

received increases over the 1970s and 1980s:

entitlements, defense, payments on the debt, and

research. 2 Consider the President’s proposed fiscal

year 1991 budget. The items in this $1.4 trillion

budget are organized under five themes. The first

theme, “Investing in the Future,” features science

and technology items most prominently among the

10 categories listed (see table 5-l). Five of these

categories explicitly mention science or research

goals.

What the Federal Government values more or less

in research can be inferred in part from the Federal

budget. The budget process compares the goals of

the President, the Office of Science and Technology

Policy (OSTP), the agencies, and Congress-not

only what each seeks to achieve, but also how they

plan to do so. However, no organization looks across

the Federal research system to determine the framework for making choices.

From the discussion in chapter 3, one could conclude that OMB has been the surrogate for such an

agent, with Congress then adding its own priorities

through budget negotiations. 3 The agencies spend

these appropriated sums based on strategic plans that

reflect their research missions, sorting long-range

from short-range investments, weighing new initiatives against ‘out-year’ commitments (in multiyear

IDoug Walgreq C “hauman of the Subcommittee on Science, Research and Ikchnology, in U.S. Congress, House Committee on Science, Space, and

lkchnology, The Hearings inadequacy, Direction, and Priorities for the American Science and Technology Effort, IOlst COng., Feb. 28-MM. 1, 1989

(Washington DC: U.S. Government Printing Wlce, 1989), p. 1.

26 ‘outlays by Category, “ Government Executive, vol. 22, September 1990, p. 44. Furthermore, within the category of “R&D,” research has seen

much greater increases than development (which has decreased in constant dollars) since the late 1960s. See Lois Ember, “Bush’s Science Advisor

Discusses Declining Mdue of R&D Dollars,’ Chem”cal & Engineering News, vol. 68, No. 17, Apr. 23, 1990, pp. 16-17.

sFor ~Ovemiew, s= EltiabethBaldti and Christopher T HiU, “The Budget Process and Large-Scale Science Fwd@,” CRS Review, Februrw

1988, pp. 13-16.

-137 –
138 ● Federal/y Funded Research: Decisions for a Decade

Table 5-l—Summary of President Bush’s $1.4 Trillion Fiscal Year 1991 Budget,

Items Listed Under Theme 1: Investing in the Future

Increasing saving, investment, and productivity

Expanding the human frontier

Space:

1. Infrastructure

2. Manned exploration (Space Station Freedom, Moon-Mars

Mission)

3. increasing scientific understanding (global change, developing commercial potential, other)

Biotechnology

Superconducting Super Collider

Enhancing research and development

1. Doubling the National Science Foundation budget

2. Global change

3. Agricultural research initiative

4. HIV/AIDS

5. R&D for advanced technology

6. Magnetic levitation transportation

7. Science and engineering education

8. Research and experimentation tax credit

9. R&D by transnational companies

Investing in human capital

Education:

1. Preparing children to learn (including Head Start)

2. Targeting resources for those most in need (including K-12, Educational Excellence Act,

mathematics and science, historically Black colleges and universities)

3. Education research and statistics

Job training

Enhancing parental choice in child care

Ending the scourge of drugs

Protecting the environment (including global climate change research)

Improving the Nation’s transportation infrastructure

Bringing hope to distressed communities

Preserving national security and advancing America’s interests abroad (including the

Department of Defense research and technology)

Preserving America’s heritage

SOURCE: “President Bush’s 1991 Budget Fact Sheet,” Jan. 29, 1990.

awards), and allocating resources by program, project, and performer. Even this picture is too simple,

however, since many decisions involve extensive

debate within the government and the public, and

developments within programs and the scientific

community also influence the decisionmaking process.

Congress wishes-perhaps now more than ever—

that the scientific community could offer priorities

at a macro level for Federal funding. However, this

community has long declined to engage in priority

setting, claiming a lack of methods to compare and

evaluate different fields of science and desiring to

maintain high levels of funding for all fields, instead

of risking cuts in any particular one. It has fallen

primarily to the Federal Government to set priorities,

both among and within fields of science, and this

situation will most likely continue through the

1990s.

In the scientific community, calls for priority

setting are also often confused with calls to direct all

research along specified lines. Even with greatly

enhanced priority setting, one goal would certainly

be the maintenance of funding for a diverse science

research base. This priority has been preeminent

since the Federal support of research began.4 Other

priorities would include training for scientists and

engineers, and supplying state-of-the-art equipment.

At present, the means to meet these goals are a

matter of continuous debate and policy revision.

4See U.S. Conuess, House committee on Science and ‘Jkchnology, Task Force on Science Pclicy, A History of Science Policy in the United States,

T940-1985, 99th Cong. ~astigon, DC: U.S. Government Printing office, September 1986).
Chapter 5—Priority Setting in Science . 139

In an era of greater priority setting, the Federal

Government would seek to target specific goals. For

instance, the allocation of additional monies to the

National Institutes of Health (NIH) for AIDS research, beginning in the late 1980s and continuing to

the present day, has been a clear designation of a

priority research area. Future decisions may center

on ranking projects designated ‘‘big science, ’ since

not all of them can be supported in the current fiscal

climate. Similarly, fields that have received large

increases in funding during the 1980s, such as the

life sciences, may grow more slowly, as others are

given precedence.

Although priority setting occurs throughout the

Federal Government, it falls short in three ways.

First, criteria used in selecting areas of research and

megaprojects (e.g., the Superconducting Super Collider (SSC) and the Space Station) are not made

explicit, and appear to vary widely. This is particularly a problem at the highest levels of priority

setting, e.g., in the President’s budget and the

congressional decision process. Second, there is

currently no formal or explicit mechanism for

evaluating the total research portfolio of the Federal

Government in terms of progress toward national

objectives. Third, the principal criteria for selection,

‘‘scientiilc merit’ and ‘‘mission relevance, ’ are in

practice coarse falters.

This chapter examines priority setting in the

Federal research system. First, it describes the

historical justification for priority setting and recent

pressures stemming from budgetary constraints.

Second, it reviews specific frameworks for setting

priorities generated by various parts of the research

system. (For a discussion of priority setting in other

countries, see appendix D.) Most proposed frameworks include a distinction between “big” and

“little” science, both as research strategies and as

accounts with certain expectations. But definitions

are murky. OTA thus discusses the criteria applied

to justify investments in various categories and the

decisions that generate agency research “portfolies. ’ Finally, the use of priority setting to clarify

Photo credit: National Aeronautics and Space Administration

Astronaut prepares experiments, which separate cells

according to their electric charge, on board the Earthorbiting Columbia Space Shuttle. The difference between

big and little science is murky, in part because the

advent of new large equipment (such as the Shuttle) often

allows new forms of what would be called “little science”

when performed in other environments.

goals, strategies, and outcomes is analyzed as part of

democratic decisionmaking.5

Historical Justification for Priority Setting

Investment in research is open-ended and uncertain in outcome. Thus, Federal decisionmakers bring

different expectations and justifications to making

choices in research. Recognizing this, Alvin Weinberg, former Director of Oak Ridge National Laboratory, proposed over a quarter-century ago a set of

‘‘criteria of scientific choice. ’ ‘6 He wrote:

Society does not a priori owe the scientist, even

the good scientist, support any more than it owes

support to the artist or to the writer or to the

musician. Science must seek its support from society

on grounds other than the science is carried out

competently and that it is ready for exploitation. . . . Thus, in seeking justification for the support of science, we are led inevitably to consider

external criteria for the validity of science, those

criteria external to science or to a given field of

science. 7

 budget process plUS cycle of authorization and hearings revisit

projects, check their progress, revise cost and time estimates, and so on. But this is done piecemeal. Some mechanism viewing the entire research portfolio

is needed, perhaps on a different cycle than the budget. A more‘‘ideal’ Federal portfolio could be constructed iteratively-a process which

could fortify the science base while allowing for the pursuit of some, but not all, new big science initiatives.

 papers on the topic, originally published in are reprinted with discussion in A.M. Weinberg, Rejections Big Science

(Cambridge, MA: MIT Press, 1966).

 p. 72.
140 . Federally Funded Research: Decisions for a Decade

Weinberg’s ‘‘external” criteria consist of social

merit and technological merit. They declare the

support of science as a priority to be judged against

conscience investments and favor the ‘‘applied’

end of the research continuum. These criteria

conjure up the potential applications and social

value of scientific research. Science for society is

epitomized by such investment criteria.

Weinberg’s ‘‘internal’ criteria, on the other hand,

are those embraced by research performers and, to a

lesser extent, agency sponsors. For them scientific

merit is the prime justification for Federal support,

one that “. . . puts value on the progress of the

scientific enterprise as a whole. Knowledge production is thus held to be a meritorious activity in its

own right. . . .“8 With no promised immediate

benefit to society, the support of research has a more

esoteric justification, such as the “ripeness” of a

field for exploitation that will advance the state of

theory or technique. The significance of this outcome may remain within a research community or be

shared only by specialists in neighboring fields. For

them, such developments become a priority. Making

this intelligible and persuasive to those who control

resources, e.g., within agencies or to one’s congressional representative, however, is what may influence the policy process. A 1988 statement of the

priorities issue suggests that the criteria have not

changed much from Weinberg’s original formulation (see box 5-A).

Historically, the notion of criteria, with scientific

merit at its core, rearticulates the social contract that

ties Federal research funding policies to investigators and research programs that bubble up to excite

other specialists and agency sponsors. For Weinberg, “. . . the purest basic science [can] be viewed

as an overhead charge on the society’s entire

scientific and technical enterprise. ’ This conception of research as overhead on society’s near-term

goals has been reasserted of late with changes in the

Federal funding climate. Under the strain of demands on the Federal budget, the call for priority

setting has grown louder.

The Funding Climate and Research Priorities

The 101st Congress engaged in what has been

characterized as “. . . six of the most consequential

and rancorous science and technology debates. ”10

Four of these six are unambiguously research

related; they are presented by Senate and House

votes in table 5-2: mathematics and science education, the SSC, environmental protection, and space/

National Aeronautics and Space Administration

(NASA). (Note the overlap between the items listed

here and in the President’s priorities.) The need for

trained people, sophisticated instrumentation, the

reduction of risk, and continued exploration of space

reflect the relation of science and technology to the

Nation’s total market basket of investments.

Even though R&D still sit in the vulnerable corner

of the budget that carries the label of “discretionary’ spending, it’s clear that science and technology

no longer are viewed as flip-of-the-coin judgment

calls. Rather, they are now seen as necessary and

strategic obligations tied to national needs, and no

matter how awful the budget deficit looks, R&D will

get better relative consideration than anything else in

the discretionary sector. .. .11

However, under tight fiscal conditions, no part of the

budget may fare well. As Association of American

Universities President Robert Rosenzweig states:

Another thing that concerns me. . . is the dynamic that seems to be set up by the next three to five

years of budget problems. We’re going to be fighting

among ourselves a lot--universities and elements

within universities. . . . The domestic discretionary

[budget] pool . . . is not supposed to grow for the

next five years, save for inflationary increases. But

8John zima.u, An ZmVdUCtiOtI to Science Studies (New York NY: Cambridge University mess, 1984), P. 163.

Weinberg, op. cit., footnote 6, pp. 97-99. Also see Harvey Brooks, “Models for Science Plaoning,’ PubZicAdministration Review, vol. 31, May/June

1971.

l~ade ROUS& “science and lkchnology in the IOISt COngIeSS,” Technology Review, vol. 93, No. 8, November-December 1990, p. 59. These six

diffemdslightly in the House and Senate, and two-having to do with the Clean Air Act and the B-2 Stealth Bomber-have arguably little science content.

llWilliam D. Caey, “R&D in the Federal Budget: 1976- 1990,” Science and Technology and the Changing World Order, colloquium proceedings,

Apr. 12-13, 1990, S.D. Sauer (cd.) (Washington DC: American Association for tbe Advancement of Science, 1990), p. 48.
Chapter 5—Priority Setting in Science . 141

Box 5-A—A Statement From the Scientific Community on the Evaluation of

Competing Scientific Initiatives

The following criteria were proposed in 1988 for evaluating competing scientific initiatives. They are presented

here (in abridged form) in the three categories developed by the authors.l

Scientific Merit

1. Scientific objective and significance

Example: What are the key scientific issues addressed by the initiative?

2. Breadth of interest

Examples: Why is the initiative important or critical to the discipline proposing it? What impact will the

science involved have on other disciplines?

3. Potential for new discoveries and understanding

Examples: Will the initiative provide powerful new techniques for probing nature? What advances beyond

previous measurements can be expected with respect to accuracy, sensitivity, comprehensiveness, and

spectral or dynamic range? In what ways will the initiative advance the understanding of widely occurring

natural processes and stimulate modeling and theoretical description of these processes?

4. Uniqueness

Example: What are the special reasons for proposing this initiative? Could the desired knowledge be

obtained in other ways? Is a special time schedule necessary for performing the initiative?

Social Benefits

1. Contribution to scientific awareness or improvement of the human condition

Examples: Are the goals of the initiative related to broader public objectives such as human welfare,

economic growth, or national security? Will the results assist in planning for the future? What is the

potential for stimulating technological developments that have application beyond this particular initiative?

Will the initiative contribute to public understanding of the goals and accomplishments of science?

2. Contribution to international understanding

Example: Will the initiative contribute to international collaboration and understanding?

3. Contribution to national pride and prestige

Example: Will the initiative create public pride because of the magnitude of the challenge, the excitement

of the endeavor, or the nature of the results?

Programmatic Concerns

1. Feasibility and readiness

Examples: Is the initiative technologically feasible? Are there adequate plans and facilities to receive,

process, analyze, store, distribute, and use data at the expected rate of acquisition?

2. Scientific logistics and infrastructure

Examples: What are the long-term requirements for special facilities or field operations? What current and

long-term infrastructure is required to support the initiative and the processing and analysis of data?

3. Community commitment and readiness

Example: In what ways will the scientific community participate in the operation of the initiative and the

analysis of the results?

4. Institutional implications

Examples: In what ways will the initiative stimulate research and education? What opportunities and

challenges will the initiative present for universities, Federal laboratories, and industrial contractors? What

will be the impact of the initiative on federally sponsored science? Can some current activities be curtailed

if the initiative is successful?

5. International involvement

Example: Are there commitments for prog rammatic support from other nations or international

organizations?

6. Cost of the proposed initiative

Examples: What are the total costs, by year, to the Federal budget? What portion of the total costs will be

borne by other nations?

l~pted fim Job A. Dutton and Lawson Crewel “Setting priorities Among Scientific Initiatives, ” Amen”can Scientist, VO1. 76,

November-December 1988, pp. 60C)-601.

—— —
142 . Federally Funded Research: Decisions for a Decade

Table 5-2—Favorable Senate and House Votes on Science Issues in the IOlst Congress

Senate votes

Mathematics and science education programs: S. 695, President Bush’s “Excellence in Education Act, “ includes $5 million for a national

Science Scholars program. Passed 92-8 on Feb. 7, 1990; R 37-8, D 55-0.a

Superconducting Super Collider authorization: H.R. 5019 appropriates $20.8 billion for energy and water programs, including $318 million

for the accelerator. Passed by voice vote on Aug. 2, 1990.

Technology programs authorization: S. 1191 authorizes $320 million in fiscal year 1990 funds for research on high-definition television and

other new technologies through the Advanced Technology Program of the National Institute of Standards and Technology. Passed by

voice vote on Oct. 26, 1989.

House votes

Superconducting Super Collider authorization: H.R. 4380 limits Federal spending on the advanced atom smasher to $5 billion, with $2.4

billion more to come from Texas and foreign sources. Passed 309-109 on May 2, 1990; R 115-57, D 194-52.

Mathematics and Science Education amendrnents:H.R.5115 authorizes $1.1 billion in fiscal years 1991 to 1995 for congressional science

scholarships and other education reforms. Passed 350-25 on July 20, 1990; R 123-25, D 227-O.

Technology programs authorization: H.R. 4329 funds the National Institute of Standards and Technology through 1992, including $100

million in fiscal year 1991 and $250 million in fiscal year 1992 for research on high-definition television and other new technologies under

the Advanced Technology Program. Passed 327-93 on July 11, 1990; R 83-90, D 244-3.

National Aeronautics and Space Administration funding:H.R.5158 appropriates $14.3 billion for NASA. Passed 355-48 on June 28, 1990;

R 128-39, D 227-9.

KEY: R= Republicans; D= Democrats; NASA=National Aeronautics and Space Administration.

aBoth the House and the Senate ~a=~ t})e ~cellen~ in Mathematics, ~ien~, and Engineering Act of 1990 (Put)l~ kw 1 ol-568) in ~tOber 1990, and

$149 million was appropriated.

SOURCE: Based on Wade Roush, “Science and Technology in the 101st Congress,” Technology Review, vol. 93, No. 8, November-December 1990, p. 65.

everybody is going to be out to get more money.

They all feel that they deserve and need more money,

and they’re probably right.12

These commentators, speaking 2 years after

National Academy of Sciences (NAS) President

Frank Press warned of constrained research budgets

as ‘the dilemma of the golden age, ’ 13 suggest some

accommodation to this reality: while the Federal

Government could invest more in science and

technology, the scientific community could do a

better job of sorting research opportunities by

whatever criteria chosen to assist decisionmakers at

all levels of the system.

Science Advisor Bromley and Former Science

Advisor Press have stated criteria and categories of

priority that they consider essential for science,

listed in table 5-3. (Projects are compared under each

category to compete for monies allocated within that

category.) Note the convergence between the Science Advisor’s (OSTP/OMB’s) and the NAS President’s (and former Science Advisor’s) formulations.

Each emphasizes the separation of large projects

requiring new infrastructure from ‘‘small science. ”

Press distinguishes human resources from national

crises and extraordinary scientific breakthroughs in

his primary category. Bromley places national

political exigencies above all else,14 whereas Press

prefers to put these items into a “political category”

of third priority. One effect of these rank orders is the

seeming creation of separate accounts, i.e., that

choices could be made within each category and

then across categories.

15 Of course, such choices are

being made by various participants in the research

12Qu~@d ~ CCA (jWd Budget for Science, But T~~ubleS Lie ~~d,” science& GoVernmen~Rep~rt,VO1. 20, No. 18, NOV. 15, 1990, pp. 1,4. hl the

President’s proposed fiscal year 1992 budge~ civilian R&D spending would rise 13 percent to $76 billioq with basic science increasing 8 percent to

$13 billion. see William Boo@ ‘‘President PUS Fisc~ Faith in Science,’ The Washington Post, Feb. 13, 1991, p. A17. Also see Jeffrey Mervis, “Bush’s

Science Budget: Will It Hold?” The Scientist, vol. 5, No. 5, Mar. 4, 1991, pp. 1,6-7.

13Frank Press, “The Dilemma of the Golden Age,” Congressional Record, May 26, 1988, pp. E1738-E1740. Press’s categories and priorities are

presented below.

1dBro~ey>s s~tement was ~umented ~ Septemkr 1990 by a brief Office of Science and ~chnology policy d~~en~ “U.S. ‘kchnoIogy poficy. ”

The document seines to bridge the roles of the private sector and the Federal Government in research and development. Justifications for the President’s

fiscal 1991 budget requests for “education and tr aining” and “Federal R&D responsibilities” are presented by agency in addition to discussion of

federally funded technology transfer and Federal-State activities. See Executive Office of the president OffIce of Science and ‘Ikchnology Policy, ‘ ‘U.S.

7kchnoIogy Policy,” unpublished document, Sept. 26, 1990.

IsNote that scientific merit is assumed in ~th formulations and not explicitly stated as a funding criterion. The iSSue bXOmeS one of f~st -g

science projects according to scientific merit and then assigning them to national goal categories, or alternatively starting from a mtional goal and

organizing a research stmtegy to meet it.
Chapter 5—Priority Setting in Science . 143

Table 5-3—Two Statements on Research Priorities

Source Criteria Categories in rank order

Press

Bromley 6’

. . . guiding principles on prioritiz- 1. National needs and international security concerns (global change,

ing the agency requests . . .“ preeminence in space, defense technology base).

2. Support for basic research (particularly university-based, individualinvestigator and small-group research-’ ’small science”).

3. Funding for scientific infrastructure and facilities (SSC, Space Station,

and”. . . in a more distilled sense . . .“ Human Genome).

“

. . . appropriate for the unprece- 1. Human resources, national crises (AIDS, space launch capacity), extradented Federal deficit. . .“ and ordinary scientific breakthroughs (high-temperature superconductivity). (’

. . . to maintain American leader- 2. Large projects (SSC, Human Genome).

ship in science and technology. . .“ 3. Political category (DOD and national security; Space Station; regional

economic development and employment; U.S. image enhancers like

manned space flight; U.S. “competitiveness” enhancers like education,

training, and civil sector R&D).

KEY: SSC=Superconducting Super Collider; DOD=U.S. Department of Defense.

SOURCES: D. AlIan Bromley, “Keynote Address” Scierwe and Technology and the Changing Wx/dorder, colloquium proceedings, Apr. 12-13, 1990, S. L.

Sauer (cd.) (Washington, DC: American Association for the Advancement of Seienee, 1990), p. 11. Also see “Q&A With D. AlIan Bromley, Bush’s

Science Advisor,” Science & Government Report, vol. 20, June 1, 1990, P. 5; and Frank Press, “The Dilemma of the Golden Age,” Congressional

Record, May 26, 1988, pp. El 738-E1740. ‘

system simultaneously. The congressional budget

process may be the final arbiter, but even after

Federal monies are obligated, choices at the agency

and program levels occur.

In addition to supporting meritorious research,

most Federal research agencies would embrace the

following as relevant to their mission:l6

●

●

●

o

to provide fiscal support to the research system

(both the infrastructure needed to conduct

research and the research itself);

to invest in human capital today (i.e., the

research work force) and tomorrow (i.e., student apprentices);

to sustain the performance sector of research

(especially the research universities) and to

build institutional capacity (especially as

viewed by region or State); and

as a factor in economic development and the

application of research to solving local problems.

Clearly, not every program in every research agency

can apply these as finding criteria without compromising any single one.

In response to a congressional request in 1988,

NAS also devised a framework for thinking about

Federal science and technology budget priorities.

The result is presented in table 5-4. In this fourcategory scheme, “agency budgets and missions”

are viewed as separate from needs of the “science

and technology (S&T) base, ” “national [political]

objectives, and ‘‘major S&T initiatives. ” All are

illustrated by NAS at the agency level, listing the

following needs: educating science and engineering

personnel; modernizing equipment and facilities;

supporting a mix of basic and applied research;

capitalizing on promising new research opportunities; promoting interactions between related fields of

science and engineering research; distributing research support by geographic region and type of

institution; maintaining a mix of research modes,

e.g., individual investigators, large groups, centers,

and university-industry partnerships; and balancing

competitiveness and cooperation with research programs in other countries.17

If these items were interpreted as listed in order of

importance, top to bottom, the projects funded by

the research agencies (indeed, the proposals received) might look quite different from the research

projects currently supported. Priorities can perturb

the funding system; they can redefine the “haves”

and ‘have nets’ (e.g., institutions, fields, investigators) by changing the value of certain criteria. For

instance, some agency funding decisions signal that

a premium has been placed on other needs (see box

5-B).

IGoTA interviews at the Federal research agencies, spring-Sumxn er 1990.

ITNatio~ A~demy of Scienws, Federal Science ad Technology Budget Priorities: New Perspectives and Procedures, a repOfi m response to the

Conference Report on the Concurrent Resolution on the Budget for Fiscal Year 1989 (H. Con. Res. 268) (Washingto~ DC: National Academy Press,

1988), p. 10.
144 . Federally Funded Research: Decisions for a Decade

Table 5-4-Framework for Assessing Science and Technology Budgets (categories are not mutually exclusive)

Category Definitions Examples

Agency budgets and Agency S&T activities viewed in terms of

missions their contributions to individual agency

goals and objectives

S&T base Activities that provide the people

knowledge, and infrastructure to carry

out S&T

Activities supported across many

agencies and under the jurisdiction of

several congressional committees

S&T applied to Stated priorities of the President and

national objectives Congress with major S&T components

(Presidential and Frequently supported by several

congressional agencies and within the purview of

priorities) several congressional committees

Major S&T initiatives Significant increase (and sometimes decreases) in budgets over several years

Budgetary consequences across

agencies

Fail in one or more of above three

categories

Nuclear alternative energy R&Din DOE

Submarine acoustics in DOD

Cell biology in HHS

Influence on learning in ED

Plant disease resistance in USDA

Fundamental research in chemistry in NSF

Standards development in NIST

Aeronautical research in NASA

Basic and applied research programs in NSF, HHS, DOD, DOE,

NASA, USDA, EPA, etc.

Student fellowships in ED, NSF, HHS, DOD, DOE, NASA, etc.

Equipment and instrumentation programs in HHS, DOE, NSF, USDA,

NASA, DOD, etc.

Facilities for research, animal care, and growing and using special

materials supported by NSF, DOD, HHS, DOE, NASA, etc.

K-12 materials development in NSF, ED, NASA, etc.

Student internships in Federal laboratories in DOE, NIH, etc.

Understanding and ameliorating global change in EPA, DOE, NSF,

NASA, USDA, NOAA, etc.

Industrial development in biotechnology, superconductivity, manufacturing technologies in HHS, DOD, Commerce, NASA, NSF,

DOE, USDA, etc.

Alternative sources of energy in DOE, NSF, DOD, USDA, etc.

AIDS in HHS, ED, DOD, State Department, etc.

Creation of nuclear defense (Strategic Defense Initiative in DOD)

Increase capacity for exploration of space (Space Station in NASA)

Superconducting Super Collider

Mapping and sequencing the human genome

Space Station

KEY: DODAJ.S. Department of Defense; DOE=U.S. Department of Energy; EIMJ.S. Department of Education; EPA=U.S. Environmental Protection Agency;

HHS=U.S. Department of Health and Human Services; NASA=National Aeronautics and Space Administration; NIH-National Institutes of Health;

NIST=National Institute of Standards and Technology; NOAA=National Oceanic and Atmospheric Administration; NSF= National Science Foundation;

R&D-research and development; S&T-science and technology; USDA=dJ.S. Department of Agriculture.

SOURCE: National Academy of Sciences, Federa/ Scierrce and T~no/ogy Bu~ef Priorities: New Perspectives and Procedures (Washington, DC: National

Academy Press, 1988), table 1, p. 7.

Concern for the S&T base closely approximates unexpected shortages of science and engineering

the needs of research. In the words of the NAS personnel, or changing institutional relationships

report: (e.g., the emergence of university-industry research

partnerships). And as if that were not a sufficient

The S&T base is the bedrock of the Nation’s challenge, budget makers and analysts must be

ability to use science and technology in the national attuned to differences among a wide range of fields.

interest and. . . it requires continual replenishment. Some changes affect many disciplines, others only a

Continuity does not imply steady funding of the part of a single discipline.l8

same activities and institutions through the same

programs and agencies year after year. On the Frameworks such as OSTP’s and NAS’s help to

contrary, the enterprise ought to be highly dynamic. demarcate the tradeoffs that could be made and assist

Policymakers must be able to respond flexibly to decisionmakers to understand that priority setting is

scientific breakthroughs that suddenly transform an a dynamic process. Priorities change with goals. As

area of research (e.g., high-temperature supercon- Weinberg put it: ductility), the invention of a powerful new instrument (e.g., gene-sequencing machine) or concep- . . . we cannot evaluate a universe of scientific

tions of new facilities that would aid research and discourse by criteria that arise solely from within that

training (e.g., supercomputer centers and networks), universe. Rather, we find that to make a value

181bid., p. 5.
Chapter S—Priority Setting in Science ● 145

Box 5-B—Criteria for Awarding a Magnet Research Laboratory: NSF, Florida State, and MIT

In August 1990, the National Science Board (NSB) of the National Science Foundation (NSF), decided to

award a $60-million grant to Florida State University to establish a national laboratory for magnet research.

Then-NSF Director Erich Bloch admitted that peer reviewers had found the proposal from the Massachusetts

Institute of Technology (MIT), home of the Francis Bitter National Magnet Laboratory, “technically

superlative, ” but cited the greater “enthusiasm” of the Florida investigators, the State of Florida’s pledge to

contribute $58 million, and other factors in funding the Florida proposal.1

The issues involved in the NSF decision are many. At one level, the award is evidence that scientific merit

is not enough to guarantee success in competition for a facility where there can be only one winner.2 NSF cited

as decisive the superior ‘‘management plan” in the Florida proposal. Clearly, the message being sent-part of

Bloch’s larger emphasis on centers and government-industry partnerships to enhance U.S. economic

competitiveness-was the rules of the game are changing: criteria other than technical merit are weighed in

determining qualification to manage and execute a multiyear research program requiring the expertise of

investigators from various institutions. 3

In the magnet laboratory competition, the commitment of MIT was found wanting. According to NSF

Assistant Director David A. Sanchez: “. . . you need support from the institution, you need support from the

State, and we did not see that ., .“ from MIT.4 NSB concurred.

The MIT protest of Florida State’s selection was not limited to NSF’s decision to overrule its reviewers’

recommendations. MIT President Paul Gray appealed on several grounds. First, the delay caused by construction

of the Florida State facility”. . . is hardly compatible with NSF’s interest in the competitive posture of the United

States. Second, some fear that projects with significant State support, so-called leveraging of Federal funding,

will put private universities at a disadvantage. Third, expertise in the Florida State physics department may be

lacking. 5

Consider, too, the symbolism of the decision. As one columnist put it: “So maybe the mandarins from MIT

got caught napping. Maybe. Or maybe not. ” 6 MIT epitomizes the Northeast science establishment.7 The

Southeast is, in a sense, an underutilized region for research. Awards such as the magnet laboratory signify that,

in specific cases, institutional collaborations can make a State or region competitive for Federal research funding.

Such awards build research capability almost from the ground up; they are a capital investment that

diversifies research performers--with short- and long-term consequences for the research community and the

Nation. Decisions such as this one also call for evaluation: what happens to magnet research while the Florida

State facility is being constructed? Will the State of Florida deliver on its pledges? And is there any impact on

the competitiveness of U.S. researchers in fields that use powerful magnets, such as superconductivity and

magnetic-resonance imaging?

Ism @l& Blwm~ 46stia= Ag~q ~~~~& Sta@ over ~as si~ for $60-Million w~t-s~dy ~,’ The Chrom”cZe

of Higher E&can-on, vol. 37, No. 1, Sept. 5, 1990, p. A21.

2~e a-d of a 5-year, $2S * eart@u& project to a eonsortiurn centered at the State University of New York at Buffalo sent

a similar signal to Cahechand aCaliforniaconsortiurn in 1987. It also led to a ~Accounting Ofi3ce (GAO) investigation of the National

Scienec Foundation (NSF) review proecss that sanctioned the award. While it sustained the fairness of the NSF process, it did question its

doenmentation procedures. See U.S. @neral Accounting Offi~ National Science Founalrtion: Problems Found in Decision Process for

Awarding Earthquake Cenrcr, GAO/RCED-87-146 (Washington DC: June 1987).

3~o~~ Swe ~ t. ~ PM ~the univeml~ of ~orida and bs *OS National LrdMmtorY b New Me* in _*~-t

laboratory a reality.

4~ B] ~Wk OP. cit., footnote 1, p. A22. National Scimce Foundation reviewers said the Massachusetts Institute of lkchndogy’s

“decriying plant’ ‘ would require substantial mod ernir.ation. ‘I%e institution wiIl submit a proposal for fhrther support until the Florida State

laboratory begins opemtions in 1993.

5~1 of -pfi~ fi~~ of ~ N~o~ ~l~c Bored(ep~y~C&kof‘‘work@ ~ientists’ arnong its members) WCikd

by a trio of Princeton physicists in Philip W. An&son et al., “NSF Magnet Lab,” lcttcz, The Scienzt”sr, vol. 4, No. 23, Nov. 26, 1990, p.

14, The Florida State proposal ineiuded a pledge from the State ‘‘. . . to add 24 new faculty members and 10 laboratory experts and to provide

20 annual fellowships for visiting scientists from around the world. ”

6Da~d w-, “WU ~~~~~e a New B~tion of ~~~Scieu?” The Washington Post, sep~ 12, 1990, p. C3.

7~e MNWhmt~ ~ti~te of ~~olo~ ~mrnm w~ to i~~de BO,SIOU B~eis, Hmvfud, Northwt~ d ~

universities,
146 . Federally Funded Research: Decisions for a Decade

Photo credit: National Aeronautics and Space Administration

Researcher studies magnetic liquids. In this example of

little science, the research is supported by the National

Aeronautics and Space Administration.

judgment, we must view the enterprise from a

broader point of view than is afforded by the

universe itself. . . . And so it is with the rest of

science. The scientific merit of a field must be judged

in large part by the contribution it makes, by the

illumination it affords, and by the cohesion it

produces in the neighboring fields.19

Leaders of the scientific community have subscribed

to the need for something other than ad hoc

policymaking for research funding. OTA next examines the problems inherent in two categories of this

funding-the science base and science megaprojects.

The Science Base

Little science is the backbone of the scientific

enterprise, and a diversity of research programs

abounds. For those who believe that scientific

discoveries are unpredictable, supporting many

creative researchers who contribute to S&T, or the

science base, is prudent science policy. In the words

of one geographer: ‘‘The continued survival of our

intellectual free market is important to scientific

progress.’ ’20 Not surprisingly, many investigators

and their teams shudder at the thought of organizing

Federal research funding around a principle other

than scientific merit. They fear that setting priorities

would change the criteria by which research funds

are awarded.21 They would run the risk of losing

what they consider their fair market share. Does

priority setting necessarily curb the search for new

knowledge, or just redirect it?

Consider the research portfolios of the Federal

Government. As shown in figure 5-1, broad field

funding, 1969 to 1990, has favored the life sciences,

almost doubling in constant dollars during that

period. Mathematics/computer, physical, and environmental sciences have also increased; engineering

has remained stable in funding; and social sciences

have decreased. In retrospect, should these be

decried as less than rational choices? With a change

in the Federal funding environment, should the

ground rules for allocating resources among broad

fields and performers also change? And what role

can peer review play?

Peer Review and Priority Setting Across

Broad Fields

Peer review is used in a variety of ways within the

Federal agencies. As seen in chapter 4, only a few

agencies, primarily the National Science Foundation

(NSF) and NIH, employ peer review throughout

their priority-setting and funding allocation processes. At NSF and NIH, peer review is considered

to be:

effective for communicating expert opinion

about what proposals definitely should arid

should not be funded (and the large gray area in

between) within a narrow band of specialization corresponding to the scope of an agency

program;

 footnote 

 Robert “Evaluating Scientific Initiatives,” letter, American Scientist, vol. 77, No. 3, May-June 1990, p. 213.

 seem to do) with tactics (how to do it). Criteria correspond to strategies, while project selection methods (e.g.,

peer review) represent tactics or ways to identify research that helps achieve stated 
Chapter 5—priority Setting in Science ● 147

Figure 5-l—Federally Funded Research by Broad

Field: Fiscal Years 1969-90

(in billions of constant 1982 dollars)

Billions of 1982 dollars

7 ~

3

2

1

( )+

I I I I I I I 1 I I I I I I oI I I I

I 1969 1974 1979 1984 1989

Life Physical

— science s

+ Engineering + sciences

~ Environ- + Social + Math/computer

mental sciences sciences

sciences

NOTE: Research includes both basic and applied. Fields not included in

this figure collectively accounted for $1.1 billion (4.9 percent) of all

federally funded research in 1990. Figures were converted to

constant 1982 dollars using the GNP Implicit Price Deflator. 1990

figures are estimates.

SOURCES: National Science Foundation, Federal Funds for Research and

In

Development, Detailed Historical Tables: fiscal Years 1955-

1990 (Washington, DC: 1990), table 25; and National Science

Foundation, Selected Data on Federal Funds for Research

and Development: Fiscal Years 1989, 1990 and 1991 (Washington, DC: December 1990), table 1.

efficient, in terms of the time, money, and

energy involved in the process of deciding how

resources should be allocated; and

accountable, ensuring that the highest standards of rigor (valid and reliable measurement),

safety (for animals, human subjects, and laboratory personnel), and freedom (e.g., to follow

hunches, train students, and exchange data) in

research are observed.

sum, peer review is expected to be robust and

responsive to changing agency and program needs.

Satisfying all of these criteria simultaneously, however, is difficult at best (see box 5-C) and, in

practice, a compromise is struck between them.

Federal monies awarded to researchers for some

expressed purpose other than or in addition to

‘‘scienttilc merit’ are seen by many as inferior to

monies for projects selected by peer review processes using scientific merit alone. Some are inclined to the view that there is something inherently

wrong with such ‘‘political allocations.” The policy

issue is whether peer review can simultaneously

serve to discern scientific merit and help in projectbased priority setting.

Reviewing for “truth,’ as science policy statesman Harvey Brooks writes, differs from reviewing

for “utility.” Peer scientists are not very helpful

with the latter.22 In Weinberg’s terms, criteria of

scientific merit clash with criteria of social or

technological merit. Peer review as a tactic tends to

break down when confronted with incommensurate

information from competing disciplines, fields, or

projects. As two commentators ask:

Should peer review operate only to evaluate merit

or should it also help establish priorities? Can it or

should it be effective in changing the direction of a

program, in allocating resources among programs

within agencies themselves? These questions are

significant because they challenge the assumption

that peer review is the best possible way to allocate

resources in the best overall interests of both science

and society.23

Recognizing the limits of specialization, agencies

maximize expertise in subject-focused programs.

Specialists are quite well-suited to the task of

making quality distinctions within disciplinary or

problem-centered boundaries. But discriminations

that must cross boundaries, no longer comparing like

with like, are rarely ever accomplished by peer

review, since reviewers in one field are very

reluctant to judge the scientific or technical merits of

information from other fields. There are no rules

inside the scientific enterprise that suggest that one

kind of information is superior to another. The

22Harvey Brooks, “The Problem of Research Priorities,’ Daeakdus, vol. 107, No. 2, spring 1978, pp. 171-190.

z~c~d C. A-on and William A. Blanpied, “Peer Review and the Public Interes~’ fssues in Science & Technology, vol. 2, s ummer 1985, p.

110.
148 . Federally Funded Research: Decisions for a Decade

Box 5-C—Peer Review in Changing Environments: Remarks at a Roundtable Discussion

In June 1990, the Forum on Research Management (FORM), consisting in equal parts of program

officers from the behavioral science divisions of various Federal agencies and of senior researched and

research managers from academia and the private sector, met to discuss peer review.1 Two dozen FORM

members discussed some of the pros and cons of peer review in an era of fiscal austerity. Their positions

as agency administrators faced with allocation decisions, as lobbyists surveying the funding scene, and as

researchers competing for scarce program dollars give them an acute sensitivity to proposal review and the

environments in which it is carried out. The remarks are as verbatim as the edited transcription allowed.

Each bulleted item represents a different speaker.

There is a connection. . . between tight funding and peer review. As money gets tighter, peer

reviewers become more conservative, less prone to take risks.

What they [peer reviewers] are doing is giving higher and higher ratings, which in effect increases

the noise in the system. So the peer review system is calling more proposals “excellent” and

‘ ‘outstanding,‘‘ and the consequence is that it is very difficult for program managers to make

evaluations. What results is a beauty contest or just chance.

Has the science changed? Has the quality of the proposals changed? I think the answer to both

questions is yes. . . . Peer reviewers used to be tightly knit groups ex amining proposals from people

they knew extremely well—it was a very closed society. Now it is a much more complicated task.

Would it really be valuable to have a peer review system and an amount of money where everything

was funded? I suspect it may lead to very bad science.

There are two things going on in peer review---one is selection, which is important, but the other

is education (of the proposer and reviewer). I think the latter function sometimes gets lost.

Unfortunately, crushing workloads are reducing the educational function of peer review.

When I serve on a [National Institutes of Health] study section, I find it extremely disconcerting and

distracting to be told by program people about what percentage of the applications are likely to be

funded. It distorts my entire approach, as well as that of my colleagues, For instance, if we’re told

only 10 percent are likely to be funded, we start playing with the ratings to ensure certain results.

Study sections are not supposed to be making funding decisions. They are to make scientific

recommendations. There should be recognition that there are two discrete sets of staff used in NIH

peer review. . . . Priority scores do not determine funding. That’s what advisory councils and

institute directors are for.

People are increasingly reluctant to get involved [in peer review]. . . . I wonder if we are losing

certain types of reviewers from the process--not just to get women and minorities on the

panels---with increasing demands on time.

These observations illustrate the challenges posed by competition and resource scarcity. Other

challenges include the consequences of age and prestige on the allocation of Federal funds, the fate of

proposals that cross disciplines and fall between agency programs, and the psychology of collective

decisionmaking. 2 Debate on the burdens absorbed by Federal peer review systems is healthy if it informs

the practices of agencies, investigators, and reviewers.

l~e Fow on Re~~h ~~~ent WW cnt~in 1982 as a working group of the mmpmfit ~~e~on of Be~io~$

Psychological, and Cognitive Sciences. Most of the attendees at the meeting were from the National Science Found@‘~ the National

Institutes of Heal@ and a few professional associations he+umtered in WashingtQ DC. The excerpts below are based on a transcript of

the meeling supplied by David JohnsoQ exezutive direetor of the Federation.

Zsome of three have &n ad&~& empfic~y. S= the special issue, ‘ ‘Peer Review d I%blic Policy,” Scieme, TeChMbgy, &

Hwnun Va/ue~, vol. 10, No. 3, summer 1985, pp. 3-86.
Chapter 5—Priority Setting in Science . 149

existence of such rules would imply that information

from different fields could be made commensurable. 24

Peer review thus cannot help to set priorities

beyond the limits imposed by agency organization.

Whereas priorities and resource allocations for

megaprojects are usually set by a tacit bargaining

and lobbying process, the science base is governed

by another dynamic altogether. As agencies evaluate

their research needs and modify the emphases of

their programs, research performers are intimately

involved. But seldom does a research community

coalesce around a single agenda (for an exception,

see box 5-D).

The Dilemma of Agency Priority Setting

Universities or States can be analyzed as aggregate categories that receive Federal research monies,

and agencies as the source of those sponsored funds.

But the actual funding decisions are made in

different agency programs and the research performance occurs in laboratories and departments.25

Decisions are thus made at several levels. Priorities

that originate outside the agencies as “national

goals” do not simply trickle down; they are adapted

to what may be called an agency research portfolio,

which in turn is comprised of various program

portfolios (“funding strategies”). Within these organizational niches, priorities are set all the time.

Thus, agencies may have the discretion to pursue

certain national needs by applying a different or

reordered set of criteria to the selection of research

performers.

Because disciplines tend to overlap agencies,

priorities in physics, for example, can be set within

an agency, but not readily across agencies. There is

simply no routine mechanism for doing so. Physics

research is distributed across three mission agencies

plus NSF. While high-energy physics is supported

primarily by the Department of Energy (DOE) and

astrophysics by NASA, theoretical physics ‘‘belongs” to no single agency.26 This is even more

dramatically apparent in the case of neuroscience.

Congress and the President declared the 1990s the

‘‘Decade of the Brain. ’ ’27 As seen in figure 5-2, the

Federal Government supports neuroscience research

in 6 institutes of NIH; in 3 within the Alcohol, Drug

Abuse, and Mental Health Administration; and in 10

other agencies, with the National Institute of Neurological Disorders and Stroke and the National

Institute of Mental Health leading the way. Unless a

“lead” agency is recognized by all participants (as

in computer science, see box 5-E) or an OSTP

Federal Coordinating Council for Science, Engi-

 rate multiple proposals and make direct comparisons of proposed work tick do

not compare their findings with those of panels in other fields, since between-field information is held to be incommensurable. Instead, they judge the

technical merit of a research the competence of the investigators, and the institutional infrastructure available for executing the proposed design.

As Harvey Brooks points out, who is the best judge of social merit? There are no experta on social which has to be a collective involving

several different kinds of expertise as well as generalists’ political judgments. Personal February 1991.

 Science “Planning and Priority-Setting in the National Science Foundation,’ a report to the On

Science, Space, and Technology of the U.S. House of Representatives, Feb. 28, 1990.

 for example, Sebastian “Condensed Matter Theory’s Fragile letter, Physics November 1990, pp. 13, 117.

 Pennisi Diana “ ‘Brain Decade’ Neuroscientist Court The Scientist, vol. 4, No. 21, Oct. 29, 1990, pp. 1,8.
150 . Federally Funded Research: Decisions for a Decade

Box 5-D—Priority Setting by the Ecological Research Community

In fall 1990, the Ecological Society of America proposed the Sustainable Biosphere Initiative (SBI), a research

initiative that focuses on the necessary role of ecological science in the wise management of Earth’s resources and

the maintenance of Earth’s life support systems. 1 The process of developing the research agenda affirms that a

community can set priorities. 2 The document was intended as a call-to-arms for all ecologists. It was also to serve

as a means of communication with individuals in other disciplines with whom ecologists must join forces. Many

of the environmental problems that challenge human society are fundamentally ecological in nature.

In response to national and international needs, the SBI represents a framework for the acquisition,

dissemination, and utilization of ecological knowledge in support of efforts to ensure the sustainability of the

biosphere. The SBI calls for: 1) basic research for the acquisition of ecological knowledge, 2) communication of

that knowledge to citizens, and 3) incorporation of that knowledge into policy and management decisions.

Research Priorities

The criteria used to evaluate research priorities were: 1) the potential to contribute to fundamental ecological

knowledge, and 2) the potential to respond to major human concerns about the sustainability of the biosphere, Based

on these criteria, the SBI proposes three research priorities:

1. global change, including the ecological causes and consequences of changes in climate; in atmospheric,

soil, and water chemistry (including pollutants); and in land- and water-use patterns;

2. biological diversity, including natural and anthropogenic changes in patterns of genetic, species, and habitat

diversity; ecological determinants and consequences of diversity; the conservation of rare and declining

species; and the effects of global and regional change on biological diversity; and

3. sustainable ecological systems, including the definition and detection of stress in natural and managed

ecological systems; the restoration of damaged systems; the management of sustainable ecological systems;

the role of pests and pathogens; the transmission of disease among humans; and the interface between

ecological processes and human social systems.

Existing national and international initiatives address parts of the first two priorities. Success of these programs

will require increased emphasis on key ecological topics. The SBI proposes three research recommendations:

1. Greater attention should be devoted toexamining the ways that ecological complexity controls global processes.

2. New research efforts should address both the importance of biological diversity in controlling ecological

processes and the role that ecological processes play in shaping patterns of diversity at different scales of

time and space.

3. A major new integrated program of research on the sustainability of ecological systems should be

established. This program would focus on understanding the underlying ecological processes in natural and

human-dominated ecosystems in order to prescribe restoration and management strategies that would

enhance the sustainability of the Earth’s ecological systems.

Implementation

Successful implementation of the SBI will require new interdisciplinary relationships that link ecologists with

the broad scientific community, with mass media and educational organizations, and with policy makers and

resource managers in all sectors of society.

In sum, while the goals and action items of the Sustainable Biosphere Initiative may not seem revolution,

few ecologists would have accepted them even a decade ago. But times have changed and so has the science. The

public is more aware of environmental issues than ever before, and opportunities for ecologists have never been greater. 3

Such statements are rare. 4 When they do appear, they can supply to policymakers an unusual tool for judging

a hierarchy of research emphases and perhaps channeling resources to agencies and programs accordingly.

l~e fouow~g is bawd On Jane Lubchenco et d., &OIOglCd SOcie& of ~eric~ “The Sustainable Biosphere Initiative: An Biological

Research Agenda, ’ dmft document, Oct. 30, 1990.

*or details on the fragile process by which the Society’s 2,000 members and its leaders reaehed consensus, see El”~ Pennisi,

‘ ‘Ecology Society Reaches Rare Consensus on Research Agenda, ’ The Scientist, vol. 4, No. 17, Sept. 3, 1990, pp. 3, 9, 20.

3

Ibid., p. 3.

4T0 t~e ~ther exmp]e, tie ~$~nomy ~omm~~, ~~r~g ~o~gh be Na~o~ ~derny of Sciences, has issued four d- surveys

of the field. For the lates~ see National Academy of Scienees, A Decude of Discovery in Asrronomy and A~mophysics (Washingto@ DC: National

Academy Press, 1991); and the statement of the study committee chairmmL John N. Bahcall, ‘‘Prioritizing Scientific Initiatives, ’ Science, vol.

251, Mar. 22, 1991, pp. 1412-1413.
150 . Federally Funded Research: Decisions for a Decade

Box 5-D—Priority Setting by the Ecological Research Community

In fall 1990, the Ecological Society of America proposed the Sustainable Biosphere Initiative (SBI), a research

initiative that focuses on the necessary role of ecological science in the wise management of Earth’s resources and

the maintenance of Earth’s life support systems. 1 The process of developing the research agenda affirms that a

community can set priorities. 2 The document was intended as a call-to-arms for all ecologists. It was also to serve

as a means of communication with individuals in other disciplines with whom ecologists must join forces. Many

of the environmental problems that challenge human society are fundamentally ecological in nature.

In response to national and international needs, the SBI represents a framework for the acquisition,

dissemination, and utilization of ecological knowledge in support of efforts to ensure the sustainability of the

biosphere. The SBI calls for: 1) basic research for the acquisition of ecological knowledge, 2) communication of

that knowledge to citizens, and 3) incorporation of that knowledge into policy and management decisions.

Research Priorities

The criteria used to evaluate research priorities were: 1) the potential to contribute to fundamental ecological

knowledge, and 2) the potential to respond to major human concerns about the sustainability of the biosphere, Based

on these criteria, the SBI proposes three research priorities:

1. global change, including the ecological causes and consequences of changes in climate; in atmospheric,

soil, and water chemistry (including pollutants); and in land- and water-use patterns;

2. biological diversity, including natural and anthropogenic changes in patterns of genetic, species, and habitat

diversity; ecological determinants and consequences of diversity; the conservation of rare and declining

species; and the effects of global and regional change on biological diversity; and

3. sustainable ecological systems, including the definition and detection of stress in natural and managed

ecological systems; the restoration of damaged systems; the management of sustainable ecological systems;

the role of pests and pathogens; the transmission of disease among humans; and the interface between

ecological processes and human social systems.

Existing national and international initiatives address parts of the first two priorities. Success of these programs

will require increased emphasis on key ecological topics. The SBI proposes three research recommendations:

1. Greater attention should be devoted toexamining the ways that ecological complexity controls global processes.

2. New research efforts should address both the importance of biological diversity in controlling ecological

processes and the role that ecological processes play in shaping patterns of diversity at different scales of

time and space.

3. A major new integrated program of research on the sustainability of ecological systems should be

established. This program would focus on understanding the underlying ecological processes in natural and

human-dominated ecosystems in order to prescribe restoration and management strategies that would

enhance the sustainability of the Earth’s ecological systems.

Implementation

Successful implementation of the SBI will require new interdisciplinary relationships that link ecologists with

the broad scientific community, with mass media and educational organizations, and with policy makers and

resource managers in all sectors of society.

In sum, while the goals and action items of the Sustainable Biosphere Initiative may not seem revolution,

few ecologists would have accepted them even a decade ago. But times have changed and so has the science. The

public is more aware of environmental issues than ever before, and opportunities for ecologists have never been greater. 3

Such statements are rare. 4 When they do appear, they can supply to policymakers an unusual tool for judging

a hierarchy of research emphases and perhaps channeling resources to agencies and programs accordingly.

l~e fouow~g is bawd On Jane Lubchenco et d., &OIOglCd SOcie& of ~eric~ “The Sustainable Biosphere Initiative: An Biological

Research Agenda, ’ dmft document, Oct. 30, 1990.

*or details on the fragile process by which the Society’s 2,000 members and its leaders reaehed consensus, see El”~ Pennisi,

‘ ‘Ecology Society Reaches Rare Consensus on Research Agenda, ’ The Scientist, vol. 4, No. 17, Sept. 3, 1990, pp. 3, 9, 20.

3

Ibid., p. 3.

4T0 t~e ~ther exmp]e, tie ~$~nomy ~omm~~, ~~r~g ~o~gh be Na~o~ ~derny of Sciences, has issued four d- surveys

of the field. For the lates~ see National Academy of Scienees, A Decude of Discovery in Asrronomy and A~mophysics (Washingto@ DC: National

Academy Press, 1991); and the statement of the study committee chairmmL John N. Bahcall, ‘‘Prioritizing Scientific Initiatives, ’ Science, vol.

251, Mar. 22, 1991, pp. 1412-1413.
152 ● Federally Funded Research: Decisions for a Decade

Box 5-E—Federal Investment in Computer Science

Since 1976, the Federal Government has had a stellar record of support for academic computer science.

Funding has grown faster than for any other scientific discipline in the United States, However, support for computer

science basic research has declined. As a consequence, questions are being raised about the amount of Federal

support for computer science and the manner in which it is being distributed. A forthcoming Association for

Computing Machinery (ACM) report on a 3-year study, A Field in Transition: Current Trends and Issues in the

Funding of Academic Computer Science Research, can be viewed as awe example of how the Federal Government

invests in academic science. 1

The Federal Government has been instrumental in the computer’s rise to strategic importance, starting with

the first electronic digital computer, ENIAC, built under Army contract during World War II. The computer might

be called an “enabling technology,” a tool for advancing research across the spectrum of disciplines. But the

Federal agencies’ long-term funding of computer science and engineering research, particularly in the universities,

has been a primary factor in the emergence and maturation of computer science as a distinctive discipline as well.

Today, there is a call for new initiatives in high-performance computing to enhance the Nation’s economic and

scientific capabilities.2

Federal Funding of Computer Research

Between fiscal years 1976 and 1989, Federal obligations for computer science research rose from about $89

million to $487 million. This is equivalent, in 1990 constant dollar terms, to an annual (compound) rate of growth

of 8 percent, or a total gain of 170 percent. About 85 percent of this increase occurred after 1980,

Historically, the Department of Defense (DOD) has provided about two-thirds of the Federal funds for

computer science research, and accounted for over 60 percent of the increase in total funds since fiscal year 1976.3

While the National Science Foundation (NSF) is considered thc second most important agency in computer science,

in funding it has jockeyed for second place with the National Aeronautics and Space Administration (NASA) sinc e

the 1980s. Except for its work on the ILLIAC IV supercomputer, NASA’s computer science funding was minimal

1

See Joel s. Yudken and Barbara Simons, “A report Summary--Final Report of the Project on Funding Policy in Computer Science."

unpublished document Oct. 15,1989, Unless otherwise indicated, the data presentedbelow are drawn from the draft report summary, as updated

by Joel Yudken Rutgers University, personal communication, February 1991 .

%atiorud Reaeamb CoUXKil, Cz.mpater Science and ‘lbclmoIogy Boar& The Nm”ond Challenge in Computer Science and Technology

(Wahingtoq DC: 1988), p. 30,

3JOIUI R. Il. ClemenL “Computer s~a~.--. Support in the W 1988 lkigc~” AAASReport XII, Research & Development

FY 19S8, AAAS Gxnmittee on SC- Bn@mmmg and Public Policy, Intemciq working Group (cd.) (wMhin@m DC.: Arlelican

A9sociatioo for the Mv mcemcal of Sciz 1%~, pp. 251-261; sod JohtJ R.B, Clement and Dianne Bdgar, ‘‘Computer Sckme and

Engineering Support in the FY 1989 Bud@’ ’AAA5ReportXJlI, Research & LkvefoptncntFY 1989, ALMS Committee on SCkxtct, Eogk@ng

and Public Policy, Intersociety Wotking Group (d.) (W-Q DC: Amcricm A3aoeiation for the Advancerncnt of Science, 1988), pp.

260-271.

success. But the metaphor breaks down here be- direct comparisons of the alternatives facing the

cause, while success in stock market investments

can be gauged by money earned, nothing as tangible

results from research-at least not in the short run.29

A program “purchases” a portfolio of research

projects in a field. The selection of projects for

inclusion in this portfolio has been determined by

their predicted or estimated quality as seen by

contemporary research performers (reviewers) or by

knowledgeable research managers (with or without

the aid of reviewers). Reviewers usually make

judgments about the quality of a project without any

investor. Priority setting forces such comparisons.

Rather than choosing projects on a one-by-one basis

up to the point of resource exhaustion, they could be

recommended with reference to their incremental

value, i.e., as projects that concentrate or divers@

strength in the portfolio. Managers, on the other

hand, compare projects with reference to the objectives of the entire program portfolio.

At least for basic research, researchers, reviewers,

and program managers are supposed to adjust their

activities so quickly that judgments about the quality

29See Harvey Averch, “New Foundations for Science and Technology Policy Analysis, ’ paper presented at the Conference on The Mutual Relevance

of Science Studies and Science Policy, May 12, 1989, p. 7.
Chapter 5—Priority Setting in Science . 15.3

until 1982, when its funding jumped up substantially. NSF supports mostly basic research not tied to missions or

applications in a full range of computer science and engineering subdiscipline, including theory, software systems

and engineering, artificial intelligence and robotics, and advanced computer architecture. The Department of Energy

(DOE) involvement in computers dates back to ENIAC in 1945, which was used for calculations for nuclear bomb

research at the Los Alamos National Laboratory. DOE (and its predecessor agencies, the Atomic Energy

Commission and the Energy Research and Development Administration) has been a major force in the development

of high-performance scientific supercomputers ever since.4

Federal funding for academic computer science research rose dramatically between 1976 and 1989, from over

$27 million to $235 million (current dollars), or 320 percent in real terms. DOD, NSF, NASA, and DOE account

for virtually all Federal funding of academic research in computer science. (The National Institutes of Health and

the National Institute of Standards and Technology both allocate a small number of extramural contracts and grants

to universities and colleges.)

DOD has historically been the largest funder of academic computer science and its role increased substantially

since 1976. DOD’s share of Federal funding for academic computer science rose from 45 percent to 62 percent in

fiscal years 1976 to 1989, accounting for over two-thirds of the total increase in this funding during this period.

Although NSF funding for academic computer science increased from roughly $14 million to $64 million (current)

between fiscal years 1976 and 1989---a real growth of 126 percent—its share of total Federal support for academic

computer science declined from 51 percent to 27 percent.

Policy Initiatives in Computer Science Research

Policy initiatives from the Federal Coordinating Council on Science, Engineering, and Technology (FCCSET)

and the Computer Science and Technology Board of the National Research Council call for substantial funding

increases in high-performance computing. The FCCSET proposal has already led to a multiagency request for a

$149 million funding augmentation (in what is now called the High Performance Computing and Communications

Program), and to new joint Defense Advanced Research Projects Agency-NSF projects.s The question of balance

and priorities—the shape of the Federal research portfolio for computer science---is likely to persist well into the

1990s. 6

AK~e~ _ ~~rgeting fhe Computer (Washington DC: The Brookings ~tu~~ 1 ~7). pp. 78-85.

5Natio~ Science Foundation Committee on phySk& ~timti~, ~~“ %iences, Grand Chalknges: High Performance

Computing and Communications (WasMngtOq DC: Fehuq lW1); Executive Office of the Prcsiti Office of Sciemx and Technology Policy,

“A Rmearch and Development Strategy for High Performan ce Computing,’ unpublished documen~ Nov. 20, 1987; and National Scitmee

FoundatiorL ‘‘Crosswalk of NSF Research Related to tbe Department of Co mmerce Emerging ‘Ikdnologies List and the Department of Defense

Critical ‘Ikdmologies List+’ in ‘ ‘Background Material for Ixmg-Range Planning: 1992- 1996,’ prepared for a meeting of the National ScicQCe

Board, June 14-15, 1990, pp. E-1 to E-6.

6For a ‘‘cm to action’ to eomputin.g researehem, see lkrfY M. W*er, ‘‘Influencing Federal Support for Computing Research+”

Cmnpm”ng Research News, vol. 2, July 1990, pp. 1, 10-11.

of any isolated single project remain congruent with The burden for priorities, then, rests not with

developments at the frontiers of knowledge. In

practice, the agency investor has no way of knowing

whether this ‘‘invisible hand’ is efficient, rapid, and

has good discriminating power. So portfolio evaluations could be used to set relative investment

priorities since they provide a check on performance

at a useful level of budgetary aggregation. But this

would require some modification of the criteria for

project selection. Reviewers would no longer be

ranking proposals by scientific merit alone, but with

respect to standards about which they as experts

have no special competence, i.e., issues of social

merit.

those who give advice, but with those who receive

and sort it along with other program arid agency

objectives. To take an example, for the period 1987

to 1991 at NSF, the increase in appropriations for

‘ ‘research and related activities’ directorates

(R&RA) was 39 percent to $1.95 billion (in current

dollars). This compares to a 153-percent increase in

‘‘science and engineering education” to $251 million and a 49-percent increase for the U.S. Antarctic

Program to $175 million. Looked at thematically, 80

percent of the requested fiscal year 1991 NSF budget

was for research and facilities, and 20 percent for

education and human resources (a virtual doubling
154 . Federally Funded Research: Decisions for a Decade

Table 5-5—lnhouse Evaluation for the NSF Strategic Plan: Research Advances and Opportunities

Lost or Postponed, Research and Related Directorates, Fiscal Years 1987-90

Percent change

in funding Opportunities

Directorate (current dollars) Research advances lost/postponed

All Research and 39.0% ●

Related

Directorates

●

●

●

Biological,

Behavioral,

and Social

Sciences

Computer and

Information

Science and

Engineering

Engineering

Geosciences

Mathematical

and Physical

Sciences

26.3

65.5

39.5

34.8

34.1

●

●

●

●

●

●

●

●

Scientific, Tech- 69.0 ●

nological, and ●

International

Affairs ●

Research initiatives to enhance economic ●

competitiveness in biotechnology, global change, manufacturing, materials, supercomputing/networking,

superconductivity.

10 new ERCs, 11 new STCs. ●

Programs on women/minorities/disabled and undergraduate research and teaching expanded. ●

Number of proposals up 11.2 percent; number of awards

up 4.6 percent.

5 new centers (3 in biotechnology, 1 in plant science- ●

cooperatively with DOE and USDA, and 1 in geography).

Other initiatives in neurobiology, human dimensions in .

global environment change.

Equipment and instrumentation increases.

NSFNET expansion. ●

New joint initiative with DARPA in parallel processing. ●

4 supercomputer centers renewed.

Infrastructure activities in minority institutions. ●

●

7 group research grants for Strategic Manufacturing ●

Initiative.

Newnitiatives in optical communications, nondestructive .

evaluation, and management of technology.

Research on Loma Prieta Earthquake. ●

Initiated active Systems Service. ●

●

Major research equipment subactivity for large research .

equipment construction projects. ●

Augmented support for new investigators.

●

Growth of EPSCoR. ●

Implementation of Scientific and Technical Personnel

Data System.

6 Minority Research Centers of Excellence initiated.

Decline of 6.1 percent in proposal

success rates and in average

annualized award amounts in

5 of 6 directorates (1987-89).

3 ERCs and 5 materials research

labs terminated.

Other STCs deferred.

Pursued all proposed, but at

reduced levels.

3-percent decline in proposal

success rates.

27-percent decline in success rates.

Fewer grants to groups than

planned.

Software engineering initiative

delayed.

1 supercomputer center phased out.

Number of proposals and awards

down slightly.

Materials synthesis and processing

initiative (with MPS) delayed.

Success rates down 3.2 percent.

Canceled some atmospheric

science filed programs.

New initiative in mesoscale

meteorology deferred.

Success rates down 13.9 percent.

Illinois Macrotron construction

canceled.

Material synthesis and processing

initiative postponed.

Success rate down 9.2 percent.

Undergraduate Education Data

System in SRS delayed.

KEY: DARPA=Defense Advanced Research Projects Agency; DOE=U.S. Department of Energy; EPSCoR=Experimental Program to Stimulate Competitive

Research; ERC=Engineering Research Centers; MPS=Mathematical and Physical Sciences; NSFNET=National Science Foundation electronic

network; SRS=Science Resources Studies; STC=Science and Technology Centers; USDA=U.S. Department of Agriculture.

SOURCE: National Science Foundation, “Background Material for Long-Range Planning: 1992 -1996,” NSB 90-81, prepared for a meeting of the National

Science Board, June 14-15, 1990, pp. C-3 to C-17.

from its share in fiscal year 1987). This reflects the opportunities seen as lost or postponed. This inhouse

congressionally mandated priority of science educa- evaluation was provided to the National Science

tion at NSF.30 Board to assist in its long-term planning. It could

also serve as a tool for organization and reorganizaTable 5-5 highlights research advances in its tion (see box 5-F), and as a priority scorecard for the

R&RA directorates since 1987, as well as research mostly little science that NSF supports.31

~~ese percentages and amOWWS are based on requests in the fiscal year 1991 budget. Still, they approximate how the research tiectorates have f~d

relative to other activities at the Natioml Science Foundation. See National Science Foundation “Background Material for Long-Range Planning:

1992-1996,” NSB 90-81, prepared for a meeting of the National Science Board, June 14-15, 1990, p. C-3.

ql~e An~CtiC ROW- is tie cMef exceptiom though the National Science Foundation also fimds research and development centers such as tie

National Center for Atmospheric Research, the Kitt Peak and Green Bank telescopes, and five National Supercomputer Centers.
Chapter 5—Priority Setting in Science ● 155

Box 5-F—Behavioral and Social Sciences: Organization and Federal Funding

In the concluding chapter of a National Research Council (NRC) committee report on achievements and

opportunities in the behavioral and social sciences, titled “Raising the Scientific Yield, ” a prescription is offered

for”. . . new investments and modifications in research infrastructures that are needed for further progress. "1 The

program of prescribed investments total $240 million annually in 1987, a year in which Federal expenditures on

behavioral and social sciences research reached the $780 million mark.2 The research frontiers singled out by the

NRC committee for investment include “. ., new inquiries into the connections among behavior, mind, and

brain, . . .“ “. . . research on the mechanisms of choice and allocation, . . .’ “. . . comparative and historical

(including prehistorical) study of the institutional and cultural origins of entire societies,. . .“ and methodological

advances in “. . . data collection, representation, and analysis. ”3 But is the level of Federal investment in a broad

field of science indicative of its potential contributions?

The behavioral and social sciences tend to get less visibility than other sciences at the Federal agencies,

especially the National Science Foundation (NSF) and the Alcohol, Drug Abuse, and Mental Health Administration

(ADAMHA ) of the Department of Health and Human Services, which are the primary providers of basic research

funding. This dilemma was addressed at a Senate hearing in 1989 by the economist-psychologist and Nobel laureate

Herbert Simon.

It is misleading to talk about “hard” and “soft” sciences. In the physical sciences, classical mechanics is hard,

but meteorology (e.g., the greenhouse effect) and the theory of high-temperature superconductivity or lowtemperature fusion can be (as recent news stories tell us) exceedingly soft. Similarly, in the social sciences, knowledge

about the operation of competitive markets or the capacity of human short-term memory is quite hard; but knowledge

about how businessmen and consumers form expectations about the future, or about motivations surrounding drug

usage can be quite soft. 4

To study scientifically what makes us human is as daunting a task as to discover the fundamental forces of the

universe or to understand how normal cells become factories of disease. 5 The problem is the priority of funding

social research, and opinions may differ on how to institutionalize a Federal commitment to behavioral and social

science research.

An Organizational Solution?

In August 1990, Reps. Walgren and Brown introduced H.R. 5543, The Behavioral and Social Science

Directorate Act of 1990. This was proposed because, according to Walgren: “NSF’s enthusiasm for the behavioral

and social sciences is at best lukewarm . . . and the cause is largely structural. Since its creation, this Biological,

Behavioral, and Social Science [BBS] Directorate has been headed by a biologist.” Brown added that: “NSF as

a whole has enjoyed a relatively large increase in funding over the past decade. . . . However, rather than sharing

in the Foundation’s good fortune, these areas of science have been languishing. ” 6 The current BBS budget totals

$293 million, including $48 million for ‘behavioral and neural sciences’ and $33 million for ‘social and economic

sciences.

While the concept of a separate directorate has been around for at least a decade--the time of Reagan-era

cuts-at the 1990 National Behavioral Science Summit, held under the auspices of the American Psychological

Society, 65 psychological

lM R. Gerstein et al.

Academy Press, 1988), p. 239.

%id., p. 249.

31bid., pp. 239-244.

and behavioral science org “ anizations endorsed the idea as a solution to needed visibility

(eds.), The Beh”orai ond Sm”al Sciences: Achiev e??unts and Opportum”ties (WaahingtorL DC: Nationat

4Hcr~ SimO~ testimony, in U.S. COWSS, Senate cm ttee en commerce, Science, and Transportation, Sukommi ttee on Science,

Twhnology, and Space, National Science and Technology Po/icy, Sept. 28-29, 1989, 10lst Cong. (WaShingtO~ DC: U.S. Gove mrnemt Print@

Office, 1989), pp. 264269.

5WS wa8 fomy RCOgDMM when the organic act of the National Science Foundation was arnendcd in 1%8, placing witi its leg~

mandate. . . a formal responsibility to look after the health of basic research in the social and behavioral seiencas. ” See Roberta Balstad Miller,

National Science Foundatio~ 4 ‘The Contribution of Social Researc&’ John Madge Memoriat Lecture, Imndon School of Economics, NOX embez

1986, quote from p. 6.

6“Be~vio~ Directorate for NSF Proposed iII Congress, ” APS Observer, vol. 3, September 1990, p. 7.

7< ~Socl~, B&vior~ Sciences Seek IJpgrade at NSF, ’ .$clence & Gove rnment Report, vol. 20, No. 15, Oct. 1, 1990, p. 6.

Continued on next page

292-863 0 - 91 - 6
156 ● Federal/y Funded Research: Decisions for a Decade

Box 5-F—Behavioral and Social Sciences: Organization and Federal Funding-Continued

and bigger budgets.g NSF appointed a task force on ‘‘Imdcing to the Twenty-First Century” to study the idea and

". . . keep several thoughts in mind: 1) BBS must have the flexibility to meet new mandates; 2) BBS must meet

the infrastructure needs of its disciplines; and 3) the zero-sum budget situation makes funding reallocations

difficult. ’

In December 1990, the task force voted its intention to recommend establishing a separate NSF directorate for

social and behavioral sciences. It would be called Social, Economic, and Psychological Sciences (SEPS).10

Foremost among the issues the task force must consider are how the boundaries for the behavioral sciences would

be drawn, and how interdisciplinary research would be affected Recommended for inclusion in SEPS are

economics, geography, law, linguistics, political science, psychology, and sociology. The interdisciplinary fields

of cognitive science and of decision, risk, and management sciences would also be included. Most of neuroscience

would stay in the biological directorate. Unresolved are the place of anthropology and some of the programs

supporting research on information, robotics, and intelligent systems (now housed in the Computer and Information

Science and Engineering Directorate) .11 In lieu of immediately developing a divisional or programma tic structure

for SEPS, a new group (including NSF program officers) may be asked to take up the issue.

Whether a separate directorate could aid the management and funding of social and behavioral science research

at NSF, and how the agency could assess the effectiveness or productivity of such a new directorate, remains to be

seen. Implementation of whatever is finally approved would not occur until fiscal year 1993. It is clear that advocates

in the vast majority of behavioral and social science fields (led by psychology) are convinced that ‘‘. . . only by

elevating representation of our scientific disciplines will we successfully compete and increase our funding

capabilities and our potential contributions to science. 12

g~e question of po~ti~ vulnerability that accompanies a mnsolidation of interesta in a single structure is the flip side of political gain.

For example, conccm for (he envkonmenti scienms lMS led to a call for COngreSS to create a new agency, the National Institnt@ for the

Environment (N’KE), modeled on the Nationai In..stitutm of Heal@ to stop the erosion of research and tndning programs relatd to environmental

biology, economics, and policy. Congress has asked the National Academy of sciences to study the concept of an NIB. See Herxy F. Howe and

Stephen P, HubbeLt, ‘‘Progress Report on Proposed National Institutes for the Environmc n~” Bioscience, vol. 40, No. 8, September 1990, p.

567; and William Boo@ “Does Earth Need a Governmen t LostitutE?” The Washington Post, Dec. 10, 1990, p. A13.

9’lBBS Tti Force Meets: se~te Directorate IS5ue on the ‘kble,” COSSA Washington Update, Sept. 21, 1990, p. 1.

I% m ~W~ of tie task force is forthcoming. See ‘‘NSF Task Force to Recmnmmd New DhW@XW e for Social and Behavioral

Sciences,” COSSA Washington Update, Dec. 7, 1990, pp. 1-2; and “NSF Task Force Discusses Upcoming Repofi, * to Support SEPS

Directorate,” COSSA Washington Upaizte, Jan. 28, 1991, pp. 1-2.

1 l-WIOH oppo~ ~ creation of a new dinxtorate, bUt & Amerim ~tiPolO$jm ~~~tion k

announced that: “were

the proposed reurg anization to occur, [anthropology] would elect to be housed . . . ‘‘ in it. See ‘“Anthropolo@ats Opt for SEPS Directorat~’

COSSA Wmhington Update, Feb. 10, 1991, p. 2, Applied statistics, “. . measurement and methodological research as well as infmnuctwe

issues . . .’ are favored by the task forw to join the Soci~ Economic and Psychological Sciences as well. See ‘‘NSF Taak Force Di~

Upcoming Repoti ” op. cit., footnote 10, pp. 1-2.

l@uo~ from tie ~~mpv~olog~ q~tion ~ ‘ ‘NSF TM Fo~ to Recommd New D~tomte for SOCiA d Behavicnal

Sciences, ” op. cit., footnote 10, p. 1. Also see Alan G. Krau~ ‘‘Statement of the American Psyctwlogical Society to the National Sci~e

Foundation’s BBS ‘lh.sk Force on I.xMking to the 21st Century,’ unpublished documen t, NOV. 29, 1990.

The science base, especially at NSF and NIH, Science Megaprojects

carries not only the traditional responsibility for

funding scientifically meritorious research, but also

for satisfying the expectations that the political

system associates with the support of research.

These expectations, together with budget constraints, create tougher and tougher choices. The

agencies cope admirably with this complex task. In

the next section, OTA examin es another category of

research funding: science megaprojects.

The Federal Government has a long history of

supporting projects such as the building and operation of dams, bridges, and transportation systems.

These projects are large-scale, complex, costly, and

long-term undertakings. In addition to performing

their primary function, these programs also provide

jobs and local public works, and have long-term

economic value. Thus, they are often called “megaprojects. ’
Chapter 5—Priority Setting in Science . 157

As the body of scientific knowledge grows more

sophisticated and costly, research instrumentation

and infrastructure are required in some fields .32 As

projects expand, they become valuable economically and politically. For example, the Hubble

Space Telescope, launched in April 1990, cost over

$2 billion, although the original estimate was as low

as $300 million.

By the time the real costs were known, it was too

far along to stop. As a practical matter, Congress

refused to write off as wasted the hundreds of

millions it already had sunk into the project.

Politically, the device had taken on a pork-barrel life

of its own, sending government money to nearly half

the 50 States and employing thousands.33

Although economic activity may be a second- or

third-order consideration among- the initial criteria

of project selection, the distribution of Federal

monies, as an interim payoff on a long-term investment, can be substantial. For instance, the three

megaprojects shown in figure 5-3, the Hubble Space

Telescope, the Space Station, and the SSC, enjoy

widespread economic and social merits, regardless

of their scientific merit.

Among the megaprojects recently listed by The

Chronicle of Higher Education are the SSC, the

Space Station, the Moon-Mars mission, the hypersonic aircraft, the Earth Observing System, the

Strategic Defense Initiative, and the Human

Genome Project (HGP). The original estimates for

these seven projects alone totaled $528 billion. A

September 1990 cost estimate for the same projects

was about $580 billion,34 or $65 billion annually.35

What Constitutes a Science Megaproject?

Megaprojects are large, lumpy, and uncertain in

outcomes and cost. ‘‘Lumpy’ refers to the discrete

nature of a project. Unlike little science projects,

there can be no information output from a megaproject until some large-scale investment has occurred.36

OTA also would define a science megaproject as

requiring very large expenditures (especially when

 that further and further we get from direct sense the more costly and complex research we 

for progress. See Nicholas Scientific Progress: A Philosophical on the Economics of Research in Natural Science PA:

University of Pittsburgh Press, 1978). The section below is based on op. cit., 28.

 Big Science Is Ready for Blastoff,” Congressional Quarterly, vol. 48, Apr. 28, 1990, p. 1254. Also see Kim A.

 “Researchers Increasingly Worried About the Unreliability of Big Science Projects,”The Chronicle of Higher Education, vol. 36, No. 48,

Aug. 15, 1990, pp. Al, A8-A9.

 Science Important Milestones in Face of Federal Budget Criticism, ”

The Chronicle of Higher Education, 37, No. 2, Sept. 12, p. A28, reports original and current cost estimates submitted to Congress just for

the R&D investments required for the seven projects (most not adjusted for inflation). High-technology R&D project cost are notoriously (but

unsurprisingly) low. For example, a number of RAND Corp. studies show the underestimation in the actual costs to develop high-tech military aircraft.

See Morns and The Anatomy of Major Projects: A Study of the Reality of Project Management (New NY: John Wiley& sons, 1987).

 “Big The New York Times, 27, 1990, June 

 build one-half of a dam, one-half of a ship, or one-half of an airplane get ce. These technologies are,

of course, well enough in hand that one can estimate or predict the results from investing in them. There is a very extensive literature on appraisal and

management of capital projects such as dams, airports, ports, and also a sizable literature on estimating the worth of private capital investments

in factories and equipment. Among the literature on managing large complex technological systems, literally nothing is written on selecting them. But

see Harvey The Polaris System Development: Bureaucratic Success in Government (Cambridge, MA: Harvard University Press, 1972).
158 ● Federal/y Funded Research: Decisions for a Decade

0

n , 

o a

L

--_——T——— 

0

L

0
Chapter 5—Priority Setting in Science . 159

compared with other investments in the same or

similar fields) to create knowledge that is unattainable by any other means.

Perhaps equally important in the definition of

science megaprojects, however, are the political

components. Each project is unique in its development, especially in its progress through the budget

processes at the research agencies. Also, science

megaprojects are supported by large political constituencies extending beyond the scientific community. In short, there are few rules for selecting and

funding science megaprojects; the process is largely

ad hoc. To illustrate, OTA presents two widely

acknowledged examples of big science projects—

the SSC and the HGP.

The Superconducting Super Collider

The SSC, when built, will accelerate two countercirculating beams of protons at energy level 20 TeV

to “create’ rarely seen elementary particles when

these beams collide. Expected to cost at least $8

billion to construct, the SSC represents one of the

world’s largest scientific instruments.37 Amidst

contentious debate in Congress, the SSC won

funding approval to begin construction in June 1989.

DOE decided to build the 54-mile-in-circumference

accelerator south of Dallas, Texas; it is expected to

begin operation in 1999.38 Director Emeritus of

Fermi National Accelerator Laboratory, Leon Lederman, has said in House testimony that “. . . instead of trying to kill off a big target like the

SSC . . . the collider should be seen as the ‘flagship’

for big increases for science [funding].”39

The SSC clearly meets the specific criteria

outlined by OTA for a big science project (high cost

and unique outcomes). It also satisfies the political

criteria. First, the SSC has important scientific goals

that can be obtained in no other way. Second, the

high-energy physics community has marshaled

support of the SSC from DOE, which administers

the project, and the State of Texas, where it will be

built. Finally, the SSC originated in DOE discussions with the high-energy physics community, and

was preferred by the Department’s High Energy

Physics Advisory Panel to the equivalent amount

invested in smaller, less costly high-energy physic

s 

40 As with many big science projects, however, .

it is also true that, without the prospect for such an

accelerator, the equivalent amount would most

likely not be available to physicists-or indeed to

scientists in other fields-at all.

The Human Genome Project

The HGP, estimated to cost $3 billion to complete, is expected to yield a high-quality genetic map

of the human genome. The HGP is ‘‘big’ biology in

lifetime costs and mode of organization (e.g.,

scientists clustered in ‘‘mapping’ centers) relative

to the rest of biomedicine. 41 An annual $200” million

appropriation would represent 5 percent of NIH’s

funding for untargeted research.42 Its fiscal year

1991 appropriation is $135 million. HGP organizers

stress that funding for the project since its inception

has been “new” money. Funds from other budget

categories at NIH have not shifted to the HGP, i.e.,

none have decreased since the inception of the

project.43

37s= David p. H~ltOu ~~~~ ssc ~~~ on a Life of Its~’Science, vol. 249, Aug. 17, 19$XJ, pp. 731-732; ad Wcia Barinaga, “The SSC

Gets Its (Official) Price Tag: $8.3 Billion,” Science, vol. 251, Feb. 15, 1991, pp. 741-742.

38sW us. D~~ent of EnmW,~~~. of tie ~~tor G.ener~ spe~”az Repo~ on r~e ~epa~~n~ @Energy’~ supercorl&cting supf?r ~Ozzder

Program, DOE/IG-0291 (Germantown, MD: Nov. 16, 1990).

XQuotd in Colleen Cordes, ‘‘Calls for Setting Science-Spending Priorities Are Renewed as Supercollider Gets Go-Ahead, NSF Faces Pinch’ The

Chronicle of Higher Education, vol. 35, No. 46, July 26, 1989, p. A23. Notice the tradeoff language in the headline.

%spring 1990, the Department of Energy’s High Energy Physics Advisory Panel mnkedresearch goals for the 1990s: building the Superconducting

Super Collider (SSC) was fm4 upgrading Fermilab’s proton-proton ‘Ikvatron collider was second, and supporting university-based investigators

received honorable mention. See “Physics Panel Sets Priorities for 1990s,” Science, vol. 248, May 11, 1990, p. 681. This also emerged from OTA staff

interviews at the Department of Energy, spring Im. In a January 191 news release, the Council of the American Physical Society, for the first time

in its 93-year history, “. . . overwhelmingly adopted a public position on fimding priorities. Top priority is given to support of individual investigators

and ‘broadly based physics research. ’ “ The statement also endorses ccmstructionof the SSC in a”. . . timely fashiom but the funding required to achieve

this goal must not beat the expense of the broadly based scientific research program of the United States. ” See American Physical Society, “First APS

Council Statement on Funding Priorities,” news release, Jan. 28, 1991.

4

1 Tom Shoop, “Biology’s Moon Shot,” Government Executive, February 1991, pp. 10-11, 13, 16-17.

Q% fiscal year 1990, the Htumm Genome Project is budgeted for $90 million at the National Institutes of Health (NIH) and $46 million at the

Department of Energy. The NIH amount represents 1 percent of its total budget. See Leslie Roberts, ‘‘A Meeting of the Minds on the Genome project?’

Science, vol. 250, Nov. 9, 1990, pp. 756-757.

43s~ Borrn~ “Human Genome Project Moving on Many Fronts,’ Chemical & Engineering News, vol. 68, No. 50, Dec. 10, 1990, pp. 6-7.
160 . Federally Funded Research: Decisions for a Decade

Critics of the HGP contend that it flaunts tradition

in the administration and performance of biomedical

research. They are also ‘‘. . . not convinced that a

crash program for analyzing the structure of

genomes will advance either health or the life

sciences for many years to come. ’ ‘44 Proponents of

the project stress its development of automated

technologies for molecular biology, including mapping and sequencing, and of new computational

approaches that apply computer science to biology.

Thus, “. . . a new type of interdisciplinary biologist

who understands technology as well as biology . . .“

is being trained. Besides, in the words of molecular

biologist Leroy Hood:

The HGP primarily funds single investigatorsponsored research. It is not big science. Rather, by

making the human chromosomal map and sequence

available to small laboratories, it allows them to

compete with large laboratories. Hence, the HGP is

the guarantor of small science.45

The HGP was originally billed as a project that

would contribute to the cures for all disease.% As

legislators skeptically claimed that they had heard

this “promise” from life science projects before,

proponents of the project began to promote the HGP

on its other potential strengths, including contributions to economic competitiveness. For instance,

Hood stated that HGP “. . . will in turn spawn new

industrial opportunities. . . . The HGP will prime the

American economic pump.’ ’47

At issue in the designation of this science project

as ‘‘big’ is more than cost and organization, but the

timing of the research investment and its impact on

both the culture and justification for biomedical

research. The 1 percent of NIH’s budget is a small

investment, but it represents the reordering of

criteria and the disruption of research-m-usual.

The Process of Megaproject Selection

From a national perspective, megaprojects are

very large projects that stand alone in the Federal

budget and cannot be subject to priority setting

within a single agency. Nor can megaprojects be

readily compared. The SSC and HGP are not big

science in the same sense. One involves construction

of a large instrument, while the other is a collection

of small projects. There also exists virtually no

scholarly literature to guide the selection of megaprojects designed to promote the state of the art of

scientific fields.48 At issue with many megaprojects

is their contribution to science. For instance, the

Space Station has little justification on scientific

grounds, 49 especially when compared with the SSC

or the Earth Observing System, which have explicit

scientific rationales. At present, the Space Station

does have considerable momentum as an economic

and social project.50 However, many question the

uniqueness of these benefits because other projects,

such as the Earth Observing System, could certainly

provide many jobs as well. Because the problem of

selecting among science megaprojects has most in

common with the selection of complex capital

projects, timeliness (why do it now rather than

later?) and scientific and social merit must all be

considered.

The tradeoffs among these criteria are complex,

even when restricted to considerations solely within

44Bemard D. Davis, ‘‘HumaIIGenome Project: Is ‘Big Science’ Bad for Biology?-Yes: It Bureaucratizes, Politicizes Research’ The Sciet~tist, vol.

4, No. 22, NOV. 12, 1990, p. 15.

ds~roy E. Hood, “H~ Genome Project: Is ‘Big Science’ Bad for Biology?-No: And Anyway, the HGP Isn’t ‘Big Science, ’ “ The Scientist,

vol. 4, No. 22, NOV. 12, 1990, p. 15. For an elaboratio~ see Walter Gilbe~ ‘‘Towards A Paradigm Shift in Biology, ” Nature, vol. 349, Jan. 10, 1991,

p. 99.

46 See U.S. Conmss, Offlce of Tec~oloW Ass~smen~ Mapping Our Genes: Gen~~ P~~jec?~~w Big, HOWI Fast? OW-BA-373 (WaS~O~

DC: U.S. Government printing OffIce, April 1988).

QTHood, op. cit., footnote 45, p. 13.

4S’rhe jo~ Tech~ozogy in socie~ devoted one full issue in 1988 and another in 1990 to the pohticd ~d social co~wences of ~ge t~~olo~~

projects, but no author discussed algorithms for selecting them. Also see William C. Boesrnq “Historical Perspective on LargeU.S. Science Facilities,”

CRSReview, February 1988, pp. 8-10; and Peter Monaghaq “Historians Seek More Detailed Study of Big Science Projects to Inform Debate Among

Researchers and Policy Makers,’ The Chronicle of Higher Education, vol. 37, No. 14, Dec. 5, 1990, pp. A5, A8.

d~or ~ ewly sbtaent of ~s view, see us. Con=ss, Offlce of ~~olo= ~sament, Civilian Space stations and the U.S. Future in Space

(Springfield, VA: National lkchnical Information Service, November 1984).

mq’his momen~ may be slowed by the latest statement of ‘‘no confidence ‘‘ in the scientilc content of Space Station Freedom in a forthcoming

National Academy of Sciences report. See David P. liarnilto~ “Space Station Shrinkage TO Affect Scientific MissioL” Science, vol.251, Mar. 8, 1991,

p. 1167; and Eliot Marshall, ‘‘Two Thumbs Down for Space Station,”Science, vol. 251, ?v?ar. 22, 1991, p. 1421.
Chapter 5—Priority Setting in Science . 161

Photo credit: National Aeronautics and Space Administration

In the early stages of megaproject development, it is often difficult to obtain firm estimates of cost because plans can change

radically. In 1982, the Earth Observing System (EOS) was conceived as a large space antenna system, as in the artist’s rendering

on the left. By 1990, the conception of EOS had changed to include “’platforms” in space and other features, as shown on the right.

the scientific community. One observer puts the initiative that could supplant older, S&T base

dilemma of weighing social and scientific merit this programs, and would be added to agency budgets if

way: the megaproject did not exist.52

Scientific progress depends heavily on scientific

capital; scientific capital is built up by investments

in training, equipment, pilot research, and the

accumulation of expertise over extended periods. A

single very large project may have great scientific

and social benefits, but if it can be done only by

shutting down existing lines of research in other

areas, the opportunity losses-the loss of the benefits these lines of research would have produced, plus

the cost of duplicating in the future the capital

investments in them-can outweigh the gains from

the larger project. It is very difficult to estimate the

losses from opportunities foregone; however, we do

know that a small proportion of studies trigger the

kind of dramatic breakthroughs that transform life in

ways the original researchers themselves rarely

envision.51

On a national scale, criteria and tradeoffs are even

more difficult to quantify, since completely separate

fields are represented. The social and scientific

benefits that will derive from investing in one are

incommensurable with those that would be derived

from investing in some other.53 So weighing the

scientific, technological, and development benefits

that will result from the projects will not suffice;

economic and labor benefits must also be considered. Other criteria might also include education and

training benefits, and the impact of the project on the

research community measured in per-investigator

costs. For instance, if one project will benefit only a

few researchers, while a second of similar cost will

benefit a larger number of researchers, then perhaps

the second should be favored.

One might also expect preference for megaprojIn addition, funds are still obligated to agencies. So ects that can be cost-shared internationally over

assurances notwithstanding, the research commu- those that cannot be. This conceives of megaproject

nity perceives megaprojects as new money for an output as a contribution to world science, i.e., as

 “Evaluating Initiatives, ” letter, Amen”can Scientist, vol. 77, No. 3, May-June 1990, pp. 211-212. The opportunities presented by should be compared with the foregone of little science at the not on average.

 funding relative to the rest of the National Institutes of Health budget. For 

 Much for AIDS The Washington Oct. 2, 1990, p. A19.

 for example, J. E. et “Allocating Resources Among AIDS Research Strategies,’ Policy Sciences, vol. 23, No. 1, February 1990, pp.

1-23. The authors asked 17 known AIDS experts to estimate the marginal or incremental value of additional funds for AIDS research

investments in terms of some social outcomes. The information that would be gained from these different investments is incommensurable,

but their expected contribution to the social outcome allows them to be ordered. The problem, however, is more like judging

research investments in AIDS v. heart disease v. cancer.
162 . Federally Funded Research: Decisions for a Decade

Table 5-6-A Comparison of Science Megaprojects (in billions of dollars)

Original Most Spent

cost recent so far Timeframe

Project estimate estimate (since) (years)

Hubble Telescope . . . . . . . . . . . . . $0.29-0.34 > $2 —a

(1973) (1978)

Space Station . . . . . . . . . . . . . . . . $37 16

(1983) (1985)

Superconducting

Super Collider. . . . . . . . . . . . . . 8.6 0.35-0.43 11

(1987) (1988)

Human Genome . . . . . . . . . . . . . . 3 0.16 15

(1988) (1988)

aProject completed

NOTE: Original cost estimates do not include inflation, while the recent estimates and the amount spent so far include

inflation. Hubble expenditures include development ($1.5 billion) and operating costs ($0.5 billion), from fiscal

years 1978-1991. Ail cost estimates are rounded.

SOURCES: Based on “The Outlook in Congress for 7 Major Big Science Projects,” T% Chroniole otl+gherfducafion,

vol. 37, No. 2, Sept. 12, 1990, p. A28; Genevieve J. Knezo, “Science Megaprojects: Status and Funding,

February 1991 ,“ CRS Report for Congress, 91-258 SPR (Washington, DC: Congressional Reseatch

Service, Mar. 12, 1991); Phil Kuntz, “Pie in the Sky: Big science Is Ready for Blastoff,” Congressional

Quarterly Weekly Report, vol. 48, Apr. 28, 1990, pp. 1254-1260; and National Aeronautics and Spaca

Administration, Office of Resources Analysis, Office of the Comptroller, personal communication, Mamh

1991.

information appropriable by all who want it and can

benefit. The SSC would not be defined as a

competitor of CERN (the European Organization for

Nuclear Research in Geneva, Switzerland) in some

private particle race pursued by U.S. high-energy

physicists; there would also be no national objective

to keep the American rate of discovery above that of

European or Japanese physicists.54 While the prevailing claim is that ‘‘priority races’ are necessary

to make progress in science, cost- and informationsharing are consistent with a view of research as an

appropriable, world public good.55

While scientific and social merit are abstract, they

provide a framework to evaluate the merits of

proposed big science projects. More concrete concerns include the range of megaproject costs and

their management.

Megaproject Costs and Management

The Federal Government buys big science initiatives, and the initial investment may represent a

point of no return. Once the “go, no-go” decision

has been made at the national level, the commitment

is expected to be honored, no matter how much the

cost estimates or timetables for completion change.

However, criteria for consideration in the funding of

a science megaproject could conceivably include:

startup and operating costs, and likely changes in the

overall cost of the project from initial estimate to

completion. Table 5-6 presents a comparison of four

projects, which shows that the cost estimates for

some big science projects double before they are

even begun.

Table 5-7 presents the budget authority for four

projects in fiscal years 1990 and 1991. The percentage increases requested are considerably larger than

the average annual increase in total budgets proposed for the cognizant research agencies. The costs

incurred in future years by most megaprojects are

enormous, and it is unclear that all of the projects

currently receiving funds can be supported in

coming years.

In addition, costs for maintenance of a big science

facility once it is operational are rarely considered.

~see S.s. Y~amOtO, “A Genuine Global Partnership?” Nature, vol. 346, Aug. 23, 1990, p. 692. This has likewise been an issue in the Human

Genome Projecq since James Watson, Director of the National Institutes of Health’s National Center for Human Genome Research has been outspoken

about the disappointing level of funding and commitment to the project by the governments of Japan and France. See Bormq op. cit., footnote 43, p.

7. Also, the benefits of megaprojects include not only the scientitlc knowledge generated, but the technological know-how gained in designing and

building the instruments.

55Two litera~es are relevant here. One is on analysis of ‘simultaneous multiple discoveries’ and Creativity in science; the O~er is on tita sharing

and the diffusion of knowledge. On the former, see Dean Keith Simontoq Scientific Genius: A Psychology of Science (Cambridge, England: Cambridge

University Press, 1988); on the latter, see David S. Cordray et al., “Sharing Research DaW With W’hoU When, and How Much?’ paper presented at

the Public Health Service Workshop on Data Management in Biomedical Research Chevy Chase, MD, Apr. 25-26, 1990; and Eliot Marshall, “Data

Sharing: A Declining Ethic?” Science, vol. 248, May 25, 1990, pp. 952,954-955,957.
Chapter 5—Priority Setting in Science . 163

Table 5-7-Four “Big Science” Initiatives in the Fiscal Year 1991 Budget

(estimates in millions of current dollars)

Fiscal year Fiscal year Proposed Fiscal year Enacted

19!30 1991 percent 1991 percent

Initiative enacted proposed increase enacted change

Strategic Defense

Initiative . . . . . . . . . . . $3,600 $4,500 25% $2,900 –1 9%

Space Station . . . . . . . . . 1,750 2,451 40%0 1,900 90/0

Superconducting

Super Collider. . . . . . . 225 331 47% 243 8%

Human Genome . . . . . . . 60 108 41% 88 47%

Total 5,635 7,390 31% 5,131 –9%

SOURCE: Michael E. Davey, Congressional Research Service, “Research and Development Funding: FYI 991 ,“ isaue

brief IB90048, Nov. 13, 1990,

The Space Station promises to require at least $1.5

billion per year in maintenance-an amount not

figured into original cost estimates.56 Much of the

maintenance support will be transported by the

Shuttle, which has proven less than reliable in recent

years. These concerns raise questions about how

realistically operations are weighed in securing

approval of megaprojects.

Another concern is the ‘‘top-down’ organization

of big science projects. For example, one critic of the

HGP endorses both the goals and the quality of the

science so far, but calls it ‘‘. . . overtargeted, overbudgeted, overprioritized, overadministered, and

. . . micromanaged. ’57 In contrast, some projects

are criticized for a lack of management: ‘‘Though

over $4 billion has been spent so far on the Space

Station, it exists only as a paper design, and with

virtually no purpose beyond serving as a platform for

the glamour of man in space. ”58 Clearly, management is an important consideration in megaproject

development.

Any big science project on the forefront of

expertise will involve considerable learning by

doing. Once a megaproject has been selected,

real-time evaluations of its progress can also be

carried out that give rapid feedback to those involved. 59 While there is no guarantee that agency

sponsors of megaprojects will listen to evaluators,

the latter can become another constituency defined

into the decisionmaking process.

In sum, megaprojects will always be selected

through a political, public process because of their

scale, lumpiness, and incommensurability. Yet, for

each initiative, as the NAS priority report reminds: "

. . . it is necessary to specify the institutions,

individuals, and organizations that will be served;

the costs; the opportunities for international cooperation and cost sharing; the management structure;

and the timeliness of the program. ”60 The cost of

investment for the Federal Government and the cost

per investigator are criteria that apply to all science

initiatives. The designations ‘‘big’ and ‘‘little’ are

quite variable when projected over time and relative

to the total value of an agency’s portfolio. Clearly,

the process of making Federal research investments

could become more iterative, less sequential, and

better oriented to national goals. OTA next examines

an alternative to current practice.

Sbcordes, op. cit., footnote 34, p. A28.

svDon Brown quoted in Roberts, op. cit., footnote 42, p, 756. For example, there are now a total of six National hISthWS of Health-supported genome

research centers and growing questions about acceptable costs and error rates in sequencing a genome before deposit in a database. See ‘‘New Human

Genome Centers Established,” Chemical & Engineering News, vol. 69, No. 6, Feb, 11, 1991, p. 16; and Leslie Roberts, “Large-Scale Sequencing Trials

Begin,” Science, vol. 250, Dec. 7, 1990, pp. 1336-1338.

s~’’Man.~-space: The se~-~icted CwSe of NASA, ” Science & Government Report, VO1. 20, No. 13. Aug. 1, 1~, p. 4.

59 See K. Guy and L. Georghiou, “Real-Time Evaluation and the Management of Mission-oriented Research: The Evaluation of the Alvey

Program--Aims, Achievements and Lessons,’ unpublished paper, presented at the ECE Seminar on Evaluation in the Management of R and D, Apr.

3-7, 1989. In additio~ifreal-time evaluation had been heeded in the construction of the Hubble TUescope, for instance, a full-scale test of the mirror

could have been performed. See Bob Davis,“NASA Management Flaws Led Agency to Overlook Hubble Defec4 Panel Finds,” Wall Street Journal,

Nov. 27, 1990; and William Booth “Hubblc Report Faults Builder, NASA, ” The Washington Post, Nov. 28, 1990, p. A8.

~ational Academy of Sciences, op. cit., footnote 17, p. 11.
164 ● Federally Funded Research: Decisions for a Decade

Research Priorities and the

“Big Picture”

Figure 5-4 depicts the projected outlays for the

science base and science megaprojects discussed

above. The projected expenditures for big science

projects rise in the 1990s as an increasingly significant portion of those for science projects as a whole.

(Since cost estimates for megaprojects tend to grow

precipitously, a similar figure that doubles those

expenditures are included for sake of comparison in

figure 5-4.61) Within the current funding climate and

that predicted for the 1990s, perhaps not all components of the current Federal research portfolio can be

supported. Choices among science projects may

need to be made. Because of the large projected

lifetime costs associated with each megaproject,

sorting and recalibrating the costs of each earlier

rather than later would be useful.

How could such choices be made? Ideally, one

might ask that Federal funds be allocated to the

science base and then add megaprojects in order of

importance until funds are depleted. However, such

a sequential approach is not realistic. First, there is

nothing that corresponds to a single research budget.

Many countries, for example, Canada, Germany,

and Sweden, have capital budgets for all functions,

including research. If the United States had a capital

budget distinct from its operating budget, then it

could rate megaprojects against one another and

compare them with other capital investments. Second, megaprojects are funded on an equal footing in

many agencies with other research programs. Finally, in the words of Albert Teich, of the American

Association for the Advancement of Science:

Advocates of systematic priority setting and those

who may be called on to advise in the process need

to recognize that any such rational analysis is just

one element of the picture. Such analysis may

influence the process, but it does not determine

priorities. Other factors and other voices will and

should be heard. Political criteria are not a contaminant in the allocation of public resources for

research; they are absolutely essential to the democratic process and to the long-run effective functioning of the system.62

An annual review of commitments across categories of investment would help to gauge balance by

field, research problem, and agency contributions to

the achievement of national goals. By revisiting

these categories year after year, Federal investments

could be appraised to add and subtract from the

Nation’s research portfolio.63

Once the context for priority setting is examined,

tradeoffs and choices take on another dimension.

What do U.S. society and the Federal Government

expect for their research investment? What does the

scientific community promise to deliver? The answers differ among participants and over time. The

answers differ because criteria and expectations

differ, because there are plural research systems, and

because participants can influence the process of

budgeting and research decisionmaking at many

levels.

Although scientific merit and program relevance

must always be the first criteria used to judge a

research program or project’s potential worth, they

cannot be the sole criteria. First, in today’s research

system, there are many more scientifically meritorious projects than can be funded. In its initial effort

to document stress on the Federal research system

created by an abundance of research applications,

OTA found that an increasing proportion could not

be funded by various research agencies due to

budget limitations, rather than to deficiencies of

quality. 64 Second, rewarding scientific merit and

relevance alone can inhibit the system from preparing for the future. This problem is seen clearly in the

finding of young investigators. Since the prospective yield of new knowledge is judged by the

technical merit (e.g., soundness of design or experimental protocol) of a project proposal, its scientific

creativity, and the track record of the scientist,

young investigators are at a disadvantage, and other

criteria must be weighed when evaluating their

proposals.

GINote tit some estimates of megapmj~ts include only capital costs, while other include capital md OPra@ costs.

62~~~ H. ~ic~ ~ $Scimtists ~d ~b~c ~lci~s Must time collaboration To Set Res~ch fiorities, ’ The ~cienfi.~t, VOI. 4, No. 3, Feb. 5, 1990,

p. 17.

63T0 ittiate is t. Plm ~d exercise flefibfi~~~a budget envelo~uch like a Natio~ B~ketb~ Assoc~tion tem shmitS rOSter tO Stlly

under the league’s imposed salary cap while enjoying a full complement of players at every position.

~u.s. Congess, Office of ‘lkchnology Assessment, “Proposal Pressure in the 1980s: An Indicator of Stress on tie Federal Research System” staff

paper of the Scienw, Educatiou and Transportation Program, April 1990.
Chapter 5—Priority Setting in Science ● 165

Figure 5-4—Cost Scenarios for the Science Base and Select Megaprojects: Fiscal Years 1990-2005

Current cost estimates for megaprojects

3 percent growth for 3 percent growth for total

science base

(megaproject funding added on)

research funding

(megaproject funding included)

Constant dollars Constant dollars

I

1990 1995 2000 2005 1990 1995 2000 2005

Doubled current cost estimates for megaprojects

3 percent growth for

science base

(megaproject funding added on)

Constant dollars

3 percent growth for total

research funding

(megaproject funding included)

Constant dollars

1990 1995 2000 2005

 Science base Human genome SSC

1990 1995 2000 2005

 EOS Space station

KEY: Super Collider; Observing System.

NOTE: These figures are schematic representations of projected costs for science projects. In the figures on the left, the base is projected to grow

at an annual rate of 3 percent above inflation. In the figures on the right, is projected to grow 3 percent above inflation. The cost

estimates forthe are based on data from “’The Outlook in Congress for 7 Major Big Science Projects,” The Chronic/e of 

vol. 37, No. 2, Sept. 12, 1990, p. A28; and Genevieve J. Knezo, “’Science Status and Funding, February 1991 ,“ Report for Congress, 91-258 SPR (Washington, DC: Congressional Research Service, Mar. 12, 1991).

SOURCE: Office of Technology Assessment, 1991.
166 . Federally Funded Research: Decisions for a Decade

There is a role for Congress to set priorities across

and within categories of science and engineering

research. The application of criteria that augment

‘‘scientific merit’ and “program relevance”—

which are today’s judgments of quality-would

clarify tomorrow’s objectives of research investment. As discussed in chapter 1, broadly stated, there

are two such criteria: strengthening education and

human resources (i.e., increasing the number and

diversity of participants); and building regional and

institutional capacity (including economic development by leveraging Federal research support).65 Both

sets address the future capability of the research

system in response to national needs, and both can

be employed in mainstream and set-aside programs.

Conclusions

Since progress begets more opportunities for

research than can be supported, setting research

priorities may be imperative for the success of

science in the 1990s.66 And while the questions raised

in this chapter have a familiar ring-how should

Federal monies for research be spent? which opportunities for scientific advance merit funding now?

who should decide?—the search for a framework to

judge criteria of choice has grown urgent. In the

pluralistic and decentralized system of research

decisionmaking, sponsorship, and performance,

there are ample voices to justify most any hierarchy

of programs and projects on the grounds of ‘social’

or “scientific” merit. The question of what do U.S.

society and the Federal Government want for their

research investments has many answers.

Long before the onset of stringency in Federal

discretionary funding, priority setting was an integra1 part of the regular budget process:

By the time any budget for science has been

pulled apart by function in the budget committees,

by agency in the legislative committees, and by

appropriations bills in the appropriations committees

(in both House and Senate at each of these levels)

and reassembled among the various other programs

of veterans’ benefits, sewage treatment grants, and

agricultural price supports, its internal priorities will

be unrecognizable.67

The problem is not a lack of priority setting. The

problem is implementing priorities in the name of

national goals and scientific needs. How can that be

achieved?

Some observers of the current priority-setting

process have suggested improvements to the process

that are structural, in particular centralizing the

budget process and intensifying research planning

within and across the agencies. This would make the

tradeoffs more explicit and less ad hoc, and the

process more transparent. At a minimum, multiyear

budgeting and an agency crosscutting budgetary

analysis (proponents like NAS say) could reduce

uncertainty in budgeting.68

To ensure that priorities are set, some persons,

committees, or bodies of the Federal Government, in

addition to the President, must be invested with the

power to set priorities. Agency managers are already

performing this function at a program level, with

oversight from the legislative branch. At the highest

level of decisionmaking, however, a crosscutting

function is required. In the executive branch, OSTP

and OMB are the only actors with the ability to play

such a sweeping role. Without additional legislative

initiatives, however, OSTP is hampered by the

powerlessness of its advisory position. And OMB,

which has been serving a crosscutting function in the

executive branch, is not receptive to incorporating

debate and public decisionmaking on these issues.

Congress already serves, in part, a crosscutting

priority-setting function. However, Congress has

traditionally been reticent to set priorities. Suggestions have been made to strengthen Congress’ hand

in research decisionmaking through structural

65somc agency ~ro=m ~mdy ~cowrate time Crittia. They are explicitly in use, for example, at the National Stimw Fo~dation (NSm (~ou~

not in every program or directorate) and there have been no claims that scientific merit has been compromised. At other agencies, however, these criteria

are seen as not as important to the research mission (OTA interviews, spring 1990). At the same time, set-aside programs at NSFand elsewhere underscore

the continuing need for ‘‘sheltered competitions”for researchers who do not fare well in mainstream disciplinary programs.

‘iBrooks writes: “Today many of the same negative signals that existed in 1971 are again evident. Will science recover to experience a new era of

prosperity as it did beginning in the late seventies, or has the day of reckoning that so many predicted ftily arrived?” Harvey Brooks, “Can Science

Survive in the Modern Age? A Revisit After Twenty Years,’ National Forum, vol. 71, No. 4, fall 1990, p. 33.

G7~ic& op. cit., footnote 62, p. 18.

6STMS was, of course, prior to the 1990 budget summit and passage of the Deficit Control Act discussed in ch. 3. The National Academy of Sciences

discussion is nevertheless instructive. See National Academy of Sciences, op. cit., footnote 17, pp. 11-16, especially table 2. Also see U.S. General

Accounting Office, U.S. Science and Engineering Base: A Synthesis of Concerns About Budget and Policy Development, GAO/RCED-87-65

(Washington DC: March 1987), especially pp. 22-56.
Chapter 5—Priority Setting in Science . 167

change in the budget process, and there has been an

evolution toward greater congressional activism.

However, Congress may wish to strengthen its

current role as the final arbiter of priorities and invest

others with the discretion to propose priorities.

Whatever Federal body is designated as having

the authority of initial choice, its task should extend

at a minimum to the iterative planning and appraisal

of accounts that results in: a) limiting the number of

(or budget commitment entailed by) megaproject

initiatives, and b) making tradeoffs among research

fields in the S&T base. For instance, the broad field

of the life sciences has received substantial increases

in funding over the last 15 years, while other fields

have climbed more slowly. Seen as part of the

Federal research portfolio, the life sciences could be

stabilized in funding, while certain other fields,

ranked according to other criteria (e.g., training of

students), could be slated for augmented funding.

Already included in most research decisionmaking

are criteria based first and foremost on scientific

merit. OTA suggests that two other criteria could be

added to scientific merit. These criteria emphasize

planning for the future-strengthening education

and human resources, and building regional and

institutional capacity. Education, human resources,

and regional and institutional capacity are valid

outcomes of Federal research investments. Progress

.

toward achieving national objectives that incorporate these criteria should be monitored with congressional oversight.

Reordering the criteria of choice changes the

process and the expectations of returns from the

investment in research. Such reconfiguration, perhaps seen most clearly in big science projects,

demonstrates how embedded science and technology have become in the myriad needs of the Nation.

These initiatives are appropriated by political actors

because they are much more than cutting-edge

research. They represent ‘‘real money’ ‘—in jobs,

industrial development, innovation, trade, and prestige regionally, nationally, and internationally. This

is why the constituencies for them are broad and why

they remain controversial within their respective

research communities years after having been proposed and the down payment made by the Federal

Government.

Enhanced priority setting could be the 1990s’

expression of the post-World War II social contract

that bound science to government. However, greater

priority setting in science is no panacea for the

problem of research tiding. It is a partial response

to the problem of how the Federal research system

can make choices in the coming decade. Another

response of comparable urgency-understanding

and coping with research expenditures-is discussed in chapter 6.
CHAPTER 6

earch

itures
Contents

Page

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

Expenditures From the Federal Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

Direct v. Indirect Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...++... ,.,..... . . . . . . . 174

Aggregate Expenditure Data for Personnel Facilities, and Equipment . . . . . . . . . . . . . . . . . . . . . 175

Research Expenditures at the National Institutes of Health. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

Research Expenditures at the National Science Foundation . . . . . . . . . . . . . . . . . . . . . . . . . . . ,..+185

The Research Performer’s Perspective on Expenditures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

Components of Research Expenditures at Universities . . . . .. .. . .. ...+..... . . . . . . . . . . . . . . . 188

Changing Expectations and Competition.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..+. 195

summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

Boxes

Box Page

6-A. Financial Planning at NIH: ACongressional Mandate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

6-B. OTA Interviews at Two Universities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

6-C. Federal Funding at Two Universities . . . . . . . . . . . . . . . ,.....4. . . . . . . . . . . . . . . . . . . . . . . . . . 190

6-D. A Federal Research Facilities Program? Perspectives From Academia . . . . . . . . . . . . . . . . . . . 194

6-E. Emulating the Research University: Beware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

6-F. Industry on Campus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Figures

Figure Page

6-1. Academic R&D Expenditures per FTE Investigator by Type ofExpenditure: 1958-88 . . . . 173

6-2. Estimated Cost Component of U.S. Academic R&D Budgets: 1958-88 .. .. ... .+. . . . . . . . 173

6-3. Indirect Cost Ratios for NSF and NIH: 1966-88 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

6-4.Total Expenditures and Unit Costs for Recent and Planned Academic Capital Projects:

1986-91 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

6-5. Relative Sources of Funds for Research Facilities: Academic Capital Projects Begun in

1986-89 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .,..+... .,.+.. 178

6-6. Direct Costs Awarded for NIH Research Projects, Research Centers, and Other Research

Grants: Fiscal Years 1979-88 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

6-7. Biomedical Research and Development Price Index: Fiscal Years 1979-89 . . . . . . . . . . . . . . 182

6-8. Comparison of BRDPI With Other Price Indices: Fiscal Years 1980-89 . . . . . . . . . . . . . . . . . 185

6-9. NIH Biomedical Research and Development Price Index: Fiscal Years 1979-89 . . . . . . . . . 185

6-10. Growth in University and College R&D Performance,by Source of Funds: Fiscal Years

1978-88 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

6-11. Federally Funded Basic Research, by Performer: Fiscal Years 1969-90 . . . . . . . . . . . . . . . . . 188

6-12. Average Salary and Benefits Paid Academic Ph.D.s in Natural Sciences and Engineering:

1958-88 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

Tables

Table Page

6-1. Selected Sources offending for 1985-86 National Stock of In-use Research Equipment

and Percent Change From 1982-83, by Field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

6-2. Types and Expenditures for Most Needed Research Equipment National Estimates for

the Biological Sciences, 1984 and 1987 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... .+... 180

6-3. Extramural Direct Costs by Budget Category at NIH: Fiscal Years 1979-88 . . . . . . . . . . . . . . 182

6-4. Biomedical Research and Development Price Index: Fiscal Years 1979-89 . . . . . . . . . . . . . .+ 186

6-5. Biomedical Research and Development Price Index: Percent Change From Prior Fiscal Year,

Fiscal Years 1980-89 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
CHAPTER 6

Understanding Research Expenditures

University research is a smokestack industry. That the research university’s capital

costs are small and easy to cope with is a myth.

Introduction

Many researchers state that the problem with

research funding in the United States is that it has not

kept pace with inflation. “Inflation” in this context

refers not to inflation in the Gross National Product,

but to the rise in apparent research costs. Several

factors contribute to research expenditures, but the

most notable is the sheer size of the enterprise.

Contributing to confusion over the issue of costs

in research are the numerous and sometimes inconsistent meanings of “costs,” and the lack of a

suitable measure of “research.” Specific research

activities generally become cheaper to complete

with time, due to increasing productivity, for example, of computers and other technologies. However,

advances in technology and knowledge also allow

deeper probing of more complex scientific problems

and create demand for greater resources. Because

success in the research environment depends heavily

on ‘‘getting there first,’ there is clear advantage to

having the financial support to acquire additional

staff and cutting-edge technology. Thus, competition drives up demand for funding. In this sense, the

demand for more resources (costs of research) will

continue to outpace any increases in Federal funding. (For a more complete discussion, see chapter 1.)

William F. Massyl

Available data suggest that an increase in the

number of scientists supported by Federal funds

combined with real growth in their salaries and

benefits have figured heavily into total Federal

expenditures. 2 In recent years growth in research

budgets (i.e., support) has also been accompanied by

a growth in researcher’s expectations (i.e., demand).

In addition to increased competition through the

1980s for available agency research funding,3 

research expenditures on scientific projects (both

direct and indirect) have grown, generally above the

rate of inflation. Some claim that research requires

more expenditures today because of complexity:

what was done yesterday can often be done cheaper

today, but ‘tomorrow’s science’ may cost more. As

understanding of natural and social phenomena

increases, the questions to be answered become

more intricate, resulting in increased expenditures or

‘‘sophistication inflation.”4 Complying with increasing layers of regulation has also been cited as

responsible for increased expenditures.5

Unfortunately, few systematic analyses have been

performed to evaluate these claims. Complicating

questions on the cost of research are incomplete and

murky data on research expenditures. Definitions of

what is being measured over time are straightforward, but the activity that they purport to capture is

constantly changing. In addition, much of the

current debate over expenditures takes place within

IWilLiam F. Massy, ‘‘Capital Investment for the FutwE of Biomedical Research: A University Chief Financial Ofilcer’s View,’ Academic Medicine,

vol. 64, August 1989, p. 433.

researchers are now submitting more proposals to improve their chances of maintaining or increasing previous support levels. Natioml Science

Foundation, “NSF Vital Signs: Trends in Research Support FY 80-89,” draft repo~ November, 1990, p. 3. The National Science Foundation (NSF)

found that the average number of proposals submitted by an investigator to win one NSF award had risen ffom about 1.5 to 1.7. NSF notes that these

data do not address the extent to which the increase in proposaf submissions to NSF is the result of perceived difficulty in winning awards or other factors

such as growth in the population of research fields or greater pressure to win awards for professional advancement.

3u.s. Conwess, office of lkchnology Assessment, ‘‘Proposal Pressure in the 1980s: An Indjcator of Stxess on the Federal Research System,” staff

paper of the Science, Education, and Transportation Program, April 1990.

4~C-~nC~: The End ~~~h~ f7ronri~r? a ~pofi from ~on M. ~de~~, president.elec~ to we BO~d of D~ectors of tie American Association fOr the

Advancement of Science (Washington DC: American Association for the Advancement of Science, January 1991), p. 6; and D. Allan Bromley,

“KeynoteAddress,’ Science and Technology and the Changing World Order, colloquium proceedings, Apr. 12-13, 1990, S.D. Sauer (cd.) (Washington

DC: American Association for the Advancement of Scjence, 1990), p. 11.

sNatioml Science Fo~~tjo~ Scientific ~~ Engineering Research Facilities at vniver~itie~ and colleges: 1990 (washgto~ DC: SeptembtX

1990), p. xvii.

–171 –
172 Federally Funded Research: Decisions for a Decade

the context of agency budget constraints and pressures felt by research performers. Determining g what

is an adequate amount of Federal money for the

conduct of research is not easy, and is only

compounded by confusion over true costs.6

Analysis of expenditures for the conduct of

research focuses on what Federal agencies are

willing to spend for personnel, facilities, and instrumentation, but gives neither an accurate picture of

what the needs are nor whether expenditures are

being totally recovered by the research performers.7

Individual components of the research budget have

not risen significantly, with the exception of salaries

and fringe benefits. However, more scientists and

engineers are doing research; they are getting paid

more for their work, and they are spending more.

Annual expenditures, including operating, equipment, and capital (facilities) spending per full-time

equivalent investigator, are estimated to have increased from $85,000 (1988 dollars) in 1958 to

about $170,000 by the late 1960s, where they

leveled off through the 1970s. In the 1980s, expenditures rose to $225,000 (see figure 6-l).

Thus, an academic scientist today spends almost

three times as much (in real terms) for research as in

1958. Figure 6-2 shows that expenditures have risen

in every component (personnel, facilities, equipment, students, other) for three decades. However,

available data point to personnel and indirect cost

expenditures as the most important components of

increases. Personnel expenditures have less accounting flexibility: unlike facilities and instrumentation

costs, they cannot be deferred or depreciated.8

Federally supported academic research is salary

intensive. This leads to salaries and fringe benefits

affecting the direct cost equation more than nonpersonnel items.

Indirect costs have been rising faster than direct

costs. Academic institutions claim that is because

the more expensive items, such as facilities and

administration, more often fall into the indirect cost

category, while controllable expenditures, such as

research personnel and graduate students, fall into

the direct cost line. The confusion about indirect v.

direct costs of research is, in part, complicated by

philosophical differences about where expenditures

should be assigned. For at least the past 20 years, a

debate has been carried on between university

administrators and Federal granting agencies over

who should pay for what in academic-based research. Although the grant is for research, it is signed

with an institution, which incurs expenditures beyond the scope of the research being performed. It

has been the practice of the Federal Government to

consider research as integral to the university

mission and, therefore, its cost should be shared by

both parties.9

This chapter looks at the issue of research

expenditures from two perspectives—the Federal

Government as funder, and the research university as

performer. 10 Available data concerning specific

budget items (e.g., salaries, instrumentation, indirect

costs, and facilities) are presented. These data are

collected by granting agencies and tend to reflect

agency expenditures rather than actual costs to the

researcher. Expenditures, as recognized from the

perspective of the university research performer, are

also discussed as a component of financial planning,

proposal-writing strategies, and changing expectations.

6For ~illu~hatiow see D~el E. Kos~~d, “me Underside of Overhead, ” Science, vol. 248, May 11, 1990, p. 645; aud letters, published as “me

Overhead Question,” in response to Koshland’s editorial, Science, vol. 249, July 6, 1990, pp. 10-13.

TA~ysi~ is ~onfo~ded by tie expenditure acco~~g schemes that vary frOm research kStitUtiOIl tO ~tihItiOU m*g compfiso~ ~ti ‘W1dt

and perilous, For au attempt to compare expenditures atwo public and two private universities associated with the performance of National Science

Foundation-funded researc~ see G.W. Baughm~ “Impact of Inflation on Resewh Expenditures of Selected Academic Disciplines 1967- 1983,’ report

prepared for the National Science Foundation and the National Center for Educational Statistics, Nov. 8, 1985. Also see Reseamh Associates of

Washington Higher Education Price Indexes: 1990 Update (Washingto@ DC: 1990).

8For at lmst he fi~ ~omeCutive Yem, faC~~ s~~es ~ve ~crem~ more ~he cost of fiving, From November 1$)89 to November 1990, the

consumer priee index increased 6.3 percent. During that same period, average faculty pay increased 7.1 percent. Reported in ‘Faculty Pay and the Cost

of Living,” cha@ The Chronicle of Higher Education, vol. 37, No. 17, Jan. 9, 1991, p. A15.

%s practice fuels what is known as the ‘ ‘full cost recovery’ debate. See Stephen P. Strickland, Research and the Healrh of Americans @x@tou

MA: Lexington Books, 1978).

l~xpendi~es h tidustri~ rese~ch are not considered here, but have been addressed in two other OTA reports. see U.S. congress, office Of

Txhnology Assessment, Making Things Better: Competing in Manufacturing, O’IA-ITE-443 (Washington, DC: U.S. Government Printing Office,

February 1990); and Government Policies and Pharmaceutical Research and Development (Washington DC: U.S. Government Printhg Offl~,

forthcoming 1991).
Chapter 6—Understanding Research Expenditures . 173

Figure 6-l—Academic R&D Expenditures per FTE

Investigator by Type of Expenditure: 1958-88

(in thousands of 1988 dollars)

1988 dollars (in thousands)

200

150

100

50

0

I I I I I I I 1 I 1 I I I I I I I I I I I I I I I I I I I

200

150

100

50

0

1958 1963 1968 1973 1978 1983 1988

+ Operating funds ~ Equipment

+ R&D facilities

NOTE: Constant dollars were calculated using the GNP Implicit Price

Deflator.

DEFINITION OF TERMS: Operating funds refer to current fund expendituresforacademic research and development (R&D) activities that

are separately budgeted and accounted for, including expenditures

forseniorscientist and graduate studentcompensation, otherdirect

costs, and indirect costs associated with conduct of academic

research. Equipment includes reported expenditures of separately

budgeted current funds for the purchase of academic research

equipment, and estimated capital expenditures for fixed or built-in

research equipment. R&D facilities include estimated capital expenditures for academic research facilities. Full-time equivalent

(17E)investigators include those scientists andengineersconducting funded (separately budgeted) academic R&D; the FTE is an

estimate, derived from the fraction of faculty time spent in those

research activities, nonfaculty scientists and engineers employed

to conduct research in campus facilities (except federally funded

R&D centers), and postdoctoral researchers working in academic

institutions.

SOURCE: Government-University-Industry Research Roundtable, Science and Technology in the Academic Enterprise: Sk@s,

Trends and Issues (Washington, DC: National Academy Press,

1989), figure 2-45.

Figure 6-2—Estimated Cost Components of

U.S. Academic R&D Budgets: 1958-88

(in billions of 1988 dollars)

1988 dollars (in billions)

6

5

4

3

2

1-

>

c

0’

1958 1963 1968 1973 1978 1983 1988

+ Senio r +- Graduate #- other direct

scientists students

+ Indirect + Equipment * Facilities

NOTE: Constant dollars were calculated using the GNP Implicit Price

Deflator.

DEFINITION OF TERMS: Estimated personnel costs for senior scientists

and graduate students include salaries and fringe benefits, such as

insurance and retirement contributions. Other direct costs include

such budget items as materials and supplies, travel, subcontractors, computer services, publications, consultants, and participant

support costs. Indirect cmstsincludegeneral administration, departmentadministration, building operation and maintenance, depreciation and use, sponsored-research projects administration, libraries, and student services administration. Equipment costs include

reported expenditures of separately budgeted current funds for the

purchase of research equipment, and estimated capital expendituresforfixedor built-in research equipment. Facilities casts include

estimated capital expenditures for research facilities, including

facilities constructed to house scientific apparatus.

DATA: National Science Foundation, Division of Policy Research and

Analysis. Database: CASPAR. Some of the data within this

database are estimates, incorporated where there are discontinuities within data series or gaps in data collection. Primary data

source: National Science Foundation, Division of Science Resources Studies, Surveyof Scientific and Engineering Expenditures

at Universities andColleges; National Institutes of Health; American

Association of University Professors; National Association of State

Universities and Land-Grant Colleges.

SOURCE: Government-University-Industry Research Roundtable, Science and Technology in the Axdemic Enterprise: Status,

Trends andkxwes (Washington, DC: National Academy Press,

1989), figure 2-43.
174 ● Federally Funded Research: Decisions for a Decade

Photo credit: Research Triangle Institute

The cost of complying with regulations of research

procedures and equipment is one of a long list of changing

expenditures for Federal research. However, indirect costs

and salaries are the largest expenditures in federally

funded research.

Expenditures From the Federal

Perspective

This section explores what is known about

research expenditures from the perspectives and

databases of two granting agencies—the National

Science Foundation (NSF) and the National Institutes of Health (NIH). It also includes a discussion

of cost data collected by NSF pertaining to all

Federal research and development (R&D) (as that is

the level at which the available data are aggregated),

as well as analyses conducted by the National

Academy of Sciences and other cost analysts.

Direct v. Indirect Costs

The Office of Management and Budget (OMB)

guidelines for indirect costs include those that are

incurred for common or joint objectives and therefore cannot be identified readily and specifically

with a particular sponsored project, an instructional

program, or any other institutional activity.ll Indirect costs reflect the contractual arrangements between the agency and a particular university, regardless of actual expenditures at that university. The

rate is negotiated based on allowable charges, past

experiences, and expectations for the period under

negotiation. The indirect cost ratio is the proportion

of total award budget applied to indirect costs. In

general, all research agencies pay the same indirect

cost rate at a given institution (the Department of

Agriculture is the exception in that formulas are

used). Direct costs are those that can be identified

with a particular sponsored project, instructional

program, or any other institutional activity; or that

can be directly assigned to such activities with a high

degree of accuracy.

The guidelines for calculating costs were developed in conjunction with OMB Circular A-21 and

have been in force since 1979. OMB also specifies

the method for calculating the indirect portion of

salaries and wages paid to professional employees.

A requirement exists for assigned workload to be

incorporated into the official records and for that

system to reflect 100 percent of the work for which

the employee is being compensated.12 Thus, the

record should show the percentage of time spent on

research, teaching, and administrative duties.

Every major research university has an indirect

rate established for the current fiscal year for

recovery of costs associated with sponsored research. These rates have evolved over many years as

a result of direct interaction and negotiation with the

cognizant Federal agency. There is a wide range of

indirect cost rates among universities, with most

noticeable differences between public and private

institutions; rates tend to be higher at private

institutions. 13 Rates vary because of: 1) real and

significant differences in facilities-related expenditures, 2) tacit or overt underrecovery by some

universities, 3) imposition of arbitrary limits by

 of Analysis, Biomedical Research Index.- Report to the Institutes

of Health DC: Mar. 30, 1990).

 of Assessment, The Regulatory Science, 

VA: National Technical Information Service, February 1986), pp. 73-76.

 on investigation Of 

cost practices at universities, beginning with Stanford. See Marcia “Stanford SailS Into a Storm,’Science, vol. 250, Dec. 21, 1990, p. 1651;

“Government Inquiry,’ Stanford Observer, November-Deeember 1990, pp. 1, 13; and Marcia ga, ‘‘Indirect Costs: How Does Stanford Compare

With Its Peers,” Science, vol. 251, Feb. 15, 1991, pp. 734-735.
Chapter 6—Understanding Research Expenditures ● 175

Figure 6-3—indirect Cost Ratios for NSF and NIH:

1966-88 (indirect cost as a percent of total R&D cost)

In percent

35

3 0

25

20

15

10

5

0 I I I I I I I I I 1

1966 1971 1976 1981 1986

— NIH + NSF

KEY: Institutes of Health; Science Foundation.

SOURCE: National Science Foundation, Policy and Analysis

Division, estimates based on unpublished and NSF data,

1990.

some government agencies in the negotiation process, and 4) diversity in assigning component

expenditures as direct or indirect.14

Figure 6-3 indicates the trends in indirect costs as

a proportion of total research expenditures for NIH

and NSF. In part, the ratios vary because NIH

separates direct and indirect costs, and proposals are

evaluated based mainly on direct costs. NSF, on the

other hand, considers total costs in making an award

(usually after merit review).

Confusion about the relationship between the

indirect cost rate and what is allowable for adminisPhoto credit: University of Michigan

Indirect cost rates vary in part because of differences in

campus facilities. The University of Michigan, pictured

here, devotes many of its facilities to research.

trative and student service components reflects the

difficulty of separating expenditures along lines of

research, instruction, and other functions.15 Equipment and facilities-related components of the rate

seem to be less controversial, perhaps because of

better documentation of expenses in these areas.

Some have advocated that two rates should be

calculated for indirect costs-one for facilities and

equipment, and one for all other components, such

as administrative, library, and student services.l6

This view has considerable relevance as universities

renovate or replace aging facilities and equipment.

Aggregate Expenditure Data for Personnel,

Facilities, and Equipment

Expenditures for facilities and equipment are

frequently cited as a drain on the academic financial

resource base. These data, however, mix actual and

planned expenditures. As universities compete

against industry and each other for resources,

funding needs grow, as does spending. Competition

in the university environment has also driven up the

“set up’ price of the average scientist.17 Data on

salaries and personnel are more reliable because

 of Indirect Costs Associated With Federal Support of Research on University Campuses: 

Change (Washington, DC: December 1988).

 C. L. Directorate for Scientific, Technological, and International Affairs, National 

“Indirect Costs of Federally Funded Academic ” unpublished Aug. 3, 1984, p. 1.

 of American Universities, op. Cit., 

 Trends, and 

 DC: National Academy Press, 1989), p. 2-33.
176 . Federally Funded Research: Decisions for a Decade

these cost categories cannot be deferred and are

documented annually.

Personnel

For the past three decades, personnel expenditures

have accounted for about 45 percent of total costs of

academic research charged to the Federal Government, consistently the largest share of the budget.

Salaries have been on the rise and from 1981 to 1988

the number of scientists and engineers employed in

academic settings increased steadily from about

275,000 to almost 340,000. 18 Increased numbers of

investigators and rising salaries (and the benefits

that go with them) have driven up the price of the

personnel component of direct costs. In future years,

it is anticipated that the personnel component of

research budgets will rise further due to a faster rate

of inflation in salaries than in other categories such

as equipment and facilities.19 And the fiscal year

1991 appropriation for NSF lifted the $95,000

annual salary cap on principal investigators that can

be charged to a grant.20

The patterns for spending on graduate students

mirror that for principal investigators. Increases in

the number of graduate students supported, however, were larger than the growth in the number of

scientists due to a greater reliance on the research

grant as a support mechanism.21

Research Facilities Construction and

Renovation

Research facilities may be defined as the environment within which research is conducted, as opposed to research instruments, or the tools that

scientists and engineers use to collect data. Facilities

currently receive about 10 percent of the Federal

R&D budget compared to about 6 percent at the

beginning of the 1980s.22 Most of the data available

on scientific and engineering research facilities are

collected by NSF on a biennial basis in response to

the 1986 National Science Foundation Authorization Act (Public Law 99-159). The act required NSF

to design, establish, and maintain a data collection

and analysis capability for the purpose of identifying

and assessing the research facilities needs of universities arid colleges. Data are not available before

1986. The assessments are based in part on estimates, relying on reported capital projects-both

actual and planned-and anticipated spending for

construction and repairs of research facilities.23

Actual expenditure data are derived from expenditures in previous years.

In constant 1988 dollars, annual capital expenditures for academic science and engineering facilities

nearly tripled during the ‘‘golden age” from $1.3

billion in 1958 to $3.5 billion a decade later.

Expenditures dropped to $1 billion in 1979 (1988

dollars) and stand at $2 billion in 1988. Presently,

the Federal share of facilities funding is 11 percent,

down from a high of 32 percent in the 1960s.24

In 1986-87, academic institutions initiated major

repair and renovation projects in academic research

space totaling $840 million. In 1988-89, this figure

rose to $1.04 billion.25 Estimated deferral rates for

repair and renovation are $4.25 for every dollar spent

in 1990, up from $3.60 in 1988.26

Institutions’ spending for new construction of

research facilities was expected to grow from $2.0

billion in 1986-87 to $3.4 billion in 1988-89, an

average increase of about 30 percent per year.27 A

1990 update revealed that costs for 1988-89 new

lg~id., p. 2.34, based on National Science Foundation ~~.

1gNatioti Science Foundatio~ The State of Academic Science and Engineering (Washingto& DC: 1990), pp. 119-149. Of course, penonnel

expenditures would be much higher if full salary and fringe benefits were charged to the Federal Government. Most universities absorb a substimtial

portion of such expenditures. Some agency programs will pay for only 2 to 3 summer months. Leonard Lederma.q Directorate for Scientific,

Technological, and International Affairs, National Science Foundatio~ personal communication, December 1990.

m’ ‘NSF Back ‘lb Normal After Budget pause, ’ Chemical & Engineering News, vol. 68, No. 48, Nov. 26, 1990, p. 12.

ZINatio~ Science Foundation, op. cit., footnote 19, pp. 124-125.

~Ibid., pp.134-139.

~Acwrding t. Nation~ Scienm Fou&tion comen~tors on ~s octo~r 1990 OTA dr~t c~pter, es~tes of ‘ne~” denote defend Of facilities

expenditures (both new construction and repair/renovation project..), not an institutiomd “wish list. ” National Science Foundation staff, personal

communicatio~ December 1990.

2 4 &v ernment-University-Industry Research Roundtable, op. cit., footnote 17, pp. 2-28, 2-29.

~Natio~ Science Foundation, op. cit., footnote 5, p. Xvii.

261bid., p. xix.

ZTNatio~ Science Fo~~tiom Scientific ~~ Engineering Re~earch Facilities at universities ad Colleges: 1988 (Washi.ngtom DC: September

1988).
Chapter 6-Understanding Research Expenditures . 177

Figure 6-4—Total Expenditures and Unit Costs for Recent and Planned Academic

Capital Projects: 1986-91

Total expenditures Unit costs

Cost per square foot of Dollars (in billions) 4.0 - research space (in dollars) 350 “

$3.5 $311

3.5 300

3.0 $2.5 $231 250

2.5 $2.1 $207

200

2.0

1.5 - 150 -

$0.84 $-1.04--$096 $91 .$111

1.0 - 100 -

--- -. ---- $62 

0.5 - 50 - “ ---

0 , , , , , , 86-87 88-89 90-91 0 86-87 88-89 90-91

(plan) (plan)

— New construction — New construction

---- Repair/renovation --- Repair/renovation

NOTE: Estimates of research space are based on net assignable square feet assigned to organized research.

SOURCE: National Science Foundation, Scientific and Engineering Research Facilities at and Colleges:

 final report, NSF 90-318 (Washington, DC: 1990), chart 4, and p. A-10.

construction projects totaled only $2.5 billion,

considerably less than projected.28 Private institutions among the top 50 recipients of Federal R&D

funds report considerably higher spending levels

than public institutions, both for construction and

repair and renovation. The reverse is true among

institutions not in the top 50, where spending levels

are higher at public institutions.29

The unit cost of new construction (the cost per net

square foot) grew in real terms from $207 in 1986-87

to $231 in 1988-89, an increase of about 12 percent

per year (see figure 6-4). Construction expenditure

increases of this magnitude, which are above the rate

of inflation, are attributed in part to changing

technical and regulatory requirements such as animal quarters,30 biohazard containment safeguards,

and toxic waste disposal facilities. These regulatory

requirements are especially relevant to the medical

and biological sciences, but vary with the institutional setting in which the research is conducted.31

The proportion of Federal support for construction is about 11 percent in private institutions and 8

percent in public institutions (see figure 6-5). The

Federal Government also pays for renovation and

repair costs in part through the indirect cost rate, and

in 1988, the Federal Government supplied nearly $1

billion to support university infrastructure through

indirect costs. Almost 20 percent was for facilities

depreciation, while the rest was recovered for

 op. cit., footnote 5. The report suggests that inability to obtain was given by

institutions for postponing or scaling back planned construction projects.

 pp. 19-24.

 Alcohol, Drug Abuse, and Mental Health “ “stration estimates that new regulations for animal care will cost $40,000 to $70,000 per grant

for the care of primates and dogs. See Constance “A Preemptive Strike Science, vol. 244, Apr. 28, 1989, pp. 415-416.

An American Association of Medical Colleges survey of 126 medical schools estimated that animal activities cost U.S. medical schools

approximately $17.6 million for increased security, insurance, and compliance over the last 5 years; as reported in Washington Fax, July

23, 1990.

 of Assessment op. cit., footnote 12, pp. 85-96. Also see Philip H. “Federal Impediments Scientific Research”

Science, vol. 251, Feb. 8, 1991, p. 605. Abelson estimates that the Federal Government imposed more than 23 administrative reporting requirements

on universities during the 1980s.
178 . Federally Funded Research: Decisions for a Decade

Figure 6-5-Relative Sources of Funds for Research Facilities: Academic Capital

Projects Begun in 1986-89

28%

2 8

Private institutions

($2.1 billion)

11%

2%

22%

Public institutions

($4.2 billion)

Sources of funds

 Federal Government 

State/local government 

 Private donations I

NOTE: Percentages may not total 100 due to rounding.

Institutional funds

Debt financing

Other sources

8%

9%

SOURCE: National Science Foundation, Scientific and Engineering Research Facilities at Universities and Colleges:

1990, final report, NSF 90-318 (Washington, DC: 1990), chart 6.

operation and maintenance costs.32 In absolute research instruments (costing $10,000 to $1 million)

terms, Federal funds for new construction of re- in engineering, and in the agricultural, biological,

search space more than doubled over the 1986 to computer, environmental, and physical sciences.

1989 period, and the increase was seen primarily at Information is collected about both quantitative and

public institutions. qualitative changes in in-use instrumentation and

Equipment and Instrumentation equipment.

There is one comprehensive source of data on Results from these surveys show that there is

research equipment and instrumentation expendi- substantial turnover in the national stock of in-use

tures. The National Survey of Academic Research academic research equipment. About one out of

Instruments and Instrumentation Needs is a congres- every four systems in research use in 1982-83 was no

sionally mandated, triennial survey program to longer being used for research by 1985-86, and about

monitor trends in academic research. It is sponsored two out of five systems in research use in 1985-86

jointly by NIH and NSF, and has been completed had been acquired in the 3-year period since the

twice, first in 1983-84 and again in 1986-87.33 A baseline data were obtained.34 Computer science had

new survey is in progress. The survey collects data the most rapid rate of expansion in stock (up 138

about expenditures, funding, and use of major percent over the 1982 to 1986 period), with slow

 Over the period 1982 1988, the Federal of university

infrastructure through indirect cost recovery grew by over 70 percent in real terms. See ‘‘Enhancing Research and Expanding the Human Frontier,”

Budget of the United States Year U.S. Government Printing Office, pp. 61-62. document further

states that: “Each academic institution must provide a certification that its research facilities are adequate (to perform the research proposed) as a

condition of accepting research grants. . . . $12 billion of needed, but unfunded capital projects has not had an apparent effect on the ability of

universities to accept Federal research funds. ’ The $12 estimate comes from the 1990 Science Foundation survey of universities.

 Science Research Science/Engineering to SRS 

 DC: June 1988).
Chapter 6—Understanding Research Expenditures . 179

Photo credit: Jay Mangum Photography

Researcher uses a CT scanner. State-of-the-art

equipment often enables researchers to push the frontiers

of scientific knowledge.

growth in mechanical engineering (up 23 percent)

and materials science (22 percent). The data indicate

that Federal and non-Federal expenditures for academic research equipment increased from $393

million (1982 dollars) in 1980 to $704 million in

1987. 35 Yet the mean purchase price per system for

all in-use equipment in 1985-86 was $36,800,

basically unchanged from 1982-83 (up only 1

percent after inflation). Computer science was the

only field to show a substantial real change in the

mean price per unit of in-use instrumentation,

dropping 22 percent after adjusting for inflation,36

Federal involvement in funding academic research equipment declined somewhat from 1982-83

to 1985-86. Fifty-five percent of all systems in use

in 1986 were acquired either partly or entirely with

Federal funding support, down from 60 percent in

1982-83. Despite this relative decline, Federal

support for in-use research equipment increased 30

percent in real dollar terms, from $663 million in

1982-83 to $906 million in 1985-86.37 Data for

select Federal funding sources are displayed in table

6-1. Not unexpectedly, funding for equipment under

development, or considered state-of-the-art, grew at

higher rates than existing systems. Qualitative

upgrading (e.g., incremental improvement in the

power and capability of existing equipment), however, varies across fields, e.g., chemistry experienced more upgrading than agricultural sciences.

Despite pronounced increases and improvements

in equipment stocks in the 1980s, 36 percent of

department heads still describe their equipment as

inadequate (to conduct state-of-the-art research). In

general, the survey data reveal that equipment stocks

have been substantially replenished and refurbished

during the period of 1982 to 1986 in all of the fields

studied and in all types of institutions. There have

been substantially increased levels of support for

instrumentation from all sources, with most of this

increased support coming from the colleges and

universities themselves, as well as from businesses,

private donors, and State governments. In relative

terms, computer science was the greatest beneficiary

of the overall increase in instrumentation support,

particularly from Federal sources, with engineering

suffering the most. Among all Federal sources of

equipment support, NSF provided the largest share

(33 percent of the total) .38

Within the biological sciences, biochemistry

more than doubled its equipment stocks between

1984 and 1987, the fastest rate of growth of any

major biological field. There appears to be an

increasing need in the biological sciences for bigticket items costing over $50,000 (1984 and 1987

prices for some items are shown in table 6-2). The

percentage of department heads reporting equipment

in this range as being their top priority for more

Federal funding increased from 20 percent to 35

percent from 1984 to 1987.39

Expenditures for equipment were $200 million

(1988 dollars) in 1958, rose to $600 million in the

1960s, fell to below $400 million in the 1970s, and

 p. 17.

 universities, however, spend more on equipment than others. The top 20 research and development universities average of

$27.9 million of equipment in the 1985-86 academic year. The average for the next 154 institutions was $6.9 million. National Science

Foundation 1989, op. cit., footnote 19, pp. 130-133.

 Science Foundation, op. cit., footnote 33, p. 

 provided 46 percent of the funding for biological equipment use 1987; the National Institutes of Health 76

percent of the Federal shine. Ibid., pp. 59-64.

 National Institutes of Academic Research Equipment and Equipment Needs in the

Biological Sciences: 1984-1987 (Washington, DC: June 1989), pp. 8-1 through 8-10.
180 ● Federally Funded Research: Decisions for a Decade

Table 6-l—Selected Sources of Funding for 1985-86 National Stock of In-use Research Equipment and

Percent Change From 1982-83,a by Field (in millions of 1985-86 dollars)

Total State/

research university Business

Field equipment NSF NIH DOD DOE funds donations , .

Computer science . . . . . . . . . . . . . . . . . . $100 (85%) $26 (123%) $1 (UE) $20 (99%) <$1 (UE) $25 (38%) $20 (61%)

Engineering: . . . . . . . . . . . . . . . . . . . . . . 372 (34%) 38 (l%) 5 (UE) 59 (16%) 17 (15%) 135 (35940) 81 (47%)

Electrical . . . . . . . . . . . . . . . . . . . . . . . . 110 (59%) 11 (4%) 1 (UE) 23 (1 1%) 2 (UE) 30 (142%) 36 (187%)

Mechanical . . . . . . . . . . . . . . . . . . . . . 71 (320/o) 7 (-12%) <1 (UE) 16 (230/o) 1 (UE) 28 (66%) 13 (520/o)

Other . . . . . . . . . . . . . . . . . . . . ... , . . . 191 (23%) 20 (5%) 5 (UE) 20 (18%) 14 (45%) 78 (10%) 32 (43%)

Materials science . . . . . . . . . . . . . . . . . . . 44 (26%) 19 (33%) <1 (UE) 3 (UE) 4 (UE) 11 (23%) 2 (UE)

Physics/astronomy . . . . . . . . . . . . . . . . . . 221 (16%) 54 (l%) 1 (UE) 28 (13%) 29 (O%) 50 (88%) 14 (2%)

Chemistry . . . . . . . . . . . . . . . . . . . . . . . . . 322 (44%) 85 (22%) 32 (64%) .15 (54%) 17 (186%) 111 (38%) 28 (78%)

Environmental sciences. . . . . . . . . . . . . . 170 (47%) 30 (71%) 1 (UE) 10 (44%) 15 (70%) 56 (50%) 27 (39%)

Agricultural sciences . . . . . . . . . . . . . . . . 62 (61%) 4 (UE) 2 (UE) <1 (UE) 1 (UE) 39 (55%) 6 (UE)

Biological sciences: . . . . . . . . . . . . . . . . . 643 (48%) 51 (39%) 226 (44%) 7 (UE) 4 (UE) 247 (54%) 48 (53%)

in colleges/universities . . . . . . . . . . . . 283 (63%) 32 (26%) 79 (51%) 5 (UE) 2 (UE) 123 (89%) 19 (45%)

in medical schools . . . . . . . . . . . . . . . . 360 (389%) 19 (66%) 148 (41%) 2 (UE) 2 (UE) 124 (31%) 29 (59%)

apercent change estimates are adjusted for inflation.

KEY: NSF = National Science Foundation; NIH = National Institutes of Health; DOD_ U.S. Department of Defense; DOE= U.S. Department of Energy;

UE = unstable estimate: 1982-83 base is less than $4 million.

SOURCE: National Science Foundation, Academic Research Equipment in Selected Science/Engineering Fields: 1982-83 to 1985-86 (Washington, DC:June

1988), table A.

Table 6-2—Types and Expenditures for Most Needed Research Equipment:

National Estimates for the Biological Sciences, 1984 and 1987a

Percent of requests Median cost per system

Types of system 1984 1987 1984 1987

Preparative (e.g., centrifuges,

scintillation counters, incubators) . . . 33%

Protein/DNA sequencers/

synthesizers . . . . . . . . . . . . . . . . . . . . . . 11

Electron microscopy. . . . . . . . . . . . . . . . . 12

Light microscopy . . . . . . . . . . . . . . . . . . . . 4

High-pressure liquid chromatography . 9

Cell sorters/counters . . . . . . . . . . . . . . . . . 4

MNR spectroscopy . . . . . . . . . . . . . . . . . . 4

General spectroscopy . . . . . . . . . . . . . . . 9

Mass spectroscopy . . . . . . . . . . . . . . . . . . 2

image analyzers . . . . . . . . . . . . . . . . . . . . 3

X-ray (other than imaging) . . . . . . . . . . . 1

Computers . . . . . . . . . . . . . . . . . . . . . . . . . . 5

25%

14

6

3

12

6

4

5

4

6

1

5

$30,000

75,000

150,000

30,000

27,000

150,000

250,000

25,000

125,000

40,000

100,000

50,000

$35,000

95,000

180,000

35,000

25,000

100,000

225,000

30,000

100,000

100,000

200,000

45,000

Other . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 9 30,000 45,000

aFindings are based on department chairs’ listings of up to three “topmost priorities” in research in-StmrnentS or

systems.

SOURCE: National Institutes of Health, Academic Research Equipment and Equipment Needs in the Biologk#

*“ences: 1984-87 (Washington, DC: June 1989).

were over $800 million in 1988. The Federal share capability have been decreasing. In addition, obsoof funding was 75 percent in 1958 and now stands at lescence time was 7 to 10 years in 1975, while the

about 60 Percent.w NSF found that instrumentation 1986 estimate was 3 to 5 years. And these instrucosts have increased (in real dollars) for state-of-the- ments require funds for maintenance and operation

art instruments, but costs for instruments of similar (about 4 percent of the purchase price) .41

40Governrnent.University-Mdus~ Research Roundtable, Op. cit., fOO~Ote 17, p. 17.

dlNafio~ Science Foundatio@ op. cit., footnote 19, pp. 130-133. The enhanced power or sophistication of anew instrument, say, anautomated DNA

sequencer, is seen by researchers as justifying its COS$ which is a relatively modest investment for sustaining, or perhaps embling for the first time, the

performance of frontier science. As Robert Borchers, associate dinxtor for computation at Lawrence Livermore National Laboratory, puts it: “We’re

scientists. When we hear about a faster machine, we’re interested. ” Quoted in Marcia Clemmit, ‘‘Livermore’s Purchase of Japanese Supercomputer Is

Blocked,” The Scientist, vol. 4, No. 23, Nov. 26, 1990, p. 3.
Chapter 6—Understanding Research Expenditures ● 181

Figure 6-6—Direct Costs Awarded for NIH Research Projects, Research Centers, and Other Research Grants:

Fiscal Years 1979-88

3

2.5

2

1.5

1

0.5

0

Current dollars

Dollars (in billions)

.

/

---- / ---------- -- ------------------------

1 1 I I 1 I 1 I

Constant dollarsa

1979 dollars (in billions)

3

3~3

2.5 2.5

2 2

i I

2.5

2

1.5 1.5- -

1 . 5

1 1 - 1

0.5 0.5- -

0.5

- - - - -- - -- ___ -_ - . _ . _- ____ --- -- - --------------

0 0 1 I I I I I 1 I 0

79 81 83 85 87 88 79 81 8 3 85 87 8 8

Year Year

— Project s

— Center s

— Project s— Center s

‘ --- Othe r ‘

- -- Othe r

aBaSed on the biomedical R&D price index, fiscal year 1979-100.

KEY: NIH=National Institutes of Health.

SOURCE: National Institutes of Health, Extramural Trends: FY 1979-1988 (Washington, DC: June 1989), p. 18.

University researchers echo the same concerns

about research expenditures for equipment: 1) instrumentation is becoming obsolete at a faster rate;

2) many are buying more computer equipment or

using a universitywide computer system (for example, the University of Michigan spends over $160

million every year on information systems—10

percent of its operating expenses); 3) most research

fields are becoming increasingly dependent on

advances in research equipment; 4) support personnel are required in increasing numbers to operate

equipment; and 5) maintaining research equipment

for a laboratory takes a large amount of researcher

time. 42

Research Expenditures at the

National Institutes of Health

The amount of dollars awarded for direct costs of

individual investigator-initiated, or RO1, research at

NIH has risen steadily since 1979. The direct costs

in current dollars awarded that year increased by

$1.5 billion, or 155 percent, to a fiscal year 1988

all-time high of $2.6 billion (see figure 6-6). In

constant dollars, growth was $408 million, or a net

increase of 41 percent. According to NIH’s Division

of Research Grants, the proportion of indirect costs

to total expenditures has increased in 8 of the years

from 1979 to 1989, ranging from just under 28 to

over 31 percent. Nevertheless, when examining

dollars awarded, and controlling for inflation, indirect costs are rising at a faster rate than direct costs

(20 percent higher).

Personnel expenditures accounted for 65 percent

of the $3.2 billion direct costs budgeted for fiscal

year 1988 RO1 research. The next largest category

of direct costs was supplies, at 12.4 percent. The

equipment category has been stable at 5.2 to 5.6

percent since 1984 (see table 6-3). Noncompeting

and competing continuation grants have higher

expenditures for personnel than do new grants (68

and 64 percent, respectively, v. 51 percent for new

oZFrom OTA hterviews at Universiv of Michigan and Stanford University, July-August 1990.
182 . Federally Funded Research: Decisions for a Decade

Table 6-3-Extramural Direct Costs by Budget Category at NIH: Fiscal Years 1979-88

Total direct All other

costs (including

(in billions Personnel Equipment Supplies hospitalization,

Fiscal year of dollars) (in percent) (in percent) (in percent) in percent)

1979 . . . . . . . . . $1.4 66.80/0 6.2% 13.OYO 14.00/0

1980 . . . . . . . . . 1.5 67.8 5.2 13.2 13.8

1981 . . . . . . . . . 1.6 68.4 4.5 13.4 13.7

1982 . . . . . . . . . 1.7 69.5 4.4 12.7 13.4

1983 . . . . . . . . . 1.9 69.4 4.8 12.7 13.1

1984 . . . . . . . . . 2.1 68.5 5.2 13.1 13.2

1985 . . . . . . . . . 2.4 66.5 5.9 12.7 14.9

1986 . . . . . . . . . 2.6 67.8 5.3 11.6 15.3

1987 . . . . . . . . . 3.0 65.8 5.8 12.0 16.3

1988 . . . . . . . . . 3.2 64.9 5.6 12.4 17.1

SOURCE: National Institutes of Health, Extramural Trends: FY 1979-1988 (Washington, DC: June 1989), p. 61.

grants). Conversely, equipment expenditures are

higher for new grants than for continuations.

In recent years, NIH has conducted “downward

negotiations’ of noncompeting grant continuations,

whereby the amount awarded in years after the initial

award will be less than the original commitment.43

This practice has resulted in uncertainty for the

investigator as to what funding will be available

from year to year. 44 It has alSO led to congressional

criticism of financial planning at NIH (see box 6-A).

Calculating the Biomedical Research and

Development Price Index

The Biomedical Research and Development Price

Index (BRDPI) is a specialized price index calculated since 1979 by the Bureau of Economic

Analysis of the Department of Commerce for NIH.

The BRDPI is a freed-weight index designed to

reflect price changes of the cost to NIH of supporting

biomedical R&D. The index is calculated for fiscal

years and is currently based on patterns of NIH

obligations for fiscal year 198845 (see figure 6-7).

The BRDPI is comprised of three major subindices—intramural activities, extramural activities,

and extramural nonacademic activities. Within each

activity, price indices are available for major components such as personnel, supplies, and equipment.

Intramural activities comprise all activities performed by NIH including R&D, as well as support

Figure 6-7—Biomedical Research and Development

Price Index: Fiscal Years 1979-89

Fiscal year 1979 ■ 100

200 I

I

191.7

/

175-

150-

125-

100 1 I 1 1 1 1 1 1 I

200

175

150

125

100

79 80 81 82 83 84 85 86 87 88* 89*

Year

aProjected.

SOURCE: National Institutes of Health, NIHDataBook: 1989( Washington,

DC: December 1989), inside back cover.

functions for intramural and extramural research.

Intramural activities are grouped into 27 categories

that correspond to available price measures.

Figure 6-8 shows a comparison of the aggregate

BRDPI with three other indices-the Consumer

Price Index, the Producer Price Index, and the Gross

National Product. The BRDPI has consistently

increased at a faster rate than the other three indices,

but has slowed in recent years. Table 6-4 displays the

BRDPI for the years 1979 through 1989 (fiscal year

1988 is the base). In 1989, the index is highest for

43MW~ c~smm “Issues Behind the Drop in the NIH Award Rate, “ ASMNews, vol. 56, September 1990, pp. 465-469. Late in 1990, the National

Institutes of Health began consideration of a plan whereby funding adjustments would be made prior to award and not through across-the-board cuts.

ah con~ast,tie Natio~ Science Fomdation awards a set amount for the grant over a multiyear period. Any cuts h the awards are across-tie-board,

leading to less uncertainty for the grantees. Uncertainty has two distinct but related repercussions for principal investigators: first their anxiety level

is raised regarding available monies for carrying out the next year’s work under a muhiyear grant, and second, their pl auning for needed personnel and

infrastructure must include several contingencies.

QsBmeau of Economic Analysis, op. cit., footnote 11, p. 1.
Chapter 6—Understanding Research Expenditures ● 183

Box 6-A—Financial Planning at NIH: A Congressional Mandate

During its review of the fiscal year 1991 budget for the National Institutes of Health (NIH), the House

Committee on Appropriations stated that:

. . . despite large increases in funding for the NIH during the last decade, the system of Federal support for the health

sciences is in crisis. While the Committee believes that this crisis has been overstated, it recognizes that problems

with low numbers of new grants, high levels of downward negotiation of grant awards, and the general lack of stability

in governme nt support for the biomedical sciences are critical problems which must be addressed if the vitality and

the morale of the research community are to be restored.1

Thus Congress, as it traditionally has done, gave NIH more than the Admini “ stration had requested for 1991,

boosting its appropriation to $8.3 million. But the House and Senate appropriations bills, using similar language,

". . . excoriated NIH administrators for inadequate financial planning. ” 2

In response to this congressional criticism, NIH drafted a plan for managing biomedical research costs. It was

circulated among scientific societies and university associations prior to a December 17, 1990, meeting of the NIH

Director’s Advisory Committee to register responses to the plan. Twenty witnesses testified. The NIH plan features

six goals:

1.

2.

3.

4.

5.

6.

Set 4 years as the average length of research project grant awards. Four years would allow NIH to provide

funding continuity to investigators while ensuring that a greater number of competing awards are made each

year.

Implement cost management measures so that the average cost of research project grants increases at the

same rate as the Biomedical Research and Development Price Index (BRDPI).

Stop using the concept of “approving” grant applications and adopt the “success rate’ method based on

the ratio of applications funded to applications reviewed-a method used by other Federal agencies.

Fund the number of research training grants recommended by the National Academy of Sciences to the

extent possible without jeopardizing NIH’s ability to provide stipend increases for research trainees.

Manage the growth of NIH research centers by controlling NIH appropriations for centers rather than by

establishing a ceiling on the number of centers,

Increase funding for other mechanisms to reflect inflation.3

As pointed out by John Briggs, deputy director of extramural research at NIH, who chaired the December 17

meeting:

NIH must remain flexible enough to allow biomedical science to respond to public health emergencies and

scientific opportunities, and this is reflected in the plan. In order for NIH to carry out its mission, each institute and

center must maintain a balanced research portfolio with an appropriate combination of research project grants, center

grants, training grants and other mechanisms. Cost management goals can and should be pursued through a

combination of peer review actions and administrative controls.4

Responses to the NIH plan have been mixed.s While increases in research project (ROl) grants are attributable

to increases in indirect costs, “. . . neither NIH officials nor researchers can pinpoint specific causes for the

increase." 6 Linking indirect costs to the BRDPI is more popular than taking a “. . . total-cost restraint approach, ” 7

lU.S. Conpess, House committee on Appmpriation.s, Departments of bbor, Health and Human Services, & Educ~”on, uti Relat~

Agencres Appropriurzon Bill, report 101-591, 10lst Cong. (WashingtO~ DC: U.S. (3OV ernrnent printing office, July 12, 1990), p. 51.

‘David L. Wheeler, ‘‘IWH Plan Would Trim hngth and Growth of Researc hGrants,” The Chronicle of Higher Education, vol. 37, Dec.

12, 1990, p. A21

3Bas~ on $ ‘NIH -emCO~sS With Six-Point Pm” Washington Fax, Mc. 18, 1990, p. 1.

41bid., pp. 1-2.

5Based on respmes to a survey conducted by Washington Fax and reported N. 14 and 17, 1990, 80 percent of M RXPOtim@ favor

a single, agencywide policy for implementing Congress’s &year plan for cost management at the National Institutes of Health.

61‘~~ut Cost he SOW of ~W~ ~l~s for RO1 Gmt~,’ t Wa)@yon Fm, ~. 7, 1990, p, 1.

7Wa~hingfon F=, D~, 17, 1990. Op~sition to swdy section consideration of kdirect COStS is IICWly un~~0u5. SIX “~~ F~us

on Quality Over Numbers,” Washington Fax, Dec. 19, 1990, p. 1.

Continued on next page
184 . Federally Funded Research: Decisions for a Decade

Box 6-A—Financial Planning at NIH: A Congressional Mandate--Continued

Developing a financial management plan is prudent for any agency; adopting and implementing one is more

difficult. 8 At a December 18, 1990, meeting the NIH Director ‘s Advisory Committee agreed that stabilizing NIH

at 6,000 new and 24,000 total grants should be a ". . . target rather than a mandate. ” Some have noted that inflation

would reduce NIH’s capacity to pay for 6,000 new grants each year, and that aiming for such a total would only

perpetuate the current budget problems, a”.. . dynamic of fat years and lean years.”9

The committee’s draft document will be reviewed by various NIH boards and councils during the first 2 months

of 1991, prior to a presentation at congressional hearings in the spring. Many biomedical scientists fear that the grant

pool would not be sustainable without sacrifices m facilities, equipment, and training, and that ”... no sector of

biomedical science should be cannibalized to serve another sector.”10

Amidst the claims that Congress is micro managing by calling for expenditure containment at NIH, the

language of appropriations is unequivocal. If financial planning does not improve, more radical options exist,

including agency-specific limits on the indirect costs that the Federal Government will pay.ll This story will most

likely unfold well into the fiscal year 1992 appropriations process.

8~ ~ abwmc.e of a pcmnanmt National Institutca of Health dhtctor , developing a f~“ mana@mW plan was seen by mnny as

~C.As hzdd Rosehwalb, director of &leral relatlons for the National Association of State Unfveraitios and Land-Grant Colleges,

puts it: “It may be good politi it may be good ~, d it may be good ~es, tit it -lexl to poor scicnec. ” Quotul

in Whoekr, op. cit., footnote 2, p. A21.

9@o~ in David L, Wheeler, “Scie@ts’ ~‘ Prompt R@skma in NIHCost<utting Pi@” ThChrOticJe@Higkr*atin,

VOi. 37, No. 17, Jan. 9, 1991, p. A19. ‘f& dhector Of the N8t@d Institutes of Health’s @l’El) division of fhmeial

through mlnputer mode@ of the tmd&\ that ~codd aupport 6,000 new grants pa y- if its

v-t has form&

eongressionld appropriations immased inthe

range of 7 to 9 percent annualiy. But ‘ ‘ . . . not everyone thinka that it’s a good idea to sa an annual goal and give it pnwedewe ova NIH-hlitiilted

projeets, grants to large groups and huxitudonalceatas, and Other forms of research SUp~” Je&ey Mcrvis, ‘ ‘NM Debates Merit of Selthlg

crant~ “2% SCunn’sr, VOL 5, No. ~ Jan 21, 1991, p. 1. In this same article, acting NIH Director William Raub notes (p. 4) that “In

the course of rnaldng our annual budget rczpest to Congms, the use of the number of new and competing grants that it would find has been

a powerful tooI. Perhaps because they do not~tidetaila of our enmprise, the committees [that oversee NIH’s budget] have found

it very useful to think in terms of the numk of pieces of rewrreh that they arc funding. And that information is refkted in the level of new

grants.’ Also see Elizabeth Pennisi,’ ‘Budget Increase for NrH won’t Meet Expectations,’ ‘ The Sciem”st, vol. 5, No. 5, Mar. 4, 1991, pp. 1,6.

1~= Wahiflgton Fti, M,14, IW, p. 1. w fuller dex.riptiom of this mee@ ~wific pow of ~~ntio~ ~ ~‘C

Cullitow “Biomedical Funding: The Eternal ‘Crisis ’,” Science, voL 250, Dec. 21, 1990, pp. 1652-1653; and Pamela Zurez, “ResearehFunding:

NIH Airs Cost-containrne nt Pm’ Chm”cal & Engineering News, vol. 68, No. 52, Dec. 24, 1990, pp. 4-5. Clearly, the issue of numbas of

grants cannot be decoupled from the duration or avenge grant amount.

1 lFor ~ d~ussion of various OptiOm, W Barbara J, CuUitoU “NH Readies Plan for Cost Containment, ” Science, vol. 250, Nov. 30,

1990, pp. 1198-1199.

extramural activities, specifically nonpersonnel annual increase heading into the 1990s. Extramural

(supplies, travel, and consultants) and indirect costs.

More telling is table 6-5, which compares percent

changes from prior fiscal years for 1980 through

1989. This table shows general slowing of the

increase in all areas, but most significantly in

indirect costs for extramural activities.

Intramural expenditures have increased at a much

slower rate than extramural expenditures, due to the

modest increases in Federal employees’ pay.46

Increases in intramural expenditures appear to be

leveling off from the sharp increases of the early

1980s (see figure 6-9). The aggregate extramural

activities index has slowed to aboul a 5.5 percent

activities comprise R&D outside of NIH and financed by grants to universities and medical

schools. A sample of universities provides the data

for this index. Three subindices comprise this

category-salary and wages, fringe benefits, and

indirect costs.

Two sources are used to compute a wage and

salary index for each institution: the Report on

Medical Faculty Salaries, published by the American Association of Medical Colleges, and Academe:

The Annual Report on the Economic Status of the

Profession, published by the American Association

of University Professors. Salaries for medical school

46Ibid., p. 13. The proposed fiscal year 1992 budget calls for a 13-percent increase in ‘‘research management and support,’” which covers the costs

of administering the National Institutes of Health extramural research programs, especially the expenses of peer review panels (that have become ‘more

labor-intensive”). Reported in “Ways and Means,” The Chronicle of Higher Education, vol. 37, No. 23, Feb. 20, 1991, p. A25.
Chapter 6—Understanding Research Expenditures . 185

Figure 6-8—Comparison of BRDPI With Other Price

Indices: Fiscal Years 1980-89

Percent change from previous year

‘“~’o

1 5 L I 15

10 -

10

5 - 5

0 - 0

-5 ! 1 I 1 1 1 1 1 1 ! -5

80 81 82 83 84 85 86 87 88 89

Year

— BRDPI ~ CPI ~ PPI — G N P

KEY: BRDPl=Biomedical Research and Development Price Index;

CPl=Consumer Price Index; PPl=Procfucer price Index; GNp_Gross

National Product.

SOURCE: U.S. Department of Commerce, Bu(eau of Economic Analysis,

13i”omedical Research and Development Price Index: Report to

the National Institutes of Health (Washington, DC: Mar. 30,

1990), chart 2.

faculty tend to be high and might have a disproportionate effect on salary calculations; biological

scientists (not M.D.s) earn lower annual salaries (8

percent less) than scientists employed in most other

fields. 47

Indirect costs are calculated as a rate applied to

direct costs. The ‘‘quantity’ of indirect costs,

therefore, is virtually impossible to define. According to the Department of Commerce, if indirect costs

increase as a result of additional R&D, the increase

is not a price change. If no additional R&D is

performed, an increase in indirect costs is a price

change. The criterion used to evaluate a change in

the composition of indirect costs is whether or not

the change has an impact on the performance of

R&D. For example, if a university purchases a more

powerful central computer and the indirect costs rate

rises because that purchase is allowable as an

indirect cost, the performance of R&D is probably

enhanced. OTA finds this exception to indirect cost

increases problematic, since it is not well defined

and enhancement of the performance of research is

not considered in other categories of expenditure.

For calculation of the indirect cost index in the

BRDPI, an indirect cost rate index and a direct cost

Figure 6-9-NIH Biomedical Research and

Development Price Index: Fiscal Years 1979-89

Fiscal year 1988 = 100

120 -

1 2 0

110- -

1 1 0

100- -

1 0 0

9 0 -

-

9 0

- 8 0

7 0 -

-

7 0

6 0 -, - 6 0

5 0 ‘

-

5 0

4 0 1 ! I I 1 1 1 I 1 40

79 80 81 82 83 84 85 86 87 88 89

Year

— BRDP I

— Extramural academic

+ Intramural — Extra-nonacademic

KEY: NIH-National Institutes of Health.

SOURCE: U.S. Department of Commerce, Bureau of Economic Analysis,

Biomedical Research and Development Price Index: Repori to

the National Institutes of Health (Washington, DC: Mar. 30,

1990), chart 1.

index are computed. These two indices are then

multiplied together. The calculation of the BRDPI is

conducted every year with the base year scheduled

to be reset in 1992.

Research Expenditures at the National

Science Foundation

Data are available on research expenditures

funded by NSF up to fiscal year 1989. Expenditures

are reported for the conduct of research, which

includes basic, applied, and development; for R&D

facilities, which include land, buildings, and fixed

equipment; and for major equipment. The portion of

the R&D budget allocated to facilities has been at

less than 1 percent for the past 10 years, until the

addition of a facilities funding initiative in 1988.

Direct costs are available for personnel, R&D

facilities, equipment, and instrumentation. Other

direct costs are reported in the aggregate, but include

supplies, publications, consultants, computer services, subcontracts, travel, and fringe benefits. This

category accounts for over 27 percent of the budget.

At NSF, equipment has risen from 9 percent of the

R&D budget in 1981 to over 13 percent in 1989.

Personnel has accounted for about 40 percent of the

R&D budget over the last decade. Indirect costs have

dT~e nu~r of inStitUtion used to create the academic salary and wage price index represents 96 percent of total obligations. Each fitimtion’s

separate salary and wage index is multiplied by its weight of obligations derived fmm the National Institutes of Health (NIH) IMPAC file, which contains

data on all NIH awards for tiect and indirect costs. These weighted &ts me summed to create the Academic Salary and Wage Price Index. The source

used to create the fiirtge beneiit index is Academe. Again, a fringe benefit rate index is created for each research institution. See National Science

t?oundatio~ Profiles4iological Sciences: Human Resources and Funding (Washington DC: 1989), p. 9.
Table 6-4--Biomedical Research and Development Price Index: Fiscal Years 1979-89 (1988 = 100)

1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989

All performers . . . . . . . . . . . . . . . . . . . . . . 56.5 61.8 67.8 73.6 78.0 82.5 86.7 90.3 95.2 100.0 105.2

Intramural activities . . . . . . . . . . . . . . . . . 76.3 79.0 83.3 87.7 90.4 91.7 93.6 94.4 97.6 100.0 104.3

Personnel . . . . . . . . . . . . . . . . . . . . . . 66.4 71.0 76.9 82.1 86.4 89.3 92.6 93.5 97.7 100.0 104.6

Nonpersonnel . . . . . . . . . . . . . . . . . . . . 84.6 85.7 88,7 92.5 93.8 93.8 94.5 95.2 97.5 100.0 104.1

Research function ... ... , <...8..,. 76.7 79.9 84.6 89.0 91.4 92.7 94.2 94.8 97.7 100.0 104.5

Support function . . . . . . . . . . . . . . . . . . 75.5 77.0 80.5 85.1 88,2 89.7 92.3 93.7 97.3 100.0 103.9

Extramural activities . . . . . . . . . . . . . . . . 52.4 58.2 64.6 70.7 75.4 80.6 85.3 89.5 94.7 100.0 105.4

Academic grants and contracts . . . . . 51.5 57.4 63.7 69.8 74.7 80.1 85.0 89.3 94.7 100.0 105.5

Personnel . . . . . . . . . . . . . . . . . . . . 52.4 57.0 62.7 69.1 73.7 78.3 84.1 88.7 94.1 100.0 105.0

Nonpersonnel . . . . . . . . . . . . . . . . . . 62.3 71.2 79.3 83.6 86.2 89.0 90.2 92.2 96.1 100.0 106.2

Indirect costs ..,..... . . . . . . . . . . 44.3 50.5 56.8 63.5 70.3 78.2 83.6 88.7 94.9 100.0 106.0

Nonacademic grants and contracts . . 55.7 61.5 68.2 74.3 78.2 82.5 86.6 90.5 64.9 100.0 105.2

SOURCE: U.S. Department of Commerce, Bureau of Economic Analysis, Biomedica/ Research and Development Price Index: Report to the National Institutes of Health (Washington, DC: Mar. 30,

1990), table 1.

Table 6-5-Biomedical Research and Development Price Index: Percent Change From Prior Fiscal Year,

Fiscal Years 1980-89

1980 1981 1982 1983 1984 1985 1986 1987 1988 1989

Ail performers. . . . . . . . . . . . . . . . . . . . 9.4 9.7 8.6 6.0 5.8 5.1 4.2 5.4 5.0 5.2

Intramural activities . . . . . . . . . . . . . . . 3.5 5.4 5.3 3.1 1.4 2.1 0.9 3.4 2.5 4.3

Personnel ... , . . . . . . . . . . . . . . . . 6.9 8.3 6.8 5.2 3.4 3.7 1.0 4.5 2.4 4.6

Nonpersonnel. . . . . . . . . . . . . . . . . . 1.3 3.5 4.3 1.4 0.0 0.7 0.7 2.4 2.6 4.1

Research function . . . . . . . . . . . . . . 4.2 5.9 5.2 2.7 1.4 1.6 0.6 3.1 2.4 4.5

Support function . . . . . . . . . . . . . . . . 2.0 4.5 5.7 3.6 1.7 2.9 1.5 3.8 2.8 3.9

Extramural activities . . . . . . . . . . . . . . 11.1 11.0 9.4 6.6 6.9 5.8 4.9 5.8 5.6 5.4

Academic grants and contracts . . . 11.5 11.0 9.6 7.0 7.2 6.1 5.1 6.0 5.6 5.5

Personnel . . . . . . . . . . . . . . . . . . 8.8 10.0 10.2 6.7 6.2 7.4 5.5 6.1 6.3 5.0

Nonpersonnel . . . . . . . . . . . . . . . 14.3 11.4 5.4 3.1 3.2 1.3 2.2 4.2 4.1 6.2

Indirect costs . . . . . . . . . . . . . . . . 14.1 12.6 11.6 10.7 11.3 6.8 6.1 7.0 5.4 6.0

Nonacademic grants and contracts . 10.4 10.9 8.9 5.2 5.5 5.0 4.5 4.9 5.4 5.2

SOURCE: U.S. Department of Commeroe, Bureau of Economic Analysis, BiomedkalResearch and DevelopmentPnce Index: Report to the National institutes

of Health (Washington, DC: Mar. 30, 1990), table 2.
Chapter 6—Understanding Research Expenditures . 187

fluctuated around 25 percent over the past 10

years. 48 The Academic Research Facilities Modernization program, authorized in 1988 as part of NSF’s

5-year reauthorization, committed up to $80 million

in fiscal year 1989, but no funds were appropriated

for it in that year. Funds were finally obligated on

September 1, 1990.49

Trend data on research expenditures administered

by NSF are spotty. Expenditure data are readily

available but do not provide a sense of actual costs

incurred (or shared) by the researcher. While NSF

routinely collects aggregate data on R&D spending

and expenditures across the Federal agencies, its

own databases are not nearly as comprehensive as

those kept by NIH.

The Research Performer% Perspective

on Expenditures

The definition of ‘research performer’ has many

components. The most obvious is the researcher or

team in a university, industrial facility, or Federal

laboratory. Another level is the department or other

organizational unit within a university or laboratory.

At the most aggregated level are the laboratories and

universities themselves. Given the concentration of

research performance in universities, most expenditure data are based on this sector.

The Federal Government supplied $9.2 billion in

research funds to universities in 1990. Industry

supplied $1.1 billion and another $1.1 billion came

from nonprofit institutions. Between 1978 and 1988,

the average annual growth above inflation was 5.5

percent (see figure 6-10). Industry has provided most

of the increase in funds, since growth of industrial

finding for university R&D averaged 12 percent

above inflation per year during that time period.

Funding from nonprofit institutions increased annually by an average of 8.2 percent in real terms over

that period, and the Federal Government increased

its support of university R&D by an annual average

of 4.6 percent above inflation. (Figure 6-11 presents

Federal basic research by performer.)

Since the 1960s, the Federal research system has

changed in many ways, not the least of which is in

the nature of the research performer. For instance,

during the 1960s, a professor with two to six students

Figure 6-10-Growth in University and College R&D

Performance, by Source of Funds: Fiscal Years

1978-88 (based on constant dollars)

Average annual percent change

O 2 4 6 8 10 12 14

Total

Federal

Non-Federal

Industry

Institutional

funds

State & local

governments a

Other funds

I I I I I

O 2 4 6 8 10 12 14

 or government appropriations that

universities use at their discretion for R&D.

NOTE: Figures were converted to constant 1982 dollars using the GNP

Implicit Price Deflator.

SOURCE: National Science Foundationp National Resources; 1990, final report, NSF 90-316 (Washington DC:

1990), chart 13.

in one discipline at a major research university was

the most prevalent production unit of research. In the

1990s, many other types of research units exist, in

particular much larger research groups with many

graduate students, nontenure track researchers, postdoctoral fellows, and technicians under one principal investigator. The rise of centers and university

research institutes now augments the traditional

array of disciplinary “departments” (see chapter 7).

Similarly, the number of universities and Federal

laboratories that conduct research has grown, expanding the group of researchers that pursue specialized forms of inquiry in the research system. These

changes have occurred primarily to accommodate

 Science budget office, 

 the 78 research institutions awarded a total of $39 Program. 

Constance “Facilities Awards,” Science, vol. 251, Feb. 8, 1991, p. 622.

292-863 0 - 91 - 7
788 . Federally Funded Research: Decisions for a Decade

Figure 6-n-Federally Funded Basic Research, by

Performer: Fiscal Years 1969-90

(in billions of 1982 dollars)

1982 dollars (in billions)

5

4

3

2

1

0

1969 1974 1979 1984 1989

— Federa l + Industry * Universities

Government and colleges

-9- Nonprofits * FFRDCs ~ Other

KEY: FFRDCs include all Federally Funded Research and Development

Centers that are not administered by the Federal Government. Other

includes Federal funds distributed to State and local governments

and foreign performers.

NOTE: Figures were converted to constant 1982 dollars using the GNP

Implicit Price Deflator. 1990 figures are estimates.

SOURCES: National Science Foundation, FederiilFunds forReseamhartd

Development, Detailed Historical Tables: Fiscal Years 1955-

1990 (Washington, DC: 1990), table 27; and National Science

Foundation, Selected Data on Federal Funds for Research

and Development: Fiscai Years 1989, 1990 and 1991 (Washington, DC: December 1990), table 1.

growth in the system.50 The annual rate of growth for

doctoral scientists and engineers employed in institutions of higher education from 1977 to 1987 was

just under 3 percent.51 Some universities are feeling

the strain felt by the scientific community, claiming

that capital needed to fund renovations and new

construction, equipment, administration, and personnel are rarely fully recovered through Federal

funds.

Components of Research Expenditures at

Universities

To document the effect of the research economy

on changing university and laboratory structures,

and to complement the Federal perspective presented above, OTA visited a public and a private

research university---the University of Michigan

(UofM) and Stanford University (SU) (see box 6-B).

In addition, OTA visited at least one laboratory for

each of the five major research agencies with

intramural laboratories. However, expenditure data

are scarce at the Federal laboratories and not

uniformly collected. This section, therefore, discusses the performer’s perspective on expenditures

(with details derived from the two universities), and

then outlines other issues that also influence spending in the conduct of research.

Both UofM and SU show evidence of robust

research organizations, with excellent human resources, facilities, and financial support. This is as it

should be in a top-ranked university with a long

history of success (see box 6-C). Nevertheless, there

was evidence of stress in the research environment

at both institutions, although it was unclear whether

this was a new phenomenon. Researchers said they

were “running harder just to stay in place.”

University administrators wondered whether their

institution could continue to expect resources to

flow from the Federal Government, student tuition,

and State and private sources.52 Graduate students

worried about whether their careers could ever be

like those of their mentors.

Salaries

University personnel spoke of the rising competition for faculty with other sectors of the economy,

and noted that faculty salaries have been rising

significantly over inflation during the last decade. In

1988 dollars, the average salary and benefits for a

full-time equivalent principal investigator in the

natural sciences and engineering increased from

$59,000 in 1981 to $70,000 in 1988. Before the

1980s, growth occurred much more slowly from

$51,000 in 1958 (1988 dollars) to over $60,000 in

~see Roga L. Geiger, “The American University and Researc~” The Academic Research Enterprise Within the Industrialized Nations:

Comparative Perspectives, Government-University-Industry Research Roundtable (cd.) (Washington DC: National Academy Press, 1990), pp. 15-35.

slNatio~l Science Fowdation, Science and Engineen”ng Personnel: A National Overview, SPCCkd report (wm~to~DC: 1990).

SZNSO .S= Susm Tiff~ “Hind Times on the Old Quad,” Time, Oct. 29, 1990, p. 92.
Chapter 6—Understanding Research Expenditures . 189

Box 6-B--OTA Interviews at Two Universities

To explore performer perspectives on research expenditures and on Federal research initiatives, OTA chose

one public and one private research university for indepth study: the University of Michigan and Stanford

University, OTA purposely selected two research institutions perennially in the top 10 of those receiving Federal

research dollars, because of the breadth of research performed on campus, Also, the problems found at these

universities in setting expenditures and expanding flexibility are thought to be indicative of problems on other

research-intensive campuses.

In addition to receiving financial and personnel data from each university, OTA interviewed members of the

administration, faculty, and graduate student population on campus. The interviews centered on two themes: 1)

research expenditures, and 2) flexibility of the university and its departments to adapt to the changing Federal

funding environment. Other issues discussed included the status of nontenure track faculty, hiring projections for

individual departments, tenure promotion standards, and the graduate student perspective on careers in different

fields.

The interviews sampled the range of personnel on campus. For example, at Stanford, OTA interviewed the

president (Donald Kennedy), the dean and associate dean of research, two department chairs, three professors, three

associate professors, one research associate (nontenure track), one postdoctoral fellow, two graduate students, the

director and other members of the Sponsored Projects Office, the assistant controller, specialists in the Office of

the Budget and the Office of Technology Licensing, and the director of the Stanford Synchrotrons Radiation

Laboratory. The interviewees were selected from a number of disciplines, including the physical sciences,

engineering, medicine, and the social sciences. At Michigan, a similar set of interviews was conducted.

OTA summarized the findings at each university and distributed the summaries to the campus hosts for

comment. Select findings are used throughout this report, especially in this chapter,

SOURCE: OTA interviews, July-August 1990.

the early 1970s, but salaries receded slightly in the classroom buildings falter and break down. 55 A 1989

late 1970s to $59,000 in 198153 (see figure 6-12). Coopers and Lybrand study found that:

Universities are encouraged by faculty attempts to ●

leverage their time with the help of postdoctoral

fellows, nontenure track researchers, and graduate

students who are paid modest salaries. Because of

the shortage of faculty positions for the numbers of

graduate students produced, young Ph.D.s have been

willing to take these positions in order to remain

active researchers. This availability of “cheap

labor’ is seen by many senior researchers as the only

way they can make ends meet in competing for

grants. 54 ●

Academic Facilities

Academic administrators claim that with growing

frequency aging utility systems in laboratories and

Since 1950, the facility space in colleges and

universities has quintupled, representing some

3 billion square feet of classrooms, libraries,

dormitories, offices, laboratories, and other

space. Not all of this space was built to last. In

particular, during the 1960s, many suboptimal

buildings were erected, in the rush to meet the

demand from the “baby boom” generation

entering college.

The capital renewal and replacement needs of

U.S. colleges and universities are roughly $60

billion, of which slightly over $20 billion is

‘‘urgent’ —requiring attention within the next

several years. Only $7.2 billion of the urgent

category was targeted to repair facilities in the

53 See Gov emment-University-Industry Research Roundtable, op. cit., footnote 17, p. 2-34.

~Labor economist ~Fechter (executive director, Offke of Scientific and Engineering Persomel, National Res=ch COWCfl, prso~

communicatio~ Nov. 15, 1990) writes: “. . . persomel costs constitute roughly45 percent of total costs and . . . this percentage has remained reasombly

stable over time. Given that salaries of faculty (i.e., principal investigators) have b-n rising during the 1980s, this suggests that the staffing pattern of

research projects has been changing, with the input of PIs decreasing relative to . . . other, less expensive resources. There is some evidence to support

this hypothesis in the report of GUIRR . . . [that] finds in academia an increasing ratio of nonfacuky to faculty.” See also ibid.

ssK~en Gmssmuck, $ ‘Colleges Scramble for Money to Reduce Huge Maintenance Backlog, Estimated to Exceed $70 Billiou New Federal Help Seen

Unlikely,” The Chronicle of Z-Zigher Education, vol. 37, Oct. 10, 1990, pp. Al, A34.
790 ● Federally Funded Research: Decisions for a Decade

Box 6-C—Federal Funding at Two Universities

The University of Michigan (UofM) and Stanford University are in the top 10 universities receiving Federal

funds. UofM was ranked fifth and Stanford was second in fiscal year 1988. Over the past decade, both universities

have been the recipients of large (real) increases in total funds and Federal research dollars. For example, in constant

(1980) dollars, UofM total revenues have risen from $600 million in 1979-80 to nearly $1 billion in 1988-89 ($1.5

billion in current dollars).

From 1979 to 1989, Federal research funds for UofM rose from $117 million (17 percent of total revenues)

to $188 million (13 percent of total revenues) in constant 1986 dollars. In the period from 1976 to 1986 at Stanford,

the total Federal R&D obligations to the university rose from $122 million to $194 million (1986 constant dollars).

In addition, from 1973 to 1986, the funds available to principal investigators (PIs) to spend directly on research

activities grew after inflation at an average annual rate of 2.6 percent.

The Department of Health and Human Services (HI-IS) is the largest Federal funder of research and

development (R&D) at both universities ($124 million at UofM and $170 million at Stanford in fiscal year 1989).

At UofM, HHS is followed by the National Science Foundation (NSF-$24 million), the National Aeronautics and

Space Administration (NASA--$1 3 million), the Department of Defense (DOD--$1O million), and the Department

of Energy (DOE-$9 million) in fiscal year 1989. At Stanford, after HHS comes NASA ($50 million), DOD ($45

million), DOE ($45 million), NSF ($30 million), and the Department of Education (ED--$1O million) in fiscal year

1989. From 1973 to 1985, Stanford’s share of total Federal R&D funding (across all agencies) has remained around

2.7 percent.

Proposal Success Rates

Stanford and UofM, like most top 10 universities, have a higher proportion of awards per proposals submitted

than the average for other research universities. For instance, although UofM does not track its proposals directly,

it estimates that two out of three proposals were awarded funds. At roughly two-thirds, its ‘‘proposal success rate’

is at least twice the national average of 20 to 35 percent for all proposals (with the exact percentage dependent on

the agency).

Roughly the same proposal success rate is found at Stanford, although they distinguish between “new

proposals’ and ‘renewals. ‘‘ For new proposals in 1989-90 (453 proposals sampled), DOD funded 35 percent; NSF,

36 percent; ED, 37 percent; DOE, 41 percent; HHS, 52 percent; NASA, 52 percent; and 57 percent averaged for

all other agencies. Renewals had much higher success rates (470 proposals sampled): HHS funded 67 percent; DOD,

75 percent; NASA, 80 percent; DOE, 81 percent; NSF, 81 percent; ED, 100 percent; and 66 percent averaged for

all other agencies.

Indirect Costs

At UofM, growth in the indirect cost expenditures for the university as a whole grew at an annual compound

rate of 10.3 percent from 1979-80 to 1988-89. However, indirect costs for organized research did not rise as

quickly-at a compound rate of 9.1 percent from 1980-81 to 1988-89. Although the cost of maintaining buildings,

equipment, and plant operation for organized research were higher than for other buildings at the university, student

services, libraries, and sponsored project research expenditures were lower.

In the early 1980s, the UofM negotiated indirect cost rate (ICR) for research underestimated the “true” rate

by as much as 20 points in 1 year (1984-85). (Note that the cog nizant agency for UofM is HHS, which traditionally

allows lower ICRs than DOD.) At present, the negotiated rate of 58 percent only slightly underestimates the ‘ ‘true’

rate of 60 percent, even though a 58 percent ICR is the highest of all public universities.

Stanford’s ICR was raised from 69 percent to 74 percent in 1987. However, this rate is currently under

investigation by the Office of Naval Research (ONR) and the House Committee on Energy and Commerce. 1

Although Stanford’s ICR is high, it is comparable to rates from other private research institutions, including Cornell

University which has an ICR of 74 percent and Yale with an ICR of 72 percent.

In response to complaints about the high ICR and the rising tuition, expenditures incurred by the 1989 I.oma

Prieta earthquake, and the lower than anticipated federally sponsored research, Stanford has embarked on a

cost-cutting campaign. It will cut $22 million over 18 months out of a $175 million administrative operating budget.

1See Marcia Barinaga, “Stanford Sails Into a Storm,” Science, vol. 250, Dec. 21, 1990, p. 1651; “Government Inquiry,” Stanford

Observer, November-December 1990, pp. 1, 13; and Marcia Barinaga, "John Dingell Takes on Stanford" Science, vol. 251, Feb. 15, 1991, pp.

734-737.
Chapter 6—Understanding Research Expenditures . 191

Salaries

Salaries at both UofM and Stanford have risen above inflation over the past two decades (exact figures were

not made available to OTA). Both institutions state that higher salaries are required to attract faculty. Experiments

with congressionally imposed (now rescinded) salary caps at NSF and the National Institutes of Health, which

provided upper limits on annual investigator salary rates charged to research grants (even though only a few months

of support maybe sought in the grant), affected both universities. (Note that faculty were not required to reduce their

salaries as the university made up the difference.) For example, if every faculty member at UofM had an NSF grant,

then one-quarter of the faculty would have been affected by the $95,000 salary cap.

Facilities

Over the last 15 years, UofM has completed nearly $1 billion in new construction and major renovation. The

amount of building space on campus totals over 22 million square feet. Many of the structures date to before 1950,

and no building is ‘‘temporary. Also, the university has demolished very few permanent, older buildings. Since

the 1960s, Federal and State funds have been limited for facilities, and expansion has occurred slowly at UofM

compared with other universities around the country.2 Recently the university has been outfitting buildings with

energy-efficient equipment, such as new thermal windows. Also, environmental regulations have required some

improvements. For instance, a large effort to install or replace fume hoods is under way.

At Stanford, an ambitious new construction project—the Near West Campus---has begun. There are plans for

at least five new buildings, providing primarily state-of-the-art laboratory and office space. Most of the funds for

construction are from private sources, but if measures are not taken (such as a successful cost-cutting campaign,

or an adjustment of the ICR by ONR), Stanford’s indirect cost rate could rise to account for the depreciation of the

new buildings.

Both Stanford and UofM question their abilities to meet their perceived need for new and renovated buildings.

Each would like to see an expanded Federal facilities program for academic research.

Projections for Future Federal Support

UofM and Stanford project an overall slowing of growth in Federal R&D support. Whereas both universities

had come to expect a 10 to 15 percent increase per year during most of the 1980s, the increase in Federal funds at

Stanford in 1989, for example, was 9 percent. University personnel forecast that similar limited growth will continue

into the 1990s. Adjustments will have to be made on both campuses to accommodate slowed growth in Federal

funding.

2Note that the university system in Michigan expanded to other campuses in Michigan during the college boom in the 1960s and 1970s.

Most of these satellite campuses have since closed.

Nation’s major research universities. Most of However, the picture is not as clear as the above

these needs, therefore, exist within other aca- data would suggest. When asked by NSF if their

demic sectors, including liberal arts and com- facilities are poor, fair, good, or excellent, a majority

munity colleges.56 of the research administrators and deans at the top 50

research universities replied that their facilities were

Many claim that facility reinvestment has not kept “good to excellent,” ‘whereas a majority of the

research administrators and deans in the schools

pace with growing needs, and Coopers and Lybrand below the top 50 estimated that their facilities were

estimate that for every dollar spent on maintenance “fair to poor. ” The average top 50 university will

and replacement, $4 are deferred. They further spend $1 to $2 million or more on facilities each

estimate that current costs to replace a laboratory are year, while the schools below the top 50 will most

roughly $200 per square foot, while classrooms often spend less than $1 million. For public universirequire less than $100 per square foot. ties, 50 to 60 percent of these funds come from the

56The Decaying Ame~”can Campus:A Ticb”ng Time Bomb is a joint report of the Association of Physical plant Admkdstito~ (APPA) of Universities

and Colleges and the National Association of College and University Business Officers in cooperation with Coopers and Lybrand, authored by Sean

C. Rush and Sandra L. Johnson, APPA (Alexandria, VA, 1989). APPA’s most recent ‘ ‘defemd maintenance” cost estimate is $70 billion. Also in 1990,

42 percent of college and university presidents surveyed by the American Council on Education called deferred maintenance a key campus issue for the

next 5 years, up from 14 percent in 1989. See ibid., p. A34.
192 ● Federally Funded Research: Decisions for a Decade

Figure 6-12—Average Salary and Benefits Paid

Academic Ph.D.s in Natural Sciences and

Engineering: 1958-88

(in thousands of 1988 dollars)

1988 dollars (in thousands)

60

40

20

0

60

40

20

0

1958 1963 1968 1973 1978 1983 1988

NOTE: Constant dollars were calculated using the GNP Implicit Price

Deflator.

DEFINITION OF TERMS: Academic in the natural sciences and

engineering include academic employees who have been awarded

the Ph.D. degree in the following fields: life sciences, including

agricultural, biological, medical, and otherhealth sciences; physical

sciences, including astronomy, chemistry, and physics; 

 including aeronautical and astronautical, chemical, civil, 

 and mechanical engineering; environmental sciences, including oceanography, and atmospheric and earth sciences; and

mathematics and computer science, including all fields of mathematics and computer-related sciences. Compensation includes

salaries and fringe benefits, including insurance and retirement

contributions.

DATA: National Science Foundation, Division of Policy Research and

Analysis. Database: Some of the data in this database

are estimates, incorporated where there are within

data series orgaps in data collection. Primary data source: National

Science Foundation, Division of Science Resources Studies,

Survey of Scientific and Engineering Expenditures at Universities

and Colleges; National Institutes of Health; American Association of

University Professors; National Association of State Universities

and Land-Grant Colleges.

SOURCE: Government-University-Industry Research Science and Technology the Academic Enterprise: Status,

Trends (Washington, DC: National Academy Press,

1989), figure 2-47.

Photo credit: Stanford University

Research universities support many facilities devoted in

part to research, such as the Seeley C. Mudd Chemistry

Building at Stanford University pictured here.

States and 30 percent from bond issues. For private

universities, roughly one-third comes from the

Federal Government, while another one-third is

from donations.57

The crux of the facilities problem is that academic

centers can always use new or augmented buildings,

but how much is enough? One method to judge

would be based on the research fostered by each new

facility. Unfortunately, there is no acceptable

method to measure the quality or quantity of

increased research capabilities or of “missed”

research opportunities. Even though “need” may

not be quantified in the different sectors of the

research enterprise, a demand certainly exists. For

example, when NSF solicited proposals for a $20

million program in 1989 to address facilities needs,

it received over 400 proposals totaling $300 million

in requests.58

Historically, the Federal Government has never

been the primary source of funding for academic

facilities, conceding support to private donors,

States, and localities. Indeed, the proportion of

Federal monies out of all the monies spent on

academic facilities has never topped one-third. Now

it is less than 10 percent.59

The issues surrounding the research infrastructure

are complicated. There is a need for improvement—

as evidenced in many research environments-but

 A of Proposals to Meet Research Facilities Needs on 

DC: Congressional Research Service, 1987).

 requires a 50-50 match for requests ranging from to $7 

President’s Science Advisor, estimate the price of academic facilities modernization to $7 billion. See Jeffrey ‘ ‘Institutions Respond in Large

Numbers to Tiny Facilities Program at NIH, NSF,” The Scientist, vol. 4, No. 8, Apr. 16, 1990, p. 2.

 cit., 
Chapter 6—Understanding Research Expenditures ● 193

the need is very hard to quantify or assess. In

addition, the extent of the Federal role—should there

be a Federal facilities program?-is in question. Do

deteriorating facilities affect the quality of research

underwritten by the Federal Government? Academic

earmarks for facilities continue to play an ad hoc role

that unfortunately fails to address the facilities

renovation issue directly or systematically (as a

formal Federal facilities program perhaps would, see

box 6-D).

Indirect Costs

One of the most worrisome issues on many

research campuses is the high cost of ‘overhead’ or

indirect costs. As part of the ‘‘fill cost recovery for

research” doctrine in the Federal Government,

universities charge the Federal Government for

facilities maintenance, administrative expenses, and

other expenditures that ensure their capacity as

research performers but cannot be directly associated with specific projects. The standard procedure

is for the university to negotiate a single rate that will

be charged to all Federal grants with the cognizant

Federal agency (either the contract audit agency of

the Department of Defense or the Department of

Health and Human Services, depending on the

institution). 60 For example, in 1990, Stanford University charged 74 percent in indirect costs to every

grant, so a grant of $100,000 in direct costs might be

submitted at a total cost of up to $174,000.61

(Overhead can be computed on only certain direct

costs, resulting in a total charge to the government

of less than $174,000.) Indirect cost rates have

evolved over the last 30 to 40 years, and clearly

reflect institutional idiosyncrasies and practices of

the cognizant agency.

Over the past three decades, indirect costs have

claimed a much larger proportion of academic R&D

finding. In 1958, federally reimbursed indirect costs

comprised 10 to 15 percent of academic R&D

finding. By 1988, that share had risen to roughly 25

percent. 62 In addition, some agencies allow more in

indirect costs. For example, in 1988, the indirect cost

as a percent of the total R&D expenditures at NIH

was 30 percent, whereas it was less than 24 percent

for NSF (a proportion unchanged since the mid1980s).63 Medical schools typically have high indirect cost rates, both because of the extra facilities

expenditures associated with their activities and

because they tend to be associated with the research

universities that have high indirect cost rates.64

The indirect cost rate at the University of Michigan is 58 percent, which is high for a public

university. Because the State assumes part of the

cost of maintaining its universities, the indirect cost

rates are usually lower than at private universities. In

addition, at State universities, indirect cost monies

are often transferred directly to State coffers, so that

the university has little incentive to pursue a higher

indirect cost rate or to employ the administrative

personnel needed to comply with Federal audit

requests to justify new rates.

Many university administrators report that the

monies received in indirect costs do not cover their

expenditures. They worry about further erosion of

the indirect cost base, due to the perception in many

quarters of high rates of overhead and resistance to

the indirect cost increases experienced by many

universities over the last decade.65 The chief recommendation offered by a 1988 Association of American Universities report was that the indirect cost rate

@&sociation of Americ~ Universities, Op. Cit., fOOtIIOte 14, p. 7.

GIAS a res~t of the Defense Contractor’s Audit Agency’s (DCAA) ongoing investigation of Stanford’s illdi.RXl COSt rote, tltl intefi mte of 70P~nt

has been negotiated as of Feb. 1, 1991. William Massy, Stanford University, personal communicatio~ February 1991. DCAA’S prelimimuy analysis

indicates that Stanfor~ which requested a new indirect cost rate of 78 percent could justify only a 62 percent rate. Stanford has yet to rebut this claim.

See Marcia Barinaga, “Was Paul Biddle lbo lbugh on Stanford?” Science, vol. 251, Jan. 11, 1991, p. 157; Kenneth J. Cooper, “Stanford Will Try

to Explain Price of Knowledge,’ The Washington Post, Mar. 13, 1991, p. A15; and Kemeth J. Cooper, “Panel Looks for Liability in Stanford Billings

Case,” The Washington Post, Mar. 15, 1991, p. A21.

GZNatio~ Science Foundation, op. cit., fOOtnote 19, p. 121.

63rbid., p. 142; ~d AssWiation of ~eficmufiversities, op. cite, footnote 14. For e~plq res~ctingpqrment for overhead expenses tO IApCTCeIlt

on research projects tided by the Department of Agriculture has aroused concern that universities would have to decline such awards since they could

not afford to do the projects. Fears that such an across-the-bored ceiling could be instituted at other agencies continue to mount. See Colleen Cordes,

“Universities Fear That U.S. Will Limit Payments for Overhead Costs Incurred by Researchers,’ The Chronicle of Higher Education, vol. 37, No. 12,

NOV. 21, 1990, pp. A19, A21.

~willim WSSY, vice president for finance, Stanford University, personal communication, Mmch 1991.

6s’ ‘Dingell Asks Defense Contract Audit Agency to Brief Staff on Stanford Overhead Expenses,”Washington Fax, Dec. 10, 1990. Also, see Susan

Tiff4 “Scandal in the Laboratories,” Time, Mar. 18, 1991, pp. 74-75; and Kemeth J. Cooper, ‘‘Five Major Universities Face GAO Audit of Research

Bills,” The Washington Post, Mar. 16, 1991, p. A2.
194 . Federally Funded Research: Decisions for a Decade

Box 6-D—A Federal Research Facilities Program? Perspectives From Academia

In August 1990, University of Wisconsin at Madison Chancellor Donna E. Shalala wrote to President Bush

urging development of a comprehensive plan to finance university research facilities. She stated:

1 recognize that there are a number of important competing claims on the Federal budget, even from the academic

community. However, a planned Federal strategy at this stage can well save us money in the long run, and make more

effective our other investments in science and engineering research and training. 1

Even if money were appropriated to a Federal facilities program, there would be tough choices, such as an equal

distribution of funds by category of institution or some weighted scheme that favors research-intensive universities.2

Stanford’s former Vice President for Finance, William E. Massy, outlines the following possible pitfalls: 3

What would happen if research sponsors were to shoulder the load of paying for needed university science

facilities? . . . Research sponsors might adopt the ‘‘pay-now’ strategy and provide the needed $5.85 billion up front,

Congress is considering a facilities grant program, but it is hard to believe that anything like this amount could be

provided over a few years without huge inroads on operating funds for research.

On the other hand, if the ‘pay-later’ strategy were adopted by sponsors, institutions would have to come up with

up-front money on their own and then try to recover it through the overhead rate. This could be done in one of two

ways: 1. Use gifts, institutional funds, or State appropriations. . . . 4

2. Use debt. . . .

Providing direct grants for facilities is an attractive option on its face, but . . . amounts would probably be modest

in relation to need because of the [Federal] deficit. There is a real danger that appropriations would be at the expense

of operating funds for science. . . . One reason for favoring a grant program is that it makes facilities available to

institutions that are unable to provide up-front finding. . . . But it is precisely the institutions that have not yet been

successful in merit-reviewed scientific competition that have the most need for a facilities grant program and would

benefit most from it. . . .

Facilities funding on a “pay-as-you-go” basis through the indirect cost rates puts the burden . . . on the

institutions, and then reimburses them for some or all of the present value of these outlays. . . . Indirect cost rates

would rise, . . . [and] institutions would have to bear the risks that a facility, once constructed, could not be filled with

sponsored research at full overhead recovery. (The Federal Government takes that risk in an u-p-front grant

program.) . . .

While the Federal Gove rnment would be unlikely to announce a ‘ ‘won’t pay” policy, that could happen by

default if deficit reduction, “no new taxes,’ social programs, and defense come to dominate the need for university

science facilities. A “won’t pay’ policy would preclude facilities grants, and it might also open the way for caps on

indirect cost rates and elimination of universities’ tax benefits . . .

When the scope of a problem is unknown, as with the need for academic research facilities, questions of which

is the most appropriate policy is even harder to answer. To this end, a six-university consortium, called the Center

for Policy Research and Education, has been established to study university finance and cost containment,

investigating ‘‘. . . growth by substitution rather than by adding on cost. In the meantime, neither universities nor

the Federal Government face the prospect of easy solutions.

IQuoted in Karen G rassmuc~ ‘‘Colleges Scramble for Money to Reduce Huge Maintenance Backlog, Estimated to Exceed $70 Billi~

New Federal Help Seen Unlikely,” The Chrom”cle of Higher Education, vol. 37, No. 6, Oct. 10, 1990, p. A34.

%ther administrative questions include: a single program or a line item in the budget of each res.txirch agency, a separate amount for repair

and renovation v. new constJuctioq and an institutional matching requirement. For further discussio~ see Governme nt-University-Industry

R esearch Roundtable, “R esearch Facility Fmancixqy Near-b @tiOIIS,” working draft, February 1991.

3~~fmm WiMarn E. MMSY, “Capital Investment for the Future of Biomedical Research: A University Chief F~ id Offkm’s

View, ” Acaden”c Medicine, vol. 64, 1989, pp. 435437, The dollar estimates for construction he cites are drawn from Nationat Science

Foun&tioQ Scientific and Engineering Research Facilities al Universities and Colleges: 1988 (WaShingtOrL DC: 1989).

@n the cost-eftlciency of academic fundraising, see Liz McMille~ “A Study to Determine the Cost of Raising a Dollar Finds That

Average Coliege Spends Just 16 Cents, ” The Chronicle of Higher Education, vol. 37, No. 1, Sept. 5, 1990, pp. A31-32.

5S@ ‘Education Fiie,’ Stanford Observer, November-December 1990, p. 14. The Department of Education’s Ofllcc of Educational

Research and Improvement is funding the consortium consisting of the University of Southern Califom@ Rutgers-The State Univemity of New

Jersey, Harvard University, Michigan State University, the University of Wisconsin at MadisoL and Stanford University. The Stanford

component, at the Institute for Higher Education Research. will be headed by William Massy.
Chapter 6—Understanding Research Expenditures . 195

should be split into two rates -one for facilities and

equipment (including operation, maintenance, and

depreciation) and a second for all other components

(including administration, library, and student services).66 The academic research community is also

hoping that OMB will reconsider Circular A-21,

which deals with indirect cost practices.67

High indirect cost rates are often seen as detrimental to the researcher, because they increase the total

expenditures for research grants while adding no

additional money for research. For example, many

Stanford University faculty are so concerned about

a proposed increase in the indirect cost rate that they

have pressured the university to cut administrative

and facilities expenditures.68

Does the overhead rate reflect true differences

related to the instructional capacity at universities or

is it due to some accounting mechanism? Before

new policies are crafted, data on actual expenditures

should be collected and presented so they are

amenable to comparisons across institutions. In the

process, both universities and the Federal Government have much to gain in making the system more

simple, transparent, and credible.

Changing Expectations and Competition

Research expenditures increase for reasons besides the line item components of a budget. Researchers also point to higher expectations for their

research, which require more spending and competition in the university environment.

Academic researchers, both young and old, are

asked today to publish more papers, shepherd more

graduate students, and bring in more Federal funding

than their predecessors.69 To boost research productivity, faculty members hire postdoctorates and

nontenure track (nonfaculty) researchers. Graduate

students can-and do-take over portions of faculty

teaching responsibility; tacticians and graduate

students can maintain equipment and run experiPhoto credit: University of Michigan

The calculation of indirect costs is often difficult because of

inherent problems with separating instruction from

research activities on a university campus.

ments; and postdoctorates and nonfaculty researchers can advise students and assist in the operation of

the laboratory. 70 All can perform research.

For example, in the chemistry departments of both

UofM and SU, the average number of graduate

students per faculty averages about six to nine. Some

professors have as many as 25 to 30. Much

chemistry research involves long hours in laboratories, so the pace of research is brisk as well as

necessitating the participation of a greater number of

graduate students. Once a faculty member in a

department or related field succeeds in expanding

his or her research group, others also expand their

groups to keep pace.71

In this very competitive research system there is

increasing pressure to perform more research and to

publish more papers. Consequently, expectations

and expenditures have risen. Perhaps one young

faculty member at Stanford put it best:

 of American Universities, Op. cit., 

 A-21 for Amendment According to Indirect ” Washington Dec. 12, 1990; “AAU Says

 Continue at High Level in To Reopen Circular A-21, ” Washington Fax, Dec. 13, 1990; and Robert M. “The Debate

Over Indirect Costs Raises Fundamental Policy Issues,’ The Chronicle of Higher Education, vol. 37, Mar. 6, 1991, p. A40.

 from of research at i.e., are at a competitive

disadvantage. Eileen Walsh and Karen Bartholomew, ‘‘Indirect Costs Subject of Three Separate Reviews,”Campus Report, Sept. 12, 1990, pp. 1,6.

 entrepreneurial such as biotechnology. See Henry ‘‘Entrepreneurial Scientists and Entrepreneurial

Universities in American Academic Science,” vol. 21, Nos. 2-3, s 1983, pp. 198-233.

“Larger Machines Are Breeding Larger Research Teams,’ The Scientist, 16, 1989, pp. 13, 15.

 at Stanford University and the University of 
196 ● Federally Funded Research: Decisions for a Decade

Each year, we have to write more papers and bring

in more money than many of the senior professors

ever produced in any year of their careers. When we

were hired as assistant professors, we already had to

have published five or ten papers, again more than

any of them had to do. And, it keeps getting worse.72

Whether the premium on productivity defined by

number of papers published is attenuating their

quality is unclear. Bibliometric data, which provide

a measure of use through citations, indicate that

about 15 percent of all U.S. scientific papers are

never cited.73 Publishing requires research; doing

more research requires spending more money. Thus,

research expenditures are not just a cost issue, but a

spending issue as well.

Relative Deprivation

When there is such pressure to compete, standards

become ‘‘whatever it takes to make it. ’ Most

professors understand how much money and how

many graduate students they will need to help them

produce enough papers to get tenure, promotions

beyond tenure, and recognition in their field. Many

hope to do more.

However, if they do not meet these expectations,

some report a sense of failure.74 Failing to meet

self-imposed expectations only intensifies these

feelings. This is true even if the economy of work

has changed such that standards of productivity need

to be revised, or if they have succeeded but not by as

much or as quickly as they had hoped. Social

scientists call this situation ‘‘relative deprivation. ’

Researchers, especially on campuses such as the

University of Michigan or Stanford University,

cannot be said to be “absolutely” deprived. They

are able to acquire laboratory space, graduate

students, and research monies; they produce very

significant amounts of research; and the success

rates for proposals at UofM and SU run at least twice

if not three times the national average (roughly two

out of three proposals are awarded funds, averaged

over these two universities).

Nevertheless, the professors at UofM and SU have

grown accustomed over the last 20 years to producing more graduate students, more publications, and

funding for perhaps four of every five proposals they

submit. 75 The adjustment to comparatively less has

been difficult. Relative deprivation is real, but so is

the greater sophistication of instruments, complexity of experiments, and amount of research that can

be completed in a short time. There is also a trend

toward an ‘‘industrial model,” where project teams

are larger and responsibilities are more distinct

within the group. 76 Research institutions are keyed

to hastening and demonstrating research productivity.

Some experiments have been attempted on U.S.

campuses to temper the drive for more research

output. For example, at Harvard Medical School,

faculty are allowed to list only five publications for

consideration at tenure, with similar numbers set for

other promotions. Thus, the quality and importance

of the candidate’s selected set of papers is stressed,

TZOTA interviews at Stanford University, August 1990.

73~s ~m~fig~y ~Wo~~ ~ ~emly ~ne~go~g ~cited. See David p.=to~“publis~g By—rind For?—the Nu.mbe~, ” SCienCe, VO1.

250, Dec. 7, 1990, pp. 1331-1332; and David P. Hamilton, “Research Papers: Who’s Uncited Now?” Science, vol. 251, Jan. 4, 1991, p. 25. The data

source, thel.nstitute for Scientfilc Inforrnatiom reports that the original estimate included notes, editorials, and meeting abstracts. When Scientilc articles

alone are considered, uncitedness by 1988 of articles published in 1984 drops to 22 percent. For U.S. authors, the proportion is even lower (14.7 percent),

one-halfthat fornon-U.S. authors. See David A. Pendlebury, ‘‘Science, Citatio& and Funding,” letter, Science, vol. 251, Mar. 22,1991, pp. 1410-1411.

Journals, however, appear to have increasingly assumed a more archival function of bestowing credit than of information exchange. Citation rates,

however, are known to vary widely by field. This may reflect more on the ways researchers credit one mother through bibliographic references than

on how they actually use that work. See Derek de Solla Price, “Citation Measures of Hard Science, Soft Science, ‘Ikdnology and Conscience,”

Communication Among Scientists and Engineers, C. Nelson and D. Pollock (eds.) (Lexingtoq MA: D.C. Hea~ 1970), pp. 3-22.

TdScience: The End of the Frontier? op. cit., footnote 4.

750TA ~terviews, op. cit., foomote 71.

761?lSewhem ~S ~S ken c~ed tie “indus~~mtion~ of ~ience, or “, . . a new collecn”vized form in which characteristics of both the academic

and industrialized modes are intermingled. ” See John Z-An Introduction to Sa”ence Studies (Cambridge, England: Cambridge University Press,

1984), p. 132, The dimensions of collectivized scien~, Xcording to z- include costly research apparatus, increasing aggregation of research

facilities, and collaboration in research performance that redefines “teamwork.” In the words of the National Science Board, “. . . modem science and

engineering research is more organized, capital intensive, multidisciplinary, and cooperative than in the past. Our universities must adapt to this need.’

National Science Foundatio~ ‘ ‘The State of U.S. Science and Engineering, ‘‘A View From the National Science Board, statement aecompanyingScience

& Engineering Zndicatorfi1989, February 1990. These dimensions are discussed fhrther in ch. 7.
Chapter 6—Understanding Research Expenditures 197

though measuring these characteristics, bibliometrics notwithstanding, remains contentious.77

Strong incentives militate against reducing research output. For instance, since most overhead is

brought into the university by a small number of

research professors (at Stanford, 5 percent of the

faculty bring in over one-half of the indirect cost

dollars78), proposals to reduce research output are

not looked on with favor by many university

administrations. Any measure that would curb the

productivity of these professors would deprive the

university of revenues. Thus, many universities try

to maximize the level of research volume and

output.79

OTA finds that research personnel at the two

universities examined are experiencing relative deprivation. This would appear to be symptomatic of

pressures felt on similar high-caliber research campuses. If so, then the Federal Government, by

sending clear signals about the importance of

scientific merit, education, and equity in allocation

decisions, could aid universities in planning for a

changing research economy (see box 6-E).

Responsiveness

Another factor leading to perceived instability in

the research environment is the difference in time

scales between changing national needs, on the one

hand, and universities’ capacity to respond to them,

on the other. To build a research infrastructure, like

any government contractor, universities must commit funds to construct facilities and to purchase

equipment. If the Federal Government then decides

to switch emphases, universities must continue to

maintain this infrastructure. Also, no matter how

resourceful one may be, there are few incentives for

a professor to change research areas after having

accumulated knowledge in one or two specialties.80

It is perceived as being more cost-effective for the

university to hire new younger faculty and buildup

their research capacity than to try to convert an older

researcher to a new field. Consequently, researchers

in universities have little recourse if research emphases shift dramatically in Federal support for their

particular field.81

An example of a growing department at the

University of Michigan may help illustrate some of

these points. In 1980-81, the Electrical Engineering

and Computer Science (EECS) Department was

perceived as weak. The central administration at

UofM decided to invest resources in the department

and pursue expansion. Most importantly, it decided

on a few key research priorities: optics, solid state

electronics, robotics, and microelectronics. The

administration encouraged retirement of many of the

older faculty and “weeded out” a few others. It

hired faculty in the designated areas to fill the vacant

slots and added a few faculty positions as well.

EECS expanded its research capacity in areas that

the Federal Government presently supports. It has

subsequently received more Federal tiding. Unfortunately, EECS does not project, for the foreseeable

future, the flexibility it experienced in the 1980s.

The department attributes its flexibility to the hiring

of new faculty, and doubts that this flexibility will

continue as the faculty ages.

The development of EECS may be unique among

university departments. As a survival tactic, universities have traditionally attempted to maintain broad

departments, covering many subdiscipline, so that

if funding in one area diminishes it has a minimal

77seeN.L. Geller et al., ‘‘Lifetime Citation Rates to Compare Scientists’ Worlq’ Social Science Research, vol. 7, 1978, pp. 345-365; and A.L. Porter

et al., “Citations and Scientillc Progress: Comparing Bibliometric Measures With Scientist Judgments,” Scientometrics, vol. 13, 1988, pp. 103-124.

The National Science Foundation now limits the number of publications it will consider, as evidence of an applicant’s track record, in reviewing grant

proposals. See Hamilton, op. cit., footnote 73.

i’8Rick Biedenweg ad Daua Shefley, ~986+7 Deca~/ l~irecr c~st Stiy (S~ord, CA: sword ufivers@, February 1988), p. Xii.

T~or exnple, Dodd Kennedy, presid~t of Stanford University (SU), said iu ~OT4 interview (Aug. 2, 1990) that if the l~s do not Promise

great increases in the Federal science budgets, SU will have to institute four means to balance its budget, and the first three have already been introduced:

1) restructure the budget for SU and instigate cuts; 2) boost the research volume to bring in more research dollars; 3) lobby for increased facilities

programs from the Federal Government; and 4) move some indirect costs to the direct cost lines to ensure full recovery.

~SeeJo~Z*Knowing EveVthingAboutNothing: Specialization and change unscientific Careers (Cambridge, EWtid: Cambridge University

Press, 1987); and John ZimW “Researches a Career, “ The Research System in Transition, S.E. Cozzens et al. (eds.) (DordrechL Holland: Kluwer, 1990),

pp. 345-359. “Thebuildup of accumulated skills and knowledge puts a lot of inertia into the academic research system, and. . . a premium on expansion

as the easiest route toward reorientation of priorities. ” Harvey Brooks, Harvard University, personal communication% February 1991.

gl~the case of a~tio~ researchmissio~ such as the War on Cancer, relabeling one’s research toqual@ formissionmoney was a workable s@ategY.

See K.E. Studer and D.E. Chub@ The Cancer Mission: Social Contexts of Biomedical Research (Beverly Hills, CA: Sage, 1980), especially ch. 3.
198 ● Federally Funded Research: Decisions for a Decade

Box 6-E—Emulating the Research University: Beware

In the last few decades, many universities have aspired to emulate the top research universities, in the hopes

of gaining a larger share of the Federal research pie. However, in the 1990s, the model of a research university may

become less attractive, functional, and reproducible.

Research universities are characterized by strength across departments and other units where research is

performed. For instance, Stanford University is not known particularly for strength in one field, or even several, but

for across-the-board excellence in every discipline-within science and engineering and without. Similarly, these

universities are the major recipients of Federal research and development funds and train the largest cadre of new

Ph.D. s in those disciplines. ]

Outside the top 50 to 100 institutions are hundreds of universities that also compete for Federal research

funding. 2 They reason that an influx of Federal monies could ease some of the financial burdens they face, but

moreover, could boost their capacity to do research and attract still more funding from other sources, Arguably, the

research university has been a model for emulation even for institutions whose missions and resource base made

them unlikely candidates to join the top 100. Institutional mobility is rare, but does occur.

Unfortunately, beneath the surface of many successful research universities lie many fiscal problems connected

with their research enterprise:

1. The demand for research funding is rising in some fields faster than funding available from Federal and

other sources, and some universities claim they cannot keep up.

2. Demands for state+ f-the-art facilities are increasing with little financial help from the Federal Government,

so universities must tap State or private coffers to allow for renovation and new construction.

3. Charges abound that re search faculty are shirking their teaching commitments, and students complain that

their education has suffered.

These problems notwithstanding, there is still an allure--and a necessity-to pursue Federal research dollars.

In addition to competing quite successfully for disciplinary agency support, universities are attracted by Federal

initiatives that enjoy ‘ ‘new’ priority funding. At the University of Michigan, for example, initiatives such as global

lb fisc~ yea 1988, 10 universities rcxxived ahnost one-quarter of the FCd~ research and development funds awarded to all academic

institutions. They were Johns Hopkins, Stanford, Massachusetts Institute of lkchnology, Wisamsin-MadisoU Michi= Washington,

Califotia-San Diego, Cornell, California-Ims Angeles, and Columbia. See National Science Foundation Academic Science and Engineering:

R&D Funak--Fiscuf Year 1988, NSF 89-326 (WashingtO~ DC: 1990), table B-37. Tbese same 10 universities produced 15 percent of all science

and engineering Ph.D.s. (3,303 of 20,738) that year. National Science Foundation Science and Engineering Doctorates: 1960-89, NSF 90-320

(Washington, DC: 1990), table 9. The disrnbution of honors, e.g., Nobel prizes and election to the Nationat kademy of Sciences, reinforces

the stratilcation among research universities and the role of a sekct few in the education and employment of U.S. scientists.

@or example, 1,719 institutions were supported by the National Institutes of Health (NW) in 1990, but only 25 accounted for 38 percent

of NIH’s extmmural funds. See ‘ *MH Policy Change Could Shake Up Distribution of NIH Extramural Funding, Official Says, ” Wa.rhington

Fax, NOV. 23, 1990.

effect on the department as a whole. On the other In the current research economy, a broad base is

hand, if one area receives increased support, the

university will be prepared to take advantage, Due to

increased competition for funds, the model of a

multifaceted yet targeted department, such as EECS,

may become more prevalent. Universities may find

that concentrating their research capabilities in

specific areas may enhance their competitiveness for

Federal funding by augmenting their research track

record and the availability of research facilities and

equipment in those areas. Another tactic is the

welcoming of non-Federal sources of research

funding to campus, often to “leverage” Federal

funding (see box 6-F).

increasingly difficult to maintain. Universities trying to achieve the status of the top 50 research

institutions are bound to face numerous obstacles if

they try to obtain-and sustain-success through a

broad-based approach.

Summary

In this chapter, OTA has reviewed data on

research expenditures from the perspectives of both

the Federal Government and academic research

performers. Fueled by increases in Federal, industrial, and academic spending on research, the national research effort and the levels of basic and
Chapter 6—Understanding Research Expenditures . 199

climate change and the Human Genome Project are seen as ‘‘bandwagons: once the university commits resources

to participate, it runs the risk of absorbing the costs of the research infrastructure and personnel atler Federal support

wanes.

3

If the top research universities are struggling to maintain excellence, even with their inherent advantages, it

may be unwise for other universities to try to become like them. It takes years to develop the breadth and depth of

resources that makes a research university; it does not happen quickly.4 Moreover, the classic research university

model may simply be maladaptive for these times. 5 Maintaining the human and physical infrastructure that has

accumulated over decades is a huge financial burden that affects all campus functions.6

Building targeted strength would appear to be a more sensible institutional strategy than striving for

across-the-board excellence. Thus, one magnet laboratory funded by the National Science Foundation (and the State

of Florida) will not transform Florida State University into a Massachusetts Institute of Technology. But Florida

State might become world-class in research on high-energy magnetism. Such targeting demands concentrated fiscal

and human resources, selective recruitment of faculty researchers, and construction of facilities. The Federal

Government has a role to play in this effort. As recently stated: “It’s the classic American challenge: What’s the

best tradeoff between the conflicting desires of preserving excellence and promoting diversity?’

Preserving excellence in research and teaching at U.S. researching universities is not a Federal obligation; it

is a good investment. Research universities have strategic plans, critical masses of researchers, and the reputation

for selectivity. However, those institutions that aspire to join the select group of top research universities in the

1990s might best reconsider the research university model-and proceed at their own risk.

31n he Shon ~resomes tit wo~d have been devoted to instruction tend to @ ~vcfi~ to ~h~pmfile r

esearch. Investing in

fashionable research is an important part of the university portfolio, but the cost of some activities may have an adverse effkct on others. This

example is based on OTA interviews with University of Michigan adrmm.strators, JuJy 1990.

4’IMS Wa a mUXTmt tie- at htigs held by the House Committee on Science and ‘Rdmology, Task FOrCe on Science policy in

1985-86. For an analysis of members’ and witnesses’ concerns, see Patrick Hamlet4 ‘‘Task Force on Science Policy: A Window on the FederaJ

Funding and Management of Researc&’ OTA contractor repo~ October 1990. Available through the National lkchnic.al Information Service,

see app. F.

5S= L&& E. p~ker M David J,. CJar~ ‘‘Departrnenud Responses to Fluctuation iI’J RWMIT h Resources,” Research Management

Review, vol. 4, spring 1990, pp. 19-34.

6P~pS tie most striking evidwe of a tiv~i~’s research intensiveness is the number of postdoctorates it en@oys. BY Ms m~,

Stanford and MichigaII are spectacular examples. At both institutions, postdoctorates in life sciences represent two to three times the number

of postdoctoral appointees in all other fields combined. (Physical sciences is the runner-up on both campuses.) At Stanford, the number of life

sciences postdoctorates doubled to 600 in 1988. This also represents aJmost twice the number of graduate (i.e., predoctoral) students in life

sciences at Stanford. At Michigan the emphasis is reversed: although postdoctorates in life sciences grew from 160 to 280 between 1980 and

1988, the number of predoctord students in these fields totaled three times the postdoctoral COUnL These differences in graduate students and

postdoctoral numbers are probably reflected in the composition of research teams.

7Michael Schrage, “Bturring the Line Between Funding Science and Funding Economic GrowtJA” The Washington Post, Oct. 5, 199(J,

p. F3.

applied research individually are at levels surpassing Reliable analyses of research expenditures by

the “golden age” of the 1960s. However, the rise in

demand for funds from the research community

continues to outpace Federal funding increases.

This rise in demand is due primarily to increased

spending on research, and only secondarily to

increases in the ‘‘costs’ of individual components

of research budgets. Increased spending appears to

stem from the growth of the size of research groups

under the direction of one principal investigator, a

tendency toward growing pressure to produce more

research, and an increasing complexity of equipment

and facilities (although advances in technology can

also decrease overall spending).

Federal agencies are not available. Information

provided to the agencies by the performers is likely

to combine actual need with the desire to pursue

boundless opportunities in research. Some trends,

however, are well documented. The number of

scientists conducting research and supporting graduate and postdoctoral students has grown. Every

agency has seen a growth in the number of grant

applications submitted. In addition, average expenditures per investigator have nearly tripled, in real

terms, since 1958. The obsolescence time for

equipment and instrumentation has shrunk more

than twofold, and facilities built in the 1950s and

1960s are in need of repair and renovation. The
200 . Federally Funded Research: Decisions for a Decade

Box 6-F—industry on Campus

Universities have been aggressive in seeking industry support for research in the 1980s, though the relationship

between the two sectors has been a ‘ ‘two-way street” for more than a half-century. 1 Industry funding of university

research, though brisk in the 1980s, still represented just over 5 percent of all university research. (The highest

proportions are found at engineering institutions--Georgia Institute of Technology, Carnegie-Mellon, Massachusetts Institute of Technology.) Most of that funding was geared to generic, nonproprietary knowledge aided by

support for faculty, graduate students, and infrastructure. In other words, technology transfer, patenting, and

commercial gain were not the primary motivations for academic-industry relations. 2

Until recently, only certain fields have had relevance to industry investment for a profit motive: computer

science, metallurgy, materials science, and chemistry.3 More recently, biology has offered industry new techniques

for the development of products and processes. As historian Roger Geiger points out:

The case for the importance of the university role in economic development rests on two pillars: that industry

has been underinvesting in generic research, and thus could profitably utilize additional research from universities;

and second, that discoveries of potential commercial value were being made in universities, but were not reaching

the market because of linking rnechanisms.4

In many fields, the demand for research funds are exceeding available funds from traditional sources--

especially in fields that require large-scale, technologically advanced equipment and instruments, as well as more

technicians with more skills. Concerns about sponsorship--military as well as industrial-skewing the research

l~ge chemi~ and drug companies funded Un.ivemity laboratories for routine services, h.ke tes@, ~in the 1930s. See Roger

L. Geiger, “indus~ and University Research: The Revolution of the 1980s,” Science and Technology and the Changing World Order,

colloquium proceedings, Apr. 12-13, 1990, S .D. Sauer (cd.) (Washington DC: American Association for the Adv aneement of Science, 1990),

PP. 138-14$; ~p~ E. my, “Advantageous Liaisons, ” Issues in Science & Technology, vol. 6, No. 3, spring 1990, pp. 40-46.

2S= Rog~ L. Geiger, ‘‘Milking the Sacred Cow: Research and the Quest for Useful Knowledge in the American University Since 1920,’

Science, Technology, & Human Wlue.r, vol. 13, Nos. 3-4, summer & autumn 1988, pp. 332-348.

3~o~~ t. ~c~d R. N~lsou 1(~ti~tiom SuppO~ T&hnical Advance in Industry, ” American Econom”c Rm’cw, vol. 76, MY

1986, pp. 186-189.

‘%eiger, op. cit., footnote 1, p, 147. For a case in poin~ see David Blumenthal et al., “University-Industry Research Relationships in

Biotechnology: Implications for the University, ” Science, vol. 232, June 13, 1986, pp. 1361-1366; U.S. Congress, Oflice of ‘Xkdmology

Assessmen~ Us. Invcsrmenr in Biotechnology, O’IA-BA-360 (Washington, DC: U.S. Governrne nt Printing Of?iee, July 1988); and Phyllis B.

Moses and Charles E. Hess, “Getting Biotech Into the Field, ” Issues in Science& Technology, vol. 4, No. 1, fait 1987, pp. 35-41.

entire enterprise has grown at a rate above inflation base, from an “expenditures/costs” perspective, it

and beyond what the current Federal budget can

perhaps afford.

Competition for funds, coupled with a failure to

meet research expectations on the part of many

researchers, contributes to relative deprivation at

many research universities. University researchers

feel deprived because their resources (and subsequent outputs) have not met their expectations.

Although many universities urge a quick infusion of

money to ensure their responsiveness to national

research missions as well as the scientific research

may not be the appropriate role of the Federal

Government simply to supply extra funds. Rather,

the Federal Government could encourage both

established and aspiring research universities to

consider the funding environment and to adjust their

research agendas, timetables, and needs accordingly. 82 Devising mechanisms for understanding

and coping with research expenditures is one of the

central challenges to the Federal system for finding

research in the 1990s.

gzIt should be noted t.hag under the Florida and Federal Demonstration Projects, no-cost extensions on research grants and other meaOS of expad@

the authority of universities to control research budgets locally have been successfully tested. By providing flexibility, these expanded authorities create

opportunities for cost savings and improved accountability. See Anne Scanley and William Sellers, Govement-utiverSi~-mduS~ Research

Roundtable, “SUmmary of Interim Reports Submitted by Grantee Organizations Participating in the Federal Demonstration projec~” unpublished

document Oct. 1, 1990.
Chapter 6—Understanding Research Expenditures ● 201

agenda and undermining the spirit of open inquiry on campus are still voiced.s But the trend has not so threatened

academic norms as to provoke a backlash. 6 Perhaps out of economic necessity, universities have accommodated.7

The “competitiveness” debate has legitimized academic involvement with industry, if not outright promoted

it. There is an assumption that strengthening the links between industry and university research will improve

America’s economic malaise. A series of legislative and executive initiatives in the 1980s encouraged collaboration,

such as the Patent and Trademark Amendments of 1980 (Public Law 96-517), the Stevenson-Wydler Technology

Innovation Act of 1980 (Public Law 96480), the 1981 Economic Recovery Tax Act (Public Law 97-34), and the

National Cooperative Research Act of 1984 (Public Law 98462). In addition, the 1980s witnessed a growth of State

economic development programs that aimed to stimulate university-industry cooperation in public universities,

More and more, linkage between universities and industry has been viewed as essential and mutually beneficial.*

The resiliency of the university will continue to be tested by rising demands for research funds and the

proliferation of missions served by experts on campus. In the 1990s, opportunistic funding of university research

may give way to a moderation of corporate-sponsored research. Or such undertakings may continue to be physically

segmented in a research institute or center as a way of detaching the industrial values it symbolizes from the core

campus organization.9 The existence of university-industry collaborations is not in doubt; the forms of these

collaborations, however, will remain in flux.

SFOI ~=@e ~ “saw-in Uni~~ty.B~ Rcs~ch who COfItfOIS’? WhO ~h?’ SCle?ICC, ~CChnOk7&Y, & Huwn ~~e$! ~id

issue, vol. 10, No. 2, spring 1985, pp. 3-1 14; and Henry Etzkowitz, “The Second Academic Rcvolutioxx The Role of the Research University

in Economic Development,” The Research System in TraM”tion, S.E!. Cozums et al. (eds.) (Dordrech6 Holland: Kluwer, IW), pp. 109-124.

61t ~-e1‘~te~w~ pm-’ I~~d~c ~ we~ as a F~~ policy issue. For discussions, see MMW1 C. LaFollette, “U.S. pdi~

on Intellectual Property in R&D: Emerging Political and Moral Issues, in S .E. Cozens et al., op. cit., footnote 5, pp. 125-139; and U.S.

Congress, 0f17ce of ‘I&hnology Assessment Intellectual Property Rights in an Age of Electronics and Information, OTA-CIT-302 (Springfield,

VA: Nationat Tlxhnical Information Service, 1986). Also see Charles Weiner, “Universities, Professors, and Patents: A Continuing

Controversy,” 72chnology Review, vol. 89, No. 2, February/March 1986, pp. 33-43.

7For ~Wplc, ~ ~ge R. McDoWel~ ‘‘m-t Colleges of A@cuhure: Renegotiating or Awox@3 a s~i~ Conmcg’

Choices, second quarter 1988, pp. 18-21.

@ffice of ‘Ikchnology Assessment op. cit., footnote 4. Also see Barry Bozeman and Michael Crow, “The Environments of U.S. R&D

Laboratories: Politicai and Market Influences,” Policy Sciences, vol. 23, No. 1, 1990, pp. 25-56.

9S= ~rothy Nelkjn and Richard Nelson, “Commentary: University-Industry Alliances,” Science, Technology, & Human kiues, vol.

12, winter 1987, pp. 65-74. Ln other words, the value of academic research is likely to persi.s~ if not grow. See Edward M. SCoh.liclq “Basic

Research and 1ts Impact on Industrial R&D,” Research-Technology Management, November-December 1990, pp. 21-26.
CHAPTER 7

Human Resources for the

Research Work Force
Contents

Page

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

Baseline Data on Science and Engineering Degrees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

Degrees, Gender, Ethnicity, Nationality, and Fields of Study . . . . . . . . . . . . . . . . . . . . . . 206

Forms of Federal Support to Graduate Students . . . . . . . . . . . . . . . . . . . . . . . . . . .. ...+.. 208

Employment of Researchers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

The Shape of the Future Research Work Force . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

The Uncertainty of Ph.D. Projections . . . . . . . . . . . . . . . . . . . .. . .+ . . . . . . . . . . . . . . . . . . . . 213

Training for an Uncertain Research System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Boxes

Box Page

7-A. Institutional Variations on National Trends: Graduate Enrollments at Four

Research Universities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .210

7-B. The Unfaculty: Who Are They and Why Should We Worry? . . . . . . . . . . . . . . . . . . . 214

7-C. Point of View of a ’’Small” Scientist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

7-D. The Priority of Research Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

7-E. Flexibility at the Beckman Institute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

7-F. Minority Biomedical Research Support Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

Figures

Figure Page

7-l. Ph.D.s Granted, by Field: 1966-88 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

7-2. Distribution of Doctorates by Science and Engineering Field, 1960-90 . . . . . . . . . . . 207

7-3. Science and Engineering Degrees, by Level and Sex: 1966-88 ++ . . . . . . . . . . . . . . . . . 207

7-4. Federal Support of Science and Engineering Graduate Students: 1969 and 1988.. 209

7-5. Distribution of Doctoral Scientists and Engineers in Academic R&D, by Field:

1977 and 1987 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

7-6. Average Annual Percentage Growth Rate of Doctoral Scientists and Engineers in

Academic R&D, by Field: 1977-87 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

7-7. Employment of Doctoral Scientists and Engineers in Basic Research, by Sector:

1987 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

Table

Table Page

7-1. Issues in Team Research Performed indifferent Organizational Contexts . . . . . . . . 224
CHAPTER 7

Human Resources for the Research Work Force

My first priority is to create an environment in which talented young people choose

careers in health sciences research. . . . My second priority would be to fashion a system

in which talented, more senior researchers could obtain stable funding for their best

work. Those two priorities cannot be achieved without setting some limits and making

difficult choices.

Leon Rosenbergl

Introduction

The scientific education system in the United

States, especially at the doctoral level, is the envy of

the world. Foreign nationals continue to seek

degrees in science and engineering at U.S. institutions at an ever growing rate, and this exemplary

“production’ of Ph.D.s has continued over at least

the past 30 years.

The U.S. graduate research and education system

trains new researchers and skilled personnel for all

sectors of the Nation’s work force (and for some

countries abroad). While new researchers have

traditionally been trained for faculty positions in

academia, in fields like computer science, the

demand for technical labor outside of academia is

great. Some fields, like chemistry, also benefit from

having a large set of potential academic and industrial employment opportunities. This diversity

makes any labor market fluid and its forecasting

difficult, but the major components can be analyzed.

This chapter focuses on Ph.D. production and

employment in the United States and the research

work force, as a subset of the total science and

engineering work force. The educational ‘pipeline’

that prepares students at the K-12 through undergraduate level for doctoral study is discussed where

needed. 2 First, the chapter discusses the overall

shape of Ph.D. production in the United States, the

Federal role in supporting graduate education, and

the present employment prospects for new Ph.D.s.

Second, the chapter focuses on projections for future

employment of Ph.D.s, and then turns to training

Photo credit: U.S. Department of Energy

Scientist mixes a chemical sample. Broadening the

participation in research of traditionally underrepresented groups, such as U.S. minorities, is a central

issue for the health of the research enterprise.

considerations for an uncertain future. The chapter

concludes with a discussion of the Federal role in

Ph.D. production and employment for the 1990s.

Baseline Data on Science and

Engineering Degrees

Trends in the award of science and engineering

(s/e) degrees highlight 20 years of growth in human

 in Dick “The Growing Crisis in Medical Science, ” Time, Dec. 17, 1990, p. 21.

 education how the Federal Government intersects it is contained 

of Technology Assessment, Education Science and 52 (Washington, DC: U.S. Government Printing Office,

March 1989).

–205–
206 ● Federally Funded Research: Decisions for a Decade

resources.

3 Scientific education has yielded a significant number of new Ph.D.s, yet the benefits of this

education have not accrued equally to all groups and,

therefore, to the Nation. Women and U.S. racial and

ethnic minorities, despite gains in Ph.D. awards

through the 1970s and 1980s, lag the achievement of

white men. Relative to their numbers in both the

general and the undergraduate populations, women

and minorities are underparticipating in the research

work force.4 Foreign nationals on temporary visas

are a growing proportion of s/e degree recipients.5

National Science Foundation (NSF) data indicate

the following trends.6

Degrees, Gender, Ethnicity, Nationality, and

Fields of Study

The total number of Ph.D.s awarded in s/e has

increased from 17,400 in 1977 to over 20,250 in

1988. In addition, the proportion of s/e Ph.D.s

awarded as compared with Ph.D.s granted in all

fields varied from 57 to 64 percent over the period

from 1966 to 1988 (see figure 7-l).

Of the approximately 34,000 Ph.D.s awarded in

1988 (in all s/e and non-s/e fields), the distribution

by s/e field ranges from 2 percent in environmental

sciences to 15 percent in biological/agricultural

(hereafter, “life”) sciences. Trends in field shares

are variable, showing percentage increases and

decreases over the period 1966 to 1988 (see figure

7-2).

One in three college graduates earns the baccalaureate degree in an s/e field. By gender, men earn

more baccalaureate degrees in s/e fields per thousand than women by a ratio of three to two.7 In 1988,

women earned 40 percent of baccalaureate degrees,

30 percent of master’s degrees, and 27 percent of the

Figure 7-1—Ph.D.s Granted, by Field: 1966-88

Percentage of all Ph.D.s granted

100

90

80

70

60

50

40

30

20

10

0

1966 1971 1976 1981 1986

n Science/engineering n Conscience/engineering

SOURCE: National %ience Foundation, Science and Engineering 12egrees: 1966-88, A Source Book, NSF 90-312 (Washington, DC:

1990), table 1; and National .%ienee Foundation, Science and

Engineering Doctorates: 1960-89, NSF 90-320 (Washington,

DC: 1990), table 1. National Science Foundation, Science and

Engineering Degrees: 1966-88, A Sowce Book, NSF 90-312

(Washington, DC: 1990), table 1; and National Seienee Foundation, Science and Engineering Doctorates: 1960-89, NSF

90-320 (Washington, DC: 1990), table 1.

doctorates awarded in s/e. At the Ph.D. level, this

proportion represents more than a tripling since

1966 (see figure 7-3). (In non-s/e fields, however,

women have achieved parity in Ph.D.s earned and

exceed the numbers of ‘menawarded baccalaureate

and master’s degrees.)

Except for life sciences, psychology, and social

sciences, the number of doctorates awarded to

women is modest. In 1988, among U.S. citizens,

men earned 90 percent of the engineering Ph.D.s, 63

percent of the science Ph.D.s, and 48 percent of the

non-s/e Ph.D.s. In fractional terms, women now earn

one in three life sciences and social sciences Ph.D.s

and more than one of every two Ph.D.s awarded in

psychology. From 1966 production rates, engineer3AMou@ OT+j uses the shorthand “scientists and engiQtXX5, “ it recognizes the diversity of fields represented by the term. These fields are those

used as degree-granting categories in the National Science Foundation’s Science Resources Studies reports: engineering, physical sciences,

environmental sciences, mathematical sciences, computer/information sciences, life (biological/agricultural) sciences, psychology, and social sciences.

dDegrees done te~ ~~complete stow of fi~e supply of scientists and engineers. For example, college atten~ce rates of 18- to 21-Year-olds ‘W

by gender and race. Since 1972, 35 to 40 percent of whites of both sexes in the cohort have attended college with Black rates in the 25 to 30 percent

range. By 1988, female attendance exceeded that of males and was rising, whereas male attendance of both races peaked in 1986-87 and has declined

thereafter. See National Science Board, Science & Engineering Indicators— 1989 (Washingto@ DC: U.S. Government Printing OffIce, 1989), figure

2-2, p. 50. The National Scienee Foundation furnishes all data reported in the Science & Engineering Indicators report.

5For ~ Ovemiew, se co~ssion on ~ofessio~s ~ Science ad T&~OIOgy, ~ea~~~”ng National Needs for Scientists tO the year 2000, mpOrt

of a workshop, Nov. 30-Dec. 1, 1988 (Washington, DC: July 1989), pp. 20-24. For more on graduate engineering education, see Elinor Barber et al.,

Choosing Futures: U.S. and Foreign Student Views of Graduate Engineering Education (New York NY: Institute of International Educatiow 1990).

6Natio~ Scienm Fom&tioq science andEngineen”ng ~egree~: ~9@-I$)&)-A sOurCe BOok, NSF 90-312 (wastingto~ DC: 1990). Doctorate data

are drawn from the multiple ageney-sponsored Survey of Earned Doctorates conducted under contract by the National Research Council and assembled

and reported by the Division of Science Resources Studies of the National Science Foundation.

TTh,is ratio ~S MITOWCXI since 1$)66 when it was nearly 3.5 to 1. See ibid., table 55, p. 43. Also see Sarah E. Iluner ~d WiNktIII G. BOWm “me

Flight From Arts and Sciences: Trends in Degrees Conferred, ” Science, vol. 250, Oct. 26, 1990, pp. 517-521.
Chapter 7—Human Resources for the Research Work Force . 207

Figure 7-2—Distribution of Doctorates by Science

and Engineering Field, 1960-90

(by decade, in percent)

Life sciences

Physical sciences

Engineering

Social sciences

Psychology

Mathematical

sciences a

Environmental

sciences

Percent of all doctorates earned

O 2 4 6 8 10 12 14 16 18

 1960-69 1970-79

 1980-89

 this were not awarded until the late 1970s: before 

 science was counted with mathematical sciences.

SOURCE: National Science Foundation, Science and Doctorates: 1960-89, NSF 90-320 (Washington, DC: 1990), table 1.

ing increased from virtually no awards to women to

almost 7 percent in 1988, while physical sciences

experienced a fourfold increase to 17 percent.8

The total number of Ph.D.s in s/e awarded to

minorities rose from 560 in 1975 to 1,100 in 1988.

However, trends by ethnic group are not as consistent. Black U.S. citizens earned 240 Ph.D.s in 1975,

which rose to a high of 290 in 1979, but by 1988 had

dropped to 230. Degrees awarded to Hispanics over

the same period increased from 130 in 1975 to 320

in 1988, exhibiting predominantly steady increases

each year. The most dramatic increase occurred

within the Asian population, which recorded increases from 190 Ph.D.s in s/e in 1975 to 440 Ph.D.s

in 1988, with gains posted in every year but one

(1985).9

Figure 7-3-Science and Engineering Degrees, by

Level and Sex: 1966-88

Bachelors

Percent of degrees earned 100

90

80

70

60

50

40

30

20

10

0

1966 1971 1976

Masters

Percent of degrees earned 100

90

80

70

60

50

40

30

20

10

0

1966

100

90

80

70

60

50

40

30

20

10

0

1986 -

100

90

80

70

60

50

40

30

20

10

n

1971 1976 1981 1986 “

Doctorates

Percent of degrees earned 100

90

80

70

60

50

40

30

20

10

0

100

90

80

70

60

50

40

30

20

10

1966 1971 1976 1981 1986 -

 Percent women Percent men

SOURCE: National Science Foundation, Science and Engineering Degrees: 1966-88, A Source Book, NSF 90-312 (Washington, DC:

1990), tables 3 and 4.

 op. cit., footnote 4, tables 23 

 pp. 55-56. The numbers bave been rounded.
208 . Federally Funded Research: Decisions for a Decade

Foreign citizens on temporary visas earn increasing proportions of the s/e doctorates awarded by U.S.

universities: one-quarter of all s/e Ph.D.s, and as

much as 40 percent in engineering and mathematic

s 

10 The number of Ph.D.s awarded in s/e to foreign .

citizens on temporary visas increased from 2,700 in

1975 to 4,800 in 1988, with the most rapid gains in

the 1980s. Foreign nationals on permanent visas, on

the other hand, decreased from 1,200 in 1975 to 820

in 1984, but experienced a rapid rise to 1,100 in

1988. At the same time, the total for U.S. citizens

dropped from 14,000 in 1975 to 12,800 in 1988, with

the most rapid decrease in the late 1970s. During the

1980s, s/e Ph.D.s awarded to U.S. citizens showed

no clear trends, ranging from a low of 12,600 (1987)

to a high of 13,300 (1981).11

Insummary, the total number of Ph.D.s awarded

in s/e in the United States has increased by nearly 50

percent from 1977 to 1988. The numbers of women

and minority recipients of Ph.D.s have also increased, with the greatest gains posted by women,

Hispanics, and Asians, but with no gain by Blacks.

Perhaps most dramatic is the increase in Ph.D.

awards to foreign citizens on temporary visas, which

almost doubled from 1978 to 1988. (For a comparison of national trends with the experiences of four

research universities-public and private, and regionally dispersed—see box 7-A.)

Forms of Federal Support to

Graduate Students

Clearly, graduate enrollments and the award of

the Ph.D. in s/e depend on more than undergraduate

degree attainment. Institutional practices and Federal policies play a significant role in graduate

student support, completion of the doctorate, and

employment aspirations. OTA notes the following

trends.

Ever since the National Defense Education Act of

1958 (NDEA, Public Law 85-864) passed in the

wake of the Sputnik launch, the Federal Government

has been pivotal in pre- and postdoctoral support of

science, engineering, and indeed, non-s/e students .12

Additional programs were soon established by NSF,

the National Aeronautics and Space Administration

(NASA), the National Institutes of Health (NIH),

and other Federal agencies. This period of growth,

beginning in the 1960s, in Federal programs offering

fellowships (portable grants awarded directly to

students for graduate study) and traineeships (grants

awarded to institutions to build training capacity)

was followed by decreases in the 1970s.13 In the

natural sciences, these declines were offset by the

rise in the number of research assistantships (RAs)

awarded on Federal research grants. Federal support

to the humanities and social sciences has always

been comparatively less since, outside of NDEA,

traineeships and fellowships were offered for the

natural sciences, and research assistantships are

rarely supported on social sciences or humanities

grants. During the 1980s, other sources of support,

including loans and family contributions, remained

constant (see figure 7-4).

In the 1980s, RAs became the principal mechanism of graduate student support, increasing at 5

percent per year since 1980 (except in agricultural

sciences, where RAs have actually declined). This

trend is consistent with the growing “research

intensiveness’ of the Nation’s universities: more

faculty report research as their primary or secondary

work activity, an estimated total in 1988 of 155,000

in academic settings.14

If the Federal agencies were to change the mix of

support to graduate students, first by increasing the

number of portable fellowships, the concentration of

support in the major research universities would be

reinforced. On the other hand, if the government

were to increase the number of traineeships, Federal

support could be directed to a broader set of

institutions. No particular mix of support mecha1°Ibid., pp. 46, 56.

llNote that tie nubers of Ph.D.s award~ to U.S. citizens and to foreign citizens on temporary and permanent visas do not add Up to tie n~ber

given for all Ph.D.s awarded. Roughly 7 percent of the total number of Ph.D.s are of unknown citizenship. Lawrence Burto~ Science Resources Studies,

National Science Foundatiorq personal communication, Dec. 10, 1990.

lzFor det~ls, s= U.S. Con=ess, Office of T~~oloW Assessmen6 De~graphic Trends ad the Scientific and Engineen”ng Work Force ,

OTA-TM-SET-35 (Sprin~leld, VA: National lkchnical Information Service, December 1985), pp. 44-49.

IsAssociation of American Universities, The Ph.D. Shortage: The Federal Role (Washington DC: Jan. 11, 1990), PP. 15-16.

14’rhe5e lss,~ repreSent 37 ~ment of employed ph.D. scientists and q@eer5 in me ufited states in 1981’. There are sever~ assumptions built

into these estimates-that Ph.D.s are most likely to do research that research is considered a “prirmq or secondary research activity” by survey

respondents, and that (although R&D are coupled here) basic and applied “~” not “D’ is performed in academic settings. National Science Board,

op. cit., footnote 4, pp. 46, 57, 115.
Chapter 7—Human Resources for the Research Work Force ● 209

Figure 7-4—Federal Support of Science and

Engineering Graduate Students: 1969 and 1988 —

(by type of support)

In percent

100

8 0

60

40

20

0

Total 1969

students

F

ellowships—

LTraineeships–

Research

assistantship

Teaching

assistantship

Other support

100

8O

60

40

20

0

supported: 51,620 54,852

NOTE: Fellowships and traineeships were not reported separately in 1969.

SOURCE: National Science Board, Sb”ence & Engineering lndicafors—

1989, NSB 89-1 (Washington, DC: U.S. Government Printing

Office, 1989), app. table 2-18; and National Science Foundation, Graduate Student Suppoti and Manpower Resources in

Graduate Science Education, Fall 1969, NSF 70-40 (Washington DC: September 1970), table C-1 la.

nisms appears to alter the decision to pursue

graduate study .15

Employment of Researchers

Since 1980, NSF estimates that the total s/e work

force (baccalaureate, master’s, and Ph.D. degree

recipients) has grown at 7.8 percent per year, which

is four times the annual rate of total employment of

1.8 percent. Scientists and engineers represented 2.4

percent of the U.S. work force in 1976 and 4.1

percent in 1988.16 Almost 2.0 million scientists and

2.6 million engineers were employed in the s/e work

force in 1988. In addition, almost 25 percent of all

scientists and 10 percent of all engineers were

employed in non-s/e jobs in 1988. At the doctoral

level, scientists numbered 351,000, which is five

times the engineers at 68,000. Total employment for

doctoral scientists and engineers grew by nearly 5

percent per year from 1981 to 1987.17 The percentage of foreign nationals who remain in the United

States after receiving their Ph.D.s remained at

roughly 50 percent through the latter half of the

1980s.

A pivotal employment sector for Ph.D. s/e researchers is academia. From 1977 to 1987, the

number of Ph.D. scientists and engineers engaged in

academic research increased by 65 percent. Figure

7-5 shows that life scientists accounted for one in

three doctoral scientists and engineers on campus, a

proportion unchanged in a decade, while figure 7-6

indicates average annual growth rates by field, with

computer and information scientists leading the

way.

The academic research work force in 1987 was 90

percent white (both sexes) and 84 percent male.

Overall participation in academic research by minorities is bifurcated-9 percent is Asian and expanding, 2 percent is Black and Hispanic and barely

inching upward. The most encouraging statistics are

for Black women who, in 1987, represented 31

percent of Black Ph.D. scientists doing research in

the academic sector.

NSF estimates a 51-percent increase, from 1977

to 1987, in the number of s/e doctorates engaged in

basic research, regardless of employment sector.

Four out of five (79 percent) worked in academia in

1987 (see figure 7-7); industry employs 8.6 percent;

the Federal Government employs 6.7 percent; nonprofit institutions support 3.5 percent; and other

groups employ the final 2.6 percent.

1%.J.s. Congress, OffIce of Technology Assessment, Educating Scientists and Engineers: Grade School to Grad School, OTA-SET-377 (Was~o~

DC: U.S. Government Prinfig oftlce, June 1988), p. 88. The administration’s fiscal year 1992 budget, however, proposes to consolidate graduate student

financial aid programs at the Department of Education into a single National Graduate Fellowships Program (NGFP) and place it under the discretionary

authority of the Secretary of Education. Included under NGFP is the Gradwte Assistance in Areas of National Need Program to support physical science

and engineering students ($25 million in fiscal year 1991). See ‘‘FY 1992 Budgets for Social and Behavioral Scienee Researeh” COSSA Washington

Update, vol. 10, Mar. 4, 1991, pp. 10-11.

l’5Natio~ Science Board, op. cit., footnote 4, p. 67.

171bid., p. 116.
210 ● Federally Funded Research: Decisions for a Decade

Box 7-A—institutional Variations on National Trends: Graduate Enrollments at

Four Research Universities

While national trends in graduate enrollments, demography, support, and distribution by field paint the “big

picture,’ they depersonalize and often mask how institutions (and their sponsors) influence those destined to join

the research work force. Profiles of graduate student enrollments at four research universities-two public, two

private-provide comparisons among key characteristics.1

In the 1980s, graduate enrollments grew by 18 percent nationally. By broad field, enrollments have, in

percentage terms, grown steadily in engineering and mathematics/computer science, decreased slightly in the life

sciences and more markedly in the social sciences, and been stable in psychology and the environmental sciences.

But these national trends are not mirrored at the four universities examin ed by OTA: the University of Houston,

the University of Califonia-Santa Barbara (UC-Santa Barbara), Carnegie-Mellon University, and the Massachusetts Institute of Technology (MIT) (see table 7A-l).

Growth in enrollments from 1980 to 1988 range from 43 percent at Carnegie-Mellon to 13 percent at UC-Santa

Barbara. MIT’s enrollment declined by 2 percent. The University of Houston, while increasing its graduate student

population by one-third, experienced the largest growth in mathematics/computer science. At Carnegie-Mellon,

there was virtually no change in the distribution by broad field during the decade. At MIT, engineering enrollments

declined (but over one-half of all graduate students there are pursuing engineering degrees), and at UC-Santa

Barbara the number of mathematics/computer science students nearly tripled.

Examined in terms of demographic characteristics, the graduate student populations at all four universities

reflect national trends, but at different levels:

. Nationally, the proportion of women is up slightly to 32 percent. At the four universities highlighted here

the trend is similar, but enrollments of women averaged one-quarter of all science and engineering (s/e)

students.

. Foreign nationals comprise almost one-half the graduate s/e students at Houston, and one-third at MIT and

Carnegie-Mellon. Nationally, foreign students were 26 percent of the graduate student population in 1988.

. Among U.S. citizens, minorities represent one-quarter at Houston, but only 13 to 14 percent at the other three

universities. The national average, unchanged since 1983, was 18 percent.2

In actuality, little is known about Ph.D. supply. 3 In the words of the National Research Council: “Basic

descriptive statistics such as the percent of entering doctoral students who never complete the degree are unknown.

More complicated issues such as determinants of degree completion (e.g., financial support, family responsibilities,

demography, and time to complete the degree) remain unanswered. ” 4

National trends in graduate student

enrollments tell only part of the story of factors affecting the renewal of human resources in science and engineering.

l~e ~n ~o~~ &low ~won Wpubfished Natio~ science Foundation data com@led by the Divition of Sciem R~o~s

studies.

2~mtlo~ dam Me tiwn from Natio~ Science Board, Science & Engineering Iti’caters---l989 (wxh@o~ ~:1989)! PP.

215-216, tables 2-7 and 2-8.

3~s ~amPle of fou in~ti~tiom alo~ mgg~fi tit research-intensive universities may under-enroll women ~ U.S. *riti@ ~

over-enroll foreign nationals m gmduate science and engineah g study relative to national trends. OTA’s analysis of Univcmity of Michigan

and Stanford University is consistent with these findings across all fields of science and engheaing as well, with women compromising 28 and

22 percent, res~tively, and foreign nationals 34 and 30 pe~en$ of graduate enrollmems. But generalizations are premature until more

systematic analysis is undertaken.

4~a Fahtm, ~mutive dir~tor, Off@ of Scientific and ~gintig Pemonnel, Natio~ ‘e

search Cound, personal communication,

Oct. 23, 1990.

Fields vary in their dependence on sectors of of all engineers and 25 percent of computer scientists

employment. Basic researchers with a Ph.D. in some doing basic research in 1987. The field experiencing

fields-mathematics, sociology/anthropology, and the largest percentage increase in industrial employeconomics— are employed almost exclusively in ment during the decade was the life sciences (in large

academia. Industry, in contrast, employed 19 percent part due to the biotechnology boom) .18

lg~id. me NatiO~ science FOUn~tiOn alSO reports that retention of male doctoral scientists and engineers is the highest for tie business/industry

and university/college sectors, around 80 percent, after 14 years (see pp. 117-118 for a discussion).
Chapter 7—Human Resources for the Research Work Force ● 211

Table 7A-l-Graduate Enrollment In Science and Engineering at Selected Universities, by Field: 1980-88

National totals

Math/ EnvironTotal social Life computer Physical mental

Year students Engineering sciences sciences science Psychology sciences sciences

1980 . . . . . . . . . . . . 208,232 20.1 % 27.7% 21 .8% 7.3% 10.4Y0 10.7% 4.9%

1983 . . . . . . . . . . . . 223,135 23.6 21.5 20.0 8.8 9.9 11.0 5.2

1986 . . . . . . . . . . . . 236,741 24.9 19.9 19.2 10.6 9.3 11.4 4.6

1988 . . . . . . . . . . . . 245,463 25.1 20.1 19.1 10.8 9.5 11.3 4.0

University of Houston

Math/ EnvironTotal social Life computer Physical mental

Year students Engineering sciences sciences science Psychology sciences sciences

1980 . . . . . . . . . . . . 766 24.80/o 12.70/o 20.3% 9.5% 1 5.5% 14.3% 2.7%

1983 . . . . . . . . . . . . 864 31.1 9.6 16.9 13.0 5.7 18.9 4.9

1986 . . . . . . . . . . . . 1,165 29.0 9.5 13.6 13.7 14.2 16.5 3.4

1988 . . . . . . . . . . . . 1,017 29,1 9.5 14.2 13.1 15.5 15.6 2.9

Carnegie-Mellon University

Math/ EnvironTotal Social Life computer Physical mental

Year students Engineering Sciences sciences science Psychology sciences sciences

1980 . . . . . . . . . . . . 796 47.4% 18.80/0 2.9% 18.20/! 1.60/0 10.9% 0.0%

1983 . . . . . . . . . . . . 965 44.0 21.7 3.2 18.8 2.1 10.3 0.0

1986 . . . . . . . . . . . . 1,115 46.4 17.9 3.3 20.3 1.7 10.4 0.0

1988 . . . . . . . . . . . . 1,137 45.8 18.6 3.5 19.5 2.5 10.0 0.0

Massachusetts Institute of Technology

Math/ EnvironTotal Social Life computer Physical

Year

mental

students Engineering Sciences sciences science Psychology sciences sciences

1980 . . . . . . . . . . . . 3,904 62.0°/0 10.2% 7.5% 3.0% 0.8% 12.9% 3.5%

1983 . . . . . . . . . . . . 3,795 61.6 10.8 7.1 2.9 0.9 13.0 3.6

1986 . . . . . . . . . . . . 3,925 56.2 11.6 6.4 8.1 1.1 13.0 3.5

1988 . . . . . . . . . . . . 3,827 54.0 11.9 6.3 3.5 1.4 13.9 3.5

University of California-Santa Barbara

Math/ EnvironTotal Social Life computer Physical mental

Year students Engineering sciences sciences science Psychology sciences sciences

1980 . . . . . . . . . . . . 1,142 25.20/. 27.8% 17.40/. 4.9% 4.5% 14.5% 5.6%

1983 . . . . . . . . . . . . 1,178 27.6 25.6 16.0 4.8 4.1 16.1 5.9

1986 . . . . . . . . . . . . 1,262 24.2 25.0 14.4 12.0 4.3 15.1 4.9

1988 . . . . . . . . . . . . 1,293 28.3 22.0 14.6 13.4 3.7 13.9 4.0

NOTE: Full-time students only.

SOURCE: National Science Board, Science & Engineering Indicators--1989 (Washington, DC: U.S. Government Printing Office, 1989), table 2-16,

p. 228; and National science Foundation, Institutional Profiles of the University of Houston, Carnegie-Mellon University, Massachusetts

Institute of Technology, and University of California-Santa Barbara, unpublished information, 1989.

Complicating the understanding of employment employment markets.19 The number of Ph.D.s taktrends is that temporary, 1- to 2-year appointments ing postdoctoral positions in U.S. universities has

are a tradition in some fields. Postdoctoral appoint- grown 5 percent annually since 1980. The availabilments are used both to augment the specialized skills ity of these appointments expand with academic

acquired in doctoral study and to wait out poor research budgets, and over one-half of these posi1 9Traditionally, the postdoctoral appointment is for 1 to 2 years. The last major national study of postdoctorates, however, is a decade old. This issue

needs to be revisited empirically. For a national perspective, see Nationa1 Research Council, Postdoctoral Appointments and Disappointments

(Washington, DC: National Academy Press, 1981). For a first-person perspective on how circumstances maybe changing, see Edward J. Hackett,

“Science as a Vocation in the 1990s,” Journal of Higher Education, vol. 61, May/June 1990, pp. 241-279.
212 . Federally Funded Research: Decisions for a Decade

Figure 7-5-Distribution of Doctoral Scientists and

Engineers in Academic R&D, by Field:

1977 and 1987 (in percent)

O 5 10 15 20 25 30 35

Life

scientists

Physical

scientists

Social

scientists

Engineers

Psychologists

Mathematical

scientists

Environmental

scientists

Computer/

information

specialists

O 5 10 15 20 25 30 35

 1977 1987

SOURCE: National Science Board, & lndkators1989, 89-1 (Washington, DC: U. S.-

Government Printing

Office, 1989), table 5-17.

tions are located in the life sciences. Foreign citizens

have been increasing their postdoctoral appointments at a rate twice that of U.S. citizens in the last

decade. 20 Also, universities have increased the

number of available nonfaculty research positions,

and that the number of nonfaculty researchers as a

percentage of the total number of Ph.D.s in s/e

employed in academia rose from 14.8 percent in

1977 to 17.5 percent in 1987 (see box 7-B).

In sum, there is a steady stream of new entrants to

the research work force. Almost 14,000 s/e Ph.D.s

are granted each year by U.S. universities to U.S.

citizens (nearly 13,000) or to foreign citizens who

are permanent residents (over 1,000). Industry and

academia have increased their employment of

Ph.D.s in s/e over the past two decades, and by rates

Figure 7-6-Average Annual Percentage Growth

Rate of Doctoral Scientists and Engineers in

Academic R&D, by Field: 1977-87

In percent

O 2 4 6 8 10 12

Total scientists

and engineers

Total scientists

Engineers

Computer/

information

specialists

Social

scientists

Psychologists

Life scientists

Environmental

scientists

Mathematical

scientists

Physical

s

Cientists 

O 2 4 6 

SOURCE: National Science Board, & Engineering lndtitors7989, 89-1 (Washington, DC: U.S. Government Printing

Office, 1989), table 5-17.

exceeding 5 percent per year in the 1980s. In

addition, relatively temporary university positions,

such as postdoctoral and nontenure-track research

slots, have increased.

Historically, the Federal Government has played

both a direct and indirect role in the production and

employment of s/e Ph.D.s. Both as the primary

supporter of graduate student salaries and tuition,

and as an employer through the Federal laboratories

and mainly through research grants, the Federal

Government has perhaps the largest role in the s/e

Ph.D. labor market. With changing demographics

and demands on the research component of this labor

 Op. 4, p. 54. There is both a “push” and “pull” factor operating the postdoctorates by non-us.

citizens with They may be ineligible for employment in some sectors, e.g., defense, and they can be productive researchers while awaiting a

change in visa status from temporary to ent. S. 358, passed in the Congress, would allow the annual number of employment-based visas

to increase from 54,000 to 140,000. Up to 40,000 visas are reserved for academicians and others with “extraordinary” ability to work in the United States.

An annual cap of 65,000 H-1, or temporary professional, visas was also imposed. See Janice Long, “U.S. Immigration Eased for Professionals,”

Chemical & News, vol. 68, No. 47, Nov. 19, 1990, p. 13.
Chapter 7—Human Resources for the Research Work Force . 213

Figure 7-7—Employment of Doctoral Scientists and

Engineers in Basic Research, by Sector: 1987

Total employed: 133,345

Other 3%

SOURCE: National Science Board, Sdence & Engineering krdicafors1989, NSB 89-1 (Washington, DC: U.S. Government Printing

Office, 1989), app. table 5-19.

force, the Federal Government could redefine that

role as more or less interventionist in the 1990s.

The Shape of the Future Research

Work Force

In recent years, the scientific community and

some Federal research agencies have intensified the

call for increased support of human resources. One

emphasis has been on the educational ‘pipeline"—

how to attract more elementary and secondary

school students to science, mathematics, and engineering. 21 Since the school-aged population will

begin to grow with the second baby boom in the

mid-1990s, most research agencies have initiated

programs that emphasize earlier stages in scientific

education, especially at the secondary and undergraduate levels.22 Not only will this population

grow, but a larger proportion of it will consist of

racial and ethnic minorities.

The Uncertainty of Ph.D. Projections

Another focus of concern is the state of graduate

education in s/e. Some recent reports have projected

that, in the 1990s, many scientific fields will

experience shortages in the supply of Ph.D. researchers.23 Based on demographic characteristics

alone, NSF has estimated that the number of new

Ph.D.s awarded in the natural sciences and engineering by U.S. universities (to U.S. citizens and foreign

nationals) would rise from roughly 14,450 in 1988

to 15,600 in 1993, but then would decrease to 14,200

by the year 2010. This projection, based on a

predicted ‘‘shortfall’ in natural science and engineering baccalaureate degrees, assumes little change

in the proportion of U.S. doctorate-seeking students

and that the number of Ph.D.s awarded to foreign

nationals remains at 4,500 per year.24

To convert these figures into a future supply of

Ph.D.s for the scientific labor market, one must

assume that some proportion of foreign nationals

will seek to remain in the United States for

employment. Most estimates assume that the current

level of 50 percent will hold throughout the 1990s,

while noting that increased scientific sophistication

of these students’ native countries may eventually

draw a larger proportion of them back home.

If the current demand for Ph.D.s in academia and

industry, at over 12,000 per year, were to remain

constant, then the aggregate supply of Ph.D.s in the

1990s would be more than adequate. (This would not

mean, of course, that the distribution among s/e

ZIFor a review of the factors influencing the early stages of the pipeline, see U.S. Congress, OffIce of ‘lkchnology Assessmen4 EZementaV ad

Seconahry Educationfor Science and Engineering, OTA-TM-SET=ll (Washington DC: U.S. Governrn ent Printing Offlce, December 1988); and D.E.

Chubin, “Misinformation and the Recruitment of Students to Science,”Bioscience, vol. 40, No. 7, July/August 1990, pp. 524-526.

zz~e Natio~ Science Foundation has ken particularly active. See Erich Bloc~ ‘‘Education and Human Resources at the National Science

Foundation” Science, vol. 249, Aug. 24, 1990, pp. 839-840; National Science Foundation “Background Material for Long-Range Planning:

1992- 1996,” NSB-90-81, prepared for a meeting of the National Science Board, June 14-15, 1990, p, A-5; Ward Worthy, “Research Universities Pay

More Heed to Freshman Science,” Chemical& Engineering News, vol. 68, No. 18, Apr. 30, 1990, pp. 27-28; and Mac WWlkenburg, “Tinning Off

Students: Our Gatekeeper Courses,” Engineering Education, vol. 80, No. 6, September/October 1990, p. 620. For a new conceptualization of

undergraduate recruitment andretentio~ see Sheila Tobias, They’re Not Dumb, They’re Diflerent: Stalh”ng the Second Tier (Thcson, AZ: Research Corp.,

1990).

23For e~ple, see hmdation of American Universities, op. cit., footnote 13; Jtice UW, “Changes in Immigration Law Eyed lb Avert Shortage

of U.S. Scientists,” Chemical& Engineering News, vol. 68, No. 34, Aug. 20, 1990, pp. 19-20; and Richard C. Atkinsou “Supply and Demand for

Scientists and Engineers: A Nationrd Crisis in the Making,’ Science, vol. 246, Apr. 27, 1990, pp. 425-432. Labor economist Michael Finn has noted

that “shortage’ is a relative concept. “Increasing shortage of scientists and engineers’ means that they will be harder to fmd than they are now, or were

intherecent past. The difilculty in measuring shortage is that hiring standards and personnel budgets adapt to supply and demand conditions. See Michael

G. FinrL “Personnel Shortage in Your Future?” Research-Technology Management, vol. 34, No. 1, January-February 1991, pp. 24-27.

Ucited ~At~on, op. cit., footnote 23.
214 . Federally Funded Research: Decisions for a Decade

Box 7-B—The Unfaculty: Who Are They and Why Should We Worry?

Graduate student enrollment increased rapidly during the 1960s, followed by a decline in the rate of increase

in the 1970s and early 1980s. Because undergraduate and graduate student enrollment levels are closely associated

with funding available for faculty salaries, it is not surprising that there was a decline in faculty job openings and

a rise of ‘‘academic marginals ” or research professionals, most possessing the Ph.D. but lacking a faculty

appointment. This cadre has also been referred to as the “unfaculty,’ ‘‘unequal peers,” or “research associates. ”

In an effort to maintain an appropriate research base in the face of increasingly tight budgets, universities employ

academic marginals on short-term contracts. These new nonfaculty positions do not depend on enrollment levels

and afford the university flexibility in fulfilling its research needs since marginal positions are more readily emptied

and reallocated than are tenured and tenure-track faculty. 1

Who are these academic marginals? Many are postdoctoral fellows who, unable to secure faculty positions,

remain in the university setting for an indefinite period of time. These positions might be viewed as extensions of

the scientific apprenticeship system, which includes graduate education and postdoctoral training.z Of the fiscal year

1972 graduates who had taken postdoctoral appointments, approximately one-third had prolonged their

appointments because they could not find other desirable employment.3 Consequently, these professional research

scientists tend to be highly qualified and capable, most earning Ph.D. s from reputable research institutions, have

impressive publication records, and are supported by National Institutes of Health and National Science Foundation

(NSF) grants.4 Yet, academic marginals are not recognized as full faculty members; they receive none of the

amenities and privileges of faculty and, more important, are ineligible for tenure.

The inferior status associated with the unfaculty fosters negative feelings and tensions within the research

community. Comparatively low salaries, little job security, and limited ‘‘rights’ to laboratory and office space or

seed money and equipment all contribute to an environment in which the marginal scientist commands little respect

from his or her full faculty peers. In fact, academic marginals often are dependent on faculty to provide part-time

teaching or research that augments employment. Similarly, the academic marginal might find it difficult to establish

him or herself in the scientific community at large: lacking an established laboratory and the accompanying prestige,

marginal scientists have a ‘‘. . . longer row to hoe than most . . . [and] have to be more perfect.

Standards vary on the research status of unfaculty and are a source of debate on many campuses. For example,

research associates at Stanford University cannot act as principal investigators on research grants, which prohibits

IU.S. Congress, OffkZ of lkc&moIogy A.ssessxneng Higher Education for Science and Engineen’ng, OTA-BP-SET-52 ~asuomw:

US. Government Printing Office, March 1989), p. 176,

~ward J. Hacket~ “Science in the Steady State: The Changing Reseamh University and Federal Funding,” OTA contractor report,

1987, p. 18. Available through the National Technical Information Service, #PB 88-177 928/AS.

3Natio~ Res~h counci~ Postdoctor& Appoin~nts and Disappointments (Washington, ~: Natiom Academy ms, 1981). p- 225.

d~cke~ op. Clt,, footnote 2; and ARXII ~. Te~Cb ‘‘Re search Centers and Non-Faculty Reseaders: A New Academic Role,” Research

in the Age oj the .Sready-Srare Um”versiry, Don Phillips and Benjamin Shen (eds.) (Boulder, CO: Westview, 1982), p. 100.

5S& ~w~d J. H~ketg “science as a vocation in the 1990’ s,’ Journal of Higher Education, vol. 61, May/June 1990, pp. 253-254.

fields would be ‘‘correct. ’ However, these calcula- ments and deaths; 2) rising college and university

tions further project that the demand for Ph.D.s, enrollments in the mid-to-late 1990s to accommowhich has been rising in the 1980s, will continue to date the second baby boom (therefore requiring

increase. Several factors are cited: 1) replacement of more faculty); and 3) increasing Federal and induscurrently employed s/e researchers due to retire- trial investment in R&D, if only at a moderate rate. 25

25Rese~ch ufiver5itics ~ve he resowces to plaD for possible discontinuities in faculty age ~d tenure s~~s. ofim categories of fititutiom~

especially comprehensive universities and 4-year colleges, wilt have less lati~de in coping with imbalanms in faculty supply and demand, which may

be exacerbated with recision of the mandatory retirement law in 1993. Some alternatives, including the consolidation or elimination of departments and

how, if at all, retiring faculty will be replaced, are discussed in Marcia Barinaga, “Howard Schachmrm Fights Retirement,” Science, vol. 249, Sept. 14,

1990, pp. 1235, 1237; Constance B. Bouchard, “The ‘Lost Generation’ of Scholars Can Help Colleges Avoid the borning Faculty Shortage, ” The

Chronicle of f-figher Education, vol. 37, No. 12, Nov. 21, 1990, pp. B1-B2; and Courtney Leathe~ “End of Mandatory Retirement Policies Seen

Having Little Effect on Professors,” The Chronicle of Higher Education, vol. 37, No. 17, Jan. 9, 1991, pp. A13-A14.
Chapter 7—Human Resources for the Research Work Force . 215

them from gaining a history of funding with any Federal research agency. (This arrangement is fairly typical for

research associate positions across the United States.)

It is possible that academic marginals are merely the product of a research environment that is shifting its focus

and reevaluating its needs. But evidence that women have long been marginalized---less likely than men to land

faculty posts or receive tenure once in them--in scientific research is unequivocal.s So funding pressures alone

cannot explain swelling of the ranks of the unfaculty. Like the rest of the research work force, a panoply of

demographic and funding changes may redefine the perceived needs of research universities. The status of those

funded entirely on soft money may be reassessed as the costs of sustaining research units are scrutinized by academic

administrators.

Surprisingly, national data beyond per-investigator costs are scarce on various ‘ ‘production’ units Table 7B-1—Research Personnel at 4-Year Colleges

of research. This includes the extent to which the ranks and Universities (based on self-identification)

of academic marginals are growing. Unpublished data

on research personnel at 4-year colleges and universi- 1977 1981 1987

ties compiled by NSF (which warns that the data may Ph.D.s with faculty rank 85.2% 83.90/o 82.5%

not be comparable for different years) are presented in (assistant, associate, full)

table 7B-1. Over the period 1977 to 1987, the proportion Ph.D.s without facuity rank 14.80/0 16.1 % 1 7.5%

(includes postdoctorates)

of Ph.D. s without faculty rank grew slightly. Growth Totai number 157,000 179,000 209,000 in the number of unfaculty, up 15,000 in the decade to

a total of 37,000, suggests that this is a sizable reserve SOURCE: National Science Foundation, Science Resources Studies

Division, unpublished data, July 1990.

research labor force to augment university faculty

capabilities. Beyond that, little is known about:

1. the cost to the Federal Government, which pays the salaries of unfaculty who are supported on research

grants;

2. the career paths and prospects of the unfaculty; and

3. the possible mobilization of the unfaculty in the face of impending faculty retirements in the 1990s and

projected shortages of Ph.D. scientists and engineers early in the next century.

Until better information is collected and analyzed on the unfaculty, Federal policymakers must worry about

who they are and how they affect the university research economy of the 1990s.

6see Wgwt W. Rossiter, Women Scientists in America: Struggles and Strategies to 1940 (Baltimore, MD: m JObQS HOPkiQS

Uruversity Press, 1982); MM G. Abir-Am and Dorinda Outram (eds.), Uneasy Careers andrntimate Lives: Women in Science, 1789-1979 (New

Brunswick NJ: Rutgers University press, 1987); and Vivian Gornic~ Women in Science: 100 Journeyslnro the Territory, revised ed. (New York,

NY: Simon & Schuster/Touch.stOne, 1990).

However, OTA cautions those who wish to use can be lured or discouraged to relieve shortages and

these projections to predict future supply and surpluses. 2 7

demand for researchers. Projections of shortages and

In addition, predicting the demand for academic surpluses in the Ph.D. s/e work force are notoriously

unreliable. 26 As OTA recently concluded: researchers is extremely complex. 28 As discussed

above, such predictions must include demographic

The job market for Ph.D.s is unusual. While it changes, immigration trends, anticipated retireresponds to demand (in particular, national R&D ments, and the orientations and intentions of new

funding) and to immediate research and training entrants to the research work force, as well as

support, the supply is particularly sensitive to shifting Federal priorities and available research

Federal policies. As for quality, at the margins talent funding. All of these are subject to change, and may

ZGOTA reached this conclusion on the basis of examin”mg various models of academic and industrial markets. See Office of Twhnology Assessment

op. cit., footnote 12, especially chs. 3 and 4. Recent independent confirmation of this conclusion comes in a critique of a National Science Foundation

model developed by its Policy Research and Analysis Division. See Alan Fechter, “Engineering Shortages and Shortfalls: Myths and Realities, ” The

Bridge, vol. 20, fall 1990, pp. 16-20.

z70ffice of TNIUIOIOO Assessment op. cit., footnote 2, p. 160.

%tkd I.K. Youn, “Studies of Academic Markets and Careers: An Historical Review,” Academic Labor Markets and Careers, David W. Breneman

and Tbd I.K. Youn (eds.) (Philadelphi~ PA: The Falrner Press, 1988), pp. 8-27.
216 . Federally Funded Research: Decisions for a Decade

vary by type of educational institution one attends,

the field of Ph.D., the region of the country in which

one seeks employment, and so on. Sorting these

factors compounds the burdens with which policymakers and educators must cope.

In general, projections also do not take into

account market adjustments:

Most of the simulation models used to assess

these labor markets assume . . . that if an imbalance

occurs between supply and demand, nothing will

occur to correct it. In fact, history demonstrates that

these labor markets do tend to equilibrate. Thus,

projected imbalances derived from such modelsboth shortages and surpluses--are always overstatements of what actually will be experienced.29

Finally, projections of future retirements and job

availability in industry vary to a large extent by field.

For instance, some expect faculty shortages to be

higher in the humanities and social sciences than in

the mathematical and physical sciences, with no

shortages foreseen in the biological and behavioral

sciences. 30 Also, the health of the pharmaceutical

and related industries has a large effect on employment prospects for Ph.D.s in medical and chemical

fields. These factors must be taken into account by

Federal policies that address projections of shortages. As science policy statesman Harvey Brooks

puts it:

Most projections are based on extrapolations of

recent history, usually considering only first derivatives, with little attention to second derivatives,

which cannot be accurately estimated anyway. In

fact the projection type of exercise has more often

than not contributed to the tendency of the technical

manpower production system to overreact, building

up alternate surpluses and deficits owing to the

delayed response of the educational pipeline to the

conditions in the market.31

Given the uncertainty of projections, OTA finds

that concentration on the preparedness of the pipeline to produce Ph.D.s (i.e., increasing the number of

undergraduates earning baccalaureates in s/e) is the

most flexible policy .32 If shortages begin to occur in

a particular field, prepared undergraduates could be

induced by increased graduate support to pursue a

Ph.D. in that research area, and Ph.D. production

would increase 4 to 7 years later. In addition, those

scientists who would have otherwise left the field

might stay longer, those who had already left might

return, and graduate students in nearby fields could

migrate to the field experiencing a shortage.33

(These are all signals of opportunity sent by the

market.)

If shortages do not materialize, then the Nation’s

work force would be enhanced by the availability of

a larger number of highly skilled workers. Research

in the United States would also benefit by the

training of a larger number of baccalaureates in the

sciences, a significant percentage of whom will

choose to pursue scientific careers regardless of

predictions of shortages, while others contribute

their acquired knowledge to other occupations.

Concerns about the demographics of Ph.D. recipients could also be addressed. Laws that prohibit

discrimin ation, such as Title VI of the Civil Rights

Act of 1964 and Title IX of the Education Amendments of 1972, justify support to groups defined by

the ascribed characteristics of race/ethnicity and sex,

z9Fechter, op. cit., footnote 26, p. 19. AlSO see Eli Ginzberg, National Research Council, Office of Scientific and Engimwing Personnel, “Scientic

and Engineering Personnel: Lessons and Policy Directions, ” The Impact of D@ense Spending in Nondefense Engineering Labor Markets (Washingto~

DC: National Academy Press, 1986), pp. 25=.12.

~or the academic administrator perspective, see Elaine E1-Khawas, Camp~ Trends-1988 (Washington, DC: American Council on Educatio~

1988); for a different perspective, see William G. Bowen and Julie Ann SOS% Prospects for Faculty in the Arts and Sciences (PrinwtoU NJ: Princeton

University Press, 1989). On the di.fllculty of applying nationat projections to regional, institutional, and field-spec~lc academic markets, see Debra Bl~

“Many Studies of Future Academic Job Market Are Said To Be of Little Use to Policymakers,’ The Chronicle of Higher Education, vol. 37, No. 23,

Feb. 20, 1991, pp. A15, A19.

3%rvey Brooks, “Are Scientists Obsolete?,” Science, vol. 186, Nov. 8, 1974, p. 503. Also see Constance HoldeU “Do We Need More Ph.D.s, Or

Is Fewer Really Better?” Science, vol. 251, Mar. 1, 1991, pp. 1017-1018.

szFrom a mmket per~ctive, Ph.D.s are skilled workers who experience low rates of unemployment. However, the Ph.D. worker W5 ~ terms of

career paths. “Underemployment” means that the person is part of the Nation’s labor force, but not working in a position that takes full advantage of

his or her training and skills. The fi~ in short, is not always good.

ssReca~ that25 percent of scientists are in nonscien~ and engineering jobs. This is a huge potential reservoir corresponding to fluctuations indema.nd.

In general, people with science and engineering training are much easier to convert to other occupations than the reverse.
Chapter 7—Human Resources for the Research Work Force ● 217

respectively .34 The 1980 reauthorization of

National Science Foundation also created the

ence and Technology Equal Opportunities

(Public Law 96-516). In it, Congress declared:

the

SciAct

. . . that the highest quality science over the long

term requires substantial support, from currently

available research and education funds, for increased

participation in science and technology by women

and minorities. The Congress further declares that

the impact on women and minorities which is

produced by advances in science and technology

must be included as essential factors in national and

international science, technology, and economic

policies.35

With reauthorization of the Higher Education Act

of 1965 scheduled for the 102d Congress, expectations are high that “significant and bold changes”

will increase Federal aid to higher education institutions and students, especially historically Black

colleges and universities. 36 This legislation empowers Congress and the research agencies that have

devised programs targeted to enhance the participation of women and minorities in science to do even

more.

37 Programs targeted to minorities, women, the

physically disabled, and students in areas of the

country where access to research institutions is

limited, could help to expand the pool of potential

scientists .38

Training for an Uncertain Research System

The unity of research and graduate teaching in

U.S. higher education has sustained a vigor and

creativity in research that is unparalleled in the

world. The training of graduate students is also

linked to the instruction of undergraduates in s/e.

This section first looks at the connection between

undergraduate teaching and research and then at the

traditional academic research model and some

alternatives.

Research and Undergraduate Teaching

Calls for a‘ ‘new paradigm’ for higher education

in the 21st century are now emanating from the

presidents of research universities.39 Most of these

reforms call for improved undergraduate education

and ‘‘. . . a better balance between research and

teaching. ’ A related need may also be to change

the reward system of the university, since asking

universities to augment the teaching of undergraduates may be misplaced if faculty continue to view

this as a drain on their time that would be better spent

doing research. This tension between the time spent

on research and teaching at the major research

universities and the use of graduate assistants as

instructors for many lower level undergraduate

~Con~oversy e~pted in December 1990 over a Department of Education ruling that ‘‘race-exclusive” scholmships me disc riminatoxy and should

be disallowed. This is seen as an assault on Title VI. Were such a ruling ever upheld, Federal aid and therefore college attendance by minority students

could be jeopardized. See Scott Jaschik, “Scholarships Set Up for Mi.nority Students Are Called Illegal,” The Chronicle cfl?igher Education, vol. 37,

No. 15, Dec. 12, 1990, pp. Al, A20; Kenneth J. Cooper, ‘‘Administration Revises Race-Based Grant Rule,’The Washington Post, Dec. 19, 1990, pp.

Al, A8; Ruth Marcus, “New Scholarship Ruling Caught in Ugal Cross-Fire,” The Washington Post, Dec. 18, 1990, p. A8; and Julie A. Miller,

“Alexander Vows to Rescind Policy on Scholarships,’ Education Week, Feb. 13, 1991, pp. 25,28.

ssNatio~ Science Foundation Authorization and Science and ‘Ikchnology Equal Opportunities Act, Dec. 12, 1980, 94 Stit. 3011.

Sbsee ThomU J. De~ug~, ‘ ‘colleges welcome News That Rep. Ford Will Chair House PostSecondary Education Subcommittee, ’ The Chronicle

of Higher Education, vol. 37, No. 12, Nov. 21, 1990, p. A23.

sTOffice of TwhnoIogy Assessmen~ op. cit., footnote 15, app. B.

sgIt is CIW tit m~ket forces alone will not increase the participation of wome~ minorities, and the physically disabled h science and en@eer@.

Policy intervention is required. Therefore, there is no inconsistency between letting the market operate for some segments of the student population while

targeting other segments through recruitment and retention programs. For examples, see Linda Dix (cd.), Minorities: Their Underrepresentation and

CareerDiflerentials in Science andEngineering, workshop proceedings, National Research Council (Washington DC: National Academy Press, 1987);

and Cynthia Lollar, “Access to Engineering: New Project for Students and Faculty With Disabilities,”Science, vol. 251, Feb. 22, 1991, p, 952.

Sgprominent among &em me the two institutions that OTA studied as case examples, Stanford Univemity and University of Michig~. See Karen

Grassmuck, “Some Research Universities Contemplate Sweeping Changes, Ranging From Management and lknure to T~ching Methods,” The

Chronicle ofHigherEducation, vol. 37, No. 2, Sept. 12, 1990, pp. Al, A29-31. Stanford’ sreforms include various programs, most funded by a$5 million

gift to the university, “.. . designed to give faculty members ranging from graduate teaching assistants to senior professors better incentives to

concentrate on effective instruction. ’ See Kenneth J. Cooper, “Stanford President Sets Initiative on Teaching,” The Washington Post, Mar. 3, 1991,

p. A12.

~~assmuc~ Op. cit., foo~ote 39, p. A29. Aso see Courtney tiatherm~‘Defiition of Faculty Scholarship Must Be Expanded to hclude ~ac~g,

Carnegie Foundation Says, ” The Chronicle of Higher Education, vol. 37, No. 14, Dec. 5, 1990, pp. Al, A16-A17; and Alliance for Undergraduate

Educatio~ The Freshman Year in Science and Engineering: Old Problems, New Perspectives for Research Universities (University Park PA: 1990).
218 . Federally Funded Research: Decisions for a Decade

Photo credit: Bob Kalmbach

Students at the University of Michigan walk on campus

between classes. Undergraduate teaching is an important

part of faculty responsibilities.

classes has caused some to question the health of

undergraduate teaching at these universities.41

Another concern that stems from the tension

between research and teaching is the relation between providing more funds for basic research and

improving both the institution’s research performance and teaching capability. A common perception

during the 1960s was that Federal dollars that

supported research also benefited undergraduate

teaching because these top researchers would communicate their excitement about developments ‘‘at

the laboratory bench” to undergraduate and graduate students alike. In the 1980s, with the separation

between research and undergraduate education becoming more pronounced at many research institutions (Particularly with many faculty “buying out”

of teaching responsibilities when awarded a large

research grant), the connection between research

progress and the cultivation of human resources

grew more tenuous.42

Consequently, many research agencies see a

larger need for funding undergraduate teaching

directly. In addition, many faculty have proposed

novel ideas. For example:

What if the four-year colleges. . . began requiring

the university departments whose doctoral students

apply for jobs at the four-year colleges to provide a

detailed description of how they had prepared those

candidates to teach, as well as specific evaluations of

their teaching skills? . . . Granted, the most selective

graduate departments at the top research universities-those that aspire chiefly to staff the faculties of”

other prestigious universities-might not be especially responsive. . . . Although institutions routinely “raid” each other for distinguished researchers, they hardly ever pursue outstanding

teachers so aggressively. As long as that disparity

exists, talented teachers will be captives of their

current employers, with little leverage to extract

greater rewards.43

Indeed, growth in the employment of Ph.D.s by

4-year institutions has been hearty for over a

decade. 44

 to junior 

they are taught primarily by faculty. The utilization of graduate students for teaching posts, however, can be quite valuable from the graduate student’s

prospective, since these classes may represent one of the few opportunities to (though not necessarily to In 1990, the Howard

Hughes Medical Institute announced a $30 grant competition to strengthen undergraduate education. Ninety-nine institutions, many of

them liberal arts colleges, will compete for 5-year grants ranging from $500,000 to $2 million. Winners will be chosen on the basis of proven success:

the proportion and number of graduates who, over the past decade, have gone on to school or to earn doctorates in biology, chemistry, physics,

or mathematics. For background, see Liz “Hughes Institute Awards $61 Million for Science The Chronicle of Higher 

vol. 35, No. 38, May 31, 1989, pp. and Linda ‘‘Howard Hughes Medical Institute Enriches Undergrad Science Studies,” The Scientist,

 5, No. 1, Jan. 7, 1991, pp. 28-29.

 B. Maddox and P. Smith-Maddox, “Developing Graduate School Awareness for Engineering and Science: A Model, ” Journal

 Negro Education, 59, 1990, 479-490. This connection has also arisen over requiring institutions of higher education receiving Federal

assistance to provide certain information on graduation rates, broken down by program and field of study. See Public Law 101-542, Title I-Student

Right-To-Know, Nov. 8, 1990, 104 Stat. 2381-2384.

 global. applies even to prospective students. best students tend flock to

institutions where faculty have the greatest external reputations. Richard ‘‘The Movement Should Try Economic Pressures,” The

Chronicle of Higher Education, vol. 36, No. 43, July 11, 1990, p. A36.

 t. employment of engineers 15 

mathematics to over 200 percent in computer science (albeit from a small base number in this field). Most fields increased by 25 to 50 percent. For a

discussion of these trends (based on National Science Foundation (NSF) data), see Commission on Professionals in Science and op. cit.,

footnote 5, pp. 18-19. The role of at these traditionally teaching institutions is cause for concern. In fiscal year 1988, of 21,000 research

proposals submitted to NSF for funding, 12 percent came investigators at undergraduate institutions’ and 8 of awards

were made to these investigators, accounting for 5 of NSF funds awarded competitively that year. See National Science Foundation 

Research Proposal by Predominantly Undergraduate Institutions, NSF 90-36 DC: March 1990), p. 3. Also see Linda

E. Parker and DavidL. ‘‘Researchat Liberal Arts Colleges: Is More Really Better?” Research ManagementReview, vol. 3, spring 1989, pp. 43-55.
Chapter 7—Human Resources for the Research Work Force ● 219

These calls for increased undergraduate teaching

by faculty seek to alter an academic research and

teaching model in the United States that may already

be under strain. What follows is an examination of

the academic research model and its contribution to

human resources at the Ph.D. level.

The Academic Research Model

The predominant mode of academic research in

the natural sciences and engineering begins with a

research group that includes a principal investigator

(most often a faculty member), a number of graduate

students, one or several postdoctoral scientists,

technicians, and perhaps an additional nonfaculty

Ph.D. researcher. While this group may be working

on a single problem funded by one or two grants,

subsets of the group may work on different but

related problems funded simultaneously by multiple

project grants. (In the social sciences, the groups

tend to be smaller, often numbering only the faculty

member and one to two graduate students. )45 The

‘‘young investigator’ problem must thus be seen in

the broader context of other changes in the university as a research training site.

During graduate study, along with self-teaching

and learning from one’s peers, much of what is

learned comes directly through mentorship by professors, postdoctoral fellows, or nonfaculty researchers. In many research universities, professors

are responsible for multiple graduate students at any

given time, so that in a professor’s career he or she

may train over 20 (sometimes many more) Ph.D.s.

As one observer commented:

Simple arithmetic shows that training in a top

laboratory at a top institution, combined with the

requisite number of high-quality publications, does

not by itself ensure anyone a position similar to that

of his or her mentor. Most top laboratories graduate

two or three postdoctoral fellows a year.

. . . Multiply that by the large number of top laboratories in the country, and it becomes clear that even

Photo credit: Chuck Painter, News and Publications Service,

Stanford University

Graduate students work in the Center for Integrated

Studies at Stanford University. Research and teaching

go hand-in-hand in much of graduate training.

in good times not all these young investigators and

their research programs can be absorbed into the

National Institutes of Health system.46

In addition, the dominant model to launch a career

as a young scientist is movement from one research

university to another with an assistant professorship,

the attainment of a first Federal research grant, and

the re-creation of the mentor’s professional lifestyle

(i.e., independent laboratory, graduate students,

postdoctorates). For an institution to subscribe to

this model, unfortunately shifts much responsibility

for awarding tenure from the department faculty to

the Federal Government. While university officials

say there is ‘‘. . . no fixed time in which researchers

are expected to become self-sufficient through

outside grants . . . researchers who have failed to

 of competent graduate students has been a perennial charge-some mentors keep them around as 

are talented cheap labor. Competition for tenure-track positions requires refereed publications on the resume of the new Ph.D. This, too, may prolong

the graduate research career. Registered time to the doctorate increased from 1%7 to 1986, yet a recent National Research Council study suggests that

the reasons for this increase cannot be readily deciphered. According to one observer, the recent leveling off of this trend in time to Ph.D. may reflect

changes in the job market for new Ph.D. recipients. It will be interesting . . . to see if the numbers fall-as market-oriented theories would

 expected increase for faculty begins later in the 1990s.’ See National Research Council, On Time the Study

of the Increased Time to Complete Doctorates in Science and Engineering (Washington DC: National Academy Press, 1990); Peter “NRC

Releases New Study on Increased Time to the Doctorate,” Communicator, vol. 23, August 1990, p. 8; and Paul “Shortening the Time

to a Ph.D.,” Chemical & Engineering News, vol. 68, No. 52, Dec. 24, 1990, pp. 25-26.

 K. ‘‘Young Scientists and the Future, ’letter, Science, vol. 249, Sept. 28, 1990, pp. 1485-1486. Of course, some of these will

neither pursue an academic career nor compete for National Institutes of Health funds. The situation is less predictable than the author’s extrapolation

suggests.

292-863 0 - 91 - 8
220 ● Federally Funded Research: Decisions for a Decade

Box 7-C—Point of View of a “Small” Scientist

The following are verbatim excerpts from a recent “Point of View” column titled: “We Need To Give a

Chance to Small, Unfashionable Science,’ published in The Chronicle of Higher Education.l

While literally millions of dollars are being spent on massive, equipment-rich projects, other ‘small’ sciences

are in real danger of drowning for lack of funds. . . . Although I’ve received 22 grants from the National Science

Foundation and a host of private foundations during the last 15 years, grant money is now much harder to get. Support

for anthropology has gone from modest to minuscule.

"There’s no real problem, ’ I used to think while responding sympathetically to my peers’ groans over another

rejected grant. ‘‘Good science will always get funded. ’ 1 learned this magic formula from my mentors while I was

in graduate school and thought that if I repeated it often enough and believed it devoutly enough, I would be protected

from disaster. But plenty of good science is not receiving support these days. . . .

‘ ‘The long-term trends are grim. In the last 10 years, the number of grant applications submitted annually to the

National Science Foundation’s anthropology program has risen, as have the indirect costs of research. But the total

budget for the program has re mained approximately constant in real dollars, Consequently, the percentage of

applications receiving support has dropped. For example, 32 per cent of the applications submitted for archaeology

research in 1980 were approved, compared with only 23 per cent in fiscal 1990. What’s more, the average dollar

amount of the grants has dwindled during a period when virtually all the costs of actually conducting research have

risen. John Yellen, director of the foundation’s anthropology program, said recently with a sigh: ‘‘What seemed a large

but reasonable grant for us to fund 10 years ago now looks out of sight”. . . .

‘‘I do not know where the next generation of field researchers will come from, because the odds against starting

up a major project are so great now. I do know that without field researchers, anthropology will stagnate into a family

feud and eventually will perish from sheer triviality. . . .

‘ ‘Some deliciously subtle ways exist for a reviewer to sabotage a grant proposal, thereby blocking or stalling

a particular line of research, They range from simply giving a vaguely lukewarm review, to cl aiming falsely that the

applicant has overlooked important work in the field, to planting poisonous questions about methodology that the

review panel will then assume have not been addressed in the proposal. The temptation to engage in such unethical

behavior is greater if everyone is feeling the pinch. . . . Under these conditions, the review process becomes one of

paring down the proposals to a manageable number for ranking, rather than deciding how many are good enough to

receive support. .

‘ ‘The dilemma for program directors is: With too little money to go around, should they spread it around like

food supplies in a famine, giving everyone enough for a taste but not enough to maintain health? Or should they

support fewer projects more fully, condemning others to oblivion?

‘‘I would love to see a box printed on federal income tax forms saying, “Check here if you want one dollar of

your return to go to support basic science research.” Another solution is to guard against spending all of our money

for megabucks projects with catchy titles that appeal to legislators. Small, unfashionable science, as well as big, sexy

science, is important. Sometimes great ideas and staggering discoveries come from the little guys with funny ideas,

pottering away in the comer by themselves. We need to give them a chance. . . . “

l~e ~u~or is ~t Shipmw he COIU app~ Sept. 26, 1990, p. A60. For another expression of related ~tiwx, sm Euge~ G~Ield,

“Fast Science vs. Slow Science, Or S1OW and Steady Wins the Race, ’ The Scientist, vol, 4, No. 18, Sept. 17, 1990, p. 14.

win such grants are less likely to earn tenure than As seen in the preceding chapter, most universitheir colleagues who have found such support.’ ’ 47 In ties cannot afford to defray faculty research costs for

many fields, young Ph.D. s and older ones as well are very long, and the cost of supporting students has in

living out this scenario (see box 7-C). part been transferred to the research budget.48 The

dTSeeDebraE. Bl~ “YougerScientis~ Feel Big Pressure in Battle fOrtits, ” The Chronicle ofHigher Education, vol. 37, No. 4, Sept. 26,190,

p. A16. As one researcher put it: “Leading universities should make their own decisions about who their faculty are going to be, and not leave it to the

study sections of NIH. ’ Quoted in David Wheeler, “Biomedical Researchers Seek New Sources of Aid for Young Scientists, ’ The Chronicle of Higher

Education, vol. 36, No. 42, July 5, 1990, p. A23.

48Reseach assismts~ps experienced tie ~mtest ficre~e as the support mechanism for science md engineering students from 1980 to 1988. SW

National Science Board, op. cit., footnote 4, figure 2-11, p. 58.
Chapter 7—Human Resources for the Research Work Force . 221

combined costs of set-up and operation alone (which

for the typical chemistry or biomedical laboratory is

on the order of $200,000 to $400,000)49 have

ballooned. The priority of this academic, individual

investigator-based research model has become increasingly difficult for some institutions, fields,

departments, and faculty mentors to maintain.

The Federal research system is presently trying to

cope with growing demand for research monies, as

measured by proposals submitted to the research

agencies. 50 A potential shift of members of the

research work force out of universities and into

another sector, or into academic work that is not

research-centered, can be recognized as normal

labor market adjustment. Such movement would

testify to the versatility and adaptability of Ph.D.

researchers and to the Nation’s ability to utilize their

talents. 51

Computer science is just such a case example.

About 800 new Ph.D.s in this field are granted

annually in North America, double that of a decade

ago. The demand for new faculty has slowed, new

departments are not being created, and existing

departments are not expanding. Industrial and government computer research leaders asked:

whether the current situation warranted concern

i.e., is there a Ph.D. surplus, or are the supply and

demand levels reasonably in balance). Reports of

graduating Ph.D.s not finding the kind of academic

positions they desired. . . [lead to] the suggestion

that the expectations of these graduates need to be

adjusted. Not every bright, new Ph.D. will find an

academic position in a top-tier research university.

Postdoctoral positions in computer science are

becoming more common, and. . . graduates will

need to look toward second-tier research universities

as well as four-year colleges in order to fulfill their

career objectives.52

There is doubtless a role for universities to play in

the diversification of research careers of recent

Ph.D.s. 53 New Ph.D.s find it difficult to entertain

alternative opportunities if they have no experience

with them. Thus, programs that offer a summer in a

corporate laboratory or part of an academic year at

a liberal arts college can help advanced graduate

students visualize working in settings other than the

university. Arrangements that link a historically

Black college or university or liberal arts college to

a research university or national laboratory stretch

the resources and experience of both participating

institutions. 54

If the career prospects that new Ph.D.s confront

are so different from what they were taught to expect

and value, there can be a crisis of confidence. As one

university administrator states:

We are giving out mixed signals. Universities are

competing intensely with one another to hire the best

young Ph.D.s. On the other hand, the positions (at

least in many fields of science) available to the

average but quite capable Ph.D. are not very

attractive. Moreover, many very good students are

turned off when they see what the young faculty are

up against.55

Wone ~epofl on .set.up ~o~ts ~ ~hemi~~ at the ufiversi~ of -eso~ shows fourfold increases from 1979-80 to 1989-90. See Paul G. G~s~

“Future Supply and Demand in Academic Institutions,”Human Resources in Science and Technology: Improving U.S. Competitiveness, Proc&gs

of a Policy Symposium for Governmen4 Academia, and Industry, Mar. 15-16, 1990, Washington, DC, Betty Vetter and Eleanor Babco (eds.)

(Washington DC: Commission on Professionals in Science and lkchnology, 1990), pp. 35-36.

Wsee U.S. Con=ess, offke of lkchnology Assessment, “Proposal Pressure in the 1980s: An Indicator of Stress on the Federal Research Systeu”

staff paper of the Science, Education, and Transportation Program, April 1990.

slFor a discussion see Natio~ Res~ch Comcfl, OffIce of Scientilc and Engineering Personnel, Fosten”ng FlexibiZiV in the Engineen”ng Work

Force (Washington DC: National Academy Press, 1990). Among the National Research Council conclusions are: a) that the production of adaptable

engineers is impeded not by the engineering curriculum, but how that curriculum is delivered; andb) that continuing education could enhance adaptability

in the engineering work force.

SZJo~ R. white, “president’s utter: Refl~tiom on Snowbkd,” Communications of the ACA4, vol. 33, Septembr 1990, p. 19.

sJBetter fac~ty ~vising of graduate students is an obvious need. See C~OlyU J. Mooney, “The Dissertation Is Still a Wluable Requirement Survey

Finds, But Graduate Students Say T’hey Need Better Faculty Advising,’ The Chronicle of Higher Education, vol. 37, No. 18, Jan. 16, 1991, pp. A15,

A22; and Association of American Universities, Institutional Policies to Improve Doctoral Education (Washingto@ DC: November 1990).

fiTo &te, such ~mgements ~ve ken most ~o-on fi ~der~duateen~~ring.One co~itio~ spe~~ded by a 5-ya, $15 million National

Science Foundation grant, will establish a communications network for information disseminatiom faculty exchange, workshops, and outreach to

elementary, secondary, and community college students. The participating universities are City College of New Yorlq Howard, Maryland, Massachusetts

Institute of ‘Rdnology, Morgan State, Pennsylvania State, and Washington. See “NSF Announces Multi-Million Dollar Grants ‘lb Form Engineering

Education Coalitions,” NSF News, Oct. 9, 1990. Also see Oak Ridge Associated Universities, 1990 Annual Report (Oak Ridge, TN: 1990). Oak Ridge

Associated Universities is a Department of Energy laboratory and a consortium of 59 colleges and universities engaged in research and educational

programs in the areas of energy, healti and the environment.

SsNeal tie, “Education~ Challenges and opportunities,’” Vetter and Babco, op. cit., footnote 49, p. 94.
222 . Federally Funded Research: Decisions for a Decade

Universities are caught between the desire to train

the next generation and the harsh reality that their

research apprentices may face a different form of

competition for resources (for a controversial example, see box 7-D).

New Models

The National Science Foundation recently announced that the 20 research universities that receive

the most NSF support “. . . will work together to

improve science and mathematics education in

schools and to increase the number of women and

minority students and faculty members in science

and engineering. ” Actions include changing tenure

policies to reflect the extra family responsibilities

often carried out by women and encouraging faculty

to work with schoolchildren and teachers.56

Other models of education could be encouraged

that feature a greater sharing of resources (e.g.,

equipment and space) and people (e.g., doctoral

students, nonfaculty researchers, and technicians).

Models that stress research in units other than

academic departments, research in nonacademic

sectors, and nonresearch roles in academia could be

entertained. These models are already being applied

in the centers programs sponsored by NSF. Centers,

which support individual researchers (as faculty and

mentors) as well, may represent a new way of doing

business for NSF and the companies that participate

in them.57

In the 1980s, centers at NSF became the focus of

political dispute.58 At issue was the appropriateness

of promoting such a mode of research organization

in view of the basic research and science education

missions of NSF. For NSF, however, the centers

complete ‘‘. . . a balanced portfolio of individual

Photo credit: U.S. Department of Energy

Three scientists work on a magnet for the Superconducting

Super Collider. Teamwork is an important part of

research groups.

investigator grants, facilities, and center activities. ’ ’59 Engineering Research Centers (ERCs) were

intended to foster university-industry collaborations. As a 1988 General Accounting Office study

found, participating companies “. . . expect to benefit over time through better personnel recruiting . . . [even though] it is too early to determine the

program’s impact on engineering education. ’ ‘60

 universities. for Minorities, ’ ‘ The Chronicle of Higher Education, vol. 37, No. 1, Sept. 5, 1990, p. A2. Incorporating

incentives for affirmative action into programs that allocate research dollars underscores the importance of human as a criterion for Federal

research funding, It to be seen whether universities respond--with or without prodding by the National Science Foundation.

 Research 

 at $48 million and $27 respectively. Thus, together they account for less than 10 of NSF’s while providing a long-term

funding base (5 to 11 years) for and high-risk projects oriented to the applied, and commercial-use end of the research

continuum, See Joseph and Eliot Marshall, Leaves NSF in Science, vol. 249, Aug. 24, 1990, p. 850. In the block-gran~

multi-investigator approach embodied by ‘‘NSF has rolled the dice on an experiment in science, and it will take some time to know whether it

has come up with a winner. ”See Joseph “NSF Centers Rise Above the Science, vol. 251, Jan. 4, 1991, pp. 19-22, quote from p. 22.

Also see Jeffrey Mervis, “NSF Cuts Back on Faltering Science, Centers, ’ The Scientist, vol. 5, No. 3, Feb. 4, 1991, pp. 1,4, 24.

 L. Next of National Science Foundation Must Press for a Greater U.S. Investment Science, ” The Chronicle

of Higher Education, vol. 36, No. 48, Aug. 15, 1990, p. B3.

 A. “Disputing Some Claims About the NSF,” The Chronicle of Higher Education, vol. 37, No. 3, Sept. 19, 1990, p. B4.

 adds “. . . although and joint the goals of the participants

believe that the quality and type of research are more important reasons for sponsoring ERCS.”See U.S. General Accounting Engineering

Research Centers: NSF Program Management and Industry Sponsorship, (Washington DC: August 1988), pp. 2-3.
Chapter 7—Human Resources for the Research Work Force ● 223

Box 7-D—The Priority of Research Training

In November 1990, the Institute of Medicine (IOM) published a report on priorities in health sciences:

The charge to the committee was to analyze the funding sources for research projects, training, facilities, and

equipment by Federal and nonfederal sources. The commi ttee was asked as well to develop a coordinated set of

funding policies to restore balance among these components of the research enterprise in order to ensure optimal use

of research dollars. . . . The goal of the study was to ensure that, at any given level of support, allocation policies would

enable the scientific community to utilize available resources in the most cfficient manner so as to create an optimal

research environment and achieve society’s goals for research into human disease.1

First, by adopting “imbalance” as a premise and then by concluding that if the National Institutes of Health

budget were not to grow over inflation in the coming decade (one of four funding scenarios considered by the IOM

committee), expenditures for research training and facilities should take priority. The report rankled the biomedical

research community. Leading the dissent was the Federation of American Societies for Experimental Biology

(FASEB), which objects to any diversion of funds at the expense of individual investigators. 2

FASEB rejected the premise of the IOM report that in an era of tight budgets, biomedical funds should be

balanced. FASEB claimed there is no imbalance, arguing that increased funds for training and construction will

jeopardize “. , . productivity in the foreseeable future. ” FASEB president Thomas Edgington, an immunology

professor at the Research Institute of Scripps Clinic, California, feared that implementation of the IOM

recommendations would “. . . diminish advances of value to the public. ” He continued: “We advocate. . . a total

increase of support of the enterprise as a whole. ’ So while committee chairman Bloom insisted: ‘ ‘We’re eating our

seed corn,” Edgington concluded: “Training has not decreased. ” Chemist Ronald Breslow, a member of the IOM

committee, added another point on the training issue:

supporting training through research grants ties training too much to the success of the research advisor. Giving

money to the student rather than the sponsor changes their relationship. Trainees can then work on their own projects

rather than being a sort of employee. It’s more encouraging to the student to be told ‘‘You are a winner’ rather than

‘ ‘Just go beg for support. ’

This public dispute highlights several points. First, it attests to the depth of anxiety that grips investigators in

search of stable, multiyear research finding. Second, by entertaining a‘ ‘no real growth’ funding scenario, the IOM

report puts into black-and-white what few investigators want to contemplate, i.e., tight funding could get tighter.

Third, by favoring an increase of research training funds under the worst-case scenario, the committee removed

research funds as the first priority. Indeed, ‘‘The committee believes that this growth in the training budget will not

enlarge the research project grant applicant pool; rather, the net effect of this gradual reallocation will be to replace

the increasing number of scientists expected to retire later this decade. ”4

This IOM report should be applauded for attempting to make forecasts and preparing for its consequences by

systematically considering priorities among resources. However, the conclusion to increase training funds is

problematic. 5 At present the system is producing an abundance of new Ph.D.s in biomedical fields. Enhancing this

production while holding the line on the research grants that must support them is rightly open to question.

Nevertheless, policy makers will welcome the IOM report for its look at a hard, complex problem and its statement

of priorities.

l~ti~te of Me&c’.ne, co~t~e on poticies for AWcating Health Sciences Research Funds, Fundng Heahh Sciences Resemch: A

.$rrafeg~ to Restore Balance, F.E. Bloom and M.A. Randolph (eds.) (Washingto~ DC: National Academy Press, 1990), p. 5.

2~F~e~on of fiencm Smletla for ~~en~ BiOlogy criticism was ~oR~ a~st ~~m~ly by: Bfi~ J. tilliton,

“FASEB ‘Rejects’ IOM Study,” Science, vol. 250, Nov. 30, 1990, p. 1199; ‘ ‘Strife Erupts Over Shares of Biomedical Funding, ” Science &

GovernmentReport. vol. 20, No. 19, Dec. 1, 1990, pp. 14; and Pameta Zurer, “Biomedical Research: Calls for Shifis in Funding Attacked,”

Chem”cal & Engineering News, vol. 68, No. 49, Dec. 3, 1990, p. 6. Matesial quoted below comes from one or all of these sources.

3Also see Ronald Breslow, “Punding Science Researc ~“ letter, Chem”cal & Engineering News, vol. 68, No. 51, Dec. 17, 1990, p. 2.

4S= ~timte of Medi~~, op. Cit., foo~te 1, p. 8. Ah see National Research Counc~ Bionudical and Behavioral Research Scientists:

Their Training and Supply (Washington, DC: Natiomd Academy Press, 1989).

5~g~on ~so ~~ed ~m~mp of tie IOM Committee, sa~% that it was composed mostly of research ausmtom~d ~d few

working scientists.
224 ● Federally Funded Research: Decisions for a Decade

Table 7-1—issues in Team Research Performed in Different Organizational Contexts

Large, university based

Key issues Small, university based (organized research unit) Small, industry-based Large, industry based

Number of disciplines

involved

Typical research

Funding

Source of research

problems to be solved

Structure of

organization

Leader’s involvement

Member’s career

prospects

Key leadership roles

small

Pure

Short term and

uncertain

Found by team

flat

Part time

Uncertain

Outside liaison

Communication Few potentially

problems

Access to scientific and Good

technical resources

Moderate

Pure-applied

Relatively certain

Found by team (largely)

Hierarchical (selective

decentralization)

Full time

Relatively certain/

predictable

Outside liaison and

coordination

Internal and external

Good

Moderate

Applied-pure

By contract and short

term

Given by environment

Flat

Part time

Uncertain

Outside liaison and

resource acquisition

External

Not SO good

Moderate to large

Applied and pureapplied

Relatively certain

Found by team and

given by environment

Hierarchical (selective

decentralization)

Full time

Relatively certain/

predictable

Motivation and

management of

different teams

Internal

Good if affordable

SOURCE: Adapted from Julie Thompson Klein, lnterdisc@/inarify: History, Theory, and Practice (Detroit, Ml: Wayne State University Press, 1990), table 1,

p. 124.

Research in general is becoming increasingly them, and Caltech’s Jet Propulsion Laboratory is

interdisciplinary, i.e., it requires the meshing of even larger than its ‘‘nominal parent, ’ Caltech.63

different specializations to advance a research

area.

61 Academic departments house specialists by Team research, whether interdisciplinary or not,

discipline whose research will be performed in takes place in many organizational contexts, subject units--centers, institutes, programs-that cut across

the traditional departmental organization on cam- to various influences (for a summary, see table

7-l). While the disciplinary department still pre- pus. Such organized research units (ORUs) have a

history on U.S. university campuses, but not as a vails, a wide range of organizational models are now

dominant structure.62 Klein writes: employed by universities to manage and conduct

In the 1960s Federal legislation gave birth to

research. The autonomy, formality, and permanence

several kinds of ORUs, including NASA space of these production units depend on adapting the

centers, water resource centers, and later, regional demands of outside patronage to local custom and

education laboratories. A number of ORUs are de need, including the support and on-the-job research

facto independent of the university that gave birth to training of graduate students.

blFor ex~ple, see A.L. porter and D.E. Chub~ “AnIndimtor of Cross-Discipliruuy Research, ” S’cienmnem”cs, VO1. 8, 1985, pp. 161-176; and Don

E. Kash, “Crossing the Boundaries of Disciplines,” Engineering Education, vol. 78, No. 10, November 1988, pp. 93-98.

62D.I. Phillips and B.P.S. Shen (eds.), Research in the Age of the Steady-State University (Boulder, CO: Westview Press, 1982). h= models of

organized research units (which are common in industry and the Federal laboratories) have taken root on campus-agricultural experiment stations, water

resources research centers, and engineering research centers. See Robert S. Friedman and Renee C. Friedrn~ ‘‘Science American Style: Three Cases

in Academy,” Policy Studies .Journal, vol. 17, No. 1, fall 1988, pp. 43-61. Several other models could be mentioned; e.g., the Joint Center for Laboratory

Astrophysics at the University of Colorado, the Center for Astrophysics (the Smithsonian Astrophysical Observatory-Harvard Astronomy Department

Partnership), and the Materials Research Laboratories (supported by the National Science Foundation).

63Jufie momp50n IWQ lnterdisciplinari~: History, Theory, and Practice (Detroit MI: Wayne State Universi& fiess, 1990, P. 123.

@see Donald pe~ and Frank D. An&ws, S’ckntists in Organizations: Productive Climatesfor Research and Development, revised ed. (Ann Arbr,

MI: University of Michig~ Institute for SociaI Research 1976).
Chapter 7—Human Resources for the Research Work Force . 225

Box 7-E—Flexibility at the Beckman Institute

The Beckman Institute for Advanced Science and Technology is located on the University of Illinois

Urbana-Champaign campus and was built with a 1985 donation of $40 million from Arnold and Mabel Beckman.

In the words of its director, Theodore L. Brown, the mission of the institute is . . to advance the life and behavioral

sciences, physical sciences, and engineering through promotion of multidisciplinary and interdisciplinary research

of the highest quality, ” in particular”.. . to transcend the limitations of departmental structure." 1

0ver 700 faculty,

postdoctoral associates, graduate students, and others work full- or part-time at the institute.

The institute divides its research programs into nine multidisciplinary groups. For example, one group includes

faculty in neuronal pattern analysis, cognitive neuroscience, perception, biomechanics, and molecular biology. It

addresses a range of problems from the molecular level to cognitive science. 2 Perhaps most well known of the nine

groups is the National Center for Supercomputing Analysis, which performs research in the diverse use and analysis

of supercomputer-generated models.

To maintain flexibility in its research missions and yet provide some measure of security for the faculty, the

Beckman Institute has taken advantage of its position outside of the university departments and developed a‘ ‘rolling

appointment’ system:3

. Roughly 35 faculty call the Beckman Institute their full-time research home, although each retains a

departmental affiliation. Faculty have 5-year appointments, which are extended every year, as long as their

research is productive for the institute. If the appointment is not extended, then the faculty member has 5

years in which to move back to his or her department (where their tenure is based).

. Another 90 to 100 faculty members are part time at the institute, and are based primarily in their departmental

homes. Each has a 3-year appointment, which is renewed like appointments of full-time faculty.

. Space is allocated in the institute only for those who use it, and is taken away if insufficiently occupied.

So far the system has worked well for the faculty at the institute; it represents a novel attempt to maintain

flexibility in the university environment. In the future, other institutes and centers may view the Beckman Institute

as a model.

1 See Ward Worthy, ‘ ‘New Beckman Institute Promotes Broad-Ranging Researc h Effort," Chemical & Engineering News, vol. 68, No.

34, Aug. 20, 1990, pp. 23-24.

2

Ibid., p. 23.

3

Ibid., p. 24.

With the ascendance of what OTA called in the be it an ERC, a supercomputer center, or another

last chapter the ‘‘industrial model’ of research, campus-wide institute that is largely federally

changes in the units producing knowledge have funded? 66 One physicist writes:

become apparent.65 One could predict that univer- Goal-oriented research and a traditional belief in

sity investigators will gravitate to those units that team methods favor large, centralized approaches in

bring together the needed personnel and instrumen- many industrial and Federal laboratories. (Small

tation, and ease the research process, especially if groups can also flourish in these environments-the

Federal and other sources of support favor centers as two discoverers of high-temperature superconducefficient sites of research performance (see box 7-E). tivity are industrial scientists.) . . . For the same

Does it matter that research laboratories in an reasons that artists use different environments or

new media to stimulate creativity, scientists need

academic department come to resemble an ORU— different ways to do science. This will help ensure

6 5Science policy statesman John Ziman claims that science iS ‘‘. . . being ‘collectivized’ in two different senses of the word. On the one hand, almost

allresearchnowadays is being carried out within the fiameworkof quite large organizations . . . In many cases, research projects are undertaken by groups

or teams of researchers who have limited control over the resources they use, and cannot claim personal responsibility for what is attempted or what

is achieved. Inotherwords, the extreme individualism embodied in the academic ethos . . . is no longer consistent with the realities of scientitlc life, where

collective action is now the rule. . . . On the other hand, the incorporation of academic science into an expanding ‘R&D’ system, drawing funds from

the central government and private industry . . . [means] academic science is losing its place as an autonomous social segment with its own standards

and goals, and is being brought under ‘collective’ control. Instead of being treated as an independent source of unpredictable social influences, it has

come to be regarded as an instrument of deliberate societal action. ’ See John Zi~ “Collectivized Science, ” An Introduction to Science Studies

(Cambridge, England: Cambridge University Press, 1984), pp. 132-139, quote from pp. 138-139.

fisee Natioml Research COunCil, The Engineering Research Centers: Leaders in Change (Washington, DC: National Academy Press, 1987).
226 ● Federally Funded Research: Decisions for a Decade

that research careers attract good minds of every

sort-those that flourish in isolation, and those that

thrive on interaction.67

The trend toward larger research teams, more

cooperation among individuals and groups, and

more multiauthored papers is unmistakable. 68 The

question is whether large research groups will

effectively duplicate or augment the mentorship

function. Whatever the campus research setting, it

must be valuable for the socialization of new Ph.D.s.

The process of scientific research is learned in

university settings, but sometimes the skills associated with the process (e.g., data handling and

communication practices) do not transfer well to

nonacademic settings.69

Other Considerations

Federal research funding serves various goals,

none more important than the strengthening of

education and human resources. The Federal Government has a special role both as a primary catalyst

of the future supply of Ph.D.s and as one employer

of the existing Ph.D. work force.70

In addition to the issues outlined above, the

Federal Government must also consider several

questions. First, given the tendency of Federal

funding to concentrate in a small set of universities,

what efforts should be made to support other

institutions? (Or is there an optimal mix?) Twenty

percent of all authors of scientific papers produce 80

percent of the scientific literature.71 Furthermore,

basic researchers are concentrated in a relatively

small subset of all academic institutions, and conventional wisdom says that scientific productivity

depends on the extraordinary achievements of relatively few researchers, laboratories, and universities.72 Writing in 1968 as a physicist and academic

dean, as well as a policy advisor, Harvey Brooks put

it this way:

The vigor of a scientific field seems to depend on

a continuing injection of new investigators with

fresh ideas and on sufficient funds to exploit new

ideas. . . . To spread the same funds more and more

thinly over a growing number of investigators,

institutions, and students would be a prescription for

the slow strangulation of science in the United

States.73

To avert the “slow strangulation of science,” a

concentration of funding in select research institutions and research groups (which is what the Federal

Government currently practices) would seem wise.

However, at what point does concentration begin to

disrupt the interdependence among researchers? Not

funding a wide array of researchers risks curbing

more than the flow of resources; it can interrupt the

flow of communications and begin to deter cooperation between specialists. Such cooperation, which

contributes to the accumulation of research findings,

is especially intense at universities.74 Sustaining

productive researchers (and the students who will

eventually join the top ranks of scientists and

engineers) fuses Federal research funding policy

b7Sidney Perkowitz, ‘‘Larger Machines Are Breeding Larger Research lkarns,” The Scientist, vol. 3, Oct. 16, 1989, pp. 13, 15.

~Natio~ Science Board, op. cit., footnote 4, p. 120.

@For a discussion, see National Academy of Sciences, Committee on the Conduct of Science, On Being A Scientist (Washingto~ DC: National

Academy Press, 1989), pp. 10-22. Also see Terry S~“Scientilc Disciplines and Organizational Specificity: The Social and Cognitive ConfQu.ration

of Laboratory Activities, ’ Scientific Establishments and Hierarchies, N. Elias et al. (eds.) (Dordrech~ Holland: D. Reidel, 1982), pp. 239-2M.

70~e Feder~ ~ve-entemploys 2w,~ ~ientists ~dengin=rs, about 11 per~nt of the Fedm~wo& force. And ~though~y of theSe people

neither have aPh.D. nor do bench resezuch, they are facilitators of research performance (as agency program managers). Micromana gernent of the Fedeml

laboratories, personnel ceilings, noncompetitive salary, inadequate fringe benefits, ethics laws, and the public image of Federal service are reported to

contribute potentially to a Federal “brain drain. ” Yet only 5 percent leave Federal employment annually. See Jauice Long, “Agencies Compete Well

in Science Job Market,” Chemical& Engineering News, vol. 68, No. 39, Sept. 24, 1990, p. 23. For further discussio~ see Alan K. Campbell and Linda

S. Dix (eds.), National Research COunCil, Recruitment, Retention, and IJtdization of Federal Scientists and Engineers (Washington, DC: National

Academy Press, 1990).

?lJo~~ R. COle and Stephen Cole, “TM ortega Hypothesis,” Science, vol. 178, Oct. 27, 1972, pp. 368-372. Also see Sharon Levin and Paula

E. Step~ “Research Productivity Over the Life Cycle: Evidence for Academic Scientists, ” American Economic Review, vol. 81, March 1991, pp.

114-132.

Tzsee ~neBy~s@, “America’s Hot Young Scientists,” Fortune, vol. 122, No. 9, Oct. 8, 1990, pp. 56-69. Also see David Pendlebury, “Science

Leaders: Researchers to Watch in the Next Decade,’ The Scientist, VO1. 4, May28, 1990, pp. 18-19, 22-24; and A.L. Porter et al., “Citations and Scientit3c

Progress: Comparing Bibliometric Measures With Scientist Judgments, ’Scientometrks, vol. 13, 1988, pp. 103-124.

73Harvey Brooks, “The Future Growth of Academic Research: Criteria aud Needs,” Science Policy and the University, Harold Orlans (cd.)

(Washingto& DC: The Brookings Institution@ 1%8), pp. 75-76.

TdRese~chersform info~ networ~ a~ording to their specialization in addition to the forrnd organizations k which they work. It is not known

how changes in funding patterns affect informal as opposed to formal communication. See Leah A. Lievrouw, ‘ ‘Four Research prO~S in Scientific

Communication” Knowledge in Society, vol. 1, summer 1988, pp. 6-22.
Chapter 7—Human Resources for the Research Work Force ● 227

with training, employment, and productivity concerns.

A further aspect of concentration of funding, and

one of mounting concern to universities and agencies alike, is the degree to which the Federal

Government could promote participation in science

by disadvantaged groups and by regions of the

Nation that have not traditionally received large

amounts of Federal research funds. Disadvantaged

groups and potential scientists and engineers who

are geographically dispersed in research are considered by many to be untapped resources that could

enhance U.S. research capacity. At present, set-aside

programs help to accomplish these goals (see box

7-F). A more concerted and sustained effort by the

Federal Government to increase participation is

warranted in the coming decade.

A final question is this: should the Federal

Government focus its educational support on graduate fellowships, traineeships, and research assistantships (regardless of the combination), or spend a

larger proportion of funds on undergraduate education and earlier segments of the pipeline? If one

believes the projections of future shortages in Ph.D.s

for the 1990s, then the Federal Government could

produce many more Ph.D.s for the short term to meet

the need, and then reconsider the question. If there

are concerns about these projections (as OTA has

ascertained), then the Federal Government could

take a long-term view and work to enhance the

supply throughout the pipeline. This would particularly argue for a greater investment in undergraduate

science and K-12 education. As labor economist

Alan Fechter writes:

The relevant policy issue should be whether the

expected equilibration mechanisms triggered to

correct . . . imbalances will be consistent with national needs and more global social objectives. . . . If

not, then policymakers will need to consider other

mechanisms that will equilibrate supply and demand

with minimal unwanted side effects.75

A key concern for the future of the research work

force is not only the size of the pipeline, but the

composition of the students in it. What is the relative

proportion, especially at the doctoral level, of U.S.

citizens and foreign nationals? The evidence prePhoto credit: University of Washington

A student in the Early Entrance Program at the

University of Washington performs a physics experiment.

Science education from K-12 through undergraduate

levels is vital for preparing the science and engineering

research work force.

sented in this chapter on the chronic underrepresentation of women, minorities, and the disabled among

science and engineering Ph.D.s suggests that financial support mechanisms have discouraged certain

segments of the talent pool from pursuing graduate

study. How can the Federal Government, through

direct financial incentives and institutional programs that can modify student aspirations, reverse

this trend? The answers have implications not only

for the size of the work force, but also for the

character of the scientific work that new participation automatically brings to the Nation’s research

enterprise. 76

Conclusions

The U.S. system of doctoral production has

consistently displayed its excellence over the past 40

years. The number of doctoral scientists and engineers employed in the United States has grown by

almost 50 percent since 1977. Nevertheless, several

trends in the production of new Ph.D.s are of

concern. Many also question whether shortages in

the future U.S. scientific and engineering work

force, and particularly its research component, are

inevitable.

 op. cit., footnote 26, p. 19. Also see “Despite Scientist Shortage, Future Fear Joblessness, ” The Scientist, vol. 5, No.

1, Jan. 7, 1991, pp. 24, 32.

 see ‘‘The Benefits of Black Participation Science,” Blacks, Science, and American Education, Willie Jr.,

and H. Kenneth (New NJ: Rutgers University Press, 1989), pp. 123-152.
228 ● Federally Funded Research: Decisions for a Decade

Box 7-F—Minority Biomedical Research Support Program

As part of the National Institutes of Health’s (NIH) effort to promote involvement of minority scientists in

ongoing research, the Minority Biomedical Research Support (MBRS) Program awards grants to institutions with

substantial minority enrollments. The purpose of the program is to support research by American Indian, Black,

Hispanic, Native Alaskan, Asian, and Native Pacific Islander faculty and students in the biomedical sciences, and

to strengthen the grantee institutions’ biomedical research capabilities.1

Initiated in 1972, the MBRS Program was originally administered through the NIH Division of Research

Resources, but in 1989 was moved to the National Institute of General Medical Sciences (NIGMS). Two- and

four-year colleges, universities, and health professional schools with either 50 percent or more minority student

enrollment or those with a demonstrated commitment to the assistance of minority students and faculty are eligible

to apply for a project grant. The average annual grant is $350,(X)0, the highest $1.5 million. MBRS program grants

may be used to support time for faculty members to engage in research; exploratory research and full-scale research

activities through the purchase of equipment, supplies, and technical assistance; and undergraduate and graduate

students working ‘‘hands on “ in a faculty member’s research. Figure 7F-1 shows the size and number of MBRS

awards for the past 15 years.

Figure 7F-1--MBRS Awards and Funding: Fiscal Years 1975-90 (In millions of 1982 dollars)

Total award amount

36 

34

32

30

28

26

24 I

22

20

18

16

14

12

1 0

19.7

28.2 27.6

24.7

29.6

26.4

32.3

30.5

33.1

31.3

75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90

(72) (73) (74) (75) (76) (80) (77) (78) (71) (79) (96) (98) (98) (95) (98)(98)

Year

(Number of awardas

NOTE: MBRS - Minority Biomedical Research Support.

SOURCE: National Institute of General Medical Sciences, unpublished data, August 1990.

Project review and management, including monitoring progress and negotiating budgets, are assumed by

NIGMS. As a line item in the appropriation for NIGMS, the MBRS Program received over $28 million in fiscal

year 1989. The program received an additional$11 million from other NIH components (which have contributed

 on Institute of General Medical Sciences, ‘ ‘Minority Biomedical Research Support Program+’

administrative April 1990.

There are pitfalls in many of the methodologies these trends should be monitored, and investments

employed in Ph.D. projections: their track record is in the educational pipeline continued to ensure a

troublesome. Most notably, OTA questions the robust supply of students. This is in the national

ability of statistical analyses to predict future interest, not just essential for science, engineering,

demand for Ph.D.s, especially when coupled with and the research enterprise. For this reason alone,

uncertainty about levels of Federal and industrial increasing the attention to teaching in research

support for s/e research in the 1990s. Nevertheless, centers is warranted.
Chapter 7—Human Resources for the Research Work Force . 229

for the past 13 years) to help support MBRS research projects of direct relevance to their own missions. Presently,

11 other NIH components and the Alcohol, Drug Abuse, and Mental Health Admini “ stration co-fund MBRS projects.

The MBRS Program has two funding mechanisms. The first, the traditional MBRS Program, provides support

of up to $1.5 million per year for faculty members to carry out research projects. To be eligible, an institution must

undertake at least 2 and no more than 25 research projects and must involve at least 2 faculty members having

different research interests. A heavy emphasis is placed on involvement of undergraduate and graduate students in

faculty research projects. The hope is that from this interactive relationship, a commitment to science will develop,

and biomedical research will become an attractive career option for those students. While faculty members are

expected to submit their work for publication in peer-reviewed scientific journals, the program also expects students

to participate actively in the research, coauthor publications, attend scientific meetings, and give presentations.

The second component of the MBRS Program, the Program for Undergraduate Colleges and Two-Year

Colleges, began in 1985 and supports enrichment activities, pilot research projects, and regular research projects.

The maximum award under this program is $450,000 in direct costs over 3 years, plus indirect costs. A portion of

these award monies must be applied toward “enrichment activities,” such as workshops, attendance at scientific

meetings, and summer research experiences in off-campus laboratories. In addition to the two primary award

programs, the MBRS Program started supplemental funding for both instrumentation and animal resource

improvement in 1983 and 1987, respectively.

Currently, institutions in 31 States are supported by the MBRS Program, with California and Puerto Rico

having the most institutions participating in the program (nine and eight, respectively). Like many programs

targeted to groups underparticipating in science, the MBRS Program represents a well-institutionalized model that

could be generalized if funding permitted. As a recruitment and retention device, the program couples students to

role models, previews science as a career, and builds institutional capability to compete for Federal funds and

conduct collaborative, cutting-edge research. A significant number of awards are concentrated in southern States

with large minority populations, such as Alabama, Georgia, and Tennessee. Thus, the MBRS Program, in

conjunction with the other NIH efforts, is intended to produce a larger minority presence in the research work force.

While OTA has presented data on human re- Based on the trends reviewed here, OTA finds that

sources in research, more detailed data could be there is no current crisis in the Nation’s ability to do

collected. For instance, it is at best possible only to

estimate, with presently available data, the size of

the research work force. Based on the NSF estimates

cited above, the U.S. s/e work force of academic

Ph.D.s totals 340,000; those reporting research as a

primary or secondary work activity total 155,000. So

less than one-half (about 45 percent) of those in the

academic sector are engaged at all in research, but

this represents an unknown fraction of the total

research work force. The proportions federally

funded and the number of scientists and engineers

leaving research careers are also unknown. Even less

information exists on the evolution of research

groups (and its effect on graduate training). Much

anecdotal evidence and some personnel expenditure

data from NIH and NSF indicate that research

groups are becoming larger on average, yet the size

of the distribution and its rate of growth are

unknown. (The next chapter reviews the state of data

on the research system, and presents suggestions for

enhanced data collection in the Federal agencies and

elsewhere.)

scientific research. The Federal Government may

wish to do nothing now to intervene in the Ph.D.

production system and let market forces operate

unencumbered. Federal funding will continue to

serve the goal of excellence in research. If funding

growth slows, however, the related goals of education and human resources risk becoming secondorder priorities. All markets would be fortified by

enhancing the education pipeline, especially at the

undergraduate level, to ensure that future shortages

can be met.

However, the composition of students in the

pipeline-by gender, race/ethnicity, and nationality-is of greater concern. The scientific community

could benefit from the diversity of research interests

and approaches that new entrants bring, especially

those from groups who have historically not participated fully in it. The Federal Government may also

wish to consider, through new legislation, more

extensive and long-term support for the recruitment

of potential scientists and engineers from groups

disadvantaged within s/e, as noted above. A diver-
230 . Federally Funded Research: Decisions for a Decade

sity of personnel and career paths will distinguish

the research work force of the 1990s.

Policies could be devised that target incentives for

s/e labor markets disaggregated by field and sector.

Research in many fields is moving toward a more

“industrial” model, with larger teams, specialized

responsibilities, and the sharing of infrastructure. In

response, the Federal Government has acknowledged changes in the (sometimes interdisciplinary)

composition and more complex structure of research

groups through centers and block-grant funding.

Perhaps it should now provide the impetus for

universities to examine and experiment with their

policies concerning the opportunities and rewards

for nonprincipal investigators, e.g., postdoctorates

and nontenure track researchers.

All policy initiatives will need to consider impacts on undergraduate and graduate teaching, the

stress on the current academic model of research and

education, and new models that might lessen institutional strain. Reauthorization of the Higher Education Act of 1965 by the 102d Congress will provide

an opportunity to encourage universities to increase

the rewards for undergraduate teaching and enhance

undergraduate and graduate experience in diverse

career paths. If this chapter has demonstrated

anything, it is that human resources are a main

business of the Federal Government, and not a

marginal concern in strengthening the Nation’s

scientific research capability.
CHAPTER 8

Data on the

Federal Research System
Contents

Page

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

What Data Are Available on the Federal Research System? . . . . . . . . . . . . . . . . . . . . . . . . . 235

What Data Are Needed? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

Research Monies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

Personnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

Research Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .240

Outcomes of Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

Utilizing Data. ... ......+. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

New Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . ..,.,... . . . . . . 249

Evaluation of Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256

Boxes

Box Page

8-A. Bibliometric Profiles of Four Research Institutions . . . . . . . . . . . . . . . ..s,..., ..,,. 244

8-B. War as a Wild Card: The Impact of the Persian Gulf Conflict on Science and

Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ,...+... . . . 250

Figures

Figure Page

8-1. Relative Citation Impact for National Laboratories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

8-2. Relative Citation Impact for Three Industries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

Tables

Table Page

8-1. Summary of OTA Data Collection and Analysis on Federally Funded Research. . 234

8-2. Categories of Data in Science & Engineering Indicators . . . . . . . . . . . . . . . . . . . . . . . . 237

8-3. Trends in Distribution of Data Among Categories in Science & Engineering

Indicators: 1972-88 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

8-4. Mismatches in Rank Between Federal Funding and Average Citations: 1988 . . . . . 243

8-5. New Indicators of Research Activity at NSF: Fiscal Years 1980-89 . . . . . . . . . . . . . . 251

8-6. Characteristics of the Research Evaluation Process for Selected Countries . . . . . . . . 255

8-7. Assessment of the Department of Energy’s (DOE) Basic Energy Sciences

Program: 1982 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256

8-8. Dimensions of Agency Research Evaluations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

8-9. Desired Data and Indicators on the Federa1 Research System . . . . . . . . . . . . . . . . . . . . 258
CHAPTER 8

Data on the Federal Research System

The measurement process is also inherently limited by the inevitable human selection

of both the phenomena to be measured and the type of data considered relevant to the

purpose of the measurement effort. . . . For measuring an area as little understood as the

science and technology enterprise, multiple models are needed to insure that as wide a

spectrum of phenomena as possible is included. - .

Introduction

While this report has characterized the Federal

research system as it enters the 1990s, its mandate

was broader. OTA was asked what data and analytical tools would be useful in describing the research

system. Preceding chapters have drawn on much

data. However, there are many areas in which

additional information would be welcome. Data, in

short, are an issue in Federal research policy,

especially their form, gaps, and uncertainties. For

example, OTA has discussed:

●

●

●

●

●

●

●

variable definitions of ‘scientists’ and ‘ ‘engineers” that can result in radically different

estimates of their numbers (chapter 1);

problems with using different deflators to

calculate constant dollar trends in research

funding (chapter 2);

potential comparisons between congressionally

earmarked and peer-reviewed projects (chapter

3);

lack of information on how agencies process

research proposals prior to awards (chapter 4);

problematic estimates of research expenditures

in megaprojects (chapter 5);

need for comparative cost-accountability data,

by institution and source, on research expenditures (chapter 6); and

lack of baseline information on the Nation’s

research work force, as opposed to all scientists

and engineers (chapter 7).

Data collected on certain aspects of the Federal

research system—sources and dollars spent for

U.S. General Accounting Officel

research, academic degrees awarded, facilities and

instruments, and various outcome measures such as

publications and citations—are extensive.

In other areas, however, data are scarce, for

example, details on the research work force (as

opposed to the total science and engineering work

force), or what proportion of investigators-across

fields and agencies—are supported by Federal

funds. Also, compared to the National Science

Foundation (NSF) and the National Institutes of

Health (NIH), the other research agencies devote

few resources to internal data collection. Consequently, most analysis and research decisionmaking

must draw conclusions from the NSF and NIH data

systems. Since these agencies represent only part of

the spectrum of research supported by the Federal

Government, these analyses may omit key results

and trends at other agencies, or skew findings toward

biomedicine or academic research.

Furthermore, it is not clear how available agency

data are used to inform decisionmaking, as some

challenge current policy assumptions and others are

reported at inappropriate levels of aggregation. For

example, while there is much attention paid to the

rising cost of instrumentation and facilities, indirect

and personnel costs are rising at faster rates and

account for larger shares of Federal expenditures. In

this case, the issue is not information, but what can

be done with it by decisionmakers.2

In this chapter, OTA first summarizes the data that

are currently available. Table 8-1 lists the new data

that OTA gathered and examined for this report.

Iu.S. General Accounting Offke, Science Indicators: Improvements Needed in Design, Construction, @tdZnte~retatiOn, pAD-79-35 ~as~gto~

DC: 1979), pp. 5-6.

2A di~fiction is made fiou~out ~s c~pter between “decisionmakers” and “PoficYm*e~. “ The former comprise a considerably larger

population especially within the Federal research agencies; the latter are found at the very top of those agencies, as well as in the O131ce of Management

and BudgeL the Office of Science and lkchnology Policy, and Congress. Data speak to research decisionmakers at all levels, some of whom we not

responsible for policies.
234 ● Federally Funded Research: Decisions for a Decade

Table 8-l—Summary of OTA Data Collection and Analysis on Federally Funded Research

Description Methods of collection Subject

Original data collection and analysis:

Federal agency analysisa

Interviews, site visits, and

document review

University case studiesbInterviews and site visits

Bibliometrics c Citation analyses

Analysis of SEId

interviews and document

review

Researchers’ viewse Surveys

Secondary data analysis:

Research cost Comparisonsf NSF, NIH, and other datasets

Country surveysg

interviews and document

review

Congressional earmarking Budget analysis and document

review

Rhetorical analyses~ Document review

Research evaluationj

interviews and document

review

Analysis of Science Policy Task Document review

Force hearingsk

Priority setting and funding allocation

Rising research costs and responsiveness to changing priorities

“Hot” fields, related fields, university comparisons, and other

indicators

Evolution of SEI volumes, data presentation, and future analysis

Sigma Xi members’ perceptions of Federal research funding

issues

Rising costs of research

Priority setting, funding allocation, and research evaluation in

other countries

Budget information on congressional funding and definitions of

“earmarks”

Historical analysis of research decisionmaking by different

branches of government and goals of different ideological

groups

Post-1 985 developments in research evacuation in the United

States and abroad

Analysis of House hearings on science policy, 1985-87

asee ~x 4-A, chapter 4-

bsee box 6-B, chapter 6.

CHenry small ad David pendlebljry, ttF~eral sup~rf of Leading Edge Research,” OTAcontraCtOr reporf, Feb~a~, 19W; and Henry small, “Bibliometrics

of Basic Research, ’’OTAcontractor report, July 1990. See appendix Fforinforrnation about howtoobtain the latter reportandallotherOTAcontractor reports

listed below.

dsu=n ~=en~, i(~ienm l~icators: Descri@ion or ~escription?” OTA contractor report July 1990.

eJohn ~mmer, 1tRe~ear~her Perswtivm on the Federal Research system,” OTA contractor report, Juiy 1990.

fKathi Hanna, !iF~era] Funding ~ ~ic Research,~~ OTA ~ntractor repo~, Novem~r, IWO; and Harvey Averch, “AnaIyzing the tidS Of Fdeml

Research,” OTA contractor report, August 1990.

gROn Johnston, “project selection Methods: International CornparisonSr” (3TA contractor report, June 1990.

hJameS Savage, 1#A~emic ~rma~ and the Distribution of Federal Research Funds: A Poli~ Interpretation,” OTA contractor rewfi, Juk 1990.

isee Mark poll~, ~lBasic Research Goals: Pemeptions of Key political Figures,” OTA contractor report, June 1 Wo; and David Birdsell and Herbert Sil’llOnS,

“Basic Research Goals: A Comparison of Political Ideologies,” OTA contractor report, June 1990.

jHarvey Averch, “Policy Uses of ‘Evaluation of Research’ Literature,” OTA contractor report, July 1990.

kpat~k Hmlett, j~ask Form on ~ewe Poliq: A wi~o~ on the F~eral Funding ad Man~ement of Research,” OTA contrac~r report, &tObSr 1990.

KEY: SEI = Scier’tce & Engineering Inc#catom; NSF= National Science Foundation; NIH = National Institutes of Health.

SOURCE: Office of Technology Assessment, 1991.

Second, OTA suggests additional information that

could be collected, concentrating in areas of policy

relevance and on data that are amenable to manipulation in the aggregate by Congress and the executive

branch (especially the Office of Science and Technology Policy (OSTP) and the Office of Management and Budget (OMB)), and at less aggregated

program and project levels within the research

agencies. The emphasis is on the analysis and

presentation of data for monitoring changes in the

Federal research system. Finally, OTA considers the

utility of data for decisionmaking, revisiting the

problems of evaluating research projects (and updating conclusions of a previous OTA study).

Information is not cost-free. While it can illuminate the operation of the Federal research system, for

all participants and at many levels, the purpose is not

to generate needless paperwork and impose new
Chapter 8-Data on the Federal Research System . 235

reporting requirements on the agencies. What may

be appropriate for decisionmakers, in fact, is less

information, not more, along with better measures

and methods of applying and coordinating it.

What Data Are Available on the

Federal Research System?

Many organizations collect and analyze data on

the research system. First and foremost, is NSF, with

its numerous surveys, reports, and electronic data

systems that are publicly available. Other sources

include the other Federal research agencies; the

National Research Council (NRC); the Congressional Research Service (CRS); professional societies, especially the American Association for the

Advancement of Science (AAAS); and other public

and special interest groups.3

Together these databases and analyses provide a

wealth of information: time series on the funding of

research and development (R&D); expenditures by

R&D performer (e.g., universities and colleges,

industry, Federal laboratories), by source of funds,

and by type (basic, applied, development); numbers

of students who enroll in and graduate with degrees

in science and engineering (s/e); characteristics of

precollege science and mathematics programs and

students in the education pipeline; and size, sectors

of employment, and activities of the s/e work force

(especially Ph.D.s in academia). Detailed analyses

of the Federal budget by research agency are

available each year, and impacts on specific disciplines and industries can often be found.

NSF publishes many annual or biennial reports.

These reports summarize budget data from the

Federal agencies, academic R&D (which is covered

extensively, as academia is NSF’s primary client),

research at the Federal laboratories, funding and

performance of research by industry, academic

equipment and instrumentation expenditures, international comparisons, geographic distributions of

R&D funds, and other topics. NSF also publishes

detailed information on students, degrees awarded,

employment by sector, and the people who perform

research. Finally, NSF issues many individual rePhoto credit: National Aeronauts and Space Administration

An astronaut spins liquid in zero gravity aboard the Space

Shuttle Columbia to test the separation of bubbles from the

liquid. Research can take place in many different settings.

ports on specific topics either requested by Congress

or of particular interest to the scientific community.

Certainly the most visible compendium of data on

the research system is the biennial report, Science&

Engineering Indicators (SEI), issued since 1973 by

the National Science Board (NSB), the governing

3

Foras of major databases on science and engineering (individuals and institutions), see National Research Council, Engineering Personnel

Data DC: National Academy Press, 1988), app. A-2.
236 . Federally Funded Research: Decisions for a Decade

body of NSF.4 At 1976 hearings,5 NSB chairman

Norman Hackerman traced the origins of SEI to a

congressional mandate to NSB for its annual report,

which was to focus on ‘‘. . . the status and health of

science and its various disciplines (including) an

assessment of such matters as national scientific

resources, . . . progress in selected areas of basic

scientific research, and an indication of those aspects

of such progress which might be applied to the needs

of American society. ’ ‘6

From the outset, then, the

SEI project aspired to measure and evaluate the

results of federally supported R&D.

Table 8-2 lists eight broad categories of data that

have appeared in SEI, including impacts and assessments, resources, scientific performance, economic

performance, international contacts, cross-sectoral

linkages, literacy, educational pipeline, and scientific work force. Table 8-3 shows the distribution of

tables among data types in the nine SEI reports. Even

this broad-brush picture reveals a highly dynamic

volume. Over the years, 79 distinct subcategories of

data have appeared, about one-half in the original

1972 volume and about one-half added later. The

categories of international and cross-sectoral contact, literacy, and pipeline show steady patterns of

expansion in types of data. Resources, impacts and

assessments, and scientific performance indicators

have been stable, with some new types added. The

economic performance indicators show high turnover—many categories added and some dropped.

Publication and citation measures are still the main

forms of scientific performance data.7

SEI stands as the most comprehensive look at the

research system that is currently available. Some

find fault with the volume, however, because it is

based on an input/output model of science (i.e.,

“people and money enter the system, research

comes out”), which is thought to be simplistic,

omitting quantitative (and qualitative) measures of

the process of research.8 Others criticize SEI for its

concentration on academic or academically based

research and lack of emphasis on the researchtechnology interface.9

Each year, CRS and AAAS publish perhaps the

most comprehensive and widely read compilations

of Federal R&D spending (the former focuses on

appropriations, the latter on the proposed “R&D

budget”). These documents help to interpret, by

placing into an historical frame, the appropriations

bills signed into law by the President. Various

professional societies, e.g., the American Chemical

Society, also compile surveys of R&D spending,

salaries, and employment opportunities that are of

particular interest to their constituencies. In addition, the American Council on Education, the

Council of Graduate Schools, and the Association of

American Universities publish annual and occasional reports that characterize trends in research

university expenditures, administrator and faculty

d~e following discussion of science and Engitzee~ing Indicators (SEI) is based on SUS~ E. Cozzem, “Science Indicators: Description or

Prescription?” OTA contractor repo~ July 1990. Available through the National lkchnical Information Service, see app. F. Note that SEI was named

Science Indicators until 1987. Science& Engineering Indicators builds on data collected, published, and issued in many other reports by the Science

Resources Studies Division of the National Science Foundation.

s~e timing of these he~gs was importrmt in the development of the Indicators series. The 1972 volume had been resembled iU the co~se of a

few months by one staff person working with unenthusiastic and energetic Board committee. Aftera stormy anduncertainprocess of approval both within

the Science Board itself (who could not agree on how the numbers should be interpreted) and at the OffIce of Management and Budget and the White

House (who thought it presented administration policy in too unfavorable a light), the volume appeared amidst considerable fanfare in the science and

general press (as reported by Robert Brainard, the National Science Foundation staff member who prepared the fust report). See U.S. Congress, House

Committee on Science and TmhnoIogy, Subcommittee on Domestic and International Scientific Pkmnin g and Analysis, Measuring and Evaluating

Results of Federally Supported Research and Development: Science Output Itiicator&a~ I. Special Oversight Hearings, 94th Cong., May 19 and

26, 1976 (Washingto~ DC: U.S. Government Printing OffIce, 1976).

‘Ibid., p. 7.

TThe bibliome~c da~base has added more to the categories of international and cross-sectoral COnt@s than it hm tO memmes of scientilc

performance.

gFor other volumes that ad~ss these issues, see National Academy of Sciences, The Quality of Research in Science: Methods for Postpetiomnce

Evaluation in the National Science Foundation (Washington, DC: National Academy Press, 1982); Y. Elkana et al., Towarda Metric of Science (New

York, NY: Wiley, 1977); and H. Zuckerman and R.B. Miller (eds.), “Science Indicators: Implications for Research and Policy,” Scientometn”cs, vol.

2, October 1989, special issue, pp. 327-448.

sCozzem, op. cit., footnote 4. Because Science and Engineering Indicators (SEI) should reflect analytical advances iU ctiacttitig science ~d

technology, provision could be made for the support of relevant research communities outside of the National Science Foundation (NSF). According

to NSF’s Carlos Kruytbosch (personal communication, December 1990), at the very least “. . .biennial post-publication workshops to evaluate the SEI

report are workable and could be productive. ”
Chapter 8—Data on the Federal Research System . 237

Table 8-2—Categories of Data in Science& Engineering Indicators

International contacts

Cross-national citations and coauthorships, publishing in foreign journals, participation in international scientific congresses,

employment plans of foreign students, U.S. students and academics going abroad.

Cross-sectoral linkages

Citations from patents to the scientific literature, cross-sectoral coauthorship, cross-sectoral citation, mobility between sectors,

university patenting.

Economic performance

Patents, trade and trade balances, productivity measures, global investments, innovation indicators, high-technology business

sector, venture capital.

Impacts and assessments

Public views on allocation of resources for science, judgments of benefit and harm from science and technology, prestige of

scientists, expectations of scientific advances and problems caused by science, differences between the attentive and general

public.

Literacy

Enrollments in science and mathematics, course content and testing requirements, achievement and test scores, teacher

characteristics and activities, public understanding of scientific concepts, public use of technologies, student attitudes toward

science and technology.

Pipeline

College and graduate school enrollments in science, engineering, and mathematics; degrees; test scores and other quality

measures; preferences and plans of high school and college students; sources of student support.

Resources

Expenditures and obligations, special research resources, instrumentation and facilities.

Scientific performance

Publication and citation counts, Nobel Prizes.

Work force

The science and engineering work force: comparative measures, demographic characteristics, career variables, sources of

support, technicians, stock and flow analysis.

SOURCE: Susan Cozzens, Science Indicators: Description or Prescription?” OTA contractor report, July 1990.

issues, and Federal support for education and

research. 10

Recently, the Government-University-Industry

Research Roundtable of the National Academy of

Sciences, with data compiled by NSF’s Policy

Research and Analysis Division, provided much

useful analysis on the state of academic R&D and

changes since the early 1960s.11 In addition, NRC

periodically publishes reports on sectors of the

research system and on the availability of data to

characterize the system. 12 These publications provide a basis for understanding the Federal research

system. But even with each of these organizations

devoting significant resources to the collection of

information, better data are needed to guide possible

improvements of the system.13

What Data Are Needed?

Recognizing that data collection is often very

difficult, and certainly time consuming, OTA concentrated on notable gaps in the empirical baseline.

One overarching problem is that comparable data

IOA montiy compendim that announces and annotates new reports containing data and analysis on trends in science and engineering k Manpower

Comments, published by the Commission on Professionals in Science and Echnology, a participating organization of the American Association for the

Advancement of Science.

1 IGov ernment-University-Industry Research Roundtable, Science and Technology in the Acade~”c Enterpn”se: Status, Trends, and Issues

(Washingto~ DC: National Academy Press, October 1989).

lzFor example, see Natio~ Resemch Council, Surveying the Nation’s Scientists and Engineers: A Data System for the 1990s ~astigto% DC:

National Academy Press, 1990). Under multiagency suppofi the National Research Council collects, analyms, and disseminates information on Ph.D.

recipients. For a statement of its cross-cutting role, see National Academy of Sciences, The National Research Council: A Unique Institun”on

(Washingto~ DC: National Academy Press, 1990).

ls~ese effo~ must alSO be s~n in the context of the massive Federal data system. The components most relevant to research me the data s~~

compiled and reported by the Census Bureau, the Bureau of Economic Analysis, the Bureau of Labor Statistics, and the National Center for Education

Statistics. The point is illustrated by calls for ways to measure how many people who aspire to attend college actually enroll. In the words of one

sociologist: “We care to know on a month-to-month basis what the unemployment rate is. I think we ought to care to know on at least a year-to-year

basis what the rate of access to higher education is. ” Quoted in Thomas J. DeLoughry, “U.S. Asked to Set Student-Aid Goals for Poor and Minority

Students,” The Chronicle of Higher Education, vol. 37, No. 20, Jan. 30, 1991, p. A20.
238 . Federally Funded Research: Decisions for a Decade

Table 8-3-Trends in Distribution of Data Among Categories in Science & Engineering Indicators: 1972-88

Number of tables

1972 1974 1976 1978 1980 1982 1984 1986 1988 Total

Resources . . . . . . . . . . . . . . . . . . 38

Work force . . . . . . . . . . . . . . . . . . . 18

Economic performance . . . . . . . . 9

Impacts and assessments. . . . . . 21

Pipeline . . . . . . . . . . . . . . . . . . . . . 23

literacy . . . . . . . . . . . . . . . . . . . . . 1

Scientific performance . . . . . . . . . 2

International contacts. . . . . . . . . . 0

Cross-sectoral contacts . . . . . . . . 0

Nonindicators . . . . . . . . . . . . . . . . 0

Total . . . . . . . . . . . . . . . . . . . . . 112

63

37

31

14

13

0

10

2

3

0

173

57

35

47

21

9

0

11

8

0

0

188

58

35

22

0

3

0

10

10

3

0

141

Percent of total

41

54

40

39

1

1

11

11

8

1

207

52

37

37

35

14

4

15

11

7

1

213

49

29

20

19

14

7

11

6

1

47

38

35

29

29

46

4

9

12

0

60

30

45

18

26

54

7

8

7

3

178 249 258

465

313

286

196

140

120

77

70

46

6

1719

1972 1974 1976 1978 1980 1982 1984 1986 1988 Total

Resources . . . . . . . . . . . . . . . . . . 34

Work force . . . . . . . . . . . . . . . . . . . 16

Economic performance . . . . . . . . 8

Impacts and assessments . . . . . . 19

Pipeline . . . . . . . . . . . . . . . . . . . . . 21

Literacy . . . . . . . . . . . . . . . . . . . . . 1

Scientific Performance . . . . . . . . . 2

International contacts . . . . . . . . . . O

Cross-sectoral contacts . . . . . . . . O

Nonindicators . . . . . . . . . . . . . . . . O

Total . . . . . . . . . . . . . . . . . . . . . 100

36

21

18

8

8

0

6

1

2

0

100

30

19

25

11

5

0

6

4

0

0

100

41

25

16

0

2

0

7

7

2

0

100

20

26

19

19

0

0

5

5

4

0

100

24

17

17

16

7

2

7

5

3

0

100

28

16

11

11

12

8

4

6

3

1

100

19

15

14

12

12

18

2

4

:

100

23

12

17

7

10

21

3

3

3

1

100

27

18

17

11

8

7

4

4

3

0

100

NOTE: Each table, text, or appendix is counted once.

SOURCE: Susan Cozzens, “Science Indicators: Description or Prescription?” OTA contractor report, July 1990.

associated with the research operations of all the

Federal agencies are lacking. NSF and NIH conscientiously log data on what proportion of proposals submitted to them are awarded finding, the

number of researchers they support, expenditures by

categories of project budgets (e.g., indirect costs and

personnel), and other dimensions related to management of their research programs. However, other

agencies collect only R&D budgetary information,

primarily in response to OMB requests and NSF

surveys of research conduct. Much more data could

be collected on research funding and performance in

these agencies. In particular, further information

could be collected on proposal submissions as well

as awards, research expenditures by line items of the

budget (requested and expended), and the size and

distribution of the research work force that is

supported. Comparable data from the agencies are

important for decisions that span agencies or broad

segments of the scientific community. With the data

that are currently available, Congress and other

Federal policymakers risk overgeneralizing from

what is known about research performance that is

supported by NSF or NIH

Some advocate that NSF should be the sole

agency to centralize and standardize the analysis,

especially since NSF has the mandate to collect

R&D funding data from the other research agencies. 14 However, OTA found that the research

agencies are sufficiently diverse in their organization and finding structures to create difficulties for

any outside agency to translate data in comparable

ways. 15 For example, breakdowns of R&D into basic

research, applied research, and development are

Idsee commission onprof~sio~ in Science and ‘lkchnology, iUeasun.ng National Needsfor Scientists to the Year2000, report of a workshop, Nov.

30-Dec. 1, 1988 (waShillgtOQ DC: July 1989).

ls~tiough probl~s IMY efit with detitio~, cc)rnpli~ by the reporting organizations with whatever definitions are uSed k mo ~isme. The

advantage of an interagency mechaniwq such as the Oft3ce of Science and ‘Ikchnology Policy’s Federal Coordinating Council for Science, Engineering,

and ‘lbdmology (IWCSET) committees, is its place in the Federal hierarchy: the agencies are likely to be responsive to requests for “crosscutting”

information where budgets are at stake. The FCCSET Committee on Physical, hlathematic~ and Engineering Sciences, for example, currently has a

“structure of science’ activity that includes the solicitation of data from the research agencies similar to those sought in tbis OTA study.
Chapter 8—Data on the Federal Research System . 239

very difficult to measure and often judgments are

made after-the-fact. NSF fields a survey to all

Federal R&D agencies asking for detailed estimates

of spending in various categories. Because of

problems with applying definitions and with converting agency accounting of research dollars into

the separate categories, however, many of the

agencies claim that it is impossible to provide

accurate answers to the NSF survey.

In 1989, NSF continued its effort to develop a

better taxonomy of the research it funds.l6 “Fundamental,’ “ directed,” and “development” seemed

to be the preferred categories, though some programs found it difficult to translate currently supported projects into these three categories. Unfortunately, any taxonomy would suffer from arbitrary

divisions of research topics among categories. Also,

“basic” and “applied,” or similar definitions, are

rarely used by managers to allocate monies; rather

these distinctions are most important to the researchers who perform the research, since basic research is

synonymous with enhanced investigator discretion

over research directions, while applied research is

often associated with the attainment of specific

objectives. 17 Consequently, basic and applied divisions are less important for decisions that concern

specific programmatic goals; however, they are

quite important to decisions about the science base

supported by the Federal Government.

Enhanced data collection at each agency would

help NSF fulfill its data mandate, and advance

development of comprehensive research strategies,

especially programs that span agencies.18 Other data

that could be very useful fall into four categories: 1)

research monies—how they are allocated and spent;

2) personnel-characteristics of the research work

force; 3) the research process-how researchers

spend their time and their needs (e.g., equipment and

communication) for the conduct of research; and 4)

outcomes—the results of research.

Research Monies

While the data collected on research sponsored by

the Federal Government are abundant, information

on research expenditures is not. In particular, direct

and indirect costs in all sectors of the research

system supported by the Federal Government could

be monitored.19 NSF and NIH have collected

longitudinal data on research expenditures in individual investigator grants, but complementary data

are needed on expenditures in Federal and industrial

laboratories, research supported by other agencies,

and on other types of research groups and cooperative ventures such as centers and university-industry

collaborations. These data would help to monitor

fluctuations in research expenditures. At present,

predictions of future spending merely extrapolate

from the gross totals disaggregated by sector, while

individual components of the budgets may be

increasing or decreasing relative to overall trends.

These data would be especially helpful for revising

estimates of start-up and operating costs in science

megaprojects.

Another measure that would refine the knowledge

of research expenditures would be breakdowns by

field. (This is available for some academic research

disciplines and Federally Funded Research and

Development Centers only.) Many claims are made

about the cost requirements of specific fields. For

instance, research in some physics specialties is

inherently more expensive than in others, because of

the equipment required by research groups. At

IGNSFTask For~ on Research and Development ~OnOmy, “Final Report,’ unpublished documen~ 1989. For anearliereffo~ see Nationrd Science

Foundation Categories of Scientific Research (Washington DC: 1979).

ITSee~ey Averc~ ‘The Political Economy of R&D Taonoti=, ‘‘ Research Po/icy, forthcoming 1991; and Richard R. Ries and Henry Hertzfeld,

“Taxonomy of Research: ‘I&t of Proposed Definitions on the NSF Budge4° unpublished documentn.d.

18For e-pie, tile Natio~ ~ti~t~ of H~th ~~ ~ide 1 Wrcent of its r~~ch budget for ms~ch ev~~tion md internal iUldySiS Of the

investigators and programs it supports. The Department of Energy, the National Aeromutics and Space Admnu“ “stratioq the OffIce of Naval Researc4

and the National Science Foundation have all conducted ad hoc inhouse evaluations of the research they support and the efficiency of the operations

needed to select and manage various research portfolios (see below). For an example of agency-based research evaluation data that could be assembled

in an ongoing way, see Daryl E. ChubQ “Designing Research Program Evaluations: A Science Studies Approac~” Science and Public Policy, vol.

14, No. 2, Apti 1987, pp. 82-90.

l~s mofitofig is not tie sme ~ fie audi~g of cost dam by Categow Of exWndi~, as ~dat~ by ~lce of Management ~d Budget CirCUIW

A-21 and as conducted by the Department of Defense and the Department of Health and Human Services contract audit agencies. That is done for

accountability purposes. Congress seeks better information m how investigators and their teams actually spend money in the course of executing

federally funded research projects, which requires some demystifkation of university accounting schemes. For examples of studies of data audit

methodologies, see the new quarterly journal, Accountability in Research: Policies and QuuZi~ Assurance, edited at the University of Maryland School

of Medicine.
240 . Federally Funded Research: Decisions for a Decade

present, there are no means to evaluate these

claimS.

20 Yet for decisions that must balance the

present and future needs of different sectors of the

research system, such cost estimates and the trends

associated with them could be very important.

Finally, data on how Federal agencies allocate

monies within project budgets could be compiled.

Agencies have much experience in negotiating

budgets. Data would illuminate how judgments are

made about specific categories of expenditure, e.g.,

in reducing “inflated’ budget requests of investigators, imposing an artificial ceiling on equipment

purchases, or adjusting allocations through NIH’s

practice of “downward negotiation.” Since personnel costs have grown quickly compared with other

research expenditures, financial analyses would be

greatly enhanced by better personnel data.

Personnel

One of the most fundamental pieces of information on the research system is the size of the research

work force, both in absolute numbers and as a

fraction of all U.S. employed scientists and engineers. These numbers depend on how ‘‘researcher”

is defined.21 While estimates exist of the rise in

Ph.D. personnel employed in research universities,

very little detailed data exist for industry or other

sectors of the research system. Estimates of the

positions held by Ph.D. personnel in academia are

inadequate. Distinguishing nonfaculty research associates from postdoctoral fellows and full-time

equivalent faculty is analytically important-and a

nightmare to sort and track over time. Accurate

estimates of the changing size of the research work

force and how many are federally funded-and are

seeking such support—would aid in measuring

current and unfunded academic research capacity. In

addition, accurate estimates of the numbers o f

researchers exiting the system would help to gauge

the attractiveness of specific fields, as well as the

category “science and engineering” relative to

other occupations .22

Another trend that has been noted in this report,

mostly with anecdotal evidence and inferences from

analyses of expenditures, is the increasing size of

research groups, both within the university structure

and through Federal support of centers. This trend

has policy implications for the cost of research, its

interdisciplinary capabilities, and the changing demographics of the work force. It also reflects how

researchers may spend their time. More data on

“production units” in research and their dependence on Federal funding relative to other sources

would augment enrollment, Ph.D. award, and work

activity data. Changes in the structure of production

units have also influenced the research process and

the volume-and perhaps the character-of outcomes.

23

Research Process

“How research is done” has evolved since the

1960s. In particular, the organization of research

groups and the settings in which research is conducted have changed.24Data on the conduct of

research would aid in understanding the opportunities and stresses on the Federal research system and

in planning how the research system can adapt to

changing conditions.

For instance, it is often claimed that researchers

are spending much more time writing proposals, and

that their research suffers as a consequence. No

systematic data exist either to support or refute this

claim. While it is in the interest of Federal sponsors

for their grantees to spend as much time as possible

in the conduct of research, investigators report that

the increased competition for Federal funds compels

2oFOr arecent effort to look comprehensively at Federal support, by agency and overtime, of one sector of one field, see Amerim Chemid Society,

Department of Government Relations and Science Policy, Federal Funding of Academ-c Chemistry Research, FY 1980-FY 1988 (Washington DC:

November 1990).

21A ‘cre~ewcher~~ could ~ defined as ~yone publi~~g a scien~lc paper (i.e., by autho~hip), possess~g a M.D. (i.e., by credential), or WOrkitlg

in a particular setting (i.e., by sector). Indeed, the problem of defining who is a“scientist’ also applies here. See Derek de Solla Price, Little Science,

Big Science (New Yorlq NY: Columbia University Press, 1963). Also see National Research Council, op. cit., footnote 3.

22For a discussion of methodological Pitfaus associated with assessing, for example, characteristics of the Federal work force, see U.S. G~e~

Accounting Office, Federal Work Force: A Framework for Studying Its Quality Over Time, GAO/PEMD-88-27 (Washington DC: August 1988).

~~e ~le of 1a~mtoW chief or tem leader mmb~es en~eprenefial ad a-s@ative/suywisoV tasks. Both ~essenti~ to the tidkg ~d

longevity of the productive research unit. On the emergence of the entrepreneurial role on carnpus, see Henry Etzkowhz, “Entrepreneurial Scientists

and Entrepreneurial Universities in American Acadmic Science, ’ Minerva, VO1. 21, surnmer-autumn 1983, pp. 198-233.

~For a prophetic discussion s= B.C. Griffith ad NC. Mum, ‘ ‘Coherent Social Groups ~ Scienfic~ge, ” Science, vol. 177, Sept. 15, 1972,

pp. 959-964.
Chapter 8—Data on the Federal Research System . 241

Photo credit: U.S. Department of Agriculture

Researcher picks blueberries-95 percent of the varieties

of blueberries in production today were developed by

Department of Agriculture scientists. Research is

performed in many settings.

proposal writing. 25 However, one might expect that

as the size of academic research groups grows,

principal investigators will spend more time seeking

money to sustain their larger research teams and

programs. 26 This phenomenon is similar to strategies in a law or consulting practice, where the

addition of less senior associates leverages the effort

of the more senior employees to spend more time

marketing and winning projects for the firm. In the

academic research community, entrepreneurial pursuits are very different from research and teaching,

and the additional burden can be a source of stress

for senior researchers.27

Many also claim that increasing time commitments required by research pursuits hamper the

ability of faculty to meet their teaching responsibilities. Data on how faculty apportion their time have

been unreliable. Ironically, self-reports in compliance with Federal accountability requirements tend

to distort estimates of time spent on various work

activities. 28 Since the Federal Government invests in

the academic research system to maintain a strong

instructional as well as knowledge-producing capability, shifts in the activities of researchers is of

central concern.

Data are needed on how apprentice, junior (e.g.,

postdoctorates), and senior researchers spend their

time on research (collecting data and analysis),

proposal writing, teaching (classroom and one-onone), travel, presenting results to scientific colleagues, and other pursuits.29 Differences between

time commitments in Federal, industrial, and academic settings could also be judged.30

More generally, data could be collected on

changing equipment needs. The average lifetime of

a scientific instrument has shrunk during the 1980s

from an average of 7 years to less than 5 years.31

Additional data could address such questions as:

how does the reliance on equipment vary across

fields? What happens to obsolete equipment? As

 see Science: End Frontier? a from M. President-Elect to of of 

Association for the Advancement of Science (Washington DC: Association for the Advancement of Science, January 1991).

 T. “Research Trails and Policies: and Negotiations of Scientific

Establishments and Hierarchies, Sociology of the Sciences, Yearbook vol. 6, N. (cd.) Holland: D. 1982), pp. 293-311.

 entrepreneurial behavior, see Karen Seashore ‘‘Entrepreneurs in An Exploration of Behaviors Among Life

Scientists,” Administrative Science Quarterly, vol. 34, 1989, pp. 110-131.

 Kennedy, ‘‘Government Policies and the Costs of Doing Research,” Science, vol. 227, Feb. 1, 1985, pp. 480-484.

 example, data could changing patterns of communication among scientific colleagues. With new communications technologies, such

as electronic mail systems and computer networks, scientists have the ability to exchange data and ideas much more often. Is science more

collaborative (or competitive) due to these innovations? Do most scientists have access to these technologies? Are some at a disadvantage without them?

 scientists, for Life: The 

Construction (Beverly CA: Sage, 1979); and Science in Action: How To Follow Scientists andEngineers Through

Society (Cambridge, MA: Harvard University Press, 1987).

 in Science/Engineering to SRS 

(Washington DC: June 1988). As the National Science Foundation’s (personal communication December 1990) points there

is no information of average ‘‘equipment use rate, ” or what proportion of available time an instrument is in use.
242 . Federally Funded Research: Decisions for a Decade

communications and other technologies progress

and the scientific community comes more to rely on

them, these questions will increasingly impact

Federal funding.

A final area of “process” on which data would be

instructive are the standards for achieving various

positions in the scientific community. Many claim

that graduate students must publish more papers to

be offered first jobs after receipt of the doctorate or

completion of a postdoctoral fellowship. What are

the average age, experience postcollege (in years),

and publication records of new hires at research

universities, and industrial and Federal laboratories?

For other promotions? Such data would help the

Federal Government to track the changing research

labor market.32

Outcomes of Research

Because of the fundamental and elusive nature of

research, measuring its outcomes—in knowledge

and education—is very difficult.33 The most elusive

outcome is cultural enrichment-the discovery and

growth of scientific knowledge. As OMB Director

Richard Dar-man has said (speaking of the proposed

Moon/Mars mission): “No one can put a price on

uplifting the Nation. Research has resulted in many

benefits to the Nation and is funded precisely

because of those benefits. This kind of benefit is

nearly impossible to measure. However, there are

some proxies.

When looking at research as a contribution to

education, numbers of degrees can be tallied and

assertions about skills added to the Nation’s work

force can be made. When looking at research as

creating new knowledge, one tangible “output” is

papers published by scientific investigators to communicate new information to their scientific peers.

Communicating the results of scientific research to

colleagues through publication in the open literature

is considered to be an important, if not essential,

feature of good research practice.34 Perhaps the best

approach is to construct workable indicators and

include a rigorous treatment of their uncertainties.

+

Bibliometrics

One tool that has been vigorously developed

(especially in western Europe during the 1980s) for

measuring the outcomes of research is bibliometrics,

the statistical analysis of scientific publications and

their attributes. Intrinsic to scientific publication is

the referencing of earlier published work on which

the current work is presumably based or has utilized

in some way. References are a common feature of

the scientific literature, and by counting how often

publications are cited, bibliometrics can arrive at a

weighted measure of publication output-not only

whether publications have been produced, but also

what impact those publications have had on the work

of other scientists.35

OTA has explored several examples of new data

sets that could be compiled using bibliometrics.36

First, universities can be ranked according to an

output or citation measure, the citation rates for

papers authored by faculty and others associated

with each institution. OTA drew on the large

electronic database created and maintained by the

Institute for Scientific Information (ISI).37 Each

institution in ISI’s Science Indicators database, 1973

to 1988, was listed by its total number of cited

32For ~tha ~ugge~tion~, SW co~~~ion on fioftisio~s ~ Science ~d ~chnology, op. cit. foo~ote 14, The above (hy’pothetic~) data alSO raise

the question of research outcomes--those relating to individual performance and that of other production units in the Federal research system.

s3For a ~omprehemive review (now a d-de old) of attempts at su~hrne~ernen~ see NatiO~ Academy Of sci~ces, OP. cit., foof.flote 8, espwidly

ch. 2.

~Ro~fi K. Mefion, ~~~e ~~ew Effect ~ Science, ~: c~~tive ~~antage ~d be spbolism of ~te~ec~ property,” l~is, VO1. 79, No. 299,

1988, pp. 606-623.

ss~terpreting citation patterns remains a subject of contention. For caveats, see D.O. Edge, “Quantitative Measures of Communication in Science:

A Critical Review,’ History o~Science, vol. 17, 1979, pp. 102-134; and S.E. CozzenS, “TakingtheMeasure of Science: AReviewof Citation Theories,”

ZSSK Newsletter, No. 7, 1981, pp. 16-21. The definitive overview is contained in Eugene Garfield, Citation Indem”ng: Its Theory and Application in

Science, Technology andHumanities (New York NY: John Wiley & Sons, 1979). Also see Francis Ntu@Evaluative Bibliometn”cs: The Use of Citation

Analysis in the Evaluation of Scientific Activity (Cherry Hill, NJ: Computer Horizons, Inc., 1976).

~See Heq S~and David Pendlebury, “Federal Support of Leading Edge Research: Report on a Method for Identifying Innovative Areas of

Scientific Research and Their Extent of Federal Support,” OTA contractor report, February 1989; and Henry Small,‘‘Bibliometrics of Basic ResearcIu”

OTA contractor report, September 1990. For OTA contractor reports available through the National lkchnica,l Information Service, see app. F.

37’f’he ~ti~te for scien~lc ~omation gsr) database mvem 7,500” jo~s published worldwide ~d fidexes g~,~ new articles ~ch year. The

files derived from the 1S1 databases cover multiple disciplines and countries, and extend back to 1973. The analysis below is based on Small, op. cit.,

footnote 36. The Science Indicators File is a specially constructed multiyear fde of publications fium 1S1’S Science Citation Index, which contains a

citation count time series for each paper in the fde that has been cited one or more times for the years 1973 through 1988 inclusive.
Chapter 8—Data on the Federal Research System . 243

Table 8-4-Mismatches in Rank Between Federal Funding and Average Citations: 1988

Of the top 100(107) federally funded universities, only 17 did not Of the top 100 cited schools, only 17 did not make the top 100

make the top 100 citation list. They are (with funding rank in (107) most funded schools. They are (with citation rank in

parentheses): parentheses):

Texas A&M (22) University of California, SantaCruz(14)

University of Florida (45) University of Oregon (23)

Woods Hole Oceanographic Institute (52) SUNY Albany (43)

New Mexico State University (61 ) Rice University (58)

Louisiana State University (72) University of California, Riverside (59)

Utah State University (74) St. Louis University (70)

North Carolina State at Raleigh (75) Creighton University (80)

Virginia Polytechnic Institute and State University (76) University of Notre Dame (81)

University of Kentucky (82) University of Houston (82)

University of Dayton (86) University of New Hampshire (84)

University of Nebraska at Lincoln (89) University of Alaska (89)

Wake Forest University (95) University of South Alabama (90)

University of Medicine and Dentistry of New Jersey (99) College of William and Mary (93)

Washington State University (100) Howard University (94)

University of Missouri, Columbia(101 ) Brigham Young University (96)

Medical College of Wisconsin (104) University of Delaware (97)

Rensselaer Polytechnic Institute (107) University of Oklahoma (98)

NOTE: To compare the top 100 rankings, some institutions in the top 100 federally funded universities were disaggregate by earnpus of the State university

system, e.g., the University of Texas, Austin. This added 7 entries to the top 100.

SOURCES: National Scienee Foundation, Academic Science@)girweting: R&D Funds, Fisca/ Year 1988, NSF 89-326 (Washington, DC: 1990), table B-37;

and Henry Small, “Bibliometrics of Basic Research,” OTA contractor report, July 1990.

papers, the total citations received by all papers

associated with each institution, and the ratio of

number of citations to the number of publications,

namely, the average citations per cited paper. This

is a more discerning measure than either publication

or citation counts alone. A ranking of institutions by

average citation rates can be used in conjunction

with the list of top universities in Federal R&D

finding received to link inputs with outputs. (Appendix E lists the top 100 academic institutions

ranked by their average citation impact for the period

1981 to 1988.) Table 8-4 lists the institutions, in

1988, that were among either the top 100 academic

institutions in average citations or the top 100

receiving Federal R&D funds (again, see appendix

B), but not both. Together, these measures illuminate differences in rank. The overlap in institutions

suggests that the funding decisions by the Federal

Government for the most part are leading to productive research. The mismatches may be indicative

both of concentrated, rather than broad-based research productivity, and either some institutional

“overachievement’ or a substantial supplementation of Federal research support by State, corporate,

and nonprofit sources.38

Trends in the average citation rate over time can

also indicate how productive an institution has been

in the published literature. The citation set can be

analyzed by broad field or other variables to try to

determine the cause of the changes (see box 8-A for

profiles of four universities). Institutions can also be

grouped to look at how, for example, “private

institutions in the Southwest” or the national laboratories are performing as a category39 (see figure 8-l).

Many companies and other types of research organizations, despite proprietary inhibitions, also publish

in the scientific literature and their work can be

similarly aggregated and displayed (see figure

8-2). In another example for future exploration,

programs receiving primarily directed funds or

block grants (e.g., in agriculture) could be compared

with those that are investigator-initiated. This comparison would help to test the claim that targeted

appropriations (e.g., earmarking) lead to the production of inferior research.

ss~ fisti~tion titr-~gh on ~~and IOW on citation impact is not necessarily an underachiever. Some research is not readily published in

the open literature, for proprietary or national security reasons.

S%or ex~ple,the publication records and citation impact of National Aeromutics ~dSPme~“ “stration research centers, 1973 to 1988, are

examined in “NASA’s Citation Impact Dims in 1980s, But Voyager Missions and JPL Shine,” Science Watch, vol. 1, No. 9, October 1990, pp. 1-2,

7-8.

‘loBiotechnology research is more often reported in the open literature than either researchfiom electronics and computing f- or from Fortune 500

companies. Thus, the samples used in figure 8-2 may not represent the full range of research activity in these industries. Indeed, the most exciting results

may be withheld from publication, but might be reflected in patents awarded later.
244 ● Federally Funded Research: Decisions for a Decade

Box 8-A—Bibliometric Profiles of Four Research Institutions

OTA selected 19 institutions, based on historical patterns in their Federal funding profiles, to examine changes

in research output and probe how they might be accounted for bibliometrically. 1 The institutions’ publication and

citation records were extracted to obtain a ‘‘citation impact’ time series. This requires specifying four time points:

a beginning and ending cited item period, and a beginning and ending citing item time period. This defines what

items are eligible to receive citations and what journal publications are eligible to give them. OTA began with 1973,

and defined the length of the period for analysis to be 8 years. This yields nine successive overlapping time windows

that can be plotted as a time series or moving picture of the citation impact for each institution through 1988.

For example, counted in the first window were the number of cited papers published from 1973 through 1980

and the number of times those papers were cited by papers published in the same period. The ratio of these quantities

is the mean citations per cited item for that time window. As a further normalization, each of the impacts is divided

by the overall average for all U.S. papers for the specified time window, e.g., 1973 to 1980. The result is a measure

of relative impact. Thus, a relative impact score of one signifies that the institution’s average is identical to the

average for all U.S. papers in the window. A score greater than one signifies an impact above the U.S. average, and

a score below one an impact below the U.S. average.

The time series plots of relative impact for 4 of the 19 selected institutions are shown in figure 8A-1. To explain

the trends observed in these graphs in terms of the fields of science involved, listings of the most cited papers were

obtained for each institution, covering items cited 100 or more times, down to a maximum of 100 items.

1) Massachusetts Institute of Technology (MIT) has been consistently among the top 10 institutions for

Federal R&D funds received. Like other top 10 institutions, which often produce relative impacts at the

national average or above, MIT exhibits relative impacts in the 1.4 to 1.5 range. MIT also shows a modest

gain in citation impact. Twenty-nine percent of its most cited papers are from the 1981 to 1988 period.

Biomedicine has become stronger, while chemistry and geoscience have tapered off, and physics remained

about the same.

l~e following IS bawd on Henry small, ‘‘Bibliometrics of Basic Research, ’ OTA contractor repo~ July 1990. Available through the

National T@mical Information Service, see app. F.

Figure 8A-1—Average Relative Citation Impact for Four Research Institutions,

1973-88

Citations /paper

1.6

1.5

1.4

1.3

1.2

1.1

1

0.9

0.8

0.7

0.6

 1.6

I

t

- 1.3

, / / ’

Carnegie Mellon 1.1

- 0.7

73-80 74-81 75-82 76-83 77-84 78-85 79-86 80-87 81-88 -

Years

SOURCE: Henry Small, “Bibliometrlcs of Baslc Research, ” OTA contractor report, July 1990.
Chapter 8—Data on the Federal Research System ● 245

2) University of Calofornia-Sarzta Barbara (UCSB) has improved its ranking among the top 100 recipients of

Federal R&D dollars from 1%7 to 1984. OTA calls such institutions “upwardly mobile.” Their patterns

of research output are even more diverse than their relative gains in funding. UCSB displays a very marked

increase in citation impact. It also has a very large number of 1981 to 1988 papers in its highly cited s@,

43 percent. Even more remarkable is the spread of these papers over various disciplines, with the emphasis

on physics. Of the recent highly cited papers, 81 percent are in physics. Other areas represented include

biomedicine, ecology, geoscience, and chemistry.

3) Carnegie-Mellon University (CMU) has been a top 20 recipient of Federal funds in engineering, and

mathematics and computer science. However, it shows a decline in relative impact, beginning in the late

1970s. An analysis of the 78 papers cited 100 times or more shows that 15 percent of these papers are in

the period 1981 to 1988. While 23 percent of the 1973 through 1980 papers were in the discipline of physics,

only 8 percent are from physics in the later period. Chemistry, biomedicine, and computer science continue

from earlier to later periods at comparable levels.

4) University of Houston(UofH) is a new comer to the select group of top 100 recipients of Federal R&D funds.

It displays one of the most marked increases in citation impact of the institutions examined, although it

started at a very low level. Its number of papers cited over 100 times is also small at 28. Nevertheless, 39

percent of these are from the recent period. Whereas chemistry and biomedicine were dominant early,

physics (and more specifically, high-temperature superconductivity) account for most of the new highly

cited papers (though biomedicine is also represented). Possibly a shift toward strengthening physics

contributed to the increase m impact for this institution.

In some of these cases, it may be possible to attribute changes in citation impact to a shift in the field orientation

of an institution. Such shifts maybe the result of deliberate org anizational changes, or perhaps due to a resourceful

faculty member who is able to move into new areas of research. One key to increasing impact is the ability to produce

a continuing flow of innovative papers that influence researchers ‘‘at the front. ” This relates to the proportion of

highly cited papers that are of recent origin. Reliance on aging ‘classics” will not ensure an upward trend in impact.

Another factor is field balance: some institutions seem to have strength across a number of fields, while other

institutions focus on one or two seemingly to the exclusion of others. It is clearly more difficult for an institution

to maintain excellence across a wide range of fields-the traditional mark of a research university--than to

specialize in one or two.2

One lesson from the institutional profiles is that maintaining a high citation impact over a generation is difficult

at best. The citation trends for UCSB and UofH confirm their upward mobility in research output as well as in

Federal funding, in contrast to the citation trends at other institutions.

2Of the top 100 institutions in Federal R&D funding in 1988, only 39 had a relative impact score above the national average. Ibid.

Not only can publishing entities be analyzed, but

fields of study as well. For instance, ‘‘hot fields,’ in

which the rate of publication and citation increases

quickly over a short period of time, can be identified.

Research areas such as high-temperature superconductivity emerge after a major discovery. Through

co-citation analysis, papers can be sorted into

‘‘clusters of publications that cite each other.

These research clusters can be grouped further and

mapped within disciplines.41 In addition, related

areas that contribute to the work can be identified

and linked across disciplinary boundaries. For example, high-temperature superconductivity research

has been connected with work in ceramics, thin

films, polymers, and other diverse areas. 42

If the papers comprising a cluster cite their

sources of funding, an estimate can be made of

Federal support of the research represented by the

clusters. To demonstrate this method, OTA requested that a small sample of papers published be

searched for funding information in a cluster repredlForelaboratiom of thealgori~~d the interpretation of resulting co-citation maps, see Henry Small and B.C. Griffith, ‘The Shucture of Scientilc

Literatures I: Identifying and Graphing Specialties,” S’cience Studies, vol. 4, 1974, pp. 17-40; and Henry Small and Eugene Garfield, “The Geography

of Science: Disciplinary and National Mappings,” Journal of Information Science, vol. 11, December 1985, pp. 147-159.

dzstil ~d pendleb~, op. cit., foo~ote 36. These connections have been confirmed independently through analysis of other, nonbibliometric da~.

See John M. Rowell, ‘Superconductivity Research: A Different View, ’ Physics Touizy, November 1988, pp. 3846; and Dorothy Robyn et al., ‘‘Bringing

Superconductivity to hhrke~” Issues in Science& 7&chnoZogy, vol. 5, No. 2, winter 1988-89, pp. 38-45.
246 ● Federally Funded Research: Decisions for a Decade

Figure 8-l—Relative Citation Impact for National Laboratories

Citations/paper

1.4

1.3

1.2

1.1

1

0.9

0.8

Includes:

Argonne National Lab

Brook haven National Lab

Fermi National Accelerator Lab

Lawrence Berkeley Lab

Lawrence Livermore National Lab

Los Alamos National Lab

Oak Ridge National Lab

Sandia National Lab

I I I I I I I I I

73-80 74-81 75-82 76-83 77-84 78-85 79-86 80-87 81-8 8

Years

SOURCE: Henry Small, “Bibliometrics of Basic Research,” OTA contractor report, July 1990.

Figure 8-2—Relative Citation Impact for Three Industries

Citations/pape r

3 . 5

/

3 -

2.5 -

2 -

1.5 -

*

1 -

0 . 5

73-80 74-81 75-82 76-83 77-84 78-85 79-86 80-87 81-88 

I I I I n~ ~—

-

Years

+ Biotechnology — Electronics and + Fortune 500 top 20

computers

NOTE: Biotechnohgycompanies indude:Amgen, Biogen, Biotech Labs, California Biotech, Centocor, Cetus, Chiron,

DNAX, Genentech, Genetics Institute, Immunex, and Molecular Genetics Inc. Electronics and computer

wmpanies include: IBM, Digital Equipment, General Electric, Westinghouse Electric, Eastman Kodak, and

Xerox. Fortune 5U0 Top 20 include: General Motors, Ford Motor, Exxon, IBM, General Electric, Mobil, Philip

Morris, Chrysler, DuPont de Nemours, Texa60, Chevron, Amoco, Shell Oil, Proctor & Gamble, Boeing,

Occidental Petroleum, United Technologies, Eastman Kodak, USX, and Dow Chemical.

SOURCE: Henry Small, “Bibliometrics of Basic Research,” OTA contractor report, July 1990.
Chapter 8—Data on the Federal Research System . 247

Photo credit: U.S. Department of Energy

Researcher holds a piece of superconducting tape.

Scientists must be able to make ceramic superconductors

in a variety of forms to be useful—from thin films for

electronics to casts for accelerator cavities. The

development of ceramic superconductors has been an

outcome of superconductivity research.

senting research directly related to high-temperature

superconductivity. (Similar analyses were conducted in four other research areas. 43) Roughly

one-half of the most cited papers in 1985 to 1987 (77

of 139 papers) were coded for funding information

and a random sample of the papers that cited them in

1989 were included (95 of 1561). More than one-half

of the funding acknowledgments were to Federal

funding agencies (with over one-half to NSF,

slightly under one-third to the Department of Energy

(DOE), and significant contributions from the Office

of Naval Research and the National Aeronautics and

Space Administration). Corporations, primarily

IBM and AT&T, funded another one-third of the

papers; Federal laboratories, private foundations,

and foreign sources supplied the remainin g funds.

The Federal Government is a continuing catalyst of

high-temperature superconductivity research.

The degree to which a field is international in

effort can also be indicated through the nationality

of authors. Again, for the high-temperature superconductivity cluster, the most cited papers in 1985 to

1987 were from the United States (64 percent),

followed by France (8 percent), Japan (8 percent),

Switzerland (4 percent), Canada (4 percent), and the

United Kingdom (3 percent). The institutions in

which these papers most often originated were

AT&T (12 percent), IBM (12 percent), University of

Houston (5 percent), University of Tokyo (4 percent), Bell Communications (4 percent), and the

University of California-San Diego (3 percent).

Countries citing the papers were more diverse, with

the United States at 43 percent; Japan, France, and

the Federal Republic of Germany at 5 to 6 percent

each; the U.S.S.R. and India at near 5 percent; and

the Peoples Republic of China at 4 percent. Similar

diversity is seen in the institutions where these

papers originated.

With these types of analyses, bibliometrics could

perhaps be used to track the evolution of fields and

subfields-by research topic and national or institutional authorship. However, there are significant

disadvantages to bibliometrics, which also must be

recognized. 44 In particular, citations are not made in

a uniform way in the scientific community, and

neither is allocation of authorship. Also, the same

discovery may be cited in different ways in different

publications. Consequently, only in the aggregate

and when comparing similar fields with similar

citation practices can judgments of hot fields,

influential papers, and prolific authors be made with

confidence. 45 The utility of bibliometrics should be

seen as ‘‘value-added’ to policy analysis, not as

stand-alone information.

Other Measurement Techniques

Another genre of outcome measures focuses on

the research-technology interface. There are many

examples of data that could be collected to illumiate the relationship of research to other parts of the

development cycle. Complicating features, however,

include technological choice within private or public

firms that develop technology, utilization of science

and engineering talent, and the transfer of knowl-

 Op. footnote 

 Eugene for Scientific Information (which pioneered citation databases 

misuse citation analysis the of my life. ’ Quoted in Gina “Who’s No. 1 in Science? Footnotes Say U.S., ” The New York

Times, Feb. 12, 1991, pp. Cl, C9.

45 See Susan E. ‘‘Literature-Based Data in Research Evaluation: A Manager’s Guide to report to the National Science

 Sept. 18, 1989.
248 . Federally Funded Research: Decisions for a Decade

edge from research centers to other sectors of the

economy.

%

Sponsors, at least for basic research, have little

control over the execution of the projects they

support. So lack of payoff may be unrelated to the

intrinsic merits of project design and substance and

have more to do with the differential competence

and efficiency of performers. But no sponsor can

ascertain the most competent, creative, and efficient

of performers .47

In the case of public programs with firm measures

of outcomes, negative evaluations suggest termination. But for programs whose output is information,

the situation is highly problematic. Information

volume and quality might be low, but this may be

because the overall level of resources is too low. Or

a program may have technical inefficiency due to

poor management. Or the lack of results may itself

have high scientific or technological value. Since

research deposits knowledge into the scientific

literature, it may take years to be applied to other

problems. Some ideas are premature, and others

remain invisible to specialists in fields different

from the authors’ own. Recognition of the utility of

research-both intended and unintended-is often

delayed.

48 This does not depreciate its value, but

does impede its use.

Historically, science and technology are full of

sudden reversals about the value of information

produced by past research. Testing hundreds of

compounds for superconductivity was, until recently, not considered high-grade science, but mundane science. And any evaluation of this work would

have suggested that this kind of research was not

worth much investment. Similarly, the funding of

the early recombinant DNA projects was not done in

the name of expected high payoffs. Certainly no one

at the time imagined a biotechnology industry as the

result. Thus, the ability of research evaluations to

provide credible estimates of the incremental information gains from additional funding is weak.

Federal agencies tend to use an insurance principle

and spread resources widely to ensure that no

reasonable bets are overlooked. (From one perspective, this is risk-averse; from another it is risk-taking,

because ideas from out of the mainstream can be

supported.)

Bibliometrics and production function data on the

research-technology interface are examples of tools

that could be used to evaluate outcomes. While not

exhaustive, they illumin ate different aspects of

science as a process and the utility of research

performance. 49 As with the examples discussed

above, data collection can be improved when the

user of the data and the purpose are targeted. The

next section explores how data on the Federal

research system is employed by policymakers and

how new data could aid the transition from analysis

to decisionmaking.

Utilizing Data

In a policy context, information must be presented

to those who are in positions to effect change by

allocating or redirecting resources. 50 In the diverse

structure of the Federal research system, many actors

play roles in research decisionmaking at many

different levels. These actors require data reported in

various forms and units to make decisions. For

example, an agency program manager requires data

specific to the purview of his or her programs, while

OMB and OSTP must be aware of trends in science

46Given be lwge, but Cheap ficr~m ~ computing power, VfiOUS models are COmmOdy used by management analysts for deciding on ex ante

investments, but these techniques remainvery sensitive to subjective and highly uncertain estimates of technical and market success. One notable

exception is Edwin Mansfield, “The Social Rate of Return From Academic Research” Research Policy, forthcoming 1991.

&’R=entadvm~esfimethods of me~~gre~ to basic res~ch~ve cente~on sophisticated econometric twhni(pesft.)r e.Sba@p?VdZfC~~O?l

jimctions (e.g., measures of the economic impact of research). Since the marginal value of research is heavily dependent on downstream events,

production functions could be embedded in fuller models of information flow and economic behavior. In additio& literally hundreds of quantitative

project selection methods exist in indushy for guiding investments. Methods include elaborate goal programrning and analytical hierarchy models; the

techniques are often known as return-on-investment, impact matrices, or checklists. See Hrmey AvercL “Policy Uses of ‘Evaluation of Research’

Literature,” OTA contractor repo~ August 1990. Available through the National ‘lkchnical Information Service, see app. F.

~See G~ther S. Stent, ‘‘pre~~ty and Uniqueness k Scientific Discovew, “ Scientific American, vol. 227, December 1972, pp. 84-93; and Julius

H. Comroe, “The Road From Research to New Diagnosis and Therapy, “ Science, vol. 200, May 26, 1978, pp. 931-937.

49’f’he application of bibhometrics t. Patenfig be~vior, i.e., m~~ the dependence of patents on the scient~lc literature, h8S pioneered new WayS

of thinking about the diffusion and application of research knowledge. See Francis Narin et al., “Patents as Indicators of Corporate lkchnological

Strenga” Research Policy, vol. 16, 1987, pp. 143-155; and Zvi Griliches, ‘‘Patent Statistics as Economic Indicators: A Survey,’ Journal of Economic

Literature, vol. 28, No. 4, December 1990, pp. 1661-1707.

~For ex~ple, see Carol H. WeiSS, “Improving the Linkage Between Social Reseamh and Public Policy,’ Knowledge and Policy: The Uncertain

Connection, L.E. Lynn (cd.) (Washington DC: National Academy of Sciences, 1978), pp. 23-81.
Chapter 8—Data on the Federal Research System . 249

that span broad fields, institutions, and agencies, as

well as those that apply only to specific fields,

performers, and sponsors. Timely data are similarly

important. For instance, world events can alter the

perception and utility of even the best information

and analysis (see box 8-B).

Providing data at each of these levels is a large

task, but one that is essential. As seen with projected

shortages of scientific and engineering personnel,

trends are often specific to disciplines and to types

of institutions, and decisions that take into account

these differences would best address impending

problems. Enhanced internal agency data collection

would help to disaggregate and distinguish trends

most relevant to the agency.

As well as targeting data collection to the needs of

decisionmakers, the data above must address policyrelevant questions, i.e., be used evaluatively, as well

as illuminate significant trends. 51 Thus, there has

developed a distinction between standard data collection (i.e., tabulations on one variable, such as

Ph.D.s awarded) and the development of indicators-data presented in such a way (e.g., comparisons between variables) as to suggest patterns not

otherwise discernible. For instance, data on the

rising cost of equipment in a specific field (or the

rate of change in this cost) have little meaning unless

compared with the cost (and percent change) of

equipment in other fields. A measure of the relative

cost of equipment in different fields would indicate

the need to make special provisions for equipment in

select fields. Similarly, data on the decline of

baccalaureate degrees in a natural science field are

more useful when they are compared to other broad

fields, and judged in terms of absolute and relative

declines and stability.

Indicators do not necessarily prescribe a course of

action, but they warn of possibly significant trends.

As part of the decision process, they offer “usable

knowledge.”

52 At present, indicators on the Federal

research system are neither comprehensive nor

objective-driven. 53 The focus of the Science &

Engineering Indicators volumes has been less on

indicators than on data. Indeed, SEI is a statistical

reference book that collates available data on the

research system. Additional efforts to produce

indicators, especially on research performers, could

greatly enhance utilization and action by decisionmakers.

New Indicators

NSF, specifically the Special Data Group attached

to the Director’s Office, has recently attempted to

develop new indicators related to research participation at NSF during the 1980 to 1989 decade.54 These

indicators are defined and summarized in table 8-5.

Though their meaning is not always straightforward ,

55 these indicators represent a significant

advance in reconstructing trends in NSF proposal

and award activity.

The first indicator in table 8-5, the Proposal Success

Rate, is driven by the change in the number of

proposals submitted. At NSF, this number increased

by 30 percent during the decade. Over 20 percent of

those originally declined resubmit proposals to NSF

(with an equal proportion submitting elsewhere).

While the Proposal Success Rate declined from 38

percent in the beginning of the decade to 31 percent

in 1989, PI (Principal Investigator) Success Rate

from 1980-82 to 1987-89 remained above 40 percent. The PI Success Rate indicator allowed NSF to

conclude that more PIs are being funded, but they

facer stiffer competition to win awards. However,

the relation between these two Success Rate indicasl~e methodologic~ pitfws in applying data to evaluate national or hlstitutiorld research perfo rmance are illustrated in John Irvine et al., “Investing

in the Future: How Much Governments Pay for Academic Researc&” Phym”cs To&zy, Septernk 1990, pp. 31-38; and Jeremy Cherfas, “University

Restructuring Based on False Premise?” Science, VO1. 247, Jan. 19, 1990, p. 278. For fiuther discussio~ see David C. Hoaglin et al., DataforDecisions:

Information Strategies for Policymakers (Cambridge, MA: Abt Books, 1982).

Szcharles E. Lindblom and David K. Cohe% Usable Know/edge: Social Science andSocial Problem Solw”ng (New Haven, CT: Yale UnivmSitypreSL

1979).

sscoz~ns, op. cit., footnote 4, pp. 15-17.

~’rhe spci~ Dab OrOUp is part of the Comptroum’s office at the National Science Foundation. It works independently Of IWO Other Stisin the

Scientit3c, ‘Ikdnological, and International Affairs Directorate that also develop science indicators+e Science Resources Studies Division (home of

the Science & Engineering Indicators volumes) and the Policy Research and Analysis Division (which in 1990 issued the data-lade~ The State of

Academic Science and Engineering).

Ss’rhese tie bas~ on National Science Foundation, “NSF Vital Signs: Trends in Research Support, Fiscal Years 1980-89,” draft repo~ Nov. 13,

1990.
250 . Federally Funded Research: Decisions for a Decade

Box 8-B-War as a Wild Card: The Impact of the Persian

Gulf Conflict on Science and Technology

After World War II came Science--The Endless Frontier. 1 A nation grateful for its success, and newly aware

of its responsibilities in the world, decided that science was an important part of that world, and that science would

benefit from government funding. Despite the negative impact of Hiroshima and Nagasaki, science came out of the

war with a positive image, a great deal of momentum, and a strong basis for Federal support.

Vietnam, an unpopular war with unclear objectives, came with defoliation, napalm, Agent Orange, and

accusations of env ironmental degradation. Science and technology, were cast in a negative light of the atomic bomb

drops on Japan and tarred with the brush of destruction. Antiscience became part of the antiwar movement, and the

remnants of this antiscience sentiment are still with us today.

The United States has just waged the most technological war the world has yet known.2For exarnple, after years

of controversy and failed test results, the Patriot missile served a cogent strategic and political purpose. Even with

its outdated technology, the Patriot strengthened the claims of some that el ectronic warfare has come into its own.

What such success may mean for the future image of science and technology is as yet unknown.

War is a wild card. Its effects on the populace at large (and on potential science and engineering students in

particular) are difficult to predict. War is a reminder that events outside of science can reverberate in many

ways--changing images and attitudes-for a long time to come. Analyses and reports on science and technology,

such as this one, can only begin to measure, much less anticipate, these impacts.

tors and inferences about PI proposal-writing behav- are supporting a more established group of researchior is unclear for decisionmaking.56

The Continuity of Support indicator shows that

nearly one in three of the PIs with NSF support in

1980 were still receiving support in 1989. The

Flexibility of Support and Continuity of Support

indicators together measure the balance between

providing stable support to (established) investigators and retaining the ability to bring new investigators into the NSF funding system. Funding of new

ers.

The Award Size/Duration indicator reflects how

NSF responded to increased demand for funding.

Early in the decade, the number of awards was held

constant but the award amounts were increased; later

more proposals were funded and median award size

did not grow. Throughout the decade, median annual

award amount represented 80 to 85 percent of the

requested amount. 57

PIs fluctuated with the decline or growth in NSF Indicators are best used to monitor trends, espeobligations. Also, directorates with higher success cially if they could be extended to other agencies as

rates (Geosciences, and Mathematics and Physical well.58 This would help to complete the picture of PI

Sciences) ranked lower in Flexibility, because they proposal-writing strategy and the distribution of

56L~d~Pmk~~, co~p~oll~~~~ offiC~, N~tio~ s~ip~~~ FOundatiO~ ~rso~ comm~catio~ January 1991, suggests that more propc)s~s Me bCiIlg

submitted and the principal investigator population is increasing, but resubmissions (of previously declined proposals) account for only 20 percent of

the growth. James McCulloug& Comptroller’s offIce, National Science Foundation personal communication, March 1991, reports that 30 percent of

the proposals received by the National Science Foundation (NSF) in any year came from researchers who had not submitted in the previous 5 years (which

NSF defines as ‘new investigators’ and another 20 percent are received from researchers who submitted only one proposal. The supply of “new blood”

and demand for funding seem hearty.

57Awad ~owts ~e negotiated. ~Cipal fivestigators inflate their requests in the expectation that they W not receive ‘Ml” ~~g. H decl~~~

their resubmissions tend to feature smaller budgets. In multiyear (e.g., 2 to 3 year) awards, which are now typical at the National Science Foundation

annual project budgets are fmexi at the outset of the award, subject only to across-the-board cuts in sucxedng years. (This contrasts with the National

Institutes of Health’s practice of annual downward negotiation in multiyear awards.) Robert P. Abel, OffIce of Budget and Control, Natiomd Science

FoundatiorL personal communication, July 1990.

sg~e Natio~ Science Foundation cautions about the i.nte~retation of indicator trends. Changes maybe due to: a) an across-the-board budgetary

upheaval, e.g., the Gramm-Rudman sequester of 1986, which reduces the capacity to fund; b) a targeted increase or decrease in appropriations to a

directorate (or more generally, any agency line item); or C) agency reorganization Or creation of programs tbat shifts proposals and awards in ways that

affect disaggregate uses of an indicator.
Chapter 8-Data on the Federal Research System . 251

Table 8-5-New Indicators of Research Activity at NSF: Fiscal Years 1980-89

Indicator Definition Comment

Proposal Success Rate

PI Success Rate

flexibility: New

PI Funding

Continuity of

Support

Award Size/

Duration

— . . . . . . . Ratio of awards to total actions (new award Measures at an aggregate level proposal activity and awards that

and decline decisions) on competitive

(peer- or merit-reviewed) proposals

Number of principal investigators (Pls) who

are successful (within a 3-year period) in

winning an award divided by total number

of investigators submitting proposals

(within the same 3 fiscal years)

Percentage share of total award dollars

going to Pls who have not had NSF

support in the previous 5 years

Percent of principal investigators receiving

support at the start of a time period who

are still receiving support at the end of

the period

Total award dollars divided by total award

years (duration)

result in National Science Foundation (NSF) commitment of new

funding.

Interpretation of the indicator is not straightforward. Assumes

estimates of growth in the research work force, rising costs of

research, change in proposal review criteria and a proliferation of

special award categories (set-asides). The indicator does require

knowledge of agency context.

Contrasts with Proposal Success Rate, which indicates NSF action

generated by proposals submitted to it.

Measures effort and success of the research population to gain NSF

support, including changes in mean number of submissions

needed to win one award.

Indicator is most revealing when compared to other indicators.

Definition of “New Pl” is only a proxy for “young investigators.”

Indicator complements “flexibility,” which measures awards to

investigators without prior awards. It can be indexed to any cohort

of grantees and calculated for prior or succeeding years.

Indicator identifies investigators with sustained support.

Change from total dollars obligated in a particular fiscal year to

amount of award over its lifetime provides a more accurate picture

of support as experienced by the PI.

Award size and duration affect the character and pace of research

activity. Reduced award size may affect number of proposals

written, while reduced duration may affect frequency of proposal

writing. Both require investigator time for research. This indicator

assumes no other, i.e., non-NSF, source of research support. It

requires caution in making inferences about time spent in proposal

writing.

SOURCE: Office of Technology Assessment, 1991, based on National Seienee Foundation, “NSF Vital Signs: Trends in Researeh Support, Fiscal Years

1980-89,” draft report, Nov. 13, 1990.

research demand by field and agency. Thus, indicators could become an important part of the prioritysetting process. Perhaps this argues for OSTP to

coordinate across agencies the development and

presentation of a prescribed set of indicators. Disaggregate to reflect disparate agency structures, such

as directorates and divisions at NSF, such indicators

could also help portray variations in fields and

research communities. Sensitivity to such disaggregations maybe most instructive for research funding

policy. As an NSF task force recently put it:

Part of the problem is a lack of understanding of

the actual size of the research community and what

fraction of a specific community should be funded.

A clear, coherent picture of community size is

essential. How many grants should be awarded and

at what budgets? Should NSF fund all fields or make

choices predicated on the investments of others?

This “snapshot” of the community should be

updated regularly in order to indicate achievement or

changes that might be necessary to minimize confusion with respect to overall NSF policy issues.59

OTA concurs. Such baseline information should

be routinely available to decisionmakers in the

1990s. Overall, sets of indicators that draw on these

data are preferable to single measures. With this in

mind, OTA (building on the new NSF indicators

reviewed above) suggests the following four sets.

They could be compiled and analyzed by all of the

research agencies or by OSTP, which would complement existing indicators constructed and reported in

SgSome of the fidicators present~ ahve were indeed used for an inhouse evaluation of how to streamline the worldoad of tie NatioM Sciace

Foundation’ sprogram staff and the external research community. Short-term recmnmendationsf ecus on simplifying the proposal preparation and review

process, including budgets; long-term recommendation include ways of balancing support modes and restructuring grant types (size and duration),

especially cross-directorate programs. See National Science Foundatio~ Reporz of the Merit Review Task Force, NSF 90-113 (Washingto~ DC: Aug.

23, 1990).

292-863 0 - 91 - 9
252 ● Federally Funded Research: Decisions for a Decade

SEI, to provide windows on various segments of the

Federal research system:

●

●

●

●

“Active research community” indicators,

which would estimate the number of researchers actively engaged in federally funded research (e.g., PIs currently supported by one or

more Federal grants plus those with a research

proposal pending at a Federal agency, and

proportion of research time that is federally

funded).

“Research expenditure” indicators to recalibrate Federal expenditures by line item of

research budgets (e.g., salaries, equipment, and

facilities) and by broad field.

Federal “proposal pressure” indicators, e.g.,

proposals submitted to the Federal Government

per investigator, ratio of Federal to (investigator’s self-reported) non-Federal proposals and

projects in force at the time of submission, and

fraction of requested project budgets actually

awarded by the funding agency.

“Production unit” indicators, e.g., the size of

the research team or other performing unit

supported in part through Federal grants (disaggregate by subfield, institution type, and

agency source).

The combination of such indicators would estimate more precisely the changing parameters of the

Federal research system.60 This information could

be invaluable to policymakers concerned about the

health of certain sectors of the system. To produce

such information, as part of ongoing agency data

collection and NSF responsibilities for collation and

presentation, extra resources would be needed. They

might come from streamlining current NSF data and

analysis activities, such as a reduction in the number

of nonmandated reports issued annually, or designating a special unit, much like the Science Indicators Unit, to expand its inhouse and extramural

“research on research. ’ If there is a premium on

timely information for research decisionmaking, it

must be declared (and funded as) a Federal priority.

The utility of data is judged by many participants

in the system: the needs of Congress are usually

agency- and budget-specific; 61 the agencies, in

contrast, worry about the performance of various

programs and their constituent research projects.

Data converge in one other underutilized source of

information-the evaluation of research projects

and programs aft

results.

Eva

:r they have (or have not) produced

uatio of Research

While data and indicators can provide valuable

information on aggregate trends in the research

system, it is much more difficult to evaluate specific

research investments in agency programs or projects

(apart from charges of fraud, incompetence, or other

gross flaws, which are investigated as part of the

congressional oversight function). The returns from

the performance of research to society are quite

diverse. They include economic, health, security,

educational, and many other benefits. Because

research is a public good, there is little incentive for

private investment (in terms of social returns). In

1986, OTA looked at ways to measure the returns

from public investments in research:

In summary, OTA finds that. . . the factors that

need to be taken into account in research planning,

budgeting, resource allocation, and evaluation are

too complex and subjective; the payoffs too diverse

and incommensurable; and the institutional barriers

“ too formidable to allow quantitative models to take

the place of mature, informed judgment.62

Five years have passed since OTA announced this

conclusion. However, demand for research evaluations has increased in all countries that make

significant investments in research. The reasons for

increased demand are the same: budgetary constraints, greater accountability to sponsors, and the

~For example, What wodd be the indications that growth in research productivity is slowing or that the stie Of a research cOmmtitY is P-Ously

large or small relative to the resources for supporting it? See Colleen Cordes, “Policy Experts Ask a Heretical Question: Has Academic Science Grown

Too Big?” The Chronicle of Higher Education, vol. 37, No. 2, Nov. 21, 1990, pp. Al, A22.

CIAS sever~ Natioti Science Foundation staff have indicated to OTA project staff (ptisoMI COmInUnimtiOm, October-December 1~), the

President’s Science Advisor draws heavily on unpublished and newly published Science & Engineen”ng Ina2”cators (SW data in preparing and presenting

the Administration’s policy proposals at congressional “posture hearings” early in the annual authorization process. Indeed, the production cycle of

SEI is geared to delivery of the volume as an input to this budget process.

CZU.S. Congress, Office of Technology AssmsmenJ Research Funding as an Investment: Can We Measure the Returns? O’rA-TM-SET-36

(Washington DC: U.S. Government Printing Office, April 1986), p. 9.
Chapter 8-Data on the Federal Research System . 253

Photo credit: U. S. Department of Agriculture

Oversized thornless blackberries, tiny strawberry plants (in

jars), and star-shaped slices of carambola (a tropical fruit

now grown in Florida) are examples of outcomes of the

Department of Agriculture’s Agricultural Research

Service programs.

desire for increased rationality in decisionmaking.63

In response, many funding agencies—here and

abroad—have formed evaluation units.64 Research

evaluators and designers of science indicators have

carried out substantial work on measuring scientific

and technological performance. There have been

refinements in existing methods for evaluating

research impacts ex post. Nevertheless, examination

of the published and unpublished literature on

research evaluation methods between 1985 and 1990

suggests that OTA’s conclusion still stands: evaluation methods are not cited as guides to research

decisionmaking by national governments.65

Since 1985, no methods have been invented that

more definitively measure the scientific or social

value of past research investments. By ‘‘definitively

measure, OTA means that evaluation outcomes,

—

whether positive or negative: 1) will be accepted

without lengthy technical and political disputes

among sponsors, clients, and constituents, and 2)

will provide unambiguous direction in resource

allocation or other kinds of decisions. While computer modeling permits greater use of ex ante (i.e.,

before the research project is attempted) project

selection methods and ex post evaluation methods,

the evidence is sparse that there is much short-term

payoff to public or private sector research administrators from making greater use of them.66

The problem, however, may reside more with

decisionmakers than the evaluation tools (and results) at their disposal. Research administrators have

little incentive to use current evaluation technologies for making decisions about awards or level of

project allocations. This lack of incentive persists

because research evaluation “. . . occurs in a political context; is inevitably seen as post hoc justification for decisions (unrelated to the content of the

evaluation); and should be anticipatory, designed to

answer specific questions raised by superiors within

the organization as well as critics from outside.”67

Below, OTA first describes evaluation practices

and processes around the world; then considers

incremental improvements in research evaluation

methods since 1985; and finally suggests that

research evaluation faces certain inherent limits.

These limits make it unlikely that however precise

the measurement of average or incremental monetary or informational returns, they may not be

embraced. Nevertheless, rather than a means of

computing returns on past public investments or

guiding prospective ones, research evaluation may

help Federal funding agencies keep ‘on their toes,’

just as environmental impact statements impel

 reported on their

evaluation efforts in the ‘‘ECE on Evaluation in the Management of R and D, Apr. 3-7, 1989, ” unpublished proceedings.

 Research Wiley& M. Gibbons L.

 Evaluation Selection of Current Practices (Paris, France: Organisation for Economic Cooperation and Development, 1987).

 to published government 

It is the basis for this section. See op. cit., footnote 47.

 and ‘‘The Use of Advanced Management Techniques in R&D,”Omega, vol. 15, January 1987, pp. 21-29. This survey,

consistent with past surveys, shows that R&D administrators prefer simple, transparent methods of project selection. Interestingly, consistently

finds in discussions with in the executive and legislative branches that the proposition“high yield from Federal investment in is

taken as axiomatic. The issue is not whether to fund, but what and how.

 Cit., footnote 18, p. 84. “Neutrality and Advocacy in Policy Policy Sciences, vol. 6, 1975, pp.

107-119.
254 ● Federally Funded Research: Decisions for a Decade

agencies to assess the effects of their programs on

the environment.68

Evaluation in Other Countries

Table 8-6 summarizes the characteristics of the

evaluation process among major scientific and

technological powers. 69 Overall, in countries with

parliamentary governments, national priority setting

in research becomes a tool both of project selection

and research evaluation.70 The United Kingdom and

France present contrasts in approaches to evaluation—the former contracting for outside analysis,

the latter incorporating analysis of research outcomes into the government’s apparatus and process

for policymaking. Smaller countries, such as The

Netherlands and Sweden, must be selective in the

areas of research they target. If a‘ ‘critical mass’ of

researchers is not available, collaboration in cooperative international projects becomes the only outlet

for research participation.71

U.S. researchers have historically operated at the

frontiers of knowledge, and other countries have

adjusted their own research ventures as scientists in

the United States and other scientifically advanced

nations uncover promising areas. “Thus it is easier

for countries off the frontier to identify what they

want to pursue. Of course, the U.S. enterprise is so

large that it uncovers more areas than smaller

countries can afford. So they have a much more

difficult choice than the United States in determining exactly what to pursue. ’72 Now that the U.S. role

as a research performer is changing in some areas so

that U.S. scientists may not always be at the

forefront, 73 the time maybe ripe to review the place

of research evaluations-especially relative to advances in other countries-in agency decisionmaking.

An Approach To Evaluating Basic

Research Projects

Because of uncertainties attached to each and

every research investment, procedures for their

evaluation can be augmented by using ex post

review by peer researchers and citation evidence

jointly. 74 One approach would apply the following

seven criteria weighted by the priority assigned to

each:

1.

2.

3.

4.

5.

6.

7.

The

value of the information produced: salience,

relevance, importance-of both positive and

negative results-to the field;

probability of use;

originality of results;

efficiency and cost;

impacts on education and human resources;

impacts on infrastructure and capability to

carry out additional research in the future; and

overall scientific merit.

overall ex post peer evaluation of particular

projects can be compared with associated bibliometric information.75 If this comparison indicates the

same quality for a project, then the sponsor can have

6SSee, for ~-pie, S. ~ylor, Making Bureauc.acie~ Think: The EnVirOnrnentolZrnpuCl strategy of Ad~”nistrative R#o~ (Stanford, CA: Stanford

University Press, 1984); E.N. Goldenberg, “The Three Faces of Evaluation“ .~ournal of Policy Analysis and Management, vol. 2, summer 1983, pp.

515-525; and R.V. Bartlett, Policy Through ImpactAssessment: InstitutionalizedAnalysis as a Policy Strategy (New Yorkj NY: Greenwood Press, 1989).

The idea is to induce deeisionmakers to incorporate research evaluation information into their planning, not to impose the information under threat of

punishment.

@The match between countries discussed in app. D of this report and those profiled in table 8-6 is not perfect. For recent comparative analyses among

some of the countries considered here, see Department of Trade and Industry, Evaluation of R~ Policymaker’s Perspective (Londo~ England: Her

Majesty’s Stationery Offke, 1988); L.L. Lederman et. al., “Research Policies and Strategies in Six Countries: A Comparative Analysis,” Science and

Public Policy, vol. 13, No. 2, April 1986, pp. 67-76; B.R. Mu-tin and J. Irvine, Research Foresight: Creating the Future (Imndo~ England: F~c~

Pinter, 1989); and A.F.J. van Raan (cd.), Handbook of Quantitative Studies of Science and Technology (Amsterdam, The Netherlands: North-Holland,

1988).

‘Averchwrites: “Most Europeau countries have ministries of science and technology that control the flow of resources for S&T. These ministries

usually construct S&T plans correlated with economic plans. They are far more able to direct research programs at universities and industrial

laboratories.” See H.A. Averc& “New Foundations for Science and lkchnology Policy Analysis,” paper presented at the Conference on The Mutual

Relevance of Science Studies and Science Policy, Blacksburg, VA, May 12, 1989.

TISee, for C-le,Jm-Frmmk~~el> ‘‘Indicators to Measure Internationalization of Science,” unpublished paper, 1989; aud Francis Narin and

Edith S. Whitlow, Measurement of Scientific Cooperation and Coauthorship in CEC-Related Areas of Science, vol. 1 (Luxembourg: Commission of

the European Communities, May 1990).

72AvercQ op. cit., fOOtXlOtC TO, P. 15”

TqsOrnc c~plcs of hot fields dominated by non-U.S. researchers are presented iu Small, op. cit., footnote 36.

Tdsee, for emle, R.N. Kostoff, ‘‘Evaluation of Proposed and Existing Accelerated Research Programs of the OffIce of Naval Research,’ ZEEE

Transactions on Engineering Management, vol. 35, November 1988, pp. 271-279.

Tssee John Irvine and Ben ~“ , Foresight in Science: Picking the Winners (hmdo% England: Frances Pinter, 1984).
Table 8-6-Characteristics of the Research Evaluation Process for Selected Countries

Types of

research Government

Government Methods of evaluation evaluated Reported utility Central evaluation units standards Reporting

United Kingdom. . . Peer review citation; publication;

rate-of-return; patents; checklists; market outcomes

FRG . . . . . . . . . . . . Economic and market indicators;

ex post peer review for basic

research projects; special committees; evaluations of disciplines; bibliometrics; patents;

market outcomes

Japan. . . . . . . . . . . Consistency with plans developed

by “foresight”; market tests for

applied commercial projects

Netherlands . . . . . . Peer review; publication-citation

indicators for basic research;

client satisfaction or utility for

applied projects; profits earned

from research contracts

Large projects;

programs;

universities;

laboratories

Large and small

projects and

programs

Projects; priority

programs

Projects; basic

research programs, universities,industrial

research

Improve policy

decisions

Improve policy

decisions

Planning

Ensure consistency with

plans; assist

with allocation

decisions

Assessment Office within Cabinet Definitions of Some public

office; Department of Trade and good practice

Industry has central unit; some

departments have Chief Scientists and Departmental Review

Committees

Federal Ministry of Service and No Some public

Technology (BMFT) has central

unit

New central science policy unit has No Some public

some evaluation responsibilities

Yes in public agencies and No All public

universities

SOURCE: Harvey Averch, “PolicyUses of ‘Evaluation of Research’ Literature,” OTAcontractor report, August 1990. Note that most of the literature on Japan discusses their R&D planning processes

and the use of “foresight” methods. There is a less open literature on ex post evaluation.
256 . Federally Funded Research: Decisions for a Decade

greater confidence that the project is, in fact, of that

quality. If the two measures are not congruent, then

the project can be subjected to more intense analysis

to explain the discrepancy .76 In addition, an agency

program can submit information on funded projects

to experts working at the research frontiers to see

whether its structure and content are judged as

significant contributions to the field. This is expensive, but has been attempted, for example, for DOE’s

Basic Energy Sciences Program.77 (For a summary,

see table 8-7.)

Information is, of course, only one component of

decisionmaking, and others may be of far more

importance. Joint, cooperative evaluation of projects

by researchers and decisionmakers-with participants inside and outside the research area being

evaluated--could clarify agency portfolios and researcher needs.78 Nevertheless, the impacts of some

internal agency research may only become known

years later.

Research evaluations can help raise difficult

questions and uncertainties, but they cannot certify

worth. There is simply no convincing way to judge

the value of different kinds of research. However,

until new techniques, which capture the research

process as well as its products, are routinely used,

research evaluations cart best be employed to alert

agencies to potential successes and problems, and to

keep their programs vigilant in research decisionmaking (see table 8-8). Finally, these measurement

techniques should be viewed only as one input to

agency decisionmaking, because nothing can replace the experienced judgment of program managers and the scientific community to craft a successful

research program.

Conclusions

There is a wealth of data on the Federal research

system. However, data are most concentrated on

Federal R&D funding in universities, degrees

Table 8-7—Assessment of the Department of Energy’s

(DOE) Basic Energy Sciences Program: 1982

Objective Assess the quality of research and performers

Evaluation

questions

Methods and

data

Recognized

constraints

outcome

Costs and

duration

Estimate the impact of the research on DOE

mission

Determine program balance

Test appropriateness of DOE support

Specific scientific problem

Research design

Findings (past, current, expected)

Impact on DOE missions

Ex post peer review

Site visits

Publication and citation counts

Matching peer review (160 reviewers) and

bibliometric data

Stratified random sample of 125 projects (10

percent of total portfolio worth $250 million)

Reviewer variability (no random assignment of

reviewers)

Sample size too small

60 percent of projects high overall quality

10 percent of projects exceptional

10 percent low quality

$700,000-$800,000

Months to complete

SOURCE: U.S. Department of Energy, Office of Energy Researeh, Office

of Program Analysis, An Assessment of the Basic Energy

Sciences Program, DO13ER-0123 (Washington, DC: 1982).

awarded in science, the science and engineering

work force (especially the Ph.D. component), and

some expenditure data by performers. Furthermore,

the most detailed analyses are done almost exclusively at NSF and NIH, and not at the other major

research agencies. The highest priority in data

collection for research policymaking in the 1990s is

comparable data from all of the agencies, to help

Congress maintain a well-rounded view of federally

supported research (for a summary, see table 8-9).

This chapter has outlined specific areas in which

useful data could be compiled. Specific examples of

data on research expenditures, personnel, the research process, and the outcomes of research were

detailed. The second priority are data presented in

forms that are instructive at disaggregated levels of

76Somepmject~ ~twere on~ly fided ~o~d~vebeenmjmtedin ~dsight.L&eWise, someproj~ts ~twereori~IyrejMted probably cotid

have delivered reasonable quality. The only way to estimate the quality of projects an agency rejeets for support is to trace its history (which requires

the cooperation of agencies and investigators). By examining samples of rejected projects funded by others, some notion of the imperfections of a

selection process that led to unwarranted rejection may be obtained. Similarly, how does an agency, or the relevant program within it, determine that

a funded project did not meet its stated objectives?

7’7For other e~ples, see National Academy of Sciences, op. cit., footnote 8, app. C.

Tssee J+ Jeffrey Fr~~ “selectivity in Funding: Evaluation of Researeh in Australia, ” Prometheus, vol. 6, June 1988, pp. 34-60. The evaluation

of an agency program would require far more than information on the projects it supports. Rather, questions of implementation-effectiveness and

efllciency of decisions, and of the program personnel who make them-would dominate. In sho~ project evaluations aggregated to the program level

would estimate the caliber of researcher performance more than success in administering the program. See Eleanor Chelimsky, ‘‘Expanding GAO’s

Capabilities in Program EvaluatiorL’ The GAO Journal, winter/spring 1990, pp. 43-52.
Chapter 8—Data on the Federal Research System . 257

Table 8-8—Dimensions of Agency Research

Evaluations

Purpose/research For example, to fund or not to fund, comparaquestion tive project performance, extent of contribution to program missions/goals.

Definitions/criteria For example, quality, priority, cost-effectiveness, innovativeness, success, accountability, impact, productivity, knowledge, growth.

Units of analysis For example, institute, division, branch/

center, program, project, individual, team,

publications, citations, awards, rates of

change, adoption/diffusion.

Outcomes Process v. product, form of research v.

content and outputs, cost per-outcome unit,

qualitative v. quantitative.

Time horizon Duration of award, short-v. long-term contribution, continuity/culmination v. new direction.

User audience Well defined v. fuzzy, disciplinary. multidisciplinary, knowledge- v. problem-oriented.

SOURCE: D.E. Chubin, “Designing Researeh Program Evaluations: A

Science Studies Approach,” Suenceand Public Policy, vol. 14,

No. 2, April 1987, p. 85.

decisionmaking. In particular, data could be presented to make suitable comparisons and to gauge

relative trends (i.e., as indicators of science and

technology activity) .79 New indicators, grounded in

the tradition of the SEI volumes and extramural

research on research, are needed to monitor changes

in the Federal research system.80

Finally, evaluation techniques of research investments in specific programs and projects were

revisited. OTA finds that research evaluation techniques cannot replace mature judgment by policymakers. However, specific evaluation tools, such

as bibliometrics and project portfolio analysis, could

be further explored. A third priority is ongoing

project evaluation, which could keep agencies alert

to changes in research performance, augment program manager judgments about performers and

projects, and serve to improve overall program

effectiveness.

In summary, one of the functions of analysis is to

raise questions about the information that decisionmakers are currently using to assess their advantages

and disadvantages, and to define a richer menu of

options. Much information could be collected on the

Federal research system to map trends at different

levels of aggregation and units of analysis for

different users.81

However, the existence of data does not ensure its

utility, for many policy issues cannot be addressed

by additional descriptive information. In particular,

external criteria involving the utility of research or

impact on objectives can be more persuasive and

salient to specific policy decisions. Depending on

one’s perspective and scope of responsibility, data on

budgets, agencies, initiatives, performers, and outcomes can nevertheless clarify understanding of the

evolving research system.82

This information, however, is not cost-free; nor is

the organization for retrieving and distilling it.

Congress could consider expanding agency resources to streamline collection and analysis of

baseline data. NSF, working in concert with OSTP

and OMB, could coordinate and reinforce this

national data function, and organizations outside of

7~or ~mple, tie En@e.fig ~poww co~55ion tit ~sembles, @yzes, ~d diss~at~ engin~ring ~bent and degree figures for

the American Association of Engineering Societies recently remarked: ‘‘There is an old gag among survey researchers that when faced with a choice

between consistency and the truth, one should always opt for consistency. When the product of the research is time series data that readers may tmck

for years, the virtues of consistency become especially obvious. ” See “Consistency Versus Relevance: EMC Changes a Statistic,” Engineering

ManpowerBulletin, June 1990, p. 1. In other words, supplementing a time series with new measures without destroying the continuity of the series is

also a virtue.

80@ti~tive datawill not s~ice. ~omtion on the contexts in which resea.rch is performed, and characteristics of tie p~ormers ~divid~lyand

collectively, will provide clues as to how the numbers can be interpreted and perhaps acted on, For example, see Daniel T. Layzell, “Most Research

on Higher Education Is Stale, Irrelevant, and of Little Use to Policymakers, ’ The Chronicle of Higher Educatiorz, vol. 37, No. 8, Oct. 24, 1990, pp. Bl,

B3.

sl~ee dec~es after historia Derek de Solla Price called for a full-blown ‘‘science Of SCienCe, ”the policy potential of “reseamh on research” as

illustrated in this chapter, has only begun to be exploited. Price’s vision is introduced in Science Since Babylon (New HaveU CT: Yale University Press,

1961) and Little Science, Big Science, op. cit., footnote 21, and elaborated in a series of analyses terminated by his death in 1984. For a retrospective,

see Susan E. Cozzens, “DerekPrice and the Pamdigm of Science Policy,’ Science, Technology, & Human Values, vol. 13, Nos. 3 and 4, summer-autumn

1988, pp. 361-372.

82~s leads (JT’ t. sugge5t tit he research agencies, especi~y me Natio~ science Fo~&tion ~(f its poficy progrms, IWIMlh h CIOSe touch

with analysts of the Federal research system. Keeping abreast of new measurement techniques and findings related to people, funding, and research

activities-perhaps through extramural support-would be a modest but fruitful investment in extending inhouse capabilities and refining knowledge

of federally sponsored research performance.
258 . Federally Funded Research: Decisions for a Decade

Table 8-9-Desired Data and Indicators on the Federal Research System

Primary users

Category Description Method Congress Agencies OMB OSTP

Agency funding

allocation method

Research expenditures

Research work force

Research process

Outcome measures

Indicators

Funding within and across fields and

agencies

Cross-agency information on proposal

submissions and awards, research costs,

and the size and distribution of the

research work force supported

Research expenditures in academia, and

Federal and industrial laboratories,

centers, and university-industry

collaborations

Agency allocations of costs within research

project budgets, by field

Megaproject expenditures: their

components, evolution over time, and

construction and operating costs

Size and how much is federally funded

Size and composition of research groups

Time commitments of researchers

Patterns of communication among

researchers

Equipment needs across fields (including the

fate of old equipment)

Requirements for new hires in research

positions

Citation impacts for institutions and sets of

institutions

International collaborations in research

areas

Research-technology interface, e.g.,

university/ industry collaboration

New production functions and quantitative

project selection measures

Comparison between earmarked and meritreviewed project outcomes

Evaluation of research projects/programs

Proposal success rate, PI success rate,

proposal pressure rates, flexibility and

continuity of support rates, project award

and duration rate, active research

community and production unit indices

Agency data x x x

collection (and

FCCSET)

Agency data

collection

x

Lead agency survey X

Lead agency

survey; onsite

studies

Bibliometrics;

surveys of

industry and

academia

x

Agency analysis x

x x

x

x

x

x

x

x

x

KEY: OMB=Office of Management and Budget; OSTP-Office of Science and Technology Policy; FCCSET.Federal Coordinating Council for Science,

Engineering, and Technology; Pl=principal investigator.

SOURCE: Office of Technology Assessment, 1991.

the government, e.g., NRC and AAAS, could also and problems, and pinpoint previously uncovered

play critical roles. Refining the measurement pro- ones, greatly enhancing research decisionmaking at

cess could help to quantify existing opportunities all levels of the Federal Government.
APPENDIXES
APPENDIX A

Major Legislation Enacted Since 1975 Affecting

U.S. Research and Development

Number and date Title Important aspects

Public Law 94-282

May 11, 1976

National Science and

Technology Policy and

Organization Act

Called for the development of a national science and technology policy, and a

national science and technology base. Created the Office of Science and

Technology Policy (OSTP) in the Executive Office of the President (EOP)

in order to advise the President on science and technology, including

budget issues, and to assess the Federal effort in science and technology.

Public Law 95-91

Aug. 4, 1977

Department of Energy

Organization Act

Created the Department of Energy, transferring all the duties of the Energy

Research and Development Administration to the Department of Energy.

Designed to coordinate climate research among the various research

agencies, this act called for a heightened effort in climate research and

defined the roles of the different agencies who do the research.

Required the President to formulate a national materials policy and submit a

plan to Congress, addressing coordination in the executive branch and

assessment of the economic, industrial, and national security needs

regarding materials policy.

Created to promote technological innovation, this act established an Office of

Industrial Technology in the Department of Commerce, and it mandated

technology transfer from the Federal laboratories to the private sector.

Established various tax breaks for research and development (R&D)

expenditures, including a deduction for charitable contributions of R&D

equipment to universities.

Public Law 95-367

Sept. 17, 1978

National Climate Program Act

Public Law 96-479

Oct. 21, 1980

Materials Policy Research and

Development Act of 1980

Public Law 96-480

Oct. 21, 1980

Stevenson-Wydler

Technology innovation Act

of 1979

Public Law 97-34

Aug. 13, 1981

Economic Recovery Tax Act

of 1981

Public Law 97-219

July 22, 1982

Small Business Innovation

Research Act

Aimed at strengthening the role of small firms in the performance of federally

funded R&D, this act required all agencies with large extramural R&D

budgets to set aside 5 percent of their budget (over 4 years) for the Small

Business Innovation Research (SBIR) program.

Public Law 98-373

June 31, 1984

National Materials and

Minerals Policy, Research

and Development Act

Created the National Critical Materials Council in EOP to coordinate Federal

materials R&D programs.

Public Law 98-462

Oct. 11, 1984

National Cooperative

Research Act of 1984

In order to stimulate industrial R&D, this act promotes more joint ventures on

research projects as it limits the effect of the antitrust laws in such cases. It

also reimburses companies for legal costs associated with frivolous antitrust

suits brought against them.

Public Law 99-502

Oct. 20, 1986

Federal Technology Transfer

Act of 1986

Amended the Stevenson-Wydler Act to allow government-operated Federal

laboratories to enter into cooperative R&D agreements, and established the

Federal Laboratories Consortium for Technology Transfer.

Included the Training Technology Transfer Act and the Technology

Competitiveness Act, as well as measures to support semiconductor R&D

and to protect intellectual property rights.

Mandated a 5-year National Action Plan on Superconductivity R&D by OSTP,

as well as an annual report updating Congress on the implementation of the

plan.

Part of a Department of Defense authorization bill, this act amended the

Stevenson-Wydler Act to allow government-owned, contractor-operated

laboratories to enter into cooperative R&D agreements.

Public Law 100-418

Aug. 23, 1988

Omnibus Trade and

Competitiveness Act

Public Law 100-697

NOV. 19, 1988

Superconductivity and

Competitiveness Act

Public Law 101-189

NOV. 29, 1989

National Competitiveness

Technology Transfer Act of

1989

–261-
262 . Federally Funded Research: Decisions for a Decade

Major Legislation Enacted Since 1975 Affecting U.S. Research and Development—Continued

Number and date Title Important aspects

Public Law 101-239

Dec. 19, 1989

Public Law 101-508

Nov. 5, 1990

Public Law 101-589

NOV. 16, 1990

Public Law 101-606

NOV. 16, 1990

Omnibus Budget Reconciliation Act of 1989

Omnibus Budget Reconciliation Act of 1990

Excellence in Mathematics,

Science and Engineering

Education Act of 1990

National Global Change

Research Act of 1990

Extended the R&D tax credit for another 9 months.

Extended the R&D tax credit for 1 more year.

Aimed at improving mathematics, science, and engineering skills, this

comprehensive act authorized various programs focusing on elementary,

secondary, and higher education, including programs promoting the use of

technology in education.

Amended the National Science and Technology Policy, Organization and

Priorities Act of 1976 to provide for a national plan to improve scientific

understanding of the Earth system and the effect of changes in that system

on climate and human well being.

SOURCES: U.S. Congress, House Committee on Science and Technology, A History of Science Policy in the United States, 1940-1985, prepared for the

Task Force on Science Policy (Washington, DC: U.S. Government Printing Office, 1986); Congressional Research Service, Science Policy

Research Division, Statutory Provisions Related to Federal /?esearch and Development, prepared for the House Committee on Science and

Technology, Subcommittee on Domestic and International Scientific Planning and Analysis (Washington, DC: U.S. Government Printing

Office, 1976); and Office of Technology Assessment, 1991.
The Top

Amount of Federal R&D Funding

100 Institutions

Received: Fiscal

APPENDIX B

Ranked by

Year 1989

Dollars

Rank Institution (in thousands)

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

34.

35.

36.

37.

38.

39.

40.

41.

42.

43.

44.

45.

46.

47.

Stanford University, CA.. . . . . . . . . . . . . . . . . . . . . . . . . . . 238,650

Massachusetts Institute of Technology . ..............215,140

University of Washington . . . . . . . . . . . . . . . . . . . . . . . . . . 182,453

University of Michigan. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174,875

University of California San Diego . . . . . . . . . . . . . . . . . . . 171,479

University of Wisconsin, Madison . . . . . . . . . . . . . . . . . . . . 169,452

The Johns Hopkins University, MD . . . . . . . . . . . . . . . . . . . 168,184

University of California, San Francisco. . . . . . . . . . . . . . . . 159,906

University of California, Los Angeles . . . . . . . . . . . . . . . . . 159,002

Cornell University, NY. . . . . . . . . . . . . . . . . . . . . . . . . . . . .157,984

Total, top 10 institutions . . . . . . . . . . . . . . . . . . . . . 1,797,125

Columbia University, NY. . . . . . . . . . . . . . . . . . . . . . . . . . . 146,712

Harvard University, MA. . . . . . . . . . . . . . . . . . . . . . . . . . . . 143,451

Yale University, CT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138,835

University of Minnesota . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132,880

University of California Berkeley. . . . . . . . . . . . . . . . . . . . 124,371

University of Pennsylvania. . . . . . . . . . . . . . . . . . . . . . . . . . 123,810

University of Southern California . . . . . . . . . . . . . . . . . . . . 119,005

Pennsylvania State University. . . . . . . . . . . . . . . . . . . . . . . 114,646

University of Illinois, Urbana. . . . . . . . . . . . . . . . . . . . . . . . 114,398

University of Colorado . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109,145

Total, top 20 institutions . . . . . . . . . . . . . . . . . . . . . 3,064,378

University of Rochester, NY.. . . . . . . . ... .............101,049

Duke University, NC.... . . . . . . . . . . . . . . . . . . . . . . . . . . . 99,036

Georgia Institute of Technology . . . . . . . . . . . . . . . . . . . . . 98,048

Washington University, MO . . . . . . . . . . . . . . . . . . . . . . . . 96,829

University of Texas, Austin . . . . . . . . . . . . . . . . . . . . . . . . . 94,311

Texas A & M University. . . . . . . . . . . . . . . . . . . . . . . . . . . . 93,584

University of North Carolina, Chapel Hill. . . . . . . . . . . . . . 93,280

University of Chicago, IL. . . . . . . . . . . . . . . . . . . . . . . . . . . 90,459

California Institute of Technology. . . . . . . . . . . . . . . . . . . . 84,167

University of Pittsburgh, PA... . . . . . . . . . . . . . . . . . . . . . . 81,217

Total, top 30 institutions . . . . . . . . . . . . . . . . . . . . . 3,996,358

New York University . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81,143

University of Arizona. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80,533

Ohio State University . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75,484

University of Iowa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74,217

University of California, Davis . . . . . . . . . . . . . . . . . . . . . . . 72,718

Baylor College of Medicine, TX . . . . . . . . . . . . . . . . . . . . . . 69,336

Case Western Reserve University, OH. . . . . . . . . . . . . . . . 68,632

University of Alabama, Birmingham. . . . . . . . . . . . . . . . . . 68,204

Carnegie-Mellon University, PA... . . . . . . . . . . . . . . . . . . . 65,079

SUNY at Buffalo, NY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64,453

Total, top 40 institutions . . . . . . . . . . . . . . . . . . . . . 4,716,211

Woods Hole Oceanographic Institute, MA. . . . . . . . . . . . . 64,333

Purdue University, IN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63,979

University of Miami, FL. . . . . . . . . . . . . . . . . . . . . . . . . . . . 63,101

University of Utah . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61,819

University of Florida . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60,731

University of Maryland, College Park . . . . . . . . . . . . . . . . . 58,924

Indiana University. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58,334

Continued on next page

–263–
264 ● Federally Funded Research: Decisions for a Decade

Dollars

Rank Institution (in thousands)

48.

49.

50.

51.

52.

53.

54.

55.

56.

57.

58.

59.

60.

61.

62.

63.

64.

65.

66.

67.

68.

69.

70.

71.

72.

73.

74.

75.

76.

77.

78.

79.

80.

81.

82.

83.

84.

85.

86.

87.

88.

89.

90.

91.

92.

93.

94.

Yeshiva University, NY . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58,224

University of Tennessee System . . . . . . . . . . . . . . . . . . . . 57,763

Northwestern University, IL.. . . . . . . . . . . . . . . . . . . . . . . . 57,510

Total, top 50 institutions . . . . . . . . . . . . . . . . . . . . . 5,320,929

University of Massachusetts . . . . . . . . . . . . . . . . . . . . . . . . 56,505

Boston University, MA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56,402

Vanderbilt University, TN. . . . . . . . . . . . . . . . . . . . . . . . . . . 56,151

Michigan State University. . . . . . . . . . . . . . . . . . . . . . . . . . . 51,741

University of Texas SW Medical Center, Dallas . . . . . . . . 51,254

University of Virginia. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51,214

SUNY at Stony Brook, NY . . . . . . . . . . . . . . . . . . . . . . . . . 49,726

Oregon State University. . . . . . . . . . . . . . . . . . . . . . . . . . . 49,112

Princeton University, NJ . . . . . . . . . . . . . . . . . . . . . . . . . . . 47,176

Colorado State University . . . . . . . . . . . . . . . . . . . . . . . . . . 46,572

Total, top 60 institutions . . . . . . . . . . . . . . . . . . . . . 5,836,782

Emory University, GA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46,497

University of California, Irvine. . . . . . . . . . . . . . . . . . . . . . . 46,492

University of Connecticut . . . . . . . . . . . . . . . . . . . . . . . . . . 46,184

New Mexico State University. . . . . . . . . . . . . . . . . . . . . . . . 45,660

University of Illinois, Chicago . . . . . . . . . . . . . . . . . . . . . . . . 43,288

University of Georgia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42,797

Utah State University. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42,449

Rockefeller University, NY. . . . . . . . . . . . . . . . . . . . . . . . . . 41,192

Tufts University, MA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40,771

University of Cincinnati, OH. . . . . . . . . . . . . . . . . . . . . . . . . 40,598

Total, top 70 institutions . . . . . . . . . . . . . . . . . . . . . 6,272,710

University of Hawaii, Manoa. . . . . . . . . . . . . . . . . . . . . . . . 40,574

Louisiana State University . . . . . . . . . . . . . . . . . . . . . . . . . 40,114

University of California, Santa Barbara. . . . . . . . . . . . . . . 39,227

Virginia Polytechnic lnstitute and State University. . . . . . . 38,597

North Carolina State University, Raleigh . . . . . . . . . . . . . . 37,783

Georgetown University, DC . . . . . . . . . . . . . . . . . . . . . . . . 37,351

CUNY Mt. Sinai School of Medicine, NY. . . . . . . . . . . . . . 37,233

University of Maryland, Baltimore Professional

schools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35,970

Rutgers University, NJ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35,896

Brown University, RI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34,506

Total, top 80 institutions . . . . . . . . . . . . . . . . . . . . . 6,649,961

Virginia Commonwealth University. . . . . . . . . . . . . . . . . . . 30,078

University of Texas Health Science Center,

Houston . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29,500

University of Texas Health Science Center,

San Antonio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29,324

University of Texas M.D. Anderson Cancer Center. . . . . . 28,992

lowa State University . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28,895

University of Vermont and State Agriculture College. , . . 28,535

Wake Forest University, NC . . . . . . . . . . . . . . . . . . . . . . . . . 28,511

Wayne State University, Ml . . . . . . . . . . . . . . . . . . . . . . . . . 28,167

University of Medicine and Dentistry of New Jersey.. . . . 27,983

Dartmouth College, NH . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27,222

Total, top 90 institutions . . . . . . . . . . . . . . . . . . . . . 6,937,168

University of Kentucky . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27,010

University of Alaska, Fairbanks . . . . . . . . . . . . . . . . . . . . . . 26,659

University of Dayton, OH . . . . . . . . . . . . . . . . . . . . . . . . . . 26,650

University of South Florida. . . . . . . . . . . . . . . . . . . . . . . . . . 26,576
Appendix B—The Top 100 Institutions Ranked by Amount of federal R&D Funding Received . 265

Dollars

Rank Institution (in thousands)

95. University of Kansas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26,420

96. University of Nebraska, Lincoln . . . . . . . . . . . . . . . . . . . . . 25,803

97. Temple University, PA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25,232

98. George Washington University, DC . . . . . . . . . . . . . . . . . . 25,220

99. Florida State University . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24,897

100. Oregon Health Sciences University . . . . . . . . . . . . . . . . . . 24,162

Total, top 100 institutions . . . . . . . . . . . . . . . . . . . . . 7,195,797

Total, all other sampled institutions . . . . . . . . . . . . 1,354,669

Total, all institutions . . . . . . . . . . . . . . . . . . . . . . . . . 8,550,466

NOTE: The Johns Hopkins University total does not include $422 million for the Applied Physics

Laboratory. Also, 95 of the 100 institutions in the fiscal year 1989 top 100 were also in the

fiscal year 1988 list. The concentration/dispersion of funding is identical in the 2 years.

SOURCE: National Science Foundation, Academic Science/Engineerirrg: R&D Fun&—Fiscal

Year 1989 (Washington, DC: forthcoming 1991 ), table B-35.
APPENDIX C

Funding Allocation in Six Federal Research Agencies’

National Institutes of Health

The primary review bodies for grant applications at the

National Institutes of Health (NIH) are the study sections,

of which there are over 90. Initially, all proposals go to the

Division of Research Grants (DRG), which assigns each

to a study section for an initial review and an institute for

second level review. A grantee can request, but not

designate, an institute; an institute can request that a grant

be directed its way, but DRG has the right to overrule that

request. Grant applications are classified according to

type, such as new, competing continuation (renewal), and

supplemental applications, and according to activities,

such as regular research projects, conferences, centers,

and fellowships. Last year DRG received over 30,000

applications.

Biennially each institute provides DRG with referral

guidelines. The referral guidelines for all institutes are

circulated and overlaps noted and negotiated through

memoranda of understanding. Overlaps can usually be

resolved in this manner. Most often, the issue then goes

to the institute directors involved for a final decision.

Grants in the areas of dispute can be assigned primarily

and secondarily to the participating institutes or there may

be a decision to send the grant to more than one institute

for dual finding.

Study sections meet several times a year to review

applications. Each proposal is discussed individually and

recommendations are determined by majority vote of the

members. If the application is recommended for approval,

each member votes privately, assigning a priority rating

from one for outstanding to five for acceptable. A priority

score for an application is determined by averaging the

individual ratings and multiplying by 100. To deal with

the diversity of rating behavior among study sections and

because of priority score ‘‘creep’ (a tendency for scores

to get better as reviewers realize that only the very best

scores will allow a proposal to be funded), a percentile

rank is now calculated for each score. The percentile

represents the relative position or rank of each priority

score among the scores assigned by the study section at its

last three meetings. The lower the numerical value of the

priority score or percentile, the better the application.

Funding units designate an approximate percentile ‘‘payline,” a priority score below which applications will not

be funded.

After a grant has been through review by the study

section, it enters a second level of review by the statutorily

mandated National Advisory Council or Board of the

institute. The councils and boards are comprised of

scientists and lay representatives. They consider the

percentiles assigned by the study sections and review

grants for their relevancy to the institute’s programs and

priorities. Councils can choose not to concur with a study

section approval based on program or policy considerations. However, they cannot reverse a disapproval action

when their decision is based on scientific and technical

merit only.

The award rate is the proportion of applications

recommended for approval that are actually funded. In

1989, the overall award rate at NIH was 29.4 percent. The

success rate is the proportion of reviewed applications

that are actually awarded. In 1989, the overall success rate

was 27.5 percent. In 1990, the award rate was 33 percent

for competing renewals at the National Institute of

General Medical Sciences (NIGMS), 14 percent for new

applications, and between 12 and 15 percent for first-time

applications. NIGMS budgets minority and training

programs separately, thereby removing them from the

same level of competition for limited resources (although

each program is competitive in its own right).

At NIGMS, nearly 3,000 grant proposals are received

each year. DRG study sections review the grants for

scientific merit and amount requested. They then send

their recommendations to the council for concurrence.

Most often the council will approve blocks of grants. If

rejected applicants wish to appeal they can submit

rebuttals. In this case, NIGMS staff then submit their

reply with the rebuttal to council. They can either support

the study section decision or the rebuttal. More often than

not, the council will concur with staff.

There has been a perception that the workload requirements associated with membership on study sections are

an impediment for recruiting members. A 1989 review of

study section workload showed that the average workload

had actually gone down between 1980 and 1988, in part,

because more study sections had been formed. In 1980,

there were about 70 study sections. In 1988, there were 90.

Still, the average study section member spends 45 days

each year preparing for and attending meetings. This does

not include site visits or mail reviews. The average tenure

of service is 4 years. The NIH peer review system has been

criticized for repeatedly using the same individuals on

study sections. In fact, only 13 percent of reviewers are

reappointed.

Some managers feel the payline has become inflexible

and creates too much of a focal point for micromanagel~s apptmdix is based on OZ4 interviews, spfig-s~er 1990.

–266-
Appendix C—Funding Allocations in Six Federal Research Agencies . 267

ment. For example, if a program chooses to go below the

payline and fund an exceptional grant, rejected applicants

who came in above the payline have been known to

request intervention by their representatives in Congress.

The program and the institute must then respond to

congressional inquiries and justify their decision. Some

managers feel that this creates a disincentive for program

managers to ignore the payline occasionally when considering innovative research. Some policies have been

created to allow flexibility around the payline to fund

young investigators and other groups.

However, every institute has an exception process for

funding. Generally about 10 percent of the research

budget can be used for exceptions or for applications

below the payline that are cutting edge and, in the eyes of

staff, deserve to be funded because of high program

relevance. At the National Cancer Institute, for example,

each division takes its exceptions to the director, where

they are put in priority order and then compete for institute

resources with the executive committee as final arbiter.

Department of Defense

The Department of Defense (DOD) solicits proposals

through Broad Agency Announcements, which detail the

interests of the services, the Defense Advanced Research

Projects Agency (DARPA), or the Strategic Defense

Initiative Organization (SDIO) research program. Program managers are allowed much latitude in funding

decisions, and their performance is judged by the impact

of the research program on issues of defense interest.

External peer review may be used, but only in an advisory

capacity. In general, inhouse review will suffice and

laboratory personnel are often integral to this review

process.

Army

Of the 6.1,6.2, and 6.3A budget categories, the Army

distributed 15 percent to basic research (6.1), 46 percent

to applied research (6.2), and 39 percent in the early stages

of development (6.3A). Within the 6.1 budget (fiscal year

1989), the Army laboratories received 68 percent, extramural single principal investigator grants accounted for

21 percent,2 Centers of Excellence encompassed 6

percent, and inhouse laboratory independent research

received the final 5 percent.

Within the Army, the 6.1 budget is disbursed by the

Army Research Office (ARO), Medical Commands, and

institutes such as the Army Research Institute for the

Behavioral and Social Sciences. In addition, each laboratory, institute, or center has its own 6.1 monies. Through

the tri-service University Research Initiative program, the

Army also sponsors 12 centers in 10 research areas. In

addition, the Army sponsors seven of its own Centers of

Excellence.

Army Research Office-Until 1985, ARO did not

solicit proposals directly. ARO has a tradition of supporting single investigators over long periods of time. ARO

feels that this stable funding environment produces highly

creative research, both because the investigator has more

time to devote to research and because stable funds are

sought by the scientific community, and so competition

is fierce.

Medical Research and Development Command

(MR&DC)--Medical research needs are addressed by the

nine laboratories of MR&DC and monies are allocated

between them. The largest, with 90 percent of the

technology base funds, is the Walter Reed Medical

Center. Walter Reed employs 1,100 scientists of which

600 are in the Institute for Research. About one-half are

uniformed and the other one-half are civilian.

Army Research Institute for the Behavioral and

Social Sciences—Research proposals are reviewed by the

Basic Research Office (BRO), the laboratories, and often

external reviewers. They are rated on five factors: 1)

scientific significance, 2) potential Army relevance, 3)

technical merit, 4) quality of executing personnel, and 5)

cost realism. Contracts and grants are awarded on the

basis of this inhouse, and partially external, review. Also,

an inhouse review committee will examine annually all of

the contracts and grants awarded in each program area.

Universities receive 80 to 90 percent of the available grant

and contract funds from BRO. Profitmaking corporations

receive another 10 to 15 percent and nonprofits receive the

remaining 2 to 5 percent.

Laboratories-Research laboratories operate primarily on 6.2 and 6.3 funds, but 6.1 monies makeup a small

proportion of the funding. Funds are distributed to

research groups through inhouse budgeting. Contracts are

awarded at program manager discretion after substantial

scientific review by inhouse personnel.3

Navy

Almost all Navy basic research money is disbursed by

the Office of Naval Research (ONR), although many of

the larger laboratories also have small 6.1 budgets to

support basic research. As part of the Navy’s investment

strategy for research, ONR stresses that, while spending

60 percent of their funds on ‘‘evolutionary’ research and

25 percent on research that is “closely associated with

ZNote that the University Research Initiative (URI) funds are not included in these figures since URI is now funded by the OffIW Of the secretary

of Defense.

qFor details, see U.S. Conwess, Offke of ‘lkchnology Assessment, Holding the Edge: Maintaining the Defense Technology Base, 0~-lSC-420

(Washington DC: U.S. Government Printing Office, April 1989), especially chs. 1 and 3.
268 . Federally Funded Research: Decisions for a Decade

transition to the fleet, ” 15 percent of the funds are

allocated to high-risk, but potentially high-payoff research. ONR also seeks to leverage funds from other

departments and industry to boost research in its programs

in civilian laboratory settings.

ONR divides its efforts into ‘core’ and “accelerated’

research initiatives. Core initiatives build on previous

efforts with slight modifications in levels of funding from

year to year. Two percent of the research program is set

aside for core enhancements. Each directorate will

compete annually for enhancement funds and will often

solicit external reviewers. Accelerated Research Initiatives (ARIs) provide increased levels of support over 3 to

7 years. The average funding is about $1 to $2 million per

year, and about 6 percent of the current research program

is set aside for new ARIs. Directorates also compete

annually for funding for their proposed ARIs and, as with

core enhancements, will solicit both inhouse and external

reviews. The core program represents about 70 percent of

the total Navy research program and ARIs total about 30

percent.

Air Force

Before 1974, inhouse laboratories controlled the 6.1

monies for the Air Force. However, in 1974, the Air Force

consolidated the direction of the 6.1 monies into one unit,

the Air Force Office of Scientific Research (AFOSR).

Each laboratory still has a portion of 6.1 monies, but the

bulk are distributed by AFOSR. The laboratories compete

for these funds along with universities and other performers.

New initiatives are usually begun with funds designated for new starts ($10 million in fiscal year 1990).

Eighteen months before the start of a fiscal year, the

program managers propose new initiatives. The seven

directorates then compete for the funds. Each directorate

is asked to bid for twice their ‘‘fair share” (or oneseventh) of the funds set aside. An extensive inhouse

review process determines the awarding of funds. Eight to

10 projects are awarded at about $1 million each. AFOSR

requires that at least 15 percent of the research portfolio

for a directorate changes composition every year.

AFOSR had about 1,200 grants and contracts in fiscal

year 1990 (about 900 grants and 300 contracts). Roughly

500 projects are initiated in a year, with an average

duration of 2 to 3 years. AFOSR works with close to 220

universities, which receive over one-half of AFOSR

funding. Laboratories receive 30 percent of AFOSR

funds. 4

DARPA and SDIO

Project managers are primarily responsible for the

selection of contract and grant awards, and usually

conduct inhouse reviews of proposals. DARPA does very

little contracting itself. ONR, AFOSR, ARO, or other

parts of DOD will administer the grant, often because the

service is the ‘customer’ for the project and will benefit

from its results. This close working relationship of

DARPA and SDIO with other parts of DOD facilitates the

technology transfer of the project findings.

Although program managers determine the specific

goals of a project, the development of these goals through

the letting of contracts and grants is left to the agent in

ONR, AFOSR, ARO, or some other part of DOD.

(Examples of specific goals include development of a

particular kind of focusing mirror or more efficient laser

using a particular kind of technology.) Managers use

whatever selection mechanism they normally use for

grants and contracts.

National Aeronautics and Space

Administration

Office of Space Science and Applications

The Office of Space Science and Applications has two

primary means of soliciting research/contract proposals.

Announcements of Opportunity (AOs) are solicited and

awarded over the associate administrator’s signature.

They usually call for hardware and experiments for an

upcoming flight mission, and are funded via contracts.

They represent one-of-a-kind opportunities with substantial monetary commitment. The AO is also the primary

means of selecting the team of scientists for a mission.

These scientists will not necessarily work together, but

their combined efforts will set the schedule for the

mission.

An AO will state the criteria and the procedure for

selecting successful candidates. Usually, each proposal

will be reviewed by panels of peers to judge scientific

merit. Further review by National Aeronautics and Space

Administration (NASA) staff will weigh feasibility,

management issues, and relevance to the mission. NASA

also keeps the prerogative of splitting up a proposal to

fund only part of it and of joining two or more

investigative teams together. The division will then rank

the proposals. The final decision is left to the associate

administrator, as advised by the division. An oversight

committee, chaired by the assistant associate administratrator, checks the selection criteria for adherence to proper

procedures and adequacy of documentation of the review

process. In descending order of importance, NASA

done ~aot ~ompwe dhwtly ti~ga~ocation ~tween universities and laboratories, because university tids reflect roug~y Ml costs including

investigator salaries whereas laboratory scientist salaries are funded through another budget line.
Appendix C—Funding Allocations in Six Federal Research Agencies ● 269

acknowledges the following criteria to rank proposals: 1)

scientific merit, 2) relevance to the goals of the mission,

3) adequacy of the research methods, 4) feasibility within

logistical constraints, 5) competence and experience of

the investigators, and 6) fiscal and other support by the

investigator’s institution.

Research Announcements (RAs) are released under the

signature of the associate administrator with the division

director as the selecting official. They are more modest in

scope than an AO, and more specific in focus. Taken

together, however, the RAs cover a broader range of

topics. In some divisions, RAs solicit “guest’ observers,

who will use an apparatus after the original investigator’s

share of the time is up. Theory, archival research, and

other disciplinary areas are also supported with RAs. Like

an AO, RAs state what selection criteria and which

procedure will be used to make awards. Funding is

primarily through grants. This procedure is usually very

similar to the one described for an AO.

In addition to AOs and RAs, NASA employs other

funding mechanisms. For example, unsolicited proposals

are encouraged. They are submitted to a peer review

process that is very similar to the procedure outlined

above for AOs. Also, discretionary money is available to

the division director, which represents a small portion

(often nearly 10 percent) of the research monies and is

disbursed by a less formal procedure (sometimes with

only internal review) for projects of higher risk or for

specific needs not addressed through other selection

methods. Discretionary money is also available to the

program manager. It is often allocated for the use of old

equipment and flight time on NASA planes, and for

technology development in preparation for new mission

proposals.

Each division has its own method of proposal review.

In general, however, for every proposal for funds, the

program managers select the peer reviewers. (Life Sciences contracts with the American Institute for Biological

Sciences (AIBS) to provide all peer review panels for their

solicited and unsolicited proposals. AIBS has similar

contracts with other agencies and it provides a ‘‘back

room” check on duplication of funding.) For a small

($100,000 to $200,000) grant, four to five reviews are

solicited. For larger proposals, as many as eight may be

requested. The reviews judge scientific merit and technical feasibility. Program relevance and all other factors are

judged by the program manager. It is generally recognized

that university reviewers give the most conservative

reviews and this type of factor is taken into account.

Each proposal is graded from A to E. (At AIBS, each

reviewer also gives a rating of how competent he or she

is to judge the science contained in the proposal.) If a

proposal receives four reviews with ratings equivalent to

four As, or two Bs and two As, it will generally get

funded. Anything below four Cs would have to be

defended forcefully within the division. However, among

the group of high scoring proposals, it is up to the program

manager to pick and choose to best satisfy his or her

programmatic goals. Occasionally, there is concern over

what proportion of the money should go to universities

and the rest to NASA laboratories and private think tanks.

Office of Aeronautics, Exploration and

Technology (OAET)

In OAET headquarters and in its laboratories, there is

less reliance on AOs and RAs and more use of Requests

for Proposals (RFPs). All proposal review for OAET is

done inhouse. Most of the small aeronautics research

grants that do not involve ‘cutting metal” are performed

by the three laboratories (Ames, Langley, and Lewis) that

operate with primarily OAET aeronautical research

funds; space technology research is performed throughout

the centers. If the contract is large, it will be farmed out

(primarily) to industry and a laboratory will oversee the

contract. The laboratories produce specifications, ask for

bidders, and then negotiate procurement. Roughly 50

percent of the total research and development funds in

OAET stays in the laboratories, 30 percent goes to

industry, and 20 percent to universities.

Department of Energy

The Department of Energy’s (DOE) Office of Energy

Research uses many of the same proposal review techniques as NASA and the offices of scientific research

within DOD, with peer or inhouse review for scientific

merit and final judgment by the program manager.

However, all individual investigator proposals are solicited through Broad Agency Announcements.

The majority of basic “research funds at DOE (twothirds of the Office of Energy Research budget, for

instance) are given to the laboratories. These expenditures

are estimated for the budget request for DOE and are

derived through an iterative process with DOE headquarters. In the defense portion of DOE, almost all of the

research is done in intramural laboratories. The money is

competed among them, using mostly inhouse review.

In the Conservation and Renewable Office and in

many of the other applied research offices, research

money is often allocated with an industrial cosponsor.

Contracts, grants, and cooperative agreements are all

used. The evaluation of industry contracts is regulated by

Federal law and is similar to that used by DOD and

NASA. For individual investigator and university grants,

peer review for scientific merit generally occurs. Inhouse

review is used, at the very least, to allocate monies.
270 ● federally Funded Research: Decisions for a Decade

National Science Foundation

Proposals are received by the Division of Administrative Services and are assigned to the appropriate National

Science Foundation (NSF) program for evaluation. Most

proposals are unsolicited, though a few are in response to

specific Announcements or RFPs. The applications are

then reviewed by the relevant program officer and sent out

for peer review. Proposers and reviewers are invited to

suggest reviewers. The program officer may convene a

panel (ad hoc or standing) to review proposals or can rely

on mail reviews, or both.5 The program officer can solicit

advice from advisory committees, review panels, or site

visits before recommending final action. Recommendations are then sent to division directors for review and

approval.

Proposals are funded on the basis of demonstrated

research performer competence, intrinsic merit of the

research, utility or relevance of the research, and the effect

of the research on the infrastructure of science and

engineering. The success rate for the agency, overall, is

about 30 percent, but is as low as 14 percent in some areas,

such as decision, risk and management sciences.

The program officer has a fair amount of discretion in

making awards recommendations. Typically, after receiving the reviews on all proposals (some programs process

grants on a continuous basis, relying on mail review rather

than panels), the manager will sit down with the reviews

and the program budget, evaluate the area of science each

proposal encompasses, consider how much money the

investigator is getting from other sources, and then make

the difficult allocation decisions.

Many program officers said they tend to give new

investigators a break. Others said they like to help out

smaller colleges. There is a specific program announcement called “Research in Undergraduate Institutions”

that encourages proposals from nondoctoral departments

at institutions that produced 20 or fewer Ph.D.s in science

and engineering in the 2 years preceding the proposal.

Targets are set for the amounts NSF funds each year in

this area, and divisions must ensure that minimums are

met. In some cases, a proposal might be consistent with

areas deemed of high priority, but would not fare well in

disciplinary program competitions.

This is a point in the process where the program officer

can also participate in the agency goal of increasing

awards to women and minorities. Most program managers

interviewed consider this an important goal and make

their best effort to fulfill it. However, it has created a new

dilemma. Money given to young investigators, women, or

minorities comes out of a pool that would normally be

given to the highest scoring proposals, which may come

from older, established scientists with lofty track records.

Program managers have to make the difficult decision of

denying grants to, or cutting the budgets of, known

performers in order to create a more equitable allocation

of funds. The increasing number of applications, stable

funding, and the process of reviewing grants have strained

the system, say some managers. The safeguards built into

peer review consume a great deal of staff time.6

Forty percent of the scientific and technical staff of

NSF is comprised of rotators on temporary assignment,

normally of 1 to 3 years duration.7 They bring direct

knowledge of forefront research to the grants process.

Many interviewees feel that this prevents NSF from

becoming an entrenched, out-of-touch bureaucracy. Rotating staff, however, can disrupt continuity in certain

research areas, as new grants managers have the potential

influence to shift the focus of research every few years.

Department of Agriculture

Agricultural Research Service (ARS)

The proposal process at ARS is unique. It is essentially

a negotiation process between National Program Staff

(NPS) and ARS scientists. ARS scientists work with their

regional directors to send ideas up to NPS program

leaders. Meanwhile, NPS sets out its budget priorities.

Once the administrator approves the plan for the upcoming year, proposals are sent forward to NPS staff. If NPS

staff want to fund a project, they send the proposal out for

external review. Proposals are only sent out for review

after the decision has been made to fund them. The

reviewers are not asked whether the project should be

funded, but how to improve the technical quality of the

research. This places an enormous amount of power in the

hands of NPS.

Obviously, there is mom for criticism of this system of

ex post peer review. In 1986, the ARS administrator asked

the Board on Agriculture of the National Research

Council (NRC) to examine the project peer review

system, assess its effectiveness, and recommend possible

improvements. NRC found a lack of agreement and

understanding among ARS staff regarding the purpose,

use, and effect of the system. There also seemed to be

inadequate understanding within ARS as to how the

administrator balances and optimizes the dual objectives

5The Natjo@ Science Foun&tionhas recatly instituted electronic proposal review panels. See National Science FoundatiorA Electronic proposal

Review Panels: An Option for NSF Program O~cers (Washington DC: forthcoming 1991).

GA forthco~ Gene~ ACCOWIting Office report on peer review procedures found no evidence of sloppy practices at the Natioti Scienw

Foundation. Conmrns were raised, however, about review processes at other agencies, especially the Department of Energy and the National Oceanic

and Atmospheric AdmmI . “stmtion. See David P. Hamiltou “NSF Off the Hook,” Science, vol. 251, Feb. 15, 1991, p. 733.

TNatio~ sci~a Fo~~tioq Report of the Merit Review Task Force, NSF 90-113 (WashingtorL w: Aug. 23, 1990), P. 9.
Appendix C—Funding Allocations in Six Federal Research Agencies . 271

of scientific excellence and mission relevance, and how

project peer review is used in the context of these

objectives.

Recently, NSF changed its rules to allow ARS principal

investigators to apply to NSF for grants. This can be done

only when ARS investigators are affiliated with a

university that becomes the primary recipient of the grant.

ARS scientists also apply directly for other sources of

outside funding support to supplement ongoing research.

Most ARS scientists will apply for other grants to support

postdoctorates or graduate students.

One might ask what motivates the ARS scientist. A

government salary and the system of getting project

money is noncompetitive. The annual performance evaluation motivates the ARS scientist to propose good

projects and perform well. In ARS, it is the scientist that

is peer reviewed, not the research. ARS uses a system

comparable to tenure review whereby a scientist is scored

by peers on the level and quality of his or her research. The

scores determine GS level. If an individual is found to be

slipping, that is, not contributing to the advancement of

the field in a manner demonstrated in the past, he or she

can be demoted.

Cooperative State Research Service (CSRS)

Special Grants—The scientific agenda and budget for

special grants are often specified in the agriculture

appropriation bill. When given discretion over the awards

process, CSRS often institutes open competition with

peer review for scientific merit. Otherwise the research

agenda is negotiated with the participating institution.

Within these institutions, there may be competition for

money (run by the institution itself), but it is often decided

informally.

Competitive Research Grants—The Competitive

Research Grants Office (CRGO) allocates funds with peer

review mechanisms that are very similar to those at NIH

and NSF. Review panels, chosen by the program manager

and associate program manager, judge the scientific merit

of proposals and their relevance to the purpose of the grant

program to rank order them. Proposals are funded in order

until the program runs out of money, and proposals are

rarely pulled out of rank.

A new, congressionally mandated experiment at

CRGO limited indirect costs in research grants to 25

percent in fiscal year 1990 and 14 percent in fiscal year

1991. The U.S. Department of Agriculture implemented

this policy in fiscal year 1990 and proposals are now

negotiated under the new terms. Program managers

projected that this ceiling would be useful in fiscal year

1990 (optimistic estimates said that it would save $3.5

million to be disbursed to other researchers). After the

first few years, universities and other organizations are

expected to bill directly for laboratory space and other

items usually claimed under indirect costs, thereby

recouping the funds.

Forest Service

Research Work Unit Descriptions (RWUDs), written

by research groups within Forest Service research centers,

charter work in a particular problem area. They usually

prescribe a plan for a 5-year duration and often will build

directly on previous work. Staffing needs are directly

related to the RWUD. The station director has a large

amount of discretion to choose projects at the RWU level,

but the RWUDs are reviewed inhouse in the Washington

office to provide balance in a nationally coordinated

program.
APPENDIX D

Academic and Basic Research Decisionmaking in

Other Countries1

Compared to the United States, other countries have

implemented vastly different organizational structures for

their research systems. OTA has surveyed research

decisionmaking practices in nine countries, including the

United Kingdom, the Federal Republic of Germany,

France, Japan, the Netherlands, Sweden, Canada, Australia, and India. It is clear that the institutional structures of

policymaking and funding are critical determinants of the

way in which governments support basic research, and

provide a powerful context to which any new methods of

selection of basic research must be adapted.2 The reasons

for this influence are at least twofold.

First, institutional structures strongly reflect the particular political, economic, and, more generally, cultural

history of a country. While there maybe certain universality in science, this does not carry over to science policy.

Thus, it is essential for any comparative study of

international science policies to place them in the context

of national culture.

A striking example of these contexts is the heterogeneity of different national research systems. As Ziman noted

in the United Kingdom and the United States, the

academic department-”. . . a multi functional organizational entity responsible for all teaching, research and other

activities in a broadly defined scientific discipline . . . “3-is

the predominant form of scientific organization. In

contrast, France with its Centre National de la Recherche

Scientifique laboratories and Germany with its Max

Planck Institutes have research institutes, staffed by

full-time researchers working in a designated problem or

disciplinary area, as the most common model of research

organization.

Second, decisionmaking structures themselves reflect

in part previous processes of selection of basic research.

Some claim, for example, that:

. . . the Big Sciences such as high-energy physics and

astronomy, which were funded generously in the past are

now, as a result, well represented on decision-making

bodies. There has, therefore, been a tendency for early

established sets of priorities and research interests to

become “frozen in” the decisionmaking structure,4

In this section, the research systems in nine countries

will be highlighted and their priority-setting mechanisms

examined (see table D-1 for a summary). Unfortunately

the only comparable figures on the funding of research are

aggregated with development. Figure D-1 shows the

United States, West Germany, and Japan with comparable

total research and development (R&D) funding levels as

a percent of GNP, but West Germany and Japan at much

higher levels for nondefense R&D. The United Kingdom

and France spend less on R&D in both categories. For

ease of comparison to the United States, the focus of this

appendix will be on academic research, the bulk of which

is carried out in the national universities of each country.5

Some generalizations about methods of priority setting

will be drawn first between the nine foreign countries

studied, and then applications to the U.S. research system

will be discussed.

United Kingdom

Three main themes run through the United Kingdom’s

government support of civilian academic science: the

importance of maintaining and enhancing quality in

science, increasing the economic and social returns from

science, and better management through greater concentration and selectivity of science activities. Strenuous

efforts have been made to introduce new policies to

achieve those objectives.

The government reviews its R&D funding annually,

but there is no overall R&D budget. The system is highly

decentralized and each academic department determines

its own R&D programs in the light of its own policy

objectives and priorities. Recent decisions to exclude the

public funding of near-market research led to some

l~s appendix is based on Ron Johnsto~ University of Wollongong, Australi~ ‘‘Project Selection hkchrinkm: hXeIIEitiOIId compfiSOns, ” Om

contractor repo% July, 1990. Available through the National TMmical Information Service, see app. F. The work on this contract was completed before

East and West Germany united. AlSO sw bonard L. Lederm~ “Science and Technology Policies and Priorities: A Comparative Analysis,” Science,

vol. 237, Sept. 4, 1987, pp. 1125-1133.

Q’ ‘~sti~tio~ s~c~es’ refer t. tie ~die=ftenatis~tive agencies, advisory councils, and review panels-that implement Policies for

priority setting and funding in science.

3Johu Z~ Restructuring .kadank Science (Tmndon, England: Science Policy Support Group, 1989).

4J. ~~e ad B. R. M~~ ‘‘~t Direction for Bmic ScientitlcResearch” Science and Technology Policy in the 1980~, M. Gibbons ~d A.

Udgaonk~(eds.) (Manchester, England: Manchester University Press, 1985).

5Except fi c~ad% where resach is conducted in provincial universities.

–272–
Appendix D—Academic and Basic Research Decisionmaking in Other Countries . 273

Table D-l—Recent Approaches to More Selective Support of Basic Research

Priority setting:

Country/agency Steering device Method Decisionmakers Status

UK

UFC Selective core funding to

universities

Interdisciplinary Research

Centers (1988)

Directorates (7 years) and

Program (5 years)

Centres (5-15 years)

NA

Submissions and internal

discussion

Internal discussion

Internal discussion

NA

Panels

Government

appointed

ABRC Appointed

SERC Council (peers and users)

Council (peers and users)

Appointed

ESRC

FRG

DFG

MPG

Appointed

Priority programs

Priority research areas

institutes

Bottom-up discussion

Bottom-up discussion

Researchers

Researchers

Elected

Elected

France

MRT Programmed mobilisateurs Identification of generic

technologies

Identification of leading

researchers

CPE and department

officials

Council peers and department officials

Elected and

appointed

Elected and

appointed

CNRS Annual strategic plans

Japan

Monbusho Priority research areas (3-6

years)

Identification of areas of

strong scientific opportunity and social need

Selection of 2 key fields and

program leader

NA

Identification of future

needs and priorities

Wide consultation, symposia, reports over 2 years;

and government departments, scientific

societies, researchers

Invitations to consortium of

university departments

to propose interdisciplinary programs

Identification of societal

problem, then elaboration of research needs

and opportunities

Computation of agencies’

research priorities,

Monbusho Science Council Appointed

Specially promoted research (5 years)

Science Council Appointed

Netherlands

MES Conditional funding of university research

Second flow funding

NA

RAWB (appointed) and

NWO

Effective consensus

Appointed

NWO Elected

Sweden

Cabinet Research policy bill NA

NFR Consortia Council (appointed and

elected)

Appointed and

elected

FRN Priority fields Council and committees

(research and users)

Appointed and

elected

Canada

ISTC Decision framework for science and technology

ISTC NA

identifying strengths

and gaps

Continued on next page

transitory increase in funds for research. However, the

funding outlook has now dimmed.6

provide funds on a competitive basis to university

researchers and in most cases maintain their own research

centers. The research councils have a high degree of

autonomy in establishing their own priorities and procedures. The Advisory Board of the Research Councils

(ABRC) also plays an important role in advising DES on

the overall budget and on the allocations to the five

councils.

The major source of funds for academic research is the

Department of Education and Science (DES). It provides

general support for university teaching and research

through the Universities Funding Council (UFC) and five

research councils, in what is commonly referred to as the

‘‘science budget. The DES-supported research councils

6For exmple, see Jeremy Chefim, “Deficits Trip U.K. Science Funding Agencies, ” Science, vol. 250, Dec. 14, 1990, pp. 1504-1505; and Peter

Aldhous, “UK Nuclear Physicists Fear SERC’S Cuts$” Narure, vol. 349, Jan. 31, 1991, p. 357.
274 . Federally Funded Research: Decisions for a Decade

Table D-l—Recent Approaches to More Selective Support of Basic Research—Continued

Priority setting:

Country/agency Steering device Method Decisionmakers Status

NSERC Strategic grants program

SSHRC Strategic research

program

Australia

ARC Priority research areas

NH&MRC Priority research fields

India

National Five-year Science and

Development Technology Plan

Council

SERC Thrust area programs

Intensification of research

in high-priority areas

scheme

Identification of 30 themes

in the 3 national priority

areas by consultation,

analysis, and workshops

Biennial seminars and

commissioned reviews

Submissions consultation

and internal discussion

Internal discussion and

consultation

Expert reports, consultation, draft reviews

National exercise of working paper preparation,

review, national seminar

Thrust area identification

exercise

Science Council of Canada

Council (department officials and researchers)

Council (researchers)

Council (researchers) and

panels

Council (government officials)

Council and program advisory committees (scientific experts and government officials)

PACS

Appointed

Appointed

Appointed

Appointed

NA

Appointed

Appointed

KEY:

ABRC = Advisory Board of the Research Councils MPG = Max Planck Gesellschaft (Max Planck Society)

ARC = Australian Research Council MRT = Ministry of Research and Technology

CNRS = Centre National de la Recherche scientifique (National Center of NA = Not available

Scientific Research) NFR = National Science Research Council

CPE = Centre de Prospective et Evaluative (Prospects and Evaluation NH&MRC - National Health and Medical Research Council

Center) NSERC - Natural Sciences and Engineering Research Council

DFG Q Deutsche Forshungsgemeinschaff (German Research Society) NWO = Netherlands Organization for Scientific Research

ESRC = Economic and Social Research Council PAC == Program Advisory Committee

FRG - Federal Republic of Germany RAWB = Science Policy Council of the Netherlands

FRN = Council for Planning and Coordination of Research SERC - Science and Engineering Research Council

ISTC - Industry, Science and Technology Canada SSHRC - Social Sciences and Humanities Research Council

MES = Ministry for Education and Science UFC = Universities Funding Council

Monbusho = Ministry of Education, Science, and Culture UK= United Kingdom

SOURCE: Ron Johnston, “Selection of Basic Research: An International Comparison,” OTA contractor report, June 1990, table 3. Available through the

National Technical Information Service, see app. F.

Recently UFC introduced a system of ranking aca- ABRC has also initiated or encouraged a number of

demic departments on their research capability as part of new procedures in the allocation of support for academic

the determination of support.7 General University Funds research. The first has been to increase the proportion of

are distributed by UFC, one component of which is for the funding flowing to “directed programs” which have

support of research, though universities may use these increased from 19 to 32 percent of the councils’ grants

funds for education as well. Since 1986, the formula to since 1980. These are designed to help coherent programs

determine how much should be given to a department of research in selected areas or to stimulate research in

favors those institutions judged to have high-quality fields judged to require more effort in the national interest.

research. In 1989, UFC further ranked departments within The second initiative has been to increase the proportion

universities on a scale of 1 to 5. The criteria used to of program grants, as opposed to project grants, from 15

determine ratings were: 1) publications, 2) success in percent in 1980 to 20 percent in 1989. Program grants are

obtaining research grants and support for students, 3) generally larger, support a bigger team of researchers, and

success in obtaining research contracts, and 4) the last for a longer period (5 years) than project grants.

professional judgment of advisory group and panel Thirdly, in 1985, it recommended a set of six criteria to be

members. There is considerable debate on the efficiency adopted by the research councils in determining funding

of this approach.8

(excellence, applicability, timeliness, pervasiveness, sig7sce ‘British science Indicators, Outlook on Science Policy, vol. 11, November 1989, pp. 112-1 13; and M.P. Carpenter etat., “Bibliometric Profiles

for British Academic Institutions: An Experiment To Develop Research Output Indicators,” Scientometrics, vol. 14, Nos. 3-4, 1988, pp. 213-233.

‘See Peter Aldhous, “University Funding Plan Collapses in Chaos, ” Nature, vol. 348, Nov. 1, 1990, p. 3.
Appendix D—Academic and Basic Research Decisionmaking in Other Countries ● 275

Figure D-l—R&D Expenditures as a Percent of Gross National Product, by Country

Total R&D expenditures

Percent

‘“~

2.8

2.6

2.4

2.2

2.0

1.8

I /

.

\

\

I

France

I ! I 1 1.44 1 1 1 1 1 1 I I I I 1 1 1 I

69 73 77 81 85

Year

KEY: FRG = Federal Republic of Germany.

3.0

2.8

2.6

Nondefense R&D expenditures

Percent

3

“

2.8

0 ~3-0

.–J t2.8

2.6 -/’l Japan —,/

2.6

/ ’

/’

/ 2.4

/

2.4

2.4

FRG —

/

2.2 /

-

2 . 2

2.2 / United States

2.0- \ ,

France

2.0

1.8-

1.8

1.6 .-

-.

. .-

1.6 1.4-

1.4 1.2~

71 75 79 8 3 87

Year

2.0

1.8

1.6

1.4

1.2

SOURCE: National Science Foundation, Nafiona/Pattems of R&D Resources: 1990, final report, NSF 90-316 (Washington, DC: 1990), tables B-18 and B-19.

nificance for education and training, and exploitability),

which have been applied by some of the councils.

The most recent initiative, together with the research

councils, UFC, and DES, has been the establishment of

Interdisciplinary Research Centers (IRCs). Funded for 6

years, their objectives are: 1) greater concentration of

research effort; 2) more interdisciplinary collaboration; 3)

increased effort in areas of ‘‘strategic’ science, i.e.,

important for economic progress; 4) stronger interface

between strategic research in higher education and

industry; 5) more positive and purposeful management of

research within higher education; and 6) more effective

collaboration between universities and the research councils in the deployment of research resources.

Thus, the general approach to academic research

selection in the United Kingdom can be summarized by

two statements: 1) there is an increasing degree of priority

setting, the priorities emerging from interaction between

peer review committees and various advisory bodies;9 and

2) there is a move to concentrate research resources by

provision of larger and longer grants to programs and

centers.

Federal Republic of Germany

In contrast to the United Kingdom, the most striking

feature of German science policy is the indirect influence

of government, which funds virtually all academic

research but accords significant autonomy to researchperforming and research-promoting, institutions.10 The

freedom of research is expressly established in the Federal

German constitution. In this context, and that of a

generous and growing budget for research, it is apparent

that there is little expressed need for, and indeed some

hostility to, notions of directed research or priority setting.

Research is performed primarily in three sets of

institutions: the 50 universities, the 60 institutes of the

Max Planck Gesellschaft (MPG), and the 13 national

research centers. Decisionmaking about research project

selection is largely made within the research institutions

or by the Deutsche Forschungsgemeinschaft (DFG-the

German Research Society).

DFG is the central, self-administering, academic research support organization (spanning basic, applied, and

strategic research) in the Federal Republic of Germany. It

receives its funds from the federal and state governments,

gwor~ G-ouP on peer Review, peer Review, a report to the Advisory Board for the Research CO~CilS(Lmdou EWl~d: NovemM Iwo).

l~is Swtion hM ~nefited from ~onard hale- Scientific and International Affairs Directorate, National Science Foundatio~ perSOIlal

communication December 1990.
276 . Federally Funded Research: Decisions for a Decade

and as the primary source of “drittmittel’’-the additional funds for research-it exerts influence on the

profile of research. The function of DFG resembles that

of the research councils in the United Kingdom. It is

organizationally independent of government, but financially dependent on it. It is a scientific society whose

membership includes the universities, other research

institutions such as the Max Planck institutes, seven of the

national research centers, and prominent scientific associations. The president and senate are elected, as are the

approximately 400 expert consultants who hold office for

3 years, to provide expert peer review.

DFG shapes the profile of German academic science

from the bottom up, augmenting government funding of

salaries, instrumentation, facilities, and MPG initiatives.

The resistance to direction of research by DFG is clear:

‘‘DFG officials are determined that targeted funds should

not exceed 10% of overall expenditure since this might

give rise to renewed alarm about academic autonomy and

flexibility. ”11

The functions of MPG are to undertake research in

areas of particular importance, newly emerging areas, or

where a concentration of effort is required. Its legal status

is that of a private nonprofit organization despite most of

its finance coming from the government. Proposals for

new institutes are received each year and undergo an

extensive evaluation process. Judgments are made by the

MPG senate on scientific merit, fruitfulness, appropriateness to MPG (as opposed to universities), and the

availability of an outstanding scientist to fill the position

of leader. Once established, an institute is subject to

review every 2 years by a visiting committee and every 7

years by a prestigious panel of overseas experts. Occasionally an institute is closed.

The national research centers were established as

essentially big science institutes in fields such as nuclear,

aviation, and space research where a large concentration

of very expensive infrastructure was necessary. Some of

these centers are now facing the challenge of missions

completed or no longer relevant, and are seeking new

orientations.

OTA concludes that basic research selection in the

Federal Republic of Germany rests essentially on bottomup proposal pressure and peer review. Priority setting is

used mainly to achieve concentrations of effort through

cooperative teams or centers. The unification of the

former German Democratic Republic and the Federal

Republic of Germany may offer opportunities of international importance for research, but it is too soon to tell.12

France

The French approach to decisionmaking for academic

research, as for all areas of the economy and society, rests

on a traditional commitment to centralized planning. In

the area of research, the strong emphasis is on economic

goals, and priority is given to industrial research.13

The Ministry of Research and Technology is responsible for recommending and implementing government

policies in the field of science and technology, and for

determining priorities for research, with the advice of the

Research and Technology Council. The performance of

basic research occurs primarily in the Center National de

la Recherche Scientifique (CNRS) laboratories located

alongside the universities and within the universities,

which receive core funding from the Ministry of National

Education and CNRS. CNRS maintains its own laboratories, independent of universities. The French system relies

on block grants to the laboratories and research groups,

rather than specific project grants to individual researchers. But there is growing academic criticism of CNRS’s

favored “inhouse” position as a research performer.

CNRS finances the entire range of academic research

from the physical sciences to the humanities. Annual

strategic plans are prepared, which rely as much on the

identification of leading research individuals and groups

as on promising areas of research. In addition, CNRS has

conducted a range of prospective studies that feed into the

planning process.14

The French Government has recently mounted a major

initiative to promote the strategic application of foresight

and evaluation to the national research system. A National

Research Evaluation Committee (CNER) was established

in response to the government’s decision to institute the

systematic periodical assessment of all researchperforming institutions. This follows the experience of

the National Evaluation Committee of the Universities,

founded in 1985, which has assessed the research of 25

universities on a voluntary basis.

CNER is responsible for the evaluation of the organization and results of a national technological research and

development policy. To achieve this goal, the committee

ensures the periodic assessment of institutes, programs,

1 IB.R. mmd J. Irvine, Research FOmsight (Londo~ England: Frances Pinter, 1989), p. 80. Despite the pronouncement, Wgeted tids me

suspected to exceed 10 percent.

Izsee Rolf H. Sime@ “Resemch L~dscape Requires Careful Gardeners: Science in Unified Gemm

TMks,” German Research Service Special Science Reports, vol. 7, January 1991, pp. 11-13.

Y—Exp@’ (@hio~ During Villa-Hugel

IsFor ~ovemiew of Fr~ce’s 24 research agencies, see “FAST Guide to French Government R& D,’ French Advances in S&T, VO1. 4, No. 1, titer

1990-91, pp. 3-6.

14see -~d ~~e, op. cit., footnote 11, PP. 47-50.
Appendix D—Academic and Basic Research Decisionmaking in Other Countries ● 277

and incentives of all kinds financed from the civilian

technological R&D budget.

In summary, there is a strong emphasis on planning and

direction of research toward technological objectives in

the French research system.15 The attempt is made within

the universities to set basic research directions largely on

the grounds of scientific excellence. However, within

CNRS, scientific departments put forward proposals that

are judged internally on grounds of merit and relationship

to priority areas.

Japan

The dense population, a deep commitment to the values

of the group, and the spiritual principles of Confucianism

have produced a culture that emphasizes the values of

harmony, respect, and decisionmaking by consensus,

even if the process is protracted. 16 These values permeate

the decisionmaking structures and procedures with respect to science.

Within the government, the Prime Minister’s Office

and its key policy body, the Council for Science and

Technology (CST), exercises the highest level of control

over the direction of scientific and technical research.17

Four main government agencies in Japan support and

target R&D: the Science and Technology Agency (STA);

the Ministry of Agriculture, Forestry, and Fisheries; the

Ministry of Education, Science, and Culture (Monbusho);

and the Ministry of International Trade and Industry

(MITI).

STA is responsible for the overall coordination of

science policy among the different ministries and agencies. It is also responsible for big science and includes

research institutes to fill this function.

While STA is the agency primarily responsible for

basic research, MITI has had considerable influence over

research policy in Japan through its emphasis on applied

R&D. MITI runs 16 national research laboratories and

develops research programs with industry. These programs are most often in technological areas in which

Japanese industry is considered to be weak, into which no

single company would enter alone, or which are for the

public good and not necessarily commercially valuable.

While MITI is most involved in raising the technological

level of Japanese industries, this is small compared with

STA’s activities.18

Monbusho is responsible for the promotion of research

across all fields and for the national university system. In

formulating policies, Monbusho consults its science

council, consisting of 27 eminent scholars whose names

are put forward by the academic societies but appointed

by the Minister. In addition to general research funds for

divisions of university faculty, construction, and equipment monies, Monbusho has a new program in which the

Science Council chooses a research field for priority

funding (generally two a year).

The Science Council is a democratic body established

by law as the representative body of Japanese scientists

and engineers. It has the right to make recommendations

directly to the government on the ways and means to

promote science and technology. Among its major

successes was the establishment of nine interuniversity

research centers. However, in recent years its influence

has waned in favor of CST. The role of CST in integrating

and coordinating research has been considerably strengthened through the establishment of a Science and Technology Promotion Coordination Fund, which is used in part

to support basic research in special priority fields

designated by CST.

Thus, the essential process of research selection in

Japan is through the time-honored mechanism of a

committee of wise men. Priorities are established by this

consensual process, involving varying degrees of interaction with an influence of academic researchers on the one

hand and government officials on the other.19

Netherlands

The Dutch are among the leaders in formulating science

policy in Europe and have carried its implementation

much further than many other countries. The major

emphasis of science and technology policy has been on a

more effective planning and linking of strategic and

applied research to national economic and social needs.

The Minister for Education and Science has recently

produced a discussion document, “Towards a Science

Policy for the Nineties, ” As a result, the government has

once again decided to elevate science on the Dutch

Issee Remi B~, “S~ategic processes and S&T Indicators: Towards a Key Role in R&D Management systems, ” The Research System in Transition,

S.E. Cozzens et al. (eds.) (Dordrech~ Holland: Kluwer, 1990), pp. 227-239. For the past 2 years, the Center for Technology Forecasting and Assessment

in the Ministere de la Recherche et de la Twhnologie, in conjunction with the Commission of the European Communities, has published an R&D

Evaluation Newsletter, which reports the results of research evaluation efforts throughout the world. Stressing evaluation has not overtly affected

plarming or resource allocation decisions.

Icsee Genevieve J. KneZO, ‘ ‘Japanese Basic Research poficies, CRS Report for Congress (Washington DC: Congressional Research Service, Aug.

1, 1990).

17see Cowcil for Science ~d ~c~olo=, pohcy Cotittee, he~ster’s Office, and co~tt~ on Guidelines for Research Evaluation Basic

View on Research Evaluation (Tokyo, Japan: 1986).

Igsee Jo~son Chalmers, Mm and the Japanese Miracle (Stanford, CA: Stanford University ~ess, 1982).

lgsee JOhII hvine et d., Investing in the Future (Worcester, England: Billings & SOIM Ltd., 1990).
278 . Federally Funded Research: Decisions for a Decade

political agenda. The three major objectives for the 1990s

are: 1) establishment of a more effective scientific basis

for key societal functions; 2) achievement of an important

role of research in the process of internationalization; and

3) the well-balanced development and application of

science and technology to economic, social, and cultural

needs.

The independent Science Policy Council of the Netherlands (RAWB) is the central advisory body on science

policy. It has had significant influence on priority setting

and resource allocation through its reports on future needs

and opportunities in particular fields. RAWB also undertakes assessments.

Basic research is performed essentially in the universities that are funded by the Ministry of Education and

Science (which administers over one-half of the government R&D budget) and in the institutes established by the

Netherlands Organization for Scientific Research (NWO).

The latter also funds some research in universities.

Before the latter half of the 1970s, university research

was largely considered in relation to educational policies,

with particular emphasis placed on the close relationship

between academic research and university teaching.

Gradually there has been a shift in approach developed in

the last decade or so, and academic research objectives are

increasingly related to external economic and social

requirements. The scope for effective planning and

steering of the direction of university research in the

context of science policy has been constrained in the past

by the funding structures for university R&D. The

Netherlands is now experimenting extensively with these

structures and performance assessments.

A feature of policy in the last few years has been the

move gradually to transfer responsibility for research

from government departments to universities or to

institutes operated by NWO. This represents one element

in a developing strategy to reshape the existing national

R&D system, which is widely seen as lacking the degree

of integration and coherence that is needed if the country

is to maintain an internationally competitive effort in key

areas over the next decade.

In summary, the Dutch Government and universities

have been particularly active over the past decade in

reshaping their science and technology policy decisionmaking procedures and capabilities. While this effort has

been directed to technology development, there has been

some attention as well to methods of project selection for

research. Experimentation with these methods has allowed new policy alternatives to emerge. In particular,

these new methods allow more funds to be allocated on a

competitive basis and in priority areas. The priorities are

determined by traditional committee methods where

scientific and government interests meet and negotiate

from their own perspectives.

Sweden

There is a long tradition of extensive government

involvement in decisionmaking in a range of research

areas in Sweden, grounded in a lengthy process of

consensus formation, planning, and evaluation. R&D has

been strongly directed, particularly through the central

establishment of priorities and funding levels every 3

years in a Government Research Policy Bill.

Policies are developed through an interactive process

between funding agencies, departments with responsibility for R&D, and a group in the Cabinet Office, with

overall responsibility vested in the latter. There is a strong

bottom-up element in the decisionmaking process, which

is set against the background of the Bill on Research. That

this bill is programmed into the legislative process allows

all the players to develop their initiatives in the period

leading up to the consideration of the bill. Background

studies, monitoring of overseas developments, and symposia that bring together representatives from academia,

industry, and the government all form part of the process.

This extensive consultation and debate ensures that all

interested parties have an opportunity to make their views

known and that the community in general is committed to

the areas and issues identified in the research bill. In the

February 1990 bill, the priority areas were: 1) strengthening basic research in universities; and 2) increasing

research in five target areas-environment, marine processes, public health, industrial safety, and cultural research.

Basic research, roughly one-quarter of Swedish R&D,

is conducted almost entirely within universities-there is

virtually no government research capacity. Three research

councils play a major role in determining research areas

and resource allocation: the Medical Research Council,

the Natural Science Research Council, and Council for

Research in the Humanities and Social Sciences. Each

council has a large degree of autonomy. A fourth council,

the Council for Planning and Coordination of Research,

is not a‘ ‘research council’ per se. It assists in government

research planning and coordination, public understanding

and participation in this process, and the assessment of

Swedish research capabilities.

Strong direction setting characterizes Swedish strategic

research. 20 Less direction is given to basic research, but its

ZoSee George Feme, Science ad Technology in Scandinavia (London, England: Ian- 1989). Not OdY is the research evaluation tradition strong

(perhaps the strongest in Western Europe), but it also involves the participation of foreign scientists andmuchpublic discussion of decisions. See Michael

Gibbons, Organisation forlkonomic Cooperation and Development, Evacuation ofResearch in Sweden (Manchester, England: University of Manchester

Press, 1984).
Appendix D—Academic and Basic Research Decisionmaking in Other Countries ● 279

priorities are affected through the connection to strategic

initiatives. The 3-year research bill provides a strong

framework for this connection.

Canada

Canadian science policy has been marked in the past

decade by a high level of debate, conflict, and change. The

major pressure for this change has been the heavy reliance

of the Canadian economy on its resource-based industries

and the recognition that such economies are becoming

increasingly vulnerable and noncompetitive. Canada has

a reputation for scientific excellence and long-established

central government laboratories.

Canada is a federal system, with the special requirements for coordination that such a system implies. The

higher education sector is funded almost entirely from

taxes collected by the national government and allocated

to the provincial governments.

There is a large set of advisory and decisionmaking

bodies in the Canadian science and technology system.

Three of the most important are: 1) the National Advisory

Board on Science and Technology, which advises the

Prime Minister on overall guidelines; 2) Industry, Science, and Technology Canada (ISTC), which coordinates

industry and academic research; and 3) the Science

Council of Canada (SCC), which provides independent

advice on science and technology. SCC has been a

long-time advocate of systematic research priority setting,

and ISTC compiles the Decision Framework for Science

and Technology, which requires departments and agencies responsible for R&D to prepare annual lists of

priorities.

Nevertheless, while there has been a strong push

toward linking research more effectively to national

needs, 21 this has been resisted in the case of basic

research. The overall framework of planning thus far

impinges only indirectly on basic research.

Australia

Australia is in many respects similar to Canada, with its

federal structure and its drive to broaden and deepen the

technological intensity of its predominantly agricultural

and minerals-reliant economic base. There is also a long

tradition of commitment to internationally excellent

research, with a particularly strong government research

capability.

In recent years Australia has developed a sectoral

model of science policy, with major R&D funding and

performing responsibilities spread across a number of

major departments. To overcome problems of fragmentation, a Coordination Committee of Science and Technology, made up of senior officials of the departments, has

been appointed. In addition, the Prime Minister receives

advice from the Science Council-composed of ministers, industrialists, and a minority of scientists, and the

Australia Science and Technology Council-composed

of appointed academics and industrialists.

With nearly 70 percent of research in the public sector,

there has been a considerable emphasis on restructuring

to give greater priority to strategic research directed to

medium- and long-term industrial needs.22 The universities, which are established under state legislation but

funded by the federal government, are also under increasing pressure to serve national interests.

In Australia the principles and practice of priority

setting have been effectively established for strategic and

applied research.

23 As in Canada, planning for basic

research has met with a degree of resistance from

researchers and universities, who have seen it as a

challenge to their autonomy. Hence, priorities have been

applied to basic research only to a modest extent.

India

Science and technology (S&T) in India has grown

under strong and sustained political support. Even before

India became independent in 1947, the national leaders

had recognized the role of S&Tin national development.

Nehru’s vision of S&T came to be accepted as an

instrument not only for industrial and economic development, but also for transforming a tradition-bound society

into a progressive nation.24

In line with the concept of socialism, the state continues

to be a strong supporter of S&T, providing 80 percent of

the funds for all R&D. It also shoulders the responsibility

for directly guiding and planning the activities of an

extensive network of S&T institutions. R&D activities are

carried out by institutions that come under central and

state government departments, industrial units, professional bodies, and by university-type structures. The

Zlsee B,aha Abu-bban (cd.), University Re.yearch and ~he Future of Canada (ottaw~ Canada: University of Ottawa ~ess, 1988).

22J. Ford, “Australia Tilts Its R&D Towards Industry, ” New Scientist, vol. 116, No. 1580, 1987, p. 19.

23M. Dodgson, “National Policies--Research and ‘Ikchnology Policy in Australia: Mgitimacy in Interventio~” Science and PubIic Po/icy, vol. 16,

No. 3, June 1989, pp. 159-166; and Australian Science and lkchnology Council, Serting Directions for Australian Research (Canberra, Australia:

Australian Government Publishing Service, June 1990).

~A. Jain, “science and ‘lkchnology Policies in India, ’ Science Policies in International Perspective: The Experience of India and the Netherlands,

P.J. Lavakare and J.G. Waardenburg (eds.) (London, England: Frances Printer, 1989), p. 139.
280 . Federally Funded Research: Decisions for a Decade

universities carry out research and provide human resources for research institutions.25

Though the basic orientation of national S&T policy

has been, and still is, to treat S&T as an integral part of

socioeconomic development, there have been several

changes in organization and planning strategies over the

years. For example, the Industrial Policy Resolution of

1948 allowed considerable scope for introducing foreign

technology into the country, but had little influence on

linking the imports with the indigenous S&T structure.

Under this policy ethos, an extensive governmentdominated research infrastructure emerged during the

next 20 years, almost undisturbed by economic developments. Priorities in S&T were set by the leaders of

science, but efforts at formulating policy instruments and

plans to couple S&T capabilities with requirements of

agriculture, industry, and other economic sectors were

weak.

The Prime Minister of India has always been the

minister-in-charge for S&T, with three advisory mechanisms at his or her disposal: the scientific adviser, the

adviser for technology missions, and an independent

1 l-member Science Advisory Council. India’s planned

approach for the development of S&T became part of the

national planning exercise. The Planning Commission

plays a central role in formulating the national S&T

5-year plan with the involvement of scientists, technologists, and representatives of concerned agencies and

departments.

International Comparisons

From the discussion above, it is clear that the methods

used for selecting basic research for government support

vary greatly among countries. For example, Canada and

Australia are pluralistic and strongly averse to directing

research initiatives, while there is growing pressure in the

United Kingdom for priorities to be determined centrally,

a tradition long observed in France.

OTA finds that in every one of the nine nations

examined there has been a substantial development of

methods for effective targeting of strategic and applied

research to national economic needs. This push for greater

economic payoff from research has also led some

countries to increase their proportions of strategic basic

research at the expense of undirected basic research.

There has also been considerable experimentation with

new methods to identify strategic avenues of high

promise. However, the development and application of

new selection methods for basic research has generally

been approached with considerable caution by research

funding agencies and been met with considerable opposition from researchers. Also, there is no evidence that

government targeting of research has increased economic

payoffs.26

Another finding is that the extent of direction of basic

research is apparently directly related to the need to do

so-the most important factor is the availability of

resources to support basic research. In countries like

Germany and Japan where there appears to be little

shortage of funds to support basic research, there is no

great enthusiasm for more central direction of research—

even though both countries have elaborate mechanisms

for targeting strategic research and linking it to industrial

and commercial opportunities. At the other extreme,

countries suffering a significant squeeze on funds available for basic research, and who have been less successful

in establishing mechanisms for pursuing an adequate

level of strategic research with a strong application

orientation, e.g. the United Kingdom, are striving hard to

achieve greater government influence over the direction

of basic research.27

The major vehicle used to influence the direction of

basic research has been the setting of priorities. Evidence

of its effectiveness, however, remains limited. Priority

setting has generally involved the identification of

research areas of special interest through interaction

between the academic members and professional staff of

research funding agencies and varying degrees of consultation with the research community. In some countries

fixed sums are allocated for competition in priority areas;

in others, the priorities become simply another criterion

to be considered in the evaluation of proposals.

A recent Organisation for Economic Cooperation and

Development report noted the broadening of the concept

of priorities to include not only “thematic” priorities

(e.g., identifying areas and problems that deserve greater

attention such as optics), but also ‘‘structural” priorities

(e.g., creating new institutes or research teams, or

purchasing equipment and facilities) .28 Indeed the report

zsme nmb~ of ~ivemities kgrOWU from 20 in 1947 to 160 in 1987. hss than 10 percent of mtioml research and development expenditie goes

to the university system for research, a proportion far smaller than that in the other countries reviewed here.

zGThis is a conclusion dawn by economist Harvey Averchin his survey of the literature. See Harvey Averch, ‘‘Policy Uses of ‘Evaluation of Research’

Literature: Post-1985 World Research EvalWtio~’ OTA contractor report, July 1990, annotated bibliography. Available through the Nationat Twhnical

Information Service, see app. F.

zT~demm, op. ~it,, foomote 10, ~autiom tit “s~ving ~d” is not the s~e as su~eeding. No co~&y wfil soon ~ome as centrally controlled

as France. And there is a culture of criticism in some cultures that masks policymaking tendencies. The UK, Canada, and Australia, like the United States,

have a tradition of criticizing government. Germany, Japan, and Sweden have little open criticism although problems and dissatisfactions of researchers

may be as widespread as in other countries.

ZSGibWns, op. cit., footnote 20.
Appendix D—Academic and Basic Research Decisionmaking in Other Countries ● 281

argues that the two are ‘indissolubly linked, ’ as thematic

priorities cannot be implemented without adequate structural support.

In the nine countries reviewed here, the major form of

structural priority is greater concentration of research

resources through an increase in the funds allocated to

long-term programs and centers. Thematic priority setting

has also been increasingly applied by the majority of the

countries, though (with the exception of the United

Kingdom) such priorities represent no more than 20

percent of research agency budgets. These thematic

priorities have been established at a number of levels,

ranging from the national arena to that of the research

agency, from a set of disciplines or problem areas to

individual disciplines.

Examination of the mechanisms and experiences of the

nine countries in developing and implementing new

approaches to the direction of basic research yields four

distinct models of priority setting and implementation:

structural, thematic disconnected, thematic connected,

and systematic. In the following section, these models are

applied to the U.S. experience.

Structural

In the United Kingdom, Netherlands, Canada, and to a

lesser extent, Australia, the emphasis has been on the

application of structural priorities. These have generally

followed the principles of concentration, either by allocating more of the resources in larger units, and/or by

increasing the small proportion of research funds distributed on a competitive basis. However, concentration

alone carries the danger of freezing national capabilities

around present or past historical strengths. Structural

priority setting thus would not appear to be an appropriate

mechanism, of its own, to identify and respond to

challenges and opportunities of the future.

The British system would appear to have a particular

defect in the extent to which the basis for implementation

of structural priorities and determination of thematic

priorities occurs behind closed doors by an appointed and

nonrepresentative elite. Such a nonparticipative approach

would appear to have grave dangers of engendering

hostility and resentment among researchers instead of

building consensus required for effective priority setting

and implementation. In contrast, in the Netherlands

bibliometric measures have been developed in an attempt

to provide a public and objective basis for structural

priorities together with an open and transparent system of

evaluation.29

Thematic Disconnected

This model is evident in the United Kingdom, Canada

and Australia, and for some programs in Japan. It

describes the system in which priority setting is effectively disconnected from the priority-implementation

process. In each of these countries the establishment of

priorities occurs in a relatively closed process, but the

implementation is through traditional methods of an open

call for competitive proposals, to be evaluated by peer

panels together with ad hoc referee reports. Such a system

has the virtues of combining thematic priority setting with

researcher freedom to develop research programs subject

to evaluation. There is a real danger, however, that the

priorities would serve as little more than signposts for

labeling of projects, and the level of implementation of

the priorities could remain quite low. The deep resistance

of Canadian and United Kingdom scientists to just such

a system would appear to confirm the likelihood of this

problem.

Thematic Connected

This model describes systems in which the prioritysetting and implementation mechanisms are tightly coupled. Within this model there are two different types. The

Federal Republic of Germany represents a bottom-up

form of the thematic connected model, in which the

scientists themselves are primarily responsible for identifying thematic priorities, but once established, it is

essentially the same researchers who are invited to

prepare proposals according to negotiated criteria.

The other type is the top-down thematic connected

model, which operates in France, India, and less so in the

Council for Planning and Coordination of Research

(FRN) in Sweden. In these cases, the thematic priorities

are identified by research agencies, composed variously

of researchers, users, government officials, and community representatives. Then the research agency can work

with select individuals and groups to develop proposals

that meet the requirements as determined by the agency.

For example, though modestly funded, the Swedish FRN

provides a positive model of building basic research

around societal needs, articulating research problems with

relevant research disciplines and specialties.

The top-down thematic connected model has considerable advantages in terms of efficiency and effective

implementation. However, it is more likely to be acceptable in a nation where a culture of cooperation and

planning is well established. In a more competitive

culture this system could be seen as being too readily open

to nepotism and political favoritism.

z~or exmple, see A.J. Nederhof and A.F.J. van RM.11, ‘‘An Internatioml Interview Round on the Use and Development of Science and Technology

Indicators, ” report to the Netherlands Ministry of Education and Sciences, Directorate-General for Science Policy, June 1988.
282 . Federally Funded Research: Decisions for a Decade

Systematic

This model is best represented in Sweden and France,

and perhaps India. In Sweden, an extraordinarily intensive systematic process of consultation, information

gathering, and preparation and review of position papers

and symposia leads to the drafting of a 3-year research bill

that sets the national directions in research. It represents

a major exercise in consensus generation that might be far

less successful in countries where consensus is not a

strong national feature.

Similarly in France, a major systematic effort is being

directed to establishing a national capability in research

intelligence, foresight, and evaluation. A wide range of

foresight exercises30 have been attempted and two public

sector think tanks-the Centre de Prospective et d'Evaluation and the Observatoire des Sciences et Techniques—

have been established to attempt to produce indicators,

conduct evaluations, and create a national capacity in

research evaluation. It remains to be seen whether the

work of these units will feed into decisionmaking.

Applications for the United States

As the research economy changes in the United States,

the experience of other countries in coping with their

research economies can be instructive. However, the

institutional structures of policymaking and funding are

critical determinants of the way in which governments

support basic research and provide a powerful setting to

which any new methods of selection must be adapted.

Hence any system operating in another country would

need to be refashioned to fit another’s political culture,

structure, and decisionmaking practices.

There are a number of features endemic to the U.S.

research system to bear in mind when considering the

applicability of the experiences of the nine countries

surveyed.

1. In each of the nine countries, the government pays

for researcher salaries, support, equipment, and

materials through general institutional grants,

which are supplemented by competitive funds for

research projects. In contrast, the primary funding

2.

3.

mechanism for academic research in the United

States is competitive support for specific research

projects.

In the vast majority of the cases in the nine

countries, individual researchers and research teams

have only one possible source of funds to support

their research beyond that available from institutional grants. In the United States, there has been a

plurality of potential funding agencies for researchers.

In the United States, there is a very high level of

absolute funding for basic research, comparable to

that of Federal Republic of Germany (once defense

research is excluded). In addition, all nine countries

have shifted almost all of their big science into

international cooperative ventures.

What, then, are the lessons for U.S. agencies? The

strategies described above represent experimental alternatives for research decisionmaking.

The structural priorities model describes the Department of Defense in the 1980s and presumably the 1990s,

especially in reference to the consolidation of the defense

laboratories. Both the thematic disconnected and the

thematic connected models might be appropriate for

consideration by the National Institutes of Health, the

National Science Foundation, the Department of Energy,

and the National Aeronautics and Space Administration.

Indeed, components of these models are already in place

at these agencies. The thematic disconnected model is

likely to be most appropriate when the identified thematic

priority area is relevant to a range of well-established

disciplines. Where a thematic priority is opening up a very

new area, sitting at the boundaries of a number of

disciplines, or demanding a large allocation of resources,

the more directive thematic connected model might prove

effective.

However, what is more important in the U.S. context is

that the research agencies experiment and evaluate

research priorities in a systematic and open way. At the

same time, these models and research policymaking in

other countries could be monitored for some possibly

valuable lessons.

30- ad ~~e, op. cit., footnote 11.
APPENDIX E

Top 100 U.S. Academic Institutions Ranked by

Citation Impact, 1981-88

Rank Institution Articles Citations Mean

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

34.

35.

36.

37.

38.

39.

40.

41.

42.

43.

44.

45.

46.

47.

48.

49.

50.

Rockefeller University. . . . . . . . . . . . . . . . . . . . . . . .

Harvard University. . . . . . . . . . . . . . . . . . . . . . . . . . .

California Institute of Technology . . . . . . . . . . . . . .

Massachusetts Institute of Technology. . . . . . . . . .

University of California, San Francisco . . . . . . . . . .

Yale University. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Stanford University. . . . . . . . . . . . . . . . . . . . . . . . . .

Washington University . . . . . . . . . . . . . . . . . . . . . . .

Brandeis University. . . . . . . . . . . . . . . . . . . . . ., , . .

Princeton University . . . . . . . . . . . . . . . . . . . . . . . . .

Yeshiva University . . . . . . . . . . . . . . . . . . . . . . . . . . .

University of Washington. . . . . . . . . . . . . . . . . . . . .

University of Chicago . . . . . . . . . . . . . . . . . . . . . . . .

University of California, San Diego . . . . . . . . . . . . .

University of California Santa Cruz. . . . . . . . . . . .

Tufts University . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The Johns Hopkins University . . . . . . . . . . . . . . . . .

University of California, Santa Barbara. . . . . . . . . .

University of Oregon . . . . . . . . . . . . . . . . . . . . . . . .

Dartmouth College . . . . . . . . . . . . . . . . . . . . . . . . . .

University of Colorado. . . . . . . . . . . . . . . . . . . . . . . .

Vanderbilt University. . . . . . . . . . . . . . . . . . . . . . . . .

New York University . . . . . . . . . . . . . . . . . . . . . . . . .

Columbia University . . . . . . . . . . . . . . . . . . . . . . . . .

University of California, Berkeley . . . . . . . . . . . . . .

Duke University . . . . . . . . . . . . . . . . . . . . . . . . . . . .

University of Pennsylvania. . . . . . . . . . . . . . . . . . . .

Baylor University . . . . . . . . . . . . . . . . . . . . . . . . . . .

University of California, Los Angeles . . . . . . . . . . .

SUNY at Stony Brook . . . . . . . . . . . . . . . . . . . . . . . .

Cornell University . . . . . . . . . . . . . . . . . . . . . ... , ,,

University of Texas . . . . . . . . . . . . . . . . . . . . . . . . . .

Boston University . . . . . . . . . . . . . . . . . . . . . . . . . . .

University of Alabama . . . . . . . . . . . . . . . . . . . . . . .

University of Virginia . . . . . . . . . . . . . . . . . . . . . . . . .

University of California, Irvine. . . . . . . . . . . . . . . . .

University of North Carolina. . . . . . . . . . . . . . . . . . .

University of Minnesota . . . . . . . . . . . . . . . . . . . . . .

University of Rochester. . . . . . . . . . . . . . . . . . . . . . .

SUNY at Albany . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Brown University . . . . . . . . . . . . . . . . . . . . . . . . . . .

University of Wisconsin . . . . . . . . . . . . . . . . . . . . . .

University of Utah . . . . . . . . . . . . . . . . . . . . . . . . . . .

University of Michigan . . . . . . . . . . . . . . . . . . . . . . .

Indiana University. . . . . . . . . . . . . . . . . . . . . . . . . . .

University of Southern California. . . . . . . . . . . . . . .

University of Massachusetts. . . . . . . . . . . . . . . . . . .

University of Connecticut. . . . . . . . . . . . . . . . . . . . .

University of California, Riverside. . . . . . . . . . . . . .

University of Pittsburgh. . . . . . . . . . . . . . . . . . . . . . .

4,058

30,228

8,536

13,486

14,910

13,516

15,834

9,384

1,839

5,199

4,522

15,770

9,710

12,675

1,541

4,850

14,325

4,022

2,356

2,380

10,587

6,044

7,694

11,781

16,347

9,891

14,108

6,993

20,271

6,698

16,910

30,272

5,766

6,436

6,414

5,641

9,205

16,278

7,109

1,676

4,296

16,831

6,650

14,494

7,396

8,393

6,307

4,916

3,200

8,264

84,718

491,663

131,923

195,921

215,487

194,410

225,929

131,454

24,355

68,802

58,201

202,848

122,860

159,907

19,353

60,747

178,437

49,473

28,125

28,402

123,560

70,514

88,943

136,067

187,872

109,954

153,468

75,588

217,515

70,367

174,873

309,240

58,567

65,206

64,925

56,419

90,821

158,981

68,519

15,744

40,059

155,350

61,225

133,444

67,852

76,612

57,508

44,749

29,030

74,572

20.88

16.26

15.45

14.53

14.45

14.38

14.27

14.01

13.24

13.23

12.88

12.86

12.65

12.62

12.56

12.53

12.45

12.30

11.94

11.93

11.67

11.67

11.56

11.55

11.49

11.12

10.88

10.81

10.73

10.51

10.34

10.22

10.16

10.13

10.12

10.00

9.87

9.77

9.64

9.39

9.32

9.23

9.21

9.21

9.17

9.13

9.12

9.10

9.07

9.02

292-863 0 - 91 - 10

–283–
284 . Federally Funded Research: Decisions for a Decade

Rank Institution Articles Citations Mean

51.

52.

53.

54.

55.

56.

57.

58.

59.

60.

61.

62.

63.

64.

65.

66.

67.

68.

69.

70.

71.

72.

73.

74.

75.

76.

77.

78.

79.

80.

81.

82.

83.

84.

85.

86.

87.

88.

89.

90.

91.

92.

93.

94.

95.

96.

97.

98.

99.

100.

University of Houston . . . . . . . . . . . . . . . . . . . . . . . . 2,704

Emory University . . . . . . . . . . . . . . . . . . . . . . . . . . . 4,317

Rice University . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1,683

Case Western Reserve . . . . . . . . . . . . . . . . . . . . . . . 6,471

Michigan State University. . . . . . . . . . . . . . . . . . . . . 6,669

Northwestern University. . . . . . . . . . . . . . . . . . . . . . 8,064

University of Arizona , . . . . . . . . . . . . . . . . . . . . . . . 8,550

University of South Alabama . . . . . . . . . . . . . . . . . . 964

Georgetown University ., . . . . . . . . . . . . . . . . . . . . . 3,000

St. Louis University . . . . . . . . . . . . . . . . . . . . . . . . . . 2,004

University of Miami. . . . . . . . . . . . . . . . . . . . . . . . . . 4,644

University of Iowa . . . . . . . . . . . . . . . . . . . . . . . . . . . 8,469

Virginia Commonwealth University . . . . . . . . . . . . . 4,003

Tulane University . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2,850

University of Hawaii . . . . . . . . . . . . . . . . . . . . . . . . . 3,355

University of Maryland . . . . . . . . . . . . . . . . . . . . . . . 10,730

Florida State University. , . . . . . . . . . . . . . . . . . . . . . 2,175

Temple University . . . . . . . . . . . . . . . . . . . . . . . . . . 3,110

George Washington University . . . . . . . . . . . . . . . . 2,676

Creighton University. . . . . . . . . . . . . . . . . . . . . . . . . 571

City University of New York . . . . . . . . . . . . . . . . . . . 7,279

University of Vermont . . . . . . . . . . . . . . . . . . . . . . . . 2,064

University of lllinois . . . . . . . . . . . . . . . . . . . . . . . . . . 18,096

Pennsylvania State University. . . . . . . . . . . . . . . . . 8,257

College of William and Mary. . . . . . . . . . . . . . . . . . . 558

Syracuse University . . . . . . . . . . . . . . . . . . . . . . . . . 1,606

Carnegie-Mellon University . . . . . . . . . . . . . . . . . . . 2,497

Brigham Young University . . . . . . . . . . . . . . . . . . . . 1,052

Wayne State University , . . . . . . . . . . . . . . . . . . . . . 4,084

SUNY at Buffalo . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5,119

University of Cincinnati . . . . . . . . . . . . . . . . . . . . . . 5,282

Purdue University . . . . . . . . . . . . . . . . . . . . . . . . . . . 7,878

University of New Hampshire . . . . . . . . . . . . . . . . . . 954

University of Kansas . . . . . . . . . . . . . . . . . . . . . . . . 4,135

University of Rhode Island. . . . . . . . . . . . . . . . . . . . 1,718

University of California, Davis . . . . . . . . . . . . . . . . . 11,024

University of Notre Dame . . . . . . . . . . . . . . . . . . . . . 1,860

University of New Mexico . . . . . . . . . . . . . . . . . . . . . 3,079

University of Tennessee . . . . . . . . . . . . . . . . . . . . . . 5,747

University of Oklahoma. . . . . . . . . . . . . . . . . . . . . . . 3,145

University of Delaware . . . . . . . . . . . . . . . . . . . . . . . 2,592

Howard University . . . . . . . . . . . . . . . . . . . . . . . . . . 941

Colorado State University. . . . . . . . . . . . . . . . . . . . . 4,003

Rutgers State University. . . . . . . . . . . . . . . . . . . . . . 4,876

University of Alaska . . . . . . . . . . . . . . . . . . . . . . . . . 903

Ohio State University . . . . . . . . . . . . . . . . . . . . . . . . 8,630

Georgia Institute of Technology . . . . . . . . . . . . . . . . 1,919

University of Georgia. . . . . . . . . . . . . . . . . . . . . . . . . 5,137

Oregon State University. . . . . . . . . . . . . . . . . . . . . . 4,087

Iowa State University . . . . . . . . . . . . . . . . . . . . . . . . 4,900

24,393

38,916

15,134

57,943

59,700

71,841

75,367

8,453

26,134

17,396

40,197

73,340

34,616

23,996

28,730

91,124

18,422

26,258

22,544

4,782

60,503

17,103

149,643

67,238

4,529

12,941

20,107

8,441

32,485

40,417

41,688

61,892

7,459

31,903

13,178

83,587

13,950

23,008

42,505

23,242

18,517

6,617

28,013

34,094

6,205

59,178

13,086

33,864

26,790

31,539

9.01

8.99

8.99

8.95

8.95

8.91

8.81

8.77

8.71

8.68

8.66

8.66

8.65

8.56

8.56

8.49

8.47

8.44

8.42

8.37

8.31

8.29

8.27

8.14

8.12

8.06

8.05

8.02

7.95

7.90

7.89

7.83

7.82

7.72

7.67

7.58

7.50

7.47

7.40

7.39

7.14

7.03

7.00

6.99

6.87

6.86

6.82

6.59

6.55

6.44

NOTE: Citation impact is for the period 1981 to 1988. The mean for all U.S. papers in this period is 9.5. Unless noted

otherwise, totals for State universities include ail campuses in the university system.

SOURCE: Henry Small, “Bibliometrics of Basic Research,” OTA contractor report, July 1990. Available through the

National Technical Information Service. See app. F.
APPENDIX F

Contractor Reports

Copies of contractor reports done for this project are available through the National Technical

Information Service (NTIS), either by mail (U.S. Department of Commerce, National Technical Information

Service, Springfield, VA 22161) or by calling NTIS directly at (703) 487-4650.

Harvey Averch, Florida International University, ‘‘Analyzing the Costs of Federal Research,’ PB 91-166629

Susan Cozzens, Rensselaer Polytechnic Institute+ “Science Indicators: Description of Prescription,” PB

91-166611

Patrick Hamlett, Cornell University, “Task Force on Science Policy: A Window on the Federal Funding and

Management of Research, ” PB 91-166603

James Savage, University of Virginia, ‘‘Academic Earmarks and the Distribution of Federal Research Funds:

A Policy Interpretation, ” PB 91-166595

Henry Small, Institute for Scientific Information, “Bibliometrics of Basic Research,” PB 91-166579

John Sommer, Political Economy Research Institute, ‘‘The Research Community Looks at Federal Funding

Issues: Results of Sigma Xi Survey s,” PB 91-166587

Rhetorical Analysis of Science Policy Literature, 1960-1990, PB 91-166637

1. David Birdsell, Baruch College, CUNY, and Herbert Simons, Temple University, “Basic Research

Goals: A Comparison of Political Ideologies”

2. Mark Pollock Temple University, “Basic Research Goals: Perceptions of Key Political Figures”

Project Selection and Research Evaluation Around the World Since 1985, PB 91-166645

1. Ron Johnston, Techmonitor Ltd., Wollongong, Australia, “Project Selection Methods: International

Comparisons”

2. Harvey Averch, Florida International University, ‘‘Policy Uses of ‘Evaluation of Research’ Literature’

–285–
APPENDIX G

Workshop Participants and Reviewers and Contributors

Costs of Research and Federal Decisionmaking Workshop, July 12, 1990

Rodney Nichols, Workshop Chair

Executive Vice President

Rockefeller University

Harvey Averch

Acting Director

Department of Public Administration

Florida International University

North Miami, FL

Essex Finney

Director of Beltsville Area

Agricultural Research Center

U.S. Department of Agriculture

Beltsville, MD

Howard Gobstein

Government Relations Officer

Research and Technology Transfer

University of Michigan

Ann Arbor, MI

Bernadine Healy

Chairman

Cleveland Clinic Foundation

Cleveland, OH

S. Allen Heininger

Vice President for Research

Monsanto Co.

Clayton, MO

Ruth L. Kirschstein

Director

National Institute of General Medical Sciences

National Institutes of Health

Bethesda, MD

William Massy

Vice President for Finance

Stanford University

Stanford, CA

Harold Orlans

Consultant

Chevy Chase, MD

Murray Schulman

Executive Assistant Director

Office of Energy Research

U.S. Department of Energy

Washington, DC

Robert Sproull

Professor

Department of Physics

University of Rochester

Rochester, NY

Amy Walton

Manager

Science Data Analysis and Computing Systems

Jet Propulsion Laboratory

Pasadena, CA

F. Karl Willenbrock

Assistant Director

Scientific, Technological and International Affairs

National Science Foundation

Washington, DC

Ron Konkel

Special Assistant to the Associate

Administrator of Space, Science and Applications

National Aeronautics and Space Administration

Washington, DC

–286-
Appendix G—Workshop Participants and Reviewers and Contributors ● 287

Reviewers and Contributors

Michael Davey

Congressional Research Service

Robert Abel

National Science Foundation

Karl Erb

Office of Science and Technology Policy

Joseph Alexander

National Aeronautics and Space Administration

Charles Falk

Consultant

John Alic

Office of Technology Assessment

Alan Fechter

National Research Council

Harvey Averch

Florida International University

Michael Feuer

Office of Technology Assessment

Miriam Baltuck

National Aeronautics and Space Administration

Gerald Garvey

Los Alamos National Laboratory

Richard Barke

Georgia Institute of Technology

Robin Gaster

Office of Technology Assessment

Frederick Bentley

Stanford University

Michael Gluck

Office of Technology Assessment

Peter Blair

Office of Technology Assessment

Howard Gobstein

University of Michigan

Jennifer Bond

National Science Foundation

Barry Gold

National Academy of Sciences

Ray Bowen

National Science Foundation

Thomas Goodnight

Northwestern University

Norman Braveman

National Institutes of Health

Edward Hackett

Rensselaer Polytechnic Institute

Harvey Brooks

Harvard University

Richard Hahn

U.S. Department of Energy

Lawrence Burton

National Science Foundation

John Campbell

National Academy of Sciences

C.I. Harris

U.S. Department of Agriculture

Henry Hertzfeld

Consultant

Judith Coakley

National Science Foundation

Charles Hess

U.S. Department of Agriculture

Vary Coates

Office of Technology Assessment

Wilmot Hess

U.S. Department of Energy

Robert Correll

National Science Foundation

Christopher Hill

National Academy of Sciences

Michael Crow

Iowa State University

John Holmfeld

Committee on Science, Space, and Technology

U.S. House of Representatives

John Crowley

Association of American Universities

Joseph Danek

National Science Foundation James Jasinski

University of Illinois, Urbana

292-863 0 - 91 - 11
288 . Federally Funded Research: Decisions for a Decade

Rhea Jezer Michael Phillips

Congressional Caucus for Women’s Issues Office of Technology Assessment

William Keller Rolf Piekarz

Office of Technology Assessment National Science Foundation

Genevieve Knezo Larry Prelli

Congressional Research Service University of New Hampshire

Carlos Kruytbosch Linda Roberts

National Science Foundation Office of Technology Assessment

Marcel LaFollette Thomas F. Rogers

George Washington University Consultant

Frank Laird Robert Rosen

University of Denver National Aeronautics and Space Administration

Neal Lane David Sanchez

Rice University National Science Foundation

Joshua Lederberg Frederick Sapp

Rockefeller University Consultant

Leonard Lederman James Savage

National Science Foundation University of Virginia

Alan Leinbach Dick Schoen

Dames & Moore, Inc. National Science Foundation

Tom Lessl Jerry Sesco

University of Wisconsin, Madison U.S. Department of Agriculture

Arnold Levine Alan Shaw

Consultant Office of Technology Assessment

Loet Leydesdorff Sonja Sperlich

University of Amsterdam National Science Foundation

Hugh Loweth Guyford Stever

Consultant Carnegie Commission on Science,

Shirley Malcolm

Technology, and Government

American Association for the Jeffrey Stine

Advancement of Science Smithsonian Institution

Edwin Mansfield Linda Stuntz

University of Pennsylvania U.S. Department of Energy

James McCullough Jane Stutsman

National Science Foundation National Science Foundation

Steve Nelson Janet Sweet

American Association for the Stanford University

Advancement of Science Albert Teich

Rodney Nichols American Association for the

Carnegie Corp. of New York Advancement of Science

Linda Parker Michael Telson

National Science Foundation Committee on the Budget

Don Phillips

U.S. House of Representatives

National Academy of Sciences
Appendix G—Workshop Participants and Reviewers and Contributors . 289

Alan Title

Lockheed Missiles & Space Co.

John Townsend

National Aeronautics and Space Administration

A.D. Van Nostrand

Georgia Institute of Technology

Betty Vetter

Commission on Professionals in

Science and Technology

Alvin Weinberg

Oak Ridge National Laboratory

John White

National Science Foundation

Ray Williamson

Office of Technology Assessment

Joan Winston

Office of Technology Assessment

Joel Yudken

Rutgers University
APPENDIX H

References

“A Good Budget for Science, But Troubles Lie Ahead,” Science &

Government Report, vol. 20, No. 18, Nov. 15, 1990, pp. 1,4.

“AAU Says Talks Continue at High Level in OMB To Reopen OMB

Circular A-21,” Washington Fax, Dec. 13, 1990.

Abelson Philip, “Federal Impediments to Scientific Research” Science, vol. 251, Feb. 8, 1991, p. 605.

Abelson Philip, ‘ ‘Future of the U.S. Space Program,” Science, vol.251,

Feb. 25, 1991, p. 357.

Abir-Am, Pnina G. and Dorinda Outram (eds.), Uneasy Careers and

Intimate Lives: Women in Science, 1789-1979 (New Brunswick, NJ:

Rutgers University Press, 1987).

Abu-Laban Baha (cd.), University Research and the Future of Canada

(Ottawa Canada: University of Ottawa Press, 1988).

Adams, James D., ‘‘Fundamental Stocks of Knowledge and Productivity

Growth” Journal of Political Economy, vol. 98, No. 4, 1990, pp.

673-702.

Adamsak Phil, “A Super Year in Science,” Visions, fall 1987, pp.

19-22.

Agres, Ted, “DARPA Bets on High-Risk R&D,” Research &

Development, November 1989, pp. 39-42.

Aldhous, Peter, ‘‘University Funding Plan Collapses in Chaos, ” Nature,

vol. 348, NOV. 1, 1990, p. 3.

Aldhous, Peter, “UK Nuclear Physicists Fear SERC’s Cuts,” Nature,

vol. 349, Jan. 31, 1991, p. 357.

Alliance for Undergraduate Education The Freshman Year in Science

and Engineering: Old Problems, New Perspectives for Research

Universities (University Park PA: 1990).

American Chemical Society, Department of Government Relations and

Science Policy, Federal Funding of Academic Chemistry Research,

FY 1980-FY 1988 (Washington DC: November 1990).

Anderson Christopher, “DOE Rallies to Save U.S. Fusion Research

Programmed, ” Nature, vol. 349, Jan. 24, 1991, p. 269.

Anderson Philip W. et al., “NSF Magnet lab, ” letter, The Scientist,

vol. 4, No. 23, NOV. 26, 1990, p. 14.

“Anthropologists Opt for SEPS Doctorate,” COSSA Washington

Update, Feb. 10, 1991, p. 2.

Association of American Universities, Indirect Costs Associated With

Federal Support of Research on University Campuses: Some

Suggestions for Change (Washington, DC: December 1988).

Association of American Universities, Institutional Policies To Improve

Doctoral Education (Washington, DC: November 1990).

Association of American Universities, The PhD. Shortage: The Federal

Role (Washington DC: Jan. 11, 1990).

Atkinson, Richard C., “Supply and Demand for Scientists and

Engineers: A National Crisis in the Making,’ Science, vol. 248, Apr.

27, 1990, pp. 425-432.

Atkinson, Richard C. and William A. Blanpied, “Peer Review and the

Public Interest,” Issues in Science & Technology, vol. 2, summer

1985, pp. 101-114.

Atkinson, Richard C. and William A. Blanpied (eds.), Science,

Technology, and Government: A Crisis of Purpose, proceedings of

a symposium, University of California San Diego, March 1988 (La

Jolla CA: 1989),

Australian Science and Technology Council, Setting Directions for

Australian Research (Canberra, Australia: Australian Government

Publishing Service, June 1990).

Averch Harvey, ‘‘New Foundations for Science and Technology Policy

Analysis,” paper presented at the Conference on the Mutual

Relevance of Science Studies and Science Policy, Blacksburg, VA,

May 12, 1989.

Averch Harvey, “Policy Uses of ‘Evaluation of Research’ Literature,”

OTA contractor report July 1990,

Averch Harvey, “Analyzing the Costs of Federal Research” OTA

contractor report August 1990.

Averch Harvey, “The Political Economy of R&D Taxonomies,”

Research Policy, forthcoming 1991.

Bahcall, John “Prioritizing Scientific Initiatives,” Science, vol. 251,

Mar. 22, 1991, pp. 1412-1413.

Baldwin Elizabeth and Christopher T. Hill, “The Budget Process and

Large-Scale Science Funding,” CRS Review, vol. 9, February 1988,

pp. 13-16.

Baltimore, David, “The Worsening Climate for Biological Research”

Technology Review, vol. 92, No. 4, May-June 1989, pp. 22,24.

Barber, Elinor et al., Choosing Futures: U.S. and Foreign Student Views

of Graduate Engineering Education (New York NY: Institute of

International Education 1990).

Barinaga Marcia, “Howard Schachman Fights Retirement” Science,

vol. 249, Sept. 14, 1990, pp. 1235, 1237.

Barinaga, Marcia “Stanford Sails Into a Storm,” Science, vol. 250,

Dec. 21, 1990, p. 1651.

Barinaga, Marcia “Was Paul Biddle Too Tough on Stanford,” Science,

vol. 251, Jan. 11, 1991, p. 157.

Barinaga, Marcia “Indirect Costs: How Does Stanford Compare With

Its Peers,” Science, vol. 251, Feb. 15, 1991, pp. 734-735.

Barinaga, Marcia “John Dingell Takes On Stanford,” Science, vol.

251, Feb. 15, 1991, pp. 734-737.

Barinaga, Marcia, “The SSC Gets Its (Official) Price Tag: $8.3

Billion,” Science, vol. 251, Feb. 15, 1991, pp. 741-742.

Barre, Remi, “Strategic Processes and S&T Indicators: Toward a Key

Role in R&D Management Systems, ” The Research System in

Transition, S.E. Cozzens et al. (eds.) (Dordrecht Holland: Kluwer,

1990), pp. 227-239.

Bartlett R. V., Policy Through Impact Assessment: Institutionalized

Analysis as a Policy Strategy (New York NY: Greenwood Press,

1989).

Baughman G.W., “Impact of Inflation on Research Expenditures of

Selected Academic Disciplines: 1967-1983,” report to the National

Science Foundation and the National Center for Educational

Statistics, NSF/PLN 8017815, NOV. 8, 1985,

“BBS Task Force Meets: Separate Directorate Issue on the Table,”

COSSA Washington Update, Sept. 21, 1990, pp. 1-3.

“Behavioral Directorate for NSF Proposed in Congress,” APS Observer, vol. 3, September 1990, p. 7.

Bernthal, Frederick M., National Science Foundation, testimony at

hearings before the House Subcommittee on Science, Committee on

Science, Space, and Technology, Feb. 20, 1991.

Biedenweg, Rick and Dana Shelley, 1986-87 Decanal Indirect Cost

Study (Stanford, CA: Stanford University Press, February 1988).

“Big Science: Is It Worth the Price?” New York Times, May 27, 1990,

p. 11, and June 5, 1990, p. Cl.

Birdsell, David and Herbert Simons, “Basic Research Goals: A

Comparison of Political Ideologies,” OTA contractor report June

1990.

Bjerklie, David, “Fast-Track Grants,” Technology Review, vol. 93, No.

6, August-September 1990, pp. 19-20.

Block Erich ‘‘Education and Human Resources at the National Science

Foundation” Science, vol. 249, Aug. 24, 1990, pp. 839-840.

Blum Debra, “Younger Scientists Feel Big Pressure in Battle for

Grants,” The Chronicle of Higher Education, vol. 37, No. 4, Sept.

26, 1990, p. A16.

Blum Debra “Many Studies of Future Academic Job Market Are Said

To Be of Little Use to Policymakers,” The Chronicle of Higher

Education, vol. 37, No. 23, Feb. 20, 1991, pp. A15, A19.

Blumenstyk, Goldie, ‘‘Science Agency Picks Florida State Over MTT as

Site for $60-Million Magnet-Study Lab,’ The Chronicle of Higher

Education, vol. 37, No. 1, Sept. 5, 1990, pp. A21-A22.

Blumenthal, David et rd., “University-Industry Research Relationships

in Biotechnology: Implications for the University,” Science, vol.

–290–
Appendix H—References . 291

232, June 13, 1986, pp. 1361-1366.

Bodkin, Dinah, “Young Scientists and the Future,” letter, Science, vol.

249, Sept. 28, 1990, pp. 1485-1486.

Boesman William C., “Historical Perspective on Large U.S. Science

Facilities,” CRS Review, February 1988, pp. 8-10.

Boesmaa William C. and Christine Matthews Rose, “Equity, Excellence and the Distribution of Federal Research and Development

Funds,’ ‘ CRS Report for Congress (Washington, DC: Apr. 25,

1989).

Boffey, Phillip, The Brain Bank of America (New York, NY: McGrawHill, 1975).

Boorstin Daniel J., The Americans: The Democratic Experience (New

York NY: Vintage Books, 1974).

Booth William “Hubble Report Faults Builder, NASA,” The Washington Post, Nov. 28, 1990, p. A8.

Booth William, “Does Earth Need a Government Institute?” The

Washington Post, Dec. 10, 1990, p. A13.

Boren, Susan H., “Appropriations Enacted for Specific Colleges and

Universities by the 96th Through 99th Congresses,’ CRSReport for

Congress, CRS 89-82 EPW (Washington, DC: Feb. 6, 1989).

Borman Stu, “Human Genome Project Moving on Many Fronts,”

Chemical & Engineering News, vol. 68, No. 50, Dec. 10, 1990, pp.

6-7.

Bouchard, Constance B., ‘‘The ‘Lost Generation’ of Scholars Can Help

Colleges Avoid the Looming Faculty Shortage,” The Chronicle of

Higher Education, vol. 37, No. 12, Nov. 21, 1990, pp. B1-B2.

Bowen, William G. and Julie Ann Sosa, Prospects for Faculty in the Arts

and Sciences (Princeton, NJ: Princeton University Press, 1989).

Bozeman, Barry and Michael Crow, “The Environments of U.S. R&D

Laboratories: Political and Market Influences,” Policy Sciences, vol.

23, No. 1, 1990, pp. 25-56.

Bradburn, Norman M., “The Ranking of Universities in the United

States and Its Effect on Their Achievement,’ Minerva, vol. 26, No.

1, spring 1988, pp. 91-100.

Brakenridge, G. Robert “Evaluating Scientific Initiatives,” letter,

American Scientist, vol. 77, No. 3, May-June 1989, pp. 212-213.

Breneman David W. and Ted I.K. Youn (eds.), Academic Labor

Markets and Careers (Philadelphia, PA: The Falmer Press, 1988).

Breslow, Ronald, “Funding Science Research” letter, Chemical &

Engineering News, vol. 68, No. 51, Dec. 17, 1990, p. 2.

“British Science Indicators, ” Outlook on Science Policy, vol. 11,

November 1989, pp. 112-113.

Bromley, D. Allan “Keynote Address,” Science and Technology and

the Changing World Order, S.D. Sauer (cd.) (Washington DC:

American Association for the Advancement of Science, 1990), pp.

7-16.

Brooks, Harvey, “The Future Growth of Academic Research: Criteria

and Needs,” Science Policy and the University, Harold Orlans (cd.)

(Washington, DC: The Brookings Institution 1968), pp. 53-76.

Brooks, Harvey, “Models for Science Planning,” Public Administration Review, vol. 31, May/June 1971, pp. 364-374.

Brooks, Harvey, “Can Science Survive in the Modern Age?” Science,

vol. 174, Oct. 1, 1971, pp. 21-30.

Brooks, Harvey, “Knowledge and Action: The Dilemma of Science

Policy in the ‘70s,” Daedalus, vol. 102, No. 2, spring 1973, pp.

125-143.

Brooks, Harvey, “Are Scientists Obsolete?” Science, vol. 186, Nov. 8,

1974, pp. 501-508.

Brooks, Harvey, “The Problem of Research Priorities, ” Daedalus, vol.

107, No. 2, spring 1978, pp. 171-190.

Brooks, Harvey, “Can Science Survive in the Modem Age? A Revisit

After Twenty Years,’ National Forum, vol. 71, No. 4, fall 1990, pp.

31-33.

Burek Deborah M. et al. (eds,), Encyclopedia of Associations (Detroit,

MI: Gale Research, Inc., 1989).

Busch Lawrence and William B. Lacy, Science, Agriculture and the

Politics of Research (Boulder, CO: Westview Press, 1983).

Bylinsky, Gene, “America’s Hot Young Scientists,” Fortune, vol. 122,

No. 9, Oct. 8, 1990, pp. 56-69.

Carey, William D., ‘‘Scientists and Sandboxes: Regions of the Mind,’

American Scientist, vol. 76, March-April 1988, pp. 143-145.

Carey, William D., “R&D in the Federal Budget: 1976-1990,” Science

and Technology and the Changing World Order, S.D. Sauer (cd.)

(Washington, DC: American Association for the Advancement of

Science, 1990), pp. 43-51.

Carpenter, M.P. et al., “Bibliometric Profiles for British Academic

Institutions: An Experiment To Develop Research Output Indicators,” Scientometrics, vol. 14, Nos. 3-4, 1988, pp. 213-233.

Carter, Grace M., What We Know and Do Not Know About the Peer

Review System, N-1878-RC/NIH (Santa Monica, CA: RAND Corp.,

June 1982).

Cassman Marvin“Issues Behind the Drop in the NIH Award Rate,”

ASM News, vol. 56, September 1990, pp. 465-469.

Chait, Richard, “The Pro-Teaching Movement Should Try Economic

Pressures,” The Chronicle of Higher Education, vol. 36, No. 43,

July 11, 1990, p. A36.

Chalk Rosemary and Patricia Woolf, “Regulating a ‘Knowledge

Business’,’ Issues in Science & Technology, vol. 5, No. 2, winter

1988-89, pp. 33-37.

Chalmers, Johnson MITI and the Japanese Miracle (Stanford, CA:

Stanford University Press, 1982).

Chelimsky, Eleanor, “Expanding GAO’s Capabilities in Program

Evaluation” The GAO Journal, winter/spring 1990, pp. 43-52.

Cherfas, Jeremy, “University Restructuring Based on False premise?”

Science, vol. 247, Jan. 19, 1990, p. 278.

Cherfas, Jeremy, “Peer Review: Software for Hard Choices,” Science,

vol. 250, Oct. 19, 1990, pp. 367-368.

Cherfas, Jeremy, “Deficits Trip U.K. Science Funding Agencies,”

Science, vol. 250, Dec. 14, 1990, pp. 1504-1505.

Chubin, Daryl E., “Designing Research Program Evaluations: A

Science Studies Approach” Science and Public Policy, vol. 14,

April 1987, pp. 82-90.

Chubin, Daryl E., ‘‘Progress, Culture, and the Cleavage of Science From

Society,” Science, Technology and Social Progress, S.L. Goldman

(cd.) (Bethlehem PA: Lehigh University Press, 1989), pp. 177-195.

Chubin, Daryl E., “Scientific Malpractice and the Contemporary

Politics of Knowledge,” Theories of Science in Society, S.E.

Cozzens and T.F. Gieryn (eds.) (Bloomington IN: Indiana University Press, 1990), pp. 149-167.

Chubin, Daryl E., ‘‘Misinformation and the Recruitment of Students to

Science,” Bioscience, vol. 40, No. 7, July/August 1990, pp.

524-526.

Chubin, D.E. and T Connolly, ‘‘Research Trails and Science Policies:

Local and Extralocal Negotiations of Scientific Work,” Scientific

Establishments and Hierarchies, Sociology of the Sciences Yearbook vol. 6, N. Elias (cd.) (Dordrecht, Holland: D. Reidel, 1982),

pp. 293-311.

Chubin D.E. and EJ. Hackett Peerless Science: Peer Review and U.S.

Science Policy (Albany, NY: SUNY Press, 1990).

Ciba Foundation The Evaluation of Scientific Research (New York

NY: John Wiley & Sons, 1989).

Clark David L., “Departmental Responses to Fluctuation in Research

Resources,’ Research Management Review, vol. 4, spring 1990, pp.

19-34.

Clement John R.B., ‘‘Computer Science and Engineering Support in the

FY 1988 Budget," AAAS Report XII, Research & Development FY

1988, AAAS Committee on Science, Engineering, and Public

Policy, Intersociety Working Group (cd.) (Washington DC: American Association for the Advancement of Science, 1987), pp.

251-261.

Clement John R.B. and Dianne Edgar, “Computer Science and

Engineering Support in the FY 1989 Budget,” AAAS Report XIII,

Research & Development FY 1989, AAAS Committee on Science,

Engineering, and Public Policy, Intersociety Working Group (cd.)

(Washington, DC: American Association for the Advancement of

Science, 1988), pp. 260-271.

Clemmit, Marcia, “Livermore’s Purchase of Japanese Supercomputer Is

Blocked,” The Scientist, vol. 4, No. 23, Nov. 26, 1990, p. 3.
292 . Federally Funded Research: Decisions for a Decade

Cole, Jonathan R. and Stephen Cole, “The Ortega Hypothesis,”

Science, vol. 178, Oct. 27, 1972, pp. 368-372.

Cole, Jonathan R. and Stephen Cole, Peer Review in the National

Science Foundation: Phase II of a Study (Washington DC: National

Academy of Sciences, 1981).

Cole, Stephen et al., Peer Review in the National Science Foundation:

Phase I of a Study (Washington DC: National Academy of Sciences,

1978).

Commission on Professionals in Science and Technology, Measuring

National Needs for Scientists to the Year 2000, report of a workshop,

Nov. 30-Dec. 1, 1988 (Washington, DC: July 1989).

Computer Horizons, Inc., “An Assessment of the Factors Affecting

Critical Cancer Research Findings,” executive summary, NIH

Evaluation Project No. 93-304, Sept. 30, 1987.

Comroe, Julius H., Jr., RetroSpectroScope: Insights Into Medical

Discovery (Menlo Park CA: Von Gehr Press, 1977).

Comroe, Julius H., Jr,, ‘The Road From Research to New Diagnosis and

Therapy,” Science, vol. 200, May 26, 1978, pp. 931-937.

“Congress Hungry for NAS Advice,” Science, vol. 250, Dec. 7, 1990,

p. 1334.

Congressional Budget Office, “Federal Investment in Intangible Assets:

Research and Development, ” unpublished document, February

1991.

Congressional Research Service, Interagency Coordination of Federal

Scientific Research and Development: The Federal Council for

Science and Technology, report to House Committee on Science and

Technology, Subcommittee on Domestic and International Scientific Planning and Analysis, 94th Cong. (Washington DC: U.S.

Government Printing Office, July 1976).

Congressional Research Service, Science Policy Research Division

Statutory Provisions Related to Federal Research and Development,

prepared for the House Committee on Science and Technology,

Subcommittee on Domestic and International Scientific Planning

and Analysis (Washington DC: U.S. Government Printing Office,

1976).

“Consistency Versus Relevance: EMC Changes a Statistic,’ Engineering Manpower Bulletin, June 1990, p. 1.

Cooper, Kenneth J., “Administration Revises Race-Based Grant Rule,”

The Washington Post, Dec. 19, 1990, pp. Al, A8.

Cooper, Kenneth J., ‘‘Stanford President Sets Initiative on Teaching,”

The Washington Post, Mar. 3, 1991, p. A12.

Cooper, Kenneth J., “Stanford Will Try to Explain Price of Knowledge,” The Washington Post, Mar. 13, 1991, p. A15.

Cooper, Kenneth J., “Panel Looks for Liability in Stanford Billings

Case,” The Washington Post, Mar. 15, 1991, p. A21.

Cooper, Kenneth J., “Five Major Universities Face GAO Audit of

Research Bills,” The Washington Post, Mar. 16, 1991, p. A2.

Cooperative State Research Service, “Budget Submission,” hearing on

Rural Development Agriculture and Related Agencies Appropriations for 1990, before the House Subcommittee of the Committee on

Appropriations, Part 2, 1989.

Cordes, Colleen “Calls for Setting Science-Spending Priorities Are

Renewed as Supercollider Gets Go-Ahead, NSF Faces Pinch” The

Chronicle of Higher Education, vol. 35, No. 46, July 26, 1989, pp.

A19, A23.

Cordes, Colleen “Congressional Practice of Earmarking Federal Funds

for Universities Offers Both Promise and Peril, ” The Chronicle of

Higher Education, vol. 36, No. 17, Jan. 10, 1990, pp. A19, A24-A25.

Cordes, Colleen, “Tiny NSF Program Hailed as Model for Broader

Distribution of U.S. Funds,” The Chronicle of Higher Education,

vol. 36, No. 45, July 25, 1990, pp. Al, A17.

Cordes, Colleen “Big Science and Technology projects Near Important

Milestones in Face of Federal Budget Crunch and Mounting

Criticism,’ The Chronicle of Higher Education, vol. 37, No. 2, Sept.

12, 1990, pp. A23, A28.

Cordes, Colleen, “Policy Experts Ask a Heretical Question: Has

Academic Science Grown Too Big?” The Chronicle of Higher

Education, vol. 37, No. 3, Nov. 21, 1990, pp. Al, A22.

Cordes, Colleen, ‘‘Universities Fear That U.S. Will Limit Payments for

Overhead Costs Incurred by Researchers,’ The Chronicle of Higher

Education, vol. 37, No. 12, Nov. 21, 1990, pp. A19, A21.

Cordray, David S. et al., “Sharing Research Data: With Whom, When

and How Much?’ paper presented at the Public Health Workshop on

Data Management in Biomedical Research Chevy Chase, MD, Apr.

25-26, 1990.

Council for Science and Technology, Policy Committee, Prime Minister’s Office, and Committee on Guidelines for Research Evaluation

Basic View on Research Evaluation (Tokyo, Japan: 1986).

Council on Competitiveness, “National Labs Meet With DOE,”

Legislative and Policy Update, vol. 3, No. 2, Jan. 28, 1991, p. 1.

Cozzens, Susan E., “Taking the Measure of Science: A Review of

Citation Theories,” ISSK Newsletter, No. 7, 1981, pp. 16-21.

Cozzens, Susan E., “Derek Price and the Paradigm of Science Policy,”

Science, Technology, & Human Values, vol. 13, Nos. 3 and 4,

summer-autumn 1988, pp. 361-372.

Cozzens, Susan E., “Literature-Based Data in Research Evaluation: A

Manager’s Guide to Bibliometrics,” final report to the National

Science Foundation Sept. 18, 1989.

Cozzens, Susan E., “Science Indicators: Description or Prescription?”

OTA contractor report July 1990.

Cozzens, S.E. and T.F. Gieryn (eds,), Theories of Science in Society,

Bloomington IN: Indiana University Press, 1990).

Cozzens, Susan E. et al. (eds.), The Research System in Transition,

proceedings of a NATO Advanced Study Institute, 11 Ciocco, Italy,

Oct. 1-13, 1989 (Dordrecht, Holland: Kluwer, 1990).

Crawford, Mark “Domenici Bill To Broaden Labs’ Missions,” The

Energy Daily, vol. 19, No. 14, Jan. 22, 1991, pp. 1-2.

Crawford, Mark “U.S. Backing for Fusion Project Seen” Science, vol.

251, Jan. 25, 1991, p. 371.

Crawford, Mark “Researchers Protest User Fees at National Labs,”

Science, vol. 251, Mar. 1, 1991, p. 1016.

Crease, Robert P. and Nicholas P. Samios, “Managing the Unmanageable,” The Atlantic Month/y, January 1991, pp. 80-88.

Culliton Barbara J., “NIH Readies Plan for Cost Containment”

Science, vol. 250, Nov. 30, 1990, pp. 1198-1199.

Culliton Barbara J., “FASEB ‘Rejects’ IOM Study,” Science, vol. 250,

Nov. 30, 1990, p. 1199.

Culliton, Barbara J., “Biomedical Funding: The Eternal ‘Crisis,’ “

Science, vol. 250, Dec. 21, 1990, pp. 1652-1653.

Czujko, Roman et al., “Their Most Productive Years: Young Physics

Faculty in 1990,” Physics Today, February 1991, pp. 37-42.

Davey, Michael, Congressional Research Service, Bricks and Mortar:A

Summary and Analysis of Proposals to Meet Research Facilities

Needs on College Campuses, report to the House Committee on

Science, Space, and Technology, Subcommittee on Science, Research and Technology, 100th Cong. (Washington DC: U.S.

Government Printing Office, September 1987).

Davey, Michael, Congressional Research Service, “Research and

Development Funding: FY 1991,” issue brief IB90048, Nov. 13,

1990.

Davis, Bernard D., “Human Genome Project: Is ‘Big Science’ Bad for

Biology?-Yes: It Bureaucratizes, Politicizes Research,” The

Scientist, vol. 4, No. 22, Nov. 12, 1990, pp. 13, 15.

Davis, Bob, “Federal Budget Pinch May Cut Amount of ‘Pork’ to

Colleges Living Off the Fat of the Land,” Wall Street Journal, May

2, 1990, p. A18.

Davis, Bob, “NASA Management Flaws Led Agency To Overlook

Hubble Defect, Panel Finds,” Wall Street Journal, Nov. 27, 1990,

p. B4.

DeLoughry, Thomas J., ‘‘Deficit-Reduction Plan Could Tighten Budgets for Student Aid and Research” The Chronicle of Higher

Education, vol. 37, No. 10, Nov. 7, 1990, pp. A18, A28.

DeLoughry, Thomas J., “Colleges Welcome News That Rep. Ford Will

Chair House Postsecondary Education Subcommittee,’ The Chronicle of Higher Education, vol. 37, No. 12, Nov. 21, 1990, pp. A19,

A23.
Appendix H—References . 293

DeLoughry, Thomas J., “U.S. Asked to Set Student-Aid Goals for Poor

and Minority Students, ” The Chronicle of Higher Education, ”vol.

37, No. 20, Jan. 30, 1991, p. A20.

Denhardt David T, “Too Much for AIDS Research,” The Washington

Post, Oct. 2, 1990, p. A19.

Department of Trade and Industry, Evaluation of R&D-A Policymaker’s Perspective (London England: Her Majesty’s Stationery

Office, 1988).

Dickson David, The New Politics of Science (New York, NY: Pantheon,

1984),

“Dingell Asks Defense Contract Audit Agency To Brief Staff on

Stanford Overhead Expenses,” Washington Fax, Dec. 10, 1990.

Dodgson M., ‘‘National Policies-Research and Technology Policy in

Australia: Legitimacy in Intervention” Science and Public Policy,

vol. 16, No. 3, June 1989, pp. 159-166.

Doniach Sebastian “Condensed Matter Theory’s Fragile Funding,”

letter, Physics Today, November 1990, pp. 13, 117.

Donnelly, John P., “Researchers Must Join Forces To Bolster Public

Confidence and Funding Support” The Scientist, vol. 4, No. 20, Oct.

15, 1990, p. 16.

Drew, David E., Science Development:An Evaluation Study, a technical

report presented to the National Board on Graduate Education

(Washington, DC: National Academy of Sciences, June 1975).

Dutton John A. and Lawson Crowe, “Setting Priorities Among

Scientific Initiatives,” Amen-can Scientist, vol. 76, NovemberDecember 1988, pp. 599-603.

Economic Report of the President Washington, DC: U.S. Government

Printing Office, 1990).

Edge, D.O., “Quantitative Measures of Communication in Science: A

Critical Review,” History of Science, vol. 17, 1979, pp. 102-134.

“Education Finance,” Stanford Observer, November-December 1990,

p. 14.

Elias, N. et al. (eds.), Scientific Establishments and Hierarchies

(Dordrecht, Holland: D. Reidel, 1982).

El-Khawas, Elaine, Campus Trends-1988 (Washington DC: American

Council on Education, 1988).

Elkana, Y. et al., Toward a Metric of Science (New York NY: Wiley,

1977).

Ember, Lois, “Bush’s Science Advisor Discusses Declining Value of

R&D Dollars,” Chemical & Engineering News, vol. 68, No. 17, Apr.

23, 1990, pp. 16-17.

England, J. Merton, A Patron for Pure Science: The National Science

Foundations’s Formative Years, 1945-57 (Washington, DC: National Science Foundation, 1982).

“Enhancing Research and Expanding the Human Frontier,’ Budget of

the United States Government, Fiscal Year 1992, part II (Washington DC: U.S. Government Printing Office, 1991), pp. 35-76.

Etzkowitz, Henry, “Entrepreneurial Scientists and Entrepreneurial

Universities in Academic Science,” Minerva, vol. 21, Nos. 2-3,

summer-autumn 1983, pp. 198-233.

Etzkowitz, Henry, ‘‘The Second Academic Revolution: The Role of the

Research University in Economic Development” The Research

System in Transition, S.E. Cozzens et al. (eds.) (Dordrecht, Holland:

Kluwer, 1990), pp. 109-124.

Executive Office of the President Office of Science and Technology

Policy, “A Research and Development Strategy for High Performance Computing,” unpublished document Nov. 20, 1987.

Executive Office of the President Office of Science and Technology

Policy, “U.S. Technology Policy,” unpublished document Sept.

26, 1990.

“Faculty Pay and the Cost of Living,” chart, The Chronicle of Higher

Education, vol. 37, No. 17, Jan. 9, 1991, p. A15.

‘‘FAST Guide to French Government R&D,’ French Advances in S&T,

vol. 4, No. 1, winter 1990-1991, pp. 3-6.

Fechter, Alan “Engineering Shortages and Shortfalls: Myths and

Realities,” The Bridge, vol. 20, fall 1990, pp. 16-20.

Ferejohn, John A., Pork Barrel Politics (Stanford CA: Stanford

University Press, 1974).

Feme, George, Science and Technology in Scandinavia (London,

England: Longman 1989).

Finn, Michael G., “Personnel Shortage in Your Future?” ResearchTechnology Management, vol. 34, No. 1, January-February 1991,

pp. 24-27.

Fiorina, Morns P., Congress: Keystone of the Washington Establishment, 2d ed. (New Haven, CT: Yale University Press, 1989).

Flamm, Kenneth Targeting the Computer (Washington, DC: The

Brookings Institution, 1987).

Ford, J., “Australia Tilts Its R&D Towards Industry,” New Scientist,

vol. 116, No. 1580, 1987, p. 19.

Franklin J. Jeffrey, “Selectivity in Funding: Evaluation of Research in

Australia” Prometheus, vol. 6, June 1988, pp. 34-60.

Friedman, Robert S. and Renee C. Friedman, “Science American Style:

Three Cases in Academe,” Policy Studies Journal, vol. 17, No. 1,

fall 1989, pp. 43-61.

Fulbright J.W., “Is the project Apollo Program To Land Astronauts on

the Moon By 1970 a Sound National Objective?” Congressional

Digest, vol. 44, February 1%5, pp. 47,49,51,53.

Fuqua, Don “Science Policy: The Evolution of Anticipation” Technology in Society, vol. 2, 1980, pp. 365-373.

“FY 1992 Budgets for Social and Behavioral Science Research”

COSSA Washington Update, vol. 10, Mar. 4, 1991, pp. 10-11.

Garfield, Eugene, Citation Indexing: Its Theory and Application in

Science, Technology and Humanities (New York, NY: John Wiley

& sons, 1979).

Garfield, Eugene, ‘‘Fast Science vs. Slow Science, Or Slow and Steady

Wins the Race,” The Scientist, vol. 4, No. 18, Sept. 17, 1990, p. 14.

Garfield, Eugene, “Assessing the Benefits of Science in Terms of

Dollars and Sense,” The Scientist, vol. 4, No. 22, Nov. 12, 1990, p.

14.

Gassman, Paul, “Future Supply and Demand In Academic Institutions,” Human Resources in Science and Technology: Improving

U.S. Competitiveness, proceedings of a Policy Symposium for

Government, Academia and Industry, March 15-16, 1990, Betty

Vetter and Eleanor Babco (eds.) (Washington DC: Commission on

professionals in Science and Technology, July 1990), pp. 29-36.

Gassman, Paul, “Shortening the Time to a Ph.D.,” Chemical &

Engineering News, vol. 68, No. 52, Dec. 24, 1990, pp. 25-26.

Gaston, Jerry, ‘The Benefits of Black Participation in Science,” Blacks,

Science and American Education, Willie Pearson, Jr. and H. Kenneth

Bechtel (eds.) (New Brunswick, NJ: Rutgers University Press,

1989), pp. 123-152.

Geiger, Roger L., “Milking the Sacred Cow: Research and the Quest for

Useful Knowledge in the American University Since 1920,”

Science, Technology, & Human Values, vol. 13, Nos. 3-4, summer

& autumn 1988, pp. 332-348.

Geiger, Roger L., “The American University and Research” The

Academic Research Enterprise Within the Industrialized Nations:

Comparative Perspectives, Government-University-Industry Research Roundtable (cd.) (Washington DC: National Academy Press,

1990), pp. 15-35.

Geller, N.L. et al., “Lifetime Citation Rates to Compare Scientists’

Work,” Social Science Research, vol. 7, 1978, pp. 345-365.

Gerstein Dean R. et al. (eds.), The Behavioral and Social Sciences:

Achievements and Opportunities (Washington, DC: National Academy Press, 1988).

Gibbons, Ann “The Salk Institute at a Crossroads,” Science, vol. 249,

July 27, 1990, pp. 360-362.

Gibbons, Michael, Organisation for Economic Cooperation and Development, Evaluation of Research in Sweden (Manchester, England:

University of Manchester Press, 1984).

Gibbons, M. and L. Georghiou, Evaluation of Research: A Selection of

Current Practices (Paris, France: Organisation for Economic

Cooperation and Development, 1987).

Gibbons, M. and A. Udgaonkar (eds.), Science and Technology Policy

in the 1980s (Manchester, England: Manchester University Press,

1985).
294 . Federally Funded Research: Decisions for a Decade

Gilbert, Walter, “Towards a Paradigm Shift in Biology,” Nature, vol.

349, Jan. 10, 1991, p. 99.

Ginzberg, Eli, “Scientific and Engineering Personnel: Lessons and

Policy Directions,” The Impact of Defense Spending in Nondefense

Engineering Labor Markets, National Research Council, Office of

Scientific and Engineering Personnel (cd.) (Washington, DC:

National Academy Press, 1986), pp. 25-42.

Gladwell, Malcolm, “Are Nobel Prizes for U.S. Vestiges of ‘Golden

Age’?” The Washington Post, Nov. 16, 1990, p. A6.

Golden, William, Science and Technology Advice to the President,

Congress, and Judiciary (New York, NY: Pergamon Press, 1988).

Goldenberg, E.N., “The Three Faces of Evaluation’ Journal of Policy

Analysis and Management, vol. 2, summer 1983, pp. 515-525.

Goldsmith, Maurice, The Science Critic (London England: Routledge

& Kegan Paul, 1986).

Goldwater, Barry M., “Is the Project Apollo Program To Land

Astronauts on the Moon by 1970 a Sound National Objective?”

Congressional Digest, vol. 44, February 1%5, pp. 53,55.

Gornick, Vivian Women in Science: 100 Journeys Into the Territory,

rev. ed. (New York, NY: Simon& Schuster/Touchstone, 1990).

Gosselin, Peter G., “A Clash of Scientific Titans: Key Groups Battle

Over Funds for Medical Projects,” The Washington Post, Dec.

18/25, 1990, health section, p, 6.

“Government Inquiry,” Stanford Observer, November-December

1990, pp. 1, 13.

Government-University-Industry Research Roundtable, Science and

Technology in the Academic Enterprise: Status, Trends and Issues

(Washington, DC: National Academy Press, September 1989).

Government-University-Industry Research Roundtable, The Academic

Research Enterprise Within the Industrialized Nations: Comparative Perspectives, report of a symposium (Washington DC: National

Academy Press, March 1990).

“The Government’s Guiding Hand: An Interview With Ex-DARPA

Director Craig Fields,” Technology Review, vol. 94, FebruaryMarch 1991, pp. 3340.

Grady, Robert E., Office of Management and Budget, “Terms of

Reference Memorandum on the FY 1992 U.S. Global Change

Research Program,’ unpublished document June 18, 1990.

Grassmuck, Karen, “Some Research Universities Contemplate Sweeping Changes Ranging From Management and Tenure to Teaching

Methods,’ The Chronicle of Higher Education, vol. 37, No. 2, Sept.

12, 1990, pp. Al, A29-A31.

Grassmuck, Karen “Colleges Scramble for Money to Reduce Huge

Maintenance Backlog, Estimated to Exceed $70 Billion; New

Federal Help Seen Unlikely,’ The Chronicle of Higher Education,

vol. 37, No. 6, Oct. 10, 1990, pp. Al, A34.

Gray, Paul E., ‘‘Advantageous Liaisons,’ Issues in Science& Technology, vol. 6, No. 3, spring 1990, pp. 40-46.

Griffith, B.C. and N.C. Mullins, “Coherent Social Groups in Scientific

Change,” Science, vol. 177, Sept. 15, 1972, pp. 959-964.

Griliches, Zvi. “Patent Statistics as Economic Indicators: A Survey,”

Journal of Economic Literature, vol. 28, No. 4, December 1990, pp.

1661-1707.

Guy, K. and L. Georghiou, ‘‘Real-Time Evaluation and the Management

of Mission-Oriented Research: The Evaluation of the Alvey

Program: Aims, Achievements and LeSSOnS, ” paper presented at the

ECE Seminar on Evaluation in the Management of R&D, Mar. 3-7,

1989.

Haas, Lawrence J., “New Rules of the Game,” National Journal, vol.

22, No. 46, NOV. 17, 1990, pp. 2793-2797.

Hackett, Edward J., “Science in the Steady State: The Changing

Research University, ” OTA contractor report, 1987.

Hackett, Edward J., “Science as a Vocation in the 1990s,” Journal of

Higher Education, vol. 61, May/June 1990, pp. 241-279.

Hamilton, David P., “The SSC Takes on a Life of Its Own,” Science,

vol. 249, Aug. 17, 1990, pp. 731-732.

Hamilton, David P., “Publishing By—and For?—the Numbers, ”

Science, vol. 250, Dec. 7, 1990, pp. 1331-1332.

Hamilton, David P., “Research Papers: Who’s Uncited Now?” Science,

vol. 251, Jan. 4, 1991, p. 25.

Hamilton, David P., “NSF Off the Hook,” Science, vol. 251, Feb. 15,

1991, p. 733.

Hamilton, David P., “Space Station Shrinkage to Affect Scientific

Mission,” Science, vol. 251, Mar. 8, 1991, p. 1167.

Hamlett, Patrick “Task Force on Science Policy: A Window on the

Federal Funding and Management of Research” OTA contractor

report, October 1990.

Harr, Karl G. and Virginia C. Lopez, “The National Aeronautics and

Space Administration: Its Social Genesis, Development and Impact” Managing Innovation: The Social Dimensions of Creativity,

Innovation and Technology, S.B. Lundstedt and E.W. Colglazier

(eds.) (New York, NY: Pergamon Press, 1982), pp. 170-189.

Hileman, Bette, “Facts and Figures for Chemical R&D, ” Chemical&

Engineering News, vol. 68, Aug. 20, 1990, pp. 28-30.

Hoaglin David C. et al., Data for Decisions: Information Strategies for

Policymakers (Cambridge, MA: Abt Books, 1982).

Holden, Constance, “A Preemptive Strike for Animal Research,”

Science, vol. 244, Apr. 28, 1989, pp. 415416.

Holden, Constance, “Facilities Awards,” Science, vol. 251, Feb. 8,

1991, p. 622.

Holden, Constance, “Do We Need More Ph.D.s, or Is Fewer Really

Better?” Science, vol. 251, Mar. l, 1991, pp. 1017-1018.

HoIt, DOn, “Recapturing the Vision: The Case for Formula Funds, ” in

proceedings of the 1989 Annual Meetings of the Agricultural

Research Institute, Bethesda, MD, May 1990.

Holt, DOn Illinois Agricultural Experiment Station, “Mechanisms for

Federal Funding of Agricultural Research and Development,”

mimeo, August 1988.

Hood, Leroy E., “Human Genome Project: Is “Big Science” Bad for

Biology?-No: And Anyway, the HGP Isn’t ‘Big Science’ “ The

Scientist, vol. 4, No. 22, Nov. 12, 1990, pp. 13, 15.

“How Iowa State University Wins Millions inEarmarked Funds,” The

Chronicle of Higher Education, vol. 37, No. 24, Feb. 27, 1991, p.

A21.

Howe, Henry F. and Stephen P. Hubbell, “Progress Report on Proposed

National Institutes for the Environment” Bioscience, vol. 40, No.

8, September 1990, p. 567.

“Indirect Cost Prime Source of Increase in Dollars for RO1 Grants,”

Dec. 7, 1990, Washington Fax, p. 1.

Inman, B.R. and Daniel F. Burton, Jr., “Technology and Competitiveness: The New Policy Frontier, ’ Foreign Affairs, VOl. 69, spring

1990, pp. 116-134.

Institute of Medicine, Funding Health Sciences Research:A Strategy to

Restore Balance (Washington, DC: National Academy Press, NOV.

9, 1990).

Intersociety Working Group, AAAS Report IX-XVI: Research &

Development, FY 1985-92 (Washington, DC: American Association

for the Advancement of Science, 1984-91).

Irvine, John and Ben_ Foresight in Science: Picking the Winners

(London, England: Frances Pinter, 1984).

Irvine, John et al., investing in the Future (Worcester, England: Billings

& Sons Ltd., 1990).

Irvine, John et al., “Investing in the Future: How Much Governments

Pay for Academic Research” Physics Today, September 1990, pp.

31-38.

“Is Knowledge a Dangerous Thing?” The Economist, vol. 318, Feb. 16,

1991, pp. 21-22.

Jaschik, Scott “Scholarships Set Up for Minority Students are Called

Illegal,” The Chronicle of Higher Education, vol. 37, No. 15, Dec.

12, 1990, pp. Al, A20.

Johnson, Bruce D., National Party Platforms: Volume II, 1960-76

(Urbana, IL: University of Illinois Press, 1978).

Johnston, Ron, University of Wollongong, Australia “Project Selection

Mechanisms: International Comparisons,” OTA contractor report,

July 1990.

Kaner, Richard B. and Alan G. MacDiarmid, “Plastics That Conduct

Electricity, ” Scientific American, vol. 258, February 1988, pp.
Appendix H—References ● 295

106-111.

Kash Don E., ‘‘Crossing the Boundaries of Disciplines,’ Engineering

Education, vol. 78, No. 10, November 1988, pp. 93-98.

Katauskas, Ted, “Shuttle Science: Is it Paying Off, ” R&D Magazine,

August 1990, pp. 43-52.

Kennedy, Donald, “Government Policies and the Costs of Doing

Research,” Science, vol. 227, Feb. 1, 1985, pp. 480-484.

Klein, Julie Thompson Interdisciplinarity: History, Theory, and

Practice (Detroit, MI: Wayne State University Press, 1990).

Knezo, Genevieve J., “White House Office of Science and Technology

Policy: An Analysis, ” CRS Report for Congress, 89-689 SPR

(Washington, DC: Congressional Research Service, Nov. 20, 1989).

Knezo, Genevieve J., “Japanese Basic Research Policies, ” CRS Report

for Congress (Washington, DC: Congressional Research Service,

Aug. 1, 1990).

Knezo, Genevieve J., ‘‘Defense Basic Research Priorities: Funding and

Policy Issues, ” CRS Report for Congress (Washington, DC:

Congressional Research Service, Oct. 24, 1990).

Knezo, Genevieve J., “Science Megaprojects: Status and Funding,

February 1991,” CRS Report for Congress, 91-258 SPR (Washington DC: Congressional Research Service, Mar. 12, 1991).

Knezo, Genevieve J. and Richard E. Rowberg, “Big and Little

Science,” CRS Review, February 1988, pp. 6-8.

Kohlstedt, S.G. and M.W. Rossiter (eds.), Historical Writing on

American Science: Perspectives and Prospects (Baltimore, MD:

Johns Hopkins University Press, 1986).

Kolata, Gina, “Who’s No. 1 in Science? Footnotes Say U.S.,” The New

York Times, Feb. 12, 1991, pp. Cl, C9.

Koprowski, Gene, "OMB to Join Summit on Space Report,” Washington Technology, vol. 5, Dec. 20, 1990, p. 1.

Koshland, Daniel E., Jr., ‘‘To See Ourselves as Others See Us,’ Science,

vol. 247, Jan. 5, 1990, p. 9.

Koshland, Daniel E., Jr., “The Underside of Overhead,” Science, vol.

248, May 11, 1990, p. 645.

Kostoff, R.N., “Evaluation of Proposed and Existing Accelerated

Research Programs of the Office of Naval Research,” IEEE

Transactions on Engineering Management, vol. 35, November

1988, pp. 271-279.

Kraut Alan G., “Statement of the American Psychological Society to

the National Science Foundation’s BBS Task Force on looking to

the 21st Century,” unpublished document Nov. 29, 1990.

Kruytbosch, Carlos and Lawrence Burton ‘‘The Search for Impact

Indicators,” Knowledge: Creation, Diffusion, Utilization, vol. 9,

December 1987, pp. 168-172.

Kuntz, Phil, “Pie in the Sky: Big Science Is Ready for Blastoff,”

Congressional Quarterly Weekly Report, vol. 48, Apr. 28, 1990, pp.

1254-1260.

LaFollette, Marcel C., “Scientists and the Media: In Search of a

Healthier Symbiosis,’ The Scientist, vol. 4, No. 14, July 9,1990, pp.

13-15.

LaFollette, Marcel C., “Congressional Oversight of Science and

Technology Programs,” paper prepared for the Committee on

Science, Technology, and Congress, Carnegie Commission on

Science, Technology, and Government New York, NY, September

1990.

LaFollette, Marcel C., Making Science Our Own: Public Images of

Science 1910-1955 (Chicago, IL: University of Chicago Press,

1990).

Lane, Neal, “Educational Challenges and Opportunities,” Human

Resources in Science and Technology: Improving U.S. Competitiveness, proceedings of a Policy Symposium for Government, Academia, and Industry, Mar. 15-16, 1990, Betty Vetter and Eleanor Babco

(eds.) (Washington, DC: Commission on Professionals in Science

and Trvchnology, July 1990), pp. 92-99.

Latour, Bruno, Science in Action: How to Follow Scientists and

Engineers Through Society (Cambridge, MA: Harvard University

Press, 1987).

Latour, Bruno and Steve Woolgar, Laboratory Life: The Social

Construction of Scientific Facts (Beverly Hills, CA: Sage, 1979).

Lavakare, P.J. and J.G. Waardenburg (eds.), Science Policies in

International Perspective: The Experience of India and the Netherlands (London England: Frances Pinter, 1989).

Layzell, Daniel T., “Most Research on Higher Education is Stale,

Irrelevant and of Little Use to Policymakers, ” The Chronicle of

Higher Education, vol. 37, No. 8, Oct. 24, 1990, pp. Bl, B3.

Weatherman, Courtney, “Definition of Faculty Scholarship Must be

Expanded to Include Teaching, Carnegie Foundation Says,” The

Chronicle of Higher Education, vol. 37, No. 14, Dec. 5, 1990, pp.

Al, A16-A17.

Weatherman, Courtney, “End of Mandatory-Retirement Policies Seen

Having Little Effect on Professors,” The Chronicle of Higher

Education, vol. 37, No. 17, Jan. 9, 1991, pp. A13-A14.

Lederberg, Joshua, “Does Scientific Progress Come From Projects or

People?’ Current Contents, vol. 29, No. 48, Nov. 27, 1989, pp. 4-12.

Lederman, Leonard L., “Science and Technology Policies and Priorities: A Comparative Analysis,” Science, vol. 237, Sept. 4, 1987, pp.

1125-1133.

Lederman, L.L. et al., “Research Policies and Strategies in Six

Countries: A Comparative Analysis, ” Science and Public Policy,

vol. 13, No. 2, April 1986, pp. 67-76.

Leeper, Pepper, ‘‘NfH Intramural Program: No Radical Changes

Needed,’ NewsReport of the National Research Council, December

1988-January 1989, pp. 2-5.

Lemonick, Michael D., ‘‘Superconductors! Time, May 11, 1987, pp.

64-70,75.

Lepkowski, Wil, ‘‘Hugh Loweth Key Science Policy Official, Retires,’

Chemical & Engineering News, vol. 64, No. 29, July 21, 1986, pp.

16-18.

Levin, Sharon and Paula E. Stephan “Research Productivity Over the

Life Cycle: Evidence for Academic Scientists,” American Economic

Review, vol. 81, March 1991, pp. 114-132.

Lievrouw, Leah A., “Four Research Programs in Scientific Communication” Knowledge in Society, vol. 1, summer 1988, pp. 6-22.

Lindblom, Charles E. and David K. Cohen, Usable Knowledge: Social

Science and Social Problem Solving (New Haven, CT: Yale

University Press, 1979).

Littlejohn Marilyn J. and Christine M. Matthews, “Scientific Misconduct in Academia: Efforts to Address the Issue,” CRS Report for

Congress, 89-392 SPR (Washington DC: Congressional Research

Service, June 30, 1989).

Lollar, Cynthia, “Access to Engineering: New Project for Students and

Faculty With Disabilities,’ Science, vol. 251, Feb. 22,1991, p. 952.

Long, Janice, “Bush’s Science Advisor Discusses Declining Value of

R&D Dollars,” Chemical & Engineering News, vol. 68, No. 17, Apr.

23, 1990, pp. 16-17.

Long, Janice, “changes in Immigration Law Eyed to Avert Shortage of

U.S. Scientists,” Chemical & Engineering News, vol. 68, No. 34,

Aug. 20, 1990, pp. 19-20.

Long, Janice, “Agencies Compete Well in Science Job Market,”

Chemical & Engineering News, vol. 68, No. 39, Sept. 24, 1990, p.

23.

Long, Janice, ‘U.S. Immigration Eased for Professionals,’ Chemical &

Engineering News, vol. 68, No. 47, Nov. 19, 1990, p. 13.

Louis, Karen Seashore et al., ‘‘Entrepreneurs in Academe: An Exploration of Behaviors Among Life Scientists,” Administrative Science

Quarterly, vol. 34, March 1989, pp. 110-131.

Loweth, Hugh “Science Advising and OMB,” The Presidency and

Science Advising, vol. 3, Kenneth W. Thompson (cd.) (Lanham,

MD: University Press of America, 1987).

Lubchenco, Jane et al., Ecological Society of America, “The Sustainable Biosphere Initiative: An Ecological Research Agenda” draft

document, Oct. 30, 1990.

Lundstedt, S.B. and E.W. Colglazier (eds.), Managing Innovation: The

Social Dimensions of Creativity, Innovation and Technology (New

York NY: Pergamon Press, 1982).

Lynn, L.E. (cd.), Knowledge and Policy: The Uncertain Connection

(Washington, DC: National Academy of Sciences, 1978).
296 . federally Funded Research: Decisions for a Decade

Maddox, Anthony B. and Renee P. Smith-Maddox, “Developing

Graduate School Awareness for Engineering and Science: A

Model,” Journal of Negro Education, vol. 59, 1990, pp. 479-490.

“Man-in-Space: The Self-Inflicted Curse of NASA,” Science &

Government Report, vol. 20, No. 13, Aug. 1, 1990, p. 4.

Mansfield, Edwin, “The Social Rate of Return From Academic

Research,” Research Policy, forthcoming 1991.

Mansfield, Edwin et al., “Social and Private Rates of Return From

Industrial Innovations,” Review of Economics and Statistics, vol.

61, 1979, pp. 49-52.

Marcus, Ruth, “New Scholarship Ruling Caught in Legal Cross-Fire,”

The Washington Post, Dec. 18, 1990, p. A8.

Marsa, Linda, “Howard Hughes Medical Institute Enriches Undergrad

Science Studies,” The Scientist, vol. 5, No. 1, Jan. 7, 1991, pp.

28-29.

Marshall, Eliot “Data Sharing: A Declining Ethic?” Science, vol. 248,

May 25, 1990, pp. 952,954-955,957.

Marshall, Eliot “Two Thumbs Down for Space Station,” Science, vol.

251, Mar. 22, 1991, p. 1421.

Marshall, Eliot and David P. Hamilton, “A Glut of Academic Pork”

Science, vol. 250, Nov. 23, 1990, pp. 1072-1073.

Martello, Angela, “Governments Led in Funding 1989-90 ‘Hot Papers’

Research,” The Scientist, vol. 4, No. 16, Aug. 20, 1990, pp. 20,23.

Martin, B.R. and J. Irvine, Research Foresight: Creating the Future

(London England: Frances Pinter, 1989).

Massy, William F., “Capital Investment for the Future of Biomedical

Research: A University Chief Financial Officer’s View,” Academic

Medicine, vol. 64, August 1989, pp. 433-437.

McCullough James, “First Comprehensive Survey of NSF Applicants

Focuses on Their Concerns About Proposal Review,” Science,

Technology, & Human Values, vol. 14, No. 1, winter 1989, pp.

78-88.

McCullough, James, Program Evaluation Staff, National Science

Foundation “Responses to Bulletin Board Message About QuickResponse, Non-Reviewed Grants, ” mimeo, Mar. 10, 1990.

McDonald, Kim A., “Panel Urges Increased Support for Superconductivity, Recommends Specific Goals for Research in Field, ” The

Chronicle of Higher Education, vol. 36, No. 48, Aug. 15, 1990, pp.

A5, A7.

McDonald, Kim A., “Researchers Increasingly Worried About the

Unreliability of Big Science Projects, ” The Chronicle of Higher

Education, vol. 36, No. 48, Aug. 15, 1990, pp. Al, A8-A9.

McDowell, George R., “Land-Grant Colleges of Agriculture: Renegotiating or Abandoning a Social Contract, ” Choices, 2d quarter 1988,

pp. 18-21.

McMillen, Liz, “Hughes Institute Awards $61 Million for Science

Education,” The Chronicle of Higher Education, vol. 36, No. 38,

May 31, 1990, pp. A19-A20.

McMillen, Liz, “A Study to Determine the Cost of Raising a Dollar

Finds That Average College Spends Just 16 Cents,” The Chronicle

of Higher Education, vol. 37, No. 1, Sept. 5, 1990, pp. A31-A32,

Merton, Robert K., “The Matthew Effect in Science, II: Cumulative

Advantage and the Symbolism of Intellectual Property,” Isis, vol.

79, No. 299, 1988, pp. 606-623.

Mervis, Jeffrey, “New Science Advisor Sees Strong Ties to Bush,

Public Support As Keys To Job,” The Scientist, vol. 3, No. 11, May

29, 1989, pp. 2-3.

Mervis, Jeffrey, “Institutions Respond in Large Numbers to Tiny

Facilities Program at NIH, NSF,” The Scientist, vol. 4, No. 8, Apr.

16, 1990, p. 2.

Mervis, Jeffrey, “When There’s Not Enough Money To Go Around,”

The Scientist, vol. 4, No. 8, Apr. 16, 1990, pp. 1, 8.

Mervis, Jeffrey, “PCAST Members Ready to Speak President Seems

Ready to Listen,” The Scientist, vol. 4, No. 10, May 14, 1990, pp.

1, 14-15.

Mervis, Jeffrey, “Science Budget: A Zero-Sum Game, ” The Scientist,

vol. 4, No. 24, Dec. 10, 1990, pp. 1, 6.

Mervis, Jeffrey, ‘‘Panel Weighs Overhaul of NSF’s Grant System,’ The

Scientist, vol. 5, No. 1, Jan. 7, 1991, pp. 1,6-7, 12.

Mervis, Jeffrey, “NIH Debates Merit of Setting Grant Minimum,” The

Scientist, vol. 5, No. 2, Jan. 21, 1991, pp. 1,4-5.

Mervis, Jeffrey, “NSF Cuts Back on Faltering Science, Technology

Centers,” The Scientist, vol. 5, No. 3, Feb. 4, 1991, pp. 1,4,24.

Mervis, Jeffrey, “Bush’s Science Budget: Will It Hold?’ The Scientist,

vol. 5, No, 5, Mar. 4, 1991, pp. 1, 6-7.

Mervis, Jeffrey, “Scientific Conflict of Interest Regulations Offer

Loophole to Small Business Program,” The Scientist, vol. 5, No. 6,

Mar. 18, 1991, pp. 1,8-9.

Miguel, Jean-Francois, “Indicators to Measure Internationalization of

Science,” unpublished paper, 1989.

Miller, Julie A., “Alexander Vows to Rescind Policy on Scholarships,’

Education Week, Feb. 13, 1991, pp. 25,28.

Miller, Neal E., The Scientist’s Responsibility for Public Information

(New York, NY: Scientists’ Institute for Public Information Media

Resource Service, 1990).

Miller, Roberta Balstad, National Science Foundation, “The Contribution of Social Research," John Madge Memorial Lecture, London

School of Economics, November 1986.

Monaghan, Peter, ‘Historians Seek More Detailed Study of Big Science

Projects to Inform Debate Among Researchers and Policy Makers,”

The Chronicle of Higher Education, vol. 37, No. 14, Dec. 5, 1990,

pp. A5, A8.

“Money for the Boffins," The Economist, vol. 318, Feb. 16, 1991, pp.

15-16.

Mooney, Carolyn J., ‘‘The Dissertation Is Still a Valuable Requirement

Survey Finds, But Graduate Students Say They Need Better Faculty

Advising,” The Chronicle of Higher Education, vol. 37, No. 18, Jan.

16, 1991, pp. A15, A22.

Morgan, Dan, “Nunn Says He’ll Investigate Some Defense Bill

Projects,” The Washington Post, Oct. 10, 1990, p. A4.

Morone, Joseph G., “Federal R&D Structure: The Need for Change,”

The Bridge, vol. 19, fall 1989, pp. 3-13.

Morns, P.W.G. and G.H. Hough, The Anatomy of Major Projects: A

Study of the Reality of Project Management (New York, NY: John

Wiley & Sons, 1987).

Morrison, David C., “Hill to NASA: Come Down,” National Journal,

vol. 22, No. 18, May 5, 1990, pp. 1077-1081.

Morrison, David C., “Pinching the Research Budget," National

Journal, vol. 22, No. 49, Dec. 8, 1990, p. 2996.

Moses, Phyllis B. and Charles E. Hess, “Getting Biotech Into the

Field,” Issues in Science & Technology, vol. 4, No. 1, fall 1987, pp.

35-41.

Mullins, Nicholas C., ‘The Structure of an Elite: The Advising Structure

of the U.S. Public Health Service,” Science Studies, vol. 2, 1972, pp.

3-29.

Narin, Francis, Evaluative Bibliometrics: The Use of Citation Analysis

in the Evaluation of Scientific Activity (Cherry Hill, NJ: Computer

Horizons, Inc., 1976).

Narin, Francis and Edith S. Whitlow, Measurement of Scientific

Cooperation and Coauthorship in CEC-Related Areas of Science,

vol. 1 (Luxembourg: Commission of the European Communities,

May 1990).

Narin, Francis et al., ‘‘Patents as Indicators of Corporate Technological

Strength,” Research Policy, vol. 16, August 1987, pp. 143-155.

“NAS Annual Meeting: Kudos From George Bush, Challenges From

Frank Press,” NewsReport of the National Research Council, vol.

40, June 1990, p. 8.

“NASA’s Citation Impact Dims in 1980s, But Voyager Missions and

JPL Shine,” Science Watch, vol. 1, No. 9, October 1990, pp. 1-2,7-8.

National Academy of Sciences, Astronomy and Astrophysics for the

1980s, report of the Astronomy Survey Committee (Washington

DC: National Academy Press, 1982).

National Academy of Sciences, The Quality of Research in Science:

Methods for Postperformance Evaluation in the National Science

Foundation (Washington, DC: National Academy Press, 1982).

National Academy of Sciences, Federal Science and Technology Budget

Priorities: New Perspectives and Procedures, a report in response to

the Conference Report on the Concurrent Resolution on the Budget
Appendix H—References ● 297

for Fiscal Year 1989 (H. Con. Res. 268) (Washington, DC: National

Academy Press, 1988).

National Academy of Sciences, The National Research Council: A

Unique Institution (Washington DC: National Academy Press,

1990).

National Academy of Sciences, A Decade of Discovery in Astronomy

and Astrophysics (Washington, DC: National Academy Press,

1991).

National Academy of Sciences, Committee on Science and Public

Policy, Basic Research and National Goals, a report to the House

Committee on Science and Astronautics (Washington DC: March

1965).

National Academy of Sciences, Committee on the Conduct of Science,

On Being a Scientist (Washington DC: National Academy Press,

1989).

National Advisory General Medical Sciences Council, “January 1990

Motion and Guidelines Regarding Funding Decisions,’ unpublished

document, Feb. 8, 1990.

National Institute of General Medical Science, “Minority Biomedical

Research Support Program,” unpublished document, April 1990.

National Institutes of Health, Extramural Trends: FY 1979-1988

(Washington, DC: June 1989)

National Institutes of Health, Academic Research Equipment and

Equipment Needs in the Biological Sciences: 1984-1987 (Washington, DC: June 1989).

National Institutes of Health, NIH Data Book: 1989 (Washington, DC:

December 1989).

National Institutes of Health, Division of Research Grants, ‘‘Briefing on

NIH FIRST Activity,’ unpublished document spring 1989.

National Institutes of Health, Grants Peer Review Study Team, Grants

PeerReview: Report to the Director, NIH Phase I (Washington, DC:

December 1976).

“National Medals Are Pinned on 30 Scientists,” The Washington Post,

Nov. 15, 1990, p. A23.

National Research Council, Postdoctoral Appointments and Disappointments (Washington DC: National Academy Press, 1981).

National Research Council, The Engineering Research Centers: Leaders in Change (Washington, DC: National Academy Press, 1987).

National Research Council, Minorities: Their Underrepresentation and

Career Differentials in Science and Engineering (Washington DC:

National Academy Press, 1987).

National Research Council, Engineering Personnel Data Needs for the

1990s (Washington DC: National Academy Press, 1988).

National Research Council, Biomedical and Behavioral Research

Scientists: Their Training and Supply (Washington DC: National

Academy Press, 1989).

National Research Council, Investing in Research: A Proposal To

Strengthen the Agricultural, Food, and Environmental System

(Washington, DC: National Academy Press, 1989).

National Research Council, On Time to the Doctorate: A Study of the

Increased Time To Complete Doctorates in Science and Engineering

(Washington, DC: National Academy Press, 1990).

National Research Council, Surveying the Nation’s Scientists and

Engineers:A Data System for the 1990s (Washington, DC: National

Academy Press, 1990).

National Research Council, Recruitment, Retention and Utilization of

Federal Scientists and Engineers (Washington, DC: National

Academy Press, 1990).

National Research Council, Computer Science and Technology Board,

The National Challenge in Computer Science and Technology

(Washington, DC: National Academy Press, 1988).

National Research Council, Office of Scientific and Engineering

Personnel, The Impact of Defense Spending on Nondefense Engineering Labor Markets (Washington, DC: National Academy Press,

1986).

National Research Council, Office of Scientific and Engineering

Personnel, Fostering Flexibility in the Engineering Work Force

(Washington, DC: National Academy Press, 1990).

National Science Board, Science& Engineering Indicators-1987, NSB

87-1 (Washington, DC: U.S. Government Printing Office, 1987).

National Science Board, Science & Engineering Indicators-1989, NSB

89-1 (Washington DC: U.S. Government Printing Office, 1989).

National Science Foundation Categories of Scientific Research (Washington, DC: 1979).

National Science Foundation, Academic Research Equipment in Selected SciencelEngineering Fields: 1982-83 to 1985$6, SRS 88-D1

(Washington, DC: June 1988).

National Science Foundation Scientific and Engineering Research

Facilities at Universities and Colleges: 1988, NSF 88-320 (Washington DC: September 1988).

National Science Foundation Report on Funding Trends and Balance

of Activities: National Science Foundation 1951-1988, special

report NSF 88-3 (Washington DC: 1988).

National Science Foundation, Profiles-Biological Sciences: Human

Resources and Funding (Washington, DC: 1989).

National Science Foundation, “Planning and Priority-Setting in the

National Science Foundation%” a report to the Committee on

Science, Space, and Technology of the U.S. House of Representatives, Feb. 28, 1990.

National Science Foundation, FY 1988 Research Proposal and Award

Activities by Predominantly Undergraduate Institutions, NSF 90-36

(Washington, DC: March 1990).

National Science Foundation, National Patterns of R&D Resources:

1990, NSF 90-316 (Washington DC: May 1990).

National Science Foundation, “Background Material for Long-Range

Planning: 1992-1996,” NSB-90-81, prepared for a meeting of the

National Science Board, June 14-15, 1990.

National Science Foundation, Report of the Merit Review Task Force,

NSF 90-113 (Washington DC: Aug. 23, 1990).

National Science Foundation Scientific and Engineering Research

Facilities at Universities and Colleges: 1990, NSF 90-318 (Washington DC: September 1990).

National Science Foundation, Selected Data on Academic Science/

Engineering R&D Expenditures, FY 1989, NSF 90-321 (Washingto% DC: October 1990).

National Science Foundation, “NSF Vital Signs: Trends in Research

Support Fiscal Years 1980-89,” Nov. 13, 1990, draft document.

National Science Foundation, Selected Data on Federal Funds for

Research and Development: Fiscal Years 1989, 1990, and 1991

(Washington, DC: December 1990).

National Science Foundation Academic Science and Engineering: R&D

Funds--Fiscal Year 1988, NSF 89-326 (Washington, DC: 1990).

National Science Foundation Federal Funds for Research and Development: Detailed Historical Tables, Fiscal Years 1955-1990 (Washington DC: 1990).

National Science Foundation, Science and Engineering Degrees:

1966-1988 Source Book, NSF 90-312 (Washington DC: 1990).

National Science Foundation, Science and Engineering Doctorates:

1960-89, NSF 90-320 (Wasbington, DC: 1990).

National Science Foundation Science and Engineering Personnel: A

National Overview, special report (Washington DC: 1990).

National Science Foundation, The State of Academic Science and

Engineering (Washington DC: 1990).

National Science Foundation Electronic Proposal Review Panels: An

Option for NSF Program Officers, (Washington, DC: forthcoming

1991).

National Science Foundation, Advisory Committee on Merit Review,

Final Report, NSF 86-93 (Washington, DC: 1986).

National Science Foundation, Committee on Physical, Mathematics

and Engineering Sciences, Grand Challenges: High Performance

Computing and Communications (Washington, DC: February

1991),

National Science Foundation, Office of Inspector General, Semiannual

Report to the Congress, No. 2 (Washington DC: Mar. 31, 1990).

National Science Foundation Program Evaluation Staff, Proposal

Review at NSF: Perceptions of Principal Investigators, NSF 88-4

(Washington, DC: February 1988).
298 . Federally Funded Research: Decisions for a Decade

Nederhof, A.J. and A.F.J. van Raan, ‘‘An International Interview Round

on the Use and Development of Science and Technology Indicators,” report to the Netherlands Ministry of Education and Sciences,

Directorate-General for Science Policy, June 1988.

Nelkin, Dorothy, “Selling Science,” Physics Today, November 1990,

p. 45.

Nelkin, Dorothy and Richard Nelson, “Commentary: UniversityIndustry Alliances,” Science, Technology, & Human Values, vol.

12, winter 1987, pp. 65-74.

Nelson, C. and D. Pollock (eds.), Communication Among Scientists and

Engineers (Lexington, MA D.C. Heath, 1970).

Nelson, Richard T., “Institutions Supporting Technical Advance in

Industry, ” American Economic Review, vol. 76, May 1986, pp.

186-189.

“New HumanGenome Centers Established,” Chemical& Engineering

News, vol. 69, No. 6, Feb. 11, 1991, p. 16.

“Newspaper Science Sections Still on the Rise,” SIP Iscope, vol. 18,

spring 1990, p. 1.

Nichols, Rodney W., “Mae West at Olympus: Five Puzzles for R&D,”

Science and Technology and the Changing World Order, S.D. Sauer

(cd.) (Washington, DC: American Association for the Advancement

of Science, 1990), pp. 53-69.

“ND-I Answers Congress With Six-Point Plan,” Washington Fax, Dec.

18, 1990, p. 1.

“NIH Crowd Seeks New Ways Out of Money Crunch,” Science &

Government Report, vol. 20, No. 13, Aug. 1, 1990, pp. 2,3.

“NIH Policy Change Could Shake Up Distribution of NIH Extramural

Funding, Official Says,’ Washington Fax, Nov. 23, 1990.

“NIH To Focus on Quality Over Numbers,” Washington Fax, Dec. 19,

1990, p. 1.

NOrman, Colin, ‘Science Budget: Growth Amid Red Ink,’ Science, vol.

251, Feb. 8, 1991, pp. 616-618.

“NSF Announces Multi-Million Dollar Grants to Form Engineering

Education Coalitions,” NSF News, Oct. 9, 1990.

“NSF Back to Normal After Budget Pause,” Chemical& Engineering

News, vol. 68, No. 48, Nov. 26, 1990, p. 12.

“NSF Task Force Discusses Upcoming Report, Agrees to Support SEPS

Directorate,” COSSA Washington Update, Jan. 28, 1991, pp. 1-2.

“NSF Task Force To Recommend New Directorate for Social and

Behavioral Sciences,” COSSA Washington Update, Dec. 7, 1990,

pp. 1-2.

“NSF, Universities Plan for Women, Minorities,” The Chronicle of

Higher Education, vol. 37, No. 1, Sept. 5, 1990, p. A2.

Oderman, Mark R., “A Viewpoint on Commercial Space Activities:

Realities and Options for the 1990s, ” Science and Technology and

the Changing World Order, S.D. Sauer (cd.) (Washington, DC:

American Association for the Advancement of Science, 1990), pp.

253-264.

Office of Management and Budget, Issues ’78 (Washington DC: U.S.

Government Printing Office, January 1977).

Ogul, Morris S. and Bert A. Rockman, “Overseeing Oversight: New

Departures and Old Problems,” Legislative Studies Quarterly, vol.

15, February 1990, pp. 5-24.

“OMB Unlikely To Open Circular A-21 for Amendment According to

Indirect Cost Observers,” Washington Fax, Dec. 13, 1990.

Orlans, Harold (cd.), Science Policy and the University (Washington,

DC: The Brookings Institution 1968).

Orlans, Harold, “Neutrality and Advocacy in Policy Research” Policy

Sciences, vol. 6, 1975, pp. 107-119.

“The Outlook in Congress for 7 Major Big Science Projects, ” The

Chronicle of Higher Education, vol. 37, No. 2, Sept. 12, 1990, p.

A28.

“The Overhead Question” letters, Science, vol. 249, July 6, 1990, pp.

10-13.

Palca, Joseph ‘‘Grants Squeeze Stirs Up Lobbyists,” Science, vol. 248,

May 18, 1990, pp. 803-804.

Palca, Joseph “NSF, NIH Apply Band-Aids,” Science, vol. 249, July

27, 1990, p. 352.

Palca, Joseph “Young Investigators at Risk” Science, vol. 249, July

27, 1990, p. 353.

Palca, Joseph, “NSF Centers Rise Above the Storm’ Science, vol. 251,

Jan. 4, 1991, pp. 19-22.

Palca, Joseph and Eliot Marshall, “Bloch Leaves NSF in Mainstream,”

Science, vol. 249, Aug. 24, 1990, p. 850.

Park, Robert L., ‘‘The Next Leader of the National Science Foundation

Must Press for a Greater U.S. Investment in Science,” The Chronicle

of Higher Education, vol. 36, No. 48, Aug. 15, 1990, p. B3.

Parker, Linda E. and David L. Clark. “Research at Liberal Arts

Colleges: Is More Really Better?’ Research Management Review,

vol. 3, spring 1990, pp. 43-55.

“Peer Review and Public Policy,” Science, Technology, & Human

Values, vol. 10, No. 3, summer 1985, pp. 3-86.

Pelz, Donald and Frank D. Andrews, Scientists in Organizations:

Productive Climates for Research and Development, rev. ed. (Ann

Arbor, MI: University of Michigan, Institute for Social Research

1976).

Pendlebury, David, “Science Leaders: Researchers lb Watch in the

Next Decade,” The Scientist, vol. 4, May 28, 1990, pp. 18-19,22-24.

Pennisi, Elizabeth, “Ecology Society Reaches Rare Consensus on

Research Agenda,’ The Scientist, vol. 4, No. 17, Sept. 3, 1990, pp.

3,9,20.

Pennisi, Elizabeth, ‘New Policies at Three Federal Agencies Lend More

Support to Outside Research,” The Scientist, vol. 5, No. 2, Jan. 21,

1991, pp. 3,5.

Pennisi, Elizabeth and Diana Morgan ” ‘Brain Decade’ Neuroscientist

Court Support” The Scientist, vol. 4, No. 21, Oct. 29, 1990, pp. 1,

8.

Perkowitz, Sidney, “Larger Machines Are Breeding Larger Research

Teams,” The Scientist, vol. 3, Oct. 16, 1989, pp. 13, 15.

Perkowitz, Sidney, “The War Science Waged,” The Washington Post,

Mar. 3, 1991, p. C2.

Phillips, Don I. and BenjaminS.P. Shen (eds.), Research in the Age of

the Steady-State University (Boulder CO: Westview Press, 1982).

“Physics Panel Sets Priorities for 1990s,” Science, vol. 248, May 11,

1990, p. 681.

Pollock, Mark, “Basic Research Goals: Perceptions of Key Political

Figures,” OTA contractor report, June 1990.

Porter, A.L. and D.E. Chubin, “An Indicator of Cross-Disciplinary

Research” Scientometrics, vol. 8, 1985, pp. 161-176.

Porter, Alan L. and Frederick A. Rossini, “Peer Review of Interdisciplinary Research Proposals,” Science, Technology, & Human Values,

vol. 10, No. 3, SUmmer 1985, pp. 33-38.

Porter, A.L. et al., “Citations and Scientific Progress: Comparing

Bibliometric Measures With Scientist Judgments,” Scientometrics,

vol. 13, 1988, pp. 103-124.

Powell, Corey S., “Universities Reach Into Pork Barrel With Help From

Friends in Congress,” Physics Today, vol. 42, No. 4, April 1990, pp.

43-45.

Press, Frank, “The Dilemma of the Golden Age,” Congressional

Record, May 26, 1988, pp. E1738-E1740.

Prewitt, Kenneth “The Public and Science Policy,” Science, Technology, & Human Values, vol. 7, No. 39, spring 1982, pp. 5-14.

Price, Derek de Solla, Science Since Babylon (New Haven CT: Yale

University Press, 1961).

Price, Derek de Solla, Little Science, Big Science (New York, NY:

Columbia University Press, 1963).

Price, Derek de Solla, “Citation Measures of Hard Science, Soft

Science, Technology and Conscience,” Communication Among

Scientists and Engineers, C. Nelson and D. Pollock (eds.) (Lexingto% MA: D.C. Health 1970), pp. 3-22.

Rabino, Isaac, “The Impact of Activist Pressures on Recombinant DNA

Research,” Science, Technology, & Human Values, vol. 16, No. 1,

winter 1991, pp. 70-87.

Report of the Advisory Committee on the Future of the U.S. Space

Program (Washington DC: U.S. Government Printing Office,

December 1990).
Appendix H—References ● 299

Rescher, Nicholas, Scientific Progress: A Philosophical Essay in the

Economics of Research in Natural Science (Pittsburg, PA: University of Pittsburgh Press, 1978).

Research Associates of Washington Higher Education Price Indexes:

1990 (Washington, DC: 1990).

Rettig, Richard A., Cancer Crusade: The Story of the National Cancer

Act of 1971 (Princeton, NJ: Princeton University Press, 1977).

Ries, Richard R. and Henry Hertzfeld, ‘‘Taxonomy of Research: Test of

Proposed Definitions on the NSF Budget,’ unpublished document,

n.d.

Roberts, Leslie, “A Meeting of the Minds on the Genome Project?”

Science, vol. 250, Nov. 9, 1990, pp. 756-757.

Roberts, Leslie, “Large-Scale Sequencing Trials Begin,” Science, vol.

250, Dec. 7, 1990, pp. 1336-1338.

Robyn, Dorothy et al., ‘‘Bringing Superconductivity to Market,” Issues

in Science& Technology, vol. 5, No. 2, winter 1988-1989, pp. 38-45.

Rosenberg, Nathan and David C. Mowery, Technology and the Pursuit

of Economic Growth (New York NY: Cambridge University Press,

1989).

Rosenberg, Nathan and L.E. Birdzell, Jr., “Science, Technology and the

Western Miracle,’ Scientific American, vol. 263, No. 5, November

1990, p. 54.

Rosenzweig, Robert M., president, Association of American Universities, ‘Address to the President’s Opening Session, The Gerontological Society of America, ” 43rd annual meeting, Boston, MA, Nov.

16, 1990.

Rosenzweig, Robert M., “The Debate Over Indirect Costa Raises

Fundamental Policy Issues,” The Chronicle of Higher Education,

vol. 37, Mar. 6, 1991, p. A40.

Rossiter, Margaret W., Women Scientists in America: Struggles and

Strategies to 1940 (Baltimore, MD: Johns Hopkins University Press,

1982).

Rossiter, Margaret W., “Science and Public Policy Since World War

II,” Historical Writing on American Science: Perspectives and

Prospects, S.G. Kohlstedt and M.W. Rossiter (eds.) (Baltimore, MD:

Johns Hopkins University Press, 1986), pp. 273-294.

Rotberg, Iris, ‘‘I Never Promised You First Place,” Delta Kappan, vol.

72, December 1990, pp. 296-303.

“Roundtable: New Challenges for the National Labs,’ Physics Today,

February 1991, pp. 24-35.

ROUSh, Wade, “Science and Technology in the 101st Congress,”

Technology Review, vol. 93, No. 8, November-December 1990, pp.

59-69.

Rowell, John M., “Superconductivity Research: A Different View,”

Physics Today, November 1988, pp. 38-46.

Rush Sean C. and Sandra L. Johnson, The Decaying American Campus:

A Ticking Time Bomb, joint report of the Association of Physical

Plant Administrators of Universities and Colleges and the National

Association of College and University Business Officers in cooperation with Coopers and Lybrand (Alexandria VA: 1989).

Sanders, Ralph and Fred R. Brown (eds.), Science and Technology: Vital

National Assets (Washington, DC: Industrial College of the Armed

Forces, 1966).

Sapolsky, Harvey, The Polaris System Development: Bureaucratic

Success in Government (Cambridge, MA: Harvard University Press,

1972).

Sauer, S.D. (cd.), Science and Technology and the Changing World

Order, colloquium proceedings, Apr. 12-13,1990 (Washington, DC:

American Association for the Advancement of Science, 1990).

Savage, James, Office of the President, University of California, “The

Distribution of Academic Earmarks in the Federal Government’s

Appropriations Bills, FY 1980-89,” mimeo, Mar. 7, 1989.

Savage, James, Office of the President University of California,

“Apparent Academic Earmarks in the FY 1990 Appropriations

Bills,” mimeo, Dec. 19, 1989.

Savage, James, University of Virginia, “Academic Earmarks and the

Distribution of Federal Research Funds: A Policy Interpretation,”

OTA contractor report, July 1990.

Sawyer, Kathy, “Truly: NASA Needs More Flexibility,’ The Washington Post, Sept. 14, 1990, p. A17.

Sawyer, Kathy, “NASA: Mission Implausible,” The Washington Post,

Nov. 4, 1990, p. C3.

Schenet, Margot A., Education and Public Welfare Division, Congressional Research Service, ‘Higher Education: Reauthorization of the

Higher Education Act” issue brief IB90028, May 15, 1990.

Schlossberg, Ken, “Earmarking by Congress Can Help Rebuild the

Country’s Research Infrastructure,” The Chronicle of Higher

Education, vol. 36, No. 19, Jan. 24, 1990, p. A48.

Schneider, Judy (updated by Carol Hardy), The Congressional Standing

Committee System-An Introductory Guide (Washington DC:

Congressional Research Service, May 1989).

Schrage, Michael, “Blurring the Line Between Funding Science and

Funding Economic Growth,” The Washington Post, Oct. 5, 1990, p.

F3.

Schriesheim, Alan, “Toward a Golden Age for Technology Transfer,”

Issues in Science& Technology, vol. 7, winter 1990-91, pp. 53-58.

Science: The End of the Frontier? a report by Leon M. Lederman,

President-Elect, to the Board of Directors of the American Association for the Advancement of Science, January 1991.

Scolnick, Edward M., “Basic Research and Its Impact on Industrial

R&D,” Research-Technology Management, November-December

1990, pp. 21-26.

“Secrecy in University-Based Research: Who Controls? Who Tells?”

Science, Technology, & Human Values, vol. 10, spring 1985, pp.

3-114.

Shapley, Deborah and Rustum Roy, Lost at the Frontier: U.S. Science

and Technology Policy Adrijt (Philadelphia PA: ISI Press, 1985).

Shapley, Willis H. et al. (eds.), Research & Development: AAAS Reports

I-VIII (Washington DC: American Association for the Advancement of Science, 1976-83).

Shinn, Teny, “Scientific Disciplines and Organizational Specificity:

The Social and Cognitive Configuration of Laboratory Activities,”

Scientific Establishments and Hierarchies, N. Elias et al. (eds.)

(Dordrecht, Holland: D. Reidel, 1982), pp. 239-264.

Shipman, Pat, “We Need to Give a Chance to Small, Unfashionable

Science,” The Chronicle of Higher Education, vol. 37, Sept. 26,

1990, p. A60.

Shoop, Tom, ‘ ‘Biology’s Moon Shot," Government Executive, vol. 23,

February 1991, pp. 10-1l, 14, 16-17.

Sigel, J.E. et al., “Allocating Resources Among Aids Research

Strategies,’ Policy Sciences, vol. 23, No. 1, February 1990, pp. 1-23.

Simen, Rolf H., “Research Landscape Requires Careful Gardeners:

Science in Unified Germany-Expert’s Opinions During VillaHugel Talks, ” German Research Service Special Science Reports,

vol. 7, January 1991, pp. 11-13.

Simon, Herbert, “Testimony at Hearings Before the U.S. Congress,

Senate Committee on Commerce, Science, and Transportation

Subcommittee on Science, Technology, and Space, ” National

Science and Technology Policy, Sept. 28-29, 1989, 101st Cong.

(Washington, DC: U.S. Government Printing Office, 1989), pp.

264-269.

Simon, Lisa, “Despite Scientist Shortage, Future Ph.D.s Fear Joblessness,” The Scientist, vol. 5, No. 1, Jan. 7, 1991, pp. 24, 32.

Simonton, Dean Keith Scientific Genius: A Psychology of Science

(Cambridge, England: Cambridge University Press, 1988).

Small, Henry, “Bibliometrics of Basic Research” OTA contractor

report, July 1990.

Small, Henry and B.C. Griffith, “The Structure of Scientific Literatures

I: Identifying and Graphing Specialties,” Science Studies, vol. 4,

1974, pp. 17-40.

Small, Henry and David Pendlebury, ‘‘Federal Support of Leading Edge

Research: Report on a Method for Identifying Innovative Areas of

Scientific Research and Their Extent of Federal Support,” OTA

contractor report, February 1989.

Small, Henry and Eugene Garfield, “The Geography of Science:

Disciplinary and National Mappings,” Journal of Information

Science, vol. 11, December 1985, pp. 147-159.
300 . Federal/y Funded Research: Decisions for a Decade

Smith, Bruce L. R., American Science Policy Since World War II

(Washington, DC: The Brookings Institution 1990).

“Social, Behavioral Sciences Seek Upgrade at NSF,” Science &

Government Report, vol. 20, No. 15, Oct. 1, 1990, p. 6.

Sommer, John W., Political Economy Research Institute, “Researcher

Perspectives on the Federal Research System,” OTA contractor

report, July 1990.

Stent, Gunther S., “Prematurity and Uniqueness in Scientific Discovery,” Scientific American, vol. 227, December 1972, pp. 84-93.

Stout, Robert L., “Evaluating Scientific Initiatives,” letter, American

Scientist, vol. 77, No. 3, May-June 1989, pp. 211-213.

Strickland, Stephen P., Politics, Science, and Dread Disease (Cambridge, MA: Harvard University Press, 1972).

Strickland, Stephen P., Research and the Health of Americans Lexingto% MA: Lexington Books, 1978).

Strickland, Stephen P., The Story of the NIH Grants Programs

MD: University Press of America, 1989).

“Strife Erupts Over Shares of Biomedical Funding,” Science &

Government Report, vol. 20, No. 19, Dec. 1, 1990, pp. 1-4.

Studer, K.E. and D.E. Chubin, The Cancer Mission: Social Contexts of

Biomedical Research (Beverly Hills, CA: Sage, 1980).

Syverson, Peter, “NRC Releases New Study on Increased Time to the

Doctorate,” CGS Communicator, vol. 23, August 1990, p. 8.

Taylor, S., Making Bureaucracies Think: The Environmental Impact

Strategy of Administrative Reform (Stanford, CA: Stanford University Press, 1984).

Teich, Albert H., “Research Centers and Non-Faculty Researchers: A

New Academic Role,” Research in the Age of the Steady-State

University, D. Phillips and B.P.S. Shen (eds.) (Boulder CO:

Westview Press, 1982), pp. 91-108.

Teich, Albert H., “Scientists and Public Officials Must Pursue

Collaboration to Set Research Priorities, ” The Scientist, vol. 4, No.

3, Feb. 5, 1990, pp. 17, 19.

Teich, AIbert H. and Kathleen M. Gramp, R&D in the 1980s: A Special

Report (Washington DC: American Association for the Advancement of Science, September 1988).

Teich, Albert et al., Congressional Action on Research and Development

in the FY 1991 Budget (Washington, DC: American Association for

the Advancement of Science, 1990).

Thomas, Eleanor C. and Leonard L. Lederman, National Science

Foundation “Indirect Costs of Federally Funded Academic Research” unpublished document, Aug. 3, 1984.

Thompson, Dick “The Growing Crisis in Medical Science,” Time,

Dec. 17, 1990, p. 21.

Thompson, Kenneth W. (cd.), The Presidency and Science Advising, vol.

3 Lanham, MD: University Press of America, 1987).

Thompson, Larry, “NIH Gets Its First Woman Director,” The

Washington Post, Mar. 26, 1991, health section, p. 8.

Tifft, Susan, “Hard Times on the Old Quad, ” Time, Oct. 29, 1990, p. 92.

Tifft, Susan “Scandal in the Laboratories,” Time, Mar. 18, 1991, pp.

7475.

“Title XIII-Budget Enforcement, ” Congressional Record-House

(Washington, DC: U.S. Government Printing Office, Oct. 26, 1990),

pp. H12743-H12744.

Tobias, Sheila, They’re Not Dumb, They’re Different: Stalking the

Second Tier (Tucson, AZ: Research Corp., 1990).

Tech, Thomas, “A Return to Teaching?” U.S. News & World Report,

Oct. 15, 1990, pp. 107-109.

Turner, Sarah E. and William G. Bowen, ‘ The Flight From ARTs and

Sciences: Trends in Degrees Conferred,’ Science, vol. 250, Oct. 26,

1990, pp. 517-521.

Turney, Jon “End of the Peer Show,” New Scientist, vol. 127, Sept. 22,

1990, pp. 38-42.

U.S. Congress, Office of Technology Assessment, Civilian Space

Stations and the U.S. Future in Space (Springfield, VA: National

Technical Information Service, November 1984).

U.S. Congress, Office of Technology Assessment, Demographic Trends

and the Scientific and Engineering Work Force, OTA-TM-SET-35

(Springfield, VA: National Technical Information Service, December 1985).

U.S. Congress, Office of Technology Assessment The Regulatory

Environment for Science, OTA-TM-SET-34 (Springfield, VA:

National Technical Information Service, February 1986).

U.S. Congress, Office of Technology Assessment Intellectual Property

Rights in an Age of Electronics and Information, OTA-CIT-302

(Washington, DC: U.S. Government Printing Office, April 1986).

U.S. Congress, Office of Technology Assessment Research Funding as

an Investment: Can We Measure the Returns? OTA-TM-SET-36

(Washington, DC: U.S. Government Printing Office, April 1986).

U.S. Congress, Office of Technology Assessment Mapping Our Genes:

Genome Projects---How Bigj How Fast? OTA-BA-373 (Washington, DC: U.S. Government Printing Office, April 1988).

U.S. Congress, Office of Technology Assessment, Educating Scientists

and Engineers: Grade School to Grad School, 0114-SET-377

(Washington, DC: U.S. Government Printing Office, June 1988).

U.S. Congress, Office of Technology Assessment U.S. Investment in

Biotechnology, OTA-BA-360 (Washington, DC: U.S. Government

Printing Office, July 1988).

U.S. Congress, Office of Technology Assessment Elementary and

Secondary Education for Science and Engineering, OTA-TM-SET41 (Washington, DC: U.S. Government Printing Office, December

1988).

U.S. Congress, Office of Technology Assessment, Higher Education for

Science and Engineering, OTA-BP-SET-52 (Washington DC: U.S.

Government Printing Office, March 1989).

U.S. Congress, Office of Technology Assessment Holding the Edge:

Maintaining the Defense Technology Base, OTA-ISC-420 (Washington, DC: U.S. Government Printing Office, April 1989).

U.S. Congress, Office of Technology Assessment Making Things

Better: Competing in Manufacturing, OTA-ITE-443 (Washington

DC: U.S. Government Printing Office, February 1990).

U.S. Congress, Office of Technology Assessment Agricultural Research and Technology Transfer Policies for the 1990s, OTA-F-447

(Washington, DC: U.S. Government Printing Office, March 1990).

U.S. Congress, Office of Technology Assessment High-Temperature

Superconductivity in Perspective, OT4-E-440 (Washington DC:

U.S. Government Printing Office, April 1990).

U.S. Congress, Office of Technology Assessment “Proposal Pressure

in the 1980s: An Indicator of Stress on the Federal Research

System” staff paper of the Science, Education and Transportation

program, April 1990.

U.S. Congress, Office of Technology Assessment, “Evaluating Defense

Department Research” background paper of the International

Security and Commerce Program, June 1990.

U.S. Congress, Office of Technology Assessment, Delivering the

Goods: Public Works Technologies and Management, OTA-SET477 (Washington, DC: U.S. Government Printing Office, April

1991).

U.S. Congress, Office of Technology Assessment, Government Policies

and Pharmaceutical Research and Development (Washington, DC:

U.S. Government Printing Office, forthcoming 1991).

U.S. Congress, House Committee on Appropriations, Departments of

Labor, Health and Human Services, and Education, and Related

Agencies Appropriation Bill, 101st Cong., report 101-591 (Washington, DC: U.S. Government Printing Office, July 12, 1990).

U.S. Congress, House Committee on Government Operations, Are

Scientific Misconduct and Conflicts of Interest Hazardous to Our

Health? 101st Cong. (Washington DC: U.S. Government Printing

Office, September 1990).

U.S. Congress, House Committee on Science and Technology, Subcommittee on Domestic and International Scientific Planning and

Analysis, Measuring and Evaluating Results of Federally Supported

Research and Development: Science Output Indicators-Part I,

Special Oversight Hearings, 94th Cong., May 19 and 26, 1976

(Washington, DC: U.S. Government Printing Office, 1976).

U.S. Congress, House Committee on Science, Space, and Technology,

Subcommittee on Investigations and Oversight, Maintaining the
Appendix H—References ● 301

lntegrity of Scientific Research, IOlst Cong., January 1990 (Washington, DC: U.S. Government Printing Office, 1990).

U.S. Congress, House Committee on Science, Space, and Technology,

Subcommittee on Energy Research and Development, Role of the

Department of Energy’s National Laboratories in Science, Engineering and Mathematics Education, IOlst Cong., June 13, 1990

(Washington, DC: U.S. Government Printing Office, 1990).

U.S. Congress, House Committee on Science, Space, and Technology,

Task Force on Science Policy, Research Project Selection, hearings,

99th Cong., Apr. 8-10, 1986 (Washington, DC: U.S. Government

Printing Office, 1986).

U.S. Congress, House Committee on Science, Space, and Technology,

Task Force on Science Policy, A History of Science Policy in the

United States, 1940-1985, 99th Cong. (Washington DC: U.S.

Government Printing Office, September 1986).

U.S. Congress, House Committee on Science, Space, and Technology,

The Hearings on Adequacy, Direction, and Priorities for the

American Science and Technology Effort, hearings, Feb. 28-Mar. 1,

1989, IOlst Cong. (Washington, DC: U.S. Government printing

Office, 1989).

U.S. Department of Agriculture, ‘‘Strategy for the ’90s for USDA Forest

Service Research,” review copy, February 1990.

U.S. Department of Agriculture, Cooperative State Research Service,

National Research Initiative Competitive Grants Program: Program Description (Washington, DC: 1990).

U.S. Department of Commerce, Bureau of Economic Analysis, Government Division Biomedical Research and Development Price Index,

report to the National Institutes of Health Mar. 30, 1990.

U.S. Department of Energy, Office of the Inspector General, Special

Report on the Department of Energy’s Superconducting Super

Collider Program, DOE/IG-0291 (Germantown, MD: Nov. 16,

1989).

U.S. General Accounting Office, Science Indicators: Improvements

Needed in Design, Construction, and Interpretation, PAD-79-35

(Washington, DC: 1979).

U.S. General Accounting Office, Biotechnology: Analysis of Federally

Funded Research (Washington, DC: U.S. Government Printing

Office, 1986).

U.S. General Accounting Office, University Funding: Assessing Federal Funding Mechanisms for University Research, GAO-RCED86-75 (Washington, DC: February 1986).

U.S. General Accounting Office, University Funding: Patterns of

Distribution of Federal Research Funds to Universities (Washington, DC: February 1987).

U.S. General Accounting Office, University Funding: Information on

the Role of Peer Review at NSF and NIH, GAO/RCED-87-87FS

(Washington, DC: March 1987).

U.S. General Accounting Office, U.S. Science and Engineering Base: A

Synthesis of Concerns About Budget and Policy Development,

GAO/RCED-87-65 (Washington, DC: March 1987).

U.S. General Accounting Office, National Science Foundation: Problems Found in Decision Process for Awarding Earthquake Center,

GAO/RCED-87-146 (Washington, DC: June 1987).

U.S. General Accounting Office, Engineering Research Centers: NSF

Program Management and Industry Sponsorship, GAOlRCED-88-

177 (Washington, DC: August 1988).

U.S. General Accounting Office, Federal Work Force:A Framework for

Studying Its Quality Over Time, GAO/PEMD-88-27 (Washington

DC: August 1988).

U.S. General Accounting Office, Budget Issues: Earmarking in the

Federal Government (Washington, DC: January 1990).

U.S. General Accounting Office, Defense Research: Information on

Selected University Research Projects, GAO/NSIAD-90-223FS

(Washington, DC: August 1990).

van Raan, A.F.J. (cd.), Handbook of Quantitative Studies of Science and

Technology (Amsterdam, The Netherlands: North-Holland, 1988).

Van Valkenburg, Mac, Turning Off Students: Our Gatekeeper

Courses,” Engineering Education, vol. 80, No. 6, September/

October 1990, p. 620.

Vetter, Betty and Eleanor Babco (eds.), Human Resources in Science

and Technology: Improving U.S. Competitiveness, proceedings of a

Policy Symposium for Government Academia and Industry, Mar.

15-16, 1990 (Washington, DC: Commission on Professionals in

Science and Technology, 1990).

Walker, Terry M., “Influencing Federal Support for Computing

Research,” Computing Research News, vol. 2, July 1990, pp. 1,

10-11.

Walsh, Eileen and Karen Bartholomew, “Indirect Costs Subject of

Three Separate Reviews,” Campus Report, Sept. 12,1990, pp. 1,6.

Warsh, David, “Will Florida Become a New Bastion of Industrial

Science?” The Washington Post, Sept. 12, 1990, p. C3.

Waterman, Alan T., “Basic Research in the United States,” Symposium

on Basic Research, Dael Wolfle (cd.) (Washington DC: American

Association for the Advancement of Science, 1959), pp. 17-40.

Watts, K.M. and J.C. Higgins, ‘‘The Use of Advanced Management

Techniques in R&D,’ Omega, vol. 15, January 1987, pp. 21-29.

“Ways and Means,’ The Chronicle of Higher Education, vol. 37, Feb.

20, 1991, p. A25.

Weinberg, A.M., Reelections on Big Science (Cambridge, MA: MIT

Press, 1966).

Weiner, Charles, “Universities, Professors, and Patents: A Continuing

Controversy,” Technology Review, vol. 89, No. 2, February-March

1986, pp. 33-43.

Weiss, Carol H., ‘Improving the Linkage Between Social Research and

Public Policy,’ Knowledge and Policy: The Uncertain Connection,

L.E. Lynn (cd.) (Washington, DC: National Academy of Sciences,

1978), pp. 23-81.

Weiss, Rick, “NIH: The Price of Neglect," Science, vol. 251, Feb. 1,

1991, pp. 508-511.

Wells, William G., School of Government and Business Admini stration

George Washington University, “Science Advice and the Presidency, 1933-76,” unpublished dissertation, 1977.

Wheeler, David L., “Biomedical Researchers Seek New Sources of Aid

for Young Scientists,’ The Chronicle of Higher Education, vol. 36,

No. 42, July 5, 1990, p. A23.

Wheeler, David L., “NH-I Plan Would Trim Length and Growth of

Research Grants, ” The Chronicle of Higher Education, vol. 37, No.

15, Dec. 12, 1990, p. A21.

Wheeler, David L., “Scientists’ Complaints Prompt Revisions in NIH

Cost-Cutting Plan,” The Chronicle of Higher Education, vol. 37,

No. 17, Jan. 9, 1991, p. A19.

White, Howard D. and Katherine W. McCain, “Bibliometrics,” Annual

Review of Information Science and Technology, vol. 23, 1989, pp.

119-186.

White, John A., “Disputing Some Claims About the NSF,” The

Chronicle of Higher Education, vol. 37, No. 3, Sept. 19, 1990, p. B4.

White, John R., “President’s Letter: Reflections on Snowbird,”

Communications of the ACM, vol. 33, September 1990, p. 19.

White, Robert M., “Science, Engineering, and the Sorcerer’s Apprentice,” presidential address to the National Academy of Engineering,

Washington, DC, Oct. 2, 1990.

Wolfle, Dael (cd.), Symposium on Basic Research (Washington, DC:

American Association for the Advancement of Science, 1959).

Working Group on Peer Review, Peer Review, a report to the Advisory

Board for the Research Councils (London England: November

1990).

Wormer, Michael D. (cd.), Congressional Quarterly’s Guide to

Congress (Washington DC: Congressional Quarterly Inc., 1982).

worthy, ward, “Research Universities Pay More Heed to Freshman

Science,” Chemical & Engineering News, vol. 68, No. 18, Apr. 30,

1990, pp. 27-28,

Worthy, Ward, “New Beckman Institute Promotes Broad-Ranging

Research Effort,” Chemical& Engineering News, vol. 68, No. 34,

Aug. 20, 1990, pp. 23-24.

Wyszomriski, Margaret Jane, “The Art and Politics of Peer Review,”

Vantage Point, spring 1990, pp. 12-13.

Yamamoto, S.S., “A Genuine Global Partnership?” Nature, vol. 346,

Aug. 23, 1990, p. 692.
302 . Federally Funded Research: Decisions for a Decade

Youn, Ted I.K., “Studies of Academic Markets and Careers: An

Historical Review,’ Academic Labor Markets and Careers, David

W. Breneman and Ted I.K. Youn (eds.) (Philadelphia, PA: The

Falmer Press, 1988), pp. 8-27.

Yudken, Joel S. and Barbara Simons, ‘‘A Report Summary-Final

Report of the Project on Funding Policy in Computer Science,”

unpublished document Oct. 15, 1989.

Ziman, John, An Introduction to Science Studies (Cambridge, England:

Cambridge University Press, 1984).

Ziman, John Knowing Everything About Nothing: Specialization and

Change in Scientific Careers (Cambridge, England: Cambridge

University Press, 1987).

Ziman, John, Restructuring Academic Science (London, England:

Science Policy Support Group, 1989).

Ziman, John, “Research as a Career,” The Research System in

Transition, S.E. Cozzens et al. (eds.) (Dordrecht, Holland: Kluwer,

1990), pp. 345-359.

Zuckerman, H. and R.B. Miller (eds.), ‘Science Indicators: Implications

for Research and Policy,” Scientometrics, vol. 2, October 1989,

special issue, pp. 32748.

Zurer, Pamela, “Biomedical Research: Calls for Shifts in Funding

Attacked,’ Chemical & Engineering News, vol. 68, No. 49, Dec. 3,

1990, p. 6.

Zurer, Pamela, “NSF Young Investigator Program May Be Slashed,”

Chemical & Engineering News, vol. 68, No. 50, Dec. 10,1990, p. 5.

Zurer, Pamela, “Research Funding: NH-I Airs Cost-Containment Plan,”

Chemical & Engineering News, vol. 68, No. 52, Dec. 24, 1990, pp.

4-5.
Index
Index

Academic research, 7-8,9-10, 11,27,28,28-36, 120, 140, 143,

172-201,263-264

agency-level data, 175, 181, 182, 185

agricultural, 114-115, 123, 271

applied research contributions, general, 49, 61

attitudes of researchers, 59-60

capital investment, 171, 189, 191

computer science, 152-153, 178-179, 181

cost accounting, 26, 172, 173, 174, 175

cost and cost effectiveness, 26, 172-201, 220-221

DOD, 61, 104, 105, 117, 120, 153,267

earmarks, 87-88, 92, 93

employment, 209, 212, 213-216, 218, 221

expenditure data, 9, 24, 54, 59, 263-265

facilities and equipment, 25-26, 171,172, 175-181,185,189,

191-193, 194,221

foreign countries, 272-282

historical perspective, 5,29,33,61,82,87,89, 90, 111, 172,

173, 182, 187, 188-191, 192, 199,200,218,278

indirect costs, 28, 172, 174-181, 184, 185, 190, 193, 195, 197

industrial models, 27,28,35,38, 196, 198-199,215,225,230

industrial applications, 49, 201

interdisciplinary approach, 35, 224, 225-226

models of, 27, 28, 33-35, 38, 219-226, 230

NIH, 181-185,214

NSF, 111, 130, 131-132, 153

OMB, 174, 195

organizational factors, 27, 28, 33-35, 38, 219-226, 272-282

post-doctoral research, 211-212,214-215

private sector spending, 177,200-201

publishing criteria, 27-28,36,38,39,66, 195-197,215,219

researcher attitudes, 241

salaries as cost, 25, 27, 176, 184-185, 188-189, 191, 192

social benefits, 49

spending data, 9, 24, 52,54, 59

State-level issues, 193

v. teaching, 33-34, 217-219

tenure, 196,214, 220, 271

young researchers, 66,219

see also graduate education and research

Accountability, 3,6,41

cost-related, 22,23,26,37-38,43-44, 172, 173, 174, 175,233

DOD, 119

DOE, 109-110

peer review, 147

Accounting, 17,26, 172, 173, 174, 175, 195,226

Advisory committees, 118

DOD, 119,267

DOE, 110-111, 119, 159

foreign system, 279,280

NASA, 108, 118

NIH, 100, 118, 128, 183-184,266

NSF, 112, 118

USDA, 113, 119

Agency issues

core v. accelerated projects, 121

cross-agency projects, 21, 27, 35,40,41,43,44,51,98, 108,

109, 117, 118, 149

cultural factors, 116, 121, 124

funding allocation, general, 124-133

.-.

historical perspectives, 51,60-62,73, 118

intramural/extramural research, 119-120

OMB interagency coordination, 78,98

portfolios, 152-153, 163, 164, 167,256

priority-setting, 143, 144, 149-156

public opinion, 54

research agencies, 97-133, 143, 144

statistical activities, 67, 240

see also management and managers; priority-setting; specific

agencies,

Agency-level statistical data (tables/graphs), 7-8, 11,97, 114

academic research, 175, 181, 182, 185

DOD, 102, 103

DOE, 109

NASA, 107

NIH, 101, 175, 181, 182, 185

NSF, 112, 175

Agricultural Research Service, 81, 113-114, 117, 127,270-271

AIDS, 73

NIH, 18, 100, 102, 139

public expectations, 52

Air Force, 103, 104, 105, 117,268

American Association for the Advancement of Science, 58-59,

77,235,236,257

American Council on Education, 236

Animal research, 177

Announcements, research opportunities, 126,127,268-269,270

Apollo program, 81, 106

Applied research, general, 61,73

academic contributions, 49, 61

defined, 3-4,238-239

DOD, 102-103, 119

DOE, 109, 111

NASA, 107

NSF, 111

spending, 4-5,52

Army, 103, 104,267

Asian Americans, 207,208

Association of American Universities, 193, 195,236

Atomic Energy Commission, 60

Attitudes

about peer review, 148

post-doctoral academia, 214

public opinion, 52,53-54,55,57,85,237

of scientists, 54, 55,57, 58-59,60, 195-196

Australia, 274,279,281

Awards and prizes

National Medals of Science/Technology, 71

Nobel, 50,55-56, 155,237

trends, 59, 237

young researchers, 65

Award Size/Duration indicator, 250

Balanced Budget Acts, 79-80,84-85

Baltimore, David, 65

Basic research, general, 73, 151, 153,248

computer science, 152, 153

defined, 34,238-239

DOD, 102-104, 121

DOE, 109

-305 –
306 ● Federally Funded Research: Decisions for a Decade

employment in, 209-210, 213

expenditures, general, 4-5, 11, 52, 54

foreign countries, 272-282

NASA, 107

NIH, 99, 119

NSF, 111, 119, 153,222

spending, 4-5, 11,52, 54

Basic Research and National Goals, 61

Behavioral and Social Science Directorate Act, 155

Behavioral sciences, 153-154

military personnel, 104, 267

see also social sciences

Bibliometrics, 38, 196, 197,226,236,237,242-247, 248,254,

281,283-284

Big science, see megaprojects

Biological sciences, non-medical, 150

Biomedical sciences, see life sciences; National Institutes of

Health

Biotechnology, 52, 130, 138,248

Black Americans

employment, 209

students, 208, 223

university earmarks, 88, 217

Block grants, 129, 230,276

Briggs, John, 183

Bromley, D. Allan, 74, 142, 143

Brooks, Harvey, 52, 147,216

Budget process, 14,21,24-25, 144, 166

congressional, 82-84, 85, 87, 89, 90, 137, 139, 143, 166-167

deficit targets, 80

discretionary spending, 78,79,84-85, 140

earmarks, 86-93, 115, 233

executive branch, 71-72, 76-81, 137-138

foreign, 164,272-282 (passim)

megaprojects, 78, 142

OMB, 71-72,74,76,77-78,82, 85,93

OSTP, 60,62,72,74,78

sequesters, 79-80

set-asides, 35-36,98, 129, 130, 133, 166

Bush Administration, 59, 71, 73, 120, 142

budget process, 78,79, 137, 138

priorities, 15, 18,76,77

space sciences, 107

Bush, Vannevar, 61

Canada, 273-274,279,281

Cancer, 52,99-100

Capital investment, 22,25,57, 161, 171, 175-181,221,249

academic research, 171, 189, 191

megaprojects, 20, 21, 160

see also facilities and equipment

Carter Administration, 73, 109

Ceramics, 50

Civil Rights Act, 216-217

Colleges and universities, see academic research; graduate

education and research; universities and colleges

Communications technologies, 38

Computers and computer science, 7,8,36-37, 152-153, 154

academic research, 152-153, 178-179, 181

basic research, 152, 153

DOD, 152, 153

evaluation tools, 253

high-performance, 15,77,78, 117, 153

historical perspective, 152-153, 178-179

Ph.D. supply/demand, 210,211,221

research agencies, 152-153, 154

Congress, 62,81-94, 140, 142, 143, 166-167

appropriations, 14, 80,84-93,98, 166, 183-184

budget process, 82-84, 137, 139, 143, 166-167

committees, general, 4, 15, 21-22,71, 82-85

coordination, 43

cost accounting, 26

earmarking, 86-93

evaluative function, 62-63

historic perspectives, 81-82,85,87,89,90

indirect cost caps, 63

legislative committees, 84-85

oversight, 86, 184-185

statistical programs, 235

taxation, 79, 80

Congressional Budget Act, 84

Congressional Budget and Impoundment Act, 86

Congressional Budget Office, 80, 85

Continuity of Support indicator, 250

Cooperative State Research Service, 114-115, 123,271

Cost and cost effectiveness, 22-28,63-64, 137, 151

academic research, 26, 172-201, 220-221

accountability, 22, 23, 26, 37-38,43-44, 172, 173, 174, 175,

195,233

animal research, 177

budgetary legislation, 61,60,77,79-80,84-85, 86, 118

competition and, 22-23, 27

definitional issues, 171-172, 173

facilities and equipment, 22,25,57, 171, 175-181,221,249

indirect, 22, 23, 24-25, 26, 28, 63, 172, 174-181, 184, 185,

193, 195, 197,239

inflation, 22,25,43, 171, 172, 173

megaprojects, 13, 18, 19, 20, 27, 37-38, 156-159, 161,

162-163, 164, 165,233,239

peer review, 147

personnel, general, 176, 181-182

salaries as factor, 22-27 (passim), 57, 172, 184-185

statistical programs and activities, 234

see also expenditures, statistics

Council of Economic Advisors, 74

Council of Graduate Schools, 236

Data collection and analysis, see statistical programs and

activities

Decisionmaking, 6,37-38,40-45,64,71

foreign systems, 272-282 (passim)

research agencies, general, 97,98

risk-taking, 21, 35, 119, 120, 121, 122, 130,268

utilization of research and, 40-41

see also management and managers; priority-setting; Project

selection

Defense Advanced Research Projects Agency (DARPA), 103,

105, 106, 117, 124, 126, 153,268

Defense research, see Department of Defense; war

deficit targets, 80

Degrees, higher education, 17,53,206

Ph.D.s, 23,28,29, 30-31,33,34, 39,64,205-213

Democratic Party, 81-82

Demography
Index ● 3 0 7

aging faculty, 187, 215, 216

disabled persons, 64,88, 217

gender differences, 206-207,208,209,210, 215,229

research teams, 28

students, 206-210, 212, 213

work force, 30-31, 36, 37, 44, 205, 233, 237, 240, 258

see also minority groups

Department of Agriculture, 7,60,61,97,98, 130,270-271

advisory committees, 113, 119

agency overview, 127

Agricultural Research Service, 81, 113-114, 117, 127, 270-

271

Cooperative State Research Service, 114-115, 123,271

earmarking, 115

environmental research, 113, 114, 115

formula funding, 129-130

history, 113-115

indirect costs, 63

managers, 270, 271

organization, 113-114, 115, 270-271

peer review, 129,270, 271

priority setting, 113-116, 117

Department of Commerce, 130

economic analyses, 56, 185

superconductivity, 51, 118

Department of Defense, 7,35,61,97,98,267-268, 282

academic research, 61, 104, 105, 117, 120, 153, 267

advisory committees, 119, 267

applied research, general, 102-103, 119

basic research, general, 102-104, 121

computer science, 152, 153

contractors, 104, 105, 267, 268

DARPA, 103, 105, 103, 105, 106, 117, 124, 126, 153,268

discretionary spending, 79

history, 73,82, 103, 104-105, 106, 152, 153

intra-agency coordination, 117, 267-268

managers, 105, 106, 126, 127, 267-268

nuclear weapons, 109, 250

organizational factors, 106, 117, 267-268, 282

peer review, 267

priority setting, 102-105, 106, 116, 117, 124

project selection, 105, 124, 126

risk-taking, 121

Strategic Defense Initiative, 73, 103, 105, 117, 157, 163,267

strategic planning, 123

superconductivity, 51, 118

war, effect on, 6, 53, 250

see specific branches of the military

Department of Energy, 35,40,60,61,97,98, 121., 269-270

accountability, 109-110

advisory committees, 110-111, 119, 159

applied research, general, 109, 111

basic research, general, 109

computer science, 153

earmarking, 89

history, 109, 110-111, 153

iterative evaluation, 110, 269

lobbying of, 81

managers, 109-110, 119, 269

mission agencies, 119

organizational factors, 269-270, 282

peer review, 127,254,256,269

priority setting, 109-111, 116, 124, 149

project selection, 110, 111

risk-taking, 119, 121

superconductivity, 51, 118

see also Superconducting Super Collider

Department of Health and Human Services, 7, 155, 190

see also National Institutes of Health

Developing countries, 104

Disabled persons, 64,88,217

Discretion, agency project managers, 121, 123, 126, 127, 128,

129, 166,267-268,270

Discretionary spending, 78,79,84-85, 106, 140, 166

Discrimination, 217

Drug abuse, 104

Earmarking, 86-93, 115,233

Earth Observing System, 108, 157, 160, 165

Earth sciences, 112, 154

see also environmental sciences

Ecological Society of America, 150

Economic cycles, 22,25,43, 171, 172, 173

Economic factors, 3

accounting, 17, 26, 172, 173, 174, 175, 195, 226

application of research, 10, 143, 272, 280

competitiveness, 81, 145, 146, 282

genome research, 160

multi-year comparisons, methodology, 55,56-57

Space Station, 20, 160

taxes, 79,80

see also capital investment; cost and cost effectiveness;

budget process; expenditures, statistics; production and

productivity; wages and salaries

Education, 3, 10,28-36,43,64, 140, 141, 142,205

data on, 37-38

elementary and secondary education, 64, 76, 138, 205, 213,

227,229,237

ethnic minorities, 28, 31-33, 44, 64, 210, 213, 216-217

geographical factors, 28

megaprojects, 21, 161

priority setting, 12, 13, 17,76,77,78

public, 53-54

see also universities and colleges

Educational research, 138

Eisenhower Administration, 74, 76, 106

Elementary and secondary education, 64,76,138,205,213,227,

229,237

Employment and unemployment, 29

academic research, 209, 212, 213-216, 218, 221

basic research, 209-210,213

blacks, 209

faculty, higher education, 209,212,213-216,218

industry-based, 209-210, 211, 213-214

Ph.D.s, 205,209-213

Ph.D. supply, 23, 28,29, 30-31,39,64,205-213

recruitment and retention, 28, 30, 60, 64, 81, 120

retirement trends, 215, 216

work force composition, 30-31,36,37,44,205,233, 237,240,

258

see also wages and salaries

Employment benefits, 172

Energy research, 73

lobbying, 81
308 ● Federally Funded Research: Decisions for a Decade

see also Department of Energy

Engineers and engineering, 7,8, 188

enrollment, 211

employment, 210

NSF, 112, 154

Entitlement programs, 79, 80

Environmental Protection Agency, 130

Environmental sciences, 7,8, 140

global issues, 15,76,77,78,85, 115, 138, 150

historical perspectives, 81, 206,207

NSF, 112

Ph.D.s, 206,207

priorities, 15,76, 150

USDA, 113, 114, 115

Equipment, see facilities and equipment

Ethnic groups, see minority groups

Evaluation issues, 3840,64,66-67, 123,252-256,257

academic research, equipment, 25-26

congressional involvement, 62-63

criteria, general, 14-16, 21, 23, 27-28, 38, 39-42, 63, 126,

139-140,143,144-146, 147,166,254,256,257, 269,274

foreign systems, 254,255,272-282 (passim)

iterative, 110, 115, 167, 269

megaprojects, 20, 21, 63, 163

methodology, general, 252-253, 257

portfolios, 152-153, 163, 164, 167,256

real time, 163

social aspects, 38, 116, 139-140, 141, 147, 151, 272

see also accountability; bibliometrics; cost and costeffectiveness; outcomes of research; peer review; utilization of research

Executive branch, Federal, 71-81

budget process, 71-72,76-81, 137-138

historical perspectives, 72-76

Science Advisor, 74-76,77,78, 80,87, 142,280

see also specific departments, agencies, and Presidential

administrations

Expedited Awards for Novel Research, 122

Expenditures, statistics, 4-5,37-38,52,56,239-240

academic research, 9, 24, 54, 59, 263-265

by agency and discipline, 7-8, 11

basic research, general, 4-5, 11,52,54

indicators, general, 252, 258

inflation and, 22,25,43, 171, 172, 173

methodology, 55, 56

student aid, 30

superconductivity, 51

Experimental Program to Stimulate Competitive Research, 33,

35, 131-132

Extramural research, 120, 184, 186

Facilities and equipment, 16, 17,38, 119, 143, 144

academic research, 25-26, 171, 172, 175-181, 185, 189,

191-193, 194,221

cost, 22, 25, 57, 171, 175-181, 221, 249

foreign countries v. US, 57

statistics on, 241-242

see also capital investment; indirect costs

Faculty, higher education

in academic research model, 27, 28, 33-35, 38, 219-226

assistantships and fellowships, 208, 214, 227

employment, 209, 212, 213-216, 218

research v. teaching, 33-34, 217-219

retirement, 187, 215, 216

tenure, 196,214, 220, 271

see also academic research

Federal Coordinating Council for Science, Engineering and

Technology (FCCSET), 21,41,74,77,78, 117,140,151

computer science, 153

Federally Funded Research and Development Centers, 104,239

Federation of American Societies for Experimental Biology,

223

Fellowships, 208, 214,227

First Independent Research Support and Transition awards, 65,

66

Fechter, Alan, 227

Ford Administration, 82

Foreign countries

bibliometrics, 247,281

budgetary processes, 164, 272-282 (passim)

evaluation efforts, 254, 255, 272-282 (passim)

organization and infrastructure, 57, 272-282

use of U.S. research, 10

see also international programs and projects; specific countries

Foreign scientists, 212

Foreign students, 29,31,205,208,210,212

Forest Service, 115-116,271

Formula funding, 129-130, 133

France, 254, 273,276-277,282

Fraud, 86

Fuqua, Don, 71

Fusion research, 110, 121

Gender differences, 229

graduate enrollment, 210

Ph.D.s, 206-207,208,209,215

see also women

General Accounting Office, 86,222,233

Genetic engineering

history, 100,248

public perceptions, 52,53

Geographical factors, see local-level action; regional development; State-level issues

Germany, Federal Republic, 254,273,275-276,281, 282

Global issues, 15,76,77,78,85, 115, 138, 150

Graduate education and research, 29, 195-196, 197

defense-related, 120

enrollment trends, 210, 211, 212, 214

Federal support, 205

megaproject benefits, 161

models, 33

Ph.D. supply, 23,28,29,30-31,39,64, 205-213

post-doctoral research, 211-212,214-215

recruitment and retention, 93, 195, 210

see also student aid

Gross National Product, 59

deflator, 55,56-57,233

Hatch Act, 113, 129

Higher Education Act, 32,217,230

High-energy physics, 149

see also Superconducting Super Collider

Hispanics, 207, 208,209
Index ● 3 0 9

Historical perspectives, 41,49,52,60-62, 114, 137, 188

academic research, 5, 29, 33, 87, 89, 90, 172, 173, 182, 187,

188-191, 192, 199,200,218,278

agency issues, general, 51, 60-62, 73, 118

biomedical sciences, 146

computer science, 152-153, 178-179

congressional budgetary role, 81-82, 85, 87, 89, 90

DOD, 73,82, 103, 104-105, 106, 152, 153

DOE, 109, 110-111, 153

environmental sciences, 81, 206, 207

executive branch, 72-76

Federal spending, 4-5,7-10,59

genetic engineering, 100,248

indirect costs, 175, 178-180, 193

NASA, 72-73, 105-107, 108, 152, 153

NIH, 175, 181, 182, 185, 186

NSF, 73, 111-113, 152-154, 175

party politics, 81-82

priority-setting, 139-140, 146, 147

Ph.D. production, 205-213

public confidence, 52,53-54,57,213

recruitment and retention, 28, 30, 60

research agencies, 97, 99-100, 101

space sciences, 72-73

statistical data, 236, 237, 238

superconductivity, 50-51, 247, 248

USDA, 113-115

women and minorities, 31

Hood, Leroy, 160

Hubble Space Telescope, 157, 158

Human Genome Project, 18,62,73,100,102,157, 159-160,163

Human resources issues, 10, 13,28-36,43,44, 143,205-230

megaprojects, 21, 166

Ph.D. SUpply, 23, 28,30-31,39, 64, 205-230

priority-setting, 13

recruitment and retention, 28, 30, 60, 64, 93, 120, 195, 210,

218

statistics, 30-31, 36-38, 44

see also demography; employment and unemployment;

faculty, higher education

India, 274,279-280, 281

Indicators, 233,249-252,258

bibliometrics, 38,196,197,226,236,237, 242-247,248,254,

281,283-284

Biomedical Research and Development Price Index, 182-185,

186

enrollment trends, 210, 211, 212, 214

funding, deflators, 55,56-57,233

Gross National Product, 55,56-57,59, 182

inflation, 22,25,43, 171, 172, 173

outcomes of research, general, 3, 23, 36, 38-40,42, 247-248,

256, 257,258

proposal/award ratio, 171, 190, 196,249-250

Science & Engineering Indicators, 36,235-236,249

utilization of research, general, 10, 12, 40-41, 247-256

see also expenditures, statistics; statistical programs and

activities;

Indirect costs, 22,23,24-25,26,63, 172, 174-175,239

academic research, 28, 172, 174-181,184, 185, 190, 193, 195,

197

defined, 24

see also facilities and equipment

Inflation, 22,25,43, 171, 172, 173

Infrastructure, see facilities and equipment

Industry-based research

academic/industry cooperation, 49, 201

academic research, modeled on, 27,28,35,38, 196,198-199,

215,225,230

as employer, 209-210, 211,213-214

expenditures, 52, 54

extramural research, 120, 184, 186

Federal relations with, 82, 119,269

space science, 107, 108

Innovative Science and Technology Office, 105

Institute of Medicine, 80, 118, 223

Institute for Scientific Information, 242-243

Instruments, see facilities and equipment

Interdisciplinary approach

academic research, 35, 224, 225-226

cross-agency projects, 21,27, 35,40,41,43,44,51,98, 108,

109, 117, 118, 149

defense research, 104-105, 106

foreign systems, 275

human genome, 160

intramural labs, 119

NSF, 122

peer review, 147, 148

statistics on, 38

Interest groups, see lobbying

International perspectives, 272-282

competition, 81, 162, 282

global issues, 15,76,77,78,85, 115, 138, 150

see also foreign countries; specific countries

International programs and projects, 163,237,247

discretionary spending, 79

environmental, 150

fusion research, 110

megaprojects, 161-162

Nobel Prize, 50,55-56, 155,237

prizes, other, 59

International Thermonuclear Experimental Reactor, 110

Iterative processes, 167

DOE, 110,269

USDA, 115

Intramural research, 119-120, 186

Japan, 255,273,277

Joint Council on Food and Agricultural Sciences, 113

Kennedy Admini “ stration, 72, 99, 106

Keyworth, George, 76,87

Killian, James, 74

Klein, JuIie, 224

Koshland, Daniel E., 53

Lederman, hen, 57, 58, 159

Legal issues

fraud, 86

patents, 59

Legislation, specific, 261-262

academic/industry cooperation, 201

Balanced Budget Acts, 79-80,84-85

Behavioral and Social Science Directorate Act, 155
370 ● Federal/y Funded Research: Decisions for a Decade

Civil Rights Act, 216-217

Congressional Budget Act, 84

Congressional Budget and Impoundment Act, 86

Hatch Act, 113, 129

Higher Education Act, 32,217, 230

Legislative Reorganization Act, 86

Military Authorization Act, 61

Merrill Land-Grant College Act, 113, 114

National Defense Education Act, 29,208

National Science and Technology Equal Opportunities Act,

217

National Science and Technology Policy, Organization, and

Priorities Act, 13

National Science Foundation Authorization Act, 176

National Superconductivity and Competitiveness Act, 51,118

Office of Science and Technology Policy Act, 74,76

Omnibus Budget Reconciliation Act, 60, 77

Omnibus Trade and Competitiveness Act, 51, 118

Science and Engineering Equal Opportunities Act, 32

Small Business Development Act, 130

Smith-Lever, Act, 129

Legislative Reorganization Act, 86

Life sciences, 6,7,8, 100

Army, 104,267

cancer, 52, 99-100

employment, 210

historical perspectives, 146

human genome, 18,62,73, 100, 102, 157, 159-160, 163

neuroscience, 151, 154

Ph.D.s, 206,207,223

political factors, 100

public expectations, 52

young rearchers, 65

see also National Institutes of Health

Little science, see small science

Local-level action, 143

Lobbying, 80-81,85-86,93, 123,149

Management and managers, 10,40,99, 117-119, 125-126

coordination, 117, 124

discretion, 121, 123, 126, 127, 128, 129, 166,267-268,270

DOD, 105, 106, 126, 127, 267-268

DOE, 109-110, 119,269

foreign systems, 272

megaprojects, 163

NASA, 107-108, 126-127,269

NIH, 117, 118,266-267

NSF, 117, 118,270

risk-taking, general, 119, 120, 121

USDA, 270,271

see also budget process

Mars, 77, 108, 157

Mass media, 53-54

Massy, William F., 171, 194

Mathematics, 7, 8

control theory, 117

education, 76, 77, 78, 140, 141, 142, 211, 237

NSF, 154

Medicine, see life sciences

Megaprojects, 63, 73

budget process, 78, 142

capital investments, general, 20, 21, 160

costs, 13, 18, 19, 20, 27, 37-38, 156-159, 161, 162-163, 164,

165,233,239

evaluation issues, 20, 21, 63, 163

political factors, 27, 157, 159, 163

priority-setting, 11, 13, 14, 18-21,27, 139, 149, 156-163

project selection, general, 157, 160-162, 165

social factors, 20, 73, 157, 160-161, 162

Methodology

of current study, 4, 97, 98, 189

evaluation, general, 252-253, 257

funding, deflators, 55,56-57,233

multi-year comparisons, 55, 56-57

see also evaluation issues; indicators; statistical programs and

activities

Military Authorization Act, 61

Military research, see defense research

Minority groups, 10, 17,227,229

earmarks, 88

educational opportunities, 28, 31-33, 44, 64, 210, 213,

216-217

NIH, 99,228-229

NSF, 113, 270

Ph.D.s, 207,208,209

set asides, 35-36, 98, 129, 228

see also specific groups

Minority Biomedical Research Support Program, 35

Models

academic research, 27, 28, 33-35, 38, 219-226, 230

foreign, 272-282

regionally based funding, 131-132

researcher supply/demand, 216

Moon missions, 77, 108, 157

Apollo, 81, 106

Merrill Land-Grant College Act, 113, 114

Multidisciplinary approach, see interdisciplinary research

National Academy of Engineering, 20-21,80, 118, 143-144

National Academy of Sciences, 27,60,61,80,97,118,142, 163,

237

energy research, 111

research training, 183

National Aeronautics and Space Adminis tration, 7, 35, 60, 61,

97,98, 140, 142, 149,268-269

advisory committees, 108, 118

agency overview, 126-127

applied research, general, 107

basic research, general, 107

computer science, 152, 153

Earth Observing System, 108, 157, 160, 165

education, 208

history, 72-73, 105-107, 108, 152, 153

Hubble Space Telescope, 157, 158

managers, 107-108, 126-127, 269

Mars missions, 77, 108, 157

Moon missions, 77,81, 106, 108, 157

priority-setting, 105-109, 116, 123

Space Shuttle, 106, 108, 163

Space Station, 20,73,77, 107, 108, 157, 158, 160, 163

strategic planning, 123

superconductivity, 51, 118

see also satellite technology

National Commission on Superconductivity, 51
Index . 311

National Critical Materials Council, 51

National Defense Education Act, 29,208

National Institute of Standards and Technology, 142, 153

National Institutes of Health, 7,60,61,85,97, 130,266-267

academic research spending, 181-185, 214

advisory committees, 100, 118, 128, 183-184, 266

AIDS, 18, 100, 102, 139

basic research, general, 99, 119

centers programs, 35

computer science, 152, 153

education, 208, 214

ethnic minorities, 99, 228-229

history, 175, 181, 182, 185, 186

indirect costs, 24-25, 175, 181, 184, 185

managers, 117, 118, 266-267

peer review, 118, 125, 126, 128

priority-setting, 99-102, 116, 118, 149

salary caps, 26, 63

statistical activities, 36,37, 174, 178, 187,229,233,238,239,

256

young researchers, 65,66

see also Human Genome Project

Nationality, see foreign scientists; foreign students

National Medals of Science/Technology, 71

National Research Council, 108, 112, 115,270

behavioral and social sciences, 155

computer science, 153

Ph.D. Supply, 210

statistical programs, 235, 257

National Science and Technology Equal Opportunities Act, 217

National Science and Technology Policy, Organization, and

Priorities Act, 13

National Science Board, 16, 112, 117, 145, 154, 236

National Science Foundation, 16, 24, 40, 61, 72, 97, 98, 110,

156, 185-187, 271

advisory committees, 112, 118

applied research, general, 111

basic research, 111, 119, 153,222

behavioral and social sciences, 155-156

centers programs, 35

competitive research, 33, 171

computer science, 152-153, 154

earmarking, 90

education, 31, 73, 113, 208, 213, 214, 222

engineering, 112, 154

evaluation criteria, 125, 131, 145, 166

historical perspectives, 111-113, 152-154, 175

indirect costs, 24-25, 175

interdisciplinary projects, 122

managers, 117, 118, 270

minorities, 113, 270

peer review, 125, 127

physics, 112, 149

priority-setting, 111-113, 116, 117, 118

project selection, general, 111, 125

publications, 28, 121

regional funding, 33, 113, 131-132, 145

salary caps, 26, 63, 176

small grants, 121, 122, 130

statistical activities, 36-37, 40, 41, 56, 102, 107, 116, 173,

174-181 (passim), 187, 213, 229, 233, 235-236, 237,

238-239, 249-252 (passim), 256,257

strategic plaing, 123

superconductivity, 51, 118

women, 113, 270

young researchers, 65,66

National Science Foundation Authorization Act, 176

National Space Council, 109

National Superconductivity and Competitiveness Act, 51, 118

Navy, 26, 103, 104-105,267-268

Netherlands, 50,254,255,273,277,281

Newspapers and magazines, 53-53

Neuroscience, 151, 154

Nixon Adm inistration, 73, 76

Nobel Prize, 50,55-56, 155,237

Nonprofit institutions, 54

see also specific institutions

Nuclear power, 60

public expectations, 52,53

weapons, 53, 109, 153, 250

Nutrition, 113

Office of Management and Budget, 14-15,40,41,60,62

academic research, 174, 195

budget process, 71-72,74,76,77-78,82, 85,93

cost-accountability, 26, 174, 195

discretionary spending, 85

economic analyses, 56

evaluation, 38

indirect costs, 174

NASA, 109

projections, 137

statistical activities, 248-249, 257

Office of Science and Technology Policy, 14, 21, 144, 151

budget process, 60,62,72,74,78

statistical activities, 40, 41, 248-249, 251, 257

superconductivity, 51, 118

see also Science Advisors, executive branch

Office of Science and Technology Policy Act, 74,76

Omnibus Budget Reconciliation Act, 60,77

Omnibus Trade and Competitiveness Act, 51, 118

Organizational factors, 12, 35

academic research, 27, 28, 33-35, 38, 219-226, 272-282

behavioral and social sciences, 153-154

bureaucracy, 120

cross-agency projects, 21, 27, 35,40,41,43,44, 51, 98, 108,

109, 117, 118, 149

DOD, 106, 117, 267-268,282

DOE, 269-270,282

foreign countries, 57,272-282

NASA, 107-108,268-269,282

NIH, 99, 100, 266-267,282

NSF, 111, 112,270, 282

Federal research system, tensions, 7

institutional capacity, 17, 21

intramural/extramural research, 119-120

megaprojects, 163

research agencies, decentralization, 97, 112, 115, 272

USDA, 113-114, 115, 270-271

see also agency issues; budget process; management and

managers

Outcomes of research, 3, 23, 36,38-40,42, 247-248,256,257,

258

see also publications; utilization of research
312 . Federally Funded Research: Decisions for a Decade

Patents, 59

Peer review, 63, 125, 126-127, 129, 146-149

attitudes about, 148

bibliometrics and, 254,256

congressional earmarks and, 87, 88

DOD, 267

DOE, 127,254,256,269

foreign systems, 276

NASA, 126-127

NIH, 118, 125, 126, 146,

NSF, 113, 118, 146

political factors, 147

research agencies, 99

risk-taking, 121, 122

USDA, 129, 270,271

Persian Gulf War, 6, 250

Personnel

costs, 176, 181-182

148,266

military, behavioral studies, 104

see also engineers and engineering; faculty, higher education;

human resources issues; scientists

Ph.D.s, 23,28,29, 30-31, 33,34, 39,64,205-213

biomedical, 206, 207, 223

Physical science, general, 7, 8, 112, 149

see also superconductivity

Political factors, 3, 61,62

biomedical research, 100

earmarks, 88

energy research, 109, 110

foreign systems, 281

lobbying, 80-81,85-86,93, 123, 149

megaprojects, 27, 157, 159, 163

NSF, 222

peer review, 147

parties, political, 81-82

priority-setting, general, 142, 164

science advisors, 74

see also public opinion

Pork barrel, see earmarking

Portfolio evaluations, 152-153, 163, 164, 167,256

Post-doctoral research, 211-212,214-215

presidential Young Investigator program, 65,66

President’s Council of Advisors in Science and Technology, 74,

76

President’s Science Advisory Committee, 74, 76

Press, Frank, 49, 142, 143

Priority setting, 6, 10, 11-22,63,78, 120-124, 137-167

Bush Administration, 15, 18,76,77

congressional oversight, 86

DOD, 102-105, 106, 116, 117, 124

DOE, 109-111, 116, 124, 149

educational, 12, 13, 17, 76, 77, 78

environmental issues, 15, 76, 150

foreign systems, 275, 276

historical perspectives, 139-140, 146, 147

indicators and, 251-252

megaprojects, 11, 13, 14, 18-21, 27, 139, 149, 156-163

NASA, 105-109, 116, 123

NIH, 99-102, 116, 118, 149

NSF, 111-113, 116, 117, 118

OMB, 77

political factors, general, 142, 164

regional development, general, 13, 16, 17

research agencies, 99-124

research training, 223

small science, 17, 18, 139, 142, 163

statistics, 37

USDA, 113-116, 117

Private sector, 82, 119,252

academic research spending, 177, 200-201

defense contractors, 104, 105,267,268

as employer, 209-210, 211, 213-214

extramural research, 120

space science, 107, 108

see also industry-based research

Prizes, see awards and prizes

Production and productivity, 3

agricultural, 113

bibliometrics, 38,196,197,226,236,237, 242-247,248,254,

281, 283-284

costs v., 171

criteria, 23, 27-28, 36, 39, 66

Ph.D. supply, 23,28,30-31,39,64,205-230

publications, academic, 27-28,36,38,39,66, 195-197,215,

219

productivity, agency, 121, 122

statistics on, 38, 40, 248

see also bibliometrics

Professional associations, 58-60, 77, 112, 193, 195, 235, 236,

257

Program officers, see management and managers

Projections, 62-64,67

academic research, 191

agricultural research, 114, 115, 116

budget deficit targets, 80

defense technology, 104, 105

degrees, higher education, 28,30,31,213-219

energy research, 110

megaprojects, 19

NIH, 183

OMB, 137

strategic, 123-124, 154

Project selection, 16-17,26,27,71, 124-133

DOD, 105, 124, 126

DOE, 110, 111

megaprojects, 157, 160-162, 165

NIH, 125

NSF, 111, 125

Proposals, research, 252

proposal/award ratio, 171, 190, 196,249-250

writing, 240-241, 249-250

Publications, 110,226

bibliometrics, 38,196,197,226,236,237, 242-247,248,254,

281, 283-284

newspapers and magazines, 53-54

opportunity announcements, research, 126, 127,268-269,270

productivity criterion, academic, 27-28,36,39,66, 195-197,

215, 219

productivity criterion, agency, 121, 122

Public opinion, 52,53-54,55,57,85, 237

Qualitative studies, 63

Quality control, see evaluation issues; standards
Index . 373

Racial differences, see minority groups

Reagan Administration, 51,73,74, 100, 109, 110, 111

behavioral and social sciences, 155

Real time, 163

Recruitment and retention, 28,30,60,64,93, 120,195,210,218

Regional development, 10, 143,277

earmarking, 89, 90, 113

educational, 28

institutional locations, 9

locations of research institutes, 9

megaprojects, 21, 166

NSF program, 33, 113, 131-132, 145

priority-setting, general, 13, 16, 17

Republican Party, 81-82

Research assistantships, 208

Research associates, 214-215

Research Initiation Awards, 65

Retirement trends, 215,216

Risk, 120, 121, 122

conservatism v., 121

DOE mission agencies, 119, 121

Naval research, 268

set-asides, 130

Rosenberg, Leon, 60,205

Rosenzweig, Robert, 140-141

Salaries and wages, see wages and salaries

Satellite technology, 106, 108, 123

Earth Observing System, 108, 157, 160, 165

Hubble Space Telescope, 157, 158

Science Advisors, executive branch, 74-76,80, 142

earmarks, 87

India, 280

OMB, relations with, 77, 78

Science and Engineering Equal Opportunities Act, 32

Science & Engineering Indicators, 36,235-236,249

Scientists

academic, 5, 175-176, 188

attitudes, 54, 55, 57, 58-59, 60, 195-196

cost per investigator, 172, 173

defined, 233

foreign, 212

young, 64,65-66

see also employment and unemployment; engineers and

engineering; wages and salaries

Sequesters, Federal budget, 79-80

Set-asides, 35-36,98, 129, 130, 133, 166, 228

Sex differences, see gender differences; women

Shalala, Donna E., 194

Simon, Herbert, 155

Small Business Development Act, 130

Small Grants for Exploratory Research, 121, 122

Small science, 35, 146, 220

human genome, 160

NSF, 121, 122

priority-setting, 17, 18, 139, 142, 163

Smith-Lever, Act, 129

Social factors, 61, 167

academic research benefits, 49

agency cultures, 116, 121, 124

cultural aspects, 49, 58, 160, 277

enrollment, 211

evaluation criteria, 38, 116, 139-140, 141, 147, 151, 272

megaprojects, 20, 73, 157, 160-161, 162

“old boys network’ 125

research organizations, 28

Space Station, 20,73, 157

see also demography

Social sciences, 7,8, 155-156

military personnel, 104, 267

student aid, 208

Soviet Union, 50, 102

Space sciences, 138

defense oriented, 106-107

private sector, 107, 108

public expectations, 52

see also National Aeronautics and Space Administration

Space Shuttle program, 106, 108, 163

Space Station program, 20,73,77, 107, 108, 157, 158, 160,163

Spending, statistics; see expenditures, statistics

Standards, statistics, 238-239

State-level issues

academic research, 193

allocation of funding to, general, 8-9, 10, 143

Cooperative State Research Service, 114-115, 123,271

earmarking, 90, 92, 93

NIH minority student grants, 229

NSF program, 33, 131, 145

set asides, 129

Superconducting Super Collider, 159

see also Cooperative State Research Service

Statistical programs and activities, 10, 13,36-41,67,233-258

bibliometrics, 38,196,197,226,236,237, 242-247,248,254,

281, 283-284

congress, 235

cost, 234

definitional issues, 233,238-238

expenditures, 55, 56

facilities and equipment, 241-242

historical perspectives, 236, 237,238

human resources, 30-31,36-38,44

interdisciplinary research, 38

NIH, 36,37, 174, 178, 187,229,233,238, 239,256

NRC, 235,257

NSF, 36-37,40,41,56,102, 107,116,173,174-181 (passim),

187, 213, 229, 233, 235-236, 237, 238-239, 249-252

(passim), 256,257

OMB, 248-249,257

OSTP, 40,41,248-249,251,257

productivity, 38,40,248

standards, 238-239

work force, 30-31, 36-38, 44

see also demography; indicators; projections

Strategic Defense Initiative, 73, 103, 105, 117, 157, 163,267

Student aid, 30,31, 64

assistantships and fellowships, 208, 214, 227

demography, 206-210, 212,213

earmarking, 93

Federal role, 205, 208-209,212,217, 223,226-229

graduate students, 30, 31,64

minority students, 228-229

Students, 6

recruitment, 93, 218

foreign, 29,31, 205,208,210,212
314 ● Federally Funded Research: Decisions for a Decade

Sustainable Biosphere Initiative, 150

Sweden, 254,273,278-279,281, 282

Superconducting Super Collider, 18, 62, 73, 77, 88, 138, 140,

141, 142, 157, 158, 159, 162, 163, 165

Superconductivity, 18, 21,62, 118, 247

coordination, 118

DOD/DOE, 51, 118

history of, 50-51, 247,248

Taxes, 79,80

Teich, Albert, 164

Television, 53

Tenure, academic, 196,214,220, 271

Traineeships, 208, 227

Transfer of technology, 12, 247-248

foreign use of U.S. research, 10

Undergraduate education, 33,205,206,214,217-219, 227,229,

270

United Kingdom, 254,255,272-275,281

Universities and colleges, 33-36, 131-132

agricultural education, 113

Federal assistance, 208,217, 226-227

undergraduate education, 33, 205, 206, 214, 217-219, 227,

229,270

see also academic research; degrees, higher education;

faculty, higher education; graduate education and research

Utilization of research, 248-256

decisionmaking, 40-41

transfer of technology, 10, 12, 247-248

Vietnam War, 250

Visas, 208, 212

Wages and salaries

academic research, 25, 27, 176, 184-185, 188-189, 191, 192

biomedical research, 100

caps, 26, 63, 176, 184

cost factor, general, 22-27 (passim), 57, 172, 184-185

defined, 192

graduate students, 212

intramural labs, 120

young scientists, 100

Walgren, Don, 137

War, 250

Persian Gulf, 6, 250

public perceptions during, 53

Waste management, 52

Weinberg, Alvin, 139-140, 144, 145, 147

White, Robert, 20-21

Women, 10, 17,98, 129

educational opportunities, 28,29,31-33,44,64,216-217, 222

graduate enrollment, 210

NSF, 113, 270

Ph.D.s, 206-207,208,209

young researchers, 65

World War II, 4

Young scientists, 64,65-66,98, 195-196, 197

academic, 66, 219

NIH, 128

salaries, 100

see also graduate education and research

U.S. GOVERNMENT PRINTING OFFICE : 1991 0 - 292-863 QL 3
Other Related OTA Reports

● Educating Scientists and Engineers: Grade School to Grad School. Presents

American public education as “all one system”; traces the formation of the student

talent pool through the science and engineering pipeline into the research work force;

discusses recruitment and retention as policy options for coping with demographic

changes and market uncertainty. SET-377, 6/88; 136 p.

NTIS order #PB 88-235 973/AS

● The Regulatory Environment for Science (TM-SET-34)

NTIS order #PB 86-182 003/AS

. Research Funding As An Investment: Can We Measure the Returns?

(TM-SET-36)

NTIS order #PB 86-218 278/AS

. Demographic Trends and the Scientific and Engineering Work Force

(TM-SET-35)

NTIS order #PB 86-206 182/AS

. Holding the Edge: Maintaining the Defense Technology Base. Examines the

management of DoD technology base programs and laboratories; analyzes the process

through which technology is introduced into defense systems; and examines the

exploitation of civilian commercial sector technology for defense needs. Concentrates

on the dual questions of expediting military access to civilian technology and keeping

the necessary base of technology alive and well in the United States. ISC-420, 4/89;

200 p.

Free summary available from OTA.

GPO stock #052-003-01 150-6$9.00 per copy

NTIS order #PB 89-196 604/AS

● Agricultural Research and Technology Transfer Policies for the 1990s: Special

Report for 1990 Farm Bill

Assesses the changing agenda for American agriculture and the responsiveness of the

agricultural research and technology transfer system to these changes. F-448, 3/90; 52p.

GPO stock #052-003-01 182-4$2.50 per copy

NTIS order #PB 90-219981

NOTE: Reports are available from the U.S. Government Printing Office, Superintendent of Documents,

Washington, DC 20402-9325, (202) 783-3238; and/or the National Technical Information Service, 5285

Port Royal Road, Springfield, VA 22161-0001, (703) 487-4650.