### Ruts in "The Royal Road "

Report on Principal Investigators' Responses to Written Questions in
NSF- Mathematica Policy Research Inc. Survey of PIs, January--March 2002

##### Submitted to

# 

Robert P. Abel

Project Director

Chief, Budget Operations and Systems Branch

Budget Division

Office of Budget, Finance and Awards Management

#  National Science Foundation 

## June 17, 2002

## Contractor: Deborah Shapley 

# 

**Table of Contents**

# 

# **Preface** 4

**Executive Summary** 7

**Introduction** 10

**Part I First Written Question -- Question 2.5** 11

A Context 11

A.1 NSF's Mission in Research and Education 12

> A.2 PI Proposals Integrate Research
>
> with Training the Next Generation 12

B Impacts of Increases in Award Amount and/or Duration 13

C Many PIs Adjust to Changes 13

D Impacts of Decreases in Award Amount and/or Duration 14

D.1 Postdocs 14

D.2 Students - Graduate Students 15

D.3 Students -- Undergraduates, REU, RUI, Minorities 15

D.4 PIs Donate Their Time 16

D.5 Summer Salary Important 16

D.6 Soft Money PIs 16

E Duration, Continuity, Completeness 16

E.1 The 2- 3- 5- Year Problem 17

E.2 Grant Duration and Churn 18

E.3 Cuts in Collaboration and Travel 19

E.4 Impacts on Scope and Risk 19

F Time Writing Proposals v. Time for Research 19

G Comparisons with NIH 20

H Praise for NSF 20

# P**art II Second Written Question -- Question 3**.**8** 21

A Added Funds -- Priorities 22

> A.1 Scientific Visions - Impacts on the Economy, Health 22
>
> A.2 Support More Students 24
>
> A.3 Development of More Minority Scientists and Engineers 24
>
> A.4 Address Number of Students in S&E Education Pipeline 25
>
> A.5 More Cross-Disciplinary and International Work 25
>
> A.6 Dissemination, Web Sites, Distribution 26
>
> A.7 Instrumentation, Infrastructure 26

B PIs Limit the Scope and Risk of Proposed Research 26

> B.1 "Failing is not an option." 27
>
> B.2 Time Writing Proposals v. Time for Research 28
>
> B.3 Loss of Continuity Creates Inefficiencies 28
>
> B.4 Preferred Award Size, Duration 28
>
> Award size - Duration - NSF Should Follow NIH's Terms

C. Other Impacts -- Stress 29

D. Unique Value of NSF Grants 29

# 

# **Part III Third Written Question -- Question 5.14** 31

A Perception of Shrinking Resources, Lengthening Odds 32

B Perception Young and New are Squeezed by Established Networks 33

C Small v. Large Institutions 33

D Peer Review 34

D.1 Multidisciplinary Research 35

D.2 Large Proposals Suffer 35

E NSF Staff 35

F Core v. Newsy 35

G International Links 36

H. What NSF Should Do 36

H.1 Longer Grants, Simpler Renewals 36

H.2 Would Larger Awards Mean Fewer Share the Pie? 37

H.3 Release Time for Writing, Summer Salaries 37

H.4 Peer Review 38

H.5 Expand NSF's Grants Program 38

# **Appendix A Sample Written Responses** 39

**Appendix B Comments Concerning NIH** 43

> **Preface**

In June 2001 the Office of Management and Budget posed this question to
Rita R. Colwell, Director of the National Science Foundation:

"The current size of NSF grants and their duration might be resulting in
an inefficient process at U.S. academic institutions. Researchers might
be spending too much time writing grant proposals instead of doing
actual research. NSF has increased grant size and duration in previous
years, particularly through its priority research areas; however there
is little documentation that this is having a positive impact on
research output. With the assistance of U.S. academic research
institutions, NSF will develop efficiency measures of the research
process and determine what is the right size for the myriad types of
research the agency funds."

In other words, has increasing the size and lengthening the duration of
some NSF grant awards, as NSF has begun to do, had any impacts on
research? And do NSF- sponsored researchers nonetheless spend "too much
time writing grant proposals instead of doing actual research?" Is there
a "right size for the myriad types of research the agency funds"? In
effect, are there ways in which the NSF grants program, which totalled
\$2.1 billion in FY2001 inefficient?

The agency was to investigate these matters in time for formulation of
its FY 2004 budget. It contracted with Mathematica Policy Research Inc.
to survey all the principal investigators who received competitively
reviewed grant awards in FY 2001. This total, 6,180, was reduced to
5,793 when those who were awarded more than one grant were assigned a
single grant to discuss in the survey**.** The questionnaire was
returned by 91 percent of those surveyed, with 4989 questionnaires
complete enough for analysis, an unusually high response.

Mathematica's quantitative analysis of the PI survey and its survey of
academic institutions, also made in response to the OMB query, appears
as "National Science Foundation Report on Efficiency of Grant Size and
Duration: Principal Investigator FY 2001 Grant Award Survey and
Institutional Survey."^1^

This report analyzes the written responses to three open-ended questions
the PIs were asked in Mathematica's survey. These were Questions 2.5,
3.8 and 5.14 in the questionnaire. For simplicity, this report refers to
them as the first, second, and third written questions. The first asked
what the impact was of changes NSF made in the PI's' grant proposals.
The second asked what would be the impact if NSF gave them "what you
need for what you want to accomplish." The third asked for "any other
comments" about their experience with the NSF grant process that was
"important."

Evidently, these questions touched a nerve -- or several. The PIs'
answers ranged from a few words to several pages.

\-- The first question, Question 2.5, got 1,401 responses out of 2,708
PIs eligible to answer. These print out at 106 pages.

\-- The second question, Question 3.8, got 2,202 answers out of 4,510
PIs eligible to answer. These print out at 198 pages.

\-- The third question, Question 5.14, got 2,548 responses out of 5038
PIs eligible to answer, printing out at 208 pages.

The total of 6,160 written responses print out at 512 pages.

"In surveys of this type, respondents normally don't take the time to
write answers to open-ended questions after they have answered the
pre-coded choices," notes Mathematica Vice President Janice Ballou. She
considers 10 -15% to be a good response rate to open-ended questions. In
this survey 48% of eligible PIs wrote answers to the first question, 47%
wrote answers to the second, and 50% wrote answers to the third
question. Respondents' guarantee of anonymity prevents this report from
tying individual comments to specific NSF offices or actions. But even
though anonymous, these comments are a new, rich lode of perspectives by
the nation's leading scientists and engineers about NSF and the conduct
of research in the United States today.

This was the first major survey of NSF principal investigators since one
conducted by Abt Associates for the agency in January 1987. ^2^ That
survey sought investigators' views about NSF\'s merit review process. It
was sent to 14,282 individuals whose proposals had been awarded or
declined by NSF in FY1985. A total of 9,204 or 64.4% responded. The 1985
survey went to both awardees and declines, whereas the current survey
went only to awardees. The subject of the current survey is different.
It is focussed on the impacts of award size and duration, though the PIs
also write about many aspects of the NSF grants program and the agency.

This report summarizes what is relevant for policymakers about:

\- how the PIs see issues related to NSF's award size and duration,

\- the efficiency of the NSF funding process,

\- how science and engineering is done,

\- the role of NSF in the nation's research and education processes,

\- both human and research outcomes of the NSF investment for the
nation,

\- the realities of the research and educational enterprise in our
universities.

It is a qualitative summary of the written responses, to complement
Mathematica's quantitative analysis of the multiple choice questions.
This report is impressionistic; it represents one science policy
analyst's culling of this large archive for the principal and most
interesting insights.

The text does not attempt to quantify how many PIs held one view or
another. It uses "many," "some," and "several" to describe a reader's
impression of how frequently a comment is made and not actual counts.
Information that is not in the comments but in Mathematica's separate
analysis is so labeled. Other background information not in the PIs'
comments but needed for understanding their context, appears in
brackets.

**Structure of Report**

The three parts of this report correspond to the three written
questions. Because the PIs discussed the same issue in answer to
different questions, comments have been grouped under issue headings.
The Table of Contents is a guide to where specific issues are discussed.

**Permissions**

# All verbatim quotes from Principal Investigators' written responses are reprinted with their express permission.

**Excerpts**.

Most excerpts represent the complete quote, omitting repetition or
detail. A three-dot ellipsis (...) means a phrase has been deleted. A
four- dot ellipsis (....) means the deleted bit starts or ends a
sentence. Spelling errors have been corrected. Where a change has been
made to correct grammar, such as to start a sentence with a capital
letter, it appears in brackets \[ \]. Double quotation marks setting off
quotes within the written comments have been changed to single quotation
marks.

Four notable and representative comments are reprinted in full in
Appendix A. Appendix B reprints all the comments PIs made in answer to
the first question that referred to the National Institutes of Health
for which permissions were given.

**PI References**

The respondents are all Principal Investigators, or PIs. Unless
self-identified, their gender was unknown, so they are referred to as
"their" in follow-on references to avoid using "his" or "her." Of the
4,989 respondents whose complete questionnaires were used, 17% were
female.

######## Author

The contractor writing this report is a journalist and author who has
written extensively for "Science" magazine, for "Nature" and many other
publications. She is author of two books and co-author of "Lost at the
Frontier: U.S. Science and Technology Policy Adrift," published by the
Institute for Scientific Information.

**Footnotes**

1 "National Science Foundation Report on Efficiency of Grant Size and
Duration: Principal Investigator FY 2001 Grant Award Survey and
Institutional Survey, " NSF, 2002.

2."National Science Foundation (U.S.). Program Evaluation Staff.
Proposal Review at NSF: Perceptions of Principal Investigators."
Washington, D.C., National Science Foundation, 1988. NSF 88-4

#### Ruts in "The Royal Road"

Executive Summary

In June 2001 the Office of Management and Budget asked in a letter to
Dr. Rita R. Colwell, Director of the National Science Foundation,
whether recent increases in the size of some grants and their duration,
as NSF had done in some areas, had had any impacts on research. What
impacts could be documented? And despite increases in some awards' size
and duration, do Principal Investigators on NSF grants spend "too much
time writing grant proposals instead of doing actual research?" Is there
a "right size for the myriad types of research the agency funds"?

The agency was charged with investigating these matters in time for
formulation of its FY 2004 budget. It contracted with Mathematica Policy
Research Inc. to survey the total population of Principal Investigators
who received competitively reviewed grant awards in FY 2001, or 6,180
PIs who were involved in the \$2.1 billion grants program that year. An
exceptionally high proportion of 91% responded. Fully 4,989
questionnaires were complete enough for analysis.

This report complements Mathematica's report on the survey
questionnaire, and an accompanying survey of US academic institutions,
titled "National Science Foundation Report on Efficiency of Grant Size
and Duration: Principal Investigator FY 2001 Grant Award Survey and
Institutional Survey."

The survey questionnaire had three open-ended questions to which PIs
could write responses of any length. This report summarizes their
answers: Part I summarizes responses to the first written question,
Question 2.5; Part II summarizes responses to the second written
question, Question 3.8; Part III summarizes responses to third written
question, Question 5.14. Because the PIs discussed the same issues in
answer to different questions, issues have been grouped under subject
headings. The Table of Contents is a guide to where specific issues are
discussed.

These comments amount to a new, rich lode of perspectives by the
nation's leading scientists and engineers about NSF's grants program and
the conduct of basic research in the United States today. They wrote
about what the impacts were from changes NSF made in their grant awards
(first question) and what they needed to accomplish their overarching
goals (second question). The third open-ended question asked them to
"write in any other comments you have about your experience with the NSF
grant process that you think are important." Responses here covered a
wide range of problems and opportunities, often including praise for
NSF's mission and execution. One wrote of the agency:

"It supports pure research and does not demand that there be a payoff in
medical or economic terms. For the ordinary scientist, pure research is
always the royal road to understanding and NSF is often the only place
to go if one wants funding to travel that royal road."

What do these comments say about NSF's grant process? Many PIs are
concerned that grant durations of three years or less are too short
\[these were 85% of the total in FY 2001\]. Many see too few
opportunities for large awards \[grants of 4 and 5 years were 16% of the
total in FY 2001\]. NSF's average award of approximately to \$100,000
per year supports about one student or postdoc after overhead and other
costs, they said. Many say they spend too much time writing proposals to
keep multiple short term grants flowing and this hinders efficiency. The
PIs say there is too much churn in NSF's process for them to attract the
most promising students and keep them the four years or more needed
through the PhD. Many say these factors impede optimal research results
and make it harder to develop the next generation of world-class US
scientists and engineers. Their take-home message is that the grant
process has discontinuities that hinder NSF's research and education
missions. Many say larger awards and more awards would smooth the way.
Many argue longer term awards would be more efficient. One could
conclude from these comments that the venue of NSF grant research seems
a rutted road, not in need of tearing up or of new direction, but in
need of repair.

# **First question -- Question 2.5** 

Here PIs were asked to comment on "impacts" from changes in the NSF
grant awards. In FY2001 47% of competitively reviewed grants were funded
up to 5% of what their PIs proposed; another 51% were reduced 5% or more
from the amount they proposed. Most of the written comments to this
question appear to be by PIs whose proposed grant amounts were cut.

Many PIs said that shorter award terms negatively impacted their
efficiency; so did lower funding amounts, by reducing the numbers of
students and post-docs on their projects.

The PIs listed cost-saving moves they make when NSF cuts their grant
award amount. They do less undergraduate teaching, including less
training of minorities who could become scientists and engineers. Some
cut their own pay and use the funds to pay to their graduate students
and postdocs. When NSF cuts a month or more of their proposed summer
salary, their research suffers, they say, because these academics count
on summer to concentrate more on research. Some donate summer time and
do the work anyway. Or they take other paying work and defer the NSF
project. When funds for collaboration and travel are cut the PIs feel
less productive; the frequency with which they expressed such concerns
reflects the growing collaborative and international character of
science.

No matter what their experience with changes to NSF awards, many said
the present award terms \[average 2.9 years\] and funding amounts
\[annual average \$113,000\] make it hard to attract the best students.
Many say they propose less risky projects to NSF in the first place
because they cannot be sure of getting the results in NSF's timeframes.

But the PIs'main problem was that duration and size of most NSF grant
awards creates churn in making them spend more time writing proposals to
assure they win one or several simultaneous grants so they can keep
their students and projects going. So much proposal-writing, with
relatively long odds of being funded, takes time from research. Many
preferred NIH's grant terms of over 4 years and average annual amounts
of \$300,000. A few said they were under pressure to stop applying to
NSF and concentrate on winning NIH's better terms.

# **Second Question - Question 3.8** 

The second question asked what the PIs would do "if NSF provided you
with what you need." Many said they would attack their scientific
problems on a broader scope and reach for the big advances and
contributions. Many said or implied that they could reach their goals
more efficiently and perhaps sooner. A number said more secure funding
would free up **t**ime now spent writing proposals, allowing them to
make better direct contributions to research. One implication was that
the quality of some NSF-funded research could be higher.

Many would use additional support to mentor more students and other
junior personnel; with larger teams they would accomplish more research.
A number said they would use additional funds to include more minority
students. This was related to risk-taking, some explained, because with
less tight funding they would feel freer to choose from a wider pool of
recruits. Several said that added support would help them integrate
research into their teaching. Many said they would use the funds for
more collaboration. Several said they would use the added support to
improve dissemination and web site development, supporting the theme
that some PIs feel that NSF does not do enough to help them communicate
their findings.

# **Third Question -- Question 5.14**

The third question asked the PIs to "write in any other comments you
have about your experience with the NSF grant process that you think are
important." One theme of importance is the discouragement expressed by
those who believe that the odds of winning an NSF grant were getting
longer and that resources from NSF grants are shrinking. Not one
mentioned that the overall grant program had grown in the prior two
years, nor that NSF's overall average grant amounts had risen to
\$113,000. (The PIs' grants evaluated in the survey averaged \$112,000.)

Some PIs expressed the belief that NSF grants go to a network of
established researchers, and that younger investigators and new
institutions have a harder time winning awards. \[In fact younger
investigators continued to receive about one fifth of all grant awards
since the 1990s; those whose chances of winning NSF grants have gotten
harder are PIs with prior year awards.\] Here as elsewhere, there were
PIs who said NSF had funded their high risk proposals at the start of
their careers, and were grateful for this spark and follow-on support.

The PIs also weighed in on how they preferred NSF staff to be proactive
in advising them and shaping the award. Several criticized panel
reviews; several said mail reviews were fairer. One theme was the wish
for a chance to correct negative or erroneous reviewer comments before
their proposal was put for decision. The biggest impression of the
comments answering this question, however, is PI frustration with the
amount of time spent writing new proposals or revising ones that had
been declined, as an inefficiency in the system.

As for making the NSF grant program more efficient and productive, many
assumed the total "pot" of funds would not grow significantly. Therefore
several were concerned that if the average amount of grant awards were
raised to, say, \$140,000 per year, then NSF would make fewer awards and
the odds of getting funded would be worse. Many proposed that NSF make
more awards to increase the odds of being funded and foster more risky
proposals. The preferred "right size" for grant term was 5 years; many
wrote that 5 years would reduce churn, help them attract better graduate
students, and encourage higher-risk research. However some had cautions
about this change.

Many PIs offered unsolicited praise for NSF. Many thanked the agency's
staff for helping them work toward their education and research goals
under a system which got mixed reviews.

The Appendices give the reader a first-hand look of how the PIs frame
the important issues raised in this study. Appendix A contains four
sample comments reprinted in full. Appendix B contains all the answers
to the first question referring to NIH for which permission was given.

####### Ruts in "The Royal Road"

Report on Principal Investigators' Responses to Written Questions in

NSF-Mathematica Policy Research Inc. Survey January--March 2002

# **Introduction** 

In FY2001 the National Science Foundation handled \$2.1 billion in
grants for science and engineering research to Principal Investigators
(PIs) at academic institutions around the United States. This amount
represented an increase over the FY2000 NSF grant total of \$1.8 billion
and the FY1999 grant total of \$1.6 billion. The FY2000 and FY2001
increases were the latest effort to compensate for the slow growth in
NSF grant awards during the 1990s; from FY 1992 to FY 1998 the NSF's
grant account remained between \$1.2 and \$1.5 billion. Meanwhile the
costs of doing research such as pay and overhead rose through the 1990s.
By decade's end there were concerns in many quarters \-- within the
agency, by the investigators, their institutions, and Congress \-- that
the size of NSF grant awards was not keeping pace with costs.

Another set of concerns was the efficiency or inefficiency in the
performance of research on NSF grants. NSF grant terms are shorter than
the corresponding awards made by the National Institutes of Health, for
example. NSF grant awards made in FY2001 averaged 2.9 years while NIH's
awards (R01 account) averaged over 4 years. Also, the average amounts of
NSF grant awards remain lower than investigators wish. Through the 1990s
NIH's average annual grants have been on the order of \$300,000 per year
compared to NSF's \$100,000. So the NIH grant program appears more
efficient than NSF's in research performed for the administration
involved.

Therefore, in June 2001, the Office of Management and Budget wrote to
the Director of the National Science Foundation asking whether the
"current size of NSF grants and their duration might be resulting in an
inefficient process at U.S. academic institutions. Researchers might be
spending too much time writing grant proposals instead of doing actual
research." It asked if "positive impact on research output" could be
documented from increased grant size and duration of some NSF grants. It
asked the agency to work with U.S. academic institutions to develop
efficiency measures and determine "what is the right size for the myriad
types of research the agency funds."

In response to OMB's query, the NSF asked Mathematica Policy Research
Inc. to survey the entire group of approximately 6,000 PIs who received
competitively reviewed NSF grant awards in FY2001. Mathematica's survey
was multiple choice. It also asked three open-ended questions to which
the PIs could write answers. Fully 91% of those surveyed responded. The
written answers alone comprise 6,160 individual comments printing out at
512 pages. This report is a qualitative summary by a science policy
analyst of these written responses. The quantitative analysis of the
entire survey is Mathematica's report, "National Science Foundation
Report on Efficiency of Grant Size and Duration: Principal Investigator
FY 2001 Grant Award Survey and Institutional Survey."

###  Part I

**First Written Question -- Question 2.5**

The survey's first written question, Question 2.5, asked:

"Please describe any other impact(s) that resulted from the change in
your 2001 NSF award or give more details on any in the list that need
further explanation."

"The list " referred to a multiple choice list of possible impacts of
changes in their NSF award amounts and duration in the preceding
Questions 2.1 - 2.4.

When deciding to make an award, NSF program directors and/or peer
reviewers sometimes change what the PI originally proposed. Sometimes
NSF changes the amount; it can change the duration, or change both, for
example a smaller grant may fund two years of research instead of three.
Of the grant awards analyzed by Mathematica for the survey 47% were
funded up to 5% of the proposed amount. Just 2% had their awards
increased over 5% from their request. Fully 51% had their awards lowered
by 5% or more from what they proposed. Stretching total grant program
dollars by lowering individual awards is common practice by NSF.

Fully 48% of those eligible to write answers to this question did so.
Mathematica found that 6% of the written answers described positive
impacts of awards that were larger or of longer duration than requested.
Another 27% wrote that the change's impact was to reduce their ability
to meet goals and objectives. The largest fraction of answers, 35%,
referred to negative effects such as on team building, students and
collaboration.

Many PIs praised how NSF staff had helped to make their projects
succeed, even when award amounts or terms were less than optimal.

Overall, PIs who had longer award terms and whose proposals were fully
funded said these features helped research efficiency. Many to whom NSF
gave less than they sought and had shorter award terms wrote that these
features hindered their efficiency. Some said they trimmed the scope of
their work and proposed less high-risk/high-payoff work to be able to
meet shorter term deadlines for results. (See Parts II and III.) The PIs
answering this first question, who had their grant awards lowered in
funds or duration, stressed the negative impact on training and on the
scope of research due to having less postdoctoral and student
assistants.

##### **A Context**

Two important pieces of context are needed to understand why larger
amounts and longer duration than NSF's standard \$100,000 and 3 years
seem more efficient and why smaller grants and shorter duration impede
efficiency, in the PIs' view. The twin contexts are NSF's mission and
the character of academic research proposals.

###### A.1 NSF's Mission in Research and Education 

The agency's mission is, in the language of the Act, "to initiate and
support basic scientific research and programs to strengthen scientific
research potential and science education programs at all levels." This
dual mission of science and engineering research *and* education shapes
how NSF research is performed. It means that most NSF grant awards go to
PIs at academic institutions who arrange for some of the work to be done
by graduate students working towards the MS or PhD degrees. Often key
parts of a project are done by those with the doctorate, known as
postdoctoral researchers or postdocs. (These are young researchers
training for junior tenure track faculty appointments.) Undergraduates
also participate under the guidance of graduate students or postdocs,
whose formal employment can be as teaching assistants (TAs) or research
assistants (RAs).

Thus the research project does two things at once: it produces
scientific results *and* provides a forum for the PI to mentor personnel
at different skill levels and career stages. For more than fifty years,
research and education of the next generation of scientists and
engineers has proceeded successfully in this integrated system.

One PI explained eloquently why the NSF system of integrated teaching
and research is a "jewel beyond value and the envy of the world in terms
of the positive returns it brings at so many levels." (See Appendix A)
"University research is first and foremost an essential foundation of
the process and functioning of education." No doubt we get "technology,
scholarship, and knowledge-base benefits" from having research performed
by junior personnel mentored by more senior personnel. But the greater
"gravity" of the system is "the tremendous synergies we build into the
education mission by having it so closely coupled to a preeminent
world-class research enterprise. This is the take home message \--
research is education!"

Thus when PIs consider the "impacts" of changes in NSF grant awards on
their efficiency, they consider the effects on both endeavors: the
performance of research, and publishing and disseminating it so the next
scientific problems can be tackled, and their mission to mentor the best
postdocs and students they can attract, so they become world-class
themselves.

The above-quoted PI wrote: "\[T\]here really is a genius that underlies
the integrated system of teaching and research that we operate in this
country.....Research lies at the heart of how we do graduate education
in the physical sciences. We bring the best students in the country (and
the world for that matter) to this involvement. We develop in them the
skills and envelop them in training experiences that, by the time they
leave, place them at the very top of the national talent pool. .... This
is a comprehensive system of training in the disciplines that works very
well. It is the envy of the world and the NSF is the essential agency of
national reach that preserves and nurtures it. We fool with this only at
our peril."

But this PI was also concerned. "We have tweaked and fooled with \[this
system\] to the point that it is now in grave danger of failing. The
resources being invested in it are below what we need." This and other
PIs' concerns are summarized in this report.

**A.2 PI Proposals Integrate Research with Training the Next
Generation**

Research proposals to NSF represent a package the PI has assembled,
including allotments of their time, full- or part- time support for
postdocs and / or students, use of instruments or field work, computing
and sharing results through conferences, databases and web distribution.
These plans rely on participation by junior personnel to do the
supporting research, for its own sake and to learn, i.e. "research as
education."

Rarely do PIs propose projects they would do alone. Overwhelmingly, they
propose collaborations postdocs and students. A very large number of
projects entail other PIs, sometimes at other institutions in the US and
abroad. The proposal is a package, a precise mix of people, activities,
equipment and collaboration. It often includes individuals the PI has
lined up in advance**,** such as a starting graduate student who agrees
to undertake a piece of the proposed research as their PhD thesis, and
hopes to publish it when it is complete.

**B** **Impacts of Increases in Award Amount and/or Duration**

As mentioned, in FY2001 the agency gave increases of 5% or more above
their grant requests to 2% of the PIs. What one PI did with the increase
shows the coupling of research with education in the integrated NSF
system.

The impact, they said, was "better support for research assistants
(graduate students working on PhD), thereby attracting more high-talent
young people to the field and getting them started on productive
careers. (All the extra support from NSF in this case went into this
category.) Of course, this will also be good for the research objectives
themselves, but the main effect is to foster future growth through
mentoring PhD students."

Those who got more than they asked for said it allowed broader scope and
higher- risk/higher-payoff research. One PI wrote of their increase,
"Mainly it allowed \[me\] to broaden the scope of the research and
undertake a more innovative, ambitious (and time consuming) approach to
the problem under consideration."

The PIs restated this theme elsewhere. (See below "Churn Relieved by 5
year Duration" and Part III.)

**C Many PIs Adjust to Changes**

In FY2001, 47% of PIs received competitively reviewed grant awards that
were within 5% of the amounts they requested. Another 2% had increases
of 5% or greater than their requests. This may explain why many PIs
indicated they rebalanced their work plans adequately.

Their institutions sometimes make up for cuts in awards. One PI wrote
that their large state university "stepped in to make up the
difference," by paying for 5 students the PI had asked for and NSF had
denied. But compensatory funding on this large scale seemed rare; most
institutional help seems to be on the order of support for one student
or for some travel.

Perhaps the easiest adjustment was when their home institution
recomputed the overhead charge to be subtracted from the grant, netting
the investigator what they sought in the first place.

In evaluating the impact of NSF's trimmed awards, account should be
taken of varying overhead requirements of institutions; Institutions
with flexibility or lower overheads may adjust better when a PI gets
less funds from NSF. On the other hand, institutions with higher
overhead requirements, whose net to the researcher from an NSF grant is
no longer worth the time and effort, may ask their PIs to apply to a
more generous agency.

One PI wrote, for example: "The drastic decrease in budget and duration
required that I take out several parts of the proposal which I then
incorporated into another grant proposal to the NIH." Indeed, a common
response to cuts from NSF is that PIs apply for compensatory funds from
another agency or from private industry.

###### D Impacts of Decreases in Award Amount and/or Duration

The NSF's dual mission in research and education explains what PIs do
when NSF makes a grant award for significantly less funds and/or shorter
duration than the PI had planned. Now they must undo the package they
had assembled \-- of co-PIs, post-docs, students, equipment, field work,
computing, dissemination -- to fit the reshaped award. Frequently the
PIs said the impact was less research output, slower results, and
concern they were not attracting the best junior personnel to mentor
into the next generation.

The written answers to this first question show the PIs intensely
concerned with having to cut personnel. A randomly chosen 10 pages of
printout from the 106 pages of answers revealed that 51 of 138 PI
comments, or 37%, mentioned the impacts on personnel, mainly students
and postdocs. Of these 51 references, just two involved increased
employment. The other 49 cited the costs and problems of not having
those personnel the PI had planned.

**D.1 Postdocs**

To cut postdocs from a project is costly in scientific results, since
postdocs have a career incentive to make discoveries on the cutting edge
and become authors or first authors on resulting papers. The PIs, being
tops in their fields themselves, sounded keenly aware of competition for
the best postdoctoral researchers. As one wrote: "The curtailment of the
postdoctoral funding makes it difficult to hire the best person. There
is a real threshold here, i.e., you can't just hire 85% of a person."

"It is the skill and independence of a postdoc," wrote another PI, "that
truly permits us to launch into innovative (and high-risk) areas. I am
fortunate to have an excellent postdoc." But due to NSF's cut in the
grant award, this postdoc would not do "the experiment initially planned
(too time-consuming and high-risk, although it would have had huge
potential returns). Instead she is focussing on more practical generic
correlation measures."

Now from the viewpoint of that postdoc, if "their" PI can only offer
work on "more practical generic...issues," when they had aspired to do
"innovative (and high risk)" research, that postdoc may be less
motivated to join the PI's project. *Not* having this postdoc work at
the cutting edge suggests an efficiency cost: the NSF grant will not
train that person maximally and is less likely to bring the greatest
scientific results.

As a result, postdoctoral participation in a new research initiative
often acts as a catalyst for pushing the boundaries of the research.
That is, the inclusion of a postdoc in a research program has more of an
impact on that program than simply the research that he/she is able to
accomplish on his/her own.

"By not providing sufficient funding for me to hire a postdoc, the NSF,
I feel, has weakened my ability to progress rapidly on the proposed
research, as too much of my time is now spent on direct stewar\[d\]ship
of graduate researchers and too little time is spent on innovation and
new insights."

This PI's last point bears on efficiency. When a postdoc is cut from a
project to meet a reduced budget, so the PI turns to a lower paid
graduate student, the PI must supervise that student more. While this
advances NSF's education mission, if it means the PI spends less time
directly on research, the project may yield fewer results.

###### D.2 Students -- Graduate Students

"Basically, less money means fewer students. \[F\]ewer students means
fewer avenues of research pursued.\[I\]t's pretty simple!" said one PI
bluntly. Another wrote that "\[r\]eduction in funding limited our
ability to have post-doc mentor a graduate student within the
proj\[ect\], which we find to be the best coupling for graduate
education."

One PI had proposed "a paid research opportunity for one post-MS level
graduate student. "Such students are more firmly grounded in research
methodology. They also generally have a broader range of usable research
skills. Eliminating the position increased orientation, training, and
start-up time for the project. "

The PIs' intimate familiarity with various mixes of postdocs, and
students show that allocating them in grant proposals, and accounting
for them administratively, are daily facts of life.

Money can be saved many ways. When NSF cuts a proposed award, some save
by having a proposed graduate student paid as a Teaching Assistant
instead of the preferred Research Assistant. This carries an efficiency
tradeoff; a bit of salary is saved but then more of the student's time
is spent on teaching and less on research. From the standpoint of the
student aspiring to a career in science, the TA position may be less
attractive than the RA position, aside from the lower pay.

**D.3 Students -- Undergraduates, REU**, **RUI, Minorities**

When NSF grants are awarded for less than the PI sought, another
cost-saving move is to hire undergraduates instead of graduate students,
paid technical staff or postdocs. Though some salary is saved,
undergraduates cannot do the caliber of independent work a graduate
student can. With rare exceptions, undergraduates cannot spot the
insights that could be publishable discoveries the way postdocs can. A
salaried skilled technician can be more efficient from the PI's
viewpoint than a series of undergraduates, who each need training for
the technician's work. When the PI directly supervises undergraduates
workers, their educations are helped; but such teaming may not the most
efficient way to get the research done.

When even undergraduate pay was cut as part of NSF- mandated reductions,
the PIs often spoke of the educational loss from not having them
involved. Several expressed gratitude for another NSF 'program, Research
Experiences for Undergraduates, or REU, which pays undergraduates to do
research.

By design, some NSF grants got to PI faculty in undergraduate-only
institutions. NSF awards help faculty there undertake original research
and interact with colleagues elsewhere. By design, these grants include
undergraduates, to help do the research and to use it in coursework.
Historically, some of the nation's outstanding scientists and engineers
started their careers in undergraduate-only institutions.

Many of these institutions are incubators of future minority scientists
and engineers. So NSF grant awards to PIs there affects the future pool
of minorities in the field. One PI, for example, would have used the
funds that were cut "to buy-out teaching time and hire underrepresented
minorities as research assistants. It would also give me more time to
work with undergraduates (I teach MBA students). Exposing minority
undergraduates to economic research is essential to increasing the
potential pool of future minority Ph.D.s."

The impacts of cutting back grants to PIs in these places are both loss
of some research, the greater difficulty of the PI in working with
colleagues elsewhere, and loss of the training of undergraduates.

"Even small grants go a long way toward inspiring, training and
especially educating undergraduate students about science. Moreover,
such grants give professors at smaller institutions the freedom and
funding to stay connected to other researchers in their field." said a
PI responding to the third question.

**D.4 PIs Donate Their Time**

Some PIs said they absorbed cuts in their original plans by donating
some of their time and doing the research unpaid. But most of the
comments to this effect, including the strongest, appeared in answers to
the third question. (See Part III.)

**D.5 Summer Salary Important**

Many PIs who teach during the academic year use summer to devote
uninterrupted time to research. When NSF cuts a month or two of summer
salary from a proposed grant awards, some PIs wrote this caused
inefficiencies. "I had to cut my summer salary....\[I\]t is in fact
where I have some extra monies that, when funded, usually end up being
used for 'new ideas' during the project or to buck up a postdoc salary
etc."

Another said the cut "negatively impacted my family because I could not
pay myself summer salary and achieve the goals of the grant." Another
reported**:** "Summer support reduced from two months to one month. I am
unable to work for free, so this will reduce the time available for
research."

Also summer is the time some can visit with collaborators elsewhere for
intensive work. One wrote: "I did not get compensation for the summer
time which may reduce my possibilities to spend time elsewhere working
together with my research collaborators."

**D.6 Soft Money PIs**

Among those PIs hurt by cuts were those who depend entirely on outside
research funds, including NSF's, for their pay. Career researchers not
in tenure track positions are in this situation, and known as "soft
money" researchers. They are most common in biomedical sciences, but
found in other fields as well. When NSF cuts the amounts 'soft money'
PIs sought, they must find other sources of income, or work for free, or
not do the project.

Here was the impact of a reduced grant award for one soft money PI. "The
biggest impact of the reduction in funds means that I could not support
the graduate students on the project for the entire year, only 2 terms
of the year, and that I have had to volunteer a great deal of my own
time on this project to keep it going. This has meant personal
sacrifice, as I live on soft money, and time spent as a 'volunteer' is
time not spent on other projects or proposals for which I can be paid.
What I see in the funding cuts is an erosion of the value attached to
academic research employment."

**E Duration, Continuity, Completeness**

The context for this survey of all PIs with competitively reviewed
grants in FY2001, as mentioned, was the fact that NSF had begun
increasing the duration of some grant awards. Another context was
anecdotal evidence that researchers prefer NIH's standard term of more
than 4 years (see below).

\[Historically, the average term of NSF grant awards was 2.5 years. In
the 1980s it moved upwards; it was 2.8 years for most of the 1990s.
Since, NSF increased the duration of more grants, so the average was 2.9
years in FY2001. That year 85% of competitively reviewed grant awards
were 3 years or less; 11% were for 4 years and 5% were for 5 years.\]

In written answers**,** PIs often said the shorter terms made them less
efficient. For example, one wrote: "Shorter term grants force you to
generate additional funds from other sources, usually on other topics,
that tends to reduce the coherence and synergism of your total research
program." Shorter term awards forced the PIs back to the drawing board
-- that is, to the proposal-writing table -- more often than they found
efficient. (See F below and Part II B.2.)

Sometimes, PIs were grateful that NSF asked them to do a shorter project
to bring their big plans into focus. One PI wrote that forcing their
project into a briefer 'proof of concept' phase had helped. "We applied
for a center grant and were given a planning grant; thus none of the
research we proposed will be performed. On the other hand, the planning
grant has been of great use to us in focusing our research
activities...."

**E.1 The 2- 3- and 5- Year Problem**

A frequent claim was that NSF's standard grant duration of 3 years
caused them to propose safer inquiries instead of risk-taking. "A 3 year
timescale is discouragement to working on those long term risky ideas
that jut might end up producing an exceptional result," wrote one PI.
"More restricted funding ultimately leads one to retrench within the
more conventional ideas, those that have a higher probability of giving
results for the next round of proposals," wrote another. The belief that
NSF's grants process encourages 'safe science' was a theme of later
answers. (See Part II A.1 and B.1.)

But PIs wrote that 2- and 3- year grant terms give a prospective
graduate student participant less assurance they would be supported
through their theses, which take 4 years or more. Many PIs expressed
concern that the duration of NSF grants made the work less attractive to
the best students and that these promising recruits would go to more
secure research environments -- such as to a PI who had a 5 year award,
or might leave academic science altogether. Anther inefficiency is churn
even before a 3 year grant ends, PIs must apply for the next round; when
it starts they must re-recruit and retrain, in effect rebalancing their
teams mid-stream**.**

\[The employment picture of the late 1990s helps explain PI concern with
attracting the best junior personnel. The number of PhD graduates in
science and engineering fields in the late 1980s was about 18,000 per
year. By the late 1990s, US institutions granted about 28,000 of these
degrees yearly.

\[But the expansion was confined to biology-related fields and
agriculture, while the numbers of MS and PhD degrees in engineering,
physical and geosciences, and mathematics, staying about level. Even
those with the BS in these fields had the prospect of lucrative careers
in the private sector, in a commercial job market with heavy demand.
Small wonder the faculty principal investigators in the survey were
concerned they be able to offer adequate and secure career paths to
attract the best. The PIs in biology-related fields faced pressure from
lucrative private sector jobs, but other research careers funded by a
fast-expanding NIH.\]

The concern some PIs voiced that the quality of graduate apprentice
students was not all they wished may be explained by this context. (See
Parts II and III.)

One PI wrote: "The fact that the award is only for 2 years means (1) I'm
reluctant to hire grad students who need 3 years support to complete
their degrees and (2) I need shorter term goals so I can report results
for the next proposal and (3) I'm going to have to write more proposals,
which takes time away from the research. The fact that it's funded at a
lower level means I can't take a chance on students whose talents and
productivity are less certain."

Another PI wrote: "The change from 4 to 3 years makes it very hard to
find and support a graduate student; there is not enough funding to last
for the entire PhD period of the student."

How these factors interact was explained by another. "The reduced level
of support limits (1) the pursuit of novel research when more time and
resources are needed at the end of the project, when most productive and
fruitful results are about to give birth; (2) the completion of a PhD
student's thesis: normally three to four years are needed to finish a
PhD thesis. This two-year funding can only accommodate a Master's
candidate, thus reducing the level of research somewhat (making Master's
level performance for a PhD caliber research project); and (3) qualified
personnel. ...\[T\]alented students intending to \[go\] on to PhD work
are not attracted to this two-year period in their 4-year career.... "

For a young scientist, the flip side of job insecurity is job security.
One PI looked back on nostalgically on the impact of the Presidential
Young Investigator award they received from the NSF years before. "The
flexibility and security that award me in developing my research program
as a new faculty member was invaluable. Years afterward I am still
reaping the benefits of the research and teaching that I nucleated under
that program."

**E.2 Grant Duration and** **Churn**

Many PIs describe NSF's 1- 2- and 3- year grants as creating churn in
their professional lives that goes beyond the start-stop impacts on the
research. Shorter terms hinder NSF's co-equal mission of attracting the
best would-be scientists and in mentoring them to the point of becoming
world leaders themselves.

Another impact some cited was personal stress, such as the PI who
concluded a long comment saying they wondered "daily" whether to "change
jobs." (See Part II C.)

Churn was reduced for one PI whose grant term was increased by NSF. "At
the time of

the grant proposal I was about to experience a complete turnover in
personnel. The result has been a rather slow restart...educating and
working with my research group. Without the longer duration grant this
would have had a disastrous impact, since there is a lag time in coming
back up to 'normal' research productivity.

"In addition I have just started to form some research collaborations,
and the longer grant will allow there to be some visible impact on my
research results before the next grant proposal is due.

"The extended duration and generous funding has also allowed me to hire
a top quality postdoc. I am very, very grateful for how my grant has
been handled."

Small wonder that PIs answering this and later questions tended to say 5
years was the right grant term. "A 5 year timescale would be much more
helpful," wrote one. One whom NSF had encouraged to ask for 5 years, and
had the 5-year project approved, wrote: "\[t\]he longer period will
encourage more speculative work and perhaps new collaborations."(See
Part II B.4)

**E.3 Cuts in Collaboration** **and Travel**

Few PIs work alone. New knowledge becomes evident faster when
researchers bring complimentary knowledge and skills together. A
mathematician wrote: "It is very important for my research to be in
constant contact with other mathematicians around the world (France,
Germany, Russia, Japan). The less supp\[ort\] I get for travel/summer
salary, the less contacts I have."

Increasingly American PIs collaborate with colleagues in other US
institutions and abroad. New technology also makes field work and
observations from big instruments more cross-national. So the need for
collaborations is rising. But even as the Internet makes distant
collaborations more necessary and feasible, face to face interaction
remains important.

When one PI was awarded a budget that reduced the visiting time of a
colleague from West Africa from 45 to 35 days, they wrote it "will
reduce the time we can work on the GIS \[global information systems\]
component of the project, which may have negative impact in terms of the
analysis of results and timeliness in publication."

Assured travel funds for hoped-for collaborators can be critical to
involving them in a US- based project, and thereby keeping American PIs
at the hub of the world's growing population of international scientific
collaborations.

As for advancing students' education and careers, the chance for them to
present results at meetings advances their knowledge and NSF's education
role.

The flip side is that cutbacks in travel to conferences, or cutting
travel by a co-PI, can set back efficiency. One wrote: "Loss of
essentially all our travel money for conferences... means that our
research results will not be disseminated to other researchers in the
field as quickly as we would have liked \[. (W\]e will have to rely on
refereed publications which have a 6+ month time lag between submission
and publication). "

If a researcher does not go to a relevant conference, the informal and
formal knowledge they would have gotten there is laborious, and in some
respects impossible to obtain afterwards. (See Parts II and III.)

**E.4 Impacts on Scope and Risk**

Increases in dollar amounts sometimes helped PIs to widen the scope of
their research. Decreases in student support decreased the scope,
because less research could be performed. One PI wrote "loss of student
support should merely delay the completion of the work, and somewhat
reduce the overall scope."

More on how PIs said the NSF grant process impacts project scope and
risk is in Part II B and Part III A.

**F Time Writing Proposals v. Time for Research**

Mathematica's analysis of the multiple choice part of the survey showed
how much time the PIs spend writing proposals. For initial submissions
to NSF, 33% spent 80 hours or less preparing their proposasl; 29% spent
81 to 150 hours; and 29% said they spent more than 150 hours (3 1/2
40-hour weeks). The length of time went up for resubmission of proposals
to NSF, with 36% saying they spent more than 150 hours on the revised
versions.

A typical written comment was this complaint. "It is very discouraging
that NSF makes it so difficult to obtain grants for longer than three
years or to obtain significant budget increases for a single grant. The
amount of time that I spend writing proposals is getting to be a larger
fraction of the available research time. NSF is not keeping up with
inflation as far as grant size is concerned and is not making the best
possible use of researcher time as far as grant duration is concerned.
The three-year grant cycle makes it harder to pursue the most difficult
projects or attempt the most innovative approaches." When NSF cut
another PI's grant term to two years, they wrote "I'm going to have to
write more proposals, which takes time away from the research." (See
Part III D.4.)

The single most recurring concern about the inefficiency of NSF grants,
is the amount of time spent writing proposals, which they claimed they
would otherwise spend doing research.

**G Comparisons to NIH**

One reason for the survey was the chance that NSF PIs find NIH's average
grant of \$300,000 per year and average term of over 4 years more
efficient. Appendix B reprints all the PI comments for which permissions
were given that referred in any way to NIH in answer to this first
written open-ended question. This is to give readers a first hand
understanding of how NSF PIs perceive NIH.

For example, a PI whose grant duration had been cut to 3 years from
4**,** wrote that as a result, "one of the postdoctoral fellows who had
planned to come in year 3 from Australia, may not participate in the
work. I had hoped to guarantee 2 years of support for that position in
order to get this person to my lab.

"As a result, I have had to go to NIH and request funding for a
different project, to try and make up the difference in funding level
and length of award. .... I guess the summary is that even for a small
lab, it is difficult to survive with a single NSF grant, and the 3 year
cycle is not long enough to allow full development of the work in time
to write the next renewal."

But the NSF PIs who were switching to NIH-- or said they were under
pressure to do so -- were ambivalent about the change.

When NSF's net to one PI was "tiny" due to their institution's 110%
overhead rate, they wrote: "I am therefore under pressure (both economic
and from my director) to redirect my research grants to NIH. I do not
want to do this as I have valued my NSF grant very much as it has
enabled me to pursue the more imaginative aspects of my research program
over the last 15 years of continuous funding from NSF. If the funding
level at NSF does not increase in future, I shall be forced to terminate
my relationship with NSF (as will other recipients at my institution)."

Another: "As NSF grants get smaller compared to, say, NIH or DOD grants,
it is harder to justify putting as much time into these projects as one
would if they were fully funded. In fact, in ... it is hard to find time
to spend on fundamental science, when it is so hard to generate funds to
support it. So one feels pressure to spend less time on NSF-supported
projects."

**H Praise for** **NSF**

Whatever concerns and frustrations the PI displayed in answer to this
and later questions, their support for the agency's program and unique
value came through.

For example, one PI whose goals were reduced drastically, wrote
nonetheless, "the reduced scope of the funded project allowed us to
submit another proposal with expanded and non overlapping scope. The
initial funding, though reduced, made possible pilot investigations to
commence and to establish credibility. Furthermore, this experience
helped us to sharpen our approaches and goals."

**Part II**

**Second Written Question - Question 3.8**

The survey's second written question, Question 3.8, asked:

"Please describe any other impact(s) that would result if NSF provided
you with what you need for what you want to accomplish, or give more
details on any in the list that needs further explanation."

In the preceding multiple choice parts of this question (3.1 to 3.7) the
PIs had been asked how much additional funding they would like over the
next five years from all sources "to achieve what would like to with
your ongoing body of research and educational activities." After
providing a figure, the PIs were to rank as "positive," "none," or
"negative" each of 18 possible impacts. Some of their written answers to
Question 3.8 gave details of impacts from the NSF list.

Question 3.8 gave the PIs the chance to say generally what they would do
"if NSF provided you with what you need." From the written answers, it
seems that this part of the question triggered the unusually high
response rate: 47% of those eligible to answer the question did so.

The largest fraction of PIs who wrote comments said, if they could have
what they needed, they would attack their scientific problems much more
broadly; they would reach directly for the big advances and
contributions that are the goals of their present NSF research.

Many said or implied that with added support they could reach their
goals more efficiently, perhaps sooner. A number said more secure
funding would free up time now spent writing shorter term proposals and
allow them to focus more on their own research. One implication was that
quality of NSF funded research could be improved.

Many would use additional support to mentor more students and other
junior personnel. As PIs explained in answer to the first question, when
they have more students they get more research done (see Part I D).
Answering this question, they see mentoring of more students as part of
expanding the effort that would help them tackle broader research goals.

A striking number said they would use additional funds to include more
minorities in their research activities. This was related to
risk-taking; some explained that if funding were less tight they would
be freer to choose from a wider pool.

The time relieved from proposal-writing they hoped to spend in equal
measure mentoring junior personnel and working directly on research.
Several said that added support would help them integrate research into
their teaching.

Many said they would use the funds for greater collaboration.
Instrumentation was also mentioned. Supporting the theme that PIs feel
that NSF does not do enough to help them communicate their research (see
Part I), many said they would use the added support to improve
dissemination and web site development.

Several wrote, as many had answering the first question, how NSF's
present funding levels and grant duration limits the scope and
risk-taking of their research. The work they propose for NSF support
tends to be low-risk, they would rather propose higher-risk/higher-yield
projects. (See also Part I E.4 and Part III A and B.)

The PI who wrote eloquently that NSF's system of integrated research and
education is a "jewel" and "the envy of the world"(see Part I A and
Appendix A) was answering this second question. This PI also wrote the
system is in "danger of failing" because it had been "tweaked" too much
and "the resources being invested ... are below what we need." Like many
others, this PI wants to "increase grant sizes and funding balance
across the portfolio of scientific interests served by the NSF,"
including "longer grant periods" to "rationalize" the system.

**A Added Funds - Priorities**

## 

## **Scientific Visions A.1** 

Mathematica's report contains how much more funding the PIs said they
needed, and the list of effects of such funding, based on the multiple
choice answers to Question 3.1 to 3.7. The largest fraction of their
written answers say that if they had the funds they need, they would
attack scientific problems on a broader scope and reach directly for the
big advances that are the goals of their careers.

Their exciting scientific visions can only be glimpsed through the brief
descriptions. But since the public supports the cause of keeping the
United States in the lead in many fields of science, it is useful to
sample these impressive goals. The 5,989 answering these questions are,
after all, among the finest scientists and engineers in the nation and
the world.

"My field is Microbial Biology and we are doing ecological genomics.
NSF-sized budgets are simply not big enough to do this type of work, so
we have to scrounge for support from everywhere. We don't have the
equivalent of NIH available to us, thus we cannot run the large
efficient labs that those in the biomedical community can. Raising
funding is almost a full time job for me," said one.

Another said their work would have "important bearing on the tectonics
of Asia as a whole" and will "help identify the ancient environments
that were responsible for the formation of the gold and rare metal
mineralization in the Trans-Baikal region."

A self-identified solar physicist hopes "to make the connection between
the direction of the coronal magnetic field and the distribution of
density in the corona. These goals could readily be achieved if NSF
provided the funding we had originally requested. The funding would also
lead to innovative techniques in studying the corona...."

"This is a very big project to survey galaxies in the distant universe.
The NSF grant does not provide enough money to thoroughly analyze the
data. It is also not large enough to allow us to develop the data
archive we would like to prepare in order to share the data (which are
unique) with the rest of the astronomical community...." wrote another.

Biomedical imaging could experience "a substantial impact" in the view
of the PI who sought "efficient computational methods for improving
images in three dimensional fluorescence microscopy ...."

The relationship between genotype and phenotype is a long-standing
mystery, but one PI is working with a postdoc on "the genetic
correlation (tradeoff) between horns and eyes in beetles" in "the most
rigorous attempt yet to quantify the impact of a developmental
'constraint' on trait evolution." The work would produce "a very big
step in the link to fill in the 'black box' between genotype and
phenotype!"

Another PI wants to "\[a\]dvance the knowledge of a new class of plant
proteins (Ribosome-Inactivating Proteins-RIPs), and its integration into
easy to use technologies." They hope to "\[I\]ncrease our understanding
on the effect of these proteins in translation ... available for humans,
yeast, and Drosophilia. ...Currently there are just three research
groups (including mine) in the US that studies this interesting group of
proteins...."

A PI interprets early climate and evolution to better understand the
present distribution of species. A pollen database is used "as a tool to
provide improved paleoclimatic and paleoecological interpretations of
Amazonian and Andean datasets. These data need to be integrated into a
larger initiative that merges climatology, human landuse studies,
biology and education...."

Another wants "eventually" funded "a high energy physics experiment
which will probe one of the most fundamental questions in the field. It
could possibly have the effect of changing the standard model of
particle physics. The impact of doing so would stimulate much
theoretical and experimental activity throughout the field of elementary
particle physics."

One PI wants to "\[c\]onvert nanoscience into nanotechnology for
societal needs and define the emerging academic discipline of
nanoengineering, including nanomanuracturing and nano processing."

An economist wrote: "We now know that the flow of information explains
in part explains in part 'irrational exuberance' in markets. But .... I
would like to explore other ideas such as: ....'Exactly what kind (lack
of) information creates irrational exuberance and what information flow
stabilizes markets?'"

"The genetics & genomics of plant resistance to insects ... are
overlooked but emerging areas....a more comprehensive idea of the
genomics of arthropod and disease pest resistance in general....." wrote
another.

"The longer term impact to the global agriculture community potentially
would be very high if the research led to...understanding...the
...mechanism underlying apomixis \[asexual reproduction in plants\]."

"The results could lead to an environmentally safe technology to
destruct harmful, toxic halogenated organic compounds," wrote another
PI.

"Langmuir circulations occur in the upper ocean and lower atmosphere as
long rolls of fluid which align with the wind; they greatly enhance the
transfer of carbon dioxide from the atmosphere into the ocean. The
intent of my work is to credibly model them. The work ultimately will
impact our ability to predict, and to some degree control, global change
via carbon dioxide."

**Impacts on the Economy, Health**

This sample shows how some PIs' work could impact innovation, the
economy, and health. Nanoengineering and nanomanfuacturing are very
important to future US competitiveness, for example. Understanding the
interrelationships between climate, landuse, and biology offers keys to
humanity's response to climate change. Carbon dioxide is a chief culprit
in global warming; predicting how oceans absorb it is key to modeling
climate change. Three-D florescence microscopy can have a big impact on
biomedicine. NSF grants support much of the nation's basic research in
economics; they also support crucial basic research in computing and
software not done by the private sector. So while most NSF research
advances basic understanding with applications in the long term, some
has near-term payoff.

Another benefit of giving the PIs what they need, some said, would be
that the pace of US discovery would be stepped up in fields where other
nations are closing in or surpassing us. Part III G takes up some PIs'
concerns that the United States is lagging internationally because of
inefficiencies in the NSF research support system.

Some mentioned that longer term, more secure funding would improve
quality. One PI wrote: "Anything NSF does that helps me spend more time
doing science, thinking, and educating students to thin\[k\] and pursue
their own ideas, will help produce higher quality science and
scientists."

Additional funding would help some publish more, said one who was
concerned that conflicts on their time had lowered production of papers
from about 3-4 average in the five years they were supported by NIH, to
1 to 2 per year because now, "when I write, I have to stop the lab
work."

# **A.2 Support More Students** 

More funds over a longer term would allow PIs to expand their mentoring
of the next generation, many said. Some indicated it would improve the
quality of junior personnel, a reminder that many PIs believe they are
in a competitive market for the best talent.

For example, one PI wrote that more funds "would allow me to expand my
current research group in a controlled manner, meaning I would be much
more selective with respect to whom I chose to join my group. This would
promote and enhance the quality of graduate education in this country."
A self-identified senior physicist wrote that with added funds, "I could
return to educating the very best post-docs in my field. I could take on
the type of lon\[g\] range, innovative work that a senior physicist
should lead." A significant increase in funding would enable "a more
aggressive recruitment of quality graduate students," wrote another.

An engineer expressed the particular need to find PhD students, instead
of MS students, to do more advanced research (see Part I D.2). "The
opportunity to attract more PhD students ... would in turn increase
productivity, given that they stay longer than the MS students that
predominate graduate programs in engineering."

Some PIs would use the funds to strengthen the educational aspect of
their projects, rather than in effect, using students as mere employees.
One PI would use added funds to "give even a stronger focus on the
education process... . \[V\]ery often compromises are made to meet
program/grant expectations that take away from the best possible
educational path for a PhD student. For example, me just telling someone
what to do to resolve a specific problem rather than letting them work
through it."

Several PIs would use the funds to improve the undergraduate mentoring.
One PI, self-identified as at "a small college," wrote: "Funding student
research assistants is critical to both my research program and my
teaching (as I incorporate my research findings and materials into the
classroom)." Another wanted to "engage students in cutting edge research
earlier in their collegiate experience....to enable sophomore students
to work alongside graduate students and ....to integrate research
experiences and findings into both undergraduate and graduate
coursework."

# **A.3 Development of More Minority Scientists and Engineers** 

A striking number indicated they would use the added funds to enlarge
the pool of minority scientists and engineers. "It is impossible to
achieve worthwhile social goals (such as increasing underrepresented
communities, enhance teaching, outreach to public) when the grants must
be no-frills to get funded. This forces a researcher to do what has
worked with who they know will work...." one wrote. Another wrote that
if the had the funds they needed, they would bring a " significant
increase in the number of Native American students interested in the
sciences...."

A PI self-identified as at a "young" university and a new laboratory,
said "continued funding would mean a lot not only to my research but
also for the morale of the university administration. Additionally this
is a minority-institution trying to build a strong research university
in an area which had no tradition. The NSF funds are helping a great
deal in this regard."

A few commenters would expand the numbers of women, such as one whose
"goal is simply to perform first rate research and to produce as many
female physics PhD graduates as I possibly can, so that we can
contribute to 'changing the face of Physics.'"

# **A.4 Address Number of Students in the S& Education Pipeline** 

One argued it was an urgent national priority to use additional NSF
grant funds to expand the ranks of scientists and engineers from the
upcoming population of undergraduates. Over the next decade, one in
three science and engineering faculty will retire, this PI said \[most
estimates place the fraction at one in four\], while the ranks of
undergraduates will swell.

"More than a third of the current faculty of the country will retire in
the next 5 -- 10 years. The enrollments of most public universities will
go up by 15% over this same period of time. If we do not pay attention
to and preserve the personnel development activities that our system of
Federal research funding supports, we are going to be in very big
trouble." (This is the "jewel" PI quoted in full Appendix A.)

# **A.5 More Cross Disciplinary and International Work** 

Some PIs specifically said that adequate funding would help them cross
academic disciplines. For example, one wrote that more support "would
give the ability to learn and pursue different disciplinary approaches
to this research area (which lies at the boundary between several
disciplines)."

But most PI comments concerning outreach said they wanted more
collaborations with other institutions and internationally.

A striking comment, quoted here in full, shows how one PI sees
globalization of training and collaboration as potentially speeding up
scientific advance. \"Given the sophistication of my area of research, I
need to collaborate with several other groups to

obtain the highest quality level of results. It would be extremely
useful to have supplemental funds, or even a separate grants program,
for collaborative research that spans different universities and
different countries.

\"An example is my last publication: five different countries are
represented. My graduate student was effectively educated in the
laboratories of three countries. A collaborative supplement or a
collaborative grant would allow for the funding of a student who is
quite literally \'shared\' among a number of different laboratories.

\"Although it is true that the costs can be shared, the benefit to US
science would be enormous were there monies for this type of student. In
effect, information flow would be toward the US rather than from the US.

Another wrote, "\[a\]dditional funding would provide for more
collaborative contact and post field work analyses with Russian
partners. T\]here is value in bringing once isolated scientists in to
the broad and open science arena...." In other words, U.S. science would
benefit if it could draw more on formerly isolated talent abroad.

Another PI quoted in full in Appendix A, stressed competitiveness.
Larger grant awards would allow them to "concentrate their efforts to
make better progress over shorter frame of time. This change will allow
US scientists to better compete with EU groups where a full professor is
usually given fundings \[sic\] to support a 10-20 person group with
minimal amount of paperwork." To this PI, the European Union's support
for science seemed more efficient (see Part I E.3 and Part III G.)

# **A.6 Dissemination, Web Sites, Distribution** 

Other forms of outreach, especially via the Internet, require findings
and sometimes raw data to be prepared for distribution. This work is not
cutting edge basic research, yet it is critical to achieve synergism
among researchers. Several PIs wanted additional funds to complete their
projects by making the results available by software and/or internet
dissemination.

With added funding "the information stored in our collection could be
made many times more accessible to researchers...." said one.
Availability "will facilitate not only basic research, but innovative
research involving meta-data and synthesis of existing research.... It
will also help in our efforts to communicate 'science' to students and
the general public."

"The principal impacts of these studies are to provide baseline data to
other scientists on when climate changed, how much it changed, and what
influence that had on natural and human communities..." wrote another.

Some mentioned dissemination to students, including elementary and
secondary students known as K-12. For example one PI would use added
funds for "creating and distributing software that is user friendly to
collaborators, industry, and to K12 teaching institutions."

**A.7 Instrumentation, Infrastructure**

The relatively small size of NSF grant awards were not enough for some
instrumentation needs. An experimentalist wrote: "As the
state-of-the-art of instrumentation improves...the funds needed for
instrumentation is far more than typically feasible with the current
size of the grants." Also the NSF's special announcements of awards for
instrumentation monies "do not have the success rate needed to sustain
genuine and well justified needs of the creative and active scientists."

Besides equipment, this PI's field needed a "substantial infusion of
funds" for "crystal growers, technical staff to maintain sophisticated
equipment, and so on." Another wrote: "Access to new equipment would
render our laboratory more competitive especially when compared to Asian
or European laboratories where the technology infrastructure has bloomed
recently."

Another PI wrote the added support could "potentially set up a
nationwide network of 10-12 other universities using similar apparatus
to share research developments."

**B PIs Limit Scope and Risk of Proposed Research**

Though this second question asked how they would be helped by having the
resources needed from NSF, many PIs nonetheless wrote of the problems of
present NSF grant duration and amounts -- their theme in answering the
first and third questions. Many wrote they follow, in effect, a
slice-and-dice strategy that nibbles at the bigger scientific issues.
(See also Part I E.4.)

#  **B.1 "Failing is not an option"** 

PIs do not propose high risk aspects of their work to NSF, according to
one PI. "I would not gamble the future funding of this program by
focussing on high risk efforts. The Foundation must recognize that a
baseline level of funding of ongoing programs will never lead to
innovation, it will only provide enough for self preservation at renewal
time....

"\[T\]his proposal was a case in point of how frustrating it is to move
in new directions with NSF money. This was an accomplishment based
renewal which I felt demonstrated clearly that we made excellent
progress in the past granting period. Half of the proposal outlined how
we will continue....However, we recognized opportunities in new
directions that were the centerpiece of the application.

"Unfortunately, when the program officer can provide only can provide a
10% increase over the previous period (my general experience with NSF
over the past 20 years) I cannot justify risking the project on the most
uncertain directions."

Wrote another: "....More stable and longer term funding would provide a
base from which to investigate innovative methods, and would permit the
opportunity to fail. In the current environment, failing is not an
option...." Another wrote: " ....A more secure source of funding would
allow us to spend more time doing what we are paid to do. And we would
take more risks in terms of expanding beyond our current area of
expertise, and developing broader collaborations."

### B.2 Time Writing Proposals v. Time On Research 

The inefficiency of frequent proposal writing takes time away from
research, many PIs insisted, as they had in answering the first question
(see Part I F). This complaint directly answers OMB's original question
to NSF (see Introduction) and may be the most frequent comment PIs made
in answering the three written questions in the survey.

Many proffered individual calculations of how long they took writing a
proposal and how many proposals they must submit to yield enough income
to support their work and students. They stressed that this took time
away from research. Typical was the one who estimated "4-6 weeks of
research time to put together a proposal."

Many PIs said they carve up their big aspirations into smaller parts
that are sure of success in three years \-- or sooner, because they lose
time waiting for the funds and getting started. The critical PI quoted
above (also in Appendix A) stressed such inefficiencies.

"I must write about 1 proposal every 3 weeks to procure enough fundings
\[sic\] to support a group of 15 students and postdoc at a private
research university because each proposal only funds 1- 1.5 students and
my success rate, as a junior faculty member, is only about 12%. On
average I spend about 30h a week dealing with the mechanics of proposal
writings and submissions. This, however, amounts to only 30% of my
time + overtime because I must work 100h/wk." After ticking off how the
balance of the 100 hours is consumed in other chores, the PI went on,
"rarely do I have the time to think about science and explore new
ideas."

Another PI said that "If NSF would just give me the funds I need to run
my group and do my research" they "could finally focus" on it "instead
of spending 50% of my time for writing likely declined proposals."

Another version of this message was: "\[T\]he funding duration standard
is \[so\] low (3 years) and the competition for funds so high (which
causes underbidding because your livelihood depends on receipt of
funds), it is difficult to put enough time into these complex problems
to complete them efficiently. The result is a cycle of proposal writing,
report writing and spreading one's efforts, that ultimately create
inefficiencies overall in the research process."

# **B.3 Loss of Continuity Creates Inefficiencies** 

# Several PIs said they need to propose something new each time they approach NSF so that earlier work breaks off. One PI participated in a "coordinated field project/experiment in climate change" that showed "exceptional potential for advancing the climate-change science for the benefit of mankind." They needed three years to finish post-experiment analysis, but the NSF cut funds for their proposed second year by half; and they had not even asked for the third year. This "is detrimental to my efficient relationship with this project...."

Another wrote NSF it does not provide the kind of long time continuity
that enables me to go really deeply into a subject; Each new NSF grant
is, to a large degree, a new grant."

"Starting up and shutting down research in response to changing funding
is very detrimental to long-term research goals," a PI answering the
third question, "These horrible gaps in funding where we're trying to
stretch out the last grant by penny pinching until we get the next one"
create more inefficiency than grant size. This was why Canadian
scientists who are "pretty much assured of continuous funding at
...about \$50 K per year in my field....are beating us to the questions
-- often coming up with answers before we get funded to do the very same
thing." (See Appendix A.)

# **B.4 Preferred Award Size, Duration** 

# **Award Size**

"NSF money is prestigious, but the grants are small relative to other
sources," wrote a PI, echoing a common view. Those PIs who addressed
grant size mainly said they should support a reasonable number of
students and postdocs. A typical comment was this from a self-identified
theoretical physicist. Researchers in their specialty "COMPLETELY rely
on the NSF funding....If one is allowed to dream, the ideal NSF grant
for a senior, well established theorist in their 40's would be for 5-6
years and at the level of about \$140K per year of REAL spending money.
" The reference to "real spending money" is to the overhead taken by
each PI's institution from the NSF grant total, discussed in the
Mathematica report and in Part I C.

"This is what is needed to restart the engines of the US enterprise in
basic natural sciences," this PI continued. "We are not biologists." As
theoretical physicists"we do not need 1M \$\$\$ grants. What we need is
the ability to hire a postdoc or two, pay two or three graduate
students, get the latest computer box, and travel to several key
conferences/workshops a year...."

**Duration -- Five Years Preferred**

#  Five years was preferred by those who mentioned an ideal grant duration. (See Part III H.) One explained: "Research always discovers new information, applications, and educational opportunities which cannot be addressed within the traditional \[3 year\] time constraint. Five year grants or two year (minimal effort) extensions would also allow for continuity in staff and more effective (among) grad student knowledge transfer. The working body of knowledge thus increases exponentially and not linearly." 

**NSF Should Follow NIH's Terms**

Here and later PIs referred to the tradeoff between assured long term
support, which some said at some other agencies leads to stagnation, and
the NSF system of intense, frequent recompetition for small short term
grants. The NIH was the only federal agency whom the PIs wanted NSF to
emulate. (See Part I G and Appendix B.)

Generally, PIs view NIH's balance between longer term stability v.
recompetition as more efficient and productive. "Even under a constant
funding level you will see greatly increased productivity if the
resources are granted with a longer time horizon (compare the longer NIH
grant periods with ours)," wrote one.

Rising costs was a factor for many who wanted NSF to drastically
increase award size and duration. Wrote one: NEW 22916 "Research has
become very expensive over the past years. Therefore the limited size of
NSF budgets compared to other granting agencies, e.g. NIH, makes it
difficult to pursue a more extensive research program. also the hiring
of qualified personnel becomes a problem. Graduate stipends along with
tuition have also increased...."

# **C Other Impacts - Stress**

Personal stress could be relieved by more secure funding, and less time
writing proposals to hedge against the loss of funds. An interesting
comment was the PI who wrote: "Five-year funding would allow me to move
further from 'safe science'. It would also decrease the stress in my
life." Another wrote: "You have hit a tender spot with this
questionnaire. I have watched closely as my most exciting colleagues,
and I myself, have been stressed to the limit by the frantic process of
funding and pursuing innovative research. We are losing young scientists
and potential scientists who see this happening.

"The US is still in a unique position among modern nations, in that the
people still believe in basic research, and the people in the agencies
are advocates and often skilled scientists themselves.

"I am so lucky to have been funded by the NSF for many years, yet my 2
to 3 year grants are accompanied by continuous stress."

**D Unique Value of NSF Grants**

Many PIs singled out what was most valuable to them about the NSF grants
program, values they want to preserve if any changes were made.

#  "NSF is the preferred funding agency for my research. It offers the simplest grant application and project administration procedures," wrote one.

"I certainly value the funding I DO have from NSF. " wrote another. "It
is wonderful 'few-strings-attached' funding that has supported all of
the exciting things we have done in my lab."

"Having the NSF support high risk, innovative ideas allows the PIs to
pursue knowledge in a way that positively impacts their creativity and
they learn how to think outside the box. I very much value my NSF
support and have leveraged my fundamental findings to secure additional
monies that put our findings to practical use."

The value to the nation of NSF grants for basic research in technology
were stressed by PIs in this area. One who works on "aging wiring"
wrote:

"We are working on a new method for detecting dangerous conditions in
aging wiring (aircraft, nuclear power plants, large machinery,
communication networks, etc.) which has been identified as an area of
critical national concern. The funding provided by NSF so far has been
the only place that I could try highly innovative (and slightly risky)
technical advances." Other funding from government and industry has been
for "simpler" development. But "NSF funding provides the 'next
generation of technologies" that are needed to take this project to
optimal conclusion. It is amazingly difficult to get funding for this
basic research any other way."

A good summary was this comment. "NSF, in my view, constitutes the
premier funding source for high-quality university research. An increase
in NSF funding will enable focusing brain--power for long term basic
research, which is nowdays scattered in many directions because PIs want
to have sufficient resources to carry out their research goals; e.g. PIs
with ability to carry out fundamental research that should be funded by
NSF \[now\] have to seek extra funds from other government agencies or
industrial sources that of course some with other constraints that
result in waste of brain power and time."

# **Part III** 

# **Third Written Question -- Question 5.14**

The third written question, Question 5.14, asked:

"Questionnaires by their nature are limited. Please write in any other
comments you have about your experience with the NSF grant process that
you think are important."

This question elicited the largest number of answers. Fully 2,521 PIs
wrote responses, or half of the total. Printout of these remarks runs
the longest of the three written questions, at 208 pages.

So 2,521 of the nation's leading scientists and engineers sounded off
about the good and the less good concerning the nation's premier basic
research agency and their circumstances. The agency would do well to
mine these comments for feedback concerning individual programs, staff
members, details of award timing and procedures and other aspects of its
grants program.

What follows are samples from this rich lode that seem most relevant to
the OMB-NSF-Mathematica study of the efficiency of the NSF grants
program.

The first important theme is the discouragement of many PIs who wrote --
rather, who assumed -- that resources from NSF grants are shrinking.
\[This perception is striking because the average NSF grant award has
increased in size in recent years.\]

A second theme is the PIs' view that younger investigators have a harder
time winning NSF grants. Many wrote that major institutions or old-boy
networks control distribution and edge out more innovative proposals new
investigators and institutions. \[This perception is also striking
because it is not true overall: the success rates of PIs with prior NSF
grants has been trending down while new applicants, who are mostly
younger, have held their own.\]

Though the PIs often praised NSF, their program directors, there were
criticisms of panel reviews, how NSF handles multidisciplinary proposals
and big- ticket proposals; both are believed to fare less well. As in
earlier answers, PIs complained they spend too much time preparing
proposals and revising them to meet reviewer's demands. Many wrote that
with 5-year instead of 3-year grants, they would run this gantlet less
often and so spend more time doing research.

Some mentioned specific NSF practices they said hinder their efficacy,
such as on summer salaries, overhead, rotator staff and others.

Yet the PIs gave high marks to the agency, most staff and the integrated
system of teaching and research that the NSF grants program sustains.
There were also concerns whether the best students are joining NSF
projects, and about the quality of the research when fewer risks are
taken. Here as earlier a few comments showed real stress, such as one
who wrote that the process "makes me wonder if this is the best way to
spend my time."

The summary ends with some of the PIs' suggestions for change, including
warnings about the move to longer grant duration and larger grant size
favored by so many.

## 

## **A Perception of Shrinking Resources and Lengthening Odds**

Not a single comment in answer to these or earlier written questions
mentioned the fact that the total NSF grant programs has risen in the
past 3 years. \[It remained in the \$1.2 - \$1.3 billion range from
FY1992-FY1996; by FY1999 it had reached \$1.6 billion. In FY2000 it rose
to \$1.8 billion and in FY 2001 to \$2.1 billion. FY2001 was the year of
the grant awards which the PIs describe in the survey. \]

\[The size of the average annual grant bottomed out below \$80,000 in
FY1995. By FY2001 it had marched upward to \$113,000. As mentioned, the
mean duration rose to 2.9 years from 2.8 the year before, as the agency
tried to award at least some grants for longer terms.

\[Overall, a PI's chances of getting funded by NSF's grants program have
gone up slightly from the early 1990s. In the 3-year period
FY1992-FY1994, 42% of the PIs who proposed got funded; by FY1999-2001,
43.5% of PIs who proposed got funded. On a per-proposal basis, 29.5% of
competitive proposals won grant awards from NSF in FY1992-FY1994; by
FY1999-FY2001 29% of such proposals were funded. So the overall odds
have stayed the same. Proposals to the comparable NIH program (RO1) have
had a 31% success rate since 1998.

\[New PIs -- i.e. those who had not had prior NSF grants -- held their
own; their success rate ranged from 17% to 20% during the 1990s; in
FY2001 19% of new PIs proposing to NSF were successful.

\[The only group of PIs whose success rates *have* gone down are those
who had prior year grants from NSF. In FY1992 41% of this group won new
grant awards; by FY2001 only 32% did, according to NSF budget
officials.\]

Yet a widespread perception runs through these written comments that
that the situation with NSF funding is getting worse. Reading some of
them, one could imagine one is in the early 1970s when NSF's resources
shrank, setting off a feeding frenzy in which too many worthy proposals
chased too few funds.

"In a downturn in resources for scientific research one could envision
the scenario...where established research teams would find themselves
spending more time writing proposals due to reduced probability of
funding spending more time reviewing colleagues' proposals (since they
are caught in a similar bind) and thus less and less time spent in the
lab and the field...making the discoveries...which is what we are
supposed to be doing," one PI wrote.

Another wrote: "Having spoken with older, well respected scientists, I
have been told that the competition for funds was considerably less
fierce starting about 15 years ago, due to fewer scientists in the
field. I am not sure this is true, per capita. Perhaps it is more likely
that the standard of living for scientists has increased along with the
rest of America, but the funding has not. As a result, to cover one's
research time within the relatively smaller budget standards, more
proposals must be written to different groups...."

The explanation may be that the rise in NSF grant size has not kept pace
with the PIs' rising costs of doing research. A PI who said they had
been supported by NSF for 15 years wrote: "\[M\]y yearly budget now is
almost the same as it was 15 years ago! The money available has not been
nearly enough to keep up with inflation. This has continually reduced
the amount of work that I can do on the project over the years."

Another wrote:"The typical amount of award has remained practically
unchanged in the last 10 years. Tuition and research assistant salaries
have gone up. \[I\]n our institution graduate student salary went up by
50% over the past 8 years. At the same time, cost sharing from the
university has significantly decreased. 2-PI proposals are practically
impossible. The biggest impact is on the quality of graduate students.
This makes our mission of educating quality graduate students through
research -- nearly impossible."

"\[A\] larger body of researchers requesting funds and a relatively flat
budget" was making the "success ratio" too low, wrote another PI. "In
the areas that I compete in it can be as low as 10% which makes me
wonder if this is the best way to spend my time." Another wrote, "some
people, after being rejected several times in a row, just give up. Much
scientific talent has probably been sidelined."

### B Perception Young and New Are Squeezed by Established Networks

"Many researchers, mainly young ones, are frustrated by the system and,
after three or four rejections, give up on NSF," wrote one PI. Some said
in some divisions the success rate was 1:15. Wrote another: "I almost
shudder at the lack of encouragement that NSF has been able to offer the
newer generation. Setting aside a certain portion of the NSF annual
budget to 'take care' of minority applicants was permitted ....but what
about a starting age/inexperience factor in setting aside a % of the
annual budget?"

The corollary of the belief that young investigators face too-long odds
is the belief that established networks have a lock on funds. One
reported "in some parts of NSF.... the institution you are and the
people you know often have more to do with funding decisions than do the
quality of the science and the educational impact of a given proposal."
Another wrote: "It would be worth a serious effort to reduce the level
of cronyism and increase objectivity in the NSF process. I think the
worst offenses are where a 'community' group gets NSF to earmark funds
for a program which is their hobby horse -- usually mediocre science."

\[As mentioned, neither of these perceptions is borne out by the facts
of the overall grant program. Investigators with prior NSF grants are
the only group becoming less successful in competing for funds; these
were 32% successful in FY2001 but had been 41% successful in FY1992. New
PIs who have not had prior NSF grant awards, most of whom are within 7
years of the year of their PhD, have had the same success rate of one in
five for many years.\]

The agency might look at what PIs did before they received that first
grant award. Some PIs told of submitting repeatedly, with excellent
reviews, before the award came through. Aside from inefficiency in
putting off research that was timely much earlier, such cycles of
rejection and resubmission could be taken by the would-be grantee as
"lack of encouragement."

**C Small v. Large Institutions**.

Another concern was, as one wrote, "that a few already well-heeled
Universities and Institutions have the lion's share of the NSF (and DOE
and NIH) pie." Another PI wrote the "long term health" of mathematics
required "that funding is not overly concentrated in the major
institutions. .... Otherwise I'm afraid that the life blood will be
circulating among a few institutions and will become ingrown. " PIs from
small institutions sometimes claimed they had difficulty breaking in.

But others insisted the agency had rewarded new ideas. One wrote: "The
NSF staff recognized our innovation and provided long term support for
myself and my lab members that significantly fostered my career." The
work had had "repercussions in other fields" and "enabled me to mentor
many students," several of which were minorities, they wrote. A number
of undergraduate-only institution faculty members and those from
minority institutions expressed gratitude for NSF grant support.

### D Peer Review 

Many PIs praised the NSF's fairness. "My experience with NSF for the
past 25 years allows me to say that the program directors have been
doing excellent jobs and the grant process is very fair." Another PI was
a good loser: "I have had very good experiences wit the NSF \...
division, where I believed that the process was always fair -- even when
I did not get a grant funded."

There were concerns that some reviewers don't take enough care. An
otherwise satisfied PI wrote that "certain (not all) reviewers' comments
which revealed a lack of attention and/or elementary ability to relate
to the ideas of another person."

"I find the review process frustrating. I know reviewers must remain
\[anonymous\] but a single misunderstanding on their part, omission of a
reference that should have been there in their opinion, and many
hundreds of hours of work is sunk..." one wrote. Another wrote:

"\[T\]he peer evaluation system which has worked so well for a number of
years since the inception of the NSF does not appear to be functioning
well in recent years because of the prevailing low success-to-failure
ratios. Any constructive comments offered in proposal reviews may be
taken as criticisms and may result in failure of proposals.

"Some reviewers are overcritical and pick on minor points. A review
board often has a hard time on the scientific basis in deciding where to
draw the line between the pass and failure. Because of the highly
limited budget at NSF, the line inevitably falls between proposals of
similar scientific merits. This is discouraging for young as well as old
scientists alike. Hence, rather than trying out a new idea, they tend to
prepare for more conservative themes rather than innovative ones."

"I sense that NSF support has become much more competitive, and that
many worthy proposals by top notch investigators must be submitted
multiple times to be funded. A 'queuing-up' mentality seems to have
crept into the funding process and I think this is deadly for innovative
high risk research." said another PI. 20983 This one also wrote:

"\[R\]eviews have become really mean-spirited and nit-picky. I edit a
major journal and process at least 2000 reviews each year. I rarely
encounter reviews as negative as the ones I routinely receive from NSF
on my own proposals. I believe in peer review, but when you get the same
review time and again, its time for the program manager to intervene. I
have discussed this with program managers who say the difference is
money."

The churn of frequent submission, and resubmission to satisfy earlier
rounds of reviewing, sparked further complaints that too much time was
spent writing proposals. One can be representative. "The odds of getting
funded are 20% in my area according to the numbers of submissions versus
funded proposals. It is so competitive that one has to spend many weeks
or months preparing a proposal. My last proposal had 12 pages of
references so that I didn't leave anyone out who worked in any area that
I was covering.

"If a grant only funds one grad student project (seems to be the general
rule) and one has 6 students\[, t\]hen it is possible to have to write
30 proposals to get them all funded. Fortunately my numbers are better
than the 20% average. But still there is so much wasted time in this
process. Can't we have bigger grants for 4 and 5 years, so we can fund 2
or 3 students at once\[?\]"

**D.1 Multidisciplinary research** "It can be difficult to obtain
funding for interdisciplinary research," wrote one. Others said that
reviewers for multidisciplinary projects were too focused on their own
specialities.

**D.2 Large proposals** **suffer** "\[R\]eviewers can be overly
influenced by the budget of the grant they are reviewing." because "they
know how small the total pool of funding is" said one PI. It should be
noted that in answer to this and earlier questions, some PIs said they
don't propose durations of more than three years or more than standard
amounts, since the odds of success are so long.

**E. NSF Staff**

The PIs wrote many testimonials to the helpfulness of NSF program
officers in advising them during the proposal preparation, review and
decision process. Many praised relationships with individual staff that
had helped them over 15 or 20 years. But some other PIs complained they
could not get through to staff or that staff were not helpful were rare.
But many said their relationships with staff were critical to getting
through the review and award gantlet; "if people don't contact the NSF
grant officer, the probability that they will have sufficient
information to get an award is zero," wrote one. The Mathematica report
quantifies the fraction of PIs who said they got lots of help from NSF
staff, some help, and very little help.

The ideal NSF program officer, as described in the PIs' written answers,
is proactive and able to make things happen, by advising how the PI
could change a proposal and facilitating arrangements once an award is
made.

Having "rotators" from academia was praised as bringing fresh outlooks
to the program offices. But an interesting critique of rotators was that
"they lack the influence or contacts of permanent staff, thus putting
their programs at a disadvantage...to programs administered by permanent
staff."

"\[I\]n my opinion, the program officers have declined in quality, and
have become more passive and more limited to proposal 'logistics.' They
are less inclined to provide any overt leadership to the panels and to
the science in general."

Another PI wrote: "\[T\]hese positions are staffed with quite passive
individuals who rarely emerge from Washington except for national
meetings." And another: "\[I\]t seems that program directors are
overwhelmed. \[T\]he review process is somewhat ad hoc since there seem
to be too many panels to participate in and the program directors are
the ones who have to put it all together."

**F. Core v. Newsy**

Some wrote that, in an environment of too few funds and too many
proposals, reviewers and staff prefer new research to proposals to
continue work in core fields. Several criticized NSF for letting "fads"
take precedence over more solid work -- an echo of the need for more
continuity that PIs mentioned answering the second question (see Part II
B.3). Some PIs wrote they found it easier to get the initial grant to
fund a research project than to get the follow on one. For example, one
wrote: "NSF continues to be trendy in its funding. For example, while
the ITR program is a good idea, the regular base programs have been
neglected as a consequence."

### G. International Links

The NSF's administration can mine these comments for feedback concerning
many details of operations such as the Fastlane electronic proposal
system, the CAREER program, and many specific program areas, such as the
ocean drilling and the Antarctic programs.

In light of PIs' earlier stated need for more collaboration, travel, and
dissemination (see Part I E.3 and Part II A.6), their feedback about
NSF's international support is relevant. As usual, their comments are
not completely consistent because individual experiences vary. Still,
most who mentioned the international dimension wrote that NSF did not
support it to the extent they needed to accomplish their best and retain
U.S. world leadership in an era of globalized science.

One complained that drawn-out cycles of proposal resubmission and
"change of reviewers every round" delayed their doing the work. It
becomes frustrating for us ...reinventing the wheel every time and also
losing precious time to competitors from EU \[European Union\]. This has
hurt us tremendously."

"\[T\]he review process needs improvement, I believe, because many of
the reviewers are not fully aware of the leading edge research,
especially what is happening in other major research countries, Europe
and Japan," wrote another who was otherwise pleased with NSF peer
review.

### G. What NSF Should Do

The survey's third written question asked for "comments you have about
your experience with the NSF grant process that you think are
important." It did [not]{.underline} ask for solutions to the problems
the PIs described. But it was clear from these and earlier written
answers that many PIs have thought long and hard about alternatives.

One of the most striking results of Mathematica's tally of the PIs'
multiple choice answers is their broad support for larger average award
sizes, for more awards, and for grant awards of longer duration. The PIs
supported these changes no matter what their experience was with their
individual grant in FY2001. For example, 32% of those who had their
FY2001 award [in]{.underline}creased said NSF should increase the size
of awards; so did 44% of those who had their FY2001 awards
[de]{.underline}creased. Almost half, or 47%, of those who got increases
of 5% or greater favored first of all increasing the
[number]{.underline} of awards NSF makes. And 40% of the PIs who got
funded up to 5% of what they asked voted that NSF should increase the
[number]{.underline} of grant awards. Thus Mathematica's multiple choice
data, as well as these written responses suggests that NSF's present
dollar amounts, number of awards and durations of NSF grants do not seem
"right" to its clientèle, the PIs.

### H.1 Longer Grants, Simpler Renewals 

Throughout the survey, the PIs generally preferred grant award lengths
of 4 and 5 years, to fit with their graduate student's timetables to get
the PhD. Just as important, many wrote, was that five years could
relieve their churn of constant proposal-writing (see Part II B.4). They
also wanted average awards of \$140,000 or higher, or NIH's average of
\$300,000, so a single grant could support a research team. The
Mathematica report spells out the financial implications.

An alternative would be for NSF to make present grants more easily
extended. "Longer grant periods for established researchers with a track
record of continued success is a good trend," wrote one. Another wrote:
"If a project has a track record, say, averaging 10 papers per year,
with solid contributions to developing human resources, then that ought
to be worth more than treading water." The agency's Accomplishment Based
Renewal program was too limited, this investigator said.

Some raised cautions about 5 year terms. One noted there would be less
money for new proposals if NSF directed more of the total to the added
two years -- though this result presumes no proportional increase in the
program total. Another cautions was that scientists have difficulty
planning beyond 3 years anyhow, so it may not be wise for reviewers and
NSF to trust that the funds will be well spent in years 4 and 5. Some
PIs argued that what would most relieve stress was a higher success rate
for all proposers, no matter how achieved.

But increasing the success rates requires an increase in the total funds
for grants at NSF. Though many assumed overall funding would not change
much, the few who addressed this wanted NSF grants program to be funded
much higher than at present.

#### H.2 Would Larger Awards Mean Fewer Share the Pie?

Only a few grappled with the implication that, if more NSF grants went
to 5 year terms, and the average grant were \$140,000 or \$300,000, NSF
most likely would support fewer projects.

One warned that this could create an even more demoralizing situation
than at present, by shutting out more hopefuls. This PI wrote that
morale had been "terrible" for the past decade "largely because it has
been incredibly difficult to get grants and to keep good, important
projects going." But "more money for longer" will "further depress
morale.....

"It will create the same sort of 'inheritance of wealth'
scheme...\[T\]hose who have will continue to get because they have the
resources to insure their competitiveness. I am not sure that this will
be good for our scientific progress overall, because I don't believe
that the top 50% of grant applicants are doing anywhere near 25% of the
excellent research in my field."

This PI said that by "casting the net more broadly" instead "we increase
our chances of capturing the scientists who will come up with something
truly innovative and exciting, ensure funding for people who do really
solid, important work that endures (where often the flashy, newsworthy
science doesn\'t) and we\'ll be able to compete internationally on a
broader scale." This PI saw the US falling behind Canadians and
Europeans who are assured of stable funding over long periods, albeit at
a low level of \$50,000 per year. "They are beating us to the questions
-- often coming up with answers before we get funded to do the very same
thing. As being first is often regarded as important, they win."

Another PI who preferred more grants to fewer larger ones, hoped this
would change the "disconcerting" fact of "my brilliant and hard working
former advisers stop getting any funding from NSF in the middle of their
careers...."

**H.3 Release Time for Writing, Summer Salari**es

Here as in earlier answers, several PIs wrote that they wanted NSF to
support time released from teaching to write books based on their NSF
research. Some asked NSF make its rule allocating funds for summer
salaries more flexible. (See Part I D.5.)

#### H.4 Peer Review

PIs also suggested changes in NSF's merit review system. One wrote that
NSF should "take the money spent on panels and use it instead to hire
better program people, and give them greater scope to utilize the mail
reviews in a way that furthers excellence in science and avoids being
sidetracked by banal panel dynamics."

"I think NSF should seriously consider getting short 2 page letters of
intent and then trying to do an initial assessment of the probable
importance and success so the community is not spending more time and
monies on trying to get grants than is really available," proposed
another PI.

Another proposed a two-tier review system. The first panel review would
be the same as now. Then NSF staff would examine the scores and
writeups. "\[I\]f there is ...inconsistency among primary reviewers,
they can retain the proposal for a second tier review." The second tier
"consists of a much smaller group of senior scientists who will review a
small number of proposals with conflicting critiques." They may revise
the score based on the "quality" of the reviewers. "The revised score
will from then on serve as the basis for the subsequent consideration of
funding of the proposal."

Another wanted NSF to use the NERC system. Investigators should submit
short 5 page proposals instead of 15 page ones. They would offer PIs the
chance to comment on reviews before they are seen by the review panel.
This would fix the problem that, "\[f\]ar too often I have had funding
rejected based on reviews that were ill informed about the work
proposed."

Others asked for a regular appeal process. "We must be given the chance
to offer a rebuttal to a frivolous review without resubmission and
another six months of waiting."

### H.5 Expand NSF's Grants Budget

Many PIs seem to believe that the inefficiencies of NSF's present grants
program can be solved if the total pot of funds is larger. "\[I\]t is
important that NSF be given the resources to fund larger numbers of
qualified researchers," wrote one.

Two comments summarize the message of many PIs who wrote answers in this
survey.

"...I think NSF is the greatest. But the size of the individual grants
is just too small to really accomplish anything significant in my
field... 200K per year will get you one post-doc and one student, and
some materials and supplies and that is it. So you need at least three
of these grants to run a decent sized group. This is a lot of
grant-writing. I wish the NSF budget could be multiplied by 10. Then we
might have something to work with."

"Clearly more money helps, but it is the TYPE of funding that NSF
provides that is unique. So many other sources (including federal
agencies) have a short-term focus on technological or economic
deliverables. More NSF funds opens up a spectrum of possibilities:
working on new ideas, allowing students to drive some of the research,
building infrastructure and a base from which more funds can be obtained
from other sources, more freedom to focus o\[n\] graduate and
undergraduate educational goals rather than 'research' goals. The NSF is
different than industry, and it needs to stay that way; and it needs to
be bigger part of academic funding."

####### APPENDIX A 

**From Answers to Second Question -- Question 3.8**

"I believe the thing we must keep in mind here is that there really is a
genius that underlies the integrated system of teaching and research
that we operate in this country. It is a jewel beyond value and
something that is the envy of the world in terms of the positive returns
it brings at so many levels. The point to focus on here is that, in the
physical sciences, University research is first and foremost an
essential foundation of the process and functioning of education. It is
the singular most important thing that supports the system of
instructional delivery in our research Universities.

" There is no doubt that the technology, scholarship, and knowledge-base
benefits are real and of great value to the nation as well. They are not
of the same gravity, though, as those that derive from the tremendous
synergies we build into the education mission by having it so closely
coupled to a preeminent, world-class research enterprise. This is the
take home message---research is education! We bring into University at
the graduate level, the absolute best undergraduates trained in the
scientific and engineering disciplines. These students form an essential
and irreplaceable component of the instructional team in every science
and engineering department. In my University, for example, we teach more
than 11,500 'instructional units' in Chemistry (in non-jargon terms,
that is an equivalent number of people taking courses in chemistry
during the fall and spring terms). That it is possible to do this, to
provide this level of service to the benefit of the children of the
diverse citizenry of the state of Illinois, is utterly FANTASTIC! There
is no way that this staggering involvement with instructional delivery
could be achieved with this sort of scale, productivity, and efficiency
of cost without the synergies that we derive from the research
components of our mission.

Research lies at heart of how we do graduate education in the physical
sciences. We bring the best students in the country (and world for that
matter) to this involvement. We develop in them skills and envelop them
in training experiences that, by the time they leave, place them at the
very top of the national talent pool. These highly motivated young
people, as an essential part of their graduate training, make enormous
and irreplaceable contributions to our undergraduate instructional
programs in the form of their service as instructors and teaching
assistants. This is a comprehensive system of training in the
disciplines that works very well. It is the envy of the world and the
NSF is the essential agency of national reach that preserves and
nurtures it. We fool with this only at our peril.

The sad news is that we have more than played with this system. We have
tweaked and fooled with it to a point that it is now in grave danger of
failing. The resources being invested in it are below what we need.
Clearly the needs to increase grant sizes and funding balance across the
portfolio of scientific interests served by the NSF are pressing ones. I
want to emphasize as well that longer grant periods will rationalize in
a needed way the duration of a grant and the training periods of the
students who actually perform the work. As it is now, it is a very
stressful process and too much time is lost in business activities (the
pursuit of ever declining resources) and not enough on scholarly
efforts. There is a real danger here. More than a third of the current
faculty of the country will retire in the next 5-10 years. The
enrollments of most public universities will go up by more than 15% over
this same period of time. If we do not pay attention to and preserve the
personnel development activities that our system of Federal research
funding supports, we are going to be in very big trouble."

**Also from answers to Second Question -- Question 3.8**

"Larger amount of funding/grant over a shorter time period will allow
PIs to hire more qualified workers and to concentrate their efforts to
make better progress over a shorter frame of time. This change will
allow US scientists to better compete with EU groups where a full
professor is usually given fundings to support a 10-20 person group with
minimal amount of paperwork.

"As an example, I must write about 1 proposal every 3 weeks to procure
enough fundings to support a group of 15 students and postdoc at a
private research university because each proposal only funds 1-1.5
students and my success rate, as a junior faculty member, is only about
12%. On average, I spend about 30h a week dealing with the mechanics of
proposal writings and submissions. This, however, amounts to only 30% of
my time + overtime because I must work about 100 h/wk. In addition to
all the bureaucratic craps (pls. pardon the language) that the
university asks me to do (15 h/week), I must put in the extra 55 h/week
on teaching (25h on teaching undergraduate), directing research in my
group (15h, 1-1.5 h/each student\[,\] and fulfilling professional
obligations (15h/wk on proposal review and manuscript review) if I want
to do a good job on all of these tasks. Rarely do I have the time to
think about science and to explore new ideas. The only time that I had
to reflect on science is when I am in the taxi or on the airplane going
somewhere.

"The system of \"sharing the wealth\" that NSF seems to be moving toward
is not working. Not all university should have a Ph.D. program and not
all professors should have a research group. It seems that there are
quite a few people out there who continue to get fundings only because
they are established and have political clouts even though they are
still doing the same thing they did 40 years ago. Not only that they
drain the resources from the system, they actually poison the new
generation of scientists by taking in enthusiastic youngsters, teaching
them outdated science, and transfering their complacency attitude
towards these kids. I have been on Ph.D. committees where the candidate
shouldn\'t have gotten the degree if he/she was in my own group.

"At the same time, there are young people who start out in academic
\[careers\] as wonderful teacher and researchers with new ideas but
cannot get fundings. I have a friend who is probably the best teacher
and researcher at his university department who couldn\'t get any
fundings after 5 yrs so he quit to go into industry. In contrast there
is an old timer in his department who continues to get NSF fundings but
has no student. This person managed to fool the system by publishing old
data and laundering the money for his personal uses. There is a very
interesting article by Greg Petsko in The Scientist (2002, 16(1, Jan 7),
72) that states exactly this problem but more eloquently than I can.

"Science should only be done by someone who will put his heart and soul
into it, not by people who only hang on, waiting to retire. The more NSF
continues to support those latter people, the worse will US Science
become."

**From Answers to Third Question - Question 5.14**

"My answers should be interpreted with the understanding that I am a 35
yr old, 100% soft-money researcher with a family including two young
children. Thus there is considerable pressure to perform across a
spectrum of responsibilities while adjusting to them at the same time.
This, of course, is just the situation I happen to be in now, and many
have been in it before. This situation has caused me to consider the
research process with some care, especially it\'s inefficiencies or
ineffectual parts.

"Having spoken with older, well-respected scientists, I have been told
that the competition for funds was considerably less fierce starting
about 15 years ago, due to fewer scientists in the field. I am not sure
this is true, per capita. Perhaps it is more likely that the standard of
living for scientists has increased along with the rest of America, but
the funding has not. As a result, to cover one\'s research time within
the relatively smaller budget standards, more proposals must be written
to different groups. The result is more time managing budgets and
writing proposals. The latter factor should not be underestimated.

"For a soft money researcher needing to cover 12 months of his/her time
and current budget standards pushing for 3 mos (or less) coverage for
senior personnel per proposal (seemingly, this is only communicated
obliquely) and an average of 3 year research duration this results in
obtaining 4 funded proposals per 3 years. This would be a limited time
sink if success was 100% (\~ 4 weeks/year of proposal writing at 3
weeks/proposal). At 50% it is 8 weeks/yr writing proposals. At 33%
success it is 12 weeks/yr and so on. It seems that a better balance can
be found. Consider that many programs have funding rates of 10-33% (you
tell me, I only hear these numbers, perhaps they are worst case).

"As a soft money researcher, I think having but one main project is too
few, but having 4 main projects is too many, so the ideal is 2 or 3 (as
opposed to side projects), which would require 4-6 months coverage for a
PI per successful proposal. Using a salary of 70k/year for a senior
person and a 2.25 multiplier on salary, the senior person, at 4 months
salary is roughly 60k/year in the budget. This leaves about 30-40k or so
for other personnel (post-docs, students, technicians), computers,
travel and publications, etc, using the typical 90-100k/yr average for
proposed work. With post-doc salaries around 40k one can see that the PI
is resigned to 1) risking a higher-cost proposal that has a higher
chance of failure (again, I think), or 2) reducing his number of
man-months covered, if he/she wishes to support staff/students. Either
choice results in more proposals and spreading oneself thinner.

"Personally, I feel I am able to maintain a good standard for my work,
but projects take longer and in fact the quality and standard of the
results could be still better if less time was spent on proposals and
the dollar amounts were higher allowing for more focus on fewer
projects.

"Another important issue is that many newer PIs are relatively savvy
with word processing, figure preparation as well as their research
activities. As a result, many of the activities that would be completed
by assistants, are now completed by the PI resulting in less research
time."

# **Also from Answers to the Third Question -- Question 5.14** 

"The rumored budgetary rule of thumb\--one RA, one PC, and one month of
summer salary\--is not enough to sustain a research program or promote
scientific education. It is a strain to maintain an apprenticeship style
of training in which senior and junior graduate students overlap on a
project (and impossible to incorporate post doctoral associates). Since
1986, I have had 9 NSF grants. More funding each time would have made
for a better educational environment.

"Last year\'s unfunded mandate that more money be devoted to children\'s
research was misguided on several fronts. Children\'s research is funded
amply at NICHD. The required funding was taken from an area\--basic
human perception and cognition\--that already has precious little. One
area that was spared, cognitive neuroscience, is one that again has
funding opportunities at NIH.

"Accomplishment Based Renewal opportunities are too limited, both in
frequency and level. If a project has a track record, say, averaging 10
papers per year, with solid contributions to developing human resources
(e.g., dissertation research that appears in the top journals, student
senior-authored publications, etc.), then that ought to be worth more
than treading water.

"These complaints should be put in the context of what NSF does that
remains special. It supports pure research and does not demand that
there must be a payoff in medical or economic terms. For the ordinary
scientist, pure research is always the royal road to understanding and
NSF is often the only place to go if one wants funding to travel that
royal road."

"Finally, with regard to the questionnaire, it would have been good to
allow changes to answers; and the question on publications: I have
coauthored 85 papers in the past 5 years. For team research, and with
students taking leading roles, the question of how many first
authorships is inappropriate."

**APPENDIX B**

**Quotes in which Principal Investigators mention the National
Institutes of Health(NIH) in answer to First Question -- Question 2.5**
\*

\-\-- "The decrease in funding forced my seeking additional support.
Because this grant is for plant science, there are limited
opportunities. NIH does not support this type of work. I received a
1-year grant (with a second year possible) from Monsanto Plant Science
that enabled my hiring a technician to do work on the hormone promoter
DNA binding project."

\-\-- "The new NIH standards for graduate and post-doctoral salaries
cannot possibly be supported on the severely trimmed budget. I am in a
region of the country that is not geographically desirable and so we
must offer competitive salaries if we are to have any hope of attracting
top notch people. This becomes increasingly difficult with small NSF
awards."

\-\-- "The total average award for the funding of molecular and
biochemical research from NSF to my knowledge averages about
\$100,00\[0\]. For the same kin\[d\] and level of research, the average
award size at NIH is \$325,000. I recognize the ... award sizes have
been kept at a lower level to support more investigators.

"In today's research, we are trying to do state of the art research, ie
'rocketship type laboratory' with rotating 'bicycle mechanics'. \[T\]his
is the analogy that I use when I explain to folks about why I have to
work 60 to 80 hrs a week, why I subsidize my research with my own
personal funds and the difficulties in maintaining a state of the art
research program. NSF needs to recognize that modern biological research
takes more funding than 10 years ago and requires long term personnel to
help maintain these sophisticated laboratories. This is also very
important for the training of students.

"In addition, not only should the awards actually pay for the research
that is being requested, the award periods in many cases should be 5
years and not 3 years. "

\-\-- "Since I was funded for 3 years instead of 4, this means that one
of the postdoctoral fellows, who had planned to come in year 3 from
Australia, may not participate in the work. I had hoped to guarantee 2
years of support for that position in order to get this person to my
lab.

"As a result, I have had to go to NIH and request funding for a
different project, to try and make up the difference in funding level
and length of award. That grant is pending. I guess the summary is that
even for a small lab, it is difficult to survive with a single NSF
grant, and the 3 year cycle is not long enough to allow full development
of the work in time to write the next renewal."

\-\-- "As new faculty, I appreciate very much the timely support from
NSF. I hope that NSF will keep its tradition in nurturing researchers
and teachers of the next generation. Unfortunately, the NIH does not
have such a wisdom yet. An informal survey among new faculty members
suggests that 80% of time are generally spent writing and rewriting
NI\[H\] proposals. How could these new faculty have time to build a new
lab, develop new courses, and do research? I salute you all at the NSF!

"I realize that the NSF budget is not big. A small increase in the
amount of award would be helpful. This is needed because the salaries
(including students' stipends) nowadays are much higher than in the
past. Yet the NSF limit of award remains the same level over many years.

"On the other hand, I also understand that Congress was not generous
enough towards NSF. Under such conditions, if NSF wants to give support
to as many young people as possible, I fully support it."

\-\-- "As our indirect cost rate is 110% on salaries the effective award
was tiny. I am therefore under pressure (both economic and from my
director) to redirect my research grants to NIH. I do not want to do
this as I have valued my NSF grant very much as it has enabled me to
pursue the more imaginative aspects of my research program over the last
15 years of continuous funding from NSF. If the funding level at NSF
does not increase in the future, I shall be forced to terminate my
relationship with NSF (as will other recipients at my institution)."

\-\-- "As NSF grants get smaller compared to, for example, NIH or DOD
grants, it is harder to justify putting as much time into these projects
as one would if they were fully funded. In fact, in this day and age it
is hard to find time to spend on fundamental science when it is so hard
to generate funds to support it. So one feels pressure to spend less
time on NSF-supported projects."

\-\-- "The drastic decrease in budget and duration required that I take
out several parts of the proposal which I then incorporated into another
grant proposal to the NIH."

\-\-- "The submitted budget was very modest in keeping with my
perception that NSF funds very small biology grants. The additional
reduction in budget makes the award less than a third of a comparable
biology grant funded by the NIH."

\-\-- "My interpretation of the results of the handling of this
application is that it was judged to be an important and innovative
proposal in an areas supported by NSF, but one that can be translated to
areas supported by NIH. The reduced support and reduced time frame were
based on the premise that it would provide the required support to fully
develop a proposal that, if the innovative aspects were demonstrated
would be funded by NIH, therefor\[e\] it was supported more as a
demonstration project rather than a fully funded NSF proposal.

"Therefor\[e\] we are pursuing the most innovative and collaborative
aspects of the work with the anticipation of submitting the 'renewal' to
the NIH and not to NSF...."

\-\-- "We have had to scale down our research program. This has put us
at a great disadvantage in our competition with other laboratories in
the U.S. (especially those funded by NIH) and in for\[ei\]gn countries
that are able to maintain research programs that are funded at
considerabl\[y\] higher levels for longer periods of time. "

\###