# Presentation: 302487

## Impact of Proposal and Award Management Mechanisms ReportAdvisory Committee Discussions

- Joanne Tornow
- Chair, IPAMM Working Group

## What is the Context?

- Between FY 2000-2005:
- The NSF budget increased by 44%.
- The average size of research awards increased by 41%.
- Research proposal submissions increased by nearly 50%.
- NSF budget increases were absorbed by the growth in the average award size, leaving little flexibility to respond to growing proposal submissions.  As a result, the research proposal funding rate decreased by 29%, from 30% to 21%.
- Directorate level trends show significant variability in rate of change, degree of change, and starting and end points of change.

## Findings: Causal Factors

- The increase in proposal submissions was due to an increased applicant pool and to an increased number of proposals per applicant.
- Increased size and capacity of the research community
- Loss of funding from other sources
- Increased use by NSF of targeted solicitations in new areas
- External institutional pressures

## Solicited vs. Unsolicited Proposal Trends

## External Institutional Pressures

**External Institutional Pressures**

***Beyond the goal of making contributions to your area of science, to what extent do the following factors motivate you to submit research proposals to any funding source?***

***Question 23:**** Building/maintaining a grant record for academic tenure/ promotion*
***Question 24:**** Contributing to the institution’s research status/reputation*
***Question 25:**** Supplementing or paying my own salary*
***Question 26:**** Building/maintaining a  research infrastructure*

*Key Findings: Goal 1, Drivers*

## Findings: Impacts on Quality and Nature of Proposed Research

- Proportion of highly-rated proposals has not declined, however, the funding rate of highly-rated proposals has decreased
- Analyzed attitudinal data to assess community perceptions about transformative research:
- 56% believe to a great or moderate extent that NSF welcomes transformative research
- NSF is the predominant choice for submitting proposals with transformative research ideas
- Significant disconnect between proposer and reviewer perceptions about prevalence of transformative projects

## Findings: Impacts on Specific Groups

- The decrease in funding rate has not had a disproportionate effect on women, minorities, beginning PIs, or PIs at particular types of institutions.
- Funding rates
- Share of proposal and award portfolios
- Maintaining funding beyond first award
- Years between degree and first award

## Findings: Impacts on Merit Review

- NSF’s peer review system is overstressed
- Reviewer workloads have increased
- Reviewer pool increased 15%, proposal load increased 50%
- Increased use of panel-only review
- Time spent on each review, as well as the thoroughness and quality of reviews, may be diminishing (based on survey data)
- Timeliness of proposal decisions did not decline, however PIs are increasingly dissatisfied with turnaround time

## Community Perceptions About Funding Rates

- More than 60% of survey respondents perceive that the level of competition at NSF is more intense than at other agencies.
- Most survey respondents underestimated actual funding rates.
- Nearly 49% of respondents estimate funding rates at 10% or lower.

## How to Improve Funding Rates?

- Limit Proposal Submissions
- Increase Number of Awards

## Limit Proposal Submissions

- Most funding opportunities do not limit submissions
- Of those that do, three primary mechanisms are used:
- Preliminary proposals
- Limiting proposals submitted by an institution
- Limiting proposals by individual

## Limit Proposal Submissions

- Institution limits primarily used for solicitations focused on infrastructure and instrumentation, centers and facilities, or education and training.
- When submission limits are used by research programs, primarily limit submissions by PI.

## Increase Number of Awards

- Primarily accomplished by increasing availability of funds:
- Two fiscal years of funds used for a single competition
- Adjustments made to the balance of standard and continuing grants
- Provides some flexibility in responding to increased proposal submissions, but can only be employed for a limited time, and with discretion

## IPAMM Recommendations to NSF

- Focus on developing strategies that are appropriate within the context of each unit, that balance long-term planning with the ability to respond to changing needs, and that help break the decline-revise-resubmit cycle for highly fundable proposals
- Improve communications with internal and external communities
- When implementing new management practices
- About sources of accurate NSF data
- Update the IPAMM trends analyses annually, and periodically reassess the practices and policies of the directorates/research offices.

## Current Status

- NSF Senior Management currently engaged in discussions of recommendations
- Implementation initiated on some recommendations
- Reaching out to NSF staff to discuss the findings of the report
- Reaching out to external communities to begin a dialogue on the implications of the report
- Alerted the NSF community about the report
- Discussed issues with the Federal Demonstration Partnership
- Engaging the Advisory Committees this Fall

## “When we try to pick out anything    

*“**When we try to pick out anything    *
*     **by itself, we find it hitched to  *
*   **everything else in the universe.”**  *
*       **	**	**	**	**	** **--*John Muir

## Acknowledgements

***Expert Resources:***
- Emily Fort, BFA
- Robert Groves, Chair, SBE AC, and Director, Survey Research Center, University of Michigan
- Brent Miller, AAAS Fellow, BIO
- Jeri Mulrow, SBE
- Louie Rivers, SBE
- Kelli Savia, BFA
- Beth Ann Velo, BFA
***Booz Allen Hamilton Survey Team:***
- George Angerbauer, Michael Carrieri, Pat Corrigan, Mary Kay Gibbons, Chris Johnson, Keisha Kelly, and Luke Monck

***IPAMM Members***
- Joanne Tornow, Chair, O/D
- Paul Herer, Executive Secretary, O/D
- Rita Teutonico (03/06 to 10/06) and
- Paul Malchow (11/06 to 08/07), BIO
- Suzi Iacono, CISE
- Daniel Litynski, EHR
- Adnan Akay, ENG
- Jarvis Moyers, GEO
- William Rundell (03/06 to 09/06) and
- Deborah Lockhart (10/06 to 08/07), MPS
- Jacqueline Meszaros, SBE
- Jeanne Hudson, OISE
- Neil Swanberg, OPP
- Vernon Ross, BFA

## Back-Up Slides

## Research Proposal Funding Rate Drops as NSF Budget Increases

## Data Sources

- Statistical data
- NSF data on proposal funding rates, PI success rates, budget data, demographic data
- Science and Engineering Indicators
- Attitudinal data
- 2007 NSF Proposer Survey developed with Booz Allen Hamilton
- Other input
- Focus groups of new rotators, COV reports, discussions with Advisory Committees and the National Science Board

## Directorate Funding Rate Trends

## Directorate Proposal Submission Trends

## Funding Rate Trends for New and Prior PIs in Underrepresented Groups

**Funding Rate Trends for New and ****Prior PIs in Underrepresented Groups**

- Prior PIs

- New PIs

## Trends in Use of Submission Limitations

## Trends in Use of Submission Limitations