# Presentation: 372597

## A New Model for Predicting the Number of NIH Grant Applications

- viaSim
- J. Chris White
- April 19, 2007

## Overview

- It is fundamentally important for all of NIH to improve the ability to forecast budget needs and improve planning related to manpower requirements and paylines that are linked to the number of applications received.
- Calculating manpower based on number of applications is straightforward, so the key issue is forecasting the incoming application stream.
- CSR is the triage organization for incoming NIH proposals and reviews approximately two-thirds of these proposals.
- _Problem_:  Results from common statistical forecasting models are sometimes unsatisfactory.
- _Objective_:  Develop a forecasting model using a simulation approach called system dynamics.
- _Major Tasks_:
- Interview research institutions to understand how they make decisions
- Interview NIH personnel to understand how NIH makes funding decisions
- Develop system dynamics (SD) model based on interviews
- Compare and contrast the traditional statistical forecasting approach with the SD structural forecasting approach

## Statistical Forecasting Example

## Previous CSR Statistical Forecasting Model

## Meyers’ Statistical Forecast

## Two Fundamental Types of Forecast Modeling

**Statistical** modeling attempts to project future results by extrapolating historical process data into trends:
- Highly dependent on the quality of data used to generate trends
- Focuses on results (sometimes using correlations with inputs), with no consideration for the activities
- Does not project well when future is significantly different from past

**Structural** modeling attempts to project future results by simulating the activities or operations:
- Does not depend on data to create the model because the flow of activities is independent of data
- Data is used to tailor and calibrate the model (as inputs)
- Focuses on activities, which generate results

- Activities

- Results

- Inputs

## Model Scope

- NIH
- Budget

- Apps
- Submitted

- Desired
- Proportion
- NIH Funds

- Institution
- Staff

## Example Feedback Loop:An increase in grants funded leads to a decrease in applications submitted

- Example Feedback Loop:An increase in grants funded leads to a decrease in applications submitted

- NIH
- Budget

- Apps
- Submitted

- Desired
- Proportion
- NIH Funds

- Institution
- Staff

**+**

**+**

- Success
- Rate

**+**

**_**

**+**

- Completed
- Grants

**+**

- NIH
- Review
- Capacity

**+**

- Apps
- Reviewed

**+**

- Apps
- Unfunded

**+**

**+**

**+**

- Required
- Funds

**+**

- Funding
- Gap

**+**

**–**

**_**

**_**

**+**

- Funded
- Grants

**+**

**+**

**_**

## Dr. Zerhouni’s “Perfect Storm”

- NIH
- Budget

- Apps
- Submitted

- Desired
- Proportion
- NIH Funds

- Institution
- Staff

**+**

**+**

- Success
- Rate

**+**

**_**

**+**

- Completed
- Grants

**+**

- NIH
- Review
- Capacity

**+**

- Apps
- Reviewed

**+**

- Apps
- Unfunded

**+**

**+**

**+**

- Required
- Funds

**+**

- Funding
- Gap

**+**

**–**

**_**

**_**

**+**

- Funded
- Grants

**+**

**+**

**_**

## Simulation Scope

- NIH
- Budget

- Apps
- Submitted

- Desired
- Proportion
- NIH Funds

- Institution
- Staff

## Simulation Model Example

- Feedback Loop:An increase in grants funded leads to a decrease in apps submitted.

- 1

- 2

- 3

- 4

## Previous Simulation Results:Apps Reviewed by CSR

## New Baseline Simulation Results

## Demonstration

## Next Steps

- Train NIH staff in use of simulation tool.
- Finalize any reporting requirements.
- Model enhancements:
- Expand to include more internal NIH processes and policies (e.g., review processes).
- Expand to include more operational processes at institutions (e.g., facility expansions).
- Integrate with PI simulation model.

## Appendix

- System Dynamics Methodology

## Why System Dynamics (SD)?

- In any system, structure guides behavior,
- and behavior determines results.

## System Dynamics Methodology

- Good analogy:  Water in bathtub

- 2. Money accumulates in “Cash Balance”

- 3. Money flows out through “Expenses”

- 1. Money flows in through “Revenue”

## System Dynamics Methodology

- Reinforcing loops push a variable to continue to grow (or decrease):
- Compound interest
- Radioactive decay
- Balancing loops attempt to bring a variable to some target value:
- Adjusting personnel based on budget available
- Thermostat in house
- Body temperature

**Number of personnel drives expenses**

**Number of personnel drives work completion**

**Work completed drives revenue**

**Available money drives hiring (or not hiring) of new personnel**

- Arrows represent connections between variables.
- Interconnections create “feedback loops”.

## What Makes SD Different?

- SD creates an “operational”, working model of activities.
- SD captures “feedback loops”, which are the function of management.
- _Ex_:  Cutting back when overbudget
- _Ex_:  Adjusting the workforce to meet demand
- SD captures time delays and non-linear relationships that can significantly impact performance.
- _Ex_: Full impact of new policies not realized until several years later.
- _Ex_: Doubling of labor force does not always double throughput.